System.OutOfMemoryException when uploading 200MB file

[agombault]

I'm facing an System.OutOfMemoryException when I try to upload a 200MB file on :

• Window Server 2012 Standard 64-bit
• 2CPU
• 4GB
• 7GB disk free space

I'm actually using the gem winrm-fs (1.0.1)

Application: wsmprovhost.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.OutOfMemoryException
Stack:
   at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.InternalClearAllResources()
   at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.CloseHelper()
   at System.Management.Automation.Runspaces.Internal.RunspacePoolInternal.CoreClose(Boolean, System.AsyncCallback, System.Object)
   at System.Management.Automation.ServerRunspacePoolDriver.Close()
   at System.Management.Automation.Remoting.ServerRemoteSession.HandleSessionDSHandlerClosing(System.Object, System.EventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerlImpl.CloseConnectionAsync(System.Exception)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.DoClose(System.Object, System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.RaiseEventPrivate(System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.ProcessEvents()
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.RaiseEvent(System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at Managed_WSMan_ServerTransportManager.DoClose(Boolean, System.Exception)
   at Managed_ShellSession.CloseOperation(OperationShutdownContext*, System.Exception)
   at Managed_WSMAN_PlugIn.CloseShellOperation(OperationShutdownContext*)
   at Managed_WSMAN_PlugIn.PerformCloseOperation(OperationShutdownContext*)
   at <Module>.CloseOperation(OperationShutdownContext*)

[agombault]

I have a different Exception with the same file on :

• Windows Server 2008 R2 Datacenter 64-bit
• 2CPU
• 2GB
• 8GB disk free space

Application: wsmprovhost.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: System.AccessViolationException
Stack:
   at <Module>.ReportOperationComplete(_WSMAN_PLUGIN_REQUEST*, UInt32, ...)
   at <Module>.ReportOperationComplete(_WSMAN_PLUGIN_REQUEST*, UInt32, ...)
   at Managed_WSMAN_PlugIn.ReportWSManOperationComplete(_WSMAN_PLUGIN_REQUEST*, System.Exception)
   at Managed_CommandSession.CloseOperation(OperationShutdownContext*, System.Exception)
   at Managed_ServerSession.Close(System.Exception)
   at Managed_ShellSession.CloseAndClearCommandSessions(System.Exception)
   at Managed_ShellSession.CloseOperation(OperationShutdownContext*, System.Exception)
   at Managed_ServerSession.Close(System.Exception)
   at System.Management.Automation.Remoting.ServerRemoteSession.Close(System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.DoCloseCompleted(System.Object, System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.RaiseEventPrivate(System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.ProcessEvents()
   at System.Management.Automation.Remoting.ServerRemoteSessionDSHandlerStateMachine.RaiseEvent(System.Management.Automation.RemoteSessionStateMachineEventArgs)
   at Managed_WSMan_ServerTransportManager.DoClose(Boolean, System.Exception)
   at Managed_ShellSession.CloseOperation(OperationShutdownContext*, System.Exception)
   at Managed_WSMAN_PlugIn.CloseShellOperation(OperationShutdownContext*)
   at Managed_WSMAN_PlugIn.PerformCloseOperation(OperationShutdownContext*)
   at <Module>.CloseOperation(OperationShutdownContext*)

[agombault]

It appears that the remote server needs Powershell v4.0 to perform uploads of large files

[fwininger]

@mwrock do you have any idea, why Powershell v4 seems to be needed to upload large files ?

[mwrock]

Prior to ps v4, the MaxMemoryPerShell was 150mb. So an out of memory error makes total sense. To add confusion, on ps v3 (2012), there was a bug that did not allow you to customize this configuration. I think you actually could change it but it was not honored. I'm not sure why you get an access violation on 2008R2. Mybe try boosting the MaxMemoryPerShell on 2008R2 and see if that allows the upload.

ps v4. bumps the limit to 1GB and v5 is effectively unlimited.

[fwininger]

Thanks @mwrock for the answer, we try to run more tests next week.
May the installation of KB2842230 can solve this issue.

https://support.microsoft.com/en-us/help/2842230/-out-of-memory-error-on-a-computer-that-has-a-customized-maxmemorypershellmb-quota-set-and-has-wmf-3.0-installed

[mwrock]

Yeah thats the hotfix you want. Even with the patch, it seems like we could be doing a better job of freeing memory so we don't have to hold the entire file in memory. Maybe specifying a buffer size on the FileStream or flushing on every x chunk.