File uploads over winrm failing

[zl4bv]

File uploads are failing when performing the following actions:

• Uploading the VirtualBox guest additons (presumably performed by the builder)
• Uploading (large) files with the file provisioner

Windows 7 Enterprise 64-bit host, Windows Server 2008 R2 Standard 64-bit virtualbox guest.

Below is the error I am seeing:

--> virtualbox-windows-iso: Error uploading file to $env:TEMP\winrmfs-54d7df54-ca3b-aa87-1b5f-a84ddd70c2a5.tmp: http err
or: 500 - <s:Envelope xml:lang="en-US" xmlns:s="http://www.w3.org/2003/05/soap-envelope" xmlns:a="http://schemas.xmlsoap
.org/ws/2004/08/addressing" xmlns:x="http://schemas.xmlsoap.org/ws/2004/09/transfer" xmlns:e="http://schemas.xmlsoap.org
/ws/2004/08/eventing" xmlns:n="http://schemas.xmlsoap.org/ws/2004/09/enumeration" xmlns:w="http://schemas.dmtf.org/wbem/
wsman/1/wsman.xsd" xmlns:p="http://schemas.microsoft.com/wbem/wsman/1/wsman.xsd"><s:Header><a:Action>http://schemas.dmtf
.org/wbem/wsman/1/wsman/fault</a:Action><a:MessageID>uuid:939C44B1-F42D-4F89-86E9-BF26DD6EDA1C</a:MessageID><a:To>http:/
/schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</a:To><a:RelatesTo>uuid:e6c94e3b-30aa-4aec-5956-0d554cde44e2</
a:RelatesTo></s:Header><s:Body><s:Fault><s:Code><s:Value>s:Sender</s:Value><s:Subcode><s:Value>w:InvalidSelectors</s:Val
ue></s:Subcode></s:Code><s:Reason><s:Text xml:lang="en-US">The WS-Management service cannot process the request because
the request contained invalid selectors for the resource. </s:Text></s:Reason><s:Detail><w:FaultDetail>http://schemas.dm
tf.org/wbem/wsman/1/wsman/faultDetail/UnexpectedSelectors</w:FaultDetail><f:WSManFault xmlns:f="http://schemas.microsoft
.com/wbem/wsman/1/wsmanfault" Code="2150858843" Machine="127.0.0.1"><f:Message>The Windows Remote Shell cannot process t
he request; the selector value 42C971CD-F030-4FAD-B724-DD716B851563 specified in the request was not found. </f:Message>
</f:WSManFault></s:Detail></s:Fault></s:Body></s:Envelope>

Below is a network trace of the messages associated with shell ID 42C971CD-F030-4FAD-B724-DD716B851563 - the body of the Fault messages represent the XML of the error message returned by packer (above).

Below is an example packer JSON. The scripts/ directory contains several PowerShell scripts and a couple of MSIs/MSUs.

{
  "builders": [{
    "type": "virtualbox-windows-iso",
    "guest_os_type": "Windows2008_64",
    "disk_size": 21440,
    "iso_url": "D:/ISOs/7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso",
    "iso_checksum": "4263be2cf3c59177c45085c0a7bc6ca5",
    "iso_checksum_type": "md5",
    "boot_wait": "2m",
    "winrm_username": "vagrant",
    "winrm_password": "vagrant",
    "winrm_wait_timeout": "10m",
    "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c \"Packer Shutdown\"",
    "vboxmanage": [
      ["modifyvm", "{{.Name}}", "--memory", "2048"],
      ["modifyvm", "{{.Name}}", "--cpus", "2"]
    ],
    "floppy_files": [
      "./answer_files/windows-2008r2-standard/Autounattend.xml"
    ]
  }],
  "provisioners": [
    {
      "type": "file",
      "source": "scripts/",
      "destination": "C:\\scripts"
    }
  ]
}

[zl4bv]

Forgot to mention - this may be related to issue #3

[dylanmei]

I've never seen this before. Is this a vanilla Win2008 iso?

[mefellows]

FYI sneal/go-winrm#3

[zl4bv]

Is this a vanilla Win2008 iso?

Yes - Win 2008 R2 Standard

FYI sneal/go-winrm#3

Like the reporter of that issue, I have executed make updatedeps and recompiled packer-windows-plugins, but still not successful.

It's worth mentioning that I only see this issue once every three or so builds.

[dylanmei]

When you get onto one of those passing builds would you mind posting as much of winrm get config/winrm as possible?

Also, I'd be curious if you could get to a faster repro using this executable: packer-community/winrmcp

[zl4bv]

Config
    MaxEnvelopeSizekb = 150
    MaxTimeoutms = 1800000
    MaxBatchItems = 32000
    MaxProviderRequests = 4294967295
    Client
        NetworkDelayms = 5000
        URLPrefix = wsman
        AllowUnencrypted = false
        Auth
            Basic = true
            Digest = true
            Kerberos = true
            Negotiate = true
            Certificate = true
            CredSSP = false
        DefaultPorts
            HTTP = 5985
            HTTPS = 5986
        TrustedHosts
    Service
        RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)S:P(AU;FA;GA;;;WD)(AU;SA;GWGX;;;WD)
        MaxConcurrentOperations = 4294967295
        MaxConcurrentOperationsPerUser = 15
        EnumerationTimeoutms = 1800000
        MaxConnections = 25
        MaxPacketRetrievalTimeSeconds = 120
        AllowUnencrypted = true
        Auth
            Basic = true
            Kerberos = true
            Negotiate = true
            Certificate = false
            CredSSP = false
            CbtHardeningLevel = Relaxed
        DefaultPorts
            HTTP = 5985
            HTTPS = 5986
        IPv4Filter = *
        IPv6Filter = *
        EnableCompatibilityHttpListener = false
        EnableCompatibilityHttpsListener = false
        CertificateThumbprint
    Winrs
        AllowRemoteShellAccess = true
        IdleTimeout = 180000
        MaxConcurrentUsers = 5
        MaxShellRunTime = 2147483647
        MaxProcessesPerShell = 15
        MaxMemoryPerShellMB = 1024
        MaxShellsPerUser = 1500

Will attempt to repro directly with the winrmcp executable and get back to you.

[mefellows]

FWIW I have actually been able to reproduce this issue on a 2012r2 box fairly reliably. It was happening in 1/2 builds for me today. Whilst I haven't yet gotten to the bottom of it, adding an arbitrary sleep (60s) makes this go away, so it seems it could be some kind of race condition if the upload is the first Provisioner.

[dylanmei]

A bit confused. Where would you put your "arbitrary sleep"? I want to repro this with winrmcp, not a dozen packer runs, if possible!

[mefellows]

Good point - this was with a Packer run.

Assuming it is related to some kind of startup condition that is required, I suppose you could repro with a script that waits for WinRM to become available and then as soon as it was start uploading a file?

Perhaps that's why it's hard to repro with winrmcp alone?

[zl4bv]

Funnily enough I very frequently experienced this issue when uploading the VirtualBox guest additions (which gets uploaded immediately after WinRM becomes available).

[zl4bv]

Finally had some free time to attempt to reproduce the error using winrmcp - with Windows Server 2008 R2.

7 attempts, 4 different errors. Results here: https://gist.github.com/zl4bv/bb5d5b6f322f29198535

Same WinRM config as before, EXCEPT for the following:
MaxMemoryPerShellMB = 300
MaxShellsPerUser = 5

The lower MaxMemoryPerShellMB value explains the error in attempt 4.
The lower MaxShellsPerUSer value explains the error in attempts 6 & 7.
Unsure about the errors in attempts 1, 2, 3 & 5.

Note that none of the 4 errors match the "The Windows Remote Shell cannot process the request; the selector value 42C971CD-F030-4FAD-B724-DD716B851563 specified in the request was not found." error from my original bug report.

[dylanmei]

That's good stuff, thanks for the time you spent on this. @mefellows Were you getting the "selector value" error 2 days ago?

[mefellows]

I believe so, yes. I can confirm once back in the office next week.

On Friday, April 3, 2015, Dylan Meissner [email protected] wrote:

That's good stuff, thanks for the time you spent on this. @mefellows
https://github.com/mefellows Were you getting the "selector value"
error 2 days ago?

—
Reply to this email directly or view it on GitHub
#25 (comment)
.

Matt Fellows

• http://www.onegeek.com.au/

[levpaul]

I'm being affected by this now too. Seems that I can't upload directories very well :/

[dragon788]

@dylanmei An arbitrary sleep could be placed in the WinRM.bat (name may vary) by calling cmd.exe with "timeout 30" as 'timeout' is the Windows version of sleep, and it will keep that batch file open until the timeout completes, which might delay enough for Virtualbox additions to get a proper connection and work.

[chrisjsmith]

Me too on 0.8.5

[dwsummersaegis]

I just started trying packer yesterday so am a complete newby on it, so there very well be something I'm not doing right.

However, I've yet to get anything to work on it. It looks like it is working and then this error occurs:
==> Some builds didn't complete successfully and had errors:
--> virtualbox-iso: Error uploading guest additions: Error uploading file to $en
v:TEMP\winrmcp-55e1d4cb-62c2-1b71-0fbb-8f3fa0f918eb.tmp: Couldn't create shell:
http error: 500 - .......

I'm running packer 0.8.6 on Windows 7 host with VirtualBox 5 and this error is happening 100% of the time. I've tried about 20 times. I've also tried the above suggestions to put at end of winrm.bat:

cmd.exe /c timeout 30

and also set the following in winrm.bat:

cmd.exe /c winrm set winrm/config/winrs @{MaxMemoryPerShellMB="2048"}
cmd.exe /c winrm set winrm/config/winrs @{MaxShellsPerUser="5"}

If needed I can give more configuration files.

I also thought of trying to disable the VirtualBox Additions upload but haven't seen a way to do that yet.

Should I try the SSH method of getting the VM configured before I come back and retry the WINRM method?

At the moment I just need to get SOMETHING to work. I would prefer not to put SSH on the box but if not possible any other way, then I can live with it.

[dylanmei]

@dwsummersaegis Get up to speed with this article: http://www.hurryupandwait.io/blog/creating-windows-base-images-for-virtualbox-and-hyper-v-using-packer-boxstarter-and-vagrant

You can disable guest additions with this in your config: "guest_additions_mode": "disable"

If you're open to SSH, try these templates: https://github.com/joefitzgerald/packer-windows

[chrisjsmith]

Workaround: Install chocolatey in the unattended XML post-install then use that to install guest additions from the chocolatey repo. Feels less coupled as well.

[dwsummersaegis]

I let it slide for a few months and when I came back to it a few weeks ago Packer 0.10.0 was out and now for the first time I can get things to work.

Thanks for the info!

[krishnagade18]

If you are using windows 2012 r2 in builder section run the below winrm powershell script as userdata
`winrm quickconfig -q
winrm set winrm/config/winrs '@{MaxMemoryPerShellMB="300"}'
winrm set winrm/config '@{MaxTimeoutms="1800000"}'
winrm set winrm/config/service '@{AllowUnencrypted="true"}'
winrm set winrm/config/client/auth '@{Basic="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'
winrm set winrm/config/service/auth '@{Basic="true"}'

netsh advfirewall firewall add rule name="WinRM 5985" protocol=TCP dir=in localport=5985 action=allow
netsh advfirewall firewall add rule name="WinRM 5986" protocol=TCP dir=in localport=5986 action=allow

net stop winrm
sc.exe config winrm start=auto
net start winrm

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope LocalMachine -Force`

i think this can solve the issue even in windows 2008

[dragon788]

I was still getting this error from boxcutter/windows, but upping the MaxMem to 2048 worked for me.

[godric7]

Still had the issue today,
Found this digging around: boxcutter/windows#73

Solved my problem and seems like it would be an elegant solution.

[jptoto]

@godric7 That solution fixed my issue. Much more elegant than upload. Thanks!