.net core MSBuild normalizes slashes when it shouldn't

[cdmihai]

Description copied from internal email

Repro steps:

1. On linux, get the latest CLI
2. Create a new project “test.csproj”(dotnet new)
3. Run dotnet msbuild /p:RestoreSources=/tmp/some-source%3Bhttps://api.nuget.org/v3/index.json /t:restore test.csproj /v:diag > output
4. In output file, search “RestoreSources” and “Sources”.

You can find
RestoreSources = /tmp/some-source;https://api.nuget.org/v3/index.json
Sources= /tmp/some-source;https:/api.nuget.org/v3/index.json

The target file: https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Build.Tasks/NuGet.targets#L415
<Sources>$(RestoreSources)<Sources>

The known workaround is to escape the slash so MSBuild treats it as a literal:
dotnet msbuild /p:RestoreSources=/tmp/some-source%3Bhttps:%2F%2Fapi.nuget.org/v3/index.json /t:restore test.csproj /v:diag > output

[cdmihai]

Another workaround is to ensure that the argument does not begin with a unix like path:
/p:RestoreSources=https://api.nuget.org/v3/index.json%3b/tmp/some-source

[rainersigwald]

dotnet/cli#5539 went in for RTW, so I'm moving this more-permanent-but-more-risky fix out.

[emgarten]

I'm hitting issues with this on Linux and OSX with the restore target. I've had to order the sources so that http is first as mentioned in @cdmihai's workaround. This will work, but it isn't ideal since the order of sources has some impact on things and restore can no longer keep the original config order.

[cparen]

I hit this as well on ItemGroups (is this the same issue?).

I see the motivation -- item groups are supposed to be for files, where this normalization would be a convenience. OTOH, a lot of built in tasks violate this assumption, like WriteLinesToFile which (ab)uses items for holding text.

For now, I'm using a replacement character (e.g. __ to mean backslash) and postprocessing the generated files with sed.

[stan-sz]

Here's a small repro:

<Project Sdk="Microsoft.Build.NoTargets">
  <Target Name="Foo" BeforeTargets="Build">
    <Exec Command="/bin/echo Display \\ oci://" />
    <Exec Command="/bin/echo Display \ oci://" />
    <Exec Command="/bin/echo Display // oci://" />
    <Exec Command="/bin/echo Display / oci://" />
  </Target>
</Project>

On Ubuntu 20.04 observe this output:

  Determining projects to restore...
  All projects are up-to-date for restore.
  Display / oci:/
  Display / oci:/
  Display // oci://
  Display / oci://

Context: Latest Helm supports pushing helm charts to a container registry, though the URL has to be prefixed with oci:// protocol.

[r3h0]

@stan-sz, more that don't work: \, $([MSBuild]::Escape(\)). However, %5C does work for me. Here's how I used it for a multi-line command:

  <Target Name="GenerateGrpcSources" BeforeTargets="CoreCompile">
    <Exec Command="/bin/bash -ce &quot;
    echo 'Generating code...';
    $(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools/linux_x64/protoc %5C
        -I$(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools/google/protobuf %5C
        --proto_path=$(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools %5C
        --proto_path=$(ProjectDir)Protobuf %5C
        --csharp_out=$(ProjectDir)Messages %5C
        $(ProjectDir)Protobuf/*.proto %5C
        ;
    echo 'Done generating code.';
    &quot;" />
  </Target>

[lordscarlet]

@stan-sz, more that don't work: \, $([MSBuild]::Escape(\)). However, %5C does work for me. Here's how I used it for a multi-line command:

  <Target Name="GenerateGrpcSources" BeforeTargets="CoreCompile">
    <Exec Command="/bin/bash -ce &quot;
    echo 'Generating code...';
    $(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools/linux_x64/protoc %5C
        -I$(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools/google/protobuf %5C
        --proto_path=$(NuGetPackageRoot)google.protobuf.tools/3.5.1/tools %5C
        --proto_path=$(ProjectDir)Protobuf %5C
        --csharp_out=$(ProjectDir)Messages %5C
        $(ProjectDir)Protobuf/*.proto %5C
        ;
    echo 'Done generating code.';
    &quot;" />
  </Target>

this did the trick for me when dealing with a regex in a property

[Happypig375]

How is this open for 6 years with Priority:1 removed

[ByronMayne]

This just burnt me as well. I was sending in a url http://website.com and it would strip the second slash to http:/website.com making it an invalid url

[lonix1]

Related to #3468. This is a major source of frustration on linux, because it takes ages to discover the underlying issue. Completely unexpected behaviour.