ReferenceCopyLocalPaths no longer content NuGetSourceType metadata

[rainersigwald]

From @joeltankam on Friday, September 20, 2019 2:22:39 PM

Steps to reproduce

I encounter some missing metadata to ReferenceCopyLocalPaths item since migrating to Microsoft.NET.Sdk format in project files.

Lets consider the following project file :

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net471</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Newtonsoft.Json" Version="12.0.2" />
  </ItemGroup>

</Project>

I use the following code to print metadata from ReferenceCopyLocalPaths items :

  <UsingTask TaskName="GetMetadataTask" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll" >
    <ParameterGroup>
      <Items ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
      <MetadataString Output="true" />
    </ParameterGroup>
    <Task>
      <Using Namespace="System"/>
      <Code Type="Fragment" Language="cs">
        <![CDATA[
            StringBuilder metadata = new StringBuilder();
            foreach (var item in Items)
            {
                metadata.AppendFormat("{0}\r\n", item);
                foreach (string name in item.MetadataNames)
                {
                    metadata.AppendFormat("  {0}={1}\r\n", name, item.GetMetadata(name));
                }
                metadata.AppendFormat("\r\n");
            }
            MetadataString = metadata.ToString();
        ]]>
      </Code>
    </Task>
  </UsingTask>

  <Target Name="_SetNuGetSourceTypeToCopyLocal" AfterTargets="ResolveAssemblyReferences">
    <ItemGroup>
      <PrintItems Include="@(ReferenceCopyLocalPaths)" Condition="'%(ReferenceCopyLocalPaths.NuGetPackageId)' != ''" />
    </ItemGroup>
    <GetMetadataTask Items="@(PrintItems)">
      <Output TaskParameter="MetadataString" PropertyName="MyBeautifulMetadata"/>
    </GetMetadataTask>
    <Message Importance="High" Text="$(MyBeautifulMetadata)" />
  </Target>

Expected behavior

When using old project files format, I had the following metadata :

  ItemName=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    NuGetPackageId=Newtonsoft.Json
    NuGetPackageVersion=12.0.2
    Private=false
    NuGetIsFrameworkReference=false
    NuGetSourceType=Package
    FullPath=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    RootDir=C:\
    Filename=Newtonsoft.Json
    Extension=.dll
    RelativeDir=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\
    Directory=Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\
    RecursiveDir=
    Identity=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    ModifiedTime=2019-04-22 01:06:16.0000000
    CreatedTime=2019-09-18 10:16:19.6451378
    AccessedTime=2019-09-18 10:16:19.6451378
    DefiningProjectFullPath=C:\MyBeautifulProject\Project.csproj
    DefiningProjectDirectory=C:\MyBeautifulProject\
    DefiningProjectName=Project
    DefiningProjectExtension=.csproj

Actual behavior

Now I get the following :

ItemName=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    NuGetPackageId=Newtonsoft.Json
    NuGetPackageVersion=12.0.2
    FullPath=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    RootDir=C:\
    Filename=Newtonsoft.Json
    Extension=.dll
    RelativeDir=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\
    Directory=Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\
    RecursiveDir=
    Identity=C:\Users\joel.tankam\.nuget\packages\newtonsoft.json\12.0.2\lib\net45\Newtonsoft.Json.dll
    ModifiedTime=2019-04-22 01:06:16.0000000
    CreatedTime=2019-09-18 10:16:19.6451378
    AccessedTime=2019-09-18 10:16:19.6451378
    DefiningProjectFullPath=C:\MyBeautifulProject\Project.csproj
    DefiningProjectDirectory=C:\MyBeautifulProject\
    DefiningProjectName=Project
    DefiningProjectExtension=.csproj

The difference being the absence of :

    Private=false
    NuGetIsFrameworkReference=false
    NuGetSourceType=Package

However, I need the NuGetSourceType metadata in my build process.
Is there please any reason why this this metadata disappeared ? Is there a way to set it back ?

I currently use this custom target to set back NuGetSourceType :

  <Target Name="_SetNuGetSourceTypeToCopyLocal" AfterTargets="ResolveReferences">
    <ItemGroup>
      <ReferenceCopyLocalPaths Update="@(ReferenceCopyLocalPaths)" Condition="'%(ReferenceCopyLocalPaths.NuGetPackageId)' != ''">
        <NuGetSourceType>Package</NuGetSourceType>
      </ReferenceCopyLocalPaths>
    </ItemGroup>
  </Target>

Environment data

msbuild /version output: 15.9.21.664
.NET Core SDK version : 2.2.108
Visual Studio : Professional 2017, v 15.9.16

Copied from original issue: dotnet/msbuild#4754

[livarcocc]

@dsplaisted

[dsplaisted]

@joeltankam

SDK style projects use different code to resolve NuGet references, and I don't think we intend to guarantee that all the metadata is the same. While everything in the build process is essentially public, the reason we have that metadata is for the MSBuild / .NET build logic, not explicitly for custom build logic.

If you're not familiar with it, I recommend using binary logs and MSBuild structured log viewer: http://msbuildlog.com/ That will let you easily see item metadata, task inputs and outputs, etc., without having to use a custom target to dump the information or trying to parse through gigantic text log files.

[joeltankam]

@dsplaisted thanks for your reply.

MSBuild / .NET build logic, not explicitly for custom build logic.

The build process I was talking about is actually in Microsoft.VsSdk.targets, used for Vsix projects. It uses '%(ReferenceCopyLocalPaths.NuGetSourceType)' == 'Package' to determine package elements :

<ItemGroup Condition="'$(IncludePackageReferencesInVSIXContainer)'=='true'">
      <_ReferenceCopyLocalPathsFromNuGet Include="@(ReferenceCopyLocalPaths)" Condition=" '%(ReferenceCopyLocalPaths.NuGetSourceType)' == 'Package' " />
</ItemGroup>

Therefore, I need this metadata since I want package assemblies to be included in the vsix archive.

That will let you easily see item metadata, task inputs and outputs, etc., without having to use a custom target to dump the information or trying to parse through gigantic text log files.

Thanks for the tip. In this case I just found more practical (across many runs) to print that through a target.

[joeltankam]

Any update please ?

[jimmylewis]

I also ran into this issue. @dsplaisted, I understand the viewpoint that

SDK style projects use different code to resolve NuGet references, and I don't think we intend to guarantee that all the metadata is the same.

Is this something that the .NET SDK would consider preserving for back-compatibility with the VS SDK? Or should this issue be routed to the VS SDK team instead to change their targets?

[dsplaisted]

I would try to get it fixed in the VS SDK. We are trying to reduce the amount of duplicate metadata we have, see here: #3850

[kfertitta]

Is there any update on this issue? I'm still seeing it from my SDK-style project with the latest VSSDK.

[wade0016]

The VSSDK BuildTools team is investigating updates to the logic which handles local reference items introduced via PackageReference