NU1505: Duplicate 'PackageDownload' items warning is raised when the SDK uses PackageDownload to get packages.

[nkolev92]

Describe the bug

Recently NuGet added warnings for duplicate PackageReference/PackageDownload/PackageVersion items. Duplicate items could lead to an unpredictable behavior. You can learn about the motivation more in NuGet/Home#9864.
https://github.com/NuGet/Home/blob/dev/proposed/2022/duplicate-nuget-item-error-handling.md

NuGet added this in 6.0.400, so customers will start seeing this very soon.

To Reproduce

Originally filled in NuGet/Home#11720.

1. Use latest SDK with NuGet version 6.3.0 or higher. Might not be 100% inserted today.
2. Create a .NET Core Console App, netcoreapp3.1. project with PackageReference package installed.
3. Run “dotnet restore” in Developer Command Prompt for VS 2022 Preview
4. The warning “warning NU1505: Duplicate 'PackageDownload' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior.” occurs as below screenshot.

This was was discovered by the NuGet test team, @v-luzh Can you please post the csproj in question?

Why is this important?

NuGet will raise a warning when it sees duplicate items, and warning as errors could cause failures for customers.

Exceptions (if any)

Further technical details

• Include the output of dotnet --info
• The IDE (VS / VS Code/ VS4Mac) you're running on, and its version
  Latest main version of NuGet tooling will have this.

[v-crchang]

@nkolev92 The repro project: Netcoreapp.zip

[Shyam-Gupta]

I am also hitting this issue while building WinForms designer repo locally: https://github.com/dotnet/winforms-designer.
It is a private repo.

[baronfel]

We appear to be adding multiple copies of the same apphost during the ResolveAppHosts task:

[baronfel]

This is now fixed for the 6.0.4xx and 7.0.1xx branches, and will flow from those.

[nkolev92]

Thanks for the quick fix!

[NickCraver]

Is it possible this shipped in 17.3.0 Preview 1? I'm hitting the same symptoms which (and I don't like to exaggerate this) does make the preview pretty much unusable (downstream it can't load types, etc. after this). If this is in the 17.3 Preview 1 bits, recommend a preview with the fix go out asap.

[NickCraver]

^ For anyone else finding this - workarounds for the moment are:

• Build from the command line where this doesn't happen
• Disable <TreatWarningsAsErrors>...and don't commit it :)

[jaredpar]

This is being hit by many members of the .NET team on the latest VS preview. Should we reactivate this bug? It's pretty clearly not fixed at this point or there is a new bug with the exact same symptoms :)

[baronfel]

@jaredpar the bug itself should be fixed, I think it's just that the SDK containing the fix hasn't flowed to the VS preview. I'm checking into that.

[nkolev92]

The NuGet change raising these warnings is in 6.0.3xx.

I should've caught it originally, but this might be a change that should be considered for that versions.

[baronfel]

@nkolev92 I only see 6.0.4xx and 7.0.100-preview4 merges associated with the NuGet.Client change, so I'm wondering how it's appearing in 6.0.300 at all? The latest 6.0.300 merge of NuGet.Client was for a 6.2.x version of NuGet.Client, so I'm unclear where the newer version was introduced. We should also be rolling forward to 6.0.400 SDK versions in future previews of VS 17.3 (per the versioning chart) so it should clear up then for early adopters.

[nkolev92]

Too early in the morning for me apparently.
I was wrong above, the NuGet change is 6.0.400, matching NuGet's 6.3, NuGet/NuGet.Client@a1afdef.

The tags start with 6.3.

[NickCraver]

I'm not sure on the urgency for this, but for what it's worth it cost me a lot of time today and for others that pinged me as well. A lot of builds have warnings as errors deeply embedded in there and we're all hacking at this passing in overrides for the properties to get by the current issue locally but for tomorrow I think it's easier for me to uninstall the preview completely. I haven't had to revert from a preview build in over a year now so this one is pretty serious IMO. I hope it makes a new preview build quickly for others hitting the issue - I think it warrants faster than a normal cadence from the number of reports I'm seeing.

[baronfel]

Hey folks, we have a good idea what happened and what the path forward is now.

Cause

NuGet in VS is independent of the NuGet shipped with the SDK. The NuGet in VS has gotten ahead of the SDK, and it includes the additional validation for duplicate PackageDownloads and PackageReferences. As a result, when the NuGet in VS interacts with a project that uses an SDK that contains the PackageDownload duplication bug, these errors appear for users.

Mitigation

All of these duplicate checks are guarded by an MSBuild condition. If you set DisableCheckingDuplicateNuGetItems to true in your project files, the error should go away. You can also set the error codes to NoWarn like so: <NoWarn>$(NoWarn);NU1505</NoWarn>. Note that since the SDK has had this bug lying latent for a while, simply moving to an older SDK version will not remove the issue. Theoretically you could also move to preview releases of either the 6.0.400 or 7.0.100 SDKs and the issue would resolve itself as well, since the SDK was fixed on both of those branches.

Resolution

The SDK will ship version 6.0.400 in VS 17.3, so it was just a matter of time until we did an insertion to that preview. We will accelerate our timeline and do the insertion early next week (after dotnet/installer#13827 merges). That will unblock the internal dogfood previews of 17.3 for internal Microsoft users. For the broader public, this will not be fixed until 17.3 Preview 2 is released, which should be roughly a month from now.

[NickCraver]

@baronfel Thank you for the update! I personally haven't used the internal previews (because you can't update unless you're on the VPN - installer hangs) - I'm on the public preview here. For what it's worth, this issue is serious/blocking enough I hopped off the preview train this morning and did a full re-install to stable. I regularly work on a few dozen solutions, so to see how common this might be (I know I'm a small sample size), I tested 19 different repos from internal and open source and this bug breaks my ability to build every single one. I was kind of hoping this was worthy of a Preview 1.1 so I could continue on preview and help find these sorts of issues.

Is it not affecting that many users overall for some reason? Or have not that many users grabbed latest preview? I'm doubting many people went to grab a nightly SDK but...maybe?

[baronfel]

@NickCraver It's not the ideal situation for sure! I'll see what levers I can pull for a faster fix on the public release, though.

[kenans]

@baronfel Just upgraded to the official 17.3.0 release, but the issue still seems to exist while building one of our NetCore3.1 apps,

Duplicate 'PackageDownload' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior. The duplicate 'PackageDownload' items are: Microsoft.NETCore.App.Host.win-x64 [3.1.25], Microsoft.NETCore.App.Host.win-x64 [3.1.25], Microsoft.NETCore.App.Host.win-x64 [3.1.25], Microsoft.NETCore.App.Host.win-x64 [3.1.25].

Right now, we set <DisableCheckingDuplicateNuGetItems> to true to mitigate the issue. But wondering how to fix the duplicate package download issue fundamentally, given Microsoft.NETCore.App.Host.win-x64 is a package that the framework references.

[baronfel]

@kenans can you confirm what version of the SDK is in use for your projects?

[kenans]

@baronfel Thanks for the quick reply. Our project has the SDK targeted to 5.0.406 in global.json, so building with dotnet CLI works fine.

The issue only happened while building in Visual Studio. I assume the cause is NuGet in VS would not honor global.json, so NU1505 (introduced in more recent NuGet versions) showed up?

[baronfel]

There's a(n overly-) complicated versioning matrix here, but I believe that's the root of the problem, yes - VS has NuGet tooling that's checking for duplicate PackageDownloads, coupled with using an SDK to build that's providing those duplicates.

I believe that your problem would be resolved if you updated your SDK to the latest 6 series - is there some blocker preventing this? You can still build net5.0-targeting projects in a higher-SDK, if that was holding you back.

[kenans]

@baronfel Thanks for the information! Will upgrade the SDK to net6 and try again.

[kenans]

@baronfel Thanks for the information! Will upgrade the SDK to net6 and try again.

For anyone who runs into the same issue -- after upgrading the SDK version, the issue is resolved.

[nolme]

Hello. Still having the problem for me using VS 2022 Pro 17.3.0 x64 Fr (up to date) :
"Duplicate 'PackageDownload' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior. The duplicate 'PackageDownload' items are: Microsoft.NETCore.App.Ref [5.0.0], Microsoft.NETCore.App.Ref [3.1.0], Microsoft.NETCore.App.Ref [3.0.0]; Microsoft.WindowsDesktop.App.Ref [5.0.0], Microsoft.WindowsDesktop.App.Ref [5.0.0], Microsoft.WindowsDesktop.App.Ref [5.0.0], Microsoft.WindowsDesktop.App.Ref [3.1.0], Microsoft.WindowsDesktop.App.Ref [3.1.0], Microsoft.WindowsDesktop.App.Ref [3.1.0], Microsoft.WindowsDesktop.App.Ref [3.0.0], Microsoft.WindowsDesktop.App.Ref [3.0.0], Microsoft.WindowsDesktop.App.Ref [3.0.0]; Microsoft.AspNetCore.App.Ref [5.0.0], Microsoft.AspNetCore.App.Ref [3.1.10], Microsoft.AspNetCore.App.Ref [3.0.1]. Nolme.Core6 C:\Users[...]\Nolme.Core6.csproj"

Installed applications :

• Microsoft .NET SDK 6.0.400 (x64) from Visual Studio
• Microsoft ASP.NET Core 6.0.8 - Shared Framework (both x86 & x64)
• Microsoft Windows Desktop Runtime - 6.0.8 (both x86 and x64

The project is attached. I found no mention of these packages :(

Nolme.Core6.zip

[knocte]

after upgrading the SDK version

To which version?

[lonix1]

I still have the problem on 6.0.400 (i.e. latest). I'm using nuget's "Central Package Management".

Repro:

MyProject/Directory.Packages.props:

<ItemGroup>
  <PackageDownload Include="minver-cli" Version="[4.1.0]" />
</ItemGroup>

MyProject/MyProject.csproj:

<ItemGroup>
  <PackageDownload Include="minver-cli" />
</ItemGroup>

dotnet build:

/home/username/code/MyProject/MyProject.csproj : warning NU1505: Duplicate 'PackageDownload' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior. The duplicate 'PackageDownload' items are: minver-cli [4.1.0], minver-cli .

[baronfel]

Hi @lonix1 - it looks like your warning isn't raised by the SDK's use of PackageDownloads (which was the cause of this issue). Your issue seems to be because you're using different PackageDownload elements with different versions, and that's not how CentralPackageManagement works (at least as far as I'm aware). CPVM requires your Directory.Package.Props to have PackageVersion items, not PackageDownload, and your project files to have PackageReference items, not PackageDownload. Can you try making those changes and see if your problem is resolved?

[julealgon]

@baronfel I just hit what appears to be an invalid case for this warning. I have some packages conditionally added in my Directory.Build.props file, but those are now being detected as duplicates even though the condition for the inclusion is false in the offending project.

Here is an example:
Directory.Build.props (slightly simplified version of my real one)

<Project>

  <PropertyGroup>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>

    <IsUnitTestProject>false</IsUnitTestProject>
    <IsUnitTestProject>$(MSBuildProjectName.EndsWith('.UnitTests'))</IsUnitTestProject>
  </PropertyGroup>

  <ItemGroup Condition="$(IsUnitTestProject)">
    <Using Include="FluentAssertions" />
    <Using Include="FluentAssertions.Execution" />
    <Using Include="NSubstitute" />
    <Using Include="Xunit" />

    <PackageReference Include="FluentAssertions" Version="6.7.0" />
    <PackageReference Include="FluentAssertions.Analyzers" Version="0.17.2">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.2.0" />
    <PackageReference Include="NSubstitute" Version="4.4.0" />
    <PackageReference Include="NSubstitute.Analyzers.CSharp" Version="1.0.15">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="coverlet.collector" Version="3.1.2">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
    <PackageReference Include="xunit" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

MyProject.Which.Is.Not.A.UnitTest.Project.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="FluentAssertions" Version="6.7.0" />
    <PackageReference Include="FluentAssertions.Analyzers" Version="0.17.2">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="MediatR" Version="10.0.1" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.2.0" />
    <PackageReference Include="NSubstitute" Version="4.4.0" />
    <PackageReference Include="NSubstitute.Analyzers.CSharp" Version="1.0.15">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="xunit.core" Version="2.4.1" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
      <PrivateAssets>all</PrivateAssets>
    </PackageReference>
  </ItemGroup>

</Project>

Error:

Error NU1504 Duplicate 'PackageReference' items found. Remove the duplicate items or use the Update functionality to ensure a consistent restore behavior. The duplicate 'PackageReference' items are: FluentAssertions 6.7.0, FluentAssertions 6.7.0; FluentAssertions.Analyzers 0.17.2, FluentAssertions.Analyzers 0.17.2; Microsoft.NET.Test.Sdk 17.2.0, Microsoft.NET.Test.Sdk 17.2.0; xunit.runner.visualstudio 2.4.5, xunit.runner.visualstudio 2.4.5.

It appears that the check is blindly looking at the presence of PackageReference elements, ignoring whether or not they are conditionally added.

I added the above code to Directory.Build.props as a means to share common setup across dozens of unit testing projects, but I have a couple of "testing libraries" that are not considered "unit test projects", but still rely on some of the same packages that unit test projects do.

This was working fine until I updated Visual Studio to latest versions (I'm using 17.3.1 right now). It is possible it was introduced in 17.3.0 but I actually don't remember if I tried to build this project in between those 2 updates.

Note that my example here has nothing to do with SDK packages... its throwing for explicit libraries that I have in the projects.

I'll add the ignore flag for now until this is fixed.

FAKE EDIT:

While I was posting this, I decided to try to create a "minimal repro" in a test solution here. Weirdly enough, it actually took a while to happen and I don't know exactly what triggers it...

I had to keep adding new projects, fiddling with the variables in Directory.Build.props and suddenly it started throwing the warning. Then, I undid some of the stuff, built again, and it stopped raising the warning. When I got back to a previous version that was raising the warning before, it just kept working... I have no idea why it is intermittent like that.

[nkolev92]

@julealgon

The NU1504, NU1505 and NU1506 warnings were added in Visual Studio 17.3, NuGet.exe 6.3 and .NET SDK 6.0.400. All those tooling versions contain the same NuGet bits.

It appears that the check is blindly looking at the presence of PackageReference elements, ignoring whether or not they are conditionally added.

That shouldn't be the case as the check is running as part of a target that's run for each framework, so the conditions will be evaluated.

I can't really say why you're seeing the intermittent behavior, but if you are seeing issues with things that aren't coming from the SDK (such as the NU1505 warnings for packages that are added by the SDK), I'd recommend filling a new issue.

You can file issues with questions about NU1504 in the NuGet/Home repo.

[michael-hawker]

It'd be nice if this error message actually pointed to the file/source locations of the project/props/target files that are including these references. I hit this issue with the new warnings in our project, but I'm just referencing the SDK and such, so no idea where the extra package reference is coming from. Changed the PackageReference from Include to Update for now to get around it... 🤷‍♂️