Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error MSB4018: The "DetectSigningIdentity" task failed unexpectedly. #5277

Closed
rolfbjarne opened this issue Dec 17, 2018 · 9 comments
Closed
Labels
bug If an issue is a bug or a pull request a bug fix iOS Issues affecting iOS macOS Issues affecting macOS vs-sync *Only use when triaging (not before)* Syncs this bug with VSTS
Milestone

Comments

@rolfbjarne
Copy link
Member

rolfbjarne commented Dec 17, 2018

There have been multiple reports about a failure in the DetectSigningIdentity task:

"tomswiftydevicetests.csproj" (default target) (1) ->
(_DetectSigningIdentity target) -> 
  /Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018: The "DetectSigningIdentity" task failed unexpectedly. [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018: System.ArgumentException: An item with the same key has already been added. Key: com.apple.developer.pass-type-identifiers [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) [0x000ad] in <71f4e3d7d65e42b0a699d02e59766d3c>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) [0x00000] in <71f4e3d7d65e42b0a699d02e59766d3c>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PDictionary.Add (System.String key, Xamarin.MacDev.PObject value) [0x00001] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PropertyListFormat+XmlFormat+Context.ReadDict (Xamarin.MacDev.PDictionary dict) [0x000dc] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PropertyListFormat+ReadWriteContext.ReadObject () [0x000ed] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PropertyListFormat+XmlFormat+Context.ReadDict (Xamarin.MacDev.PDictionary dict) [0x000c9] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PropertyListFormat+ReadWriteContext.ReadObject () [0x000ed] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PObject.FromByteArray (System.Byte[] array, System.Int32 startIndex, System.Int32 length, System.Boolean& isBinary) [0x00035] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PDictionary.FromByteArray (System.Byte[] array, System.Int32 startIndex, System.Int32 length, System.Boolean& isBinary) [0x00001] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.PDictionary.FromBinaryXml (System.Byte[] array) [0x00041] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.MobileProvision.LoadFromFile (System.String fileName) [0x0000e] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.MobileProvisionIndex.GetMobileProvisions (Xamarin.MacDev.MobileProvisionPlatform platform, System.String bundleIdentifier, Xamarin.MacDev.MobileProvisionDistributionType type, System.Collections.Generic.IList`1[T] developerCertificates, System.Boolean includeExpired, System.Boolean unique, System.Collections.Generic.List`1[T] failures) [0x0041d] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.Tasks.DetectSigningIdentityTaskBase.GetProvisioningProfiles (Xamarin.MacDev.MobileProvisionPlatform platform, Xamarin.MacDev.MobileProvisionDistributionType type, Xamarin.MacDev.Tasks.DetectSigningIdentityTaskBase+CodeSignIdentity identity, System.Collections.Generic.IList`1[T] certs) [0x0002a] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Xamarin.MacDev.Tasks.DetectSigningIdentityTaskBase.Execute () [0x00945] in <1619c6ec775046a2b5639a9558f65f7a>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute () [0x00023] in <2338fdb301034c1d9c6b95e564d514b0>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(646,3): error MSB4018:   at Microsoft.Build.BackEnd.TaskBuilder+<ExecuteInstantiatedTask>d__26.MoveNext () [0x00212] in <2338fdb301034c1d9c6b95e564d514b0>:0  [/Users/builder/jenkins/workspace/swift-o-matic/maccore/tools/tom-swifty/tests/tom-swifty-test/tomswiftydevicetests/iphone/tomswiftydevicetests.csproj]

Sources:

https://stackoverflow.com/questions/53794998/duplicate-key-in-provisioning-profile-causes-build-failure-error-msb4018
https://forums.xamarin.com/discussion/146770
https://twitter.com/craigajohnson/status/1074031845275099136?s=21

It seems to be an Apple change in how they generate provisioning profiles:

https://forums.developer.apple.com/message/343811#343811

VS bug #752874, VS bug #752875

@rolfbjarne rolfbjarne added bug If an issue is a bug or a pull request a bug fix macOS Issues affecting macOS iOS Issues affecting iOS labels Dec 17, 2018
@rolfbjarne rolfbjarne added this to the Future milestone Dec 17, 2018
@rolfbjarne
Copy link
Member Author

I'm not sure it's possible to create an exact test case anymore, because:

  • We load all the provisioning profiles in a system, and cache the information.
  • If there's a problem loading a provisioning profile, we don't add it to the cache (but we don't surface the exception to the user). We just fail to load the profile, and we won't use/find it during our build, nor will it show up in VSfM's Bundle Signing property page.
  • Modifying an existing provisioning profile effectively re-creates it with a new filename, which means the cache won't have the modified provisioning profile.
  • The reported exception occurs when loading the profile from the cache.

This all means that the problem can only occur when an existing provisioning profile (which we've already cached) is updated (in a way that the filename does not change - just re-downloading existing profiles didn't seem to generate problematic ones with duplicate keys, so there's something else triggering this on Apple's side).

That said, I can reproduce the exception itself locally when trying to cache a provisioning profile with duplicate keys (even though the exception isn't surfaced and doesn't cause a build failure).

@rolfbjarne
Copy link
Member Author

Fix in progress: dotnet/macios-devtools#39

rolfbjarne added a commit to dotnet/macios-devtools that referenced this issue Dec 17, 2018
rolfbjarne added a commit that referenced this issue Dec 17, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@2029b17 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277. (#39)
* dotnet/macios-devtools@80498aa Get rid of spurious <ItemGroup>
* dotnet/macios-devtools@9921b48 Remove packagerefs. Leads to unintended problems (see discussion xamarin/md-addins#3946) (#38)
* dotnet/macios-devtools@179f720 Add in MicroBuild to all projects for signing (#37)
* dotnet/macios-devtools@45e26fc Cache EntitlementExtensions.AllKeys

Diff: dotnet/macios-devtools@49a17b1...2029b17
rolfbjarne added a commit to rolfbjarne/Xamarin.MacDev that referenced this issue Dec 17, 2018
rolfbjarne added a commit to rolfbjarne/Xamarin.MacDev that referenced this issue Dec 17, 2018
@rolfbjarne
Copy link
Member Author

Reopening since the fix has not yet been bumped into xamarin-macios.

@rolfbjarne rolfbjarne reopened this Dec 17, 2018
@spouliot spouliot added the vs-sync *Only use when triaging (not before)* Syncs this bug with VSTS label Dec 17, 2018
@spouliot spouliot modified the milestones: Future, d15-9 Dec 17, 2018
@xamarin-release-manager xamarin-release-manager modified the milestones: d15-9, Future Dec 17, 2018
@xamarin-release-manager
Copy link
Collaborator

[VS sync] The field 'Milestone' contains the value '15.9' that is not in the list of supported values

rolfbjarne added a commit that referenced this issue Dec 17, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@2029b17 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277. (#39)
* dotnet/macios-devtools@80498aa Get rid of spurious <ItemGroup>
* dotnet/macios-devtools@9921b48 Remove packagerefs. Leads to unintended problems (see discussion xamarin/md-addins#3946) (#38)
* dotnet/macios-devtools@179f720 Add in MicroBuild to all projects for signing (#37)
* dotnet/macios-devtools@45e26fc Cache EntitlementExtensions.AllKeys

Diff: dotnet/macios-devtools@49a17b1...2029b17
rolfbjarne added a commit to dotnet/macios-devtools that referenced this issue Dec 17, 2018
rolfbjarne added a commit that referenced this issue Dec 17, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@44e90a2 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277. (#39) (#41)
* dotnet/macios-devtools@c81c625 Get rid of spurious <ItemGroup>
* dotnet/macios-devtools@d1d3fa4 Remove packagerefs. Leads to unintended problems (see discussion xamarin/md-addins#3946) (#38)

Diff: dotnet/macios-devtools@179f720...44e90a2

Fixes #5277.
rolfbjarne added a commit to dotnet/macios-devtools that referenced this issue Dec 17, 2018
rolfbjarne added a commit that referenced this issue Dec 17, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@ba1629e Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277. (#39) (#40)
* dotnet/macios-devtools@21b7270 Cache EntitlementExtensions.AllKeys
* dotnet/macios-devtools@44b2fee Added new Xcode10 Watch Complication values (#34)
* dotnet/macios-devtools@e9e1bb1 Merge pull request #36 from xamarin/vsts-702327-d15-9
* dotnet/macios-devtools@e348034 VSTS 702327: Launch image sizes for iPhone XS Max and iPhone XR (#35)
* dotnet/macios-devtools@61e811c Use Version.TryParse
* dotnet/macios-devtools@62cdeac Added API to get the recommended Xcode version value
* dotnet/macios-devtools@3adc4a9 Merge pull request #33 from xamarin/vsts-692176-release-7.7
* dotnet/macios-devtools@25883d4 VSTS-692176: Add missing AutoFill Credential Provider entitlement

Diff: dotnet/macios-devtools@39ea45b...ba1629e

Fixes #5277.
rolfbjarne added a commit to dotnet/macios-devtools that referenced this issue Dec 17, 2018
rolfbjarne added a commit that referenced this issue Dec 17, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@9b55404 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277.

Diff: dotnet/macios-devtools@39ea45b...9b55404

Fixes #5277.
rolfbjarne added a commit that referenced this issue Dec 17, 2018
Bump Xamarin.MacDev to get fix for #5277, and use a custom Xamarin.MacDev
branch to only get this commit from Xamarin.MacDev.

Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@9b55404 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277.

Diff: dotnet/macios-devtools@39ea45b...9b55404

Fixes #5277.
@rolfbjarne rolfbjarne pinned this issue Dec 17, 2018
@rolfbjarne
Copy link
Member Author

This error might still happen after the fix:

/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/iOS/Xamarin.iOS.Common.targets(1490,3): error : Could not locate the provisioning profile with a Name or UUID of [UUID].

This is because of how we cache some data for provisioning profiles. This is what happens:

  • Previously (before the fix) we'd detect the new change in any of the provisioning profiles, load them all again, fail to load the broken one, ignore it, and finally update the cache with the results we loaded (which would not ignore the broken profile).
  • After the fix, none of the provisioning profiles changed, so the cache is considered up-to-date. At build time we find the profile in the system (without looking in the cache), and at a later point in the same build we try to get it from the cache (where it's not), and the build fails.

This can be fixed in two ways:

  1. Invalidate the cache by touching the directory where the provisioning profiles are stored:
touch ~/Library/MobileDevice/"Provisioning Profiles"
  1. Invalidate the cache by removing it completely:
rm -f ~/Library/Xamarin/"Provisioning Profiles.index"

@rolfbjarne
Copy link
Member Author

It seems Apple has fixed the issue on their end, and they're serving provisioning profiles without duplicate keys now.

A customer confirmed this here: #5286 (comment), and two customers here as well: https://forums.developer.apple.com/message/343811#343811

I was also able to confirm it:

  • I re-downloaded (without re-generating it) the provisioning profile I used to test this yesterday, and it hadn't changed (the duplicate key was still there).
  • I re-generated the provisioning profile, and now the duplicate key was gone.

@rolfbjarne rolfbjarne modified the milestones: d15-9, d16-0 Dec 18, 2018
@xamarin-release-manager
Copy link
Collaborator

[VS sync] The field 'Milestone' contains the value '16.0' that is not in the list of supported values

@rolfbjarne
Copy link
Member Author

Closing since the fix is already in master, and we won't be backporting to d15-9 (we'll backport to d16-0, but there's no urgency there).

rolfbjarne added a commit that referenced this issue Dec 19, 2018
Commit list for xamarin/Xamarin.MacDev:

* dotnet/macios-devtools@44e90a2 Keys in property list dictionaries aren't necessarily unique. Fixes xamarin-macios#5277. (#39) (#41)
* dotnet/macios-devtools@c81c625 Get rid of spurious <ItemGroup>
* dotnet/macios-devtools@d1d3fa4 Remove packagerefs. Leads to unintended problems (see discussion xamarin/md-addins#3946) (#38)

Diff: dotnet/macios-devtools@179f720...44e90a2

Fixes #5277.
@rolfbjarne rolfbjarne unpinned this issue Dec 19, 2018
@imsam67
Copy link

imsam67 commented Apr 1, 2022

Everything was working fine for me. I upgraded my Visual Studio 2022 to the latest version (17.1.3) and now I'm wrestling with this issue! I removed all provisioning profiles and had VS download a fresh one for me but that didn't fix the issue. Here's my SO question: https://stackoverflow.com/questions/71702055/prepareappbundle-task-failed-unexpectedly-xamarin-forms

@ghost ghost locked as resolved and limited conversation to collaborators May 4, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug If an issue is a bug or a pull request a bug fix iOS Issues affecting iOS macOS Issues affecting macOS vs-sync *Only use when triaging (not before)* Syncs this bug with VSTS
Projects
None yet
Development

No branches or pull requests

4 participants