System.PlatformNotSupportedException : The PKCS#12 Exportable flag is not supported on this platform. breaking many tests on tvOS simulator

[danmoseley]

This is failing many tests in System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_*

System.PlatformNotSupportedException : The PKCS#12 Exportable flag is not supported on this platform.


Stack trace
   at Internal.Cryptography.Pal.AppleCertificatePal.FromDerBlob(ReadOnlySpan`1 rawData, X509ContentType contentType, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at Internal.Cryptography.Pal.AppleCertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_SingleCert.ReadPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedCert, Action`1 otherWork, X509KeyStorageFlags flags)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_SingleCert.ReadMultiPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedSingleCert, X509Certificate2[] expectedOrder, X509KeyStorageFlags nonExportFlags, Action`1 perCertOtherWork)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests.ReadMultiPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedSingleCert, X509Certificate2[] expectedOrder, Action`1 perCertOtherWork)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests.TwoCerts_TwoKeys_ManySafeContentsValues(Boolean invertCertOrder, Boolean invertKeyOrder)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

https://dev.azure.com/dnceng/public/_build/results?buildId=1313534&view=ms.vss-test-web.build-test-results-tab&runId=38657078&resultId=142723&paneView=debug

@mdh1418 this is making a lot of noise when looking through staging failures.

@dotnet/area-system-security is the correct change here to just disable these tests permanently for tvOS?

[ghost]

Tagging subscribers to this area: @bartonjs, @vcsjones, @krwq, @GrabYourPitchforks
See info in area-owners.md if you want to be subscribed.

Issue Details

---

This is failing many tests in System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_*

System.PlatformNotSupportedException : The PKCS#12 Exportable flag is not supported on this platform.


Stack trace
   at Internal.Cryptography.Pal.AppleCertificatePal.FromDerBlob(ReadOnlySpan`1 rawData, X509ContentType contentType, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at Internal.Cryptography.Pal.AppleCertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at Internal.Cryptography.Pal.CertificatePal.FromBlob(ReadOnlySpan`1 rawData, SafePasswordHandle password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_SingleCert.ReadPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedCert, Action`1 otherWork, X509KeyStorageFlags flags)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests_SingleCert.ReadMultiPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedSingleCert, X509Certificate2[] expectedOrder, X509KeyStorageFlags nonExportFlags, Action`1 perCertOtherWork)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests.ReadMultiPfx(Byte[] pfxBytes, String correctPassword, X509Certificate2 expectedSingleCert, X509Certificate2[] expectedOrder, Action`1 perCertOtherWork)
   at System.Security.Cryptography.X509Certificates.Tests.PfxFormatTests.TwoCerts_TwoKeys_ManySafeContentsValues(Boolean invertCertOrder, Boolean invertKeyOrder)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

https://dev.azure.com/dnceng/public/_build/results?buildId=1313534&view=ms.vss-test-web.build-test-results-tab&runId=38657078&resultId=142723&paneView=debug

@mdh1418 this is making a lot of noise when looking through staging failures.

@dotnet/area-system-security is the correct change here to just disable these tests permanently for tvOS?

Author:	danmoseley
Assignees:	-
Labels:	area-System.Security, os-tvos
Milestone:	-

[danmoseley]

was in #57992

[danmoseley]

Looks like it's also net6.0-MacCatalyst-Release-x64-Mono_Release-OSX.1015.Amd64.Open

[mdh1418]

Thanks! There were recent changes to throw PNSE on that flag for iOS,tvOS,MacCatalyst in #57153, same for Android with #57494
Theres a PR open to skip the tests because of the PNSE, in addition to skipping other test suites crashing on iOS,tvOS,MacCatalyst as well #57208

[danmoseley]

There were recent changes to throw PNSE on that flag for iOS,tvOS,MacCatalyst in #57153, same for Android with #57494
Theres a PR open to skip the tests because of the PNSE, in addition to skipping other test

@mdh1418 just curious, why didn't the first PR disable the tests? Didn't the failures show up in PR validation? That caused a bunch of noise, me opening this issue, etc. and then folks are less inclined to look at runtime-staging results as so many are failing.

[mdh1418]

@danmoseley Ah, it was a lapse. I didn't properly check the runtime-staging lane for iOS/tvOS/MacCatalyst and missed out on System.Security.Cryptography test suite actually having failures vs being a crashed suite since there are suites crashing on each PR that are partially logged here #53624. We are currently investigating why they're crashing

[danmoseley]

It's cool, just wondering whether we had a process failure.

Yes, there's a chicken and egg here, the more existing noise in the staging pipeline, the easier it is to add more failures to the staging pipeline. 😢