Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File I/O Abstraction: Replace IFileResolver.TryAcquireFileLock #15826

Merged
merged 3 commits into from
Dec 12, 2024

Conversation

shenglol
Copy link
Contributor

@shenglol shenglol commented Dec 10, 2024

As part of the ongoing file I/O abstraction migration, this PR removes IFileResolver.TryAcquireFileLock and transitions it to the new file I/O API. The updates, including changes to IFeatureProvider, *ArtifactRegistry, and several related tests, are inevitable side effects of what initially seemed like a straightforward update.

Microsoft Reviewers: Open in CodeFlow

@shenglol shenglol changed the title File I/O Abstraction: Remove IFileResolver.TryAcquireFileLock File I/O Abstraction: Replace IFileResolver.TryAcquireFileLock Dec 10, 2024
Copy link
Contributor

github-actions bot commented Dec 10, 2024

Test this change out locally with the following install scripts (Action run 12290549161)

VSCode
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-vsix.sh) --run-id 12290549161
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-vsix.ps1) } -RunId 12290549161"
Azure CLI
  • Mac/Linux
    bash <(curl -Ls https://aka.ms/bicep/nightly-cli.sh) --run-id 12290549161
  • Windows
    iex "& { $(irm https://aka.ms/bicep/nightly-cli.ps1) } -RunId 12290549161"

Copy link
Contributor

github-actions bot commented Dec 10, 2024

Dotnet Test Results

    78 files   -     39      78 suites   - 39   30m 22s ⏱️ - 20m 42s
11 511 tests  -     23  11 511 ✅  -     23  0 💤 ±0  0 ❌ ±0 
26 753 runs   - 13 363  26 753 ✅  - 13 363  0 💤 ±0  0 ❌ ±0 

Results for commit 8a124ec. ± Comparison against base commit c730403.

This pull request removes 1842 and adds 633 tests. Note that renamed tests count towards both.

		nestedProp1: 1
		nestedProp2: 2
		prop1: true
		prop2: false
	*|\u0004z�-����\u0016a\u0012\u0010��\u0001�Lr\u0000`��+��U�K��\u0018g��@A]g����T���\u00101��Ptb�\u000e]{$�\u0017]\u001f��HM�Nչh���o_�������S�q���\u0001\u0001��\u0012�|,8o�?J&�<�7/Nv�U<�/��ěB;��ݓ�8�3�;�J��\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
	1
	2
	\$'")
	prop1: true
…
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
��K
�0\u0010\u0006�=EN��L^��{�^!h�\u0007M�\u000f,�w7]\u0008.Z��V0�2�!��_$;�o\u000bw(�F(�(\u0011ln\u0010\u0018�F�\u0007���}\u0001\u0006Vkb���O2�kZW�Q�8�\u0007Q�]{*�\u001c���AmQ���P�	��d
�յǼ�\u0017_ݼ87�\u001f\u001aʩ��/\u001a�z��o=ѧ�\u0003\u0010Ce� �\u0004\u0019�\u0014v�E2����?֞ ��(Z�\u0013\u001ef��\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
��K
�0\u0010\u0006�=E�\u0001b�i���\u0008n<@lG��Z�
\u0005��\u000bq��\u000f�|��@&	��zo�-�\u0014��H\u001e�5Ac����w�äA̼QD�R\u0006!܎>I�{��ʍ2�^?���6Y\u000e13B��)#\u0008\u0013�\u0019�W��\u000c�X�Kۜ�H�%��Vt��P����\u0006|�o�+���*$r�\u0000׌SA\u0015�ZI��?K&�<�\u000f\u001c\u001c
hKH\u001aHw�\u001f�
6�d�5<����<ϛ�\u000bf���\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000
��M\u000b�0\u0018\u0007��\u0014�\u0007�{�\u000c<\u0004\u001d2��k�\u001cd��\u001a\u0008}��!�(]Ԃ�;n\u000f������N7+�SSV���\u0004\u0010\u000c
Y����\u0016f\u0012`��\u0000��K\u0000�f�N:ܫZ���)��ADy��r\u0013aI�\u0012�K
1V�r>�{��=\u001cz7]���HM\u0003/յh���o_�������S�\u0011a��\u0000"0A\u0014q�\u0004g����L�y�\u001f~�\u001c\u0016�xy\﷉?��\u0004�vO��8������\u0013\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003�Խ
� \u0010\u0007p�>�O`�ԨK��}\u0005i\u0003� �$�\u0006J߽f(t0tIc��F��\u0013�ˊ�\u001d��=�]�\u0004((\u0018Y\u001a\u0004J���\u0004K$�_��.%'���O\u00121��va�5z� n�����P\u000ba\u0014J\u0003̘\u0012\u000c�M����!�Z��\u0006wq�ͱsߺ���+H}�,������S�\u00018A��B\u001e��)���i�J&�<��G�	�,˲\u0014�^x\�\u0000\u000c\u0000\u0000,"The path: index.json was not found in artifact contents")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003�Խ
�0\u0010\u0000��>E�\u0003�I�&Q� 8X�*\u0008�\u0012l�
��V(��^\u0007qiq�`��rp\u0017�;��u�6:1E�}"��Q�\u0008\u0010���\u001b�KD�\u001bAD\u0006\u0010AN�{'-\u001ee�\u000bhe�Z?�)GWifB*}_	�\u0015�JR\u001a05��s�\u0019�;w]]�4OL���-oҳ����iu���p5��?a\u001c�\u000e`�2\u0002�\u0005\u0011\u0011p\u0006�?�L���?�(>.���9�bw��\u0006��'˲,kx/��϶\u0000\u000c\u0000\u0000,"'7' is an invalid end of a number. Expected a delimiter. Path: $.INVALID_JSON | LineNumber: 0 | BytePositionInLine: 20.")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Bicep_compiler_handles_corrupted_extension_package_gracefully (\u001f�\u0008\u0000\u0000\u0000\u0000\u0000\u0000\u0003��K
�0\u0010\u0006�=E�\u0001Ҽ�F�{7�\u001b\u000f\u0010�\u0011+���BA���Bܴ��C0�r20\u0013�\u001f\u0012�M�\u0005�Am����\u0004��:J��z��\u0018���"\u001aG��p;�&=�1�[e�Y?�kl�����BhŤ�Dk�$�+w�\u000e��5�LsN�2��\��ڋ�����
���t3��
��\u001f�\u0015�E\u001cQ\u0015I��?K&�<�\u000f\u001c\u001cJh+H\u001b�vP\u001c�\u000e6�d�\u0016�K��y��M�\u0005�\u0000\u000c\u0000\u0000,"Value cannot be null. (Parameter 'source')")
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Repository_not_found_in_registry (ArtifactRegistryAddress { RegistryAddress = mcr.microsoft.com, RepositoryPath = unknown/path/az, ExtensionVersion = 0.0.0-placeholder },Azure.RequestFailedException: The artifact does not exist in the registry.
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 138,[(BCP192, Error, Unable to restore the artifact with reference "br:mcr.microsoft.com/unknown/path/az:0.0.0-placeholder": The artifact does not exist in the registry.)])
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Repository_not_found_in_registry (ArtifactRegistryAddress { RegistryAddress = mcr.microsoft.com, RepositoryPath = unknown/path/az, ExtensionVersion = 0.0.0-placeholder },Azure.RequestFailedException: The artifact does not exist in the registry.
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 138,[(BCP192, Error, Unable to restore the artifact with reference "br:mcr.microsoft.com/unknown/path/az:0.0.0-placeholder": The artifact does not exist in the registry.)])
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Repository_not_found_in_registry (ArtifactRegistryAddress { RegistryAddress = unknown.registry.azurecr.io, RepositoryPath = bicep/extensions/az, ExtensionVersion = 0.0.0-placeholder },System.AggregateException: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443))
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.<>c__DisplayClass4_0.<<PullArtifactAsync>g__DownloadManifestInternalAsync|0>d.MoveNext() in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 44
--- End of stack trace from previous location ---
   at Bicep.Core.Registry.AzureContainerRegistryManager.PullArtifactAsync(RootConfiguration configuration, IOciArtifactReference artifactReference) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 51
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.<>c__DisplayClass4_0.<<PullArtifactAsync>g__DownloadManifestInternalAsync|0>d.MoveNext() in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 44
--- End of stack trace from previous location ---
   at Bicep.Core.Registry.AzureContainerRegistryManager.PullArtifactAsync(RootConfiguration configuration, IOciArtifactReference artifactReference) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/AzureContainerRegistryManager.cs:line 63
   at Bicep.Core.Registry.OciArtifactRegistry.TryRestoreArtifactAsync(RootConfiguration configuration, OciArtifactReference reference) in /home/runner/work/bicep/bicep/src/Bicep.Core/Registry/OciArtifactRegistry.cs:line 495,[(BCP192, Error, Unable to restore the artifact with reference "br:unknown.registry.azurecr.io/bicep/extensions/az:0.0.0-placeholder": Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)))])
Bicep.Core.IntegrationTests.AzTypesViaRegistryTests ‑ Repository_not_found_in_registry (ArtifactRegistryAddress { RegistryAddress = unknown.registry.azurecr.io, RepositoryPath = bicep/extensions/az, ExtensionVersion = 0.0.0-placeholder },System.AggregateException: Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443))
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.<>c__DisplayClass4_0.<<PullArtifactAsync>g__DownloadManifestInternalAsync|0>d.MoveNext() in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 44
--- End of stack trace from previous location ---
   at Bicep.Core.Registry.AzureContainerRegistryManager.PullArtifactAsync(RootConfiguration configuration, IOciArtifactReference artifactReference) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 51
   at Bicep.Core.Registry.AzureContainerRegistryManager.DownloadManifestAndLayersAsync(IOciArtifactReference artifactReference, ContainerRegistryContentClient client) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 138
   at Bicep.Core.Registry.AzureContainerRegistryManager.<>c__DisplayClass4_0.<<PullArtifactAsync>g__DownloadManifestInternalAsync|0>d.MoveNext() in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 44
--- End of stack trace from previous location ---
   at Bicep.Core.Registry.AzureContainerRegistryManager.PullArtifactAsync(RootConfiguration configuration, IOciArtifactReference artifactReference) in D:\a\bicep\bicep\src\Bicep.Core\Registry\AzureContainerRegistryManager.cs:line 63
   at Bicep.Core.Registry.OciArtifactRegistry.TryRestoreArtifactAsync(RootConfiguration configuration, OciArtifactReference reference) in D:\a\bicep\bicep\src\Bicep.Core\Registry\OciArtifactRegistry.cs:line 495,[(BCP192, Error, Unable to restore the artifact with reference "br:unknown.registry.azurecr.io/bicep/extensions/az:0.0.0-placeholder": Retry failed after 4 tries. Retry settings can be adjusted in ClientOptions.Retry or by configuring a custom retry policy in ClientOptions.RetryPolicy. (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)) (No such host is known. (unknown.registry.azurecr.io:443)))])
…

♻️ This comment has been updated with latest results.

@shenglol shenglol requested a review from Copilot December 10, 2024 23:24

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 53 out of 68 changed files in this pull request and generated no suggestions.

Files not reviewed (15)
  • src/Bicep.Cli.IntegrationTests/packages.lock.json: Language not supported
  • src/Bicep.Cli.UnitTests/packages.lock.json: Language not supported
  • src/Bicep.Cli/packages.lock.json: Language not supported
  • src/Bicep.Core.IntegrationTests/packages.lock.json: Language not supported
  • src/Bicep.Core.Samples/packages.lock.json: Language not supported
  • src/Bicep.Core.UnitTests/BicepTestConstants.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/Features/FeatureProviderOverrides.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/Features/OverriddenFeatureProvider.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/Features/FeatureProviderTests.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/ApiVersion/ApiVersionProviderTests.cs: Evaluated as low risk
  • src/Bicep.Core.IntegrationTests/SourceArchiveTests.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/Assertions/OciModuleRegistryAssertions.cs: Evaluated as low risk
  • src/Bicep.Core.UnitTests/Registry/CachedModules.cs: Evaluated as low risk
  • src/Bicep.Cli.IntegrationTests/RestoreCommandTests.cs: Evaluated as low risk
  • src/Bicep.Core.IntegrationTests/ExtensionRegistryTests.cs: Evaluated as low risk
Comments skipped due to low confidence (1)

src/Bicep.Core.IntegrationTests/RegistryTests.cs:45

  • [nitpick] The variable 'badCacheDirectory' is reassigned to a file handle, which can be confusing. Consider using a different variable name for the file handle, such as 'badCacheFileHandle'.
badCacheDirectory = badCacheDirectory.GetDirectory("file.txt");
{
if (!OperatingSystem.IsWindows())
{
this.FileSystem.File.SetUnixFileMode(this.Uri.GetFileSystemPath(), UnixFileMode.UserRead | UnixFileMode.UserWrite | UnixFileMode.UserExecute);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UserWrite

Is UserWrite needed to mark it as executable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not, but chmod 700 seems to be common. Also, the original code has the UserWrite flag, so keeping it just in case.

@@ -8,6 +8,7 @@

<ItemGroup>
<PackageReference Include="System.IO.Abstractions" Version="21.1.3" />
<PackageReference Include="System.Memory.Data" Version="6.0.0" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

6.0.0

Any reason not to take 8.0.1 or 9.0.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question...the package was added by VS automatically via completion. Not sure why the old version is used. Let me bump the version.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah it's because Azure.Core has a transitive dependency on System.Memory.Data 6.0.0, so VS choosed the local version. Let's keep the current version to avoid any incompatible issues since Azure.Core is used by Bicep.Core.

Copy link
Member

@majastrz majastrz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@shenglol shenglol merged commit c404992 into main Dec 12, 2024
47 checks passed
@shenglol shenglol deleted the shenglol/file-io-abstraction-part-3 branch December 12, 2024 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants