Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local exporter hangs if attestation content is larger than 2^22 (4194304) bytes #3454

Closed
jedevc opened this issue Jan 4, 2023 · 1 comment

Comments

@jedevc
Copy link
Member

jedevc commented Jan 4, 2023

If the attestation content exceeds the specified size (4194304 bytes = 4 MB), then the GRPC layer rejects the malformed packet during file transfer with fsutil.

#14 exporting to client directory
#14 copying files
#14 copying files 9.59MB 5.1s
#14 copying files 56.60MB 10.2s
#14 copying files 61.14MB 15.2s
#14 copying files 64.32MB 20.3s
#14 copying files 68.11MB 25.3s
#14 copying files 70.21MB 30.4s
#14 copying files 72.55MB 35.4s
#14 copying files 135.74MB 40.5s
#14 copying files 184.37MB 45.6s
#14 copying files 188.00MB 50.6s
#14 copying files 213.86MB 55.6s
#14 copying files 217.27MB 60.6s
#14 copying files 225.96MB 65.6s
#14 copying files 229.45MB 70.7s
#14 copying files 276.34MB 75.7s
#14 copying files 346.11MB 80.8s
#14 copying files 361.03MB 85.9s
#14 copying files 363.68MB 90.9s
#14 copying files 364.19MB 96.0s
#14 copying files 367.78MB 101.1s
#14 copying files 376.29MB 106.1s
#14 copying files 377.83MB 111.2s
#14 copying files 379.99MB 116.6s
#14 copying files 385.10MB 121.7s
#14 copying files 389.25MB 126.7s
#14 copying files 395.87MB 131.8s
#14 copying files 396.40MB 136.8s
#14 copying files 399.14MB 141.9s
#14 copying files 442.43MB 146.9s
#14 copying files 444.43MB 150.8s done
#14 ERROR: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (5338399 vs. 4194304)
------
 > exporting to client directory:
------
ERROR: failed to solve: ResourceExhausted: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (5338399 vs. 4194304)

To prevent this, we need to patch fsutil, e.g. with tonistiigi/fsutil#144. The StaticFS implementation in buildkit returns bytes.Reader objects, which implement the io.WriterTo interface, which circumvents the packet size limits in fsutil.

@jedevc
Copy link
Member Author

jedevc commented Jan 6, 2023

Fixed by #3461.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant