Add the ability to run starting from a specific task (fixes #227) #377
+720
−639
firefoxci-taskcluster / toolchain-cuda-toolkit
succeeded
Feb 13, 2024 in 16m 46s
FirefoxCI (push)
CUDA Toolkit 12 preparation
Details
View task in Taskcluster
View logs in Taskcluster
[taskcluster 2024-02-13 19:58:25.375Z] Task ID: XKFBSMoJRdO-Evx0QApuSQ
[taskcluster 2024-02-13 19:58:25.375Z] Worker ID: 7660731726463137687
[taskcluster 2024-02-13 19:58:25.375Z] Worker Group: us-west1
[taskcluster 2024-02-13 19:58:25.375Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-13 19:58:25.375Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-02-13 19:58:25.375Z] Worker Version: 38.0.5
[taskcluster 2024-02-13 19:58:25.375Z] Public IP: 34.145.29.59
[taskcluster 2024-02-13 19:58:25.376Z] Hostname: translations-1-b-linux-large-gcp-b5hm8wx0s3mdnk1r2gs0yw
[taskcluster 2024-02-13 19:58:25.376Z] using cache "translations-level-1-checkouts-sparse-v3-7afeb851dd97df8f3607-Jew_2ep5Q0SJc7IaKa41Pg" -> /builds/worker/checkouts
[taskcluster 2024-02-13 19:58:30.193Z] Downloading artifact "public/image.tar.zst" from task ID: Jew_2ep5Q0SJc7IaKa41Pg.
[taskcluster 2024-02-13 19:58:35.195Z] Download Progress: 39.81%
[taskcluster 2024-02-13 19:58:40.195Z] Download Progress: 91.41%
[taskcluster 2024-02-13 19:58:41.163Z] Downloaded artifact successfully.
[taskcluster 2024-02-13 19:58:41.164Z] Downloaded 776.726 mb
[taskcluster 2024-02-13 19:58:41.164Z] Decompressing downloaded image
[taskcluster 2024-02-13 19:58:47.748Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-13 19:59:28.438Z] Image 'public/image.tar.zst' from task 'Jew_2ep5Q0SJc7IaKa41Pg' loaded. Using image ID sha256:0bda823ded0479e9bfa01cc4b80273ad1987c66c2b4e567901ee6a02a0583651.
[taskcluster 2024-02-13 19:59:28.796Z] === Task Starting ===
[setup 2024-02-13T19:59:37.075Z] run-task started in /builds/worker
[setup 2024-02-13T19:59:37.075Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --firefox_translations_training-sparse-profile=build/sparse-profiles/toolchain-build --task-cwd /builds/worker/checkouts -- vcs/taskcluster/scripts/toolchain/build-cuda-toolkit.sh
[setup 2024-02-13T19:59:37.076Z] Python version: 3.10.12
[cache 2024-02-13T19:59:37.078Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-13T19:59:37.078Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-02-13T19:59:37.078Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-13T19:59:37.078Z] running as worker:worker
[vcs 2024-02-13T19:59:37.078Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:59:37.080Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:59:37.082Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-13T19:59:38.033Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:59:38.324Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:59:38.324Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:59:38.330Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:59:38.606Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:59:38.606Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:59:38.612Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/start-specific', '37fbf272d7eb316897377144111a3ef057becfd4']
[vcs 2024-02-13T19:59:38.655Z] Switched to a new branch 'refs/heads/start-specific'
[vcs 2024-02-13T19:59:38.655Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-13T19:59:38.677Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-13T19:59:38.677Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-13T19:59:38.677Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-13T19:59:38.678Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-13T19:59:38.678Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-13T19:59:38.678Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-13T19:59:38.679Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-13T19:59:38.701Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-13T19:59:40.950Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-13T19:59:41.318Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-13T19:59:41.688Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-13T19:59:43.174Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-13T19:59:44.638Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-13T19:59:45.151Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-13T19:59:45.161Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-13T19:59:45.172Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-13T19:59:45.199Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-13T19:59:45.313Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-13T19:59:45.333Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-13T19:59:45.334Z] cleaning git checkout...
[vcs 2024-02-13T19:59:45.334Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-13T19:59:45.336Z] removing []
[vcs 2024-02-13T19:59:45.337Z] successfully cleaned git checkout!
[vcs 2024-02-13T19:59:45.338Z] TinderboxPrint:<a href='https://github.com/mozilla/firefox-translations-training/commit/37fbf272d7eb316897377144111a3ef057becfd4' title='Built from firefox-translations-training commit 37fbf272d7eb316897377144111a3ef057becfd4'>37fbf272d7eb316897377144111a3ef057becfd4</a>
[setup 2024-02-13T19:59:45.338Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[setup 2024-02-13T19:59:45.338Z] UPLOAD_DIR is /builds/worker/artifacts
[fetches 2024-02-13T19:59:45.338Z] fetching artifacts
[fetches 2024-02-13T19:59:45.339Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W1x9jZ3-RBuHmJNYmC1GJQ/artifacts/public/cuda-source.run to /builds/worker/fetches/cuda-source.run
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W1x9jZ3-RBuHmJNYmC1GJQ/artifacts/public/cuda-source.run
/usr/local/bin/fetch-content:192: DeprecationWarning: cafile, capath and cadefault are deprecated, use a custom context instead.
with urllib.request.urlopen(
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/W1x9jZ3-RBuHmJNYmC1GJQ/artifacts/public/cuda-source.run resolved to 4245586997 bytes with sha256 68699036c12d71adb9ad2799dce2ff070270fab4488b90920b9756ab3f52c41c in 53.927s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 53.929287682999984, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-13T20:00:39.363Z] finished fetching artifacts
[task 2024-02-13T20:00:39.363Z] executing ['vcs/taskcluster/scripts/toolchain/build-cuda-toolkit.sh']
[task 2024-02-13T20:00:39.365Z] + export CUDA_INSTALLER=/builds/worker/fetches/cuda-source.run
[task 2024-02-13T20:00:39.365Z] + CUDA_INSTALLER=/builds/worker/fetches/cuda-source.run
[task 2024-02-13T20:00:39.365Z] + TARFILE=/builds/worker/artifacts/cuda-toolkit.tar
[task 2024-02-13T20:00:39.365Z] + chmod +x /builds/worker/fetches/cuda-source.run
[task 2024-02-13T20:00:39.367Z] ++ pwd
[task 2024-02-13T20:00:39.367Z] + EXTRACT_DIR=/builds/worker/checkouts/cuda-toolkit
[task 2024-02-13T20:00:39.367Z] + /builds/worker/fetches/cuda-source.run --toolkit --toolkitpath=/builds/worker/checkouts/cuda-toolkit --silent --override
[task 2024-02-13T20:02:49.985Z] + tar --zstd -cf /builds/worker/artifacts/cuda-toolkit.tar.zst cuda-toolkit
[fetches 2024-02-13T20:03:32.434Z] removing /builds/worker/fetches
[fetches 2024-02-13T20:03:32.822Z] finished
[taskcluster 2024-02-13 20:03:34.143Z] === Task Finished ===
[taskcluster 2024-02-13 20:04:20.260Z] Successful task run with exit code: 0 completed in 354.886 seconds
Loading