Upgrade to Taskgraph 7 #316
Merged
firefoxci-taskcluster / extract-best-ru-en-2/2
succeeded
Dec 20, 2023 in 37m 3s
FirefoxCI (pull_request)
extract best ru-en 2/2
Details
View task in Taskcluster
View logs in Taskcluster
[taskcluster 2023-12-20 15:40:22.133Z] Task ID: VFUkymJQQV6-g1Ca5dHYrw
[taskcluster 2023-12-20 15:40:22.133Z] Worker ID: 9098964554170153616
[taskcluster 2023-12-20 15:40:22.133Z] Worker Group: us-central1
[taskcluster 2023-12-20 15:40:22.133Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2023-12-20 15:40:22.133Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2023-12-20 15:40:22.133Z] Worker Version: 38.0.5
[taskcluster 2023-12-20 15:40:22.133Z] Public IP: 34.132.154.70
[taskcluster 2023-12-20 15:40:22.133Z] Hostname: translations-1-b-linux-large-gcp-300gb-aykln-kprnqufopxtsrakw
[taskcluster 2023-12-20 15:40:22.133Z] using cache "translations-level-1-checkouts-v3-58974d7dcf0417b3fe53-ARGg7JgtR3C0C2BCSaAWqA" -> /builds/worker/checkouts
[taskcluster 2023-12-20 15:40:22.673Z] Downloading artifact "public/image.tar.zst" from task ID: ARGg7JgtR3C0C2BCSaAWqA.
[taskcluster 2023-12-20 15:40:25.883Z] Downloaded artifact successfully.
[taskcluster 2023-12-20 15:40:25.883Z] Downloaded 251.882 mb
[taskcluster 2023-12-20 15:40:25.883Z] Decompressing downloaded image
[taskcluster 2023-12-20 15:40:27.228Z] Loading docker image from downloaded archive.
[taskcluster 2023-12-20 15:40:34.330Z] Image 'public/image.tar.zst' from task 'ARGg7JgtR3C0C2BCSaAWqA' loaded. Using image ID sha256:fa53620b2ece3428f41eb15ffb863544aa69d70369d1fa2d4412339db9f7115c.
[taskcluster 2023-12-20 15:40:34.455Z] === Task Starting ===
[setup 2023-12-20T15:40:40.893Z] run-task started in /builds/worker
[setup 2023-12-20T15:40:40.893Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu
[setup 2023-12-20T15:40:40.893Z] Python version: 3.10.12
[cache 2023-12-20T15:40:40.895Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2023-12-20T15:40:40.895Z] volume /builds/worker/checkouts is a cache
[setup 2023-12-20T15:40:40.895Z] running as worker:worker
[vcs 2023-12-20T15:40:40.896Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2023-12-20T15:40:40.897Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2023-12-20T15:40:40.898Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2023-12-20T15:40:41.598Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'taskgraph-7']
[vcs 2023-12-20T15:40:41.908Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2023-12-20T15:40:41.908Z] * branch taskgraph-7 -> FETCH_HEAD
[vcs 2023-12-20T15:40:41.911Z] executing ['git', 'checkout', '-f', '-B', 'taskgraph-7', '084cd68b442168a104ef509f6ea7bf320b139e2a']
[vcs 2023-12-20T15:40:41.947Z] Switched to a new branch 'taskgraph-7'
[vcs 2023-12-20T15:40:41.947Z] executing ['git', 'submodule', 'init']
[vcs 2023-12-20T15:40:41.966Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2023-12-20T15:40:41.966Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2023-12-20T15:40:41.967Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2023-12-20T15:40:41.967Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2023-12-20T15:40:41.968Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2023-12-20T15:40:41.968Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2023-12-20T15:40:41.969Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2023-12-20T15:40:41.988Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2023-12-20T15:40:43.154Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2023-12-20T15:40:43.467Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2023-12-20T15:40:43.798Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2023-12-20T15:40:44.444Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2023-12-20T15:40:45.722Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2023-12-20T15:40:46.235Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2023-12-20T15:40:46.244Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2023-12-20T15:40:46.253Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2023-12-20T15:40:46.276Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2023-12-20T15:40:46.373Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2023-12-20T15:40:46.390Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2023-12-20T15:40:46.391Z] cleaning git checkout...
[vcs 2023-12-20T15:40:46.391Z] executing ['git', 'clean', '-nxdff']
[vcs 2023-12-20T15:40:46.393Z] removing []
[vcs 2023-12-20T15:40:46.393Z] successfully cleaned git checkout!
[vcs 2023-12-20T15:40:46.394Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/084cd68b442168a104ef509f6ea7bf320b139e2a' title='Built from firefox-translations-training commit 084cd68b442168a104ef509f6ea7bf320b139e2a'>084cd68b442168a104ef509f6ea7bf320b139e2a</a>
[setup 2023-12-20T15:40:46.394Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2023-12-20T15:40:46.394Z] fetching artifacts
[fetches 2023-12-20T15:40:46.394Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
attempt 1/5Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst to /builds/worker/fetches/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst to /builds/worker/fetches/file.2.nbest.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst resolved to 440499 bytes with sha256 d3ce2cf9a44f2cdcea9660201edb2db691420a24fce2f24c22caaa206f74fdfa in 0.117s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst resolved to 159733 bytes with sha256 d15e5f64e794b8a535cf27e52eb2d31fc61cd7bf75d6b6a35fcaf0ba12853c8f in 0.170s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.17208386600009362, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2023-12-20T15:40:46.642Z] finished fetching artifacts
[task 2023-12-20T15:40:46.642Z] executing ['bash', '-c', 'zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu']
[task 2023-12-20T15:40:46.644Z]
[task 2023-12-20T15:40:46.668Z] Decompress: 1/ 2 files. Current: ...ile.2.nbest.zst : 0 MB...
[task 2023-12-20T15:40:46.669Z]
[task 2023-12-20T15:40:46.669Z]
[task 2023-12-20T15:40:46.669Z]
[task 2023-12-20T15:40:46.669Z] 2 files decompressed : 31512619 bytes total
[fetches 2023-12-20T15:40:53.514Z] removing /builds/worker/fetches
[fetches 2023-12-20T15:40:53.518Z] finished
[taskcluster 2023-12-20 15:40:54.015Z] === Task Finished ===
[taskcluster 2023-12-20 15:40:54.280Z] Successful task run with exit code: 0 completed in 32.147 seconds
Loading