Set TERM for docker worker images #464
Merged
firefoxci-taskcluster / extract-best-ru-en-2/2
succeeded
Feb 28, 2024 in 28m 54s
FirefoxCI (pull_request)
extract best ru-en 2/2
Details
View task in Taskcluster
View logs in Taskcluster
[taskcluster 2024-02-28 02:51:55.481Z] Task ID: VNcBIKDYRTmhjJ5Oyhk8dw
[taskcluster 2024-02-28 02:51:55.481Z] Worker ID: 664258403898696148
[taskcluster 2024-02-28 02:51:55.481Z] Worker Group: us-west1
[taskcluster 2024-02-28 02:51:55.481Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-28 02:51:55.481Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-02-28 02:51:55.481Z] Worker Version: 38.0.5
[taskcluster 2024-02-28 02:51:55.481Z] Public IP: 35.227.148.246
[taskcluster 2024-02-28 02:51:55.481Z] Hostname: translations-1-b-linux-large-gcp-300gb-sug-adt9rluwi4lp0kkaog
[taskcluster 2024-02-28 02:51:55.481Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-e_fnyhlhT6CGxAxd6QIFSA" -> /builds/worker/checkouts
[taskcluster 2024-02-28 02:51:55.936Z] Downloading artifact "public/image.tar.zst" from task ID: e_fnyhlhT6CGxAxd6QIFSA.
[taskcluster 2024-02-28 02:51:57.893Z] Downloaded artifact successfully.
[taskcluster 2024-02-28 02:51:57.893Z] Downloaded 265.144 mb
[taskcluster 2024-02-28 02:51:57.894Z] Decompressing downloaded image
[taskcluster 2024-02-28 02:51:59.308Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-28 02:52:07.508Z] Image 'public/image.tar.zst' from task 'e_fnyhlhT6CGxAxd6QIFSA' loaded. Using image ID sha256:4a564314aaa610230a33ae7b01128154df8057c24ea44b0f87454c1d2ac59a3d.
[taskcluster 2024-02-28 02:52:07.605Z] === Task Starting ===
[setup 2024-02-28T02:52:15.398Z] run-task started in /builds/worker
[setup 2024-02-28T02:52:15.398Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu
[setup 2024-02-28T02:52:15.398Z] Python version: 3.10.12
[cache 2024-02-28T02:52:15.400Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-28T02:52:15.400Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-28T02:52:15.401Z] running as worker:worker
[vcs 2024-02-28T02:52:15.401Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T02:52:15.402Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T02:52:15.404Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-28T02:52:16.114Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'term']
[vcs 2024-02-28T02:52:16.475Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2024-02-28T02:52:16.475Z] * branch term -> FETCH_HEAD
[vcs 2024-02-28T02:52:16.477Z] executing ['git', 'checkout', '-f', '-B', 'term', '38340942e4bdac1ff2749e0647f6fc7f10d602db']
[vcs 2024-02-28T02:52:16.530Z] Switched to a new branch 'term'
[vcs 2024-02-28T02:52:16.531Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-28T02:52:16.549Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-28T02:52:16.550Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-28T02:52:16.550Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-28T02:52:16.551Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-28T02:52:16.551Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-28T02:52:16.552Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-28T02:52:16.552Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-28T02:52:16.571Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-28T02:52:17.787Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-28T02:52:18.175Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-28T02:52:18.546Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-28T02:52:19.166Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-28T02:52:21.297Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-28T02:52:21.785Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-28T02:52:21.794Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-28T02:52:21.803Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-28T02:52:21.825Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-28T02:52:21.923Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-28T02:52:21.939Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-28T02:52:21.940Z] cleaning git checkout...
[vcs 2024-02-28T02:52:21.940Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-28T02:52:21.942Z] removing []
[vcs 2024-02-28T02:52:21.942Z] successfully cleaned git checkout!
[vcs 2024-02-28T02:52:21.943Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/38340942e4bdac1ff2749e0647f6fc7f10d602db' title='Built from firefox-translations-training commit 38340942e4bdac1ff2749e0647f6fc7f10d602db'>38340942e4bdac1ff2749e0647f6fc7f10d602db</a>
[setup 2024-02-28T02:52:21.943Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-02-28T02:52:21.943Z] fetching artifacts
[fetches 2024-02-28T02:52:21.943Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst to /builds/worker/fetches/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst to /builds/worker/fetches/file.2.nbest.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst resolved to 162814 bytes with sha256 bab62317e04ca97c3cf3e593e1b174b58a8fa869b5896151d62ab18c02bd1542 in 0.173s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst resolved to 411420 bytes with sha256 2787e180642f023c1c2354ca9ed99fee4d5827ecb9c3ccca3b590ba888b8d108 in 0.178s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.1810777279999911, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-28T02:52:22.201Z] finished fetching artifacts
[task 2024-02-28T02:52:22.201Z] executing ['bash', '-c', 'zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu']
[task 2024-02-28T02:52:22.204Z]
[task 2024-02-28T02:52:22.216Z] Decompress: 1/ 2 files. Current: ...ile.2.nbest.zst : 0 MB...
[task 2024-02-28T02:52:22.217Z]
[task 2024-02-28T02:52:22.217Z]
[task 2024-02-28T02:52:22.217Z]
[task 2024-02-28T02:52:22.217Z] 2 files decompressed : 14839943 bytes total
[fetches 2024-02-28T02:52:25.497Z] removing /builds/worker/fetches
[fetches 2024-02-28T02:52:25.499Z] finished
[taskcluster 2024-02-28 02:52:25.991Z] === Task Finished ===
[taskcluster 2024-02-28 02:52:26.249Z] Successful task run with exit code: 0 completed in 30.769 seconds
Loading