Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set TERM for docker worker images #464

Merged
merged 2 commits into from
Feb 28, 2024

Merge branch 'main' into term

3834094
Select commit
Loading
Failed to load commit list.
Merged

Set TERM for docker worker images #464

Merge branch 'main' into term
3834094
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / extract-best-ru-en-2/2 succeeded Feb 28, 2024 in 28m 54s

FirefoxCI (pull_request)

extract best ru-en 2/2

Details

View task in Taskcluster
View logs in Taskcluster


[taskcluster 2024-02-28 02:51:55.481Z] Task ID: VNcBIKDYRTmhjJ5Oyhk8dw
[taskcluster 2024-02-28 02:51:55.481Z] Worker ID: 664258403898696148
[taskcluster 2024-02-28 02:51:55.481Z] Worker Group: us-west1
[taskcluster 2024-02-28 02:51:55.481Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-28 02:51:55.481Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-02-28 02:51:55.481Z] Worker Version: 38.0.5
[taskcluster 2024-02-28 02:51:55.481Z] Public IP: 35.227.148.246
[taskcluster 2024-02-28 02:51:55.481Z] Hostname: translations-1-b-linux-large-gcp-300gb-sug-adt9rluwi4lp0kkaog
[taskcluster 2024-02-28 02:51:55.481Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-e_fnyhlhT6CGxAxd6QIFSA" -> /builds/worker/checkouts

[taskcluster 2024-02-28 02:51:55.936Z] Downloading artifact "public/image.tar.zst" from task ID: e_fnyhlhT6CGxAxd6QIFSA.
[taskcluster 2024-02-28 02:51:57.893Z] Downloaded artifact successfully.
[taskcluster 2024-02-28 02:51:57.893Z] Downloaded 265.144 mb
[taskcluster 2024-02-28 02:51:57.894Z] Decompressing downloaded image
[taskcluster 2024-02-28 02:51:59.308Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-28 02:52:07.508Z] Image 'public/image.tar.zst' from task 'e_fnyhlhT6CGxAxd6QIFSA' loaded.  Using image ID sha256:4a564314aaa610230a33ae7b01128154df8057c24ea44b0f87454c1d2ac59a3d.
[taskcluster 2024-02-28 02:52:07.605Z] === Task Starting ===
[setup 2024-02-28T02:52:15.398Z] run-task started in /builds/worker
[setup 2024-02-28T02:52:15.398Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu
[setup 2024-02-28T02:52:15.398Z] Python version: 3.10.12
[cache 2024-02-28T02:52:15.400Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-28T02:52:15.400Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-28T02:52:15.401Z] running as worker:worker
[vcs 2024-02-28T02:52:15.401Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T02:52:15.402Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T02:52:15.404Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-28T02:52:16.114Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'term']
[vcs 2024-02-28T02:52:16.475Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2024-02-28T02:52:16.475Z]  * branch            term       -> FETCH_HEAD
[vcs 2024-02-28T02:52:16.477Z] executing ['git', 'checkout', '-f', '-B', 'term', '38340942e4bdac1ff2749e0647f6fc7f10d602db']
[vcs 2024-02-28T02:52:16.530Z] Switched to a new branch 'term'
[vcs 2024-02-28T02:52:16.531Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-28T02:52:16.549Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-28T02:52:16.550Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-28T02:52:16.550Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-28T02:52:16.551Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-28T02:52:16.551Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-28T02:52:16.552Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-28T02:52:16.552Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-28T02:52:16.571Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-28T02:52:17.787Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-28T02:52:18.175Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-28T02:52:18.546Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-28T02:52:19.166Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-28T02:52:21.297Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-28T02:52:21.785Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-28T02:52:21.794Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-28T02:52:21.803Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-28T02:52:21.825Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-28T02:52:21.923Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-28T02:52:21.939Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-28T02:52:21.940Z] cleaning git checkout...
[vcs 2024-02-28T02:52:21.940Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-28T02:52:21.942Z] removing []
[vcs 2024-02-28T02:52:21.942Z] successfully cleaned git checkout!
[vcs 2024-02-28T02:52:21.943Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/38340942e4bdac1ff2749e0647f6fc7f10d602db' title='Built from firefox-translations-training commit 38340942e4bdac1ff2749e0647f6fc7f10d602db'>38340942e4bdac1ff2749e0647f6fc7f10d602db</a>
[setup 2024-02-28T02:52:21.943Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-02-28T02:52:21.943Z] fetching artifacts
[fetches 2024-02-28T02:52:21.943Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst to /builds/worker/fetches/file.2.ref.zst

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst to /builds/worker/fetches/file.2.nbest.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/WytyhWd5Rv-TaiBeNA5rxA/artifacts/public/build/file.2.ref.zst resolved to 162814 bytes with sha256 bab62317e04ca97c3cf3e593e1b174b58a8fa869b5896151d62ab18c02bd1542 in 0.173s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CQk_vbFOS6mcnGEnKb5XVQ/artifacts/public/build/file.2.nbest.zst resolved to 411420 bytes with sha256 2787e180642f023c1c2354ca9ed99fee4d5827ecb9c3ccca3b590ba888b8d108 in 0.178s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.1810777279999911, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-28T02:52:22.201Z] finished fetching artifacts
[task 2024-02-28T02:52:22.201Z] executing ['bash', '-c', 'zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu']
[task 2024-02-28T02:52:22.204Z] 
[task 2024-02-28T02:52:22.216Z] Decompress:  1/ 2 files. Current: ...ile.2.nbest.zst : 0 MB...    
[task 2024-02-28T02:52:22.217Z]                                                                                
[task 2024-02-28T02:52:22.217Z] 
[task 2024-02-28T02:52:22.217Z]                                                                                
[task 2024-02-28T02:52:22.217Z] 2 files decompressed : 14839943 bytes total 
[fetches 2024-02-28T02:52:25.497Z] removing /builds/worker/fetches
[fetches 2024-02-28T02:52:25.499Z] finished
[taskcluster 2024-02-28 02:52:25.991Z] === Task Finished ===
[taskcluster 2024-02-28 02:52:26.249Z] Successful task run with exit code: 0 completed in 30.769 seconds