Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to Taskgraph 7 #316

Merged
merged 2 commits into from
Dec 21, 2023

Taskgraph 7.0 changes

084cd68
Select commit
Loading
Failed to load commit list.
Merged

Upgrade to Taskgraph 7 #316

Taskgraph 7.0 changes
084cd68
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / extract-best-ru-en-2/2 succeeded Dec 20, 2023 in 37m 3s

FirefoxCI (pull_request)

extract best ru-en 2/2

Details

View task in Taskcluster
View logs in Taskcluster


[taskcluster 2023-12-20 15:40:22.133Z] Task ID: VFUkymJQQV6-g1Ca5dHYrw
[taskcluster 2023-12-20 15:40:22.133Z] Worker ID: 9098964554170153616
[taskcluster 2023-12-20 15:40:22.133Z] Worker Group: us-central1
[taskcluster 2023-12-20 15:40:22.133Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2023-12-20 15:40:22.133Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2023-12-20 15:40:22.133Z] Worker Version: 38.0.5
[taskcluster 2023-12-20 15:40:22.133Z] Public IP: 34.132.154.70
[taskcluster 2023-12-20 15:40:22.133Z] Hostname: translations-1-b-linux-large-gcp-300gb-aykln-kprnqufopxtsrakw
[taskcluster 2023-12-20 15:40:22.133Z] using cache "translations-level-1-checkouts-v3-58974d7dcf0417b3fe53-ARGg7JgtR3C0C2BCSaAWqA" -> /builds/worker/checkouts

[taskcluster 2023-12-20 15:40:22.673Z] Downloading artifact "public/image.tar.zst" from task ID: ARGg7JgtR3C0C2BCSaAWqA.
[taskcluster 2023-12-20 15:40:25.883Z] Downloaded artifact successfully.
[taskcluster 2023-12-20 15:40:25.883Z] Downloaded 251.882 mb
[taskcluster 2023-12-20 15:40:25.883Z] Decompressing downloaded image
[taskcluster 2023-12-20 15:40:27.228Z] Loading docker image from downloaded archive.
[taskcluster 2023-12-20 15:40:34.330Z] Image 'public/image.tar.zst' from task 'ARGg7JgtR3C0C2BCSaAWqA' loaded.  Using image ID sha256:fa53620b2ece3428f41eb15ffb863544aa69d70369d1fa2d4412339db9f7115c.
[taskcluster 2023-12-20 15:40:34.455Z] === Task Starting ===
[setup 2023-12-20T15:40:40.893Z] run-task started in /builds/worker
[setup 2023-12-20T15:40:40.893Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu
[setup 2023-12-20T15:40:40.893Z] Python version: 3.10.12
[cache 2023-12-20T15:40:40.895Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2023-12-20T15:40:40.895Z] volume /builds/worker/checkouts is a cache
[setup 2023-12-20T15:40:40.895Z] running as worker:worker
[vcs 2023-12-20T15:40:40.896Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2023-12-20T15:40:40.897Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2023-12-20T15:40:40.898Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2023-12-20T15:40:41.598Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'taskgraph-7']
[vcs 2023-12-20T15:40:41.908Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2023-12-20T15:40:41.908Z]  * branch            taskgraph-7 -> FETCH_HEAD
[vcs 2023-12-20T15:40:41.911Z] executing ['git', 'checkout', '-f', '-B', 'taskgraph-7', '084cd68b442168a104ef509f6ea7bf320b139e2a']
[vcs 2023-12-20T15:40:41.947Z] Switched to a new branch 'taskgraph-7'
[vcs 2023-12-20T15:40:41.947Z] executing ['git', 'submodule', 'init']
[vcs 2023-12-20T15:40:41.966Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2023-12-20T15:40:41.966Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2023-12-20T15:40:41.967Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2023-12-20T15:40:41.967Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2023-12-20T15:40:41.968Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2023-12-20T15:40:41.968Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2023-12-20T15:40:41.969Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2023-12-20T15:40:41.988Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2023-12-20T15:40:43.154Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2023-12-20T15:40:43.467Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2023-12-20T15:40:43.798Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2023-12-20T15:40:44.444Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2023-12-20T15:40:45.722Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2023-12-20T15:40:46.235Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2023-12-20T15:40:46.244Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2023-12-20T15:40:46.253Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2023-12-20T15:40:46.276Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2023-12-20T15:40:46.373Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2023-12-20T15:40:46.390Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2023-12-20T15:40:46.391Z] cleaning git checkout...
[vcs 2023-12-20T15:40:46.391Z] executing ['git', 'clean', '-nxdff']
[vcs 2023-12-20T15:40:46.393Z] removing []
[vcs 2023-12-20T15:40:46.393Z] successfully cleaned git checkout!
[vcs 2023-12-20T15:40:46.394Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/084cd68b442168a104ef509f6ea7bf320b139e2a' title='Built from firefox-translations-training commit 084cd68b442168a104ef509f6ea7bf320b139e2a'>084cd68b442168a104ef509f6ea7bf320b139e2a</a>
[setup 2023-12-20T15:40:46.394Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2023-12-20T15:40:46.394Z] fetching artifacts
[fetches 2023-12-20T15:40:46.394Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
attempt 1/5Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst to /builds/worker/fetches/file.2.ref.zst

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst to /builds/worker/fetches/file.2.nbest.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LnF6HQIGRzW9ZLAQLBAiVA/artifacts/public/build/file.2.nbest.zst resolved to 440499 bytes with sha256 d3ce2cf9a44f2cdcea9660201edb2db691420a24fce2f24c22caaa206f74fdfa in 0.117s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Fi_b4lq3RHipuCEqMPDP4w/artifacts/public/build/file.2.ref.zst resolved to 159733 bytes with sha256 d15e5f64e794b8a535cf27e52eb2d31fc61cd7bf75d6b6a35fcaf0ba12853c8f in 0.170s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.17208386600009362, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2023-12-20T15:40:46.642Z] finished fetching artifacts
[task 2023-12-20T15:40:46.642Z] executing ['bash', '-c', 'zstd -d --rm $MOZ_FETCHES_DIR/*.zst && python3 $VCS_PATH/pipeline/translate/bestbleu.py -i "$MOZ_FETCHES_DIR/file.2.nbest" -r "$MOZ_FETCHES_DIR/file.2.ref" -o /builds/worker/artifacts/file.2.nbest.out -m bleu']
[task 2023-12-20T15:40:46.644Z] 
[task 2023-12-20T15:40:46.668Z] Decompress:  1/ 2 files. Current: ...ile.2.nbest.zst : 0 MB...    
[task 2023-12-20T15:40:46.669Z]                                                                                
[task 2023-12-20T15:40:46.669Z] 
[task 2023-12-20T15:40:46.669Z]                                                                                
[task 2023-12-20T15:40:46.669Z] 2 files decompressed : 31512619 bytes total 
[fetches 2023-12-20T15:40:53.514Z] removing /builds/worker/fetches
[fetches 2023-12-20T15:40:53.518Z] finished
[taskcluster 2023-12-20 15:40:54.015Z] === Task Finished ===
[taskcluster 2023-12-20 15:40:54.280Z] Successful task run with exit code: 0 completed in 32.147 seconds