[taskcluster 2024-05-24 17:43:50.500Z] Task ID: S2YTFNyBSTGMFD4176hUEw
[taskcluster 2024-05-24 17:43:50.500Z] Worker ID: 8473823959597281218
[taskcluster 2024-05-24 17:43:50.500Z] Worker Group: us-central1-a
[taskcluster 2024-05-24 17:43:50.500Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-05-24 17:43:50.500Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2024-05-24 17:43:50.500Z] Worker Version: 38.0.5
[taskcluster 2024-05-24 17:43:50.500Z] Public IP: 34.68.90.76
[taskcluster 2024-05-24 17:43:50.500Z] Hostname: translations-1-b-linux-large-gcp-300gb-caku8zaariqigg7fbvfz7q
[taskcluster 2024-05-24 17:43:50.500Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-Q_deBg9LRdWXU21Qhn7s6Q" -> /builds/worker/checkouts

[taskcluster 2024-05-24 17:43:53.246Z] Image 'public/image.tar.zst' from task 'Q_deBg9LRdWXU21Qhn7s6Q' loaded.  Using image ID sha256:1e62b245ce9b86147f4b5a4cdcf3708c9a763f8cdaa58df95c74f81427d083ab.
[taskcluster 2024-05-24 17:43:53.302Z] === Task Starting ===
[setup 2024-05-24T17:43:53.592Z] run-task started in /builds/worker
[setup 2024-05-24T17:43:53.593Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c $VCS_PATH/pipeline/translate/collect.sh fetches artifacts/corpus.en.zst $MOZ_FETCHES_DIR/corpus.en.zst
[setup 2024-05-24T17:43:53.593Z] Python version: 3.10.12
[cache 2024-05-24T17:43:53.594Z] cache /builds/worker/checkouts exists; requirements: gid=1000 uid=1000 version=1
[volume 2024-05-24T17:43:53.594Z] volume /builds/worker/checkouts is a cache
[setup 2024-05-24T17:43:53.594Z] running as worker:worker
[vcs 2024-05-24T17:43:53.595Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-05-24T17:43:53.596Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/firefox-translations-training', 'revert_d2g_pool']
[vcs 2024-05-24T17:43:53.802Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-05-24T17:43:53.802Z]  * branch            revert_d2g_pool -> FETCH_HEAD
[vcs 2024-05-24T17:43:53.807Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/firefox-translations-training', 'revert_d2g_pool']
[vcs 2024-05-24T17:43:54.028Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-05-24T17:43:54.028Z]  * branch            revert_d2g_pool -> FETCH_HEAD
[vcs 2024-05-24T17:43:54.034Z] executing ['git', 'checkout', '-f', '-B', 'revert_d2g_pool', '9db52842e17a9fa0ececcfc30809128fa1c52288']
[vcs 2024-05-24T17:43:54.038Z] Reset branch 'revert_d2g_pool'
[vcs 2024-05-24T17:43:54.039Z] executing ['git', 'submodule', 'init']
[vcs 2024-05-24T17:43:54.056Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-05-24T17:43:54.138Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-05-24T17:43:54.148Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-05-24T17:43:54.158Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-05-24T17:43:54.182Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-05-24T17:43:54.294Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-05-24T17:43:54.311Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-05-24T17:43:54.311Z] cleaning git checkout...
[vcs 2024-05-24T17:43:54.311Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-05-24T17:43:54.313Z] removing []
[vcs 2024-05-24T17:43:54.313Z] successfully cleaned git checkout!
[vcs 2024-05-24T17:43:54.315Z] TinderboxPrint:<a href='https://github.com/mozilla/firefox-translations-training/commit/9db52842e17a9fa0ececcfc30809128fa1c52288' title='Built from firefox-translations-training commit 9db52842e17a9fa0ececcfc30809128fa1c52288'>9db52842e17a9fa0ececcfc30809128fa1c52288</a>
[setup 2024-05-24T17:43:54.315Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-05-24T17:43:54.315Z] fetching artifacts
[fetches 2024-05-24T17:43:54.315Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OIze0iuHQKyomqR2eGeoiQ/artifacts/public/build/file.1.nbest.out to /builds/worker/fetches/file.1.nbest.out
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cYkdMxNwSn-qJqVvYXfSLA/artifacts/public/build/file.2.nbest.out to /builds/worker/fetches/file.2.nbest.out
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OIze0iuHQKyomqR2eGeoiQ/artifacts/public/build/file.1.nbest.out
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CNkzq1f4TQW7VSM1Md0CAg/artifacts/public/build/corpus.en.zst to /builds/worker/fetches/corpus.en.zstDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cYkdMxNwSn-qJqVvYXfSLA/artifacts/public/build/file.2.nbest.out

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CNkzq1f4TQW7VSM1Md0CAg/artifacts/public/build/corpus.en.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/CNkzq1f4TQW7VSM1Md0CAg/artifacts/public/build/corpus.en.zst resolved to 264226 bytes with sha256 ecfc3718f9cf0dcb3f5e38f1aa221ce730b491884deecf52c0f3a105c396efc0 in 0.145s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/OIze0iuHQKyomqR2eGeoiQ/artifacts/public/build/file.1.nbest.out resolved to 3963211 bytes with sha256 40477367d57ab653dd37116b9e7a43ab4781d80229c243e63427c6005cac530a in 0.232s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cYkdMxNwSn-qJqVvYXfSLA/artifacts/public/build/file.2.nbest.out resolved to 4743683 bytes with sha256 5d1cbec918a96427e5984524ae648cdb6824eebb498bab27263a7e3d9bf1972f in 0.248s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.25080823199999713, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-05-24T17:43:54.641Z] finished fetching artifacts
[task 2024-05-24T17:43:54.641Z] executing ['bash', '-c', '$VCS_PATH/pipeline/translate/collect.sh fetches artifacts/corpus.en.zst $MOZ_FETCHES_DIR/corpus.en.zst']
[task 2024-05-24T17:43:54.644Z] + set -euo pipefail
[task 2024-05-24T17:43:54.644Z] + chunks_dir=fetches
[task 2024-05-24T17:43:54.644Z] + output_path=artifacts/corpus.en.zst
[task 2024-05-24T17:43:54.644Z] + mono_path=/builds/worker/fetches/corpus.en.zst
[task 2024-05-24T17:43:54.644Z] + COMPRESSION_CMD=zstdmt
[task 2024-05-24T17:43:54.644Z] + echo '### Collecting translations'
[task 2024-05-24T17:43:54.644Z] ### Collecting translations
[task 2024-05-24T17:43:54.644Z] + find fetches -name '*.out'
[task 2024-05-24T17:43:54.644Z] + sort -t . -k2,2n
[task 2024-05-24T17:43:54.644Z] + xargs cat
[task 2024-05-24T17:43:54.644Z] + zstdmt
[task 2024-05-24T17:43:54.657Z] + echo '### Comparing number of sentences in source and artificial target files'
[task 2024-05-24T17:43:54.657Z] ### Comparing number of sentences in source and artificial target files
[task 2024-05-24T17:43:54.658Z] ++ zstdmt -dc /builds/worker/fetches/corpus.en.zst
[task 2024-05-24T17:43:54.658Z] ++ wc -l
[task 2024-05-24T17:43:54.661Z] + src_len=7028
[task 2024-05-24T17:43:54.661Z] ++ zstdmt -dc artifacts/corpus.en.zst
[task 2024-05-24T17:43:54.661Z] ++ wc -l
[task 2024-05-24T17:43:54.668Z] + trg_len=7028
[task 2024-05-24T17:43:54.668Z] + '[' 7028 '!=' 7028 ']'
[fetches 2024-05-24T17:43:54.669Z] removing /builds/worker/fetches
[fetches 2024-05-24T17:43:54.670Z] finished
[taskcluster 2024-05-24 17:43:54.891Z] === Task Finished ===
[taskcluster 2024-05-24 17:43:55.107Z] Successful task run with exit code: 0 completed in 4.608 seconds