[taskcluster 2023-12-15 19:03:25.937Z] Task ID: VVnCkPmzQrOV-QlZ5xvQTg
[taskcluster 2023-12-15 19:03:25.937Z] Worker ID: 7367342897027423326
[taskcluster 2023-12-15 19:03:25.937Z] Worker Group: us-central1
[taskcluster 2023-12-15 19:03:25.937Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2023-12-15 19:03:25.937Z] Worker Pool: translations-1/b-linux-large-gcp-300gb
[taskcluster 2023-12-15 19:03:25.937Z] Worker Version: 38.0.5
[taskcluster 2023-12-15 19:03:25.937Z] Public IP: 35.223.7.63
[taskcluster 2023-12-15 19:03:25.937Z] Hostname: translations-1-b-linux-large-gcp-300gb-poba4ma0qustggehpkkqsa
[taskcluster 2023-12-15 19:03:25.937Z] using cache "translations-level-1-checkouts-hg58-v3-2922149087d95338df26" -> /builds/worker/checkouts

[taskcluster 2023-12-15 19:03:30.135Z] Downloading artifact "public/image.tar.zst" from task ID: coL2A8X7TtKPvOvddcYtiw.
[taskcluster 2023-12-15 19:03:32.152Z] Downloaded artifact successfully.
[taskcluster 2023-12-15 19:03:32.152Z] Downloaded 221.499 mb
[taskcluster 2023-12-15 19:03:32.153Z] Decompressing downloaded image
[taskcluster 2023-12-15 19:03:33.530Z] Loading docker image from downloaded archive.
[taskcluster 2023-12-15 19:03:42.128Z] Image 'public/image.tar.zst' from task 'coL2A8X7TtKPvOvddcYtiw' loaded.  Using image ID sha256:a6912c5313466b965d7bb74ac22f26d3fe1e8789f9d2bbd3c516122194ef152e.
[taskcluster 2023-12-15 19:03:42.261Z] === Task Starting ===
[setup 2023-12-15T19:03:49.371Z] run-task started in /builds/worker
[setup 2023-12-15T19:03:49.371Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c export BIN=$MOZ_FETCHES_DIR && $VCS_PATH/taskcluster/scripts/pipeline/split-mono-taskcluster.sh 2 /builds/worker/artifacts 5000 $MOZ_FETCHES_DIR/mono.$LOCALE.zst
[setup 2023-12-15T19:03:49.371Z] Python version: 3.10.12
[cache 2023-12-15T19:03:49.373Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2023-12-15T19:03:49.373Z] volume /builds/worker/checkouts is a cache
[setup 2023-12-15T19:03:49.373Z] running as worker:worker
[vcs 2023-12-15T19:03:49.373Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2023-12-15T19:03:49.374Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2023-12-15T19:03:50.141Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'followup']
[vcs 2023-12-15T19:03:50.432Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2023-12-15T19:03:50.432Z]  * branch            followup   -> FETCH_HEAD
[vcs 2023-12-15T19:03:50.434Z] executing ['git', 'checkout', '-f', '-B', 'followup', '4aa774c2a9f31b93627bcac82e0c235839cb7c0e']
[vcs 2023-12-15T19:03:50.469Z] Switched to a new branch 'followup'
[vcs 2023-12-15T19:03:50.469Z] executing ['git', 'submodule', 'init']
[vcs 2023-12-15T19:03:50.487Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2023-12-15T19:03:50.488Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2023-12-15T19:03:50.488Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2023-12-15T19:03:50.489Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2023-12-15T19:03:50.489Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2023-12-15T19:03:50.490Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2023-12-15T19:03:50.490Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2023-12-15T19:03:50.508Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2023-12-15T19:03:51.645Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2023-12-15T19:03:51.957Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2023-12-15T19:03:52.332Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2023-12-15T19:03:53.572Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2023-12-15T19:03:54.841Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2023-12-15T19:03:55.363Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2023-12-15T19:03:55.372Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2023-12-15T19:03:55.381Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2023-12-15T19:03:55.404Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2023-12-15T19:03:55.506Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2023-12-15T19:03:55.523Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2023-12-15T19:03:55.523Z] cleaning git checkout...
[vcs 2023-12-15T19:03:55.523Z] executing ['git', 'clean', '-nxdff']
[vcs 2023-12-15T19:03:55.525Z] removing []
[vcs 2023-12-15T19:03:55.525Z] successfully cleaned git checkout!
[vcs 2023-12-15T19:03:55.527Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/4aa774c2a9f31b93627bcac82e0c235839cb7c0e' title='Built from firefox-translations-training commit 4aa774c2a9f31b93627bcac82e0c235839cb7c0e'>4aa774c2a9f31b93627bcac82e0c235839cb7c0e</a>
[setup 2023-12-15T19:03:55.527Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2023-12-15T19:03:55.527Z] fetching artifacts
[fetches 2023-12-15T19:03:55.527Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KAFn2cQEQgSu_gb9x2LXpg/artifacts/public/build/mono.en.zst to /builds/worker/fetches/mono.en.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AgPlxUayScSDmqgddgY9pw/artifacts/public/build/dedupe.tar.zst to /builds/worker/fetches/dedupe.tar.zstDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KAFn2cQEQgSu_gb9x2LXpg/artifacts/public/build/mono.en.zst

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AgPlxUayScSDmqgddgY9pw/artifacts/public/build/dedupe.tar.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AgPlxUayScSDmqgddgY9pw/artifacts/public/build/dedupe.tar.zst resolved to 133246 bytes with sha256 1e567b7eabfbbac81c949ba165181bf3b4b5a15d4fcd114a0fc1d7ec82cfac07 in 0.200s
Extracting /builds/worker/fetches/dedupe.tar.zst to /builds/worker/fetches
/builds/worker/fetches/dedupe.tar.zst extracted in 0.003s
Removing /builds/worker/fetches/dedupe.tar.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/KAFn2cQEQgSu_gb9x2LXpg/artifacts/public/build/mono.en.zst resolved to 510192 bytes with sha256 bd03cb3d9b71ce25a8aaf9bd47895b3bdc311c02373292cff778f8132944e5e7 in 0.252s
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.2543548069999986, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2023-12-15T19:03:55.859Z] finished fetching artifacts
[task 2023-12-15T19:03:55.860Z] executing ['bash', '-c', 'export BIN=$MOZ_FETCHES_DIR && $VCS_PATH/taskcluster/scripts/pipeline/split-mono-taskcluster.sh 2 /builds/worker/artifacts 5000 $MOZ_FETCHES_DIR/mono.$LOCALE.zst']
[task 2023-12-15T19:03:55.862Z] + set -euo pipefail
[task 2023-12-15T19:03:55.862Z] + chunks=2
[task 2023-12-15T19:03:55.862Z] + output_dir=/builds/worker/artifacts
[task 2023-12-15T19:03:55.862Z] + length=5000
[task 2023-12-15T19:03:55.862Z] + lang_args=("${@:4}")
[task 2023-12-15T19:03:55.862Z] ++ dirname /builds/worker/checkouts/vcs/taskcluster/scripts/pipeline/split-mono-taskcluster.sh
[task 2023-12-15T19:03:55.863Z] + pushd /builds/worker/checkouts/vcs/taskcluster/scripts/pipeline/../../..
[task 2023-12-15T19:03:55.863Z] ++ pwd
[task 2023-12-15T19:03:55.863Z] + VCS_ROOT=/builds/worker/checkouts/vcs
[task 2023-12-15T19:03:55.863Z] + popd
[task 2023-12-15T19:03:55.863Z] + /builds/worker/checkouts/vcs/pipeline/translate/split-mono.sh /builds/worker/fetches/mono.en.zst /builds/worker/artifacts 5000
[task 2023-12-15T19:03:55.864Z] + set -euo pipefail
[task 2023-12-15T19:03:55.864Z] + test -v BIN
[task 2023-12-15T19:03:55.864Z] + mono_path=/builds/worker/fetches/mono.en.zst
[task 2023-12-15T19:03:55.864Z] + output_dir=/builds/worker/artifacts
[task 2023-12-15T19:03:55.864Z] + length=5000
[task 2023-12-15T19:03:55.864Z] + COMPRESSION_CMD=zstdmt
[task 2023-12-15T19:03:55.865Z] + ARTIFACT_EXT=zst
[task 2023-12-15T19:03:55.865Z] + mkdir -p /builds/worker/artifacts
[task 2023-12-15T19:03:55.866Z] + zstdmt -dc /builds/worker/fetches/mono.en.zst
[task 2023-12-15T19:03:55.866Z] + /builds/worker/fetches/dedupe
[task 2023-12-15T19:03:55.866Z] + split -d -l 5000 - /builds/worker/artifacts/file.
[task 2023-12-15T19:03:55.867Z] File stdin isn't normal.  Using slower read() instead of mmap().  No progress bar.
[task 2023-12-15T19:03:55.870Z] Kept 9932 / 9932 = 1
[task 2023-12-15T19:03:55.870Z] + cd /builds/worker/artifacts
[task 2023-12-15T19:03:55.871Z] + ls file.00 file.01
[task 2023-12-15T19:03:55.871Z] + sort
[task 2023-12-15T19:03:55.872Z] ++ seq 1 2
[task 2023-12-15T19:03:55.872Z] ++ tr '\n' ' '
[task 2023-12-15T19:03:55.873Z] + for i in $(seq 1 ${chunks} | tr '\n' ' ')
[task 2023-12-15T19:03:55.873Z] ++ split -n l/1/2 out-files.txt
[task 2023-12-15T19:03:55.873Z] ++ tr '\n' ' '
[task 2023-12-15T19:03:55.874Z] + out_files='file.00 '
[task 2023-12-15T19:03:55.874Z] + '[' 'file.00 ' = '' ']'
[task 2023-12-15T19:03:55.874Z] + cat file.00
[task 2023-12-15T19:03:55.875Z] + zstd --rm out-file.1
[task 2023-12-15T19:03:55.878Z] 
[task 2023-12-15T19:03:55.878Z]                                                                                
[task 2023-12-15T19:03:55.883Z] Read :  0 /  0 MB ==> 45%
[task 2023-12-15T19:03:55.883Z]                                                                                
[task 2023-12-15T19:03:55.883Z] out-file.1           : 42.02%   (616048 => 258849 bytes, out-file.1.zst) 
[task 2023-12-15T19:03:55.883Z] + for i in $(seq 1 ${chunks} | tr '\n' ' ')
[task 2023-12-15T19:03:55.884Z] ++ split -n l/2/2 out-files.txt
[task 2023-12-15T19:03:55.884Z] ++ tr '\n' ' '
[task 2023-12-15T19:03:55.885Z] + out_files='file.01 '
[task 2023-12-15T19:03:55.885Z] + '[' 'file.01 ' = '' ']'
[task 2023-12-15T19:03:55.885Z] + cat file.01
[task 2023-12-15T19:03:55.886Z] + zstd --rm out-file.2
[task 2023-12-15T19:03:55.889Z] 
[task 2023-12-15T19:03:55.889Z]                                                                                
[task 2023-12-15T19:03:55.894Z] Read :  0 /  0 MB ==> 45%
[task 2023-12-15T19:03:55.894Z]                                                                                
[task 2023-12-15T19:03:55.894Z] out-file.2           : 41.89%   (618858 => 259210 bytes, out-file.2.zst) 
[task 2023-12-15T19:03:55.894Z] + rm file.00 file.01 out-files.txt
[fetches 2023-12-15T19:03:55.895Z] removing /builds/worker/fetches
[fetches 2023-12-15T19:03:55.896Z] finished
[taskcluster 2023-12-15 19:03:56.377Z] === Task Finished ===
[taskcluster 2023-12-15 19:04:02.266Z] Successful task run with exit code: 0 completed in 36.33 seconds