[taskcluster 2024-02-28 03:29:39.068Z] Task ID: BYSbr6ABQpW9v1IQ--WodA
[taskcluster 2024-02-28 03:29:39.068Z] Worker ID: 1278820223401793231
[taskcluster 2024-02-28 03:29:39.068Z] Worker Group: us-central1
[taskcluster 2024-02-28 03:29:39.068Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-28 03:29:39.068Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-02-28 03:29:39.068Z] Worker Version: 38.0.5
[taskcluster 2024-02-28 03:29:39.068Z] Public IP: 34.172.166.48
[taskcluster 2024-02-28 03:29:39.068Z] Hostname: translations-1-b-linux-large-gcp-gqpi5wyrq9ubg-galrod5g
[taskcluster 2024-02-28 03:29:39.068Z] using cache "translations-level-1-checkouts-v3-7afeb851dd97df8f3607-e_fnyhlhT6CGxAxd6QIFSA" -> /builds/worker/checkouts

[taskcluster 2024-02-28 03:29:39.563Z] Downloading artifact "public/image.tar.zst" from task ID: e_fnyhlhT6CGxAxd6QIFSA.
[taskcluster 2024-02-28 03:29:41.613Z] Downloaded artifact successfully.
[taskcluster 2024-02-28 03:29:41.613Z] Downloaded 265.144 mb
[taskcluster 2024-02-28 03:29:41.613Z] Decompressing downloaded image
[taskcluster 2024-02-28 03:29:42.989Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-28 03:29:51.414Z] Image 'public/image.tar.zst' from task 'e_fnyhlhT6CGxAxd6QIFSA' loaded.  Using image ID sha256:4a564314aaa610230a33ae7b01128154df8057c24ea44b0f87454c1d2ac59a3d.
[taskcluster 2024-02-28 03:29:51.538Z] === Task Starting ===
[setup 2024-02-28T03:29:53.806Z] run-task started in /builds/worker
[setup 2024-02-28T03:29:53.806Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c export BMT_MARIAN=$MOZ_FETCHES_DIR && export BIN=$MOZ_FETCHES_DIR && export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$MOZ_FETCHES_DIR/cuda-toolkit/lib64" && zstd --rm -d $MOZ_FETCHES_DIR/*.zst && pip install -r $VCS_PATH/pipeline/quantize/requirements/quantize.txt && mkdir -p scripts/alphas && mv $MOZ_FETCHES_DIR/extract_stats.py scripts/alphas && $VCS_PATH/pipeline/quantize/quantize.sh $MOZ_FETCHES_DIR/final.model.npz.best-chrf.npz $MOZ_FETCHES_DIR/vocab.spm $MOZ_FETCHES_DIR/lex.s2t.pruned $MOZ_FETCHES_DIR/devset.ru /builds/worker/artifacts
[setup 2024-02-28T03:29:53.806Z] Python version: 3.10.12
[cache 2024-02-28T03:29:53.808Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-28T03:29:53.808Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-28T03:29:53.808Z] running as worker:worker
[vcs 2024-02-28T03:29:53.809Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T03:29:53.810Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-28T03:29:53.812Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-28T03:29:54.487Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'term']
[vcs 2024-02-28T03:29:54.751Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2024-02-28T03:29:54.751Z]  * branch            term       -> FETCH_HEAD
[vcs 2024-02-28T03:29:54.753Z] executing ['git', 'checkout', '-f', '-B', 'term', '38340942e4bdac1ff2749e0647f6fc7f10d602db']
[vcs 2024-02-28T03:29:54.807Z] Switched to a new branch 'term'
[vcs 2024-02-28T03:29:54.807Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-28T03:29:54.826Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-28T03:29:54.826Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-28T03:29:54.826Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-28T03:29:54.826Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-28T03:29:54.827Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-28T03:29:54.827Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-28T03:29:54.827Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-28T03:29:54.847Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-28T03:29:55.942Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-28T03:29:56.240Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-28T03:29:56.552Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-28T03:29:57.204Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-28T03:29:58.536Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-28T03:29:59.043Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-28T03:29:59.052Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-28T03:29:59.061Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-28T03:29:59.085Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-28T03:29:59.185Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-28T03:29:59.202Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-28T03:29:59.203Z] cleaning git checkout...
[vcs 2024-02-28T03:29:59.203Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-28T03:29:59.205Z] removing []
[vcs 2024-02-28T03:29:59.205Z] successfully cleaned git checkout!
[vcs 2024-02-28T03:29:59.207Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/38340942e4bdac1ff2749e0647f6fc7f10d602db' title='Built from firefox-translations-training commit 38340942e4bdac1ff2749e0647f6fc7f10d602db'>38340942e4bdac1ff2749e0647f6fc7f10d602db</a>
[setup 2024-02-28T03:29:59.207Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[fetches 2024-02-28T03:29:59.207Z] fetching artifacts
[fetches 2024-02-28T03:29:59.207Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/TeG3mvo2SKGDLG0a3Jk3Tw/artifacts/public/build/lex.s2t.pruned.zst to /builds/worker/fetches/lex.s2t.pruned.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EPrawNzOQuebx6oMcYdhBg/artifacts/public/build/final.model.npz.best-chrf.npz to /builds/worker/fetches/final.model.npz.best-chrf.npzDownloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/TeG3mvo2SKGDLG0a3Jk3Tw/artifacts/public/build/lex.s2t.pruned.zst

attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eDv8skAsQjK65TLNhrebgg/artifacts/public/build/devset.ru.zst to /builds/worker/fetches/devset.ru.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NfpnGL4vSwuFi03NcIL6OA/artifacts/public/build/marian.tar.zst to /builds/worker/fetches/marian.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NfpnGL4vSwuFi03NcIL6OA/artifacts/public/build/marian.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eDv8skAsQjK65TLNhrebgg/artifacts/public/build/devset.ru.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EPrawNzOQuebx6oMcYdhBg/artifacts/public/build/final.model.npz.best-chrf.npz
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XKFBSMoJRdO-Evx0QApuSQ/artifacts/public/build/cuda-toolkit.tar.zst to /builds/worker/fetches/cuda-toolkit.tar.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Sdhuw2JhT1CUXzwMxYnmHQ/artifacts/public/build/vocab.spm to /builds/worker/fetches/vocab.spm
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XKFBSMoJRdO-Evx0QApuSQ/artifacts/public/build/cuda-toolkit.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Sdhuw2JhT1CUXzwMxYnmHQ/artifacts/public/build/vocab.spm
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/TeG3mvo2SKGDLG0a3Jk3Tw/artifacts/public/build/lex.s2t.pruned.zst resolved to 7181 bytes with sha256 9a2120a7695ac1b5ee2bc0b04c6b10043f2e57462831587be9043b6b4a660879 in 0.127s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Sdhuw2JhT1CUXzwMxYnmHQ/artifacts/public/build/vocab.spm resolved to 252750 bytes with sha256 cb63c2c60d98d0a71d760497b22a2e877fd67667c7fbc722743ad8cc0a6c5fa5 in 0.151s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/eDv8skAsQjK65TLNhrebgg/artifacts/public/build/devset.ru.zst resolved to 201773 bytes with sha256 ef00e00684936ff256ec66377157e92dccec58b0e28be045dfc82b1901739a19 in 0.226s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/EPrawNzOQuebx6oMcYdhBg/artifacts/public/build/final.model.npz.best-chrf.npz resolved to 35790072 bytes with sha256 2929287187e742434f7b1c34e793bcf0dc9eb432ac4d2c76e904a1ab71791b4c in 0.714s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/NfpnGL4vSwuFi03NcIL6OA/artifacts/public/build/marian.tar.zst resolved to 166234971 bytes with sha256 d415f5051ea22f5b6fad59198fcf94456eac5c7f3b58a04e6bbc03c845d7a2aa in 4.961s
Extracting /builds/worker/fetches/marian.tar.zst to /builds/worker/fetches
/builds/worker/fetches/marian.tar.zst extracted in 1.268s
Removing /builds/worker/fetches/marian.tar.zst
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/XKFBSMoJRdO-Evx0QApuSQ/artifacts/public/build/cuda-toolkit.tar.zst resolved to 3296632591 bytes with sha256 8235ffabba1ea2012dc266b0123cbdbd2ecb03512190c8f36ee3355407fcf464 in 49.453s
Extracting /builds/worker/fetches/cuda-toolkit.tar.zst to /builds/worker/fetches
/builds/worker/fetches/cuda-toolkit.tar.zst extracted in 14.458s
Removing /builds/worker/fetches/cuda-toolkit.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 64.21199836100001, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-28T03:31:03.501Z] finished fetching artifacts
[task 2024-02-28T03:31:03.501Z] executing ['bash', '-c', 'export BMT_MARIAN=$MOZ_FETCHES_DIR && export BIN=$MOZ_FETCHES_DIR && export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:$MOZ_FETCHES_DIR/cuda-toolkit/lib64" && zstd --rm -d $MOZ_FETCHES_DIR/*.zst && pip install -r $VCS_PATH/pipeline/quantize/requirements/quantize.txt && mkdir -p scripts/alphas && mv $MOZ_FETCHES_DIR/extract_stats.py scripts/alphas && $VCS_PATH/pipeline/quantize/quantize.sh $MOZ_FETCHES_DIR/final.model.npz.best-chrf.npz $MOZ_FETCHES_DIR/vocab.spm $MOZ_FETCHES_DIR/lex.s2t.pruned $MOZ_FETCHES_DIR/devset.ru /builds/worker/artifacts']
[task 2024-02-28T03:31:03.504Z] 
[task 2024-02-28T03:31:03.506Z] Decompress:  1/ 2 files. Current: ...s/devset.ru.zst : 0 MB...    
[task 2024-02-28T03:31:03.506Z]                                                                                
[task 2024-02-28T03:31:03.506Z] 
[task 2024-02-28T03:31:03.506Z]                                                                                
[task 2024-02-28T03:31:03.506Z] 2 files decompressed : 667365 bytes total 
[task 2024-02-28T03:31:03.840Z] WARNING: The directory '/builds/worker/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
[task 2024-02-28T03:31:03.840Z] Defaulting to user installation because normal site-packages is not writeable
[task 2024-02-28T03:31:04.391Z] Collecting numpy==1.25.0
[task 2024-02-28T03:31:04.547Z]   Downloading numpy-1.25.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.6 MB)
[task 2024-02-28T03:31:04.906Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.6/17.6 MB 92.5 MB/s eta 0:00:00
[task 2024-02-28T03:31:04.997Z] Installing collected packages: numpy
[task 2024-02-28T03:31:06.657Z] Successfully installed numpy-1.25.0
[task 2024-02-28T03:31:06.737Z] + set -euo pipefail
[task 2024-02-28T03:31:06.737Z] + echo '###### Quantizing a model'
[task 2024-02-28T03:31:06.737Z] ###### Quantizing a model
[task 2024-02-28T03:31:06.737Z] + test -v BMT_MARIAN
[task 2024-02-28T03:31:06.737Z] + test -v BIN
[task 2024-02-28T03:31:06.737Z] + test -v SRC
[task 2024-02-28T03:31:06.737Z] + test -v TRG
[task 2024-02-28T03:31:06.737Z] + model=/builds/worker/fetches/final.model.npz.best-chrf.npz
[task 2024-02-28T03:31:06.737Z] + vocab=/builds/worker/fetches/vocab.spm
[task 2024-02-28T03:31:06.737Z] + shortlist=/builds/worker/fetches/lex.s2t.pruned
[task 2024-02-28T03:31:06.737Z] + devtest_src=/builds/worker/fetches/devset.ru
[task 2024-02-28T03:31:06.737Z] + output_dir=/builds/worker/artifacts
[task 2024-02-28T03:31:06.738Z] ++ dirname /builds/worker/checkouts/vcs/pipeline/quantize/quantize.sh
[task 2024-02-28T03:31:06.738Z] + cd /builds/worker/checkouts/vcs/pipeline/quantize
[task 2024-02-28T03:31:06.738Z] + res_model=/builds/worker/artifacts/model.intgemm.alphas.bin
[task 2024-02-28T03:31:06.738Z] + mkdir -p /builds/worker/artifacts
[task 2024-02-28T03:31:06.740Z] + cp /builds/worker/fetches/vocab.spm /builds/worker/artifacts
[task 2024-02-28T03:31:06.741Z] + echo '### Decoding a sample test set in order to get typical quantization values'
[task 2024-02-28T03:31:06.741Z] ### Decoding a sample test set in order to get typical quantization values
[task 2024-02-28T03:31:06.741Z] + test -s /builds/worker/artifacts/quantmults
[task 2024-02-28T03:31:06.741Z] + /builds/worker/fetches/marian-decoder -m /builds/worker/fetches/final.model.npz.best-chrf.npz -v /builds/worker/fetches/vocab.spm /builds/worker/fetches/vocab.spm -c decoder.yml -i /builds/worker/fetches/devset.ru -o /builds/worker/artifacts/output.en --shortlist /builds/worker/fetches/lex.s2t.pruned false --quiet --quiet-translation --log /builds/worker/artifacts/cpu.output.log --dump-quantmult
[task 2024-02-28T03:32:39.837Z] /builds/worker/checkouts/vcs/pipeline/quantize/quantize.sh: line 41:   298 Aborted                 (core dumped) "${BMT_MARIAN}"/marian-decoder -m "${model}" -v "${vocab}" "${vocab}" -c "decoder.yml" -i "${devtest_src}" -o "${output_dir}/output.${TRG}" --shortlist "${shortlist}" false --quiet --quiet-translation --log "${output_dir}/cpu.output.log" --dump-quantmult 2> "${output_dir}/quantmults"
[fetches 2024-02-28T03:32:39.837Z] removing /builds/worker/fetches
[fetches 2024-02-28T03:32:40.696Z] finished
[taskcluster 2024-02-28 03:32:40.871Z] === Task Finished ===
[taskcluster 2024-02-28 03:32:42.448Z] Unsuccessful task run with exit code: 134 completed in 183.381 seconds