Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish comet metrics #621

Merged
merged 19 commits into from
May 22, 2024

Revert "Enable publication"

6d213f8
Select commit
Loading
Failed to load commit list.
Merged

Publish comet metrics #621

Revert "Enable publication"
6d213f8
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / translate-corpus-ru-en-1/2 succeeded May 22, 2024 in 20m 42s

FirefoxCI (pull_request)

translate corpus for ru-en

Details

View task in Taskcluster
View logs in Taskcluster


  with urllib.request.urlopen(
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cCr_z2sITDGiow53WiHs8g/artifacts/public/build/marian.tar.zst to /home/ubuntu/tasks/task_171641228457277/fetches/marian.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cCr_z2sITDGiow53WiHs8g/artifacts/public/build/marian.tar.zst
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/final.model.npz.best-chrf.npz to /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz
attempt 1/5Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/final.model.npz.best-chrf.npz
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/vocab.spm to /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm

Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/vocab.spm
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/vocab.spm resolved to 252230 bytes with sha256 9d8ac29958816890d18f1ab76bc153d1b73b59a3e447780e1021e936802d92ac in 0.277s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/O9MiNHaZRm2xl-PqmER1ng/artifacts/public/build/file.1.zst resolved to 151390 bytes with sha256 c5dbdd1c1371dade005f5c0100a4894707c8f5706588de085cafb70f17e72b16 in 0.324s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/VLppB-nISxaB4CSrbocOCw/artifacts/public/build/final.model.npz.best-chrf.npz resolved to 178659619 bytes with sha256 14817d599221de9942895e5d2cb0324ff66b53c2abae01b4ab29280057f33044 in 1.440s
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/DtQ5zn1HS4-Q7G_cWlRc0Q/artifacts/public/build/cuda-toolkit.tar.zst resolved to 3295138224 bytes with sha256 e3bdbf0078b8eb8c7804039d6cf6a10b673212bedf1d016a7b4a7eed3ead5f04 in 39.574s
Extracting /home/ubuntu/tasks/task_171641228457277/fetches/cuda-toolkit.tar.zst to /home/ubuntu/tasks/task_171641228457277/fetches
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/cCr_z2sITDGiow53WiHs8g/artifacts/public/build/marian.tar.zst resolved to 684170918 bytes with sha256 674b3ad4462d1b7ef22e1f2a5a1e340385136821b0e5f4b48af357281a41d819 in 41.375s
Extracting /home/ubuntu/tasks/task_171641228457277/fetches/marian.tar.zst to /home/ubuntu/tasks/task_171641228457277/fetches
/home/ubuntu/tasks/task_171641228457277/fetches/marian.tar.zst extracted in 6.442s
Removing /home/ubuntu/tasks/task_171641228457277/fetches/marian.tar.zst
/home/ubuntu/tasks/task_171641228457277/fetches/cuda-toolkit.tar.zst extracted in 22.590s
Removing /home/ubuntu/tasks/task_171641228457277/fetches/cuda-toolkit.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 62.16762520900001, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-05-22T21:13:31.547Z] finished fetching artifacts
[task 2024-05-22T21:13:31.547Z] executing ['bash', '-xc', 'export MARIAN=$MOZ_FETCHES_DIR && $VCS_PATH/taskcluster/scripts/pipeline/translate-taskcluster.sh $MOZ_FETCHES_DIR/file.1.zst artifacts nbest $MOZ_FETCHES_DIR/vocab.spm $MOZ_FETCHES_DIR/model*/*.npz  --mini-batch-words 1000 --precision float16']
[task 2024-05-22T21:13:31.549Z] + export MARIAN=/home/ubuntu/tasks/task_171641228457277/fetches
[task 2024-05-22T21:13:31.549Z] + MARIAN=/home/ubuntu/tasks/task_171641228457277/fetches
[task 2024-05-22T21:13:31.549Z] + ./checkouts/vcs/taskcluster/scripts/pipeline/translate-taskcluster.sh /home/ubuntu/tasks/task_171641228457277/fetches/file.1.zst artifacts nbest /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz --mini-batch-words 1000 --precision float16
[task 2024-05-22T21:13:31.551Z] + set -euo pipefail
[task 2024-05-22T21:13:31.551Z] + input=/home/ubuntu/tasks/task_171641228457277/fetches/file.1.zst
[task 2024-05-22T21:13:31.551Z] + output_dir=artifacts
[task 2024-05-22T21:13:31.551Z] + type=nbest
[task 2024-05-22T21:13:31.551Z] + other_args=("${@:4}")
[task 2024-05-22T21:13:31.551Z] ++ dirname ./checkouts/vcs/taskcluster/scripts/pipeline/translate-taskcluster.sh
[task 2024-05-22T21:13:31.553Z] + pushd ./checkouts/vcs/taskcluster/scripts/pipeline/../../..
[task 2024-05-22T21:13:31.553Z] ++ pwd
[task 2024-05-22T21:13:31.553Z] + VCS_ROOT=/home/ubuntu/tasks/task_171641228457277/checkouts/vcs
[task 2024-05-22T21:13:31.553Z] + popd
[task 2024-05-22T21:13:31.554Z] + mkdir -p artifacts
[task 2024-05-22T21:13:31.555Z] + zstd -d --rm /home/ubuntu/tasks/task_171641228457277/fetches/file.1.zst
[task 2024-05-22T21:13:31.821Z] 
[task 2024-05-22T21:13:31.823Z] 7/fetches/file.1.zst : 0 MB...     
[task 2024-05-22T21:13:31.823Z]                                                                                
[task 2024-05-22T21:13:31.823Z] /home/ubuntu/tasks/task_171641228457277/fetches/file.1.zst: 756288 bytes 
[task 2024-05-22T21:13:31.824Z] + input=/home/ubuntu/tasks/task_171641228457277/fetches/file.1
[task 2024-05-22T21:13:31.824Z] + outfile=/home/ubuntu/tasks/task_171641228457277/fetches/file.1.out
[task 2024-05-22T21:13:31.824Z] + '[' nbest = nbest ']'
[task 2024-05-22T21:13:31.824Z] + outfile=/home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest
[task 2024-05-22T21:13:31.824Z] + '[' -s /home/ubuntu/tasks/task_171641228457277/fetches/file.1 ']'
[task 2024-05-22T21:13:31.824Z] + '[' nbest = plain ']'
[task 2024-05-22T21:13:31.824Z] + '[' nbest = nbest ']'
[task 2024-05-22T21:13:31.824Z] + /home/ubuntu/tasks/task_171641228457277/checkouts/vcs/pipeline/translate/translate-nbest.sh /home/ubuntu/tasks/task_171641228457277/fetches/file.1 /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz --mini-batch-words 1000 --precision float16
[task 2024-05-22T21:13:31.826Z] + set -euo pipefail
[task 2024-05-22T21:13:31.826Z] + test -v GPUS
[task 2024-05-22T21:13:31.826Z] + test -v MARIAN
[task 2024-05-22T21:13:31.826Z] + test -v WORKSPACE
[task 2024-05-22T21:13:31.826Z] + input=/home/ubuntu/tasks/task_171641228457277/fetches/file.1
[task 2024-05-22T21:13:31.826Z] + vocab=/home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm
[task 2024-05-22T21:13:31.826Z] + models=("${@:3}")
[task 2024-05-22T21:13:31.826Z] ++ dirname /home/ubuntu/tasks/task_171641228457277/checkouts/vcs/pipeline/translate/translate-nbest.sh
[task 2024-05-22T21:13:31.828Z] + cd /home/ubuntu/tasks/task_171641228457277/checkouts/vcs/pipeline/translate
[task 2024-05-22T21:13:31.828Z] + /home/ubuntu/tasks/task_171641228457277/fetches/marian-decoder -c decoder.yml -m /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz --mini-batch-words 1000 --precision float16 -v /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm -i /home/ubuntu/tasks/task_171641228457277/fetches/file.1 -o /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest --log /home/ubuntu/tasks/task_171641228457277/fetches/file.1.log --n-best -d 0 1 2 3 -w 12000
[task 2024-05-22T21:13:50.470Z] [2024-05-22 21:13:50] [marian] Marian v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-05-22T21:13:50.470Z] [2024-05-22 21:13:50] [marian] Running on translations-1-b-linux-v100-gpu-4-1tb-fsom-gvtri2nuicnufrtoq as process 3749 with command line:
[task 2024-05-22T21:13:50.470Z] [2024-05-22 21:13:50] [marian] /home/ubuntu/tasks/task_171641228457277/fetches/marian-decoder -c decoder.yml -m /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz --mini-batch-words 1000 --precision float16 -v /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm -i /home/ubuntu/tasks/task_171641228457277/fetches/file.1 -o /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest --log /home/ubuntu/tasks/task_171641228457277/fetches/file.1.log --n-best -d 0 1 2 3 -w 12000
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] alignment: ""
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] allow-special: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] allow-unk: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] authors: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] beam-size: 8
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-class-symbol: "[CLS]"
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-mask-symbol: "[MASK]"
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-masking-fraction: 0.15
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-sep-symbol: "[SEP]"
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-train-type-embeddings: true
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] bert-type-vocab-size: 2
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] best-deep: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] build-info: ""
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] check-nan: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] cite: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-augment-bad: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-dropout: 0.1
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-final-sigmoid: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-mix: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-mix-norm: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-mixup: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-mixup-reg: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-pooler-ffn:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 2048
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 1024
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-prepend-zero: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] comet-stop-grad: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] cpu-threads: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] data-threads: 8
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dec-cell: gru
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dec-cell-base-depth: 2
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dec-cell-high-depth: 1
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dec-depth: 6
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] devices:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 1
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 2
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 3
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dim-emb: 512
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dim-rnn: 1024
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dim-vocabs:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 1000
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - 1000
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] dump-config: ""
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] enc-cell: gru
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] enc-cell-depth: 1
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] enc-depth: 6
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] enc-type: bidirectional
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] factors-combine: sum
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] factors-dim-emb: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] force-decode: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] gemm-type: float32
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] ignore-model-config: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] input:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - /home/ubuntu/tasks/task_171641228457277/fetches/file.1
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] input-types:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] interpolate-env-vars: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] layer-normalization: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] lemma-dependency: ""
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] lemma-dim-emb: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] log: /home/ubuntu/tasks/task_171641228457277/fetches/file.1.log
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] log-level: info
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] log-time-zone: ""
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] max-length: 200
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] max-length-crop: true
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] max-length-factor: 3
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] maxi-batch: 1000
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] maxi-batch-sort: src
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] mini-batch: 16
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] mini-batch-words: 1000
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] model-mmap: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] models:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] n-best: true
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] no-spm-decode: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] no-spm-encode: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] normalize: 1.0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] optimize: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] output: /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] output-approx-knn:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] output-omit-bias: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] output-sampling:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] precision:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   - float16
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] quantize-range: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] quiet: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] quiet-translation: True
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] relative-paths: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] right-left: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] seed: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] shortlist:
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] skip: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] skip-cost: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] stat-freq: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] tied-embeddings: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] tied-embeddings-all: true
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] tied-embeddings-src: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-aan-activation: swish
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-aan-depth: 2
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-aan-nogate: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-decoder-autoreg: self-attention
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-decoder-dim-ffn: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-decoder-ffn-depth: 0
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-depth-scaling: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-dim-aan: 2048
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-dim-ffn: 2048
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-ffn-activation: relu
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-ffn-depth: 2
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-guided-alignment-layer: last
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-heads: 8
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-no-affine: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-no-bias: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-no-projection: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-pool: false
[task 2024-05-22T21:13:50.472Z] [2024-05-22 21:13:50] [config] transformer-postprocess: dan
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-postprocess-emb: d
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-postprocess-top: ""
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-preprocess: ""
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-rnn-projection: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-tied-layers:
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] transformer-train-position-embeddings: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] tsv: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] tsv-fields: 0
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] type: transformer
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] ulr: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] ulr-dim-emb: 0
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] ulr-trainable-transformation: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] version: v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] vocabs:
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config]   - /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config]   - /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] weights:
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config]   []
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] word-penalty: 0
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] word-scores: false
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] workspace: 12000
[task 2024-05-22T21:13:50.473Z] [2024-05-22 21:13:50] [config] Loaded model has been created with Marian v1.12.14 2d067af 2024-02-16 11:44:13 -0500
[task 2024-05-22T21:13:50.474Z] [2024-05-22 21:13:50] [data] Loading SentencePiece vocabulary from file /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm
[task 2024-05-22T21:13:50.478Z] [2024-05-22 21:13:50] [data] Loading SentencePiece vocabulary from file /home/ubuntu/tasks/task_171641228457277/fetches/vocab.spm
[task 2024-05-22T21:13:50.481Z] [2024-05-22 21:13:50] Loading model from /home/ubuntu/tasks/task_171641228457277/fetches/model1/final.model.npz.best-chrf.npz
[task 2024-05-22T21:13:53.481Z] [2024-05-22 21:13:53] [memory] Extending reserved space to 12032 MB (device gpu0)
[task 2024-05-22T21:13:53.481Z] [2024-05-22 21:13:53] [memory] Extending reserved space to 12032 MB (device gpu3)
[task 2024-05-22T21:13:53.481Z] [2024-05-22 21:13:53] [memory] Extending reserved space to 12032 MB (device gpu2)
[task 2024-05-22T21:13:53.481Z] [2024-05-22 21:13:53] [memory] Extending reserved space to 12032 MB (device gpu1)
[task 2024-05-22T21:13:53.775Z] [2024-05-22 21:13:53] Loaded model config
[task 2024-05-22T21:13:53.775Z] [2024-05-22 21:13:53] Loaded model config
[task 2024-05-22T21:13:53.776Z] [2024-05-22 21:13:53] Loaded model config
[task 2024-05-22T21:13:53.794Z] [2024-05-22 21:13:53] Loaded model config
[task 2024-05-22T21:13:53.923Z] [2024-05-22 21:13:53] Loading scorer of type transformer as feature F0
[task 2024-05-22T21:13:53.925Z] [2024-05-22 21:13:53] Loading scorer of type transformer as feature F0
[task 2024-05-22T21:13:53.925Z] [2024-05-22 21:13:53] Loading scorer of type transformer as feature F0
[task 2024-05-22T21:13:53.954Z] [2024-05-22 21:13:53] Loading scorer of type transformer as feature F0
[task 2024-05-22T21:13:54.128Z] [2024-05-22 21:13:54] [memory] Reserving 85 MB, device gpu3
[task 2024-05-22T21:13:54.132Z] [2024-05-22 21:13:54] [memory] Reserving 85 MB, device gpu2
[task 2024-05-22T21:13:54.134Z] [2024-05-22 21:13:54] [memory] Reserving 85 MB, device gpu1
[task 2024-05-22T21:13:54.175Z] [2024-05-22 21:13:54] [memory] Reserving 85 MB, device gpu0
[task 2024-05-22T21:13:56.975Z] [2024-05-22 21:13:56] [gpu] 16-bit TensorCores enabled for float32 matrix operations
[task 2024-05-22T21:15:20.255Z] [2024-05-22 21:15:20] Total time: 86.01268s wall
[task 2024-05-22T21:15:22.070Z] ++ wc -l
[task 2024-05-22T21:15:22.096Z] ++ wc -l
[task 2024-05-22T21:15:22.098Z] + test 28112 -eq 28112
[task 2024-05-22T21:15:22.098Z] + zstd --rm /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest
[task 2024-05-22T21:15:22.100Z] 
[task 2024-05-22T21:15:22.100Z]                                                                                
[task 2024-05-22T21:15:22.118Z] Read :  0 /  6 MB ==>  0%
[task 2024-05-22T21:15:22.118Z]                                                                                
[task 2024-05-22T21:15:22.118Z] /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest :  5.05%   (7029314 => 354672 bytes, /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest.zst) 
[task 2024-05-22T21:15:22.121Z] + cp /home/ubuntu/tasks/task_171641228457277/fetches/file.1.nbest.zst artifacts
[fetches 2024-05-22T21:15:22.123Z] removing /home/ubuntu/tasks/task_171641228457277/fetches
[fetches 2024-05-22T21:15:23.809Z] finished
[taskcluster 2024-05-22T21:15:23.819Z]    Exit Code: 0
[taskcluster 2024-05-22T21:15:23.819Z]    User Time: 5m59.549679s
[taskcluster 2024-05-22T21:15:23.819Z]  Kernel Time: 33.926253s
[taskcluster 2024-05-22T21:15:23.819Z]    Wall Time: 3m58.609044948s
[taskcluster 2024-05-22T21:15:23.819Z]       Result: SUCCEEDED
[taskcluster 2024-05-22T21:15:23.819Z] === Task Finished ===
[taskcluster 2024-05-22T21:15:23.820Z] Task Duration: 3m58.610782707s
[taskcluster 2024-05-22T21:15:23.854Z] Uploading artifact public/build/file.1.nbest.zst from file /home/ubuntu/tasks/task_171641228457277/artifacts/file.1.nbest.zst with content encoding "identity", mime type "application/zstd" and expiry 2024-08-20T20:54:41.991Z
[taskcluster 2024-05-22T21:15:24.032Z] [mounts] Preserving cache: Moving "/home/ubuntu/tasks/task_171641228457277/checkouts" to "/home/ubuntu/caches/O1xyCeovS2mwLy0zRlA6kw"
[taskcluster 2024-05-22T21:15:24.091Z] Uploading link artifact public/logs/live.log to artifact public/logs/live_backing.log with expiry 2024-08-20T20:54:41.991Z