[taskcluster 2023-12-15 19:00:06.274Z] Task ID: JFsmTdjPSAuRu00B4DF2rg
[taskcluster 2023-12-15 19:00:06.275Z] Worker ID: 4535462932033642296
[taskcluster 2023-12-15 19:00:06.275Z] Worker Group: us-central1
[taskcluster 2023-12-15 19:00:06.275Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2023-12-15 19:00:06.275Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2023-12-15 19:00:06.275Z] Worker Version: 38.0.5
[taskcluster 2023-12-15 19:00:06.275Z] Public IP: 34.27.137.195
[taskcluster 2023-12-15 19:00:06.275Z] Hostname: translations-1-b-linux-large-gcp-h1smykxfsw-9jx64nicptg
[taskcluster 2023-12-15 19:00:06.275Z] using cache "translations-level-1-checkouts-hg58-v3-2922149087d95338df26" -> /builds/worker/checkouts

[taskcluster 2023-12-15 19:00:10.261Z] Downloading artifact "public/image.tar.zst" from task ID: boZy5pKYRgODoLNelqxZNQ.
[taskcluster 2023-12-15 19:00:15.262Z] Download Progress: 78.80%
[taskcluster 2023-12-15 19:00:16.587Z] Downloaded artifact successfully.
[taskcluster 2023-12-15 19:00:16.587Z] Downloaded 655.676 mb
[taskcluster 2023-12-15 19:00:16.588Z] Decompressing downloaded image
[taskcluster 2023-12-15 19:00:22.737Z] Loading docker image from downloaded archive.
[taskcluster 2023-12-15 19:00:59.340Z] Image 'public/image.tar.zst' from task 'boZy5pKYRgODoLNelqxZNQ' loaded.  Using image ID sha256:16372f64b8140fab260d7d32d4959601b619c569d29d8c1fc951a187f9302655.
[taskcluster 2023-12-15 19:00:59.738Z] === Task Starting ===
[setup 2023-12-15T19:01:06.228Z] run-task started in /builds/worker
[setup 2023-12-15T19:01:06.228Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ -- bash -c pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/data/requirements/data.txt && python3 $VCS_PATH/pipeline/data/dataset_importer.py --type corpus --dataset opus_ada83/v1 --output_prefix /builds/worker/artifacts/ada83_v1
[setup 2023-12-15T19:01:06.228Z] Python version: 3.10.12
[cache 2023-12-15T19:01:06.230Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2023-12-15T19:01:06.230Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2023-12-15T19:01:06.230Z] volume /builds/worker/checkouts is a cache
[setup 2023-12-15T19:01:06.230Z] running as worker:worker
[vcs 2023-12-15T19:01:06.231Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2023-12-15T19:01:06.232Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2023-12-15T19:01:06.944Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'followup']
[vcs 2023-12-15T19:01:07.261Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2023-12-15T19:01:07.261Z]  * branch            followup   -> FETCH_HEAD
[vcs 2023-12-15T19:01:07.263Z] executing ['git', 'checkout', '-f', '-B', 'followup', '4aa774c2a9f31b93627bcac82e0c235839cb7c0e']
[vcs 2023-12-15T19:01:07.301Z] Switched to a new branch 'followup'
[vcs 2023-12-15T19:01:07.301Z] executing ['git', 'submodule', 'init']
[vcs 2023-12-15T19:01:07.322Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2023-12-15T19:01:07.322Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2023-12-15T19:01:07.322Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2023-12-15T19:01:07.323Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2023-12-15T19:01:07.323Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2023-12-15T19:01:07.323Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2023-12-15T19:01:07.324Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2023-12-15T19:01:07.345Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2023-12-15T19:01:10.345Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2023-12-15T19:01:10.659Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2023-12-15T19:01:11.007Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2023-12-15T19:01:13.077Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2023-12-15T19:01:16.571Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2023-12-15T19:01:17.231Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2023-12-15T19:01:17.241Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2023-12-15T19:01:17.252Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2023-12-15T19:01:17.278Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2023-12-15T19:01:17.390Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2023-12-15T19:01:17.410Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2023-12-15T19:01:17.410Z] cleaning git checkout...
[vcs 2023-12-15T19:01:17.410Z] executing ['git', 'clean', '-nxdff']
[vcs 2023-12-15T19:01:17.413Z] removing []
[vcs 2023-12-15T19:01:17.413Z] successfully cleaned git checkout!
[vcs 2023-12-15T19:01:17.415Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/4aa774c2a9f31b93627bcac82e0c235839cb7c0e' title='Built from firefox-translations-training commit 4aa774c2a9f31b93627bcac82e0c235839cb7c0e'>4aa774c2a9f31b93627bcac82e0c235839cb7c0e</a>
[task 2023-12-15T19:01:17.415Z] executing ['bash', '-c', 'pip3 install --upgrade pip setuptools && pip3 install -r $VCS_PATH/pipeline/data/requirements/data.txt && python3 $VCS_PATH/pipeline/data/dataset_importer.py --type corpus --dataset opus_ada83/v1 --output_prefix /builds/worker/artifacts/ada83_v1']
[task 2023-12-15T19:01:17.771Z] Defaulting to user installation because normal site-packages is not writeable
[task 2023-12-15T19:01:17.797Z] Requirement already satisfied: pip in /usr/lib/python3/dist-packages (22.0.2)
[task 2023-12-15T19:01:17.989Z] Collecting pip
[task 2023-12-15T19:01:18.184Z]   Downloading pip-23.3.1-py3-none-any.whl (2.1 MB)
[task 2023-12-15T19:01:18.406Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 9.8 MB/s eta 0:00:00
[task 2023-12-15T19:01:18.417Z] Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (59.6.0)
[task 2023-12-15T19:01:18.751Z] Collecting setuptools
[task 2023-12-15T19:01:18.793Z]   Downloading setuptools-69.0.2-py3-none-any.whl (819 kB)
[task 2023-12-15T19:01:18.804Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 819.5/819.5 KB 97.1 MB/s eta 0:00:00
[task 2023-12-15T19:01:18.982Z] Installing collected packages: setuptools, pip
[task 2023-12-15T19:01:20.344Z] Successfully installed pip-23.3.1 setuptools-69.0.2
[task 2023-12-15T19:01:20.719Z] Defaulting to user installation because normal site-packages is not writeable
[task 2023-12-15T19:01:20.777Z] Collecting opustrainer@ git+https://github.com/hplt-project/OpusTrainer.git@9133e1525c7ee37f53ea14ee6a180152bf7ea192 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 11))
[task 2023-12-15T19:01:20.777Z]   Cloning https://github.com/hplt-project/OpusTrainer.git (to revision 9133e1525c7ee37f53ea14ee6a180152bf7ea192) to /tmp/pip-install-wqiba4ly/opustrainer_0eaeb310bcc0479ca74e31c78214aba6
[task 2023-12-15T19:01:20.779Z]   Running command git clone --filter=blob:none --quiet https://github.com/hplt-project/OpusTrainer.git /tmp/pip-install-wqiba4ly/opustrainer_0eaeb310bcc0479ca74e31c78214aba6
[task 2023-12-15T19:01:21.532Z]   Running command git rev-parse -q --verify 'sha^9133e1525c7ee37f53ea14ee6a180152bf7ea192'
[task 2023-12-15T19:01:21.534Z]   Running command git fetch -q https://github.com/hplt-project/OpusTrainer.git 9133e1525c7ee37f53ea14ee6a180152bf7ea192
[task 2023-12-15T19:01:21.790Z]   Running command git checkout -q 9133e1525c7ee37f53ea14ee6a180152bf7ea192
[task 2023-12-15T19:01:22.065Z]   Resolved https://github.com/hplt-project/OpusTrainer.git to commit 9133e1525c7ee37f53ea14ee6a180152bf7ea192
[task 2023-12-15T19:01:22.070Z]   Installing build dependencies: started
[task 2023-12-15T19:01:24.189Z]   Installing build dependencies: finished with status 'done'
[task 2023-12-15T19:01:24.191Z]   Getting requirements to build wheel: started
[task 2023-12-15T19:01:24.410Z]   Getting requirements to build wheel: finished with status 'done'
[task 2023-12-15T19:01:24.412Z]   Preparing metadata (pyproject.toml): started
[task 2023-12-15T19:01:24.636Z]   Preparing metadata (pyproject.toml): finished with status 'done'
[task 2023-12-15T19:01:24.773Z] Collecting click==8.1.7 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 7))
[task 2023-12-15T19:01:24.938Z]   Downloading click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
[task 2023-12-15T19:01:24.992Z] Collecting joblib==1.3.2 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 9))
[task 2023-12-15T19:01:25.027Z]   Downloading joblib-1.3.2-py3-none-any.whl.metadata (5.4 kB)
[task 2023-12-15T19:01:25.089Z] Collecting pyyaml==6.0.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 13))
[task 2023-12-15T19:01:25.124Z]   Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.1 kB)
[task 2023-12-15T19:01:25.523Z] Collecting regex==2023.10.3 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 15))
[task 2023-12-15T19:01:25.558Z]   Downloading regex-2023.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
[task 2023-12-15T19:01:25.584Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 40.9/40.9 kB 1.6 MB/s eta 0:00:00
[task 2023-12-15T19:01:25.615Z] Collecting sacremoses==0.0.53 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 17))
[task 2023-12-15T19:01:25.649Z]   Downloading sacremoses-0.0.53.tar.gz (880 kB)
[task 2023-12-15T19:01:25.765Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 880.6/880.6 kB 7.8 MB/s eta 0:00:00
[task 2023-12-15T19:01:25.805Z]   Preparing metadata (setup.py): started
[task 2023-12-15T19:01:25.963Z]   Preparing metadata (setup.py): finished with status 'done'
[task 2023-12-15T19:01:26.032Z] Collecting sentencepiece==0.1.99 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 19))
[task 2023-12-15T19:01:26.067Z]   Downloading sentencepiece-0.1.99-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
[task 2023-12-15T19:01:26.108Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 33.5 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.113Z] Requirement already satisfied: six==1.16.0 in /usr/local/lib/python3.10/dist-packages (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 21)) (1.16.0)
[task 2023-12-15T19:01:26.189Z] Collecting tqdm==4.66.1 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 23))
[task 2023-12-15T19:01:26.224Z]   Downloading tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
[task 2023-12-15T19:01:26.228Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.6/57.6 kB 21.8 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.254Z] Collecting typo==0.1.5 (from -r /builds/worker/checkouts/vcs/pipeline/data/requirements/data.txt (line 25))
[task 2023-12-15T19:01:26.289Z]   Downloading typo-0.1.5.tar.gz (7.0 kB)
[task 2023-12-15T19:01:26.296Z]   Preparing metadata (setup.py): started
[task 2023-12-15T19:01:26.447Z]   Preparing metadata (setup.py): finished with status 'done'
[task 2023-12-15T19:01:26.515Z] Downloading click-8.1.7-py3-none-any.whl (97 kB)
[task 2023-12-15T19:01:26.520Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 97.9/97.9 kB 31.6 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.557Z] Downloading joblib-1.3.2-py3-none-any.whl (302 kB)
[task 2023-12-15T19:01:26.563Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 302.2/302.2 kB 79.2 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.597Z] Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
[task 2023-12-15T19:01:26.606Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 118.5 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.641Z] Downloading regex-2023.10.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (773 kB)
[task 2023-12-15T19:01:26.717Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 773.9/773.9 kB 10.3 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.752Z] Downloading tqdm-4.66.1-py3-none-any.whl (78 kB)
[task 2023-12-15T19:01:26.756Z]    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.3/78.3 kB 32.8 MB/s eta 0:00:00
[task 2023-12-15T19:01:26.770Z] Building wheels for collected packages: sacremoses, typo, opustrainer
[task 2023-12-15T19:01:26.770Z]   Building wheel for sacremoses (setup.py): started
[task 2023-12-15T19:01:27.146Z]   Building wheel for sacremoses (setup.py): finished with status 'done'
[task 2023-12-15T19:01:27.150Z]   Created wheel for sacremoses: filename=sacremoses-0.0.53-py3-none-any.whl size=895241 sha256=aad72730d6369e1f637f40e601d50c30675b48b304bbfcff2b71befce6398b26
[task 2023-12-15T19:01:27.150Z]   Stored in directory: /builds/worker/.cache/pip/wheels/00/24/97/a2ea5324f36bc626e1ea0267f33db6aa80d157ee977e9e42fb
[task 2023-12-15T19:01:27.153Z]   Building wheel for typo (setup.py): started
[task 2023-12-15T19:01:27.378Z]   Building wheel for typo (setup.py): finished with status 'done'
[task 2023-12-15T19:01:27.379Z]   Created wheel for typo: filename=typo-0.1.5-py3-none-any.whl size=6837 sha256=7fd8ffb595de86aa0755ef258352e7450f345f7c5debab1c80e2389f40638cf9
[task 2023-12-15T19:01:27.379Z]   Stored in directory: /builds/worker/.cache/pip/wheels/2e/2f/73/60e0ce42d1375a386b9171a37cd5536e173ad950a98e7dc6b1
[task 2023-12-15T19:01:27.384Z]   Building wheel for opustrainer (pyproject.toml): started
[task 2023-12-15T19:01:27.632Z]   Building wheel for opustrainer (pyproject.toml): finished with status 'done'
[task 2023-12-15T19:01:27.633Z]   Created wheel for opustrainer: filename=opustrainer-0.2-py3-none-any.whl size=39889 sha256=c1927978e1afc08397c1927b2c9db08996a5e96ba2540cbbb49764dd21de8c10
[task 2023-12-15T19:01:27.633Z]   Stored in directory: /builds/worker/.cache/pip/wheels/be/de/18/46a4ed14bd505e2a41b3680ce76f5ec719db6334485446d524
[task 2023-12-15T19:01:27.635Z] Successfully built sacremoses typo opustrainer
[task 2023-12-15T19:01:27.785Z] Installing collected packages: typo, sentencepiece, tqdm, regex, pyyaml, joblib, click, sacremoses, opustrainer
[task 2023-12-15T19:01:28.257Z] Successfully installed click-8.1.7 joblib-1.3.2 opustrainer-0.2 pyyaml-6.0.1 regex-2023.10.3 sacremoses-0.0.53 sentencepiece-0.1.99 tqdm-4.66.1 typo-0.1.5
[task 2023-12-15T19:01:30.563Z] Running with arguments: ['/builds/worker/checkouts/vcs/pipeline/data/dataset_importer.py', '--type', 'corpus', '--dataset', 'opus_ada83/v1', '--output_prefix', '/builds/worker/artifacts/ada83_v1']
[task 2023-12-15T19:01:30.564Z] Starting dataset import and augmentation.
[task 2023-12-15T19:01:30.564Z] Downloading parallel dataset
[task 2023-12-15T19:01:30.564Z] + set -euo pipefail
[task 2023-12-15T19:01:30.564Z] + [[ -z ru ]]
[task 2023-12-15T19:01:30.564Z] + [[ -z en ]]
[task 2023-12-15T19:01:30.564Z] + dataset=opus_ada83/v1
[task 2023-12-15T19:01:30.564Z] + output_prefix=/builds/worker/artifacts/ada83_v1
[task 2023-12-15T19:01:30.564Z] + echo '###### Downloading dataset opus_ada83/v1'
[task 2023-12-15T19:01:30.564Z] ###### Downloading dataset opus_ada83/v1
[task 2023-12-15T19:01:30.564Z] ++ dirname /builds/worker/checkouts/vcs/pipeline/data/download-corpus.sh
[task 2023-12-15T19:01:30.564Z] + cd /builds/worker/checkouts/vcs/pipeline/data
[task 2023-12-15T19:01:30.564Z] ++ dirname /builds/worker/artifacts/ada83_v1
[task 2023-12-15T19:01:30.564Z] + dir=/builds/worker/artifacts
[task 2023-12-15T19:01:30.564Z] + mkdir -p /builds/worker/artifacts
[task 2023-12-15T19:01:30.564Z] + name=ada83/v1
[task 2023-12-15T19:01:30.564Z] + type=opus
[task 2023-12-15T19:01:30.564Z] + bash importers/corpus/opus.sh ru en /builds/worker/artifacts/ada83_v1 ada83/v1
[task 2023-12-15T19:01:30.564Z] + set -euo pipefail
[task 2023-12-15T19:01:30.564Z] + echo '###### Downloading opus corpus'
[task 2023-12-15T19:01:30.564Z] ###### Downloading opus corpus
[task 2023-12-15T19:01:30.564Z] + src=ru
[task 2023-12-15T19:01:30.564Z] + trg=en
[task 2023-12-15T19:01:30.564Z] + output_prefix=/builds/worker/artifacts/ada83_v1
[task 2023-12-15T19:01:30.564Z] + dataset=ada83/v1
[task 2023-12-15T19:01:30.564Z] + COMPRESSION_CMD=zstdmt
[task 2023-12-15T19:01:30.564Z] + ARTIFACT_EXT=zst
[task 2023-12-15T19:01:30.564Z] + name=ada83
[task 2023-12-15T19:01:30.564Z] + name_and_version=ada83_v1
[task 2023-12-15T19:01:30.564Z] ++ dirname /builds/worker/artifacts/ada83_v1
[task 2023-12-15T19:01:30.564Z] + tmp=/builds/worker/artifacts/opus/ada83_v1
[task 2023-12-15T19:01:30.564Z] + mkdir -p /builds/worker/artifacts/opus/ada83_v1
[task 2023-12-15T19:01:30.564Z] + archive_path=/builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip
[task 2023-12-15T19:01:30.564Z] + wget -O /builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip https://object.pouta.csc.fi/OPUS-ada83/v1/moses/ru-en.txt.zip
[task 2023-12-15T19:01:30.564Z] --2023-12-15 19:01:28--  https://object.pouta.csc.fi/OPUS-ada83/v1/moses/ru-en.txt.zip
[task 2023-12-15T19:01:30.564Z] Resolving object.pouta.csc.fi (object.pouta.csc.fi)... 86.50.254.18, 86.50.254.19
[task 2023-12-15T19:01:30.564Z] Connecting to object.pouta.csc.fi (object.pouta.csc.fi)|86.50.254.18|:443... connected.
[task 2023-12-15T19:01:30.564Z] HTTP request sent, awaiting response... 404 Not Found
[task 2023-12-15T19:01:30.564Z] 2023-12-15 19:01:29 ERROR 404: Not Found.
[task 2023-12-15T19:01:30.564Z] 
[task 2023-12-15T19:01:30.564Z] + wget -O /builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip https://object.pouta.csc.fi/OPUS-ada83/v1/moses/en-ru.txt.zip
[task 2023-12-15T19:01:30.564Z] --2023-12-15 19:01:29--  https://object.pouta.csc.fi/OPUS-ada83/v1/moses/en-ru.txt.zip
[task 2023-12-15T19:01:30.564Z] Resolving object.pouta.csc.fi (object.pouta.csc.fi)... 86.50.254.18, 86.50.254.19
[task 2023-12-15T19:01:30.564Z] Connecting to object.pouta.csc.fi (object.pouta.csc.fi)|86.50.254.18|:443... connected.
[task 2023-12-15T19:01:30.564Z] HTTP request sent, awaiting response... 200 OK
[task 2023-12-15T19:01:30.564Z] Length: 278237 (272K) [application/zip]
[task 2023-12-15T19:01:30.564Z] Saving to: ‘/builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip’
[task 2023-12-15T19:01:30.564Z] 
[task 2023-12-15T19:01:30.564Z]      0K .......... .......... .......... .......... .......... 18%  366K 1s
[task 2023-12-15T19:01:30.564Z]     50K .......... .......... .......... .......... .......... 36%  366K 0s
[task 2023-12-15T19:01:30.564Z]    100K .......... .......... .......... .......... .......... 55%  367K 0s
[task 2023-12-15T19:01:30.564Z]    150K .......... .......... .......... .......... .......... 73%  160M 0s
[task 2023-12-15T19:01:30.564Z]    200K .......... .......... .......... .......... .......... 92%  228M 0s
[task 2023-12-15T19:01:30.564Z]    250K .......... .......... .                               100%  159K=0.5s
[task 2023-12-15T19:01:30.564Z] 
[task 2023-12-15T19:01:30.564Z] 2023-12-15 19:01:30 (497 KB/s) - ‘/builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip’ saved [278237/278237]
[task 2023-12-15T19:01:30.564Z] 
[task 2023-12-15T19:01:30.564Z] + unzip -o /builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip -d /builds/worker/artifacts/opus/ada83_v1
[task 2023-12-15T19:01:30.564Z] Archive:  /builds/worker/artifacts/opus/ada83_v1/ada83.txt.zip
[task 2023-12-15T19:01:30.564Z]   inflating: /builds/worker/artifacts/opus/ada83_v1/README  
[task 2023-12-15T19:01:30.564Z]   inflating: /builds/worker/artifacts/opus/ada83_v1/LICENSE  
[task 2023-12-15T19:01:30.564Z]   inflating: /builds/worker/artifacts/opus/ada83_v1/ada83.en-ru.en  
[task 2023-12-15T19:01:30.564Z]   inflating: /builds/worker/artifacts/opus/ada83_v1/ada83.en-ru.ru  
[task 2023-12-15T19:01:30.564Z]   inflating: /builds/worker/artifacts/opus/ada83_v1/ada83.en-ru.xml  
[task 2023-12-15T19:01:30.564Z] + for lang in ${src} ${trg}
[task 2023-12-15T19:01:30.564Z] + zstdmt -c /builds/worker/artifacts/opus/ada83_v1/ada83.ru-en.ru
[task 2023-12-15T19:01:30.564Z] zstd: can't stat /builds/worker/artifacts/opus/ada83_v1/ada83.ru-en.ru : No such file or directory -- ignored 
[task 2023-12-15T19:01:30.564Z] + zstdmt -c /builds/worker/artifacts/opus/ada83_v1/ada83.en-ru.ru
[task 2023-12-15T19:01:30.564Z] + for lang in ${src} ${trg}
[task 2023-12-15T19:01:30.564Z] + zstdmt -c /builds/worker/artifacts/opus/ada83_v1/ada83.ru-en.en
[task 2023-12-15T19:01:30.564Z] zstd: can't stat /builds/worker/artifacts/opus/ada83_v1/ada83.ru-en.en : No such file or directory -- ignored 
[task 2023-12-15T19:01:30.564Z] + zstdmt -c /builds/worker/artifacts/opus/ada83_v1/ada83.en-ru.en
[task 2023-12-15T19:01:30.564Z] + rm -rf /builds/worker/artifacts/opus/ada83_v1
[task 2023-12-15T19:01:30.564Z] + echo '###### Done: Downloading opus corpus'
[task 2023-12-15T19:01:30.564Z] ###### Done: Downloading opus corpus
[task 2023-12-15T19:01:30.564Z] + echo '###### Done: Downloading dataset opus_ada83/v1'
[task 2023-12-15T19:01:30.564Z] ###### Done: Downloading dataset opus_ada83/v1
[task 2023-12-15T19:01:30.564Z] 
[task 2023-12-15T19:01:30.564Z] Finished dataset import and augmentation.
[taskcluster 2023-12-15 19:01:31.041Z] === Task Finished ===
[taskcluster 2023-12-15 19:01:31.503Z] Successful task run with exit code: 0 completed in 85.23 seconds