Add the ability to run starting from a specific task (fixes #227) #377
firefoxci-taskcluster / toolchain-extract-lex
succeeded
Feb 13, 2024 in 11m 33s
FirefoxCI (push)
extract-lex
Details
View task in Taskcluster
View logs in Taskcluster
[taskcluster 2024-02-13 19:57:59.688Z] Task ID: ImXVV4GuQP-epXQ3biawsQ
[taskcluster 2024-02-13 19:57:59.688Z] Worker ID: 2770288943112288151
[taskcluster 2024-02-13 19:57:59.688Z] Worker Group: us-central1
[taskcluster 2024-02-13 19:57:59.688Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-13 19:57:59.688Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-02-13 19:57:59.688Z] Worker Version: 38.0.5
[taskcluster 2024-02-13 19:57:59.688Z] Public IP: 35.226.154.108
[taskcluster 2024-02-13 19:57:59.688Z] Hostname: translations-1-b-linux-large-gcp-wewnhml9qxkdi-qvknkfiq
[taskcluster 2024-02-13 19:57:59.688Z] using cache "translations-level-1-checkouts-sparse-v3-7afeb851dd97df8f3607-Jew_2ep5Q0SJc7IaKa41Pg" -> /builds/worker/checkouts
[taskcluster 2024-02-13 19:58:03.985Z] Downloading artifact "public/image.tar.zst" from task ID: Jew_2ep5Q0SJc7IaKa41Pg.
[taskcluster 2024-02-13 19:58:08.987Z] Download Progress: 61.09%
[taskcluster 2024-02-13 19:58:11.635Z] Downloaded artifact successfully.
[taskcluster 2024-02-13 19:58:11.635Z] Downloaded 776.726 mb
[taskcluster 2024-02-13 19:58:11.636Z] Decompressing downloaded image
[taskcluster 2024-02-13 19:58:17.265Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-13 19:58:54.683Z] Image 'public/image.tar.zst' from task 'Jew_2ep5Q0SJc7IaKa41Pg' loaded. Using image ID sha256:0bda823ded0479e9bfa01cc4b80273ad1987c66c2b4e567901ee6a02a0583651.
[taskcluster 2024-02-13 19:58:55.007Z] === Task Starting ===
[setup 2024-02-13T19:58:57.065Z] run-task started in /builds/worker
[setup 2024-02-13T19:58:57.065Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --firefox_translations_training-sparse-profile=build/sparse-profiles/toolchain-build --task-cwd /builds/worker/checkouts -- vcs/taskcluster/scripts/toolchain/build-extract-lex.sh
[setup 2024-02-13T19:58:57.065Z] Python version: 3.10.12
[cache 2024-02-13T19:58:57.068Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-13T19:58:57.068Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-02-13T19:58:57.068Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-13T19:58:57.068Z] running as worker:worker
[vcs 2024-02-13T19:58:57.068Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:57.070Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:57.072Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-13T19:58:57.801Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:58.019Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:58.019Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:58.025Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:58.248Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:58.248Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:58.253Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/start-specific', '37fbf272d7eb316897377144111a3ef057becfd4']
[vcs 2024-02-13T19:58:58.308Z] Switched to a new branch 'refs/heads/start-specific'
[vcs 2024-02-13T19:58:58.309Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-13T19:58:58.328Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-13T19:58:58.328Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-13T19:58:58.328Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-13T19:58:58.329Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-13T19:58:58.329Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-13T19:58:58.329Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-13T19:58:58.330Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-13T19:58:58.350Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-13T19:58:59.482Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-13T19:58:59.780Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-13T19:59:00.093Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-13T19:59:00.727Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-13T19:59:02.059Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-13T19:59:02.561Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-13T19:59:02.570Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-13T19:59:02.580Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-13T19:59:02.605Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-13T19:59:02.707Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-13T19:59:02.726Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-13T19:59:02.726Z] cleaning git checkout...
[vcs 2024-02-13T19:59:02.726Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-13T19:59:02.729Z] removing []
[vcs 2024-02-13T19:59:02.729Z] successfully cleaned git checkout!
[vcs 2024-02-13T19:59:02.731Z] TinderboxPrint:<a href='https://github.com/mozilla/firefox-translations-training/commit/37fbf272d7eb316897377144111a3ef057becfd4' title='Built from firefox-translations-training commit 37fbf272d7eb316897377144111a3ef057becfd4'>37fbf272d7eb316897377144111a3ef057becfd4</a>
[setup 2024-02-13T19:59:02.731Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[setup 2024-02-13T19:59:02.731Z] UPLOAD_DIR is /builds/worker/artifacts
[fetches 2024-02-13T19:59:02.731Z] fetching artifacts
[fetches 2024-02-13T19:59:02.731Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bI1JG54vSnKw-0ySVorwWA/artifacts/public/extract-lex.tar.zst to /builds/worker/fetches/extract-lex.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bI1JG54vSnKw-0ySVorwWA/artifacts/public/extract-lex.tar.zst
/usr/local/bin/fetch-content:192: DeprecationWarning: cafile, capath and cadefault are deprecated, use a custom context instead.
with urllib.request.urlopen(
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/bI1JG54vSnKw-0ySVorwWA/artifacts/public/extract-lex.tar.zst resolved to 15477 bytes with sha256 103d6febcb9703a26b8515aa9bf20847da9ebefe913733e7bf9fcec82ea22ff4 in 0.211s
Extracting /builds/worker/fetches/extract-lex.tar.zst to /builds/worker/fetches
/builds/worker/fetches/extract-lex.tar.zst extracted in 0.002s
Removing /builds/worker/fetches/extract-lex.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.21579900899999416, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-13T19:59:03.032Z] finished fetching artifacts
[task 2024-02-13T19:59:03.032Z] executing ['vcs/taskcluster/scripts/toolchain/build-extract-lex.sh']
[task 2024-02-13T19:59:03.033Z] + EXTRACT_LEX_DIR=/builds/worker/fetches/extract-lex
[task 2024-02-13T19:59:03.034Z] ++ mktemp -d
[task 2024-02-13T19:59:03.034Z] + build_dir=/tmp/tmp.mDWdz4Oodh
[task 2024-02-13T19:59:03.034Z] + cd /tmp/tmp.mDWdz4Oodh
[task 2024-02-13T19:59:03.034Z] + cmake /builds/worker/fetches/extract-lex
[task 2024-02-13T19:59:03.084Z] -- The C compiler identification is GNU 11.4.0
[task 2024-02-13T19:59:03.136Z] -- The CXX compiler identification is GNU 11.4.0
[task 2024-02-13T19:59:03.144Z] -- Detecting C compiler ABI info
[task 2024-02-13T19:59:03.200Z] -- Detecting C compiler ABI info - done
[task 2024-02-13T19:59:03.208Z] -- Check for working C compiler: /usr/bin/cc - skipped
[task 2024-02-13T19:59:03.208Z] -- Detecting C compile features
[task 2024-02-13T19:59:03.208Z] -- Detecting C compile features - done
[task 2024-02-13T19:59:03.212Z] -- Detecting CXX compiler ABI info
[task 2024-02-13T19:59:03.279Z] -- Detecting CXX compiler ABI info - done
[task 2024-02-13T19:59:03.286Z] -- Check for working CXX compiler: /usr/bin/c++ - skipped
[task 2024-02-13T19:59:03.287Z] -- Detecting CXX compile features
[task 2024-02-13T19:59:03.287Z] -- Detecting CXX compile features - done
[task 2024-02-13T19:59:03.288Z] CMake Deprecation Warning at CMakeLists.txt:2 (cmake_minimum_required):
[task 2024-02-13T19:59:03.288Z] Compatibility with CMake < 2.8.12 will be removed from a future version of
[task 2024-02-13T19:59:03.288Z] CMake.
[task 2024-02-13T19:59:03.288Z]
[task 2024-02-13T19:59:03.288Z] Update the VERSION argument <min> value or use a ...<max> suffix to tell
[task 2024-02-13T19:59:03.288Z] CMake that the project does not need compatibility with older versions.
[task 2024-02-13T19:59:03.288Z]
[task 2024-02-13T19:59:03.288Z]
[task 2024-02-13T19:59:03.295Z] -- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found version "1.74.0") found components: filesystem iostreams
[task 2024-02-13T19:59:03.296Z] -- Configuring done
[task 2024-02-13T19:59:03.298Z] -- Generating done
[task 2024-02-13T19:59:03.299Z] -- Build files have been written to: /tmp/tmp.mDWdz4Oodh
[task 2024-02-13T19:59:03.302Z] ++ nproc
[task 2024-02-13T19:59:03.302Z] + make -j32
[task 2024-02-13T19:59:03.332Z] [ 75%] Building CXX object CMakeFiles/extract_lex.dir/src/utils.cpp.o
[task 2024-02-13T19:59:03.332Z] [ 75%] Building CXX object CMakeFiles/extract_lex.dir/src/extract-lex-main.cpp.o
[task 2024-02-13T19:59:03.332Z] [ 75%] Building CXX object CMakeFiles/extract_lex.dir/src/exception.cpp.o
[task 2024-02-13T19:59:07.292Z] [100%] Linking CXX executable extract_lex
[task 2024-02-13T19:59:07.387Z] [100%] Built target extract_lex
[task 2024-02-13T19:59:07.394Z] + cd /tmp/tmp.mDWdz4Oodh
[task 2024-02-13T19:59:07.394Z] + chmod +x extract_lex
[task 2024-02-13T19:59:07.395Z] + tar --zstd -cf /builds/worker/artifacts/extract_lex.tar.zst extract_lex
[fetches 2024-02-13T19:59:07.414Z] removing /builds/worker/fetches
[fetches 2024-02-13T19:59:07.414Z] finished
[taskcluster 2024-02-13 19:59:07.747Z] === Task Finished ===
[taskcluster 2024-02-13 19:59:07.967Z] Successful task run with exit code: 0 completed in 68.28 seconds
Loading