Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the ability to run starting from a specific task (fixes #227) #377

Merged
merged 2 commits into from
Feb 14, 2024

Update poetry dependencies to pull in newer taskgraph version

37fbf27
Select commit
Loading
Failed to load commit list.
Merged

Add the ability to run starting from a specific task (fixes #227) #377

Update poetry dependencies to pull in newer taskgraph version
37fbf27
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / toolchain-preprocess succeeded Feb 13, 2024 in 11m 26s

FirefoxCI (push)

preprocess

Details

View task in Taskcluster
View logs in Taskcluster


[taskcluster 2024-02-13 19:57:56.153Z] Task ID: IeWGiM5NRAWLgCYlUo7s1Q
[taskcluster 2024-02-13 19:57:56.153Z] Worker ID: 4501438579064862311
[taskcluster 2024-02-13 19:57:56.153Z] Worker Group: us-central1
[taskcluster 2024-02-13 19:57:56.153Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-13 19:57:56.153Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-02-13 19:57:56.153Z] Worker Version: 38.0.5
[taskcluster 2024-02-13 19:57:56.153Z] Public IP: 34.29.74.66
[taskcluster 2024-02-13 19:57:56.153Z] Hostname: translations-1-b-linux-large-gcp-urozm8kvq-gptr9k4ddjew
[taskcluster 2024-02-13 19:57:56.153Z] using cache "translations-level-1-checkouts-sparse-v3-7afeb851dd97df8f3607-Jew_2ep5Q0SJc7IaKa41Pg" -> /builds/worker/checkouts

[taskcluster 2024-02-13 19:57:56.605Z] Downloading artifact "public/image.tar.zst" from task ID: Jew_2ep5Q0SJc7IaKa41Pg.
[taskcluster 2024-02-13 19:58:01.606Z] Download Progress: 54.59%
[taskcluster 2024-02-13 19:58:04.996Z] Downloaded artifact successfully.
[taskcluster 2024-02-13 19:58:04.996Z] Downloaded 776.726 mb
[taskcluster 2024-02-13 19:58:04.996Z] Decompressing downloaded image
[taskcluster 2024-02-13 19:58:09.960Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-13 19:58:41.666Z] Image 'public/image.tar.zst' from task 'Jew_2ep5Q0SJc7IaKa41Pg' loaded.  Using image ID sha256:0bda823ded0479e9bfa01cc4b80273ad1987c66c2b4e567901ee6a02a0583651.
[taskcluster 2024-02-13 19:58:42.007Z] === Task Starting ===
[setup 2024-02-13T19:58:49.966Z] run-task started in /builds/worker
[setup 2024-02-13T19:58:49.966Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --firefox_translations_training-sparse-profile=build/sparse-profiles/toolchain-build --task-cwd /builds/worker/checkouts -- vcs/taskcluster/scripts/toolchain/build-preprocess.sh
[setup 2024-02-13T19:58:49.966Z] Python version: 3.10.12
[cache 2024-02-13T19:58:49.968Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-13T19:58:49.968Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-02-13T19:58:49.968Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-13T19:58:49.968Z] running as worker:worker
[vcs 2024-02-13T19:58:49.968Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:49.970Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:49.971Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-13T19:58:50.642Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:50.851Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:50.851Z]  * branch            start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:50.856Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:51.057Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:51.057Z]  * branch            start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:51.062Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/start-specific', '37fbf272d7eb316897377144111a3ef057becfd4']
[vcs 2024-02-13T19:58:51.118Z] Switched to a new branch 'refs/heads/start-specific'
[vcs 2024-02-13T19:58:51.119Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-13T19:58:51.137Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-13T19:58:51.137Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-13T19:58:51.137Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-13T19:58:51.137Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-13T19:58:51.138Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-13T19:58:51.138Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-13T19:58:51.138Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-13T19:58:51.158Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-13T19:58:52.281Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-13T19:58:52.568Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-13T19:58:52.877Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-13T19:58:53.529Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-13T19:58:54.823Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-13T19:58:55.332Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-13T19:58:55.341Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-13T19:58:55.351Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-13T19:58:55.375Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-13T19:58:55.476Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-13T19:58:55.493Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-13T19:58:55.494Z] cleaning git checkout...
[vcs 2024-02-13T19:58:55.494Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-13T19:58:55.496Z] removing []
[vcs 2024-02-13T19:58:55.496Z] successfully cleaned git checkout!
[vcs 2024-02-13T19:58:55.498Z] TinderboxPrint:<a href='https://github.com/mozilla/firefox-translations-training/commit/37fbf272d7eb316897377144111a3ef057becfd4' title='Built from firefox-translations-training commit 37fbf272d7eb316897377144111a3ef057becfd4'>37fbf272d7eb316897377144111a3ef057becfd4</a>
[setup 2024-02-13T19:58:55.498Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[setup 2024-02-13T19:58:55.498Z] UPLOAD_DIR is /builds/worker/artifacts
[fetches 2024-02-13T19:58:55.498Z] fetching artifacts
[fetches 2024-02-13T19:58:55.498Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst to /builds/worker/fetches/preprocess.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst
/usr/local/bin/fetch-content:192: DeprecationWarning: cafile, capath and cadefault are deprecated, use a custom context instead.
  with urllib.request.urlopen(
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst resolved to 224786 bytes with sha256 6aae2dced65f027bbca8b005533ba32fbd28644c9d1679ca3d37d42a98620863 in 0.197s
Extracting /builds/worker/fetches/preprocess.tar.zst to /builds/worker/fetches
/builds/worker/fetches/preprocess.tar.zst extracted in 0.007s
Removing /builds/worker/fetches/preprocess.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.20632764700002326, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-13T19:58:55.785Z] finished fetching artifacts
[task 2024-02-13T19:58:55.785Z] executing ['vcs/taskcluster/scripts/toolchain/build-preprocess.sh']
[task 2024-02-13T19:58:55.787Z] + PREPROCESS_DIR=/builds/worker/fetches/preprocess
[task 2024-02-13T19:58:55.787Z] ++ mktemp -d
[task 2024-02-13T19:58:55.788Z] + build_dir=/tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:55.788Z] + cd /tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:55.788Z] + cmake /builds/worker/fetches/preprocess -DBUILD_TYPE=Release
[task 2024-02-13T19:58:55.837Z] -- The C compiler identification is GNU 11.4.0
[task 2024-02-13T19:58:55.890Z] -- The CXX compiler identification is GNU 11.4.0
[task 2024-02-13T19:58:55.898Z] -- Detecting C compiler ABI info
[task 2024-02-13T19:58:55.954Z] -- Detecting C compiler ABI info - done
[task 2024-02-13T19:58:55.961Z] -- Check for working C compiler: /usr/bin/cc - skipped
[task 2024-02-13T19:58:55.961Z] -- Detecting C compile features
[task 2024-02-13T19:58:55.962Z] -- Detecting C compile features - done
[task 2024-02-13T19:58:55.965Z] -- Detecting CXX compiler ABI info
[task 2024-02-13T19:58:56.031Z] -- Detecting CXX compiler ABI info - done
[task 2024-02-13T19:58:56.039Z] -- Check for working CXX compiler: /usr/bin/c++ - skipped
[task 2024-02-13T19:58:56.039Z] -- Detecting CXX compile features
[task 2024-02-13T19:58:56.040Z] -- Detecting CXX compile features - done
[task 2024-02-13T19:58:56.046Z] -- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found suitable version "1.74.0", minimum required is "1.41.0") found components: program_options 
[task 2024-02-13T19:58:56.072Z] -- Found ICU: /usr/lib/x86_64-linux-gnu/libicui18n.so;/usr/lib/x86_64-linux-gnu/libicuuc.so;/usr/lib/x86_64-linux-gnu/libicudata.so;/usr/lib/x86_64-linux-gnu/libicuio.so  
[task 2024-02-13T19:58:56.079Z] -- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11") 
[task 2024-02-13T19:58:56.081Z] -- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR) 
[task 2024-02-13T19:58:56.084Z] -- Could NOT find LibLZMA (missing: LIBLZMA_LIBRARY LIBLZMA_INCLUDE_DIR LIBLZMA_HAS_AUTO_DECODER LIBLZMA_HAS_EASY_ENCODER LIBLZMA_HAS_LZMA_PRESET) 
[task 2024-02-13T19:58:56.089Z] -- Configuring done
[task 2024-02-13T19:58:56.123Z] -- Generating done
[task 2024-02-13T19:58:56.125Z] CMake Warning:
[task 2024-02-13T19:58:56.125Z]   Manually-specified variables were not used by the project:
[task 2024-02-13T19:58:56.125Z] 
[task 2024-02-13T19:58:56.125Z]     BUILD_TYPE
[task 2024-02-13T19:58:56.125Z] 
[task 2024-02-13T19:58:56.125Z] 
[task 2024-02-13T19:58:56.125Z] -- Build files have been written to: /tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:56.130Z] ++ nproc
[task 2024-02-13T19:58:56.130Z] + make -j32
[task 2024-02-13T19:58:56.162Z] [  1%] Building CXX object preprocess/CMakeFiles/fields.dir/fields.cc.o
[task 2024-02-13T19:58:56.162Z] [  2%] Building CXX object preprocess/CMakeFiles/base64.dir/base64.cc.o
[task 2024-02-13T19:58:56.163Z] [  3%] Building CXX object preprocess/CMakeFiles/captive_child.dir/captive_child.cc.o
[task 2024-02-13T19:58:56.163Z] [  5%] Building CXX object preprocess/CMakeFiles/warc.dir/warc.cc.o
[task 2024-02-13T19:58:56.164Z] [  6%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/bignum-dtoa.cc.o
[task 2024-02-13T19:58:56.164Z] [  7%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/bignum.cc.o
[task 2024-02-13T19:58:56.164Z] [  9%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/double-conversion.cc.o
[task 2024-02-13T19:58:56.164Z] [ 10%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/fixed-dtoa.cc.o
[task 2024-02-13T19:58:56.164Z] [ 11%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/diy-fp.cc.o
[task 2024-02-13T19:58:56.165Z] [ 13%] Building CXX object util/CMakeFiles/preprocess_util.dir/compress.cc.o
[task 2024-02-13T19:58:56.165Z] [ 14%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/cached-powers.cc.o
[task 2024-02-13T19:58:56.165Z] [ 15%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/fast-dtoa.cc.o
[task 2024-02-13T19:58:56.166Z] [ 17%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/strtod.cc.o
[task 2024-02-13T19:58:56.166Z] [ 18%] Building CXX object util/CMakeFiles/preprocess_util.dir/ersatz_progress.cc.o
[task 2024-02-13T19:58:56.166Z] [ 19%] Building CXX object util/CMakeFiles/preprocess_util.dir/exception.cc.o
[task 2024-02-13T19:58:56.167Z] [ 21%] Building CXX object util/CMakeFiles/preprocess_util.dir/file.cc.o
[task 2024-02-13T19:58:56.167Z] [ 22%] Building CXX object util/CMakeFiles/preprocess_util.dir/file_piece.cc.o
[task 2024-02-13T19:58:56.169Z] [ 25%] Building CXX object util/CMakeFiles/preprocess_util.dir/integer_to_string.cc.o
[task 2024-02-13T19:58:56.169Z] [ 25%] Building CXX object util/CMakeFiles/preprocess_util.dir/float_to_string.cc.o
[task 2024-02-13T19:58:56.171Z] [ 26%] Building CXX object util/CMakeFiles/preprocess_util.dir/mmap.cc.o
[task 2024-02-13T19:58:56.171Z] [ 28%] Building CXX object util/CMakeFiles/preprocess_util.dir/mutable_vocab.cc.o
[task 2024-02-13T19:58:56.171Z] [ 28%] Building CXX object util/CMakeFiles/preprocess_util.dir/murmur_hash.cc.o
[task 2024-02-13T19:58:56.172Z] [ 30%] Building CXX object util/CMakeFiles/preprocess_util.dir/pool.cc.o
[task 2024-02-13T19:58:56.172Z] [ 31%] Building CXX object util/CMakeFiles/preprocess_util.dir/scoped.cc.o
[task 2024-02-13T19:58:56.172Z] [ 34%] Building CXX object util/CMakeFiles/preprocess_util.dir/spaces.cc.o
[task 2024-02-13T19:58:56.172Z] [ 34%] Building CXX object util/CMakeFiles/preprocess_util.dir/string_piece.cc.o
[task 2024-02-13T19:58:56.187Z] [ 35%] Building CXX object util/CMakeFiles/preprocess_util.dir/utf8.cc.o
[task 2024-02-13T19:58:56.454Z] /builds/worker/fetches/preprocess/preprocess/captive_child.cc: In function ‘pid_t preprocess::Launch(char**, util::scoped_fd&, util::scoped_fd&)’:
[task 2024-02-13T19:58:56.454Z] /builds/worker/fetches/preprocess/preprocess/captive_child.cc:57:10: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
[task 2024-02-13T19:58:56.454Z]    57 |     write(status_out.get(), &errno, sizeof(int));
[task 2024-02-13T19:58:56.454Z]       |     ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2024-02-13T19:58:56.536Z] [ 36%] Linking CXX static library ../lib/libcaptive_child.a
[task 2024-02-13T19:58:56.560Z] [ 36%] Built target captive_child
[task 2024-02-13T19:58:56.582Z] [ 38%] Linking CXX static library ../lib/libfields.a
[task 2024-02-13T19:58:56.606Z] [ 38%] Built target fields
[task 2024-02-13T19:58:56.632Z] [ 39%] Linking CXX static library ../lib/libwarc.a
[task 2024-02-13T19:58:56.656Z] [ 39%] Built target warc
[task 2024-02-13T19:58:56.668Z] [ 40%] Linking CXX static library ../lib/libbase64.a
[task 2024-02-13T19:58:56.694Z] [ 40%] Built target base64
[task 2024-02-13T19:58:56.832Z] [ 42%] Linking CXX static library ../lib/libpreprocess_util.a
[task 2024-02-13T19:58:56.863Z] [ 42%] Built target preprocess_util
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/commoncrawl_dedupe.dir/commoncrawl_dedupe_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/dedupe.dir/dedupe_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/b64filter.dir/b64filter_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 47%] Building CXX object preprocess/CMakeFiles/foldfilter.dir/foldfilter_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 48%] Building CXX object preprocess/CMakeFiles/remove_invalid_utf8.dir/remove_invalid_utf8_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 50%] Building CXX object util/CMakeFiles/preprocess_icu.dir/utf8_icu.cc.o
[task 2024-02-13T19:58:56.884Z] [ 51%] Building CXX object preprocess/CMakeFiles/cache.dir/cache_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 52%] Building CXX object preprocess/CMakeFiles/docenc.dir/docenc_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 59%] Building CXX object preprocess/CMakeFiles/order_independent_hash.dir/order_independent_hash_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/gigaword_unwrap.dir/gigaword_unwrap_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/mmhsum.dir/mmhsum_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/idf.dir/idf_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/shard.dir/shard_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 60%] Building CXX object preprocess/CMakeFiles/substitute.dir/substitute_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 61%] Building CXX object preprocess/CMakeFiles/remove_long_lines.dir/remove_long_lines_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 63%] Building CXX object preprocess/CMakeFiles/warc_parallel.dir/warc_parallel_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 64%] Building CXX object preprocess/CMakeFiles/vocab.dir/vocab_main.cc.o
[task 2024-02-13T19:58:57.173Z] [ 65%] Linking CXX executable ../bin/order_independent_hash
[task 2024-02-13T19:58:57.197Z] [ 67%] Linking CXX executable ../bin/remove_invalid_utf8
[task 2024-02-13T19:58:57.266Z] [ 68%] Linking CXX executable ../bin/mmhsum
[task 2024-02-13T19:58:57.275Z] [ 68%] Built target order_independent_hash
[task 2024-02-13T19:58:57.299Z] [ 68%] Built target remove_invalid_utf8
[task 2024-02-13T19:58:57.359Z] [ 68%] Built target mmhsum
[task 2024-02-13T19:58:57.359Z] [ 69%] Linking CXX executable ../bin/gigaword_unwrap
[task 2024-02-13T19:58:57.424Z] [ 71%] Linking CXX executable ../bin/b64filter
[task 2024-02-13T19:58:57.442Z] [ 72%] Linking CXX executable ../bin/substitute
[task 2024-02-13T19:58:57.465Z] [ 72%] Built target gigaword_unwrap
[task 2024-02-13T19:58:57.493Z] [ 73%] Linking CXX executable ../bin/commoncrawl_dedupe
[task 2024-02-13T19:58:57.516Z] [ 73%] Built target b64filter
[task 2024-02-13T19:58:57.521Z] [ 73%] Built target substitute
[task 2024-02-13T19:58:57.570Z] [ 73%] Built target commoncrawl_dedupe
[task 2024-02-13T19:58:57.593Z] [ 75%] Linking CXX executable ../bin/vocab
[task 2024-02-13T19:58:57.595Z] [ 76%] Linking CXX executable ../bin/docenc
[task 2024-02-13T19:58:57.646Z] [ 77%] Linking CXX executable ../bin/idf
[task 2024-02-13T19:58:57.695Z] [ 77%] Built target docenc
[task 2024-02-13T19:58:57.695Z] [ 77%] Built target vocab
[task 2024-02-13T19:58:57.727Z] [ 77%] Built target idf
[task 2024-02-13T19:58:57.880Z] [ 78%] Linking CXX executable ../bin/remove_long_lines
[task 2024-02-13T19:58:57.958Z] [ 78%] Built target remove_long_lines
[task 2024-02-13T19:58:57.979Z] [ 80%] Linking CXX executable ../bin/foldfilter
[task 2024-02-13T19:58:58.056Z] [ 80%] Built target foldfilter
[task 2024-02-13T19:58:58.071Z] [ 81%] Linking CXX static library ../lib/libpreprocess_icu.a
[task 2024-02-13T19:58:58.097Z] [ 81%] Built target preprocess_icu
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/apply_case.dir/apply_case_main.cc.o
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/process_unicode.dir/process_unicode_main.cc.o
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/train_case.dir/train_case_main.cc.o
[task 2024-02-13T19:58:58.114Z] [ 86%] Building CXX object preprocess/CMakeFiles/select_latin.dir/select_latin_main.cc.o
[task 2024-02-13T19:58:58.114Z] [ 88%] Building CXX object preprocess/CMakeFiles/truecase.dir/truecase_main.cc.o
[task 2024-02-13T19:58:58.567Z] [ 89%] Linking CXX executable ../bin/select_latin
[task 2024-02-13T19:58:58.655Z] [ 89%] Built target select_latin
[task 2024-02-13T19:58:58.748Z] [ 90%] Linking CXX executable ../bin/truecase
[task 2024-02-13T19:58:58.837Z] [ 90%] Built target truecase
[task 2024-02-13T19:58:59.313Z] [ 92%] Linking CXX executable ../bin/train_case
[task 2024-02-13T19:58:59.329Z] [ 93%] Linking CXX executable ../bin/apply_case
[task 2024-02-13T19:58:59.407Z] [ 93%] Built target train_case
[task 2024-02-13T19:58:59.410Z] [ 94%] Linking CXX executable ../bin/cache
[task 2024-02-13T19:58:59.422Z] [ 94%] Built target apply_case
[task 2024-02-13T19:58:59.490Z] [ 94%] Built target cache
[task 2024-02-13T19:58:59.719Z] [ 96%] Linking CXX executable ../bin/warc_parallel
[task 2024-02-13T19:58:59.731Z] [ 97%] Linking CXX executable ../bin/process_unicode
[task 2024-02-13T19:58:59.793Z] [ 98%] Linking CXX executable ../bin/dedupe
[task 2024-02-13T19:58:59.797Z] [ 98%] Built target warc_parallel
[task 2024-02-13T19:58:59.825Z] [ 98%] Built target process_unicode
[task 2024-02-13T19:58:59.873Z] [ 98%] Built target dedupe
[task 2024-02-13T19:58:59.897Z] [100%] Linking CXX executable ../bin/shard
[task 2024-02-13T19:58:59.978Z] [100%] Built target shard
[task 2024-02-13T19:58:59.988Z] + cd /tmp/tmp.aAG8UQxCLs/bin
[task 2024-02-13T19:58:59.988Z] + chmod +x dedupe
[task 2024-02-13T19:58:59.988Z] + tar --zstd -cf /builds/worker/artifacts/dedupe.tar.zst dedupe
[fetches 2024-02-13T19:58:59.996Z] removing /builds/worker/fetches
[fetches 2024-02-13T19:59:00.002Z] finished
[taskcluster 2024-02-13 19:59:00.311Z] === Task Finished ===
[taskcluster 2024-02-13 19:59:00.525Z] Successful task run with exit code: 0 completed in 64.372 seconds