Add the ability to run starting from a specific task (fixes #227) #377
firefoxci-taskcluster / toolchain-preprocess
succeeded
Feb 13, 2024 in 11m 26s
FirefoxCI (push)
preprocess
Details
View task in Taskcluster
View logs in Taskcluster
[taskcluster 2024-02-13 19:57:56.153Z] Task ID: IeWGiM5NRAWLgCYlUo7s1Q
[taskcluster 2024-02-13 19:57:56.153Z] Worker ID: 4501438579064862311
[taskcluster 2024-02-13 19:57:56.153Z] Worker Group: us-central1
[taskcluster 2024-02-13 19:57:56.153Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2024-02-13 19:57:56.153Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2024-02-13 19:57:56.153Z] Worker Version: 38.0.5
[taskcluster 2024-02-13 19:57:56.153Z] Public IP: 34.29.74.66
[taskcluster 2024-02-13 19:57:56.153Z] Hostname: translations-1-b-linux-large-gcp-urozm8kvq-gptr9k4ddjew
[taskcluster 2024-02-13 19:57:56.153Z] using cache "translations-level-1-checkouts-sparse-v3-7afeb851dd97df8f3607-Jew_2ep5Q0SJc7IaKa41Pg" -> /builds/worker/checkouts
[taskcluster 2024-02-13 19:57:56.605Z] Downloading artifact "public/image.tar.zst" from task ID: Jew_2ep5Q0SJc7IaKa41Pg.
[taskcluster 2024-02-13 19:58:01.606Z] Download Progress: 54.59%
[taskcluster 2024-02-13 19:58:04.996Z] Downloaded artifact successfully.
[taskcluster 2024-02-13 19:58:04.996Z] Downloaded 776.726 mb
[taskcluster 2024-02-13 19:58:04.996Z] Decompressing downloaded image
[taskcluster 2024-02-13 19:58:09.960Z] Loading docker image from downloaded archive.
[taskcluster 2024-02-13 19:58:41.666Z] Image 'public/image.tar.zst' from task 'Jew_2ep5Q0SJc7IaKa41Pg' loaded. Using image ID sha256:0bda823ded0479e9bfa01cc4b80273ad1987c66c2b4e567901ee6a02a0583651.
[taskcluster 2024-02-13 19:58:42.007Z] === Task Starting ===
[setup 2024-02-13T19:58:49.966Z] run-task started in /builds/worker
[setup 2024-02-13T19:58:49.966Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --firefox_translations_training-sparse-profile=build/sparse-profiles/toolchain-build --task-cwd /builds/worker/checkouts -- vcs/taskcluster/scripts/toolchain/build-preprocess.sh
[setup 2024-02-13T19:58:49.966Z] Python version: 3.10.12
[cache 2024-02-13T19:58:49.968Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2024-02-13T19:58:49.968Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2024-02-13T19:58:49.968Z] volume /builds/worker/checkouts is a cache
[setup 2024-02-13T19:58:49.968Z] running as worker:worker
[vcs 2024-02-13T19:58:49.968Z] executing ['git', 'config', '--global', '--add', 'safe.directory', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:49.970Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2024-02-13T19:58:49.971Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2024-02-13T19:58:50.642Z] executing ['git', 'fetch', '--tags', '--force', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:50.851Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:50.851Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:50.856Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/mozilla/firefox-translations-training', 'refs/heads/start-specific']
[vcs 2024-02-13T19:58:51.057Z] From https://github.com/mozilla/firefox-translations-training
[vcs 2024-02-13T19:58:51.057Z] * branch start-specific -> FETCH_HEAD
[vcs 2024-02-13T19:58:51.062Z] executing ['git', 'checkout', '-f', '-B', 'refs/heads/start-specific', '37fbf272d7eb316897377144111a3ef057becfd4']
[vcs 2024-02-13T19:58:51.118Z] Switched to a new branch 'refs/heads/start-specific'
[vcs 2024-02-13T19:58:51.119Z] executing ['git', 'submodule', 'init']
[vcs 2024-02-13T19:58:51.137Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2024-02-13T19:58:51.137Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2024-02-13T19:58:51.137Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2024-02-13T19:58:51.137Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2024-02-13T19:58:51.138Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2024-02-13T19:58:51.138Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2024-02-13T19:58:51.138Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2024-02-13T19:58:51.158Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2024-02-13T19:58:52.281Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2024-02-13T19:58:52.568Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2024-02-13T19:58:52.877Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2024-02-13T19:58:53.529Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2024-02-13T19:58:54.823Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2024-02-13T19:58:55.332Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2024-02-13T19:58:55.341Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2024-02-13T19:58:55.351Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2024-02-13T19:58:55.375Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2024-02-13T19:58:55.476Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2024-02-13T19:58:55.493Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2024-02-13T19:58:55.494Z] cleaning git checkout...
[vcs 2024-02-13T19:58:55.494Z] executing ['git', 'clean', '-nxdff']
[vcs 2024-02-13T19:58:55.496Z] removing []
[vcs 2024-02-13T19:58:55.496Z] successfully cleaned git checkout!
[vcs 2024-02-13T19:58:55.498Z] TinderboxPrint:<a href='https://github.com/mozilla/firefox-translations-training/commit/37fbf272d7eb316897377144111a3ef057becfd4' title='Built from firefox-translations-training commit 37fbf272d7eb316897377144111a3ef057becfd4'>37fbf272d7eb316897377144111a3ef057becfd4</a>
[setup 2024-02-13T19:58:55.498Z] MOZ_FETCHES_DIR is /builds/worker/fetches
[setup 2024-02-13T19:58:55.498Z] UPLOAD_DIR is /builds/worker/artifacts
[fetches 2024-02-13T19:58:55.498Z] fetching artifacts
[fetches 2024-02-13T19:58:55.498Z] executing ['/usr/bin/python3', '-u', '/usr/local/bin/fetch-content', 'task-artifacts']
attempt 1/5
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst to /builds/worker/fetches/preprocess.tar.zst
Downloading https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst
/usr/local/bin/fetch-content:192: DeprecationWarning: cafile, capath and cadefault are deprecated, use a custom context instead.
with urllib.request.urlopen(
https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/LDfXeYHYQI2ikpL4OybN4A/artifacts/public/preprocess.tar.zst resolved to 224786 bytes with sha256 6aae2dced65f027bbca8b005533ba32fbd28644c9d1679ca3d37d42a98620863 in 0.197s
Extracting /builds/worker/fetches/preprocess.tar.zst to /builds/worker/fetches
/builds/worker/fetches/preprocess.tar.zst extracted in 0.007s
Removing /builds/worker/fetches/preprocess.tar.zst
PERFHERDER_DATA: {"framework": {"name": "build_metrics"}, "suites": [{"name": "fetch_content", "value": 0.20632764700002326, "lowerIsBetter": true, "shouldAlert": false, "subtests": []}]}
[fetches 2024-02-13T19:58:55.785Z] finished fetching artifacts
[task 2024-02-13T19:58:55.785Z] executing ['vcs/taskcluster/scripts/toolchain/build-preprocess.sh']
[task 2024-02-13T19:58:55.787Z] + PREPROCESS_DIR=/builds/worker/fetches/preprocess
[task 2024-02-13T19:58:55.787Z] ++ mktemp -d
[task 2024-02-13T19:58:55.788Z] + build_dir=/tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:55.788Z] + cd /tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:55.788Z] + cmake /builds/worker/fetches/preprocess -DBUILD_TYPE=Release
[task 2024-02-13T19:58:55.837Z] -- The C compiler identification is GNU 11.4.0
[task 2024-02-13T19:58:55.890Z] -- The CXX compiler identification is GNU 11.4.0
[task 2024-02-13T19:58:55.898Z] -- Detecting C compiler ABI info
[task 2024-02-13T19:58:55.954Z] -- Detecting C compiler ABI info - done
[task 2024-02-13T19:58:55.961Z] -- Check for working C compiler: /usr/bin/cc - skipped
[task 2024-02-13T19:58:55.961Z] -- Detecting C compile features
[task 2024-02-13T19:58:55.962Z] -- Detecting C compile features - done
[task 2024-02-13T19:58:55.965Z] -- Detecting CXX compiler ABI info
[task 2024-02-13T19:58:56.031Z] -- Detecting CXX compiler ABI info - done
[task 2024-02-13T19:58:56.039Z] -- Check for working CXX compiler: /usr/bin/c++ - skipped
[task 2024-02-13T19:58:56.039Z] -- Detecting CXX compile features
[task 2024-02-13T19:58:56.040Z] -- Detecting CXX compile features - done
[task 2024-02-13T19:58:56.046Z] -- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.74.0/BoostConfig.cmake (found suitable version "1.74.0", minimum required is "1.41.0") found components: program_options
[task 2024-02-13T19:58:56.072Z] -- Found ICU: /usr/lib/x86_64-linux-gnu/libicui18n.so;/usr/lib/x86_64-linux-gnu/libicuuc.so;/usr/lib/x86_64-linux-gnu/libicudata.so;/usr/lib/x86_64-linux-gnu/libicuio.so
[task 2024-02-13T19:58:56.079Z] -- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
[task 2024-02-13T19:58:56.081Z] -- Could NOT find BZip2 (missing: BZIP2_LIBRARIES BZIP2_INCLUDE_DIR)
[task 2024-02-13T19:58:56.084Z] -- Could NOT find LibLZMA (missing: LIBLZMA_LIBRARY LIBLZMA_INCLUDE_DIR LIBLZMA_HAS_AUTO_DECODER LIBLZMA_HAS_EASY_ENCODER LIBLZMA_HAS_LZMA_PRESET)
[task 2024-02-13T19:58:56.089Z] -- Configuring done
[task 2024-02-13T19:58:56.123Z] -- Generating done
[task 2024-02-13T19:58:56.125Z] CMake Warning:
[task 2024-02-13T19:58:56.125Z] Manually-specified variables were not used by the project:
[task 2024-02-13T19:58:56.125Z]
[task 2024-02-13T19:58:56.125Z] BUILD_TYPE
[task 2024-02-13T19:58:56.125Z]
[task 2024-02-13T19:58:56.125Z]
[task 2024-02-13T19:58:56.125Z] -- Build files have been written to: /tmp/tmp.aAG8UQxCLs
[task 2024-02-13T19:58:56.130Z] ++ nproc
[task 2024-02-13T19:58:56.130Z] + make -j32
[task 2024-02-13T19:58:56.162Z] [ 1%] Building CXX object preprocess/CMakeFiles/fields.dir/fields.cc.o
[task 2024-02-13T19:58:56.162Z] [ 2%] Building CXX object preprocess/CMakeFiles/base64.dir/base64.cc.o
[task 2024-02-13T19:58:56.163Z] [ 3%] Building CXX object preprocess/CMakeFiles/captive_child.dir/captive_child.cc.o
[task 2024-02-13T19:58:56.163Z] [ 5%] Building CXX object preprocess/CMakeFiles/warc.dir/warc.cc.o
[task 2024-02-13T19:58:56.164Z] [ 6%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/bignum-dtoa.cc.o
[task 2024-02-13T19:58:56.164Z] [ 7%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/bignum.cc.o
[task 2024-02-13T19:58:56.164Z] [ 9%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/double-conversion.cc.o
[task 2024-02-13T19:58:56.164Z] [ 10%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/fixed-dtoa.cc.o
[task 2024-02-13T19:58:56.164Z] [ 11%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/diy-fp.cc.o
[task 2024-02-13T19:58:56.165Z] [ 13%] Building CXX object util/CMakeFiles/preprocess_util.dir/compress.cc.o
[task 2024-02-13T19:58:56.165Z] [ 14%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/cached-powers.cc.o
[task 2024-02-13T19:58:56.165Z] [ 15%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/fast-dtoa.cc.o
[task 2024-02-13T19:58:56.166Z] [ 17%] Building CXX object util/CMakeFiles/preprocess_util.dir/double-conversion/strtod.cc.o
[task 2024-02-13T19:58:56.166Z] [ 18%] Building CXX object util/CMakeFiles/preprocess_util.dir/ersatz_progress.cc.o
[task 2024-02-13T19:58:56.166Z] [ 19%] Building CXX object util/CMakeFiles/preprocess_util.dir/exception.cc.o
[task 2024-02-13T19:58:56.167Z] [ 21%] Building CXX object util/CMakeFiles/preprocess_util.dir/file.cc.o
[task 2024-02-13T19:58:56.167Z] [ 22%] Building CXX object util/CMakeFiles/preprocess_util.dir/file_piece.cc.o
[task 2024-02-13T19:58:56.169Z] [ 25%] Building CXX object util/CMakeFiles/preprocess_util.dir/integer_to_string.cc.o
[task 2024-02-13T19:58:56.169Z] [ 25%] Building CXX object util/CMakeFiles/preprocess_util.dir/float_to_string.cc.o
[task 2024-02-13T19:58:56.171Z] [ 26%] Building CXX object util/CMakeFiles/preprocess_util.dir/mmap.cc.o
[task 2024-02-13T19:58:56.171Z] [ 28%] Building CXX object util/CMakeFiles/preprocess_util.dir/mutable_vocab.cc.o
[task 2024-02-13T19:58:56.171Z] [ 28%] Building CXX object util/CMakeFiles/preprocess_util.dir/murmur_hash.cc.o
[task 2024-02-13T19:58:56.172Z] [ 30%] Building CXX object util/CMakeFiles/preprocess_util.dir/pool.cc.o
[task 2024-02-13T19:58:56.172Z] [ 31%] Building CXX object util/CMakeFiles/preprocess_util.dir/scoped.cc.o
[task 2024-02-13T19:58:56.172Z] [ 34%] Building CXX object util/CMakeFiles/preprocess_util.dir/spaces.cc.o
[task 2024-02-13T19:58:56.172Z] [ 34%] Building CXX object util/CMakeFiles/preprocess_util.dir/string_piece.cc.o
[task 2024-02-13T19:58:56.187Z] [ 35%] Building CXX object util/CMakeFiles/preprocess_util.dir/utf8.cc.o
[task 2024-02-13T19:58:56.454Z] /builds/worker/fetches/preprocess/preprocess/captive_child.cc: In function ‘pid_t preprocess::Launch(char**, util::scoped_fd&, util::scoped_fd&)’:
[task 2024-02-13T19:58:56.454Z] /builds/worker/fetches/preprocess/preprocess/captive_child.cc:57:10: warning: ignoring return value of ‘ssize_t write(int, const void*, size_t)’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
[task 2024-02-13T19:58:56.454Z] 57 | write(status_out.get(), &errno, sizeof(int));
[task 2024-02-13T19:58:56.454Z] | ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[task 2024-02-13T19:58:56.536Z] [ 36%] Linking CXX static library ../lib/libcaptive_child.a
[task 2024-02-13T19:58:56.560Z] [ 36%] Built target captive_child
[task 2024-02-13T19:58:56.582Z] [ 38%] Linking CXX static library ../lib/libfields.a
[task 2024-02-13T19:58:56.606Z] [ 38%] Built target fields
[task 2024-02-13T19:58:56.632Z] [ 39%] Linking CXX static library ../lib/libwarc.a
[task 2024-02-13T19:58:56.656Z] [ 39%] Built target warc
[task 2024-02-13T19:58:56.668Z] [ 40%] Linking CXX static library ../lib/libbase64.a
[task 2024-02-13T19:58:56.694Z] [ 40%] Built target base64
[task 2024-02-13T19:58:56.832Z] [ 42%] Linking CXX static library ../lib/libpreprocess_util.a
[task 2024-02-13T19:58:56.863Z] [ 42%] Built target preprocess_util
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/commoncrawl_dedupe.dir/commoncrawl_dedupe_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/dedupe.dir/dedupe_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 46%] Building CXX object preprocess/CMakeFiles/b64filter.dir/b64filter_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 47%] Building CXX object preprocess/CMakeFiles/foldfilter.dir/foldfilter_main.cc.o
[task 2024-02-13T19:58:56.883Z] [ 48%] Building CXX object preprocess/CMakeFiles/remove_invalid_utf8.dir/remove_invalid_utf8_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 50%] Building CXX object util/CMakeFiles/preprocess_icu.dir/utf8_icu.cc.o
[task 2024-02-13T19:58:56.884Z] [ 51%] Building CXX object preprocess/CMakeFiles/cache.dir/cache_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 52%] Building CXX object preprocess/CMakeFiles/docenc.dir/docenc_main.cc.o
[task 2024-02-13T19:58:56.884Z] [ 59%] Building CXX object preprocess/CMakeFiles/order_independent_hash.dir/order_independent_hash_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/gigaword_unwrap.dir/gigaword_unwrap_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/mmhsum.dir/mmhsum_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/idf.dir/idf_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 59%] Building CXX object preprocess/CMakeFiles/shard.dir/shard_main.cc.o
[task 2024-02-13T19:58:56.885Z] [ 60%] Building CXX object preprocess/CMakeFiles/substitute.dir/substitute_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 61%] Building CXX object preprocess/CMakeFiles/remove_long_lines.dir/remove_long_lines_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 63%] Building CXX object preprocess/CMakeFiles/warc_parallel.dir/warc_parallel_main.cc.o
[task 2024-02-13T19:58:56.886Z] [ 64%] Building CXX object preprocess/CMakeFiles/vocab.dir/vocab_main.cc.o
[task 2024-02-13T19:58:57.173Z] [ 65%] Linking CXX executable ../bin/order_independent_hash
[task 2024-02-13T19:58:57.197Z] [ 67%] Linking CXX executable ../bin/remove_invalid_utf8
[task 2024-02-13T19:58:57.266Z] [ 68%] Linking CXX executable ../bin/mmhsum
[task 2024-02-13T19:58:57.275Z] [ 68%] Built target order_independent_hash
[task 2024-02-13T19:58:57.299Z] [ 68%] Built target remove_invalid_utf8
[task 2024-02-13T19:58:57.359Z] [ 68%] Built target mmhsum
[task 2024-02-13T19:58:57.359Z] [ 69%] Linking CXX executable ../bin/gigaword_unwrap
[task 2024-02-13T19:58:57.424Z] [ 71%] Linking CXX executable ../bin/b64filter
[task 2024-02-13T19:58:57.442Z] [ 72%] Linking CXX executable ../bin/substitute
[task 2024-02-13T19:58:57.465Z] [ 72%] Built target gigaword_unwrap
[task 2024-02-13T19:58:57.493Z] [ 73%] Linking CXX executable ../bin/commoncrawl_dedupe
[task 2024-02-13T19:58:57.516Z] [ 73%] Built target b64filter
[task 2024-02-13T19:58:57.521Z] [ 73%] Built target substitute
[task 2024-02-13T19:58:57.570Z] [ 73%] Built target commoncrawl_dedupe
[task 2024-02-13T19:58:57.593Z] [ 75%] Linking CXX executable ../bin/vocab
[task 2024-02-13T19:58:57.595Z] [ 76%] Linking CXX executable ../bin/docenc
[task 2024-02-13T19:58:57.646Z] [ 77%] Linking CXX executable ../bin/idf
[task 2024-02-13T19:58:57.695Z] [ 77%] Built target docenc
[task 2024-02-13T19:58:57.695Z] [ 77%] Built target vocab
[task 2024-02-13T19:58:57.727Z] [ 77%] Built target idf
[task 2024-02-13T19:58:57.880Z] [ 78%] Linking CXX executable ../bin/remove_long_lines
[task 2024-02-13T19:58:57.958Z] [ 78%] Built target remove_long_lines
[task 2024-02-13T19:58:57.979Z] [ 80%] Linking CXX executable ../bin/foldfilter
[task 2024-02-13T19:58:58.056Z] [ 80%] Built target foldfilter
[task 2024-02-13T19:58:58.071Z] [ 81%] Linking CXX static library ../lib/libpreprocess_icu.a
[task 2024-02-13T19:58:58.097Z] [ 81%] Built target preprocess_icu
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/apply_case.dir/apply_case_main.cc.o
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/process_unicode.dir/process_unicode_main.cc.o
[task 2024-02-13T19:58:58.113Z] [ 85%] Building CXX object preprocess/CMakeFiles/train_case.dir/train_case_main.cc.o
[task 2024-02-13T19:58:58.114Z] [ 86%] Building CXX object preprocess/CMakeFiles/select_latin.dir/select_latin_main.cc.o
[task 2024-02-13T19:58:58.114Z] [ 88%] Building CXX object preprocess/CMakeFiles/truecase.dir/truecase_main.cc.o
[task 2024-02-13T19:58:58.567Z] [ 89%] Linking CXX executable ../bin/select_latin
[task 2024-02-13T19:58:58.655Z] [ 89%] Built target select_latin
[task 2024-02-13T19:58:58.748Z] [ 90%] Linking CXX executable ../bin/truecase
[task 2024-02-13T19:58:58.837Z] [ 90%] Built target truecase
[task 2024-02-13T19:58:59.313Z] [ 92%] Linking CXX executable ../bin/train_case
[task 2024-02-13T19:58:59.329Z] [ 93%] Linking CXX executable ../bin/apply_case
[task 2024-02-13T19:58:59.407Z] [ 93%] Built target train_case
[task 2024-02-13T19:58:59.410Z] [ 94%] Linking CXX executable ../bin/cache
[task 2024-02-13T19:58:59.422Z] [ 94%] Built target apply_case
[task 2024-02-13T19:58:59.490Z] [ 94%] Built target cache
[task 2024-02-13T19:58:59.719Z] [ 96%] Linking CXX executable ../bin/warc_parallel
[task 2024-02-13T19:58:59.731Z] [ 97%] Linking CXX executable ../bin/process_unicode
[task 2024-02-13T19:58:59.793Z] [ 98%] Linking CXX executable ../bin/dedupe
[task 2024-02-13T19:58:59.797Z] [ 98%] Built target warc_parallel
[task 2024-02-13T19:58:59.825Z] [ 98%] Built target process_unicode
[task 2024-02-13T19:58:59.873Z] [ 98%] Built target dedupe
[task 2024-02-13T19:58:59.897Z] [100%] Linking CXX executable ../bin/shard
[task 2024-02-13T19:58:59.978Z] [100%] Built target shard
[task 2024-02-13T19:58:59.988Z] + cd /tmp/tmp.aAG8UQxCLs/bin
[task 2024-02-13T19:58:59.988Z] + chmod +x dedupe
[task 2024-02-13T19:58:59.988Z] + tar --zstd -cf /builds/worker/artifacts/dedupe.tar.zst dedupe
[fetches 2024-02-13T19:58:59.996Z] removing /builds/worker/fetches
[fetches 2024-02-13T19:59:00.002Z] finished
[taskcluster 2024-02-13 19:59:00.311Z] === Task Finished ===
[taskcluster 2024-02-13 19:59:00.525Z] Successful task run with exit code: 0 completed in 64.372 seconds
Loading