Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation in transforms #305

Merged
merged 2 commits into from
Dec 15, 2023

Add basic documentation to most transforms

4aa774c
Select commit
Loading
Failed to load commit list.
Merged

Improve documentation in transforms #305

Add basic documentation to most transforms
4aa774c
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / tests-taskgraph-diff succeeded Dec 15, 2023 in 2m 20s

FirefoxCI (pull_request)

Create diffs of taskgraphs vs. base revision.

Details

View task in Taskcluster
View logs in Taskcluster


[taskcluster 2023-12-15 19:00:08.228Z] Task ID: Lp7o3qhzRsSGoMPk1oLJzw
[taskcluster 2023-12-15 19:00:08.229Z] Worker ID: 4369086099533930297
[taskcluster 2023-12-15 19:00:08.229Z] Worker Group: us-central1
[taskcluster 2023-12-15 19:00:08.229Z] Worker Node Type: projects/887720501152/machineTypes/n2-highmem-32
[taskcluster 2023-12-15 19:00:08.229Z] Worker Pool: translations-1/b-linux-large-gcp
[taskcluster 2023-12-15 19:00:08.229Z] Worker Version: 38.0.5
[taskcluster 2023-12-15 19:00:08.229Z] Public IP: 34.28.226.252
[taskcluster 2023-12-15 19:00:08.229Z] Hostname: translations-1-b-linux-large-gcp-mi8arsjcrvwcl2tguzeeoa
[taskcluster 2023-12-15 19:00:08.229Z] using cache "translations-level-1-checkouts-hg58-v3-2922149087d95338df26" -> /builds/worker/checkouts

[taskcluster 2023-12-15 19:00:11.623Z] Downloading artifact "public/image.tar.zst" from task ID: UaK5100GRL6z4GYFGM8fPA.
[taskcluster 2023-12-15 19:00:13.086Z] Downloaded artifact successfully.
[taskcluster 2023-12-15 19:00:13.086Z] Downloaded 170.191 mb
[taskcluster 2023-12-15 19:00:13.086Z] Decompressing downloaded image
[taskcluster 2023-12-15 19:00:14.403Z] Loading docker image from downloaded archive.
[taskcluster 2023-12-15 19:00:21.204Z] Image 'public/image.tar.zst' from task 'UaK5100GRL6z4GYFGM8fPA' loaded.  Using image ID sha256:6763e055623ff48a8f2aea716efd277ff41032cf9948bac9b78e6b2c19919355.
[taskcluster 2023-12-15 19:00:21.332Z] === Task Starting ===
[setup 2023-12-15T19:00:22.628Z] run-task started in /builds/worker
[setup 2023-12-15T19:00:22.628Z] Invoked by command: --firefox_translations_training-checkout=/builds/worker/checkouts/vcs/ --task-cwd /builds/worker/checkouts/vcs -- bash -cx make diff-taskgraph BASE_REV=1273a0ef3f4a06f4eef9d8493e40a5848662b8de OUTPUT_DIR=/builds/worker/artifacts
[setup 2023-12-15T19:00:22.628Z] Python version: 3.10.12
[cache 2023-12-15T19:00:22.630Z] cache /builds/worker/checkouts is empty; writing requirements: gid=1000 uid=1000 version=1
[volume 2023-12-15T19:00:22.630Z] changing ownership of volume /builds/worker/.cache to 1000:1000
[volume 2023-12-15T19:00:22.630Z] volume /builds/worker/checkouts is a cache
[setup 2023-12-15T19:00:22.631Z] running as worker:worker
[vcs 2023-12-15T19:00:22.631Z] executing ['git', 'clone', 'https://github.com/mozilla/firefox-translations-training', '/builds/worker/checkouts/vcs']
[vcs 2023-12-15T19:00:22.632Z] Cloning into '/builds/worker/checkouts/vcs'...
[vcs 2023-12-15T19:00:23.528Z] executing ['git', 'fetch', '--no-tags', 'https://github.com/bhearsum/firefox-translations-training', 'followup']
[vcs 2023-12-15T19:00:23.815Z] From https://github.com/bhearsum/firefox-translations-training
[vcs 2023-12-15T19:00:23.815Z]  * branch            followup   -> FETCH_HEAD
[vcs 2023-12-15T19:00:23.817Z] executing ['git', 'checkout', '-f', '-B', 'followup', '4aa774c2a9f31b93627bcac82e0c235839cb7c0e']
[vcs 2023-12-15T19:00:23.854Z] Switched to a new branch 'followup'
[vcs 2023-12-15T19:00:23.855Z] executing ['git', 'submodule', 'init']
[vcs 2023-12-15T19:00:23.875Z] Submodule '3rd_party/browsermt-marian-dev' (https://github.com/browsermt/marian-dev) registered for path '3rd_party/browsermt-marian-dev'
[vcs 2023-12-15T19:00:23.875Z] Submodule 'extract-lex' (https://github.com/marian-nmt/extract-lex) registered for path '3rd_party/extract-lex'
[vcs 2023-12-15T19:00:23.876Z] Submodule 'fast_align' (https://github.com/clab/fast_align) registered for path '3rd_party/fast_align'
[vcs 2023-12-15T19:00:23.876Z] Submodule '3rd_party/kenlm' (https://github.com/kpu/kenlm) registered for path '3rd_party/kenlm'
[vcs 2023-12-15T19:00:23.877Z] Submodule '3rd_party/marian-dev' (https://github.com/marian-nmt/marian-dev) registered for path '3rd_party/marian-dev'
[vcs 2023-12-15T19:00:23.877Z] Submodule '3rd_party/preprocess' (https://github.com/kpu/preprocess.git) registered for path '3rd_party/preprocess'
[vcs 2023-12-15T19:00:23.877Z] executing ['git', 'submodule', 'update', '--force']
[vcs 2023-12-15T19:00:23.899Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/browsermt-marian-dev'...
[vcs 2023-12-15T19:00:26.972Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/extract-lex'...
[vcs 2023-12-15T19:00:27.300Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/fast_align'...
[vcs 2023-12-15T19:00:27.654Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/kenlm'...
[vcs 2023-12-15T19:00:28.903Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/marian-dev'...
[vcs 2023-12-15T19:00:30.675Z] Cloning into '/builds/worker/checkouts/vcs/3rd_party/preprocess'...
[vcs 2023-12-15T19:00:31.381Z] Submodule path '3rd_party/browsermt-marian-dev': checked out '11c6ae7c46be21ef96ed10c60f28022fa968939f'
[vcs 2023-12-15T19:00:31.391Z] Submodule path '3rd_party/extract-lex': checked out '42fa605b53f32eaf6c6e0b5677255c21c91b3d49'
[vcs 2023-12-15T19:00:31.401Z] Submodule path '3rd_party/fast_align': checked out 'cab1e9aac8d3bb02ff5ae58218d8d225a039fa11'
[vcs 2023-12-15T19:00:31.428Z] Submodule path '3rd_party/kenlm': checked out 'bbf4fc511266c5d4515047055d7bdec659a6e158'
[vcs 2023-12-15T19:00:31.539Z] Submodule path '3rd_party/marian-dev': checked out 'e8a1a2530fb84cbff7383302ebca393e5875c441'
[vcs 2023-12-15T19:00:31.558Z] Submodule path '3rd_party/preprocess': checked out '64307314b4d5a9a0bd529b5c1036b0710d995eec'
[vcs 2023-12-15T19:00:31.559Z] cleaning git checkout...
[vcs 2023-12-15T19:00:31.559Z] executing ['git', 'clean', '-nxdff']
[vcs 2023-12-15T19:00:31.561Z] removing []
[vcs 2023-12-15T19:00:31.561Z] successfully cleaned git checkout!
[vcs 2023-12-15T19:00:31.563Z] TinderboxPrint:<a href='https://github.com/bhearsum/firefox-translations-training/commit/4aa774c2a9f31b93627bcac82e0c235839cb7c0e' title='Built from firefox-translations-training commit 4aa774c2a9f31b93627bcac82e0c235839cb7c0e'>4aa774c2a9f31b93627bcac82e0c235839cb7c0e</a>
[task 2023-12-15T19:00:31.563Z] executing ['bash', '-cx', 'make diff-taskgraph BASE_REV=1273a0ef3f4a06f4eef9d8493e40a5848662b8de OUTPUT_DIR=/builds/worker/artifacts']
[task 2023-12-15T19:00:31.564Z] + make diff-taskgraph BASE_REV=1273a0ef3f4a06f4eef9d8493e40a5848662b8de OUTPUT_DIR=/builds/worker/artifacts
[task 2023-12-15T19:00:31.565Z] pip3 install -r taskcluster/requirements.txt
[task 2023-12-15T19:00:31.565Z] taskgraph full -p "taskcluster/test/params" -o "/builds/worker/artifacts" --diff "1273a0ef3f4a06f4eef9d8493e40a5848662b8de" -J
[task 2023-12-15T19:00:31.920Z] Defaulting to user installation because normal site-packages is not writeable
[task 2023-12-15T19:00:32.095Z] Collecting appdirs==1.4.4
[task 2023-12-15T19:00:32.270Z]   Downloading appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
[task 2023-12-15T19:00:32.316Z] Collecting arrow==1.2.3
[task 2023-12-15T19:00:32.350Z]   Downloading arrow-1.2.3-py3-none-any.whl (66 kB)
[task 2023-12-15T19:00:32.396Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.4/66.4 KB 1.5 MB/s eta 0:00:00
[task 2023-12-15T19:00:32.418Z] Collecting binaryornot==0.4.4
[task 2023-12-15T19:00:32.453Z]   Downloading binaryornot-0.4.4-py2.py3-none-any.whl (9.0 kB)
[task 2023-12-15T19:00:32.490Z] Collecting certifi==2022.12.7
[task 2023-12-15T19:00:32.526Z]   Downloading certifi-2022.12.7-py3-none-any.whl (155 kB)
[task 2023-12-15T19:00:32.565Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 155.3/155.3 KB 3.9 MB/s eta 0:00:00
[task 2023-12-15T19:00:32.586Z] Collecting chardet==5.1.0
[task 2023-12-15T19:00:32.633Z]   Downloading chardet-5.1.0-py3-none-any.whl (199 kB)
[task 2023-12-15T19:00:32.647Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 199.1/199.1 KB 16.8 MB/s eta 0:00:00
[task 2023-12-15T19:00:32.836Z] Collecting charset-normalizer==3.0.1
[task 2023-12-15T19:00:32.872Z]   Downloading charset_normalizer-3.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (198 kB)
[task 2023-12-15T19:00:32.881Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 198.8/198.8 KB 26.9 MB/s eta 0:00:00
[task 2023-12-15T19:00:32.919Z] Collecting click==8.1.3
[task 2023-12-15T19:00:32.954Z]   Downloading click-8.1.3-py3-none-any.whl (96 kB)
[task 2023-12-15T19:00:32.959Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 96.6/96.6 KB 29.3 MB/s eta 0:00:00
[task 2023-12-15T19:00:32.988Z] Collecting cookiecutter==2.1.1
[task 2023-12-15T19:00:33.024Z]   Downloading cookiecutter-2.1.1-py2.py3-none-any.whl (36 kB)
[task 2023-12-15T19:00:33.045Z] Collecting giturlparse==0.10.0
[task 2023-12-15T19:00:33.080Z]   Downloading giturlparse-0.10.0-py2.py3-none-any.whl (14 kB)
[task 2023-12-15T19:00:33.106Z] Collecting idna==3.4
[task 2023-12-15T19:00:33.140Z]   Downloading idna-3.4-py3-none-any.whl (61 kB)
[task 2023-12-15T19:00:33.144Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.5/61.5 KB 32.4 MB/s eta 0:00:00
[task 2023-12-15T19:00:33.175Z] Collecting jinja2==3.1.2
[task 2023-12-15T19:00:33.210Z]   Downloading Jinja2-3.1.2-py3-none-any.whl (133 kB)
[task 2023-12-15T19:00:33.217Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 133.1/133.1 KB 29.0 MB/s eta 0:00:00
[task 2023-12-15T19:00:33.235Z] Collecting jinja2-time==0.2.0
[task 2023-12-15T19:00:33.278Z]   Downloading jinja2_time-0.2.0-py2.py3-none-any.whl (6.4 kB)
[task 2023-12-15T19:00:33.302Z] Collecting json-e==4.5.0
[task 2023-12-15T19:00:33.337Z]   Downloading json_e-4.5.0-py3-none-any.whl (13 kB)
[task 2023-12-15T19:00:33.436Z] Collecting markupsafe==2.1.2
[task 2023-12-15T19:00:33.472Z]   Downloading MarkupSafe-2.1.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
[task 2023-12-15T19:00:33.491Z] Collecting mozilla-repo-urls==0.1.1
[task 2023-12-15T19:00:33.526Z]   Downloading mozilla_repo_urls-0.1.1-py3-none-any.whl (6.2 kB)
[task 2023-12-15T19:00:33.554Z] Collecting python-dateutil==2.8.2
[task 2023-12-15T19:00:33.589Z]   Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
[task 2023-12-15T19:00:33.601Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 247.7/247.7 KB 24.8 MB/s eta 0:00:00
[task 2023-12-15T19:00:33.629Z] Collecting python-slugify==8.0.1
[task 2023-12-15T19:00:33.664Z]   Downloading python_slugify-8.0.1-py2.py3-none-any.whl (9.7 kB)
[task 2023-12-15T19:00:33.747Z] Collecting pyyaml==6.0
[task 2023-12-15T19:00:33.782Z]   Downloading PyYAML-6.0-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (682 kB)
[task 2023-12-15T19:00:33.822Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 682.2/682.2 KB 17.7 MB/s eta 0:00:00
[task 2023-12-15T19:00:33.844Z] Collecting redo==2.0.4
[task 2023-12-15T19:00:33.879Z]   Downloading redo-2.0.4-py2.py3-none-any.whl (12 kB)
[task 2023-12-15T19:00:33.935Z] Collecting requests==2.28.2
[task 2023-12-15T19:00:33.970Z]   Downloading requests-2.28.2-py3-none-any.whl (62 kB)
[task 2023-12-15T19:00:33.974Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.8/62.8 KB 30.4 MB/s eta 0:00:00
[task 2023-12-15T19:00:33.991Z] Collecting requests-unixsocket==0.3.0
[task 2023-12-15T19:00:34.026Z]   Downloading requests_unixsocket-0.3.0-py2.py3-none-any.whl (11 kB)
[task 2023-12-15T19:00:34.053Z] Collecting six==1.16.0
[task 2023-12-15T19:00:34.088Z]   Downloading six-1.16.0-py2.py3-none-any.whl (11 kB)
[task 2023-12-15T19:00:34.107Z] Collecting slugid==2.0.0
[task 2023-12-15T19:00:34.142Z]   Downloading slugid-2.0.0-py2.py3-none-any.whl (8.2 kB)
[task 2023-12-15T19:00:34.179Z] Collecting taskcluster-taskgraph==6.2.0
[task 2023-12-15T19:00:34.213Z]   Downloading taskcluster_taskgraph-6.2.0-py3-none-any.whl (187 kB)
[task 2023-12-15T19:00:34.218Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 188.0/188.0 KB 76.0 MB/s eta 0:00:00
[task 2023-12-15T19:00:34.257Z] Collecting taskcluster-urls==13.0.1
[task 2023-12-15T19:00:34.292Z]   Downloading taskcluster_urls-13.0.1-py3-none-any.whl (10 kB)
[task 2023-12-15T19:00:34.310Z] Collecting text-unidecode==1.3
[task 2023-12-15T19:00:34.345Z]   Downloading text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
[task 2023-12-15T19:00:34.349Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 78.2/78.2 KB 38.6 MB/s eta 0:00:00
[task 2023-12-15T19:00:34.410Z] Collecting urllib3==1.26.14
[task 2023-12-15T19:00:34.444Z]   Downloading urllib3-1.26.14-py2.py3-none-any.whl (140 kB)
[task 2023-12-15T19:00:34.449Z]      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 KB 64.1 MB/s eta 0:00:00
[task 2023-12-15T19:00:34.475Z] Collecting voluptuous==0.13.1
[task 2023-12-15T19:00:34.509Z]   Downloading voluptuous-0.13.1-py3-none-any.whl (29 kB)
[task 2023-12-15T19:00:34.772Z] Installing collected packages: voluptuous, text-unidecode, taskcluster-urls, slugid, redo, json-e, charset-normalizer, appdirs, urllib3, six, pyyaml, python-slugify, markupsafe, idna, giturlparse, click, chardet, certifi, requests, python-dateutil, mozilla-repo-urls, jinja2, binaryornot, requests-unixsocket, arrow, jinja2-time, cookiecutter, taskcluster-taskgraph
[task 2023-12-15T19:00:35.632Z] Successfully installed appdirs-1.4.4 arrow-1.2.3 binaryornot-0.4.4 certifi-2022.12.7 chardet-5.1.0 charset-normalizer-3.0.1 click-8.1.3 cookiecutter-2.1.1 giturlparse-0.10.0 idna-3.4 jinja2-3.1.2 jinja2-time-0.2.0 json-e-4.5.0 markupsafe-2.1.2 mozilla-repo-urls-0.1.1 python-dateutil-2.8.2 python-slugify-8.0.1 pyyaml-6.0 redo-2.0.4 requests-2.28.2 requests-unixsocket-0.3.0 six-1.16.0 slugid-2.0.0 taskcluster-taskgraph-6.2.0 taskcluster-urls-13.0.1 text-unidecode-1.3 urllib3-1.26.14 voluptuous-0.13.1
[task 2023-12-15T19:00:36.004Z] Generating full_task_graph @ followup
[task 2023-12-15T19:00:37.393Z] Note: switching to '1273a0ef3f4a06f4eef9d8493e40a5848662b8de'.
[task 2023-12-15T19:00:37.393Z] 
[task 2023-12-15T19:00:37.393Z] You are in 'detached HEAD' state. You can look around, make experimental
[task 2023-12-15T19:00:37.393Z] changes and commit them, and you can discard any commits you make in this
[task 2023-12-15T19:00:37.393Z] state without impacting any branches by switching back to a branch.
[task 2023-12-15T19:00:37.393Z] 
[task 2023-12-15T19:00:37.394Z] If you want to create a new branch to retain commits you create, you may
[task 2023-12-15T19:00:37.394Z] do so (now or later) by using -c with the switch command. Example:
[task 2023-12-15T19:00:37.394Z] 
[task 2023-12-15T19:00:37.394Z]   git switch -c <new-branch-name>
[task 2023-12-15T19:00:37.394Z] 
[task 2023-12-15T19:00:37.394Z] Or undo this operation with:
[task 2023-12-15T19:00:37.394Z] 
[task 2023-12-15T19:00:37.394Z]   git switch -
[task 2023-12-15T19:00:37.394Z] 
[task 2023-12-15T19:00:37.394Z] Turn off this advice by setting config variable advice.detachedHead to false
[task 2023-12-15T19:00:37.394Z] 
[task 2023-12-15T19:00:37.394Z] HEAD is now at 1273a0e Fix the docs building by using the remote theme (#310)
[task 2023-12-15T19:00:37.395Z] Generating full_task_graph @ 1273a0ef3f4a
[task 2023-12-15T19:00:38.787Z] Previous HEAD position was 1273a0e Fix the docs building by using the remote theme (#310)
[task 2023-12-15T19:00:38.788Z] Switched to branch 'followup'
[task 2023-12-15T19:00:38.796Z] Dumping result with parameters from large-lt-en:
[task 2023-12-15T19:00:38.798Z] Dumping result with parameters from small-ru-en:
[task 2023-12-15T19:00:38.798Z] See '/builds/worker/.cache/taskgraph/log/vcs' for logs
[task 2023-12-15T19:00:38.798Z] Files full_task_graph@1273a0ef3f4a and full_task_graph@followup are identical
[task 2023-12-15T19:00:38.798Z] 
[task 2023-12-15T19:00:38.798Z] 
[task 2023-12-15T19:00:38.798Z] Files full_task_graph@1273a0ef3f4a and full_task_graph@followup are identical
[task 2023-12-15T19:00:38.798Z] 
[task 2023-12-15T19:00:38.798Z] 
[taskcluster 2023-12-15 19:00:39.186Z] === Task Finished ===
[taskcluster 2023-12-15 19:00:39.186Z] Successful task run with exit code: 0 completed in 30.959 seconds