Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation in transforms #305

Merged
merged 2 commits into from
Dec 15, 2023

Add basic documentation to most transforms

4aa774c
Select commit
Loading
Failed to load commit list.
Merged

Improve documentation in transforms #305

Add basic documentation to most transforms
4aa774c
Select commit
Loading
Failed to load commit list.
firefoxci-taskcluster / clean-mono-news-crawl-en-news_2007-mono-trg succeeded Dec 15, 2023 in 4m 49s

FirefoxCI (pull_request)

Clean news-crawl news_2007 dataset mono-trg en

Details

View task in Taskcluster
View logs in Taskcluster


[task 2023-12-15T19:03:05.577Z] 117150K .......... .......... .......... .......... .......... 91%  312M 0s
[task 2023-12-15T19:03:05.577Z] 117200K .......... .......... .......... .......... .......... 91%  310M 0s
[task 2023-12-15T19:03:05.577Z] 117250K .......... .......... .......... .......... .......... 91%  283M 0s
[task 2023-12-15T19:03:05.577Z] 117300K .......... .......... .......... .......... .......... 91%  337M 0s
[task 2023-12-15T19:03:05.578Z] 117350K .......... .......... .......... .......... .......... 91%  349M 0s
[task 2023-12-15T19:03:05.578Z] 117400K .......... .......... .......... .......... .......... 91%  327M 0s
[task 2023-12-15T19:03:05.578Z] 117450K .......... .......... .......... .......... .......... 91%  290M 0s
[task 2023-12-15T19:03:05.579Z] 117500K .......... .......... .......... .......... .......... 91% 50.8M 0s
[task 2023-12-15T19:03:05.579Z] 117550K .......... .......... .......... .......... .......... 91%  319M 0s
[task 2023-12-15T19:03:05.579Z] 117600K .......... .......... .......... .......... .......... 91%  315M 0s
[task 2023-12-15T19:03:05.579Z] 117650K .......... .......... .......... .......... .......... 91%  271M 0s
[task 2023-12-15T19:03:05.580Z] 117700K .......... .......... .......... .......... .......... 91%  338M 0s
[task 2023-12-15T19:03:05.580Z] 117750K .......... .......... .......... .......... .......... 91%  155M 0s
[task 2023-12-15T19:03:05.580Z] 117800K .......... .......... .......... .......... .......... 91%  308M 0s
[task 2023-12-15T19:03:05.580Z] 117850K .......... .......... .......... .......... .......... 91%  270M 0s
[task 2023-12-15T19:03:05.580Z] 117900K .......... .......... .......... .......... .......... 92%  325M 0s
[task 2023-12-15T19:03:05.580Z] 117950K .......... .......... .......... .......... .......... 92%  347M 0s
[task 2023-12-15T19:03:05.581Z] 118000K .......... .......... .......... .......... .......... 92%  325M 0s
[task 2023-12-15T19:03:05.581Z] 118050K .......... .......... .......... .......... .......... 92%  268M 0s
[task 2023-12-15T19:03:05.581Z] 118100K .......... .......... .......... .......... .......... 92%  310M 0s
[task 2023-12-15T19:03:05.581Z] 118150K .......... .......... .......... .......... .......... 92%  334M 0s
[task 2023-12-15T19:03:05.581Z] 118200K .......... .......... .......... .......... .......... 92%  333M 0s
[task 2023-12-15T19:03:05.581Z] 118250K .......... .......... .......... .......... .......... 92%  260M 0s
[task 2023-12-15T19:03:05.582Z] 118300K .......... .......... .......... .......... .......... 92%  302M 0s
[task 2023-12-15T19:03:05.582Z] 118350K .......... .......... .......... .......... .......... 92%  335M 0s
[task 2023-12-15T19:03:05.582Z] 118400K .......... .......... .......... .......... .......... 92%  336M 0s
[task 2023-12-15T19:03:05.582Z] 118450K .......... .......... .......... .......... .......... 92%  301M 0s
[task 2023-12-15T19:03:05.582Z] 118500K .......... .......... .......... .......... .......... 92%  339M 0s
[task 2023-12-15T19:03:05.584Z] 118550K .......... .......... .......... .......... .......... 92% 30.1M 0s
[task 2023-12-15T19:03:05.584Z] 118600K .......... .......... .......... .......... .......... 92%  299M 0s
[task 2023-12-15T19:03:05.584Z] 118650K .......... .......... .......... .......... .......... 92%  266M 0s
[task 2023-12-15T19:03:05.584Z] 118700K .......... .......... .......... .......... .......... 92%  339M 0s
[task 2023-12-15T19:03:05.584Z] 118750K .......... .......... .......... .......... .......... 92%  353M 0s
[task 2023-12-15T19:03:05.585Z] 118800K .......... .......... .......... .......... .......... 92%  304M 0s
[task 2023-12-15T19:03:05.585Z] 118850K .......... .......... .......... .......... .......... 92%  275M 0s
[task 2023-12-15T19:03:05.585Z] 118900K .......... .......... .......... .......... .......... 92%  324M 0s
[task 2023-12-15T19:03:05.585Z] 118950K .......... .......... .......... .......... .......... 92%  355M 0s
[task 2023-12-15T19:03:05.585Z] 119000K .......... .......... .......... .......... .......... 92%  346M 0s
[task 2023-12-15T19:03:05.585Z] 119050K .......... .......... .......... .......... .......... 92%  272M 0s
[task 2023-12-15T19:03:05.586Z] 119100K .......... .......... .......... .......... .......... 92%  324M 0s
[task 2023-12-15T19:03:05.586Z] 119150K .......... .......... .......... .......... .......... 92%  330M 0s
[task 2023-12-15T19:03:05.586Z] 119200K .......... .......... .......... .......... .......... 93%  331M 0s
[task 2023-12-15T19:03:05.586Z] 119250K .......... .......... .......... .......... .......... 93%  300M 0s
[task 2023-12-15T19:03:05.586Z] 119300K .......... .......... .......... .......... .......... 93%  321M 0s
[task 2023-12-15T19:03:05.586Z] 119350K .......... .......... .......... .......... .......... 93%  310M 0s
[task 2023-12-15T19:03:05.586Z] 119400K .......... .......... .......... .......... .......... 93%  350M 0s
[task 2023-12-15T19:03:05.587Z] 119450K .......... .......... .......... .......... .......... 93%  297M 0s
[task 2023-12-15T19:03:05.587Z] 119500K .......... .......... .......... .......... .......... 93%  357M 0s
[task 2023-12-15T19:03:05.588Z] 119550K .......... .......... .......... .......... .......... 93% 48.0M 0s
[task 2023-12-15T19:03:05.588Z] 119600K .......... .......... .......... .......... .......... 93%  241M 0s
[task 2023-12-15T19:03:05.588Z] 119650K .......... .......... .......... .......... .......... 93%  291M 0s
[task 2023-12-15T19:03:05.588Z] 119700K .......... .......... .......... .......... .......... 93%  326M 0s
[task 2023-12-15T19:03:05.588Z] 119750K .......... .......... .......... .......... .......... 93%  338M 0s
[task 2023-12-15T19:03:05.589Z] 119800K .......... .......... .......... .......... .......... 93%  311M 0s
[task 2023-12-15T19:03:05.589Z] 119850K .......... .......... .......... .......... .......... 93%  266M 0s
[task 2023-12-15T19:03:05.589Z] 119900K .......... .......... .......... .......... .......... 93%  344M 0s
[task 2023-12-15T19:03:05.589Z] 119950K .......... .......... .......... .......... .......... 93%  322M 0s
[task 2023-12-15T19:03:05.589Z] 120000K .......... .......... .......... .......... .......... 93%  326M 0s
[task 2023-12-15T19:03:05.589Z] 120050K .......... .......... .......... .......... .......... 93%  284M 0s
[task 2023-12-15T19:03:05.590Z] 120100K .......... .......... .......... .......... .......... 93%  321M 0s
[task 2023-12-15T19:03:05.590Z] 120150K .......... .......... .......... .......... .......... 93%  341M 0s
[task 2023-12-15T19:03:05.590Z] 120200K .......... .......... .......... .......... .......... 93%  315M 0s
[task 2023-12-15T19:03:05.590Z] 120250K .......... .......... .......... .......... .......... 93%  258M 0s
[task 2023-12-15T19:03:05.590Z] 120300K .......... .......... .......... .......... .......... 93%  337M 0s
[task 2023-12-15T19:03:05.590Z] 120350K .......... .......... .......... .......... .......... 93%  337M 0s
[task 2023-12-15T19:03:05.590Z] 120400K .......... .......... .......... .......... .......... 93%  332M 0s
[task 2023-12-15T19:03:05.591Z] 120450K .......... .......... .......... .......... .......... 94%  290M 0s
[task 2023-12-15T19:03:05.591Z] 120500K .......... .......... .......... .......... .......... 94%  318M 0s
[task 2023-12-15T19:03:05.591Z] 120550K .......... .......... .......... .......... .......... 94%  325M 0s
[task 2023-12-15T19:03:05.591Z] 120600K .......... .......... .......... .......... .......... 94%  348M 0s
[task 2023-12-15T19:03:05.591Z] 120650K .......... .......... .......... .......... .......... 94%  272M 0s
[task 2023-12-15T19:03:05.591Z] 120700K .......... .......... .......... .......... .......... 94%  307M 0s
[task 2023-12-15T19:03:05.592Z] 120750K .......... .......... .......... .......... .......... 94%  309M 0s
[task 2023-12-15T19:03:05.592Z] 120800K .......... .......... .......... .......... .......... 94%  343M 0s
[task 2023-12-15T19:03:05.592Z] 120850K .......... .......... .......... .......... .......... 94%  313M 0s
[task 2023-12-15T19:03:05.592Z] 120900K .......... .......... .......... .......... .......... 94%  313M 0s
[task 2023-12-15T19:03:05.592Z] 120950K .......... .......... .......... .......... .......... 94%  293M 0s
[task 2023-12-15T19:03:05.592Z] 121000K .......... .......... .......... .......... .......... 94%  224M 0s
[task 2023-12-15T19:03:05.593Z] 121050K .......... .......... .......... .......... .......... 94%  216M 0s
[task 2023-12-15T19:03:05.593Z] 121100K .......... .......... .......... .......... .......... 94%  236M 0s
[task 2023-12-15T19:03:05.593Z] 121150K .......... .......... .......... .......... .......... 94%  244M 0s
[task 2023-12-15T19:03:05.593Z] 121200K .......... .......... .......... .......... .......... 94%  276M 0s
[task 2023-12-15T19:03:05.593Z] 121250K .......... .......... .......... .......... .......... 94%  243M 0s
[task 2023-12-15T19:03:05.594Z] 121300K .......... .......... .......... .......... .......... 94%  258M 0s
[task 2023-12-15T19:03:05.594Z] 121350K .......... .......... .......... .......... .......... 94%  250M 0s
[task 2023-12-15T19:03:05.594Z] 121400K .......... .......... .......... .......... .......... 94%  251M 0s
[task 2023-12-15T19:03:05.594Z] 121450K .......... .......... .......... .......... .......... 94%  222M 0s
[task 2023-12-15T19:03:05.594Z] 121500K .......... .......... .......... .......... .......... 94%  227M 0s
[task 2023-12-15T19:03:05.595Z] 121550K .......... .......... .......... .......... .......... 94%  264M 0s
[task 2023-12-15T19:03:05.595Z] 121600K .......... .......... .......... .......... .......... 94%  241M 0s
[task 2023-12-15T19:03:05.595Z] 121650K .......... .......... .......... .......... .......... 94%  225M 0s
[task 2023-12-15T19:03:05.595Z] 121700K .......... .......... .......... .......... .......... 94%  269M 0s
[task 2023-12-15T19:03:05.595Z] 121750K .......... .......... .......... .......... .......... 95%  234M 0s
[task 2023-12-15T19:03:05.596Z] 121800K .......... .......... .......... .......... .......... 95%  275M 0s
[task 2023-12-15T19:03:05.596Z] 121850K .......... .......... .......... .......... .......... 95%  225M 0s
[task 2023-12-15T19:03:05.596Z] 121900K .......... .......... .......... .......... .......... 95%  257M 0s
[task 2023-12-15T19:03:05.596Z] 121950K .......... .......... .......... .......... .......... 95%  254M 0s
[task 2023-12-15T19:03:05.596Z] 122000K .......... .......... .......... .......... .......... 95%  259M 0s
[task 2023-12-15T19:03:05.597Z] 122050K .......... .......... .......... .......... .......... 95%  241M 0s
[task 2023-12-15T19:03:05.597Z] 122100K .......... .......... .......... .......... .......... 95%  259M 0s
[task 2023-12-15T19:03:05.597Z] 122150K .......... .......... .......... .......... .......... 95%  254M 0s
[task 2023-12-15T19:03:05.597Z] 122200K .......... .......... .......... .......... .......... 95%  260M 0s
[task 2023-12-15T19:03:05.597Z] 122250K .......... .......... .......... .......... .......... 95%  242M 0s
[task 2023-12-15T19:03:05.598Z] 122300K .......... .......... .......... .......... .......... 95%  276M 0s
[task 2023-12-15T19:03:05.598Z] 122350K .......... .......... .......... .......... .......... 95%  262M 0s
[task 2023-12-15T19:03:05.598Z] 122400K .......... .......... .......... .......... .......... 95%  256M 0s
[task 2023-12-15T19:03:05.598Z] 122450K .......... .......... .......... .......... .......... 95%  240M 0s
[task 2023-12-15T19:03:05.598Z] 122500K .......... .......... .......... .......... .......... 95%  266M 0s
[task 2023-12-15T19:03:05.599Z] 122550K .......... .......... .......... .......... .......... 95%  263M 0s
[task 2023-12-15T19:03:05.599Z] 122600K .......... .......... .......... .......... .......... 95%  274M 0s
[task 2023-12-15T19:03:05.599Z] 122650K .......... .......... .......... .......... .......... 95%  233M 0s
[task 2023-12-15T19:03:05.599Z] 122700K .......... .......... .......... .......... .......... 95%  274M 0s
[task 2023-12-15T19:03:05.599Z] 122750K .......... .......... .......... .......... .......... 95%  261M 0s
[task 2023-12-15T19:03:05.599Z] 122800K .......... .......... .......... .......... .......... 95%  275M 0s
[task 2023-12-15T19:03:05.600Z] 122850K .......... .......... .......... .......... .......... 95%  238M 0s
[task 2023-12-15T19:03:05.600Z] 122900K .......... .......... .......... .......... .......... 95%  274M 0s
[task 2023-12-15T19:03:05.600Z] 122950K .......... .......... .......... .......... .......... 95%  272M 0s
[task 2023-12-15T19:03:05.600Z] 123000K .......... .......... .......... .......... .......... 95%  246M 0s
[task 2023-12-15T19:03:05.600Z] 123050K .......... .......... .......... .......... .......... 96%  216M 0s
[task 2023-12-15T19:03:05.601Z] 123100K .......... .......... .......... .......... .......... 96%  270M 0s
[task 2023-12-15T19:03:05.601Z] 123150K .......... .......... .......... .......... .......... 96%  266M 0s
[task 2023-12-15T19:03:05.601Z] 123200K .......... .......... .......... .......... .......... 96%  280M 0s
[task 2023-12-15T19:03:05.601Z] 123250K .......... .......... .......... .......... .......... 96%  239M 0s
[task 2023-12-15T19:03:05.601Z] 123300K .......... .......... .......... .......... .......... 96%  274M 0s
[task 2023-12-15T19:03:05.602Z] 123350K .......... .......... .......... .......... .......... 96%  262M 0s
[task 2023-12-15T19:03:05.602Z] 123400K .......... .......... .......... .......... .......... 96%  243M 0s
[task 2023-12-15T19:03:05.602Z] 123450K .......... .......... .......... .......... .......... 96%  238M 0s
[task 2023-12-15T19:03:05.602Z] 123500K .......... .......... .......... .......... .......... 96%  311M 0s
[task 2023-12-15T19:03:05.602Z] 123550K .......... .......... .......... .......... .......... 96%  289M 0s
[task 2023-12-15T19:03:05.602Z] 123600K .......... .......... .......... .......... .......... 96%  249M 0s
[task 2023-12-15T19:03:05.603Z] 123650K .......... .......... .......... .......... .......... 96%  236M 0s
[task 2023-12-15T19:03:05.603Z] 123700K .......... .......... .......... .......... .......... 96%  301M 0s
[task 2023-12-15T19:03:05.603Z] 123750K .......... .......... .......... .......... .......... 96%  306M 0s
[task 2023-12-15T19:03:05.603Z] 123800K .......... .......... .......... .......... .......... 96%  285M 0s
[task 2023-12-15T19:03:05.603Z] 123850K .......... .......... .......... .......... .......... 96%  246M 0s
[task 2023-12-15T19:03:05.604Z] 123900K .......... .......... .......... .......... .......... 96%  279M 0s
[task 2023-12-15T19:03:05.604Z] 123950K .......... .......... .......... .......... .......... 96%  287M 0s
[task 2023-12-15T19:03:05.604Z] 124000K .......... .......... .......... .......... .......... 96%  262M 0s
[task 2023-12-15T19:03:05.604Z] 124050K .......... .......... .......... .......... .......... 96%  242M 0s
[task 2023-12-15T19:03:05.604Z] 124100K .......... .......... .......... .......... .......... 96%  263M 0s
[task 2023-12-15T19:03:05.604Z] 124150K .......... .......... .......... .......... .......... 96%  275M 0s
[task 2023-12-15T19:03:05.605Z] 124200K .......... .......... .......... .......... .......... 96%  285M 0s
[task 2023-12-15T19:03:05.605Z] 124250K .......... .......... .......... .......... .......... 96%  220M 0s
[task 2023-12-15T19:03:05.605Z] 124300K .......... .......... .......... .......... .......... 97%  272M 0s
[task 2023-12-15T19:03:05.605Z] 124350K .......... .......... .......... .......... .......... 97%  277M 0s
[task 2023-12-15T19:03:05.605Z] 124400K .......... .......... .......... .......... .......... 97%  283M 0s
[task 2023-12-15T19:03:05.606Z] 124450K .......... .......... .......... .......... .......... 97%  241M 0s
[task 2023-12-15T19:03:05.606Z] 124500K .......... .......... .......... .......... .......... 97%  277M 0s
[task 2023-12-15T19:03:05.606Z] 124550K .......... .......... .......... .......... .......... 97%  298M 0s
[task 2023-12-15T19:03:05.606Z] 124600K .......... .......... .......... .......... .......... 97%  268M 0s
[task 2023-12-15T19:03:05.606Z] 124650K .......... .......... .......... .......... .......... 97%  227M 0s
[task 2023-12-15T19:03:05.607Z] 124700K .......... .......... .......... .......... .......... 97%  274M 0s
[task 2023-12-15T19:03:05.607Z] 124750K .......... .......... .......... .......... .......... 97%  281M 0s
[task 2023-12-15T19:03:05.607Z] 124800K .......... .......... .......... .......... .......... 97%  285M 0s
[task 2023-12-15T19:03:05.607Z] 124850K .......... .......... .......... .......... .......... 97%  259M 0s
[task 2023-12-15T19:03:05.607Z] 124900K .......... .......... .......... .......... .......... 97%  287M 0s
[task 2023-12-15T19:03:05.607Z] 124950K .......... .......... .......... .......... .......... 97%  275M 0s
[task 2023-12-15T19:03:05.608Z] 125000K .......... .......... .......... .......... .......... 97%  285M 0s
[task 2023-12-15T19:03:05.608Z] 125050K .......... .......... .......... .......... .......... 97%  244M 0s
[task 2023-12-15T19:03:05.608Z] 125100K .......... .......... .......... .......... .......... 97%  288M 0s
[task 2023-12-15T19:03:05.608Z] 125150K .......... .......... .......... .......... .......... 97%  240M 0s
[task 2023-12-15T19:03:05.608Z] 125200K .......... .......... .......... .......... .......... 97%  260M 0s
[task 2023-12-15T19:03:05.609Z] 125250K .......... .......... .......... .......... .......... 97%  245M 0s
[task 2023-12-15T19:03:05.609Z] 125300K .......... .......... .......... .......... .......... 97%  278M 0s
[task 2023-12-15T19:03:05.609Z] 125350K .......... .......... .......... .......... .......... 97%  277M 0s
[task 2023-12-15T19:03:05.609Z] 125400K .......... .......... .......... .......... .......... 97%  289M 0s
[task 2023-12-15T19:03:05.609Z] 125450K .......... .......... .......... .......... .......... 97%  238M 0s
[task 2023-12-15T19:03:05.609Z] 125500K .......... .......... .......... .......... .......... 97%  271M 0s
[task 2023-12-15T19:03:05.610Z] 125550K .......... .......... .......... .......... .......... 97%  252M 0s
[task 2023-12-15T19:03:05.610Z] 125600K .......... .......... .......... .......... .......... 98%  287M 0s
[task 2023-12-15T19:03:05.610Z] 125650K .......... .......... .......... .......... .......... 98%  248M 0s
[task 2023-12-15T19:03:05.610Z] 125700K .......... .......... .......... .......... .......... 98%  264M 0s
[task 2023-12-15T19:03:05.610Z] 125750K .......... .......... .......... .......... .......... 98%  278M 0s
[task 2023-12-15T19:03:05.611Z] 125800K .......... .......... .......... .......... .......... 98%  297M 0s
[task 2023-12-15T19:03:05.611Z] 125850K .......... .......... .......... .......... .......... 98%  244M 0s
[task 2023-12-15T19:03:05.611Z] 125900K .......... .......... .......... .......... .......... 98%  283M 0s
[task 2023-12-15T19:03:05.611Z] 125950K .......... .......... .......... .......... .......... 98%  291M 0s
[task 2023-12-15T19:03:05.611Z] 126000K .......... .......... .......... .......... .......... 98%  288M 0s
[task 2023-12-15T19:03:05.611Z] 126050K .......... .......... .......... .......... .......... 98%  254M 0s
[task 2023-12-15T19:03:05.612Z] 126100K .......... .......... .......... .......... .......... 98%  299M 0s
[task 2023-12-15T19:03:05.612Z] 126150K .......... .......... .......... .......... .......... 98%  261M 0s
[task 2023-12-15T19:03:05.612Z] 126200K .......... .......... .......... .......... .......... 98%  269M 0s
[task 2023-12-15T19:03:05.612Z] 126250K .......... .......... .......... .......... .......... 98%  257M 0s
[task 2023-12-15T19:03:05.612Z] 126300K .......... .......... .......... .......... .......... 98%  285M 0s
[task 2023-12-15T19:03:05.612Z] 126350K .......... .......... .......... .......... .......... 98%  299M 0s
[task 2023-12-15T19:03:05.613Z] 126400K .......... .......... .......... .......... .......... 98%  295M 0s
[task 2023-12-15T19:03:05.613Z] 126450K .......... .......... .......... .......... .......... 98%  226M 0s
[task 2023-12-15T19:03:05.613Z] 126500K .......... .......... .......... .......... .......... 98%  285M 0s
[task 2023-12-15T19:03:05.613Z] 126550K .......... .......... .......... .......... .......... 98%  315M 0s
[task 2023-12-15T19:03:05.613Z] 126600K .......... .......... .......... .......... .......... 98%  287M 0s
[task 2023-12-15T19:03:05.614Z] 126650K .......... .......... .......... .......... .......... 98%  237M 0s
[task 2023-12-15T19:03:05.614Z] 126700K .......... .......... .......... .......... .......... 98%  280M 0s
[task 2023-12-15T19:03:05.614Z] 126750K .......... .......... .......... .......... .......... 98%  279M 0s
[task 2023-12-15T19:03:05.614Z] 126800K .......... .......... .......... .......... .......... 98%  259M 0s
[task 2023-12-15T19:03:05.614Z] 126850K .......... .......... .......... .......... .......... 98%  251M 0s
[task 2023-12-15T19:03:05.614Z] 126900K .......... .......... .......... .......... .......... 99%  265M 0s
[task 2023-12-15T19:03:05.615Z] 126950K .......... .......... .......... .......... .......... 99%  260M 0s
[task 2023-12-15T19:03:05.615Z] 127000K .......... .......... .......... .......... .......... 99%  258M 0s
[task 2023-12-15T19:03:05.615Z] 127050K .......... .......... .......... .......... .......... 99%  226M 0s
[task 2023-12-15T19:03:05.615Z] 127100K .......... .......... .......... .......... .......... 99%  269M 0s
[task 2023-12-15T19:03:05.615Z] 127150K .......... .......... .......... .......... .......... 99%  281M 0s
[task 2023-12-15T19:03:05.616Z] 127200K .......... .......... .......... .......... .......... 99%  290M 0s
[task 2023-12-15T19:03:05.616Z] 127250K .......... .......... .......... .......... .......... 99%  259M 0s
[task 2023-12-15T19:03:05.616Z] 127300K .......... .......... .......... .......... .......... 99%  254M 0s
[task 2023-12-15T19:03:05.616Z] 127350K .......... .......... .......... .......... .......... 99%  285M 0s
[task 2023-12-15T19:03:05.616Z] 127400K .......... .......... .......... .......... .......... 99%  257M 0s
[task 2023-12-15T19:03:05.617Z] 127450K .......... .......... .......... .......... .......... 99%  245M 0s
[task 2023-12-15T19:03:05.617Z] 127500K .......... .......... .......... .......... .......... 99%  312M 0s
[task 2023-12-15T19:03:05.617Z] 127550K .......... .......... .......... .......... .......... 99%  281M 0s
[task 2023-12-15T19:03:05.617Z] 127600K .......... .......... .......... .......... .......... 99%  302M 0s
[task 2023-12-15T19:03:05.617Z] 127650K .......... .......... .......... .......... .......... 99%  250M 0s
[task 2023-12-15T19:03:05.617Z] 127700K .......... .......... .......... .......... .......... 99%  271M 0s
[task 2023-12-15T19:03:05.618Z] 127750K .......... .......... .......... .......... .......... 99%  280M 0s
[task 2023-12-15T19:03:05.618Z] 127800K .......... .......... .......... .......... .......... 99%  290M 0s
[task 2023-12-15T19:03:05.618Z] 127850K .......... .......... .......... .......... .......... 99%  248M 0s
[task 2023-12-15T19:03:05.618Z] 127900K .......... .......... .......... .......... .......... 99%  282M 0s
[task 2023-12-15T19:03:05.618Z] 127950K .......... .......... .......... .......... .......... 99%  274M 0s
[task 2023-12-15T19:03:05.618Z] 128000K .......... .......... .......... .......... .......... 99%  288M 0s
[task 2023-12-15T19:03:05.619Z] 128050K .......... .......... .......... .......... .......... 99%  259M 0s
[task 2023-12-15T19:03:05.619Z] 128100K .......... .......... .......... .......... .......... 99%  285M 0s
[task 2023-12-15T19:03:05.619Z] 128150K .......... .......... .......... .........            100%  260M=0.6s
[task 2023-12-15T19:03:05.619Z] 
[task 2023-12-15T19:03:05.619Z] 2023-12-15 19:03:05 (210 MB/s) - ‘tools/lid.176.bin’ saved [131266198/131266198]
[task 2023-12-15T19:03:05.619Z] 
[task 2023-12-15T19:03:05.620Z] + zstdmt -dc /builds/worker/artifacts/news_2007.en.monofix.zst
[task 2023-12-15T19:03:05.621Z] + grep -P '^en\t'
[task 2023-12-15T19:03:05.621Z] + cut -f2
[task 2023-12-15T19:03:05.621Z] + zstdmt
[task 2023-12-15T19:03:05.621Z] ++ echo 32/4
[task 2023-12-15T19:03:05.621Z] ++ bc
[task 2023-12-15T19:03:05.622Z] + parallel --no-notice --pipe -k -j 8 --block 50M 'python3 tools/langid_fasttext.py'
[task 2023-12-15T19:03:06.143Z] Warning : `load_model` does not return WordVectorModel or SupervisedModel any more, but a `FastText` object which is very similar.
[task 2023-12-15T19:03:06.159Z] + echo '### Rule-based filtering'
[task 2023-12-15T19:03:06.159Z] ### Rule-based filtering
[task 2023-12-15T19:03:06.159Z] + zstdmt -dc /builds/worker/artifacts/news_2007.en.langid.zst
[task 2023-12-15T19:03:06.159Z] + parallel --no-notice --pipe -k -j 32 --block 50M 'python3 tools/clean_mono.py -l en --debug'
[task 2023-12-15T19:03:06.159Z] + zstdmt
[task 2023-12-15T19:03:06.703Z] + test -s /builds/worker/artifacts/news_2007.en.zst
[task 2023-12-15T19:03:06.703Z] + echo '### Remove data from intermediate steps'
[task 2023-12-15T19:03:06.703Z] ### Remove data from intermediate steps
[task 2023-12-15T19:03:06.703Z] + rm -rf /builds/worker/artifacts/news_2007.en.nrm.zst /builds/worker/artifacts/news_2007.en.langid.zst /builds/worker/artifacts/news_2007.en.monofix.zst
[task 2023-12-15T19:03:06.704Z] + echo '### Clean data is written to  /builds/worker/artifacts/news_2007'
[task 2023-12-15T19:03:06.704Z] ### Clean data is written to  /builds/worker/artifacts/news_2007
[task 2023-12-15T19:03:06.704Z] + echo '###### Done: Cleaning monolingual data'
[task 2023-12-15T19:03:06.704Z] ###### Done: Cleaning monolingual data
[fetches 2023-12-15T19:03:06.704Z] removing /builds/worker/fetches
[fetches 2023-12-15T19:03:06.705Z] finished
[taskcluster 2023-12-15 19:03:07.973Z] === Task Finished ===
[taskcluster 2023-12-15 19:03:08.676Z] Successful task run with exit code: 0 completed in 5.742 seconds