Add local registry with most popular crates from crates.io

[dhovart]

This fixes exercism/rust#1307
This will download to a local registry the top 100 most downloaded crates from crates.io
To regenerate the registry, specify /opt/test-runner/bin/regenerate-registry.sh" as the entrypoint.

[coriolinus]

This looks reasonable to me, thank you! Note: I have not yet attempted this locally.

Assuming this also looks good to @iHiD, I'll test it thoroughly.

[ErikSchierboom]

This is a nice way to approach this! Do you have any indication on the size increase of the Docker image this PR adds?

[ErikSchierboom]

It would be nice if at some later point there would be an error messageif the student uses a non-supported crate.

[dhovart]

Yes, I was thinking about this. I'll think of a way to support this.

[iHiD]

How big are crates? Is it worth doing 1000 instead of 100? Or are they big?

[dhovart]

For the top 100 crates it's just 17MB

[iHiD]

I'd be up for expanding it to the top 500 then? @ErikSchierboom?

[ErikSchierboom]

🤷 It all depends on how likely it is those crates are used. I have no data to back up any claims though. @coriolinus @dhovart how likely is it that students would used a crate that is not in the top 100 crates?

[dhovart]

I have absolutely no idea, sorry! I just know that this list should maybe be fine-tuned. There are crates we pull that make no sense in the context of exercism, such as for gui development.
Maybe there should be a message for students somewhere (below the list of supported crates maybe) that say that if they want support for a given crate, they should open a GH issue?

[dhovart]

Instead of the crates.io API, we would get the list from somewhere else, where it was edited.
I guess I can add an endpoint to exercism's site API. But I'm not familiar with the site codebase yet, is there some administration interface where someone could create this list ? (if my proposal makes sense)

[ErikSchierboom]

I have absolutely no idea, sorry! I just know that this list should maybe be fine-tuned.

I think having a curated list makes the most sense. We definitely don't need to have any GUI crates included, and that would also allow us to cherry-pick crates that are not in the top 100, but still useful for students in the context of Exercism.

Instead of the crates.io API, we would get the list from somewhere else, where it was edited.

We haven't yet come up with a spec for this. I think we want this list to be in the track repo at exercism/rust. Having it there means that we'll be able to programmatically access it from the website. The next question then becomes: where to store it? I think there are basically two options:

1. Add a section to the root config.json file
2. Add a new file just to list the supported libraries

I don't yet have a real preference, but the first option might be best.

[iHiD]

I think it would be fine as MVP to get it just as a file in this repo that's read when Docker builds. That should be quite trivial. Then we can improve it later. In my eyes this is a real priority, so the quickest solution is probably the best one! :)

[ErikSchierboom]

I think it would be fine as MVP to get it just as a file in this repo that's read when Docker builds.

I wouldn't want to touch the existing Docker push workflow (we want to sync that across all repos at some point) and secondly, I'd prefer it to be in exercism/rust as there will be more visibility with maintainers (and students will figure out that location sooner than they will this repo).

[iHiD]

@dhovart Brilliant. Thanks! I'll leave @ErikSchierboom to help get it over the line from a Docker perspective, and @coriolinus to test it all out, but theoretically seems great to me! :)

[iHiD]

I think we need to let users know which crates are available. Is there a way we can reliably do that as the "top x crates" might change between deployment and someone looking at the website if we do ad hoc lookups? Could the GHA write a file when it builds the docker with the list of crates, which gets read by the website. Thoughts?

[ErikSchierboom]

Could the GHA write a file when it builds the docker with the list of crates, which gets read by the website. Thoughts?

I think so. Might have to open a PR instead of writing directly to a file, but such a thing should be possible.

[ErikSchierboom]

Crate	Occurences	Versions
unicode-segmentation	2888	*, 0.1.0, 0.1.2, 0.1.3, 1, 1.0, 1.0.8, 1.1.0, 1.2, 1.2.0, 1.2.1, 1.3, 1.3.0, 1.5, 1.5.0, 1.6, 1.6.0, 1.7, 1.7.0, 1.7.1, 1.8, 1.8.0, ^0, ^1.1.0, ^1.3.0, ^1.6, ^1.6.0, ^1.7, ^1.7.0, ^1.7.1, ~1.2, ~1.7.1
chrono	902	*, 0, 0.2, 0.3, 0.4, 0.4.0, 0.4.10, 0.4.11, 0.4.13, 0.4.15, 0.4.19, 0.4.4, 0.4.6, 0.4.7, 0.4.9, ^0.4
itertools	845	*, 0, 0.10, 0.10.0, 0.10.1, 0.5.9, 0.6.0, 0.6.1, 0.7, 0.7.11, 0.7.2, 0.7.4, 0.7.6, 0.7.7, 0.7.8, 0.7.9, 0.8, 0.8.0, 0.8.1, 0.8.2, 0.9, 0.9.0, ^0.10, ^0.8.0, ^0.9, ~0.9.0
rand	480	*, 0.3, 0.3.12, 0.3.14, 0.4, 0.4.2, 0.4.6, 0.5, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.6, 0.6.0, 0.6.2, 0.6.5, 0.7, 0.7.0, 0.7.2, 0.7.3, 0.8, 0.8.0, 0.8.2, 0.8.3, 0.8.4, ^0.7, ^0.7.3, ~0.7
regex	432	*, 0.1, 0.1.70, 0.1.77, 0.1.80, 0.2, 0.2.11, 0.2.2, 0.2.6, 1, 1.0, 1.0.0, 1.0.1, 1.0.2, 1.1, 1.1.0, 1.1.5, 1.1.6, 1.1.7, 1.1.8, 1.1.9, 1.2, 1.2.0, 1.2.1, 1.3, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.3.6, 1.3.7, 1.3.9, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.5, 1.5.3, 1.5.4, ^1.1.0, ^1.3, ^1.5.4
lazy_static	409	*, 0.2, 0.2.1, 0.2.8, 1, 1.0, 1.1.0, 1.2.0, 1.3, 1.3.0, 1.4, 1.4.0
rayon	220	*, 1, 1.0, 1.0.0, 1.0.2, 1.0.3, 1.1, 1.1.0, 1.2, 1.2.0, 1.2.1, 1.3, 1.3.0, 1.3.1, 1.4, 1.4.0, 1.4.1, 1.5, 1.5.0, 1.5.1, ^1.2
time	216	*, 0.1, 0.1.24, 0.1.37, 0.1.39, 0.1.40, 0.1.42, 0.2.26, ^0.1.36
num	216	*, 0.1, 0.1.29, 0.1.37, 0.1.42, 0.2, 0.2.0, 0.2.1, 0.3, 0.3.0, 0.3.1, 0.4, 0.4.0, ^0.2
unicode-reverse	169	*, 1, 1.0, 1.0.8, ^1.0.8
maplit	115	*, 1, 1.0.1, 1.0.2
crossbeam	110	0.2, 0.4.1, 0.6, 0.7, 0.7.1, 0.7.2, 0.7.3, 0.8, 0.8.0, 0.8.1, ^0.7.2
num-traits	91	*, 0.1.39, 0.1.40, 0.2, 0.2.0, 0.2.10, 0.2.11, 0.2.12, 0.2.14, 0.2.5, 0.2.6, 0.2.8, ~0.2.12
primes	69	*, 0.2.0, 0.2.3, 0.2.4, 0.3, 0.3.0
num-bigint	63	*, 0.1, 0.1.40, 0.2, 0.2.1, 0.2.2, 0.2.3, 0.2.6, 0.3, 0.3.0, 0.3.1, 0.4, 0.4.0, ~0.3.0
modulo	53	0.1, 0.1.2
criterion	53	*, 0.1.2, 0.2, 0.2.10, 0.2.11, 0.3, 0.3.1, 0.3.2, 0.3.3, 0.3.4
num-integer	50	*, 0.1, 0.1.39, 0.1.41, 0.1.42, 0.1.43, 0.1.44
crossbeam-utils	34	*, 0.6, 0.6.5, 0.6.6, 0.7, 0.7.2, 0.8, 0.8.0, 0.8.1, 0.8.3, 0.8.5
bit-vec	31	0.4, 0.5.0, 0.6, 0.6.0, 0.6.1, 0.6.2, 0.6.3
strum	30	0.15.0, 0.16.0, 0.17.1, 0.18.0, 0.19, 0.19.2, 0.20, 0.20.0, 0.21.0
strum_macros	28	0.15.0, 0.16.0, 0.17.1, 0.18.0, 0.19, 0.19.2, 0.20, 0.20.1, 0.21, 0.21.1
primal	26	*, 0.2, 0.2.3, 0.3, 0.3.0, ^0.2
once_cell	26	*, 0.2.6, 1.2, 1.3.1, 1.4.0, 1.5.2, 1.7.2, 1.8, 1.8.0
failure	23	0.1, 0.1.1, 0.1.3, 0.1.5, 0.1.6
counter	20	0.4.3, 0.5, 0.5.2
permutator	17	0.3, 0.3.3, 0.4.0, ^0.4
rstest	15	0.10.0, 0.6, 0.6.4, 0.7.0
crossbeam-channel	15	*, 0.3, 0.4, 0.4.2, 0.5, 0.5.0, 0.5.1
bitflags	15	1.0.3, 1.1.0, 1.2.1
permutohedron	14	0.2, 0.2.4
num-derive	14	0.2, 0.3, 0.3.3
ndarray	14	0.12.1, 0.13, 0.13.1, 0.14.0, 0.15.1
anyhow	14	1.0, 1.0.38
nom	13	5, 5.0.0, 5.0.1, 6, 6.0, 6.1, 6.1.2, ^3.1
threadpool	12	1.7.1, 1.8.1
cached	11	0.11.0, 0.12.0, 0.23.0, 0.8.0
unicase	10	2.6, 2.6.0
phf	10	0.10.0, 0.7.15, 0.7.21, 0.7.24, 0.8.0, 0.9
libmath	10	0.2.1, ^0.1
unicode-normalization	9	0.1, 0.1.16, 0.1.8
rstest_reuse	9	0.1.0, 0.1.3
petgraph	9	0, 0.5, 0.5.0
im	9	15.0.0
hexlit	9	0.3.0, 0.4.0, 0.5.0
enum-iterator	9	*, 0.3.0, 0.5.0, 0.6, 0.6.0
boolinator	9	2.4.0
uuid	8	0.8, 0.8.1, 0.8.2
num_enum	8	0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.5.1
duplicate	8	0.2.2, 0.2.9, 0.3.0
divrem	8	0.1.0
pest	7	0.4, 2, 2.1.3
modinverse	7	0.1.1
dashmap	7	3.1.1, 3.11.10, 3.2.1, 4.0.2
bit-set	7	0.5, 0.5.0, 0.5.1, 0.5.2
bimap	7	0.4.0, 0.5.2
voca_rs	6	1.13.0, 1.9.1, ^1.10
unic-segment	6	0.9, 0.9.0
thiserror	6	*, 1.0, 1.0.18, 1.0.23
multiset	6	0.0.5
kmp	6	0.1.1
gcd	6	1.2.0, 2.0.1, ^2.0.0
fnv	6	1, 1.0, 1.0.3, 1.0.6, 1.0.7
derivative	6	^2.2
ascii	6	0.9, 1.0
unicode_reader	5	1.0.0, 1.0.1
unic-char-range	5	0.9.0
structopt	5	*, 0.3.20, 0.3.21
scan_fmt	5	0.2.4, 0.2.5
pretty_assertions	5	0.6.1
pest_derive	5	2, 2.1.0
multimap	5	0.4.0, 0.8.1, 0.8.2, 0.8.3
try_opt	4	0.1.1
integer-sqrt	4	0.1.2, 0.1.3
int-enum	4	0.4.0
indexmap	4	1.0.2, 1.6.1, 1.7.0
hashers	4	1.0.1
grapheme	4	0.0.0, 1.3.0
factorial	4	0.1.1, 0.2.0, 0.2.1
eval	4	0.4.2, 0.4.3
derive_more	4	0.15.0, 0.99.11, 0.99.14
counted-array	4	0.1.2
case	4	1.0.0
bitvec	4	0.16, 0.19.0, 0.20
arrayvec	4	0, 0.5.1
Inflector	4	0.10.1, 0.11.4
watch	3	0.1.0
stringsort	3	2.0.0
serde	3	1, 1.0, 1.0.0
rusqlite	3	0.13.0, 0.20.0
reikna	3	0.12.3
modexp	3	0.2.2
mod_exp	3	*, 1.0.1
lazysort	3	0.1.1, 0.2, 0.2.1
evalexpr	3	5, 6
enumflags2	3	0.6, 0.6.2
enum_derive	3	0.1.7
custom_derive	3	0.1.7
char_stream	3	*
bytecount	3	0.6.2
bencher	3	0.1.5
unzip-n	2	0.1.1
unicode_segmentation	2	*, 1.7.1
unic-normal	2	0.9.0
transpose	2	0.2.0
test-case	2	1.0.0
subslice	2	0.2, 0.2.2
strfmt	2	~0.1.6
slow_primes	2	0.1.14
serde_derive	2	*, 1.0
say-number	2	1.0.0
rustversion	2	1.0
rs_poker	2	0.6.0, 1.0.0
recur-fn	2	2.1.0
readonly	2	0.2.0
rand_chacha	2	0.3.0
ramp	2	0.5
quickcheck_macros	2	0.8
quickcheck	2	0.8
proptest	2	0.10.1
prime_tools	2	0.2.2, 0.3.4
peg	2	0.6.3
pcre	2	0.2.3
ordered-float	2	0.4.0
onig	2	4.3.3, 5.0.0
num_cpus	2	1.0, 1.13.0
luhn	2
linked_hash_set	2	0.1.4
linked-hash-map	2	0.5.3, 0.5.4
libm	2	0.2.1
inflections	2	1.1.1
indicatif	2	0.15.0
imageproc	2	0.18.0
image	2	0.21.2
if_chain	2	1.0.1
hashbag	2	0.1.3
generational-arena	2	0.2.6
frunk	2	0.3.0
factor	2	0.4
enum-primitive-derive	2	0.1.2
either	2	1.5, 1.5.3
digits_iterator	2	0.1.0
csv	2	1.1
conv	2	0.3.3
compare	2	0.1.0
char-iter	2	0.1.0
caseless	2	0.2
binomial-iter	2	0.1.0
bigint	2	4.4.0
bigdecimal	2	0.1.0
async-std	2	1.6
async-channel	2	1.4
assert_float_eq	2	1
array_tool	2	1.0.0, 1.0.3
z3	1	0.10.0
xvii	1	0.3.4
workerpool	1	1.2
unidecode	1	0.3.0
unicode-case-mapping	1	0.2.0
unic-langid	1	0.9
tramp	1	0.3.0
tinyset	1	*
stringreader	1	0.1.1
static_assertions	1	1.1.0
spinning_top	1	0.2.2
sparse-bitfield	1	0.8.1
sliding_windows	1	3.0.0
simplelog	1	0.8.0
simple_parallel	1	0.3.0
serial_test	1	*
scoped_threadpool	1	0.1.9
scoped-pool	1	0.1.9
rustc-hash	1	1.1.0
rug	1	1.3.0
reduce	1	0.1.4
queues	1	1.0.2
pprof	1	0.3
phf_macros	1	0.7.21
pathfinding	1	2.1
out	1	5.0.0
numtoa	1	0.2.3
num-format	1	0.4
num-digitize	1	0.4
mustache	1	*
memoize	1	*
macro-attr	1	*
log	1	0.4.11
lexical-sort	1	0.3.0
left-pad	1	1.0.1
lalrpop-util	1	0.19.5
itoa	1	0.4.6
indoc	1	*
incremental-topo	1	0.1
humantime	1	2.0.1
hashbrown	1	0.9.1
genawaiter	1	0.99.1
futures-core	1	0.3
free-ranges	1	1.0.7
foreach	1	0.3.0
fluent	1	0.14.1
flamer	1	0.3.0
flame	1	0.2.2
flagset	1	0.3.0
fixedbitset	1	0.1.9
fasteval	1	0.2.4
fancy-regex	1	0.4.1
failure_derive	1	0.1
enumset	1	1.0.4
enum-utils	1	0.1.1
encode_unicode	1	0.3.6
easybench	1	0.1.4
divisors	1	0.1.5
dia-time	1	4.0.0
deunicode	1	1.1.1
defaultmap	1	0.5.0
cranelift-simplejit	1	0.37
cranelift-module	1	0.37
cranelift	1	0.37
cow-utils	1	0.1.2
convert_case	1	0.4.0
const_fn_assert	1	0.1.2
clap	1	2.33.3
chomp	1	0.3.1
chashmap	1	2.2.2
change-case	1	0.2.0
cast	1	*
case_insensitive_hashmap	1	1.0.0
bstr	1	0.2.13
binary-heap-plus	1	0.2.0
as-slice	1	0.2.0
arr_macro	1	0.1.3
alloc_counter	1	0.0.4
ahash	1	^0.7

[ErikSchierboom]

The above comment is an analysis of all the dependencies that were used and their counts and versions.

[dhovart]

I've made a small change to generate the registry from the list of crates above, using cargo search to retrieve their latest versions (see latest commit)

[ErikSchierboom]

It would be great if you could add some tests to verify that exercises that use one of these crates work fine.

Looking at this repo, AFAICT the existing test are all unit tests. While useful, I'd encourage test runner repos to also have some golden (integration) tests, where you have exercises at test data along with an expected results.json file. Running the test runner on those exercises should then produce the exact same results.json file.

For an example of how this works, see:

• https://github.com/exercism/prolog-test-runner/tree/main/tests/example-empty-file
• https://github.com/exercism/prolog-test-runner/tree/main/tests
• https://github.com/exercism/prolog-test-runner/blob/main/bin/run-tests.sh
• https://github.com/exercism/prolog-test-runner/blob/main/bin/run-tests-in-docker.sh

I'd be totally fine with adding tests in a follow-up PR.

Apart from that, this PR looks good to me. I'll leave it up to Rust maintainers (@coriolinus) to confirm that they're happy with this PR.

Great work!

[coriolinus]

I still haven't actually tested this, sorry. Lots going on all at once on my end.

From a Rust perspective this looks both straightforward and acceptable. I just want someone on the maintainer side to run through it once locally, and the time in which I expected to do that myself hasn't materialized. Sorry.

[ErikSchierboom]

I've tried running this on the exemplar solution for the gigasecond exercise.

The command I've run is:

docker run --network none --read-only --mount type=bind,src="/home/erik/exercism/rust/exercises/practice/gigasecond",dst=/solution --mount type=bind,src="/home/erik/exercism/rust/exercises/practice/gigasecond",dst=/output --mount type=tmpfs,dst=/tmp exercism/rust-test-runner "gigasecond" /solution /output

This results in the following error:

WARNING: student did not upload Cargo.lock. This may cause build errors.
   Unpacking autocfg v1.0.1 (registry `/opt/test-runner/local-registry`)
error: failed to download `autocfg v1.0.1`

Caused by:
  unable to get packages from source

Caused by:
  failed to download replaced source registry `crates-io`

Caused by:
  failed to unpack package `autocfg v1.0.1 (registry `/opt/test-runner/local-registry`)`

Caused by:
  failed to create directory `/usr/local/cargo/registry/src/-0177ef2a2962ff86/autocfg-1.0.1`

Caused by:
  Read-only file system (os error 30)

It works fine for hello-world. Could it be that autocfg is a transitive dependency and that the local registry only includes the crates for the direct dependencies?

[dhovart]

I did try with gigasecond (before my most recent changes) and it worked. I'll run some more tests later today or tomorrow if I can find the time.
One need to submit the Cargo.lock file with the correct crates along with the solution for it to work.

[ErikSchierboom]

@dhovart Ah, that might have been the issue then. I don't think the exemplar implementation has one. I'll check tomorrow.

[ErikSchierboom]

@dhovart I've found the issue. This solution doesn't work with a read-only file system. That's not an issue at this moment, as we don't yet run on a read-only file system. We might at some point want to enable this, but we can safely ignore this for now.

This is the output BTW:

Unpacking autocfg v1.0.1 (registry `/opt/test-runner/local-registry`)
   Unpacking chrono v0.4.19 (registry `/opt/test-runner/local-registry`)
   Unpacking libc v0.2.103 (registry `/opt/test-runner/local-registry`)
   Unpacking num-integer v0.1.44 (registry `/opt/test-runner/local-registry`)
   Unpacking num-traits v0.2.14 (registry `/opt/test-runner/local-registry`)
   Unpacking time v0.1.44 (registry `/opt/test-runner/local-registry`)
   Compiling autocfg v1.0.1
   Compiling libc v0.2.103
   Compiling num-traits v0.2.14
   Compiling num-integer v0.1.44
   Compiling time v0.1.44
   Compiling chrono v0.4.19
   Compiling gigasecond v2.0.0 (/solution)
    Finished test [unoptimized + debuginfo] target(s) in 5.44s
     Running unittests (target/debug/deps/gigasecond-24c7f82a404bdcee)
     Running tests/gigasecond.rs (target/debug/deps/gigasecond-bc633be8efe96735)
   Doc-tests gigasecond

I'm noticing that the dependencies seem to be compiled before being able to run the tests. Is there any way in which we can pre-compile all the packages?

[ErikSchierboom]

Thanks a ton @dhovart!

[dhovart]

You're welcome! I'll check if there's a way to use precompiled libraries.

[SleeplessByte]

This is great.

(This comment is just here so I can find it back using GitHub's history, since we probably want to do this for other languages too)

[dhovart]

we probably want to do this for other languages too

I was thinking that one way to generalize this for different languages would be to have packages managed by Nix, but I'm no nix expert.

Maybe something to look into at some later date if someone is willing to.