Create CI

[TriplEight]

@drahnr, please describe here how do you want it.

[drahnr]

sccache already provides quite a bit of unit tests as well as a custom test harness.

So my idea (feel free to improve upon this) is twofold:

Unit Tests

Be able to launch unit tests and the tiny integration tests with the regular cargo test.

Some of the bigger/half-integration tests require docker to launch a container with a subset of binaries created (see DistSystem, only relevant for the sccache-dist server case).

There are two ways to do this:

• move to podman based execution to avoid all the daemon infused hassle (preferred) docker -> podman #9
• make sure the docker daemon is running in the CI env and cleaned properly
• s3 like storage, i.e. minio local server that can be relaunched quickly to assure consistent clean state and re-entrance

Either way container execution (overlay fs, linux namespace caps) need to be avail in the CI.

Integration Tests

Obsolete

Pending the coverage of the dist harness and test cases, we should consider running a defined dummy crate with a certain set of uglinesses (FFI, build.rs, include_bytes!) and verify that the stats recorded by sccache -s --stats-format json after a fresh run with a clean state are always identical (timings excluded).

• s3 like storage, i.e. minio local server that can be relaunched quickly to assure consistent clean state and re-entrance
• deployment config declaration for the 2 services to automatically launch them and fail once one errors out, ansible seems to be the standard here, don't really care if it's a VM or container process
• non-interleaved output collection

Note that this is just a write down from the top of my head, possibly some details pending.

[drahnr]

Having had a few minutes to look into the harness code, it seems that would be sufficient, no need to go to extensive lengths re-implenting this in some typeless scripting language.

[TriplEight]

Shall I introduce -Dwarnings or it's a bit too early for it?

[TriplEight]

After removing openssl it should become easier to assemble the CI env for sccache-dist tests in a new image.

[drahnr]

openssl is removed, and the docker variant within the CI seems to be working just fine, so I'd be in favour of dropping the podman migration for now.

[TriplEight]

Ok, will attempt to assemble a new CI env image. Anything specific that you think should be included there?

[drahnr]

Requirements:

Binaries:

• gcc
• clang
• rustc +stable
• rustc +nightly
• nvcc
• broot (cargo)
• ripgrep (cargo)
• sd (cargo)
• binutils
• coreutils
• gdbserver (or something for remote debugging, not sure if needed)

Libs (including headers and devel):

• miniz
• zlib
• openssl (for unit tests)

In a perfect world we'd have fedora, centos, and ubuntu (the latest each to run the test inside to avoid fallout from users)

[TriplEight]

I see stuff like broot/ripgrep/sd and wonder, are you going to use this env for the development as well?

And also, could you please mention some versions other than clang 11?

[drahnr]

I was more thinking the latest that is available as a base container, so depending on the base container we get common setup profiles, which we test against. I.e. latest fedora with updates and latest ubuntu with updates.

My idea with those tools is mostly to be able to dig into issues that are not reproducible locally, but in the container env.

[TriplEight]

What's the success criterion? I mean, besides obvious "able to run the current CI jobs". What would be the command to test dist?

[drahnr]

cargo test --features=dist-tests,dist-client,dist-server should be fair game and succeed on all platforms.

[TriplEight]

I've built a first CI image version, you can check it out from docker pull tripleight/sccache-ci-ubuntu:nvcc.
Config is here, pls review.

Getting this after running the above command:

running 4 tests
test test_dist_failingserver ... FAILED
test test_dist_nobuilder ... FAILED
test test_dist_basic ... FAILED
test test_dist_restartedserver ... FAILED

failures:

---- test_dist_failingserver stdout ----
thread 'test_dist_failingserver' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', tests/harness/mod.rs:221:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/option.rs:1268:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/result.rs:973:23
   4: dist::harness::DistSystem::new
             at ./tests/harness/mod.rs:215:25
   5: dist::test_dist_failingserver
             at ./tests/dist.rs:203:22
   6: dist::test_dist_failingserver::{{closure}}
             at ./tests/dist.rs:195:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- test_dist_nobuilder stdout ----
thread 'test_dist_nobuilder' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', tests/harness/mod.rs:221:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/option.rs:1268:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/result.rs:973:23
   4: dist::harness::DistSystem::new
             at ./tests/harness/mod.rs:215:25
   5: dist::test_dist_nobuilder
             at ./tests/dist.rs:138:22
   6: dist::test_dist_nobuilder::{{closure}}
             at ./tests/dist.rs:130:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- test_dist_basic stdout ----
thread 'test_dist_basic' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', tests/harness/mod.rs:221:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/option.rs:1268:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/result.rs:973:23
   4: dist::harness::DistSystem::new
             at ./tests/harness/mod.rs:215:25
   5: dist::test_dist_basic
             at ./tests/dist.rs:69:22
   6: dist::test_dist_basic::{{closure}}
             at ./tests/dist.rs:61:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- test_dist_restartedserver stdout ----
thread 'test_dist_restartedserver' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', tests/harness/mod.rs:221:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/std/src/panicking.rs:493:5
   1: core::panicking::panic_fmt
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/panicking.rs:92:14
   2: core::option::expect_none_failed
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/option.rs:1268:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/result.rs:973:23
   4: dist::harness::DistSystem::new
             at ./tests/harness/mod.rs:215:25
   5: dist::test_dist_restartedserver
             at ./tests/dist.rs:102:22
   6: dist::test_dist_restartedserver::{{closure}}
             at ./tests/dist.rs:94:1
   7: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
   8: core::ops::function::FnOnce::call_once
             at /rustc/1c389ffeff814726dec325f0f2b0c99107df2673/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    test_dist_basic
    test_dist_failingserver
    test_dist_nobuilder
    test_dist_restartedserver

test result: FAILED. 0 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--test dist'