Skip to content
This repository has been archived by the owner on Sep 28, 2022. It is now read-only.

Commit

Permalink
[cargo-compare] add some comparison testing to CI
Browse files Browse the repository at this point in the history
This works by:
* setting up a real, fairly complex fixture
* generating random queries using proptest
* running them through cargo and guppy
* ensuring that they produce the same results

Not only does this give us high confidence that cargo
and guppy produce the same results, it also gets us
most of the way to #83.

Most of the work in this commit is simply setting up
all the necessary test infrastructure. We piggy-back on
existing proptest infrastructure as much as possible.
  • Loading branch information
sunshowers committed May 5, 2020
1 parent 679f41e commit d4085f4
Show file tree
Hide file tree
Showing 39 changed files with 593 additions and 51 deletions.
21 changes: 12 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions fixtures/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,7 @@ The fixtures are organized into several folders.

* `small`: relatively simple examples that cover basic and some edge case functionality
* `large`: complex examples pulled from real-world Rust repositories, that test a variety of edge cases
* `invalid`: examples that are *representable* as cargo metadata (i.e. they are valid JSON and follow the general
schema) but are *invalid* in some way; `cargo metadata` should never be able to generate these
* `invalid`: examples that are [*representable*](https://oleb.net/blog/2018/03/making-illegal-states-unrepresentable/)
as cargo metadata (i.e. they are valid JSON and follow the general schema) but are *invalid* in some way; `cargo
metadata` should never be able to generate these
* `workspace`: real workspaces, used for comparison testing with Cargo
2 changes: 2 additions & 0 deletions fixtures/workspace/inside-outside/external/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/target
Cargo.lock
27 changes: 27 additions & 0 deletions fixtures/workspace/inside-outside/external/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "external"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

[dependencies]
transitive = { path = "../transitive" }
bytes = { version = "0.5", optional = true }

[dev-dependencies]
transitive = { path = "../transitive", features = ["dev-feature"] }

[build-dependencies]
transitive = { path = "../transitive", features = ["build-feature"] }

[features]
internal-dev-feature = []
internal-build-feature = []
main-dev-feature = []
main-build-feature = []
macro-normal-feature = []
macro-build-feature = []
macro-dev-feature = []
side-feature = ["bytes"]

[workspace]
7 changes: 7 additions & 0 deletions fixtures/workspace/inside-outside/external/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
2 changes: 2 additions & 0 deletions fixtures/workspace/inside-outside/inactive/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/target
Cargo.lock
21 changes: 21 additions & 0 deletions fixtures/workspace/inside-outside/inactive/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "inactive"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
transitive = { path = "../transitive", features = ["inactive-normal"] }

[build-dependencies]
transitive = { path = "../transitive", features = ["inactive-build"] }

[dev-dependencies]
transitive = { path = "../transitive", features = ["inactive-dev"] }

[features]
extra = ["transitive/extra"]

[workspace]
7 changes: 7 additions & 0 deletions fixtures/workspace/inside-outside/inactive/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
2 changes: 2 additions & 0 deletions fixtures/workspace/inside-outside/transitive/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/target
Cargo.lock
17 changes: 17 additions & 0 deletions fixtures/workspace/inside-outside/transitive/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
[package]
name = "transitive"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[features]
build-feature = []
dev-feature = []
inactive-normal = []
inactive-build = []
inactive-dev = []
extra = []

[workspace]
7 changes: 7 additions & 0 deletions fixtures/workspace/inside-outside/transitive/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
1 change: 1 addition & 0 deletions fixtures/workspace/inside-outside/workspace/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
/target
75 changes: 75 additions & 0 deletions fixtures/workspace/inside-outside/workspace/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions fixtures/workspace/inside-outside/workspace/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[workspace]
members = ["main", "internal", "internal-macro", "side"]
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
[package]
name = "internal-macro"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

[lib]
proc-macro = true

[dependencies]
external = { path = "../../external", features = ["macro-normal-feature"] }

[build-dependencies]
external = { path = "../../external", features = ["macro-build-feature"] }

[dev-dependencies]
external = { path = "../../external", features = ["macro-dev-feature"] }

[features]
main-build-feature = []
internal-normal-feature = []
Empty file.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
24 changes: 24 additions & 0 deletions fixtures/workspace/inside-outside/workspace/internal/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
[package]
name = "internal"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
lazy_static = "1"
external = { path = "../../external" }
internal-macro = { path = "../internal-macro", features = ["internal-normal-feature"] }

[dev-dependencies]
lazy_static = {version = "1", features = ["spin"] }
bytes = "0.5"
external = { path = "../../external", features = ["internal-dev-feature"] }

[build-dependencies]
external = { path = "../../external", features = ["internal-build-feature"] }

[features]
dev-feature = []
build-feature = []
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
27 changes: 27 additions & 0 deletions fixtures/workspace/inside-outside/workspace/main/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
[package]
name = "main"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"
build = "build.rs"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
external = { path = "../../external" }
internal = { path = "../internal" }

[build-dependencies]
external = { path = "../../external", features = ["main-build-feature"] }
internal = { path = "../internal", features = ["build-feature"] }
internal-macro = { path = "../internal-macro", features = ["main-build-feature"] }

[dev-dependencies]
external = { path = "../../external", features = ["main-dev-feature"] }
internal = { path = "../internal", features = ["dev-feature"] }

[target.'cfg(all(unix, not(unix)))'.build-dependencies]
inactive = { path = "../../inactive" }

[features]
inactive-extra = ["inactive/extra"]
2 changes: 2 additions & 0 deletions fixtures/workspace/inside-outside/workspace/main/build.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
fn main() {
}
7 changes: 7 additions & 0 deletions fixtures/workspace/inside-outside/workspace/main/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
10 changes: 10 additions & 0 deletions fixtures/workspace/inside-outside/workspace/side/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[package]
name = "side"
version = "0.1.0"
authors = ["Fake Author <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
external = { path = "../../external", features = ["side-feature"] }
7 changes: 7 additions & 0 deletions fixtures/workspace/inside-outside/workspace/side/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
#[cfg(test)]
mod tests {
#[test]
fn it_works() {
assert_eq!(2 + 2, 4);
}
}
Loading

0 comments on commit d4085f4

Please sign in to comment.