Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature Requst: Add regex support to test filters #53278

Open
TyOverby opened this issue Aug 11, 2018 · 9 comments
Open

Feature Requst: Add regex support to test filters #53278

TyOverby opened this issue Aug 11, 2018 · 9 comments
Labels
A-libtest Area: `#[test]` / the `test` library C-feature-request Category: A feature request, i.e: not implemented / a PR. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.

Comments

@TyOverby
Copy link
Contributor

TyOverby commented Aug 11, 2018 •
edited by rustbot
Loading

Proposal

I'd like to filter my tests based on a more complex pattern than just substring.

The current behavior of rust tests is that when a pattern is specified, then only tests that contain that pattern as a substring are run.

cargo test -- foo will run only tests that contain the substring foo.

I propose to amend this to allow for a regex syntax.

cargo test -- /regex/ which will run tests that match the regex regex.

This is "technically" a breaking change because it's possible that people are already using filters with slashes in them, but since test names can't contain slashes these invocations would always be running 0 tests.

Misc

  • Let me know if this should be an RFC. The feature seems pretty small, but I'd understand if it should be more speced out.
  • I'd be happy to implement this if people are interested.
@estebank estebank added C-feature-request Category: A feature request, i.e: not implemented / a PR. A-libtest Area: `#[test]` / the `test` library labels Jan 11, 2019
@ririsoft
Copy link

Hello,

I came to this issue while trying to run only one test in a test suite with several tests sharing the same base name. Coming from Go I wanted to run a test with foo$ for instance.

However reading this comment on Cargo, I discovered the --exact option that perfectly suite my needs.

While I believe this feature request would be a plus, I recognize that the substring feature is enough for me so far.

@kennytm
Copy link
Member

kennytm commented Sep 24, 2020 •
edited
Loading

This is "technically" a breaking change because it's possible that people are already using filters with slashes in them,

We could make it truly backward-compatible by introducing a --regex flag like the existing --exact flag (#38181)

cargo test -Z unstable-options --regex 'foo$'

One potential issue is that this makes libtest depends on regex.

@kennytm kennytm added the T-libs-api Relevant to the library API team, which will review and decide on the PR/issue. label Sep 24, 2020
jszwedko added a commit to vectordotdev/vector that referenced this issue Oct 17, 2020
@jszwedko
chore(ci): Add transforms-metric_to_log transform dependency to sink-…
1bf6466 
…humio

Given that the humio metrics sink depends on the log_to_metric
transform.

Also fix Makefile to run humio integration tests. I couldn't find a good
way to use a wildcard for the module (like
"::humio::*::integration_tests") so I just did the logs submodule for
now. It seems like there are plans to add regex support for test
filters: rust-lang/rust#53278

Signed-off-by: Jesse Szwedko <[email protected]>
@jszwedko jszwedko mentioned this issue Oct 17, 2020
Merged
jamtur01 pushed a commit to vectordotdev/vector that referenced this issue Oct 17, 2020
@jszwedko
chore(ci): Add transforms-metric_to_log transform dependency to sink-…
8ea0848 
…humio (#4617)

Given that the humio metrics sink depends on the log_to_metric
transform.

Also fix Makefile to run humio integration tests. I couldn't find a good
way to use a wildcard for the module (like
"::humio::*::integration_tests") so I just did the logs submodule for
now. It seems like there are plans to add regex support for test
filters: rust-lang/rust#53278

Signed-off-by: Jesse Szwedko <[email protected]>
juchiast pushed a commit to vectordotdev/vector that referenced this issue Oct 19, 2020
@jszwedko
chore(ci): Add transforms-metric_to_log transform dependency to sink-…
2a686c9 
…humio (#4617)

Given that the humio metrics sink depends on the log_to_metric
transform.

Also fix Makefile to run humio integration tests. I couldn't find a good
way to use a wildcard for the module (like
"::humio::*::integration_tests") so I just did the logs submodule for
now. It seems like there are plans to add regex support for test
filters: rust-lang/rust#53278

Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Duy Do <[email protected]>
@Jasonoro
Copy link

This looks like useful functionality, and I would like to take a stab at trying to implement it. Using a --regex flag seems like a good idea.

@kennytm You said that making libtest depend on regex might be an issue. Do you have any idea on who to ping to find out if this is a big issue or not? The dev guide just has a TODO there, and while the regex crate is approved I'd like to get some clarification on that.

@rustbot claim

mengesb pushed a commit to jacobbraaten/vector that referenced this issue Dec 9, 2020
@jszwedko
chore(ci): Add transforms-metric_to_log transform dependency to sink-…
9fd950b 
…humio (vectordotdev#4617)

Given that the humio metrics sink depends on the log_to_metric
transform.

Also fix Makefile to run humio integration tests. I couldn't find a good
way to use a wildcard for the module (like
"::humio::*::integration_tests") so I just did the logs submodule for
now. It seems like there are plans to add regex support for test
filters: rust-lang/rust#53278

Signed-off-by: Jesse Szwedko <[email protected]>
Signed-off-by: Brian Menges <[email protected]>
@Jasonoro
Copy link

Jasonoro commented Feb 7, 2021

This is currently blocked on rust-lang/regex#476, due to the fact that regex requires std which isn't available when building libtest. See also the following zulipchat conversation.

@Jasonoro
Copy link

Some more discussion has been started in this conversation. I'll update the issue when a conclusion has been reached.

@RalfJung RalfJung mentioned this issue Aug 11, 2022
Closed
@RalfJung
Copy link
Member

/filter/ seems nice because it lets one mix regex and non-regex filters.

due to the fact that regex requires std which isn't available when building libtest

libtest itself depends on std, so it should be okay for its dependencies to depend on std.

@RalfJung
Copy link
Member

RalfJung commented Aug 11, 2022 •
edited
Loading

I think all that needs to happen on the regex side is that it needs to depend on rustc-std-workspace-std; see https://github.com/rust-lang/rust/tree/master/library/rustc-std-workspace-core for how to use that dependency.

This should probably be enabled by a rustc-dep-of-std feature similar to libc.

@RalfJung
Copy link
Member

However, adding a new dependency to the standard library is still a big deal. regex is approved for use in the compiler but that does not mean it can be used in the standard library. I am not sure what the process for that looks like.

@futurist
Copy link

+1, any update for this feature?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-libtest Area: `#[test]` / the `test` library C-feature-request Category: A feature request, i.e: not implemented / a PR. T-libs-api Relevant to the library API team, which will review and decide on the PR/issue.
Projects
testing-devex backlog
Status: No status
Milestone
No milestone
Development

No branches or pull requests
7 participants
@kennytm @futurist @RalfJung @TyOverby @estebank @Jasonoro @ririsoft