Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-enable test coverage that the python test suite runner disabled. #12282

Merged
merged 3 commits into from
Nov 26, 2021
Merged

Re-enable test coverage that the python test suite runner disabled. #12282

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
510f97e
Re-enable test coverage that the python test suite runner disabled.
bzbarsky-apple Nov 26, 2021
9f55946
Update the glob for skipping TV apps in darwin tests
andy31415 Nov 26, 2021
385c2eb
make globs lowercase as well, since we match against lowercase names
andy31415 Nov 26, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/tests.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ jobs:
run: |
./scripts/run_in_build_env.sh \
"./scripts/tests/run_test_suite.py \
--target-skip-glob 'tv-*' \
--target-skip-glob 'TV_*' \
run \
--iterations 2 \
--chip-tool ./out/debug/standalone/chip-tool \
Expand Down
10 changes: 4 additions & 6 deletions scripts/tests/chiptest/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,17 +32,15 @@ def AllTests(root: str):
logging.debug('Found YAML: %s' % path)

# grab the name without the extension
name = path.stem.lower()
name = path.stem

if 'simulated' in name:
if 'Simulated' in name:
continue

if name.startswith('tv_'):
if name.startswith('TV_'):
target = TestTarget.TV
name = 'tv-' + name[3:]
elif name.startswith('test_'):
elif name.startswith('Test'):
target = TestTarget.ALL_CLUSTERS
name = 'app-' + name[5:]
else:
continue

Expand Down
27 changes: 20 additions & 7 deletions scripts/tests/run_test_suite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,18 +110,31 @@ def main(context, log_level, target, target_glob, target_skip_glob, no_log_times
coloredlogs.install(level=__LOG_LEVELS__[log_level], fmt=log_fmt)

# Figures out selected test that match the given name(s)
tests = [test for test in chiptest.AllTests(root)]
all_tests = [test for test in chiptest.AllTests(root)]
tests = all_tests
if 'all' not in target:
target = set([name.lower() for name in target])
tests = [test for test in tests if test.name in target]
tests = []
for name in target:
targeted = [test for test in all_tests if test.name.lower()
== name.lower()]
if len(targeted) == 0:
logging.error("Unknown target: %s" % name)
tests.extend(targeted)

if target_glob:
matcher = GlobMatcher(target_glob)
tests = [test for test in tests if matcher.matches(test.name)]
matcher = GlobMatcher(target_glob.lower())
tests = [test for test in tests if matcher.matches(test.name.lower())]

if len(tests) == 0:
logging.error("No targets match, exiting.")
logging.error("Valid targets are (case-insensitive): %s" %
(", ".join(test.name for test in all_tests)))
exit(1)

if target_skip_glob:
matcher = GlobMatcher(target_skip_glob)
tests = [test for test in tests if not matcher.matches(test.name)]
matcher = GlobMatcher(target_skip_glob.lower())
tests = [test for test in tests if not matcher.matches(
test.name.lower())]

tests.sort(key=lambda x: x.name)

Expand Down