Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor test_epochs.py::test_split_saving (1 out of 2) #11880

Merged
merged 15 commits into from
Aug 14, 2023

Conversation

dmalt
Copy link
Contributor

@dmalt dmalt commented Aug 14, 2023

Second attempt on PR #11876.

Start with refactoring tests related to saving epochs splits, while keeping the library code intact.

Related issues:
#11870
#7897
#5102

@dmalt dmalt marked this pull request as draft August 14, 2023 17:01
@larsoner
Copy link
Member

ignore pip-pre for now, will be fixed by #11878

"junk": junk,
}
)
epochs.metadata = metadata
epochs.drop_bad()
Copy link
Contributor Author

@dmalt dmalt Aug 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drop_bad() is needed so assert len(epochs) == n_epochs doesn't break. before epochs.get_data() was calling drop_bads() behind the scenes.

@larsoner
Copy link
Member

You can ignore CircleCI for now. To get it to stop being red feel free to add [circle skip] to your commits (but I'm happy enough to ignore-by-eye as well)

@dmalt dmalt force-pushed the refactor-epochs-splits-naming-test branch 3 times, most recently from 07e02c5 to 2b0df72 Compare August 14, 2023 17:45
@dmalt dmalt closed this Aug 14, 2023
@dmalt dmalt reopened this Aug 14, 2023
@dmalt dmalt force-pushed the refactor-epochs-splits-naming-test branch from 3bc5804 to da265fb Compare August 14, 2023 18:31
mne/tests/test_epochs.py Outdated Show resolved Hide resolved
@dmalt dmalt force-pushed the refactor-epochs-splits-naming-test branch from 921ac90 to 3aadc09 Compare August 14, 2023 18:48
Comment on lines +1514 to +1525
@pytest.fixture(
params=[
("1.5MB", 9, True, True, 6),
("1.5MB", 9, True, False, 6),
("1.5MB", 9, False, True, 6),
("1.5MB", 9, False, False, 6),
("3MB", 18, True, True, 3),
("3MB", 18, True, False, 3),
("3MB", 18, False, True, 3),
("3MB", 18, False, False, 3),
]
)
Copy link
Contributor Author

@dmalt dmalt Aug 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parametrisation now includes metadata and concat flags. I do it like this because n_files can potentially depend on these flags and I want to add corner case when these flags increase the number of splits later.

Comment on lines +1570 to +1576
epochs.save(
split_fpath, split_size="1.4MB", split_naming=split_naming, verbose=True
)

# check that the filenames match the intended pattern
assert split_fpath.is_file()
assert (tmp_path / split_fname_part1(n_files)).is_file()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

number of splits with split_size="1.4MB" is completely unrelated to n_files, so checks in this test will break if data generation ever changes.
Also ...part1 in the variable name is misleading and n_files + 1 is wrong. I think it's supposed to check the last split, but the last split is n_files - 1.

split_fname = tmp_path / "test_epo.fif"
split_fname_neuromag_part1 = tmp_path / f"test_epo-{n_files + 1}.fif"
split_fname_bids_part1 = tmp_path / f"test_split-{n_files + 1:02d}_epo.fif"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Part1 is really part_n and the indexing is wrong.

@dmalt
Copy link
Contributor Author

dmalt commented Aug 14, 2023

So far the code should semantically be the same. Next I want to add some new stuff to the tests. @larsoner could you check up till now?

@dmalt dmalt force-pushed the refactor-epochs-splits-naming-test branch from 655c2f8 to ef716f2 Compare August 14, 2023 20:07
@larsoner
Copy link
Member

So far the code should semantically be the same. Next I want to add some new stuff to the tests. @larsoner could you check up till now?

Looks good so far. But please do wait for CIs to come back green before you add anything. Or I could mark for merge-when-green and you could continue in a follow-up PR (which would be a little easier review-wise since the main diff page can be used rather than having to look at single commits)

@dmalt dmalt marked this pull request as ready for review August 14, 2023 21:42
@dmalt
Copy link
Contributor Author

dmalt commented Aug 14, 2023

Ok, let's merge when green then and I'll continue tomorrow in a follow-up PR.

@dmalt dmalt changed the title Refactor test_epochs.py::test_split_saving Refactor test_epochs.py::test_split_saving (1 out of 2) Aug 14, 2023
@larsoner larsoner merged commit 039122a into mne-tools:main Aug 14, 2023
@larsoner
Copy link
Member

Done @dmalt , thanks for taking the time to make things easier to digest !

@dmalt
Copy link
Contributor Author

dmalt commented Aug 15, 2023

No worries! I'm learning a lot. Besides, I like how it's coming out.

larsoner added a commit to larsoner/mne-python that referenced this pull request Aug 15, 2023
* upstream/main:
  Refactor test_epochs.py::test_split_saving (1 out of 2) (mne-tools#11880)
  FIX: Missing Saccade information in Eyelink File (mne-tools#11877)
  Improve drawing of annotations with matplotlib (mne-tools#11855)
  MAINT: Work around NumPy deprecation (mne-tools#11878)
@dmalt dmalt deleted the refactor-epochs-splits-naming-test branch August 16, 2023 07:00
larsoner added a commit to drammock/mne-python that referenced this pull request Aug 22, 2023
* upstream/main:
  [pre-commit.ci] pre-commit autoupdate (mne-tools#11911)
  [BUG, MRG] Remove check on `mne.viz.Brain.add_volume_labels` (mne-tools#11889)
  Small splits fix (mne-tools#11905)
  adds niseq package to "Related software" (mne-tools#11909)
  Minor fixes for ERDS maps example (mne-tools#11904)
  FIX: Fix pyvista rendering (mne-tools#11896)
  BUG: Fix epoch splits naming (mne-tools#11876)
  ENH: Use section-title for HTML anchors in Report (mne-tools#11890)
  CI: Deploy [circle deploy]
  MAINT: Clean up whats_new and doc versions (mne-tools#11888)
  Refactor test_epochs.py::test_split_saving (2 out of 2) (mne-tools#11884)
  Cross-figure event passing system (mne-tools#11685)
  MAINT: Post-release deprecations, updates [circle deploy] (mne-tools#11887)
  MAINT: Release 1.5.0 (mne-tools#11886)
  [pre-commit.ci] pre-commit autoupdate (mne-tools#11883)
  Refactor test_epochs.py::test_split_saving (1 out of 2) (mne-tools#11880)
  FIX: Missing Saccade information in Eyelink File (mne-tools#11877)
  Improve drawing of annotations with matplotlib (mne-tools#11855)
  MAINT: Work around NumPy deprecation (mne-tools#11878)
snwnde pushed a commit to snwnde/mne-python that referenced this pull request Mar 20, 2024
)

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Larson <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants