Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clone euxfel data file structure #467

Merged
merged 9 commits into from
Dec 5, 2023
Merged

Clone euxfel data file structure #467

merged 9 commits into from
Dec 5, 2023

Conversation

tmichela
Copy link
Member

Add a little tool to generate EuXFEL data files by copying existing run data, but without any of the original data in the resulting files.

I made this originaly to build test for the exdf-tools as this is critical to test on real-like data from various cycles, but I thought this would be useful more generally, hence it ends up here.

… of real data files. The resulting files do not hold any of the original data, only the group structure and dataset type/shape.
Copy link
Member

@takluyver takluyver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more comment: naming!

I'd move this out of tests, since it's useful for things beyond EXtra-data's own tests.

gendata doesn't really describe what it does - it's not generating anything, and the output doesn't have most of the data. 😉 What about something like these options:

  • extra_data.mkskeleton
  • extra_data.mkemptyfiles
  • extra_data.hollowfiles
  • extra_data.stripfiles

extra_data/tests/utils/gendata.py Outdated Show resolved Hide resolved
extra_data/tests/utils/gendata.py Outdated Show resolved Hide resolved
extra_data/tests/utils/gendata.py Outdated Show resolved Hide resolved
extra_data/tests/utils/gendata.py Outdated Show resolved Hide resolved
replicate softlinks to new file

...
@tmichela
Copy link
Member Author

I'd move this out of tests

Fine with that, I though it is niche enough to hide it there, but 🤷

One more comment: naming!

yea, I gave it purposely a terrible name to get ideas back 😁
I don't have any particular preference, so if you have a favourite?

Maybe also to consider:
clone_empty
blanc_copy
empty_replica

@takluyver
Copy link
Member

Or copy_structure, to give yet another option. I think that might be my favourite so far, but I don't have a strong preference.

@takluyver
Copy link
Member

extra_data.copy_structure is still the name I like most so far, but I'd be OK with any of our suggestions if you're developing a preference.

Other than that, LGTM

extra_data/copy.py Fixed Show fixed Hide fixed
extra_data/tests/test_copy.py Fixed Show fixed Hide fixed
@tmichela
Copy link
Member Author

So I added support for soft/hard links anyway, if you want to have an other look at it.
(+ some bonus unit tests)

extra_data/copy.py Outdated Show resolved Hide resolved
@takluyver takluyver added this to the 1.16 milestone Dec 4, 2023
@takluyver takluyver added the enhancement New feature or request label Dec 4, 2023
@takluyver
Copy link
Member

Thanks, LGTM 👍

@tmichela
Copy link
Member Author

tmichela commented Dec 5, 2023

thanks for the review! :)

@tmichela tmichela merged commit 9acd861 into master Dec 5, 2023
8 checks passed
@takluyver
Copy link
Member

prnote: Add a development tool to copy the structure of EuXFEL data files without the data.

@takluyver takluyver deleted the cloneDataStructure branch February 26, 2024 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants