Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use hierarchical groups in libSplash #60

Closed
f-schmitt opened this issue Sep 19, 2013 · 12 comments
Closed

Use hierarchical groups in libSplash #60

f-schmitt opened this issue Sep 19, 2013 · 12 comments
Assignees
Labels
component: plugin in PIConGPU plugin

Comments

@f-schmitt
Copy link
Member

Use hierarchical groups to better distinguish particles, fields, ghost cells, indices

Current idea:
fileset 1 (simulation data):

/data/timestamp/...

fields/FieldB/x
fields/FieldB/y

particles/electrons/momentum
particles/electrons/positions
particles/ions/

fileset 2 (simulation restart data):

as above + 
fields/FieldB/_ghosts/top/x
fields/FieldB/_ghosts/bottom/x
@ghost ghost assigned f-schmitt Sep 19, 2013
@f-schmitt
Copy link
Member Author

Will be integrated with or after @psychocoderHPC changes

@f-schmitt
Copy link
Member Author

The group level shouldn't get too deep so it does not affect performance.
Especially h5* tools such as h5list seem to have problems with very deep hierarchies.
I propose the following internal hierarchy below /data/timestamp/:

fields/FieldB/x
fields/FieldB/y
fields/FieldB/ghosts/x
fields/FieldB/ghosts/y

particles/electrons/momentum
particles/electrons/positions
particles/electrons/ghosts/momentum

particles/ions/

@ax3l
Copy link
Member

ax3l commented Jan 24, 2014

👍 that would be perfect.

We always have to keep in mind, that a user just wants to skip the ghosts at all during post-processing (or he/she will ask for them explicitly).
The ghost group will have "holes" in the domain, will that cause huge problems?

@f-schmitt
Copy link
Member Author

I am pretty sure this will trigger some problems, yes. This would be even weird for writing the data, since you would try to combine data from top and bottom gpus (?) into one dataset. Maybe we should just split this once more time in top and bottom _ghosts ?

@psychocoderHPC
Copy link
Member

Do we need ghosts if we implement #114? Than we write the sliding window data and the restart data in a own file.
I am not sure if a physicist need the ghosts outside of the sliding window. But for our implementation all would be more easy without explicit writing out ghosts.

@ax3l
Copy link
Member

ax3l commented Jan 24, 2014

@psychocoderHPC is probably right.

Else, holes could be interpreted as zero-value data transparently, but maybe that is harder to achieve than dividing restart hdf5 and output hdf5 in #114 .

@f-schmitt
Copy link
Member Author

+1, let's write to files so the poor physicists are not confused :)
I updated the issue description.

@ax3l
Copy link
Member

ax3l commented Jan 24, 2014

yeah, let us write to files :P

@f-schmitt
Copy link
Member Author

so it will look like this

hdf5_layout

@f-schmitt
Copy link
Member Author

I agree and we can drop ghosts and write them to a seperate file later on. For now, let's just move over the existing code.

@psychocoderHPC
Copy link
Member

Cool 👍

@ax3l
Copy link
Member

ax3l commented Jan 27, 2014

Nice! 👍

ax3l added a commit that referenced this issue Jan 27, 2014
Use hierarchical groups in libSplash Close #60
@ax3l ax3l closed this as completed Jan 27, 2014
ax3l pushed a commit to ax3l/picongpu that referenced this issue Feb 17, 2015
…gfix-initialize_regions

Regions are now correctly initialized
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: plugin in PIConGPU plugin
Projects
None yet
Development

No branches or pull requests

3 participants