Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Brainstorm organization of framework-supported IO packages #1197

Open
Tracked by #1158
makortel opened this issue Jan 28, 2025 · 3 comments
Open
Tracked by #1158

Brainstorm organization of framework-supported IO packages #1197

makortel opened this issue Jan 28, 2025 · 3 comments
Assignees
Labels

Comments

@makortel
Copy link
Collaborator

No description provided.

@makortel
Copy link
Collaborator Author

makortel commented Jan 28, 2025

Presently

IOPool/Common (RootServiceChecker, getWrapperBasePtr(); edmProvDump, edmFileUtil etc executables)
IOPool/Input
IOPool/Output
IOPool/Provenance (for CommonProvenanceFiller.h that is used also in NanoAODOutputModule)
IOPool/SecondaryInput (only tests)
IOPool/Streamer (contains both input and output)
IOPool/TFileAdaptor

Utilities/DCacheAdaptor
Utilities/DavixAdaptor
Utilities/LStoreAdaptor
Utilities/StorageFactory
Utilities/XrdAdaptor

Notes

  • TTree and RNTuple can be opened with bare ROOT, streamer not
    • In a way streamer using ROOT serialization can be thought as an implementation detail, that could change "at any time"
  • Decided organization should naturally extend to hypothetical inclusion of other file formats (PDS, HDF5)
  • Another consideration in addition to storage technology would be maintenance ownership (e.g. FWIO)
  • DataFormats is a layering system (at least to some degree...)
  • We could think of moving FWCore/{Integration,TestModules} outside of FWCore to reduce intertwining between FWCore and IOPool

@makortel makortel moved this from 🔖 Ready to 🏗 In progress in Framework tasks Jan 28, 2025
@makortel
Copy link
Collaborator Author

Outcome

FWIO/Common (from IOPool/Common)

FWIO/PoolInput (from IOPool/Input and IOPool/SecondaryInput)
FWIO/PoolOutput (from IOPool/Output)
FWIO/TTreeProvenance (from IOPool/Provenance)

FWIO/RNTuple

FWIO/Streamer (from IOPool/Streamer)

# hypothetical extension
FWIO/HDF5
FWIO/PDS

FWStorage/TFileAdaptor (from IOPool/TFileAdaptor)
FWStorage/DCacheAdaptor (from Utilities/DCacheAdaptor)
FWStorage/LStoreAdaptor (from Utilities/LStoreAdaptor)
FWStorage/StorageFactory (from Utilies/StorageFactory)
FWStorage/XrdAdaptor (from Utilities/XrdAdaptor)

We probably want to split -lTree out from rootcore into a separate tool, and add similar tool for RNTuple.

@makortel makortel moved this from 🏗 In progress to 👀 In review in Framework tasks Jan 28, 2025
@makortel
Copy link
Collaborator Author

makortel commented Feb 2, 2025

cms-sw/cmssw#45349 suggested renaming IOPool to IOCore, and merging {Common,Input,Output} into TTree. Maybe we follow the result of our brainstorming above nevertheless. The PR moving things around should then close cms-sw/cmssw#45349.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: 👀 In review
Development

No branches or pull requests

3 participants