Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a TFile cache for RootEmbeddedFileSequence #263

Open
Tracked by #1144
dan131riley opened this issue Sep 9, 2021 · 1 comment
Open
Tracked by #1144

Implement a TFile cache for RootEmbeddedFileSequence #263

dan131riley opened this issue Sep 9, 2021 · 1 comment

Comments

@dan131riley
Copy link
Collaborator

In replay mode, RootEmbeddedFileSequence tends to do a lot of TFile thrashing opening PU files because thread assignments don't match, so a sequence that followed one thread/TFile in the previous step gets randomly assigned to threads in the following step. Caching the LFN-hash to LFN mapping would help some, caching the LFN-hash to TFile* would help a lot more.

I commented on this when I made the MixingModule threaded, but at the time it didn't seem like a problem. With more threads it looks like it is becoming more of an issue, and could be high impact for some specific WFs, especially HI production.

@dan131riley
Copy link
Collaborator Author

This is related to, but not dependent on #126 (and should be much simpler than the general TTreeCache usage)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Paused
Development

No branches or pull requests

2 participants