Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use TTaskGroup interface to unzip baskets in parallel. #1010

Merged
merged 6 commits into from
Feb 20, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add more explanations
zzxuanyuan committed Jan 26, 2018

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit d7a0560eb28950edd2c1d5466ca5337719655681
21 changes: 12 additions & 9 deletions tree/tree/src/TTreeCacheUnzip.cxx
Original file line number Diff line number Diff line change
@@ -100,15 +100,18 @@ Bool_t TTreeCacheUnzip::UnzipState::IsFinished(Int_t index) const {
}

////////////////////////////////////////////////////////////////////////////////
/// Check if the basket is unzipped already.
/// Check if the basket is unzipped already. We must make sure the length in
/// fUnzipLen is larger than 0.

Bool_t TTreeCacheUnzip::UnzipState::IsUnzipped(Int_t index) const {
return (fUnzipStatus[index].load() == kFinished) && (fUnzipChunks[index].get()) && (fUnzipLen[index] > 0);
}

////////////////////////////////////////////////////////////////////////////////
/// Reset all baskets' state arrays. This function is only called by main thread.
/// Other threads should not call this function since it is not thread-safe.
/// Reset all baskets' state arrays. This function is only called by main
/// thread and parallel processing from upper layers should be disabled such
/// as IMT in TTree::GetEntry(). Other threads should not call this function
/// since it is not thread-safe.

void TTreeCacheUnzip::UnzipState::Reset(Int_t oldSize, Int_t newSize) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since fUnzipStatus point to atomic, they are presumably accessed from multiple thread. Can 'Reset' be called when there is a potential access from other threads? If so, was the ordering of the operation here checked for thread safety. If so, can you add documentation/explanation. If Reset does not need to be thread, please note in comment why this is the case.

std::vector<Int_t> aUnzipLen = std::vector<Int_t>(newSize, 0);
@@ -562,7 +565,7 @@ Int_t TTreeCacheUnzip::UnzipCache(Int_t index)
}

if ((myCycle != fCycle) || !fIsTransferred) {
fUnzipState.SetFinished(index); // Set it as not done
fUnzipState.SetFinished(index); // Set it as not done, main thread will take charge
return 1;
}

@@ -579,7 +582,7 @@ Int_t TTreeCacheUnzip::UnzipCache(Int_t index)
readbuf = ReadBufferExt(locbuff, rdoffs, rdlen, loc);

if (readbuf <= 0) {
fUnzipState.SetFinished(index); // Set it as not done
fUnzipState.SetFinished(index); // Set it as not done, main thread will take charge
if (locbuff) delete [] locbuff;
return -1;
}
@@ -595,7 +598,7 @@ Int_t TTreeCacheUnzip::UnzipCache(Int_t index)
if (gDebug > 0)
Info("UnzipCache", "Block %d is too big, skipping.", index);

fUnzipState.SetFinished(index); // Set it as done
fUnzipState.SetFinished(index); // Set it as not done, main thread will take charge
if (locbuff) delete [] locbuff;
return 0;
}
@@ -605,14 +608,14 @@ Int_t TTreeCacheUnzip::UnzipCache(Int_t index)
Int_t loclen = UnzipBuffer(&ptr, locbuff);
if ((loclen > 0) && (loclen == objlen + keylen)) {
if ((myCycle != fCycle) || !fIsTransferred) {
fUnzipState.SetFinished(index); // Set it as not done
fUnzipState.SetFinished(index); // Set it as not done, main thread will take charge
if (locbuff) delete [] locbuff;
return 1;
}
fUnzipState.SetUnzipped(index, ptr, loclen);
fUnzipState.SetUnzipped(index, ptr, loclen); // Set it as done
fNUnzip++;
} else {
fUnzipState.SetFinished(index);
fUnzipState.SetFinished(index); // Set it as not done, main thread will take charge
}

if (locbuff) delete [] locbuff;