-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cache DataStub data #448
cache DataStub data #448
Conversation
Codecov Report
@@ Coverage Diff @@
## master #448 +/- ##
==========================================
+ Coverage 85.92% 85.96% +0.03%
==========================================
Files 126 126
Lines 5216 5236 +20
==========================================
+ Hits 4482 4501 +19
- Misses 734 735 +1
Help us with your feedback. Take ten seconds to tell us how you rate us. |
Since these are also cached.
- datastub constructor can now take dataset and file ids. - modify parse dataset to use new datastub constructor
Originally took two arguments. Now keyword arguments in External Links
better organize and isolate control of file id lifetimes when using data stub caching.
ffe056f
to
d22e7d9
Compare
H5F.reopen does not create a new id in older versions of MATLAB. Defaulting back into legacy behavior with ids.
@lawrence-mbf is this ready to merge? |
@bendichter I don't think so. There was still some instability in the last hackathon that caused full MATLAB crashes and the "solution" in the last commit only really resolved it at the high level. Unfortunately, I narrowed this down to a MATLAB bug present in versions older than R2021a where H5F.close() would close all file IDs, even if you've cloned it properly. To reproduce: filename = 'test.nwb';
datasetPath = '/file_create_date';
fid = H5F.open(filename, 'H5F_ACC_RDONLY', 'H5P_DEFAULT');
sourceDid = H5D.open(fid, datasetPath);
sourceGid = H5G.open(fid, '/general');
% this can occur with both H5F.open and H5F.reopen
fidReopened = H5F.open(filename, 'H5F_ACC_RDONLY', 'H5P_DEFAULT');
% fidReopened = H5F.reopen(fid);
didReopened = H5D.open(fidReopened, datasetPath);
% The error only occurs when ALL HDF5 IDs are closed including the source file id.
% not closing any of the below ids keeps it "alive" and does not cause a crash.
H5D.close(sourceDid);
H5G.close(sourceGid);
H5F.close(fid);
data = H5D.read(didReopened); % MATLAB crashes here. |
I see. Is there any way we could file this with Mathworks? |
I suppose you could but it is technically fixed in R2021a and later. You could also bring it up in the working group I suppose. |
I think they still maintain patches for older versions |
Fixes #412
Motivation
Add caching to data stubs.
How to test the behavior?
Checklist
fix #XX
whereXX
is the issue number?