Skip to content

Commit

Permalink
Merge branch 'master' into wip/codecsShards
Browse files Browse the repository at this point in the history
  • Loading branch information
cmhulbert committed Jan 2, 2025
2 parents bc5d103 + bdb8cde commit f294898
Show file tree
Hide file tree
Showing 8 changed files with 17 additions and 60 deletions.
7 changes: 7 additions & 0 deletions .github/setup.sh
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
#!/bin/sh
curl -fsLO https://raw.githubusercontent.com/scijava/scijava-scripts/main/ci-setup-github-actions.sh
sh ci-setup-github-actions.sh

# Let the Linux build handle artifact deployment.
if [ "$(uname)" != Linux ]
then
echo "No deploy -- non-Linux build"
echo "NO_DEPLOY=1" >> $GITHUB_ENV
fi
23 changes: 0 additions & 23 deletions .github/workflows/build-pr.yml

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@ on:
- master
tags:
- "*-[0-9]+.*"
pull_request:
branches:
- master

jobs:
build:
Expand All @@ -24,8 +27,10 @@ jobs:
cache: 'maven'
- name: Set up CI environment
run: .github/setup.sh
shell: bash
- name: Execute the build
run: .github/build.sh
shell: bash
env:
GPG_KEY_NAME: ${{ secrets.GPG_KEY_NAME }}
GPG_PASSPHRASE: ${{ secrets.GPG_PASSPHRASE }}
Expand Down
30 changes: 0 additions & 30 deletions .github/workflows/platform-test.yml

This file was deleted.

2 changes: 1 addition & 1 deletion LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## BSD 2-Clause License

Copyright (c) 2017-2024, Stephan Saalfeld
Copyright (c) 2017-2025, Stephan Saalfeld

All rights reserved.

Expand Down
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ N5 group is not a single file but simply a directory on the file system. Meta-d

1. All directories of the file system are N5 groups.
2. A JSON file `attributes.json` in a directory contains arbitrary attributes. A group without attributes may not have an `attributes.json` file.
3. The version of this specification is 1.0.0 and is stored in the "n5" attribute of the root group "/".
3. The version of this specification is 4.0.0 and is stored in the "n5" attribute of the root group "/".
4. A dataset is a group with the mandatory attributes:
* dimensions (e.g. [100, 200, 300]),
* blockSize (e.g. [64, 64, 64]),
Expand All @@ -38,7 +38,7 @@ N5 group is not a single file but simply a directory on the file system. Meta-d
* xz with parameters
* preset (integer, default 6).

Custom compression schemes with arbitrary parameters can be added using [compression annotations](#extensible-compression-schemes), e.g. [N5 Blosc](https://github.com/saalfeldlab/n5-blosc).
Custom compression schemes with arbitrary parameters can be added using [compression annotations](#extensible-compression-schemes), e.g. [N5 Blosc](https://github.com/saalfeldlab/n5-blosc) and [N5 ZStandard](https://github.com/JaneliaSciComp/n5-zstandard/).
5. Chunks are stored in a directory hierarchy that enumerates their positive integer position in the chunk grid (e.g. `0/4/1/7` for chunk grid position p=(0, 4, 1, 7)).
6. Datasets are sparse, i.e. there is no guarantee that all chunks of a dataset exist.
7. Chunks cannot be larger than 2GB (2<sup>31</sup>Bytes).
Expand Down Expand Up @@ -134,4 +134,3 @@ Custom compression schemes can be implemented using the annotation discovery mec
HDF5 is a great format that provides a wealth of conveniences that I do not want to miss. It's inefficiency for parallel writing, however, limit its applicability for handling of very large n-dimensional data.
N5 uses the native filesystem of the target platform and JSON files to specify basic and custom meta-data as attributes. It aims at preserving the convenience of HDF5 where possible but doesn't try too hard to be a full replacement.
Please do not take this project too seriously, we will see where it will get us and report back when more data is available.
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@

<groupId>org.janelia.saalfeldlab</groupId>
<artifactId>n5</artifactId>
<version>3.3.1-SNAPSHOT</version>
<version>3.3.2-SNAPSHOT</version>

<name>N5</name>
<description>Not HDF5</description>
Expand Down
3 changes: 1 addition & 2 deletions src/main/java/org/janelia/saalfeldlab/n5/N5Reader.java
Original file line number Diff line number Diff line change
Expand Up @@ -565,10 +565,9 @@ default String[] deepList(
final LinkedBlockingQueue<Future<String>> datasetFutures = new LinkedBlockingQueue<>();
deepListHelper(this, normalPathName, false, filter, executor, datasetFutures);

datasetFutures.poll().get(); // skip self
while (!datasetFutures.isEmpty()) {
final String result = datasetFutures.poll().get();
if (result != null)
if (result != null && !result.equals(normalPathName))
results.add(result.substring(normalPathName.length() + groupSeparator.length()));
}

Expand Down

0 comments on commit f294898

Please sign in to comment.