Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple multiscales #44

Open
will-moore opened this issue Nov 7, 2022 · 6 comments
Open

Multiple multiscales #44

will-moore opened this issue Nov 7, 2022 · 6 comments

Comments

@will-moore
Copy link
Member

will-moore commented Nov 7, 2022 •
edited
Loading

I don't know if this is an issue with ZarrReader or elsewhere, but just need somewhere to document this strange behaviour.

To test the support for various NGFF viewers for "multiple multiscales", I created a sample at https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr:

  • omero zarr export 4995115 -> 4995115.zarr
  • omero zarr export 4995097 -> 4995097.zarr
  • Copied each dataset from e.g. 4995097.zarr/0 -> 4995115.zarr/multiscales2_0
  • Copied multiscales metadata from 4995097.zarr/.zattrs into 4995115.zarr/.zattrs to add a second multiscales to the list.

When imported into OMERO, we get 2 images and the 2nd looks OK. But the first one seems to show the same planes for every Z-index and the 3 channels show mostly the same thing, but not exactly the same!

Screenshot 2022-11-07 at 18 43 49

Viewing the same NGFF in vizarr looks fine (although the 2nd multiscales image is ignored): https://hms-dbmi.github.io/vizarr/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr

If I remove the 2nd multiscales item from the list in 4995115.zarr/.zattrs and re-import, the image imports as expected (and the 'orphaned' .zarrays get imported as separate images...

Screenshot 2022-11-07 at 19 23 18

import log 

2022-11-07 19:26:21,964 248        [      main] INFO          ome.formats.importer.ImportConfig - OMERO.blitz Version: 5.5.12
2022-11-07 19:26:21,980 264        [      main] INFO          ome.formats.importer.ImportConfig - Bioformats version: 6.10.0 revision: f8b46c2458c43cffdf5bc67cc4bf9dfc6e93167b date: 31 May 2022
2022-11-07 19:26:22,032 316        [      main] INFO   formats.importer.cli.CommandLineImporter - Log levels -- Bio-Formats: ERROR OMERO.importer: INFO
2022-11-07 19:26:22,332 616        [      main] INFO      ome.formats.importer.ImportCandidates - Depth: 20 Metadata Level: MINIMUM
2022-11-07 19:26:23,281 1565       [      main] INFO      ome.formats.importer.ImportCandidates - 209 file(s) parsed into 1 group(s) with 1 call(s) to setId in 643ms. (949ms total) [0 unknowns]
2022-11-07 19:26:23,322 1606       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Attempting initial SSL connection to merge-ci-devspace.openmicroscopy.org:4064
2022-11-07 19:26:23,701 1985       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Insecure connection requested, falling back
2022-11-07 19:26:23,950 2234       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Pinging session every 300s.
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Server: 5.6.3
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Client: 5.5.12
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - Java Version: 11.0.9.1-internal
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Name: Linux
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Arch: amd64
2022-11-07 19:26:23,961 2245       [      main] INFO       ome.formats.OMEROMetadataStoreClient - OS Version: 3.10.0-1160.45.1.el7.x86_64
2022-11-07 19:26:24,324 2608       [      main] INFO          ome.formats.importer.ImportConfig - Using import target: Dataset:62870
2022-11-07 19:26:24,347 2631       [2-thread-1] INFO         ome.formats.importer.ImportLibrary - Import target specifies container: Dataset:62870
2022-11-07 19:26:24,350 2634       [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_PREPARATION
2022-11-07 19:26:42,550 20834      [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_START
2022-11-07 19:26:42,562 20846      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/2/0/0...
2022-11-07 19:26:42,589 20873      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/2/0/0
2022-11-07 19:26:42,625 20909      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/2/0/0
2022-11-07 19:26:42,638 20922      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/.zarray...
2022-11-07 19:26:42,658 20942      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/.zarray
2022-11-07 19:26:42,685 20969      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/.zarray
2022-11-07 19:26:42,698 20982      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/1/0/0...
2022-11-07 19:26:42,716 21000      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/1/0/0
2022-11-07 19:26:42,746 21030      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/1/0/0
2022-11-07 19:26:42,759 21043      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/0/0/0...
2022-11-07 19:26:42,777 21061      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/0/0/0
2022-11-07 19:26:42,806 21090      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/0/0/0
2022-11-07 19:26:42,816 21100      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/3/.zarray...
2022-11-07 19:26:42,834 21118      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/3/.zarray
2022-11-07 19:26:42,860 21144      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/3/.zarray

.....

2022-11-07 19:26:52,734 31018      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/11/0/0
2022-11-07 19:26:52,745 31029      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/5/0/0...
2022-11-07 19:26:52,772 31056      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/5/0/0
2022-11-07 19:26:52,817 31101      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/5/0/0
2022-11-07 19:26:52,828 31112      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/2/0/0...
2022-11-07 19:26:52,844 31128      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/2/0/0
2022-11-07 19:26:53,008 31292      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/2/0/0
2022-11-07 19:26:53,022 31306      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/4/0/0...
2022-11-07 19:26:53,043 31327      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/4/0/0
2022-11-07 19:26:53,090 31374      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/4/0/0
2022-11-07 19:26:53,101 31385      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/1/0/0...
2022-11-07 19:26:53,119 31403      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/1/0/0
2022-11-07 19:26:53,159 31443      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/1/0/0
2022-11-07 19:26:53,171 31455      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/0/0/0...
2022-11-07 19:26:53,189 31473      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/0/0/0
2022-11-07 19:26:53,227 31511      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/0/0/0
2022-11-07 19:26:53,239 31523      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/9/0/0...
2022-11-07 19:26:53,256 31540      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/9/0/0
2022-11-07 19:26:53,295 31579      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/9/0/0
2022-11-07 19:26:53,307 31591      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/10/0/0...
2022-11-07 19:26:53,325 31609      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/10/0/0
2022-11-07 19:26:53,361 31645      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/10/0/0
2022-11-07 19:26:53,373 31657      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/8/0/0...
2022-11-07 19:26:53,405 31689      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/8/0/0
2022-11-07 19:26:53,449 31733      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/1/0/8/0/0
2022-11-07 19:26:53,462 31746      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/.zarray...
2022-11-07 19:26:53,481 31765      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/.zarray
2022-11-07 19:26:53,503 31787      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/.zarray
2022-11-07 19:26:53,516 31800      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/2/0/0...
2022-11-07 19:26:53,543 31827      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/2/0/0
2022-11-07 19:26:53,568 31852      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/2/0/0
2022-11-07 19:26:53,581 31865      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/1/0/0...
2022-11-07 19:26:53,598 31882      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/1/0/0
2022-11-07 19:26:53,629 31913      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/1/0/0
2022-11-07 19:26:53,641 31925      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/0/0/0...
2022-11-07 19:26:53,664 31948      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/0/0/0
2022-11-07 19:26:53,692 31976      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_2/0/0/0
2022-11-07 19:26:53,704 31988      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/.zarray...
2022-11-07 19:26:53,730 32014      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/.zarray
2022-11-07 19:26:53,751 32035      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/.zarray
2022-11-07 19:26:53,764 32048      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/2/0/0...
2022-11-07 19:26:53,785 32069      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/2/0/0
2022-11-07 19:26:53,878 32162      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/2/0/0
2022-11-07 19:26:53,891 32175      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/1/0/0...
2022-11-07 19:26:53,916 32200      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/1/0/0
2022-11-07 19:26:54,029 32313      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/1/0/0
2022-11-07 19:26:54,042 32326      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/0/0/0...
2022-11-07 19:26:54,067 32351      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/0/0/0
2022-11-07 19:26:54,153 32437      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_0/0/0/0
2022-11-07 19:26:54,165 32449      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/.zarray...
2022-11-07 19:26:54,192 32476      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/.zarray
2022-11-07 19:26:54,213 32497      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/.zarray
2022-11-07 19:26:54,225 32509      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/7/0/0...
2022-11-07 19:26:54,251 32535      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/7/0/0
2022-11-07 19:26:54,324 32608      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/7/0/0
2022-11-07 19:26:54,333 32617      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/6/0/0...
2022-11-07 19:26:54,369 32653      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/6/0/0
@     
2022-11-07 19:26:54,452 32736      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/3/0/0...
2022-11-07 19:26:54,479 32763      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/3/0/0
2022-11-07 19:26:54,549 32833      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/3/0/0
2022-11-07 19:26:54,562 32846      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/11/0/0...
2022-11-07 19:26:54,580 32864      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/11/0/0
2022-11-07 19:26:54,639 32923      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/11/0/0
2022-11-07 19:26:54,653 32937      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/5/0/0...
2022-11-07 19:26:54,670 32954      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/5/0/0
2022-11-07 19:26:54,736 33020      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/5/0/0
2022-11-07 19:26:54,750 33034      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/2/0/0...
2022-11-07 19:26:54,768 33052      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/2/0/0
2022-11-07 19:26:54,843 33127      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/2/0/0
2022-11-07 19:26:54,856 33140      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/4/0/0...
2022-11-07 19:26:54,871 33155      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/4/0/0
2022-11-07 19:26:54,934 33218      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/4/0/0
2022-11-07 19:26:54,948 33232      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/1/0/0...
2022-11-07 19:26:54,966 33250      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/1/0/0
2022-11-07 19:26:55,042 33326      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/1/0/0
2022-11-07 19:26:55,054 33338      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/0/0/0...
2022-11-07 19:26:55,072 33356      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/0/0/0
2022-11-07 19:26:55,146 33430      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/0/0/0
2022-11-07 19:26:55,159 33443      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/9/0/0...
2022-11-07 19:26:55,178 33462      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/9/0/0
2022-11-07 19:26:55,240 33524      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/9/0/0
2022-11-07 19:26:55,253 33537      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/10/0/0...
2022-11-07 19:26:55,271 33555      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/10/0/0
2022-11-07 19:26:55,331 33615      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/10/0/0
2022-11-07 19:26:55,343 33627      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/8/0/0...
2022-11-07 19:26:55,361 33645      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/8/0/0
2022-11-07 19:26:55,422 33706      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/2/8/0/0
2022-11-07 19:26:55,434 33718      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/7/0/0...
2022-11-07 19:26:55,451 33735      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/7/0/0
2022-11-07 19:26:55,542 33826      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/7/0/0
2022-11-07 19:26:55,555 33839      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/6/0/0...
2022-11-07 19:26:55,581 33865      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/6/0/0
2022-11-07 19:26:55,661 33945      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/6/0/0
2022-11-07 19:26:55,674 33958      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/3/0/0...
2022-11-07 19:26:55,701 33985      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/3/0/0
2022-11-07 19:26:55,795 34079      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/3/0/0
2022-11-07 19:26:55,808 34092      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/11/0/0...
2022-11-07 19:26:55,840 34124      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/11/0/0
2022-11-07 19:26:55,919 34203      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/11/0/0
2022-11-07 19:26:55,932 34216      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/5/0/0...
2022-11-07 19:26:55,959 34243      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/5/0/0
2022-11-07 19:26:56,045 34329      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/5/0/0
2022-11-07 19:26:56,058 34342      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/2/0/0...
2022-11-07 19:26:56,085 34369      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/2/0/0
2022-11-07 19:26:56,182 34466      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/2/0/0
2022-11-07 19:26:56,195 34479      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/4/0/0...
2022-11-07 19:26:56,222 34506      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/4/0/0
2022-11-07 19:26:56,316 34600      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/4/0/0
2022-11-07 19:26:56,329 34613      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/1/0/0...
2022-11-07 19:26:56,355 34639      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/1/0/0
2022-11-07 19:26:56,453 34737      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/1/0/0
2022-11-07 19:26:56,467 34751      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/0/0/0...
2022-11-07 19:26:56,494 34778      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/0/0/0
2022-11-07 19:26:56,595 34879      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/0/0/0
2022-11-07 19:26:56,608 34892      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/9/0/0...
2022-11-07 19:26:56,635 34919      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/9/0/0
2022-11-07 19:26:56,727 35011      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/9/0/0
2022-11-07 19:26:56,740 35024      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/10/0/0...
2022-11-07 19:26:56,768 35052      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/10/0/0
2022-11-07 19:26:56,862 35146      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/10/0/0
2022-11-07 19:26:56,875 35159      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/8/0/0...
2022-11-07 19:26:56,904 35188      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/8/0/0
2022-11-07 19:26:57,001 35285      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/1/8/0/0
2022-11-07 19:26:57,015 35299      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/7/0/0...
2022-11-07 19:26:57,033 35317      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/7/0/0
2022-11-07 19:26:57,126 35410      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/7/0/0
2022-11-07 19:26:57,133 35417      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/6/0/0...
2022-11-07 19:26:57,152 35436      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/6/0/0
2022-11-07 19:26:57,242 35526      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/6/0/0
2022-11-07 19:26:57,253 35537      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/3/0/0...
2022-11-07 19:26:57,271 35555      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/3/0/0
2022-11-07 19:26:57,366 35650      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/3/0/0
2022-11-07 19:26:57,377 35661      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/11/0/0...
2022-11-07 19:26:57,394 35678      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/11/0/0
2022-11-07 19:26:57,465 35749      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/11/0/0
2022-11-07 19:26:57,473 35757      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/5/0/0...
2022-11-07 19:26:57,491 35775      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/5/0/0
2022-11-07 19:26:57,563 35847      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/5/0/0
2022-11-07 19:26:57,574 35858      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/2/0/0...
2022-11-07 19:26:57,595 35879      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/2/0/0
2022-11-07 19:26:57,671 35955      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/2/0/0
2022-11-07 19:26:57,681 35965      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/4/0/0...
2022-11-07 19:26:57,707 35991      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/4/0/0
2022-11-07 19:26:57,788 36072      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/4/0/0
2022-11-07 19:26:57,799 36083      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/1/0/0...
2022-11-07 19:26:57,828 36112      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/1/0/0
2022-11-07 19:26:57,913 36197      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/1/0/0
2022-11-07 19:26:57,924 36208      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/0/0/0...
2022-11-07 19:26:57,951 36235      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/0/0/0
2022-11-07 19:26:58,030 36314      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/0/0/0
2022-11-07 19:26:58,041 36325      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/9/0/0...
2022-11-07 19:26:58,067 36351      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/9/0/0
2022-11-07 19:26:58,141 36425      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/9/0/0
2022-11-07 19:26:58,152 36436      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/10/0/0...
2022-11-07 19:26:58,169 36453      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/10/0/0
2022-11-07 19:26:58,241 36525      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/10/0/0
2022-11-07 19:26:58,251 36535      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/8/0/0...
2022-11-07 19:26:58,268 36552      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/0/0/8/0/0
@  
2022-11-07 19:26:58,351 36635      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/zattrs_copy...
2022-11-07 19:26:58,368 36652      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/zattrs_copy
2022-11-07 19:26:58,390 36674      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/zattrs_copy
2022-11-07 19:26:58,399 36683      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/.zgroup...
2022-11-07 19:26:58,416 36700      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/.zgroup
2022-11-07 19:26:58,436 36720      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/.zgroup
2022-11-07 19:26:58,445 36729      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/.zarray...
2022-11-07 19:26:58,461 36745      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/.zarray
2022-11-07 19:26:58,482 36766      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/.zarray
2022-11-07 19:26:58,491 36775      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/2/0/0...
2022-11-07 19:26:58,507 36791      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/2/0/0
2022-11-07 19:26:58,527 36811      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/2/0/0
2022-11-07 19:26:58,537 36821      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/1/0/0...
2022-11-07 19:26:58,599 36883      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/1/0/0
2022-11-07 19:26:58,637 36921      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/1/0/0
2022-11-07 19:26:58,647 36931      [3-thread-1] INFO   ts.importer.transfers.UploadFileTransfer - Transferring /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/0/0/0...
2022-11-07 19:26:58,663 36947      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_STARTED: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/0/0/0
2022-11-07 19:26:58,682 36966      [3-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILE_UPLOAD_COMPLETE: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_4/0/0/0
2022-11-07 19:26:59,734 38018      [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - FILESET_UPLOAD_END
2022-11-07 19:26:59,826 38110      [2-thread-1] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_STARTED Logfile: 1703492
2022-11-07 19:27:00,883 39167      [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - METADATA_IMPORTED Step: 1 of 5  Logfile: 1703492
2022-11-07 19:27:04,168 42452      [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - PIXELDATA_PROCESSED Step: 2 of 5  Logfile: 1703492
2022-11-07 19:27:04,826 43110      [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - THUMBNAILS_GENERATED Step: 3 of 5  Logfile: 1703492
2022-11-07 19:27:04,852 43136      [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - METADATA_PROCESSED Step: 4 of 5  Logfile: 1703492
2022-11-07 19:27:04,872 43156      [l.Client-0] INFO   ormats.importer.cli.LoggingImportMonitor - OBJECTS_RETURNED Step: 5 of 5  Logfile: 1703492
2022-11-07 19:27:04,962 43246      [l.Client-1] INFO   ormats.importer.cli.LoggingImportMonitor - IMPORT_DONE Imported file: /uod/idr/objectstore/minio/idr/v0.4/idr0050/4995115.zarr/multiscales2_3/2/0/0
Other imported objects:
Fileset:132559

==> Summary
209 files uploaded, 1 fileset created, 2 images imported, 0 errors in 0:00:40.896
@will-moore
Copy link
Member Author

I created another multiple-multiscale NGFF at https://ome.github.io/ome-ngff-validator/?source=https://minio-dev.openmicroscopy.org/idr/v0.4/idr0050/4995115.zarr which seems to import fine - both images look correct.
So, just something strange about the first test image above? Maybe something to do with images of different dimensions?

@will-moore will-moore mentioned this issue Nov 9, 2022
Merged
@will-moore
Copy link
Member Author

Having tested some more with another sample, it seems that the presence of a single Z/T image is what triggers the "corruption" of the planes on previous images.
Sample currently available at https://ome.github.io/ome-ngff-validator/?source=https://minio-dev.openmicroscopy.org/idr/v0.4/idr0062/6001240.zarr
gives 3 Z-stack images where all Z/C planes are the same:

Screenshot 2022-11-10 at 13 07 17

@joshmoore
Copy link
Member

To test the support for various NGFF viewers for "multiple multiscales", I created a sample at https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr:

So a file under idr/zarr/v0.4 is in an unknown state? If so, can this be moved to test-data until we've figured this out?

Viewing the same NGFF in vizarr looks fine (although the 2nd multiscales image is ignored)

This is the behavior that I would actually expect, though I assume we need to work on the semantics of multiple multiscales. @dgault, what behavior would you expect?

@will-moore
Copy link
Member Author

I'm pretty sure that https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr is valid.

ZarrReader imports both images (as expected 👍 ) but the first image is 'corrupted' (not expected).

@joshmoore
Copy link
Member

I'm pretty sure that https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr is valid.

Even the "pretty sure" worries me.

@will-moore
Copy link
Member Author

Well, I'm as sure as I can be. https://ome.github.io/ome-ngff-validator/?source=https://uk1s3.embassy.ebi.ac.uk/idr/zarr/v0.4/idr0050A/4995115.zarr says it's valid.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@joshmoore @will-moore