Read neuroglancer precomputed

[normanrz]

Detailed Description

We already read Zarr and N5. Now, we also want to read Neuroglancer's precomputed for image and segmentation data (no meshes for now).
Here is the specification: https://github.com/google/neuroglancer/blob/master/src/neuroglancer/datasource/precomputed/volume.md

There are 2 flavours that we should support: legacy (single file per chunk) and sharded (multiple chunks per file).
There is also a python implementation in wkconnect that can be reused: https://github.com/scalableminds/webknossos-connect/tree/master/wkconnect/backends/neuroglancer

These neuroglancer formats also need to be "explorable" for users to be able to import them.

[normanrz]

Demo data:
Sharded:

• EM: https://www.googleapis.com/storage/v1/b/h01-release/o/data%2F20210601%2F4nm_raw%2Finfo?alt=media&neuroglancerOrigin=http%3A%2F%2Fh01-dot-neuroglancer-demo.appspot.com
• Segmentation: https://www.googleapis.com/storage/v1/b/h01-release/o/data%2F20210601%2Fc3%2Finfo?alt=media&neuroglancerOrigin=http%3A%2F%2Fh01-dot-neuroglancer-demo.appspot.com
• Viewer: http://h01-dot-neuroglancer-demo.appspot.com/#!%7B%22dimensions%22:%7B%22x%22:%5B8e-9%2C%22m%22%5D%2C%22y%22:%5B8e-9%2C%22m%22%5D%2C%22z%22:%5B3.3e-8%2C%22m%22%5D%7D%2C%22position%22:%5B247489.015625%2C193590.765625%2C2002.4998779296875%5D%2C%22crossSectionScale%22:66.56481577338745%2C%22projectionScale%22:237864.89707126172%2C%22layers%22:%5B%7B%22type%22:%22image%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/4nm_raw%22%2C%22tab%22:%22source%22%2C%22name%22:%224nm%20EM%22%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%7B%22url%22:%22precomputed://gs://h01-release/data/20210601/c3%22%2C%22subsources%22:%7B%22default%22:true%2C%22bounds%22:true%2C%22properties%22:true%2C%22mesh%22:true%7D%2C%22enableDefaultSubsources%22:false%7D%2C%22tab%22:%22source%22%2C%22segments%22:%5B%2212857642016%22%2C%2213046505828%22%2C%2213951650786%22%2C%2214794814799%22%2C%2215539757452%22%2C%222120664699%22%2C%2221348269172%22%2C%2221635455184%22%2C%2222948012913%22%2C%2227216368299%22%2C%2228104985461%22%2C%2228862498148%22%2C%2229210178754%22%2C%222950980117%22%2C%2229672577804%22%2C%2229730222317%22%2C%2229977680734%22%2C%2230211441416%22%2C%2230224642109%22%2C%2230501462297%22%2C%2230501870437%22%2C%2230516077733%22%2C%2230573824944%22%2C%2230588382028%22%2C%2230605127877%22%2C%2230865392105%22%2C%2230941549387%22%2C%2231068084020%22%2C%223106947392%22%2C%2231082523968%22%2C%2231083559920%22%2C%2231140649483%22%2C%2231274759074%22%2C%2231349195650%22%2C%2231576899712%22%2C%2231591499830%22%2C%2232385180272%22%2C%223296832824%22%2C%2233049905842%22%2C%223308836336%22%2C%2233269227364%22%2C%2233544164420%22%2C%2233663789818%22%2C%2233779853124%22%2C%2233779985123%22%2C%223512752660%22%2C%223513951463%22%2C%223588284587%22%2C%223631312996%22%2C%223645170418%22%2C%2237626413608%22%2C%223773716567%22%2C%223788376389%22%2C%223788551051%22%2C%223790844645%22%2C%223833230503%22%2C%2238611762212%22%2C%2238660748322%22%2C%2238689468673%22%2C%223907682045%22%2C%223937466277%22%2C%223949920978%22%2C%2239540618731%22%2C%223994337380%22%2C%224006748353%22%2C%224068088366%22%2C%224079680017%22%2C%224094078022%22%2C%224094501608%22%2C%224127236696%22%2C%224140858954%22%2C%224152247000%22%2C%224152861452%22%2C%224153108523%22%2C%224154686065%22%2C%224185377684%22%2C%2241933255298%22%2C%2242002712851%22%2C%2242108919642%22%2C%224254761989%22%2C%224255432378%22%2C%224312742466%22%2C%224326699742%22%2C%224326818082%22%2C%224327955860%22%2C%224370590371%22%2C%224372430258%22%2C%224372868243%22%2C%224474214654%22%2C%224475484255%22%2C%224475995296%22%2C%224489268853%22%2C%224500948443%22%2C%224501838736%22%2C%224503782000%22%2C%224504145716%22%2C%224516965446%22%2C%224519199538%22%2C%224519243274%22%2C%224530384028%22%2C%224573881259%22%2C%224575925341%22%2C%224617509729%22%2C%224619654531%22%2C%224619873240%22%2C%224723890949%22%2C%224735542348%22%2C%224766773877%22%2C%224780076410%22%2C%224809949027%22%2C%224823076755%22%2C%224839473227%22%2C%224852629019%22%2C%224896065848%22%2C%224924801658%22%2C%224926903194%22%2C%224938804262%22%2C%224955754739%22%2C%224970736617%22%2C%224997163340%22%2C%225028934840%22%2C%225042588326%22%2C%225055801374%22%2C%225103080167%22%2C%225114905708%22%2C%225115534039%22%2C%225115869424%22%2C%225144808371%22%2C%225159205830%22%2C%225174375994%22%2C%225231832117%22%2C%225247730833%22%2C%225248446458%22%2C%225291856456%22%2C%225317129594%22%2C%225348623756%22%2C%225364977088%22%2C%225376877616%22%2C%225377446229%22%2C%225377927964%22%2C%225406487849%22%2C%225480120932%22%2C%225492969654%22%2C%225565128350%22%2C%225595249249%22%2C%225623693160%22%2C%225624832585%22%2C%225682402703%22%2C%225845146472%22%2C%225855979281%22%2C%225887240031%22%2C%225900396223%22%2C%225902484088%22%2C%225942899024%22%2C%225975984947%22%2C%225989857178%22%2C%226163710175%22%2C%226177990420%22%2C%226209498692%22%2C%226220712283%22%2C%226222230785%22%2C%226263276083%22%2C%226278138984%22%2C%226309603153%22%2C%226322687028%22%2C%22635743109%22%2C%226384842151%22%2C%226411635598%22%2C%226412233719%22%2C%226484377108%22%2C%226613785802%22%2C%226614928166%22%2C%226631059913%22%2C%226643164727%22%2C%226658406895%22%2C%226702705861%22%2C%226745384646%22%2C%226777258586%22%2C%226804898281%22%2C%226863842503%22%2C%227037463002%22%2C%227067117794%22%5D%2C%22segmentQuery%22:%22#neuron%20#spiny-stellate%22%2C%22colorSeed%22:4270253886%2C%22name%22:%22c3%20segmentation%22%7D%2C%7B%22type%22:%22annotation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/synapses/precomputed%22%2C%22tab%22:%22source%22%2C%22ignoreNullSegmentFilter%22:false%2C%22shader%22:%22void%20main%28%29%20%7B%5Cn%20%20if%20%28prop_type%28%29%20==%20uint%281%29%29%20%7B%5Cn%20%20%20%20setColor%28vec3%280.%2C0.%2C1.%29%29%3B%5Cn%20%20%7D%20else%20%7B%5Cn%20%20%20%20setColor%28vec3%281.%2C1.%2C0.%29%29%3B%5Cn%20%20%7D%5Cn%20%20%5Cn%20%20setEndpointMarkerBorderWidth%280.0%29%3B%5Cn%20%20setEndpointMarkerSize%284.0%29%3B%5Cn%7D%5Cn%22%2C%22linkedSegmentationLayer%22:%7B%22post_synaptic_cell%22:%22c3%20segmentation%22%7D%2C%22filterBySegmentation%22:%5B%22post_synaptic_cell%22%5D%2C%22name%22:%22synapse%20annotations%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22annotation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/subcompartments/annotations%22%2C%22tab%22:%22source%22%2C%22ignoreNullSegmentFilter%22:false%2C%22shader%22:%22void%20main%28%29%20%7B%5Cn%20%20switch%20%28prop_class_label%28%29%29%20%7B%5Cn%20%20case%200:%20%20//%20axon%5Cn%20%20%20%20setColor%28vec3%280%2C%200%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201:%20%20//%20dendrite%5Cn%20%20%20%20setColor%28vec3%281%2C%200%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%202:%20%20//%20astrocyte%5Cn%20%20%20%20setColor%28vec3%280%2C%201%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%203:%20%20//%20soma%5Cn%20%20%20%20setColor%28vec3%281%2C%201%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%204:%20%20//%20cilium%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%205:%20%20//%20AIS%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20%20%20%5Cn%20%20case%201000:%20%20//%20myelinated%20axon%5Cn%20%20case%201001:%5Cn%20%20%20%20setColor%28vec3%281%2C%200.25%2C%200.75%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201004:%20%20//%20fragments%5Cn%20%20case%201005:%5Cn%20%20default:%5Cn%20%20%20%20discard%3B%5Cn%20%20%7D%5Cn%7D%22%2C%22linkedSegmentationLayer%22:%7B%22skeleton_id%22:%22c3%20segmentation%22%7D%2C%22filterBySegmentation%22:%5B%22skeleton_id%22%5D%2C%22name%22:%22subcompartment%20annotations%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/synapses/incoming_excitatory%22%2C%22tab%22:%22source%22%2C%22linkedSegmentationGroup%22:%22c3%20segmentation%22%2C%22linkedSegmentationColorGroup%22:false%2C%22colorSeed%22:2613304347%2C%22segmentDefaultColor%22:%22#ffff00%22%2C%22name%22:%22incoming%20excitatory%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/synapses/incoming_inhibitory%22%2C%22tab%22:%22source%22%2C%22linkedSegmentationGroup%22:%22c3%20segmentation%22%2C%22linkedSegmentationColorGroup%22:false%2C%22colorSeed%22:679993271%2C%22segmentDefaultColor%22:%22#0000ff%22%2C%22name%22:%22incoming%20inhibitory%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22annotation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/embeddings/combined_umap%22%2C%22tab%22:%22source%22%2C%22shader%22:%22void%20main%28%29%20%7B%5Cn%20%20setColor%28vec3%28%5Cn%20%20%20%201.0%20-%20%28prop_ue0%28%29%20+%204.6017108%29%20/%2018.7%2C%20%5Cn%20%20%20%201.0%20-%20%28prop_ue1%28%29%20+%200.13594195%20%29/%209.28%2C%20%5Cn%20%20%20%20%28prop_ue2%28%29%20-%206.69197893%29%20/%207.59%29%29%3B%20%5Cn%20%20setPointMarkerBorderColor%28vec3%28%5Cn%20%20%20%201.0%20-%20%28prop_ue0%28%29%20+%204.6017108%29%20/%2018.7%2C%20%5Cn%20%20%20%201.0%20-%20%28prop_ue1%28%29%20+%200.13594195%20%29/%209.28%2C%20%5Cn%20%20%20%20%28prop_ue2%28%29%20-%206.69197893%29%20/%207.59%29%29%3B%20%5Cn%20%20setPointMarkerSize%285.0%29%3B%5Cn%20%20setPointMarkerBorderWidth%280.%29%3B%5Cn%7D%22%2C%22linkedSegmentationLayer%22:%7B%22skeleton_id%22:%22c3%20segmentation%22%7D%2C%22filterBySegmentation%22:%5B%22skeleton_id%22%5D%2C%22name%22:%22embeddings%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22annotation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/subcompartments/annotations/cilia%22%2C%22tab%22:%22source%22%2C%22projectionAnnotationSpacing%22:1.3607900001743771%2C%22shader%22:%22void%20main%28%29%20%7B%5Cn%20%20switch%20%28prop_class_label%28%29%29%20%7B%5Cn%20%20case%200:%20%20//%20axon%5Cn%20%20%20%20setColor%28vec3%280%2C%200%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201:%20%20//%20dendrite%5Cn%20%20%20%20setColor%28vec3%281%2C%200%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%202:%20%20//%20astrocyte%5Cn%20%20%20%20setColor%28vec3%280%2C%201%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%203:%20%20//%20soma%5Cn%20%20%20%20setColor%28vec3%281%2C%201%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%204:%20%20//%20cilium%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%205:%20%20//%20AIS%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20%20%20%5Cn%20%20case%201000:%20%20//%20myelinated%20axon%5Cn%20%20case%201001:%5Cn%20%20%20%20setColor%28vec3%281%2C%200.25%2C%200.75%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201004:%20%20//%20fragments%5Cn%20%20case%201005:%5Cn%20%20default:%5Cn%20%20%20%20discard%3B%5Cn%20%20%7D%5Cn%7D%22%2C%22name%22:%22cilia%20annotations%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22annotation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/subcompartments/annotations/ais%22%2C%22tab%22:%22source%22%2C%22projectionAnnotationSpacing%22:0.14324743353719752%2C%22shader%22:%22void%20main%28%29%20%7B%5Cn%20%20switch%20%28prop_class_label%28%29%29%20%7B%5Cn%20%20case%200:%20%20//%20axon%5Cn%20%20%20%20setColor%28vec3%280%2C%200%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201:%20%20//%20dendrite%5Cn%20%20%20%20setColor%28vec3%281%2C%200%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%202:%20%20//%20astrocyte%5Cn%20%20%20%20setColor%28vec3%280%2C%201%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%203:%20%20//%20soma%5Cn%20%20%20%20setColor%28vec3%281%2C%201%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%204:%20%20//%20cilium%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%200%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%205:%20%20//%20AIS%5Cn%20%20%20%20setColor%28vec3%280.5%2C%200.5%2C%201%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20%20%20%5Cn%20%20case%201000:%20%20//%20myelinated%20axon%5Cn%20%20case%201001:%5Cn%20%20%20%20setColor%28vec3%281%2C%200.25%2C%200.75%29%29%3B%5Cn%20%20%20%20break%3B%5Cn%20%20case%201004:%20%20//%20fragments%5Cn%20%20case%201005:%5Cn%20%20default:%5Cn%20%20%20%20discard%3B%5Cn%20%20%7D%5Cn%7D%22%2C%22name%22:%22AIS%20annotations%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/c3/subcompartments%22%2C%22tab%22:%22source%22%2C%22segmentColors%22:%7B%22100%22:%22#0000ff%22%2C%22101%22:%22#ff0000%22%2C%22102%22:%22#00ff00%22%2C%22103%22:%22#ffffff%22%2C%22104%22:%22#7f7f00%22%2C%22105%22:%22#7f7fff%22%2C%221100%22:%22#ff3fbf%22%2C%221101%22:%22#ff3fbf%22%2C%221102%22:%22#000000%22%2C%221103%22:%22#000000%22%2C%221104%22:%22#ff3fbf%22%2C%221105%22:%22#ff3fbf%22%7D%2C%22name%22:%22subcompartments%20render%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/layers%22%2C%22tab%22:%22source%22%2C%22selectedAlpha%22:0.3%2C%22segments%22:%5B%221%22%2C%222%22%2C%223%22%2C%224%22%2C%225%22%2C%226%22%2C%227%22%5D%2C%22segmentQuery%22:%221%2C2%2C3%2C4%2C5%2C6%2C7%22%2C%22name%22:%22cortical%20layers%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/masking%22%2C%22tab%22:%22source%22%2C%22name%22:%22tissue%20mask%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/blood_vessels_segmented%22%2C%22tab%22:%22segments%22%2C%22selectedAlpha%22:0.25%2C%22segmentDefaultColor%22:%22#ff0000%22%2C%22name%22:%22blood%20vessels%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/blood_vessels%22%2C%22tab%22:%22source%22%2C%22name%22:%22blood%20vessel%20cells%22%2C%22archived%22:true%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://h01-release/data/20210601/cell_bodies%22%2C%22tab%22:%22source%22%2C%22name%22:%22cell%20bodies%22%2C%22archived%22:true%7D%5D%2C%22showDefaultAnnotations%22:false%2C%22showSlices%22:false%2C%22prefetch%22:false%2C%22selectedLayer%22:%7B%22flex%22:0.65%2C%22visible%22:true%2C%22layer%22:%224nm%20EM%22%7D%2C%22layout%22:%7B%22type%22:%22xy-3d%22%2C%22orthographicProjection%22:true%7D%2C%22helpPanel%22:%7B%22flex%22:0.92%2C%22visible%22:true%7D%2C%22selection%22:%7B%22flex%22:0.45%2C%22size%22:309%2C%22visible%22:false%7D%2C%22layerListPanel%22:%7B%22flex%22:1.08%2C%22visible%22:true%7D%7D

Legacy:

• EM: https://www.googleapis.com/storage/v1/b/neuroglancer-public-data/o/flyem_fib-25%2Fimage%2Finfo
• Segmentation: https://www.googleapis.com/storage/v1/b/neuroglancer-public-data/o/flyem_fib-25%2Fground_truth%2Finfo
• Viewer: https://neuroglancer-demo.appspot.com/#!%7B%22dimensions%22:%7B%22x%22:%5B8e-9%2C%22m%22%5D%2C%22y%22:%5B8e-9%2C%22m%22%5D%2C%22z%22:%5B8e-9%2C%22m%22%5D%7D%2C%22position%22:%5B2914.500732421875%2C3088.243408203125%2C4045%5D%2C%22crossSectionScale%22:3.762185354999915%2C%22projectionOrientation%22:%5B0.31435418128967285%2C0.8142172694206238%2C0.4843378961086273%2C-0.06040274351835251%5D%2C%22projectionScale%22:4593.980956070107%2C%22layers%22:%5B%7B%22type%22:%22image%22%2C%22source%22:%22precomputed://gs://neuroglancer-public-data/flyem_fib-25/image%22%2C%22tab%22:%22source%22%2C%22name%22:%22image%22%7D%2C%7B%22type%22:%22segmentation%22%2C%22source%22:%22precomputed://gs://neuroglancer-public-data/flyem_fib-25/ground_truth%22%2C%22tab%22:%22source%22%2C%22segments%22:%5B%22158571%22%2C%2221894%22%2C%2222060%22%2C%2224436%22%2C%222515%22%5D%2C%22name%22:%22ground-truth%22%7D%5D%2C%22showSlices%22:false%2C%22layout%22:%224panel%22%7D