Implement zarr backend

[LouiseDck]

No description provided.

[ivirshup]

I think this will be blocked on R zarr libraries. AFAICT there isn't enough support in either pizzarr or rarr, as neither has string support last I checked.

[keller-mark]

I implemented the VLen-UTF8 support yesterday keller-mark/pizzarr#55 - Example with a remote anndata-zarr store here https://keller-mark.github.io/pizzarr/articles/remote-anndata.html - feel free to report any issues if you end up trying it out

[keller-mark]

I have a preliminary implementation working on this branch (compare). Some of the functions emit warnings at the moment but most of the reading/writing tests are passing, as well as tests that compare values between h5ad and zarr

[Artur-man]

Gave it a quick try, looks promising ... here

library(pizzarr)
library(Seurat)

adata <- AnnData(
  X = t(GetAssayData(object = pbmc_small, layer = "counts")),
  obs = [email protected],
  obs_names = colnames(pbmc_small),
  var_names = rownames(pbmc_small)
)
write_zarr(adata, store = "data/example.zarr", compression = "none")

> adata
[1] "layers" "obs"    "obsm"   "obsp"   "uns"    "var"    "varm"   "varp"   "X"  

Still has some problems reading though.

data <- read_zarr(path = "data/example.zarr/", to = "ZarrAnnData")
data$X

> data$X
80 x 230 sparse Matrix of class "dgCMatrix"
                                                                                                                                                   
[1,] . 1 . . . 1 . .  3 . . 1 . . . . . . . . . . 1 . . . 
> data
Error in if (!is.na(path)) { : argument is of length zero

[Artur-man]

Are maintainers still interested in pushing ZarrAnnData to main ? We have a version where we test data types being properly wrote/read from/to zarr stores: keller-mark#5. (tests and functions updated to current version of anndataR)

This could be nice since some of us will be gathering to start building the R-native SpatialData package where Tables are still written using anndata and with reticulate/basilisk. https://github.com/HelenaLC/SpatialData/blob/45b166ac441933ff46432cd5f6a8368d041fffa7/R/read.R#L77-L92

[lazappi]

@rcannood can comment better on where things are currently at but I think in principal this is something we want. At the hackathon where we started {anndataR} we had discussed R {SpatialData} depending on it for AnnData related stuff and I think it's better for everyone to avoid duplicating work if possible.

[rcannood]

@keller-mark @Artur-man thanks for pinging me ! I'll take a look tonight 😊