Document object storage and scratch bucket

[rabernat]

Context

Our hubs are in the cloud, which means that a powerful and scalable way to interact with data is via cloud object storage. The Pangeo-style hubs are (afaik) all configured with an object storage "scratch bucket", meaning that object storage is part of 2i2c's service.

Our new LEAP and M2LInES hubs will need user-facing documentation that explains how to work with object storage. This documentation can live inside or outside 2i2c. Currently 2i2c docs do not address object storage at all. I believe it would be advantageous to us all if this were part of 2i2c documentation.

Proposal

I propose we start by migrating some of the content in

https://pangeo.io/cloud.html#cloud-object-storage

to

https://docs.2i2c.org/en/latest/admin/howto/data.html

Overall, a longer and more detailed "data management" section, without more information about how to work with data in the cloud, should be our goal.

I can commit to writing this documentation if we agree it is in-scope for 2i2c.

Updates and actions

No response

[yuvipanda]

@rabernat would be lovely for you to write it! We can keep it inside 2i2c for now as I don't see another obvious location, and we can move it around later as we find places for it.

[damianavila]

Related PRs (besides the one linked):

• Setup gh-scoped-creds on pangeo hub infrastructure#1219
• Add gh-scoped-creds to base-notebook pangeo-data/pangeo-docker-images#305

[yuvipanda]

And conda-forge/staged-recipes#18710

[damianavila]

And 2i2c-org/infrastructure#1221