Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extension proposal for zarr accumulation #205

Open
wants to merge 29 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
494def1
Add zarr-accumulation/v1.0.rst draft
HailiangZhangNASA Feb 13, 2023
a31ce72
zarr-accumulation.rst added
HailiangZhangNASA Feb 14, 2023
64dc873
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
1690563
zarr-accumulation/v1.0.rst hyperlink polished
HailiangZhangNASA Feb 14, 2023
412a440
zarr-accumulation/v1.0.rst minor cleaning
HailiangZhangNASA Feb 14, 2023
bc022ea
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
e1c4412
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
6887e22
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
4f6eeda
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
63b8897
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
6329a42
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
b489021
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
7b40c00
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
e76ed28
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
800ff95
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
ac27f9f
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
ec3ac00
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
4030f36
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
dc1e559
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
5ddad67
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
4181d85
zarr-accumulation/v1.0.rst code formatting
HailiangZhangNASA Feb 14, 2023
be5a3aa
zarr-accumulation/v1.0.rst more content added
HailiangZhangNASA Feb 14, 2023
27ecba5
more content added in Motivation
HailiangZhangNASA Feb 14, 2023
68bf791
Detailed description added
HailiangZhangNASA Feb 14, 2023
82c2948
minor cleaning
HailiangZhangNASA Feb 14, 2023
0d2e1e7
minor cleaning
HailiangZhangNASA Feb 14, 2023
87f3d70
minor cleaning
HailiangZhangNASA Feb 14, 2023
d6da2f2
minor cleaning
HailiangZhangNASA Feb 14, 2023
c302847
hyperlink provided
HailiangZhangNASA Feb 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
minor cleaning
  • Loading branch information
HailiangZhangNASA committed Feb 14, 2023
commit 0d2e1e7f72119896dbae98cf79feb11eb982ddce
2 changes: 1 addition & 1 deletion docs/extensions/zarr-accumulation/v1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The fundamental of this Zarr-based Chunk-level Accumulation algorithm is to pre-

For example:

For a sequence `A=(a, b, c, d)`, the cumulative sums are `S=(s0, s1, s2, s3)` where `s0=a, s1=a+b, s2=a+b+c, s3=a+b+c+d`. The average of the sequence over a range can now be calculated using the cumulative sums. For example, assuming zero-based indexing, average `(A[1:])=(S[3]-S[0])/3=(s3-s0)/3`.
For a sequence ``A=(a, b, c, d)``, the cumulative sums are ``S=(s0, s1, s2, s3)`` where ``s0=a, s1=a+b, s2=a+b+c, s3=a+b+c+d``. The average of the sequence over a range can now be calculated using the cumulative sums. For example, assuming zero-based indexing, average ``(A[1:])=(S[3]-S[0])/3=(s3-s0)/3``.

The above example can be extended and generalized for multiple dimensions. This makes the problem of computing averages *O(1)* vs *O(N^m)* for the dimensions being averaged, where *N* is the number of data values and *m* is the number of dimensions to be averaged. See our `ESIP 2022 presentation <https://www.youtube.com/watch?v=ac_UKunUrNM&t=2250s>`_ (and the `slides <https://docs.google.com/presentation/d/1RNvkIlCFvtoy89OTMzQNn_0jixOpdhnu/edit?usp=sharing&ouid=106287227661991623566&rtpof=true&sd=true>`_) for a more detailed description.

Expand Down