218 coda transforms #244

em-t · 2023-11-28T14:19:02Z

Implement the requested logratio transformations for compositional data (issue #218):

ILR is only partially implemented: it's included only as a single ILR operation (instead of a full set of a given dataset), and there inverse ILR hasn't been implemented. (ILR is waiting on further details).
Add a decorator/wrapper for checking that the requirements for compositional data are fulfilled.
Include a notebook demonstrating use.

example of using ALR, CLR & ILR from the package pyrolite.

arg for keeping the redundant column. Use existing exception classes where possible. Move some utility functions into more appropriate places.

testing notebook.

… test.

…mon to coda transforms.

…ion, normalization and some checks.

versions of simplex check and normalizing functions. Fix some issues discovered during writing tests.

em-t · 2023-11-28T14:21:56Z

For the purposes of testing that the functionality is correct, this paper was a good overall source (and the first source I found with a proper definition for pivot logratio transform), and this paper has a simple, follow-along example for CLR, ALR and ILR.

nmaarnio

Hey! Overall code looks great and is easy to follow! My comments mostly concern some naming conventions and suggestion – feel free to disagree or counter-suggest something for these. I tried to check the different logratio logics too, but I have to admit I am not familiar with these methods myself.

eis_toolkit/transformations/coda/alr.py

nmaarnio · 2023-12-01T14:54:57Z

eis_toolkit/transformations/coda/alr.py

+    denominator_column = df.columns[idx]
+    columns = [col for col in df.columns]
+
+    if not keep_redundant_column and denominator_column in columns:
+        columns.remove(denominator_column)


If we decide to keep the denominator column, we will divide it by itself in the inner function, or am I wrong in here? This seems intuitively unintended to me, but I could be wrong

Yup, that's the case. I thought there should be the option to keep the column even though it becomes "redundant", since I don't know what the actual use cases for the function are. But default behavior will still be to return a dataframe with one less columns than the input frame.

eis_toolkit/transformations/coda/alr.py

eis_toolkit/transformations/coda/pairwise.py

eis_toolkit/utilities/aitchison_geometry.py

eis_toolkit/utilities/checks/compositional.py

eis_toolkit/utilities/miscellaneous.py

tests accordingly.

…ecorator to inverse_alr.

…ogratio.

…nction.

…s in VS code weren't working properly with chained decorator calls. Instead, perform the check within each function that needs it. Fix one of the compositional check tests that was resulting in a false positive assertion.

em-t · 2023-12-04T13:26:15Z

@nmaarnio All the suggested changes should now be addressed!

nmaarnio

Thanks for the quick changes – looks really good to me now! I only had one comment/question about the alr_transform parameterization.

nmaarnio · 2023-12-05T07:33:55Z

eis_toolkit/transformations/coda/alr.py

+        column: The integer position based index of the column of the dataframe to be used as denominator.
            If not provided, the last column will be used.


Did you leave the type of this parameter int on purpose? What I suggested was to let the user give the desired column name (str) instead of the index. This solution isn't bad, but can be a bit tricky if there are tens of columns in a DF :)

I must have missed it or forgotten halfway through going through the change suggestions. 😄 It's now fixed!

…ete check_column_index_in_dataframe function as unused. Update notebook.

nmaarnio · 2023-12-07T14:56:20Z

One more thing I noticed: the doc files are missing for these functions. After you have added them I will merge!

em-t · 2023-12-11T16:01:54Z

I added doc files similar to those for other modules. But I couldn't get mkdocs serve or build to work as described in the instructions.

I'm getting (the same issue is with all of them):

ERROR   -  mkdocstrings: eis_toolkit.transformations.coda.alr could not be found
ERROR   -  Error reading page 'transformations/coda/alr.md':
ERROR   -  Could not collect 'eis_toolkit.transformations.coda.alr'

Is there something that needs to be done in addition to adding the files for them to work?

nmaarnio · 2023-12-12T07:34:20Z

You need to add an empty __init__.py file in the coda folder (the folder with the implementations) so that mkdocs will recognize it. It should work after that.

nmaarnio

LGTM, merging!

Emmu T added 30 commits October 31, 2023 12:40

Add dependency pyrolite.

4b76996

Add a notebook for testing existing coda transformations. Include

cb56aa7

example of using ALR, CLR & ILR from the package pyrolite.

Add logratio.py.

679ad6c

Add notebook for testing logratio transformations.

ced375e

Add some utility functions.

1d4e16a

Work on ALR transform.

28b89c5

Edits to logratio transformation testing notebook.

b4c2797

More work on ALR transform: improve exception handling & docstring, add

4321389

arg for keeping the redundant column. Use existing exception classes where possible. Move some utility functions into more appropriate places.

Fix ALR transform calculation.

4d2d42a

Finishing touches to ALR transform.

a21328f

Rename logratio.py to alr.py and move into subpackage coda. Update

dd7dea4

testing notebook.

Add file for CLR transform.

3efd2bc

Convert df.columns to a list to be able to remove items.

65bceed

Fix logic error in index check.

2fc69ba

Add some tests for ALR transform.

79c5658

Add test file for CLR. Work on CLR: check for zeros + add appropriate…

7f5152c

… test.

Implement clr transform.

bd4d4e0

Perform CLR for selected columns only.

4d6e09f

Add module & test file for ILR transform.

f9e273f

Merge branch 'master' into 218-coda-transforms

b4c225c

Fix broken module paths.

c3b08b8

Fix CLR logic.

676839c

Tidy up clr_test.py.

76fa9f4

Add public-facing CLR transform function.

62552df

Add public-facing ALR transform function + some cleanup.

87f47b0

Add a test + tidy up ALR tests.

58ab17f

Add a utility module called aitchison_geometry to store functions com…

9ecc820

…mon to coda transforms.

Work on common utilities for coda transformations. Add closure operat…

894fbf6

…ion, normalization and some checks.

Work on ILR (mostly boilerplate).

665ca4e

Add some tests for aitchison_geometry module. Add generalized (non-unit)

260813a

versions of simplex check and normalizing functions. Fix some issues discovered during writing tests.

Emmu T added 3 commits November 28, 2023 14:29

Update notebook.

28bf1fd

Allow placing denominator columns at specific index in inverse ALR.

c5ed399

Merge branch 'master' into 218-coda-transforms

1f78c06

em-t requested review from nmaarnio, lehtonenp and tomiturunen1 November 28, 2023 14:19

nmaarnio requested changes Dec 1, 2023

View reviewed changes

Emmu T added 11 commits December 4, 2023 09:58

Merge branch 'master' into 218-coda-transforms

0e60aaa

Rename alr_transform args. Remove idx argument from inverse_alr. Edit

6754b55

tests accordingly.

Add check for non-positive scale value in inverse_alr. Add beartype d…

cfb4ba0

…ecorator to inverse_alr.

Remove unimplemented inverse_ilr function stub.

0166fc7

Change arg type from float to Number.

690788b

Rename some args + change arg type from float to Number in pairwise l…

4f740b7

…ogratio.

Fix logic error in rename_columns.

11043e9

Change arg type from float to Number in a few more places.

38c3d0a

Split inverse_alr and inverse_clr into a public-facing and private fu…

c1b19c7

…nction.

Rename argument as requested.

e5d62c1

nmaarnio reviewed Dec 5, 2023

View reviewed changes

Change alr_transform denominator column parameter type to string. Del…

d43e160

…ete check_column_index_in_dataframe function as unused. Update notebook.

Add doc files.

0ec1449

Add init file.

6c14a1a

nmaarnio approved these changes Dec 12, 2023

View reviewed changes

nmaarnio merged commit 014b825 into master Dec 12, 2023
4 checks passed

nmaarnio mentioned this pull request Dec 13, 2023

Add functions for CoDA #218

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

218 coda transforms #244

218 coda transforms #244

em-t commented Nov 28, 2023

em-t commented Nov 28, 2023

nmaarnio left a comment

nmaarnio Dec 1, 2023

em-t Dec 4, 2023

em-t commented Dec 4, 2023

nmaarnio left a comment

nmaarnio Dec 5, 2023

em-t Dec 7, 2023

nmaarnio commented Dec 7, 2023

em-t commented Dec 11, 2023

nmaarnio commented Dec 12, 2023 •

edited

Loading

nmaarnio left a comment

		column: The integer position based index of the column of the dataframe to be used as denominator.
		If not provided, the last column will be used.

218 coda transforms #244

218 coda transforms #244

Conversation

em-t commented Nov 28, 2023

em-t commented Nov 28, 2023

nmaarnio left a comment

Choose a reason for hiding this comment

nmaarnio Dec 1, 2023

Choose a reason for hiding this comment

em-t Dec 4, 2023

Choose a reason for hiding this comment

em-t commented Dec 4, 2023

nmaarnio left a comment

Choose a reason for hiding this comment

nmaarnio Dec 5, 2023

Choose a reason for hiding this comment

em-t Dec 7, 2023

Choose a reason for hiding this comment

nmaarnio commented Dec 7, 2023

em-t commented Dec 11, 2023

nmaarnio commented Dec 12, 2023 • edited Loading

nmaarnio left a comment

Choose a reason for hiding this comment

nmaarnio commented Dec 12, 2023 •

edited

Loading