Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timelapse option in transform #80

Merged
merged 29 commits into from
Jul 29, 2022

Conversation

yfukai
Copy link
Collaborator

@yfukai yfukai commented Jun 27, 2022

An attempt to add timelapse option in transform. Also did some refactoring of comments and examples.

@yfukai yfukai added this to the Beta release milestone Jun 30, 2022
@yfukai
Copy link
Collaborator Author

yfukai commented Jul 2, 2022

I also set the flatfield and darkfield shape to the original images. (closes #85)

@yfukai
Copy link
Collaborator Author

yfukai commented Jul 2, 2022

Docstring examples are also changed (closes #86 )

@Nicholas-Schaub Nicholas-Schaub merged commit 4571862 into peng-lab:dev Jul 29, 2022
@yfukai yfukai deleted the timelapse_option_in_transform branch July 29, 2022 01:55
Nicholas-Schaub added a commit that referenced this pull request Dec 2, 2022
* Add badges to README

* Update .gitignore

* Add noxfile

* Move files to src

* Add config files

* Placeholder dct functions

* Update gitignore

* Add class for settings

* Update setup.cfg

* Update requirements.txt

* Add docstring to settings class

* Add validation to Settings input

* Update github CI

* Add gpu option to init

* Update README formatting

* Rename 'figures' folder to 'resources'

* Update gitignore

* Update gitignore

* Move API from README to docs

* Refactor, update docstrings

* Change folder ExampleData to data

* Remove environment file

* API development

* Read the docs config

* Remove png from output formats

* Update Profile docs

* Update docstrings

* Fix imports

* Fix docs... again

* Update tests

* Update workflow

* Fix workflow

* Fix workflows

* Fix tests

* Delete requirements.txt

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Create VERSION

* Update README.md

* Settings becomes pydantic dataclass

* Update dependencies

* Update types

* Update pre-commit

* Update types, variable names

* Rename variables

* Add properties to Profile class

* Add properties for getting/setting BaSiC settings

* Add dct types to docs

* Add abstract base class for DCT

* Feat/benchmarks (#7)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Delete CONTRIBUTING.rst

* Changed project/repository names

* Changed to BaSiCPy

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Converted `BaSiC` to a pydantic class (#23)

* Converted BaSiC to pydantic class.

* Changed get_dark to get_darkfield

* Added CLI parsing (#26)

* Converted BaSiC to pydantic class.

* Created command line inputs.

* Added metavar indicator to input arguments.

* Check package installed before making DCT class

* Remove unused import

* Added abstract base class for DCT (#25)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Added predict method, does not include baseline estimation (#29)

* Implemented shading model fit algorithm from main branch (#27)

* Implemented shading model fit algorithm from main branch

* Fixed idct in inexact_alm.py

* Fix/fit (#34)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Set score values to object attributes

* Added caching to `predict` method (#35)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Set score values to object attributes

* Added caching for resizing of flatfield/darkfield

* Feat/autodoc pydantic (#38)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

* Implement autodoc_pydantic

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Feat/cleanup (#39)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

* Remove Profiles class and methods from basic

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Convert relative to absolute imports

* Add pytest-benchmark

* Update noxfile for pytest-benchmark

* ignored python-version

* Update .gitignore

* switched to commented line

* refactor name to basicpy

* further refactored...

* changed dir name

* refactor `predict` to `transform`

* Update README.md

change PyBaSiC to BaSiCPy

* Implement logging (#44)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* fixed main.py logging level and added warning in dct2d_tools.

* made BaSiC logger to file-level

* Feat/jax dct (#46)

* Add custom JAX idct function

* Refactor idct

* Fix tests

Co-authored-by: Fukai Yohsuke <[email protected]>

* Add save and load methods (#43)

* Add save and load methods

* Update docstring

* Update load_model test

* Add overwrite option and tests to save_model

* Update save/load tests

Co-authored-by: Fukai Yohsuke <[email protected]>

* Update README.md (#49)

* Update README.md

* Update README.md

* removed dropbox link from the link

* Update README.md

Co-authored-by: Fukai Yohsuke <[email protected]>

* fixed pydantic version lower bound to get test_basic_save_model
succeeded

* set the license to MIT

* removed pydantic version

* Testing fit method with synthetic data (#53)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* added synthesized fit test

* changed threshold for error

* fixed typo for sign

* added pooch for Zenodo datasets

* some fix...

* fixed pooch data bug

* Revert "added pooch for Zenodo datasets"

This reverts commit 4fdf0d7.

* removed logger from class

* refactor and added code for debug plotting in test_basic_fit_synthesized

* Data in package (#57)

* added files

* added fetch code to data module

* formatted data...

* added data download function

* added docstring

* updated comment

* added docstring for all

* added test download

* added pooch for dependency

* typo fix...

* fixed typo...

* parametrized tests

* disabled hash checking

* fixed bug

* Revert "fixed bug"

This reverts commit 6b24503.

* Revert "disabled hash checking"

This reverts commit d0dbd8a.

* added data and tests

* changed data url to the dev branch

* added tests

* added pydantic version to get test_basic_save_model succeed

* added tests for original dataset

* added code for original fetch

* added exception for invalid data cases

* reference commit with

* switched to dev branch

* Fixed typo for data.py comments

* Major update with LADMAP implementation (#62)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* Add custom JAX idct function

* Refactor idct

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* added synthesized fit test

* changed threshold for error

* fixed typo for sign

* added pooch for Zenodo datasets

* some fix...

* fixed pooch data bug

* Revert "added pooch for Zenodo datasets"

This reverts commit 4fdf0d7.

* wrote tests for experimental data

* added files

* added fetch code to data module

* formatted data...

* added data download function

* added docstring

* updated comment

* added docstring for all

* added test download

* added pooch for dependency

* typo fix...

* fixed typo...

* parametrized tests

* disabled hash checking

* fixed bug

* Revert "fixed bug"

This reverts commit 6b24503.

* Revert "disabled hash checking"

This reverts commit d0dbd8a.

* added jax experimental code and tested tingying's implementation

* experimental implementation wroking to approximate darkfield

* at least jit version is working

* deleted old version codes

* refactor to include D_R and D_Z

* added darkfield 1

* experimental code working

* loop jit

* fixed package name

* merged Tim's idct

* jax impl working with Tim's idct

* ladmap code is fed into the basicpy class (still a bit slow?)

* converted _fit_ladmap_single to the member method

* class method working...

* refactored jnp.newaxis

* in-class version working without jit

* trying to add pytree property to the model but aborting this approach

* writing jitted routine with new pydantic classes

* added fields to jax routines

* jit working

* all jax working!!

* apprioximate method not working with jax

* confirmed approximate fit working in jax

* organized files

* import organize

* single fit is working, but reweighting iteration is not working

* reweighting loops is working!

* still working example

* still running...

* resize implemented

* fixed resize in transform

* added _step_only_baseline for approximate fit

* custom weight calc working

* removed logger from class

* approximate fit optimization changed to original imp.

* working version with custom weight

* working before baseline calculation

* baseline calc working except approximate cases

* trying to get baseline work for approximate fit

* dropped python 3.7

* still approximate version not working...

* some working version

* approximate version working except baseline

* baseline calc not working for approximate cases

* sort_intensity option working for approximate fit!

* removed comments

* analyzing difference b/w original implementation

* refactor and added code for debug plotting in test_basic_fit_synthesized

* added data and tests

* changed data url to the dev branch

* added tests

* added pydantic version to get test_basic_save_model succeed

* added tests for original dataset

* added code for original fetch

* added exception for invalid data cases

* reference commit with

* switched to dev branch

* added comparison w/ reference implementation at this time

* Fixed typo for data.py comments

* updated notebook

* at least original impl is reproduced to some extent

* slightly changed transform tests to get them working

* Revert "Merge branch 'data_in_package' into jax_implementation"

This reverts commit 9d30051, reversing
changes made to a3d35ff.

* deleted experimental note

* added relative fitting weight argment

* added fitting results at commit 11b0747 for testing

* test rewriting but still failing

* test working

* added test with experimental data

* added pytest-datadir in nox

* migrated to pytest-datadir

* made darkfield sparsity coef tunable

* Revert "made darkfield sparsity coef tunable"

This reverts commit 2b8cd5d.

* loosen the isclose torelance

* fixed typo in tests

* deleted reference results

* removed reference results

* added additional darkfield coef and test passing

* trying to modify dimensions

* rewriting notebooks

* deleted previous notebooks

* notebook working

* readme updated

* example notebook added

* fixed type

* at least approximate method is working!

* fixed some bugs

* tests are running with loosened checks for experimental data

* added 3D DCT

* loosen the torelance for approximate fit

* loosen darkfield torelance for approximate fit

* changed DCT from 2D to 3D in ladmap fit

* loosen torelance for ladmap

* small modification

* added constraint for S mean

* get the test working

* before adding positivity constraint

* changed initial config

* added naive mean S regularization

* rewrote organizing test data notebook

* added value_diff for update torelance

* fixed convergence bug

* ladmap working with modified stopping cond

* working with modified stopping cond. (still harmonic ones not added)

* added test data

* removed cell output for jupyter notebook

* removed cell output for jupyter notebook

* updated test

* loosened the darkfield test constraint

* added guard for S==0

* updated example for lambda_flatfield_coef=100

* lambda_flatfield_coef=100 example update

* updated flatfield and darkfield dimension for 3d images

* updating organize test notebook

* updated notebook path

* notebook updated

* re-added python 3.7

* added 3D test

* refactored synthetic test

* fixed tests

* reduced z size in test

* added rescue for RESOURCE_EXHAUSTED: Out of memory

* updated enum type to str, Enum

Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Nicholas-Schaub <[email protected]>

* Update contribution notes (#66)

* initial draft

* added git version for the original

* 👥 Add @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Add @tying84 as a contributor

* 👥 Update @tying84 as a contributor

* 👥 Update @tying84 as a contributor

* 👥 Add @tdmorello as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Add @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* trying to see how all-contributors works

* reordered contributors

* regenerated readme

* commenting on contrinbution

* organized README

* updated nick email, added contributions to Tingying

* removed experimental_code reference

* updated jaxlib version to avoid importerror (#69)

* Merge Yu's docs

* deleting files not used at this point

* enab

* re-added autoflake and isort

* removed opencv

* Feat/release (#72)

* setup.cfg

* Create release pipeline

* Updated config and added new details to README

* Fixed build pipeline, added MANIFEST

* Fixed formatting issues

* Fixed typo in README

* Moved bumpversion config to separate file

* Fixed whitespace in bumpversion config

* Exclude bumpversion config from whitespace checks

* Bump version: 0.1.0-dev0 → 0.2.0-dev0

* Fix/release (#75)

* setup.cfg

* Create release pipeline

* Updated config and added new details to README

* Fixed build pipeline, added MANIFEST

* Fixed formatting issues

* Fixed typo in README

* Moved bumpversion config to separate file

* Fixed whitespace in bumpversion config

* Exclude bumpversion config from whitespace checks

* Bump version: 0.1.0-dev0 → 0.2.0-dev0

* Modified config, updated VERSION

* Bump version: 0.2.0-dev0 → 0.2.0-dev1

* Update secrets environment

* Bump version: 0.2.0-dev1 → 0.2.0-dev2

* Updated twine upload command

* Bump version: 0.2.0-dev2 → 0.2.0-dev3

* Updated CI pipeline

* Bump version: 0.2.0-dev3 → 0.2.0-dev4

* Use pypa publishing container

* Bump version: 0.2.0-dev4 → 0.2.0-dev5

* Update build

* Bump version: 0.2.0-dev5 → 0.2.0-dev6

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Add @YuLiu-web as a contributor

* update readme

* 👥 Update @tying84 as a contributor

* add tingying organizing meetings

* Fix `lambda flatfield` bug (#74)

* fixed bug for 3d images and lambda_flatfield calc

* reorganized test data

* Bump version: 0.2.0-dev6 → 0.2.1-dev0

* fixed save bug for 3D images and added tests (#77)

* updated jaxlib dependency

* updated notebooks

* 👥 Update @tdmorello as a contributor

* some update

* fixed typos

* Update docs

* fixed typos

* changed default parameters

* updated examples

* Add comments for notebook gallery

* Update notebooks

* compared lagrangian with the origin implementation

* notebook update

* formatted notebooks with black

* added markdowns to notebooks

* ifurther updated notebooks

* Add contributing guide to docs

* Add autoenum to docs

* updated notebooks

* updated thumbnail data for docs

* updated readme

* ignored package lock file for yarm

* fixed type

* added windows instruction

* 👥 Update @YuLiu-web as a contributor

* fixed notebook folder

* fixed typo

* added macos and windows tests

* test falling back to ubuntu

* added jax installation in noxfile for windows

* migrated contribution details to the docs

* update badges

* fixed badges

* a bit more update...

* fixed readme

* fixed badgee stype

* further fix...

* Timelapse option in `transform` (#80)

* udpated transform method

* fixed tests

* fixed typo

* solving problems with tests

* fixed transform bug

* removed tests for rescaling-fitting

* removed resized transform example

* added dask test

* set baseline type to optinal

* updated testdata

* deleted notebook outputs

* Update timelapse

* re-organized transform part

* tests running...

* made the resize method changable

* resize method tested

* tested resize method

* added comments for the transform function

* minor fix

* fixed fitting_weight shape bug

Co-authored-by: Tim Morello <[email protected]>

* Update example data (#92)

* updating notebooks

* updated rescaled data

* updated test data

* update notebooks

* Fixed most unit tests

* re-added reference comparison notebook

* re-added test data

Co-authored-by: Nicholas-Schaub <[email protected]>

* Fix `fit` bug for dask array inputs (#93)

* udpated transform method

* fixed tests

* fixed typo

* solving problems with tests

* fixed transform bug

* removed tests for rescaling-fitting

* removed resized transform example

* added dask test

* set baseline type to optinal

* updated testdata

* deleted notebook outputs

* Update timelapse

* re-organized transform part

* tests running...

* made the resize method changable

* resize method tested

* tested resize method

* added comments for the transform function

* minor fix

* fixed fitting_weight shape bug

* fixed dask-related bug

Co-authored-by: Tim Morello <[email protected]>

* Update hashes (#95)

* Minor change before beta release (#96)

* DCT tools update (#98)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* updated tests and fixed jax version to solve problems

* Update coef definitions (#103)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* trying to add darkfield weight and changing weight for ladmap

* fixed typo

* added attribute

* updated tests and fixed jax version to solve problems

* rewrite ladmap to normalize B

* working ver?

* updated test data

* udpated weight for darkfield

* updated test data

* removed print

* fixed normalization const

* added test to show basic does not depend on the intensity scale

* small update

* Update config names (#106)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* trying to add darkfield weight and changing weight for ladmap

* fixed typo

* added attribute

* updated tests and fixed jax version to solve problems

* rewrite ladmap to normalize B

* working ver?

* updated test data

* udpated weight for darkfield

* updated test data

* removed print

* fixed normalization const

* added test to show basic does not depend on the intensity scale

* small update

* updated coefs

* updated field names

* fixed typo in log

* updated the definition for self._sparse_cost_darkfield in approximate

* Update documents (#107)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* trying to add darkfield weight and changing weight for ladmap

* fixed typo

* added attribute

* updated tests and fixed jax version to solve problems

* rewrite ladmap to normalize B

* working ver?

* updated test data

* udpated weight for darkfield

* updated test data

* removed print

* fixed normalization const

* added test to show basic does not depend on the intensity scale

* small update

* updated coefs

* updated field names

* fixed typo in log

* updated parameters for notebooks

* added docs_build

* added ci for docs

* changed the notebooks so that it only installs basicpy for colab

* updated docs

* added pandoc in ci

* fixed error typo

* Cleaned up code and logging.

* Changed error message comment

* Add single-step convergence warning after the loop

* Cleanup (#109)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Cleaned up code and logging.

* Changed error message comment

* Add single-step convergence warning after the loop

Co-authored-by: Yohsuke T. Fukai <[email protected]>

* Change default log level to warning

* removed device variable from the basic object (#110)

Co-authored-by: Nicholas-Schaub <[email protected]>

* fixed requirements for gallery fix (#112)

* Bump version: 0.2.1-dev0 → 1.0.0

* Bump version: 0.2.1-dev0 → 1.0.0

* Updated docs version and bumpversion config

* Bump version: 1.0.0 → 1.0.1

Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>
Co-authored-by: Tingying Peng <[email protected]>
Co-authored-by: YuLiu-web <[email protected]>
Nicholas-Schaub added a commit that referenced this pull request Dec 15, 2022
* Add badges to README

* Update .gitignore

* Add noxfile

* Move files to src

* Add config files

* Placeholder dct functions

* Update gitignore

* Add class for settings

* Update setup.cfg

* Update requirements.txt

* Add docstring to settings class

* Add validation to Settings input

* Update github CI

* Add gpu option to init

* Update README formatting

* Rename 'figures' folder to 'resources'

* Update gitignore

* Update gitignore

* Move API from README to docs

* Refactor, update docstrings

* Change folder ExampleData to data

* Remove environment file

* API development

* Read the docs config

* Remove png from output formats

* Update Profile docs

* Update docstrings

* Fix imports

* Fix docs... again

* Update tests

* Update workflow

* Fix workflow

* Fix workflows

* Fix tests

* Delete requirements.txt

* Create VERSION

* Update README.md

* Settings becomes pydantic dataclass

* Update dependencies

* Update types

* Update pre-commit

* Update types, variable names

* Rename variables

* Add properties to Profile class

* Add properties for getting/setting BaSiC settings

* Add dct types to docs

* Add abstract base class for DCT

* Feat/benchmarks (#7)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Delete CONTRIBUTING.rst

* Changed project/repository names

* Changed to BaSiCPy

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Converted `BaSiC` to a pydantic class (#23)

* Converted BaSiC to pydantic class.

* Changed get_dark to get_darkfield

* Added CLI parsing (#26)

* Converted BaSiC to pydantic class.

* Created command line inputs.

* Added metavar indicator to input arguments.

* Check package installed before making DCT class

* Remove unused import

* Added abstract base class for DCT (#25)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Added predict method, does not include baseline estimation (#29)

* Implemented shading model fit algorithm from main branch (#27)

* Implemented shading model fit algorithm from main branch

* Fixed idct in inexact_alm.py

* Fix/fit (#34)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Set score values to object attributes

* Added caching to `predict` method (#35)

* Added benchmark library asv

* Revert "Delete CONTRIBUTING.rst"

This reverts commit 01d8fe1.

* Revert "Added benchmark library asv"

This reverts commit d067f04.

* Deleted CONTRIBUTING.rst

* Set score values to object attributes

* Added caching for resizing of flatfield/darkfield

* Feat/autodoc pydantic (#38)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

* Implement autodoc_pydantic

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Feat/cleanup (#39)

* Add abstract base class for DCT

* Update pre-commit

* Add dct backends and tests

* Update setup.cfg

* Quickfix for failing tests

* Update inexact_alm_rspca_l1.py (#1)

-  optimise SVD choice
- small other cleanup

* Check package installed before making DCT class

* Remove unused import

* Remove Profiles class and methods from basic

Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>

* Convert relative to absolute imports

* Add pytest-benchmark

* Update noxfile for pytest-benchmark

* ignored python-version

* Update .gitignore

* switched to commented line

* refactor name to basicpy

* further refactored...

* changed dir name

* refactor `predict` to `transform`

* Update README.md

change PyBaSiC to BaSiCPy

* Implement logging (#44)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* fixed main.py logging level and added warning in dct2d_tools.

* made BaSiC logger to file-level

* Feat/jax dct (#46)

* Add custom JAX idct function

* Refactor idct

* Fix tests

Co-authored-by: Fukai Yohsuke <[email protected]>

* Add save and load methods (#43)

* Add save and load methods

* Update docstring

* Update load_model test

* Add overwrite option and tests to save_model

* Update save/load tests

Co-authored-by: Fukai Yohsuke <[email protected]>

* Update README.md (#49)

* Update README.md

* Update README.md

* removed dropbox link from the link

* Update README.md

Co-authored-by: Fukai Yohsuke <[email protected]>

* fixed pydantic version lower bound to get test_basic_save_model
succeeded

* set the license to MIT

* removed pydantic version

* Testing fit method with synthetic data (#53)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* added synthesized fit test

* changed threshold for error

* fixed typo for sign

* added pooch for Zenodo datasets

* some fix...

* fixed pooch data bug

* Revert "added pooch for Zenodo datasets"

This reverts commit 4fdf0d7.

* removed logger from class

* refactor and added code for debug plotting in test_basic_fit_synthesized

* Data in package (#57)

* added files

* added fetch code to data module

* formatted data...

* added data download function

* added docstring

* updated comment

* added docstring for all

* added test download

* added pooch for dependency

* typo fix...

* fixed typo...

* parametrized tests

* disabled hash checking

* fixed bug

* Revert "fixed bug"

This reverts commit 6b24503.

* Revert "disabled hash checking"

This reverts commit d0dbd8a.

* added data and tests

* changed data url to the dev branch

* added tests

* added pydantic version to get test_basic_save_model succeed

* added tests for original dataset

* added code for original fetch

* added exception for invalid data cases

* reference commit with

* switched to dev branch

* Fixed typo for data.py comments

* Major update with LADMAP implementation (#62)

* added logging package to all

* trying to fix logging in main

* initial attempt to add logger logging to each files

* fixed typo

* fixed main logger typo

* Add custom JAX idct function

* Refactor idct

* fixed import style

* added actual logs in the calculation

* changed logger to class-level

* added log for inexact_alm

* added synthesized fit test

* changed threshold for error

* fixed typo for sign

* added pooch for Zenodo datasets

* some fix...

* fixed pooch data bug

* Revert "added pooch for Zenodo datasets"

This reverts commit 4fdf0d7.

* wrote tests for experimental data

* added files

* added fetch code to data module

* formatted data...

* added data download function

* added docstring

* updated comment

* added docstring for all

* added test download

* added pooch for dependency

* typo fix...

* fixed typo...

* parametrized tests

* disabled hash checking

* fixed bug

* Revert "fixed bug"

This reverts commit 6b24503.

* Revert "disabled hash checking"

This reverts commit d0dbd8a.

* added jax experimental code and tested tingying's implementation

* experimental implementation wroking to approximate darkfield

* at least jit version is working

* deleted old version codes

* refactor to include D_R and D_Z

* added darkfield 1

* experimental code working

* loop jit

* fixed package name

* merged Tim's idct

* jax impl working with Tim's idct

* ladmap code is fed into the basicpy class (still a bit slow?)

* converted _fit_ladmap_single to the member method

* class method working...

* refactored jnp.newaxis

* in-class version working without jit

* trying to add pytree property to the model but aborting this approach

* writing jitted routine with new pydantic classes

* added fields to jax routines

* jit working

* all jax working!!

* apprioximate method not working with jax

* confirmed approximate fit working in jax

* organized files

* import organize

* single fit is working, but reweighting iteration is not working

* reweighting loops is working!

* still working example

* still running...

* resize implemented

* fixed resize in transform

* added _step_only_baseline for approximate fit

* custom weight calc working

* removed logger from class

* approximate fit optimization changed to original imp.

* working version with custom weight

* working before baseline calculation

* baseline calc working except approximate cases

* trying to get baseline work for approximate fit

* dropped python 3.7

* still approximate version not working...

* some working version

* approximate version working except baseline

* baseline calc not working for approximate cases

* sort_intensity option working for approximate fit!

* removed comments

* analyzing difference b/w original implementation

* refactor and added code for debug plotting in test_basic_fit_synthesized

* added data and tests

* changed data url to the dev branch

* added tests

* added pydantic version to get test_basic_save_model succeed

* added tests for original dataset

* added code for original fetch

* added exception for invalid data cases

* reference commit with

* switched to dev branch

* added comparison w/ reference implementation at this time

* Fixed typo for data.py comments

* updated notebook

* at least original impl is reproduced to some extent

* slightly changed transform tests to get them working

* Revert "Merge branch 'data_in_package' into jax_implementation"

This reverts commit 9d30051, reversing
changes made to a3d35ff.

* deleted experimental note

* added relative fitting weight argment

* added fitting results at commit 11b0747 for testing

* test rewriting but still failing

* test working

* added test with experimental data

* added pytest-datadir in nox

* migrated to pytest-datadir

* made darkfield sparsity coef tunable

* Revert "made darkfield sparsity coef tunable"

This reverts commit 2b8cd5d.

* loosen the isclose torelance

* fixed typo in tests

* deleted reference results

* removed reference results

* added additional darkfield coef and test passing

* trying to modify dimensions

* rewriting notebooks

* deleted previous notebooks

* notebook working

* readme updated

* example notebook added

* fixed type

* at least approximate method is working!

* fixed some bugs

* tests are running with loosened checks for experimental data

* added 3D DCT

* loosen the torelance for approximate fit

* loosen darkfield torelance for approximate fit

* changed DCT from 2D to 3D in ladmap fit

* loosen torelance for ladmap

* small modification

* added constraint for S mean

* get the test working

* before adding positivity constraint

* changed initial config

* added naive mean S regularization

* rewrote organizing test data notebook

* added value_diff for update torelance

* fixed convergence bug

* ladmap working with modified stopping cond

* working with modified stopping cond. (still harmonic ones not added)

* added test data

* removed cell output for jupyter notebook

* removed cell output for jupyter notebook

* updated test

* loosened the darkfield test constraint

* added guard for S==0

* updated example for lambda_flatfield_coef=100

* lambda_flatfield_coef=100 example update

* updated flatfield and darkfield dimension for 3d images

* updating organize test notebook

* updated notebook path

* notebook updated

* re-added python 3.7

* added 3D test

* refactored synthetic test

* fixed tests

* reduced z size in test

* added rescue for RESOURCE_EXHAUSTED: Out of memory

* updated enum type to str, Enum

Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Nicholas-Schaub <[email protected]>

* Update contribution notes (#66)

* initial draft

* added git version for the original

* 👥 Add @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Update @yfukai as a contributor

* 👥 Add @tying84 as a contributor

* 👥 Update @tying84 as a contributor

* 👥 Update @tying84 as a contributor

* 👥 Add @tdmorello as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Add @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @Nicholas-Schaub as a contributor

* trying to see how all-contributors works

* reordered contributors

* regenerated readme

* commenting on contrinbution

* organized README

* updated nick email, added contributions to Tingying

* removed experimental_code reference

* updated jaxlib version to avoid importerror (#69)

* Merge Yu's docs

* deleting files not used at this point

* enab

* re-added autoflake and isort

* removed opencv

* Feat/release (#72)

* setup.cfg

* Create release pipeline

* Updated config and added new details to README

* Fixed build pipeline, added MANIFEST

* Fixed formatting issues

* Fixed typo in README

* Moved bumpversion config to separate file

* Fixed whitespace in bumpversion config

* Exclude bumpversion config from whitespace checks

* Bump version: 0.1.0-dev0 → 0.2.0-dev0

* Fix/release (#75)

* setup.cfg

* Create release pipeline

* Updated config and added new details to README

* Fixed build pipeline, added MANIFEST

* Fixed formatting issues

* Fixed typo in README

* Moved bumpversion config to separate file

* Fixed whitespace in bumpversion config

* Exclude bumpversion config from whitespace checks

* Bump version: 0.1.0-dev0 → 0.2.0-dev0

* Modified config, updated VERSION

* Bump version: 0.2.0-dev0 → 0.2.0-dev1

* Update secrets environment

* Bump version: 0.2.0-dev1 → 0.2.0-dev2

* Updated twine upload command

* Bump version: 0.2.0-dev2 → 0.2.0-dev3

* Updated CI pipeline

* Bump version: 0.2.0-dev3 → 0.2.0-dev4

* Use pypa publishing container

* Bump version: 0.2.0-dev4 → 0.2.0-dev5

* Update build

* Bump version: 0.2.0-dev5 → 0.2.0-dev6

* 👥 Update @Nicholas-Schaub as a contributor

* 👥 Update @tdmorello as a contributor

* 👥 Add @YuLiu-web as a contributor

* update readme

* 👥 Update @tying84 as a contributor

* add tingying organizing meetings

* Fix `lambda flatfield` bug (#74)

* fixed bug for 3d images and lambda_flatfield calc

* reorganized test data

* Bump version: 0.2.0-dev6 → 0.2.1-dev0

* fixed save bug for 3D images and added tests (#77)

* updated jaxlib dependency

* updated notebooks

* 👥 Update @tdmorello as a contributor

* some update

* fixed typos

* Update docs

* fixed typos

* changed default parameters

* updated examples

* Add comments for notebook gallery

* Update notebooks

* compared lagrangian with the origin implementation

* notebook update

* formatted notebooks with black

* added markdowns to notebooks

* ifurther updated notebooks

* Add contributing guide to docs

* Add autoenum to docs

* updated notebooks

* updated thumbnail data for docs

* updated readme

* ignored package lock file for yarm

* fixed type

* added windows instruction

* 👥 Update @YuLiu-web as a contributor

* fixed notebook folder

* fixed typo

* added macos and windows tests

* test falling back to ubuntu

* added jax installation in noxfile for windows

* migrated contribution details to the docs

* update badges

* fixed badges

* a bit more update...

* fixed readme

* fixed badgee stype

* further fix...

* Timelapse option in `transform` (#80)

* udpated transform method

* fixed tests

* fixed typo

* solving problems with tests

* fixed transform bug

* removed tests for rescaling-fitting

* removed resized transform example

* added dask test

* set baseline type to optinal

* updated testdata

* deleted notebook outputs

* Update timelapse

* re-organized transform part

* tests running...

* made the resize method changable

* resize method tested

* tested resize method

* added comments for the transform function

* minor fix

* fixed fitting_weight shape bug

Co-authored-by: Tim Morello <[email protected]>

* Update example data (#92)

* updating notebooks

* updated rescaled data

* updated test data

* update notebooks

* Fixed most unit tests

* re-added reference comparison notebook

* re-added test data

Co-authored-by: Nicholas-Schaub <[email protected]>

* Fix `fit` bug for dask array inputs (#93)

* udpated transform method

* fixed tests

* fixed typo

* solving problems with tests

* fixed transform bug

* removed tests for rescaling-fitting

* removed resized transform example

* added dask test

* set baseline type to optinal

* updated testdata

* deleted notebook outputs

* Update timelapse

* re-organized transform part

* tests running...

* made the resize method changable

* resize method tested

* tested resize method

* added comments for the transform function

* minor fix

* fixed fitting_weight shape bug

* fixed dask-related bug

Co-authored-by: Tim Morello <[email protected]>

* Update hashes (#95)

* Minor change before beta release (#96)

* DCT tools update (#98)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* updated tests and fixed jax version to solve problems

* Update coef definitions (#103)

* updated hashes to newer ones

* changed source to main

* ci experiment with dev branch

* updated branch

* added tests for wrong argments

* updated classifier to beta

* updated copyright

* just added spaces

* updated downsampled data to Zenodo

* renamed data to datasets

* typo fix...

* deleted depricated tools

* fixed package names ...

* import fix

* 👥 Update @yfukai as a contributor

* added my contribution for ci...

* removed deprecated routines from old DCT2d

* trying to add darkfield weight and changing weight for ladmap

* fixed typo

* added attribute

* updated tests and fixed jax version to solve problems

* rewrite ladmap to normalize B

* working ver?

* updated test data

* udpated weight for darkfield

* updated test data

* removed print

* fixed normalization const

* added test to show basic does not depend on the intensity scale

* small update

* added installation doc for M1 chips

* updated installation instruction for M1 mac

* reverted some merge changes

* Update README.md

Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Tim Morello <[email protected]>
Co-authored-by: Nicholas-Schaub <[email protected]>
Co-authored-by: Mahdi Lamb <[email protected]>
Co-authored-by: Tingying Peng <[email protected]>
Co-authored-by: YuLiu-web <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants