Skip to content

Commit

Permalink
Upload license.
Browse files Browse the repository at this point in the history
  • Loading branch information
Ethan-Tseng committed Nov 2, 2021
1 parent 6255745 commit b061f3d
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 4 deletions.
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:

The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
23 changes: 20 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,15 @@

This code implements a differentiable proxy model for simulating meta-optics and a neural feature propagation deconvolution method. These components are optimized end-to-end using machine learning optimizers.

The experimental results from the manuscript and the supplemental information are reproducible with this implementation. The proposed differentiable proxy model, neural feature propagation, and end-to-end optimization framework are implemented completely in TensorFlow, without dependency on third-party libraries. While the fastest inference and training times are achieved by running on state-of-the-art GPUs, this code can be run solely with standard TensorFlow packages, ensuring high portability and ease of reproducibility.

The experimental results from the manuscript and the supplemental information are reproducible with this implementation. The proposed differentiable proxy model, neural feature propagation, and end-to-end optimization framework are implemented completely in TensorFlow, without dependency on third-party libraries.

## Training
To perform end-to-end training (of meta-optic and deconvolution) execute the 'run_train.sh' script. The model checkpoint which includes saved parameters for both the meta-optic and deconvolution will be saved to 'training/ckpt'. The folder 'training/data' contains a subset of the training and test data that we used for optimizing our end-to-end imaging pipeline.


## Testing
To perform inference on real-world captures launch the "test.ipynb" notebook in Jupyter Notebook and step through the cells. The notebook will load in a finetuned checkpoint of our neural feature propagation network from 'experimental/ckpt' which will process captured sensor measurements located in 'experimental/data'. The reconstructed images will be displayed within the notebook.

Additional captured sensor measurements can be found in the [data repository](https://drive.google.com/drive/folders/1fsAvN9MPtN5jJPeIFjWuLUY9Hp8NNkar?usp=sharing).

## Requirements
This code has been tested with Python 3.6.10 using TensorFlow 2.2.0 running on Linux with an Nvidia P100 GPU with 16GB RAM.
Expand All @@ -29,3 +28,21 @@ Scipy
matplotlib
jupyter-notebook
```

## Citation
If you find our work useful in your research, please cite:
```
@article{Tseng2021NeuralNanoOptics,
title = "Neural Nano-Optics for High-quality Thin Lens Imaging",
author = "Tseng, Ethan and Colburn, Shane and Whitehead, James and Huang, Luocheng
and Baek, Seung-Hwan and Majumdar, Arka and Heide, Felix",
journal = "Nature Communications",
volume = ,
number = ,
pages = ,
year = 2021
}
```

## License
Our code is licensed under BSL-1. By downloading the software, you agree to the terms of this License. The training data in the folder 'training/data' comes from the [INRIA Holidays Dataset](https://lear.inrialpes.fr/~jegou/data.php).
2 changes: 1 addition & 1 deletion test.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"source": [
"## Inference Code for 'Neural Nano-Optics for High-quality Thin Lens Imaging'\n",
"\n",
"#### This Jupyter Notebook can be used to produce the experimental reconstructions shown in the manuscript and in the supplemental information."
"#### This notebook can be used to produce the experimental reconstructions shown in the manuscript and in the supplemental information."
]
},
{
Expand Down

0 comments on commit b061f3d

Please sign in to comment.