diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..ae3da63 --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +Permission is hereby granted, free of charge, to any person or organization +obtaining a copy of the software and accompanying documentation covered by +this license (the "Software") to use, reproduce, display, distribute, +execute, and transmit the Software, and to prepare derivative works of the +Software, and to permit third-parties to whom the Software is furnished to +do so, all subject to the following: + +The copyright notices in the Software and this entire statement, including +the above license grant, this restriction and the following disclaimer, +must be included in all copies of the Software, in whole or in part, and +all derivative works of the Software, unless such copies or derivative +works are solely in the form of machine-executable object code generated by +a source language processor. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT +SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE +FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, +ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER +DEALINGS IN THE SOFTWARE. diff --git a/README.md b/README.md index a33e24d..78c1f03 100644 --- a/README.md +++ b/README.md @@ -5,16 +5,15 @@ This code implements a differentiable proxy model for simulating meta-optics and a neural feature propagation deconvolution method. These components are optimized end-to-end using machine learning optimizers. -The experimental results from the manuscript and the supplemental information are reproducible with this implementation. The proposed differentiable proxy model, neural feature propagation, and end-to-end optimization framework are implemented completely in TensorFlow, without dependency on third-party libraries. While the fastest inference and training times are achieved by running on state-of-the-art GPUs, this code can be run solely with standard TensorFlow packages, ensuring high portability and ease of reproducibility. - +The experimental results from the manuscript and the supplemental information are reproducible with this implementation. The proposed differentiable proxy model, neural feature propagation, and end-to-end optimization framework are implemented completely in TensorFlow, without dependency on third-party libraries. ## Training To perform end-to-end training (of meta-optic and deconvolution) execute the 'run_train.sh' script. The model checkpoint which includes saved parameters for both the meta-optic and deconvolution will be saved to 'training/ckpt'. The folder 'training/data' contains a subset of the training and test data that we used for optimizing our end-to-end imaging pipeline. - ## Testing To perform inference on real-world captures launch the "test.ipynb" notebook in Jupyter Notebook and step through the cells. The notebook will load in a finetuned checkpoint of our neural feature propagation network from 'experimental/ckpt' which will process captured sensor measurements located in 'experimental/data'. The reconstructed images will be displayed within the notebook. +Additional captured sensor measurements can be found in the [data repository](https://drive.google.com/drive/folders/1fsAvN9MPtN5jJPeIFjWuLUY9Hp8NNkar?usp=sharing). ## Requirements This code has been tested with Python 3.6.10 using TensorFlow 2.2.0 running on Linux with an Nvidia P100 GPU with 16GB RAM. @@ -29,3 +28,21 @@ Scipy matplotlib jupyter-notebook ``` + +## Citation +If you find our work useful in your research, please cite: +``` +@article{Tseng2021NeuralNanoOptics, + title = "Neural Nano-Optics for High-quality Thin Lens Imaging", + author = "Tseng, Ethan and Colburn, Shane and Whitehead, James and Huang, Luocheng + and Baek, Seung-Hwan and Majumdar, Arka and Heide, Felix", + journal = "Nature Communications", + volume = , + number = , + pages = , + year = 2021 +} +``` + +## License +Our code is licensed under BSL-1. By downloading the software, you agree to the terms of this License. The training data in the folder 'training/data' comes from the [INRIA Holidays Dataset](https://lear.inrialpes.fr/~jegou/data.php). diff --git a/test.ipynb b/test.ipynb index d9be124..79ee9d7 100644 --- a/test.ipynb +++ b/test.ipynb @@ -6,7 +6,7 @@ "source": [ "## Inference Code for 'Neural Nano-Optics for High-quality Thin Lens Imaging'\n", "\n", - "#### This Jupyter Notebook can be used to produce the experimental reconstructions shown in the manuscript and in the supplemental information." + "#### This notebook can be used to produce the experimental reconstructions shown in the manuscript and in the supplemental information." ] }, {