Merge pull request #158 from EhsanGharibNezhad/main

main -> dev

README.md

@@ -2,20 +2,48 @@
 
 ![logo](logo_TelescopeML.png)
 
-** `TelescopeML`: Deep Convolutional Neural Networks and Machine Learning Models for Analyzing Stellar and Exoplanetary Telescope Spectra**
-
-## Brief introduction
-
 ``TelescopeML`` is a Python package comprising a series of modules, each equipped with specialized machine learning and
 statistical capabilities for conducting Convolutional Neural Networks (CNN) or Machine Learning (ML) training on
-datasets captured from the atmospheres of extrasolar planets and brown dwarfs. 
+datasets captured from the atmospheres of extrasolar planets and brown dwarfs. The tasks executed by the ``TelescopeML``
+modules are outlined below:
+
+
+- *DataMaster module*: Performs various tasks to process the datasets, including:
+
+    - Preparing inputs and outputs
+    - Splitting the dataset into training, validation, and test sets
+    - Scaling/normalizing the data
+    - Visualizing the data
+    - Conducting feature engineering
+
+- *DeepTrainer module*: Utilizes different methods/packages such as TensorFlow to:
+
+  - Build Convolutional Neural Networks (CNNs) model using the training examples
+  - Utilize tuned hyperparameters
+  - Fit/train the ML models
+  - Visualize the loss and training history, as well as the trained model's performance
+
+- *Predictor module*: Implements the following tasks to predict atmospheric parameters:
+
+  - Processes and predicts the observational datasets
+  - Deploys the trained ML/CNNs model to predict atmospheric parameters
+  - Visualizes the processed observational dataset and the uncertainty in the predicted results
+
+- *StatVisAnalyzer module*: Provides a set of functions to perform the following tasks:
+
+  - Explores and processes the synthetic datasets
+  - Performs the chi-square test to evaluate the similarity between two datasets
+  - Calculates confidence intervals and standard errors
+  - Functions to visualize the datasets, including scatter plots, histograms, boxplots
+
+or simply...
+
+    - Load the pre-trained CNN models based on the latest synthetic datasets
+    - Predict the stellar/(exo-)planetary parameters
+    - Report the statistical analysis
 
-## Main Features and Modules:
+![](docs/figures/TelescopeML_modules.png)
 
-- *StatVisAnalyzer*: Explore and process the synthetic datasets (or the training examples) and perform statistical analysis.
-- *DeepBuilder*: Specify training and target features, normalize/scale datasets, and construct a CNN model.
-- *DeepTrainer*: Create an ML model, train the model with the training examples, and utilize hyperparameters.
-- *Predictor*: Train the module using specified hyperparameters.
 
 
 ## Documentation
@@ -24,5 +52,4 @@ datasets captured from the atmospheres of extrasolar planets and brown dwarfs.
 - Installation: https://ehsangharibnezhad.github.io/TelescopeML/installation.html
 - Tutorials: https://ehsangharibnezhad.github.io/TelescopeML/tutorials.html
 - The code: https://ehsangharibnezhad.github.io/TelescopeML/code.html
-- Concepts: https://ehsangharibnezhad.github.io/TelescopeML/knowledgebase.html
 

docs/Makefile

@@ -6,7 +6,7 @@
 SPHINXOPTS    ?=
 SPHINXBUILD   ?= sphinx-build
 SOURCEDIR     = .
-BUILDDIR      = ../../TelescopeML_docs/TelescopeML_docs
+BUILDDIR      = ../../TelescopeML_docs/
 
 # Put it first so that "make" without argument is like "make help".
 help:

docs/index.rst

@@ -63,9 +63,11 @@ or simply...
    Installation <installation>
    Tutorials <tutorials>
    The Code <code>
-   KnowledgeBase <knowledgebase> 
    Github <https://github.com/ehsangharibnezhad>
-   Publications <publications>
-   What to Cite <cite>
+
+
+.. KnowledgeBase <knowledgebase>
+.. Publications <publications>
+.. What to Cite <cite>
 
 

publications/joss/paper.bib

@@ -1,3 +1,109 @@
+
+
+@article{KIRANYAZ2021,
+title = {1D convolutional neural networks and applications: A survey},
+journal = {Mechanical Systems and Signal Processing},
+volume = {151},
+pages = {107398},
+year = {2021},
+issn = {0888-3270},
+doi = {https://doi.org/10.1016/j.ymssp.2020.107398},
+url = {https://www.sciencedirect.com/science/article/pii/S0888327020307846},
+author = {Serkan Kiranyaz and Onur Avci and Osama Abdeljaber and Turker Ince and Moncef Gabbouj and Daniel J. Inman},
+keywords = {Artificial Neural Networks, Machine learning, Deep learning, Convolutional neural networks, Structural health monitoring, Condition monitoring, Arrhythmia detection and identification, Fault detection, Structural damage detection},
+abstract = {During the last decade, Convolutional Neural Networks (CNNs) have become the de facto standard for various Computer Vision and Machine Learning operations. CNNs are feed-forward Artificial Neural Networks (ANNs) with alternating convolutional and subsampling layers. Deep 2D CNNs with many hidden layers and millions of parameters have the ability to learn complex objects and patterns providing that they can be trained on a massive size visual database with ground-truth labels. With a proper training, this unique ability makes them the primary tool for various engineering applications for 2D signals such as images and video frames. Yet, this may not be a viable option in numerous applications over 1D signals especially when the training data is scarce or application specific. To address this issue, 1D CNNs have recently been proposed and immediately achieved the state-of-the-art performance levels in several applications such as personalized biomedical data classification and early diagnosis, structural health monitoring, anomaly detection and identification in power electronics and electrical motor fault detection. Another major advantage is that a real-time and low-cost hardware implementation is feasible due to the simple and compact configuration of 1D CNNs that perform only 1D convolutions (scalar multiplications and additions). This paper presents a comprehensive review of the general architecture and principals of 1D CNNs along with their major engineering applications, especially focused on the recent progress in this field. Their state-of-the-art performance is highlighted concluding with their unique properties. The benchmark datasets and the principal 1D CNN software used in those applications are also publicly shared in a dedicated website. While there has not been a paper on the review of 1D CNNs and its applications in the literature, this paper fulfills this gap.}
+}
+
+
+@book{Goodfellow_2016DeepLearning,
+    title={Deep Learning},
+    author={Ian Goodfellow and Yoshua Bengio and Aaron Courville},
+    publisher={MIT Press},
+    note={\url{http://www.deeplearningbook.org}},
+    year={2016}
+}
+
+@ARTICLE{Zingales2018ExoGAN,
+       author = {{Zingales}, Tiziano and {Waldmann}, Ingo P.},
+        title = "{ExoGAN: Retrieving Exoplanetary Atmospheres Using Deep Convolutional Generative Adversarial Networks}",
+      journal = {\aj},
+     keywords = {methods: statistical, planets and satellites: atmospheres, radiative transfer, techniques: spectroscopic, Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Earth and Planetary Astrophysics},
+         year = 2018,
+        month = dec,
+       volume = {156},
+       number = {6},
+          eid = {268},
+        pages = {268},
+          doi = {10.3847/1538-3881/aae77c},
+archivePrefix = {arXiv},
+       eprint = {1806.02906},
+ primaryClass = {astro-ph.IM},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2018AJ....156..268Z},
+      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
+}
+
+@ARTICLE{Miles2023ApJ,
+       author = {{Miles}, Brittany E. and {Biller}, Beth A. and {Patapis}, Polychronis and {Worthen}, Kadin and {Rickman}, Emily and {Hoch}, Kielan K.~W. and {Skemer}, Andrew and {Perrin}, Marshall D. and {Whiteford}, Niall and {Chen}, Christine H. and {Sargent}, B. and {Mukherjee}, Sagnick and {Morley}, Caroline V. and {Moran}, Sarah E. and {Bonnefoy}, Mickael and {Petrus}, Simon and {Carter}, Aarynn L. and {Choquet}, Elodie and {Hinkley}, Sasha and {Ward-Duong}, Kimberly and {Leisenring}, Jarron M. and {Millar-Blanchaer}, Maxwell A. and {Pueyo}, Laurent and {Ray}, Shrishmoy and {Sallum}, Steph and {Stapelfeldt}, Karl R. and {Stone}, Jordan M. and {Wang}, Jason J. and {Absil}, Olivier and {Balmer}, William O. and {Boccaletti}, Anthony and {Bonavita}, Mariangela and {Booth}, Mark and {Bowler}, Brendan P. and {Chauvin}, Gael and {Christiaens}, Valentin and {Currie}, Thayne and {Danielski}, Camilla and {Fortney}, Jonathan J. and {Girard}, Julien H. and {Grady}, Carol A. and {Greenbaum}, Alexandra Z. and {Henning}, Thomas and {Hines}, Dean C. and {Janson}, Markus and {Kalas}, Paul and {Kammerer}, Jens and {Kennedy}, Grant M. and {Kenworthy}, Matthew A. and {Kervella}, Pierre and {Lagage}, Pierre-Olivier and {Lew}, Ben W.~P. and {Liu}, Michael C. and {Macintosh}, Bruce and {Marino}, Sebastian and {Marley}, Mark S. and {Marois}, Christian and {Matthews}, Elisabeth C. and {Matthews}, Brenda C. and {Mawet}, Dimitri and {McElwain}, Michael W. and {Metchev}, Stanimir and {Meyer}, Michael R. and {Molliere}, Paul and {Pantin}, Eric and {Quirrenbach}, Andreas and {Rebollido}, Isabel and {Ren}, Bin B. and {Schneider}, Glenn and {Vasist}, Malavika and {Wyatt}, Mark C. and {Zhou}, Yifan and {Briesemeister}, Zackery W. and {Bryan}, Marta L. and {Calissendorff}, Per and {Cantalloube}, Faustine and {Cugno}, Gabriele and {De Furio}, Matthew and {Dupuy}, Trent J. and {Factor}, Samuel M. and {Faherty}, Jacqueline K. and {Fitzgerald}, Michael P. and {Franson}, Kyle and {Gonzales}, Eileen C. and {Hood}, Callie E. and {Howe}, Alex R. and {Kraus}, Adam L. and {Kuzuhara}, Masayuki and {Lagrange}, Anne-Marie and {Lawson}, Kellen and {Lazzoni}, Cecilia and {Liu}, Pengyu and {Llop-Sayson}, Jorge and {Lloyd}, James P. and {Martinez}, Raquel A. and {Mazoyer}, Johan and {Quanz}, Sascha P. and {Redai}, Jea Adams and {Samland}, Matthias and {Schlieder}, Joshua E. and {Tamura}, Motohide and {Tan}, Xianyu and {Uyama}, Taichi and {Vigan}, Arthur and {Vos}, Johanna M. and {Wagner}, Kevin and {Wolff}, Schuyler G. and {Ygouf}, Marie and {Zhang}, Xi and {Zhang}, Keming and {Zhang}, Zhoujian},
+        title = "{The JWST Early-release Science Program for Direct Observations of Exoplanetary Systems II: A 1 to 20 {\ensuremath{\mu}}m Spectrum of the Planetary-mass Companion VHS 1256-1257 b}",
+      journal = {ApJL},
+     keywords = {Brown dwarfs, Exoplanet atmospheres, Extrasolar gaseous giant planets, 185, 487, 509, Astrophysics - Earth and Planetary Astrophysics, Astrophysics - Solar and Stellar Astrophysics},
+         year = 2023,
+        month = mar,
+       volume = {946},
+       number = {1},
+          eid = {L6},
+        pages = {L6},
+          doi = {10.3847/2041-8213/acb04a},
+archivePrefix = {arXiv},
+       eprint = {2209.00620},
+ primaryClass = {astro-ph.EP},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2023ApJ...946L...6M},
+      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
+}
+
+
+
+@ARTICLE{Alderson2023Nature,
+       author = {{Alderson}, Lili and {Wakeford}, Hannah R. and {Alam}, Munazza K. and {Batalha}, Natasha E. and {Lothringer}, Joshua D. and {Adams Redai}, Jea and {Barat}, Saugata and {Brande}, Jonathan and {Damiano}, Mario and {Daylan}, Tansu and {Espinoza}, N{\'e}stor and {Flagg}, Laura and {Goyal}, Jayesh M. and {Grant}, David and {Hu}, Renyu and {Inglis}, Julie and {Lee}, Elspeth K.~H. and {Mikal-Evans}, Thomas and {Ramos-Rosado}, Lakeisha and {Roy}, Pierre-Alexis and {Wallack}, Nicole L. and {Batalha}, Natalie M. and {Bean}, Jacob L. and {Benneke}, Bj{\"o}rn and {Berta-Thompson}, Zachory K. and {Carter}, Aarynn L. and {Changeat}, Quentin and {Col{\'o}n}, Knicole D. and {Crossfield}, Ian J.~M. and {D{\'e}sert}, Jean-Michel and {Foreman-Mackey}, Daniel and {Gibson}, Neale P. and {Kreidberg}, Laura and {Line}, Michael R. and {L{\'o}pez-Morales}, Mercedes and {Molaverdikhani}, Karan and {Moran}, Sarah E. and {Morello}, Giuseppe and {Moses}, Julianne I. and {Mukherjee}, Sagnick and {Schlawin}, Everett and {Sing}, David K. and {Stevenson}, Kevin B. and {Taylor}, Jake and {Aggarwal}, Keshav and {Ahrer}, Eva-Maria and {Allen}, Natalie H. and {Barstow}, Joanna K. and {Bell}, Taylor J. and {Blecic}, Jasmina and {Casewell}, Sarah L. and {Chubb}, Katy L. and {Crouzet}, Nicolas and {Cubillos}, Patricio E. and {Decin}, Leen and {Feinstein}, Adina D. and {Fortney}, Joanthan J. and {Harrington}, Joseph and {Heng}, Kevin and {Iro}, Nicolas and {Kempton}, Eliza M. -R. and {Kirk}, James and {Knutson}, Heather A. and {Krick}, Jessica and {Leconte}, J{\'e}r{\'e}my and {Lendl}, Monika and {MacDonald}, Ryan J. and {Mancini}, Luigi and {Mansfield}, Megan and {May}, Erin M. and {Mayne}, Nathan J. and {Miguel}, Yamila and {Nikolov}, Nikolay K. and {Ohno}, Kazumasa and {Palle}, Enric and {Parmentier}, Vivien and {Petit dit de la Roche}, Dominique J.~M. and {Piaulet}, Caroline and {Powell}, Diana and {Rackham}, Benjamin V. and {Redfield}, Seth and {Rogers}, Laura K. and {Rustamkulov}, Zafar and {Tan}, Xianyu and {Tremblin}, P. and {Tsai}, Shang-Min and {Turner}, Jake D. and {de Val-Borro}, Miguel and {Venot}, Olivia and {Welbanks}, Luis and {Wheatley}, Peter J. and {Zhang}, Xi},
+        title = "{Early Release Science of the exoplanet WASP-39b with JWST NIRSpec G395H}",
+      journal = {Nature},
+     keywords = {Astrophysics - Earth and Planetary Astrophysics, Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Solar and Stellar Astrophysics},
+         year = 2023,
+        month = feb,
+       volume = {614},
+       number = {7949},
+        pages = {664-669},
+          doi = {10.1038/s41586-022-05591-3},
+archivePrefix = {arXiv},
+       eprint = {2211.10488},
+ primaryClass = {astro-ph.EP},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2023Natur.614..664A},
+      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
+}
+
+
+
+@ARTICLE{JWST2023PASP,
+       author = {{Gardner}, Jonathan P. and {Mather}, John C. and {Abbott}, Randy and {Abell}, James S. and {Abernathy}, Mark and {Abney}, Faith E. and {Abraham}, John G. and {Abraham}, Roberto and {Abul-Huda}, Yasin M. and {Acton}, Scott and et al.},
+        title = "{The James Webb Space Telescope Mission}",
+      journal = {\pasp},
+     keywords = {Space vehicle instruments, Astronomical instrumentation, Infrared astronomy, Infrared observatories, Space observatories, History of astronomy, 1548, 799, 786, 791, 1543, 1868, Astrophysics - Instrumentation and Methods for Astrophysics},
+         year = 2023,
+        month = jun,
+       volume = {135},
+       number = {1048},
+          eid = {068001},
+        pages = {068001},
+          doi = {10.1088/1538-3873/acd1b5},
+archivePrefix = {arXiv},
+       eprint = {2304.04869},
+ primaryClass = {astro-ph.IM},
+       adsurl = {https://ui.adsabs.harvard.edu/abs/2023PASP..135f8001G},
+      adsnote = {Provided by the SAO/NASA Astrophysics Data System}
+}
+
+
+
 @ARTICLE{SpectRes,
        author = {{Carnall}, A.~C.},
         title = "{SpectRes: A Fast Spectral Resampling Tool in Python}",

publications/joss/paper.md

@@ -26,7 +26,6 @@ authors:
     orcid: 0000-0002-2188-0807
     affiliation: "3, 4"
   - name: Gopal Nookula 
-    orcid: (Ask Gopal to fix this!)
     affiliation: "5"
 affiliations:
   - name: Space Science and Astrobiology Division, NASA Ames Research Center, Moffett Field, CA, 94035 USA
@@ -40,7 +39,7 @@ affiliations:
   - name: Department of Computer Science, University of California, Riverside, Riverside, CA 92507 USA
     index: 5
 
-date: 13 November 2023
+date: 17 November 2023
 bibliography: paper.bib
 
 aas-doi: LINK OF ApJ PAPER
@@ -50,27 +49,23 @@ aas-journal: The Astrophysical Journal
 
 # Statement of Need
 
-We are on the verge of a revolutionary era in space exploration, thanks to advancements in ground- and space-based 
-telescopes, such as the James Webb and CRIRES. These remarkable instruments collect an enormous amount of data 
-from extrasolar atmospheres [e.g., @bean2018transiting]. Without an accurate interpretation of this data, the main objectives 
-of space missions will not be fully accomplished. Different analytical and statistical methods, such as the chi-test and 
+We are on in a new era of space exploration, thanks to advancements in ground- and space-based 
+telescopes, such as the James Webb Space Telescope [e.g., @JWST2023PASP] and CRIRES. These remarkable instruments collect high-resolution, high-signal-to-noise spectra 
+from extrasolar planets [e.g., @Alderson2023Nature], and brown dwarfs [e.g., @Miles2023ApJ] atmospheres. Without an accurate interpretation of this data, the main objectives 
+of space missions will not be fully accomplished. Different analytical and statistical methods, such as the chi-squared-test, 
 Bayesian statistics as well as radiative-transfer atmospheric modeling packages have been developed 
-[e.g., @batalha2017pandexo; @batalha2019picaso; @MacDonald2023POSEIDON]
-and are utilized in the context of forward- and retrieval-radiative transfer modeling to analyze these datasets and 
-extract crucial information, such as atmospheric temperature, metallicity, carbon-to-oxygen ratio, and surface gravity 
-[@line2014systematic; @Iyer2023Sphinx; @Marley2015]. These atmospheric models rely on the synthetic training 
-datasets generated by simulating the physics and chemistry of these atmospheres for a wide range of thermal structures 
-and compositions. In addition, machine learning and deep learning methods have been developed in recent years 
+[e.g., @batalha2019picaso; @MacDonald2023POSEIDON] to interpret the spectra. They utilize either forward- and/or retrieval-radiative transfer modeling to analyze the spectra and 
+extract physical information, such as atmospheric temperature, metallicity, carbon-to-oxygen ratio, and surface gravity 
+[@line2014systematic; @Iyer2023Sphinx; @Marley2015]. These atmospheric models rely on generating the physics and chemistry of these atmospheres for a wide range of thermal structures 
+and compositions. In addition to Bayesian-based techniques, machine learning and deep learning methods have been developed in recent years 
 for various astronomical problems, including confirming the classification of light curves for 
-exoplanet validation [e.g., @Valizadegan2021] as well as interpreting brown dwarfs spectra using Random Forest technique 
-[e.g., @Lueber2023RandomForesr_BDs]. However, to the best of
-our knowledge, this is the first time, deep learning and convolutional neural networks is implemented on brown dwarf 
-atmospheric datasets to predict parameters such as temperature and gravity simultaneously.  
-
-With the continuous observation of these objects and the increasing amount of data coming to Earth each day, there is a 
-critical need for a systematic pipeline to explore the datasets and extract important information from them. This 
-invaluable information will be utilized to provide insights about the diversity of exoplanets and brown dwarfs' 
-atmospheric compositions and will help facilitate the long-term analysis of this data in research. `TelescopeML`
+exoplanet validation [e.g., @Valizadegan2021], recognizing molecular features  [@Zingales2018ExoGAN] as well as interpreting brown dwarfs spectra using Random Forest technique 
+[e.g., @Lueber2023RandomForesr_BDs]. Here, we present one of the first applications of deep learning and convolutional neural networks on the interpretation brown dwarf 
+atmospheric datasets. The configuration of a CNN and the key concepts can be found in [@Goodfellow_2016DeepLearning; @KIRANYAZ2021].
+
+With the continuous observation of these objects and the increasing amount of data, there is a 
+critical need for a systematic pipeline to quickly explore the datasets and extract important physical from them. In the future we can expand our pipeline to exoplanet atmospheres, and use it to provide insights about the diversity of exoplanets and brown dwarfs' 
+atmospheric compositions. Ultimately,  `TelescopeML` will help facilitate the long-term analysis of this data in research. `TelescopeML`
 is an ML Python package with Sphinx-ed user-friendly documentation that provides both trained ML models and ML tools 
 for interpreting observational data captured by telescopes.