Data product docs #37

cmccully · 2025-01-28T15:53:57Z

This PR adds documentation explaining the data format of the reduction products of the pipeline.

sfoale · 2025-01-29T11:31:29Z

docs/README.rst

 ----------------------------------------

 .. image:: http://img.shields.io/badge/powered%20by-AstroPy-orange.svg?style=flat
    :target: http://www.astropy.org
    :alt: Powered by Astropy Badge

+BANZAI-FLOYDS is the Las Cumbres Observatory pipeline to process `FLOYDS <https://lco.global/observatory/instruments/floyds/>` data. We currently have two FLOYDS insturments in operation: one on Haleakala in Hawaii and one at Siding Springs in Australia. The FLOYDS spectrographs are low-resolution, workhorse spectrographs for supernovae and other explosive transients. FLOYDS has a resolution of R~500 and covers from 320nm - 1000nm. FLOYDS is double dispersed so the images of the slit are curved on the detector (similar to eschelle spectrographs). The sky lines are also tilted to better sample the line spread funciton to enable better sky subtraction.


instruments typo
function typo

sfoale · 2025-01-29T11:36:53Z

docs/banzai_floyds/data-products.rst

+  are 'order', 'wavelength', 'center', 'center_error', 'sigma', and 'sigma_error'. These points are estimated by taking
+  slices in the y-direction and stepping along the dispersion direction.
+
+- 'PRFOILE' Extension: This extension has a 2-D image of the profile weights. This is for convenience so the user can


PROFILE rather than PRFOILE?

sfoale · 2025-01-29T11:40:17Z

docs/banzai_floyds/processing.rst

+
+Profile Fitting
+---------------
+To estimte the center of the profile that we want to extract, we perform a match filter with a Gaussian, fixing Gaussian's


Typo in estimate

sfoale · 2025-01-29T11:41:43Z

docs/banzai_floyds/processing.rst

+----------------------------------------
+Flux standards are observed periodically and are calibrated with known reference fluxes. The README in the 
+banzai_floyds/data/standards directory provides more details. The telluric features are treated as a multiplicative 
+correction and are based on the same flux standard observations. The initial sensitivity function to disenangle the 


Typo in disentangle

sfoale

A few typos but looks good.

jchate6

I'm not sure exactly what's going on, but when I try to build the docs myself locally using make html in the docs directory, I get the following error:

Configuration error:
There is a programmable error in your configuration file:

Traceback (most recent call last):
  File "/home/jchatelain/.local/lib/python3.12/site-packages/sphinx/config.py", line 358, in eval_config_file
    exec(code, namespace)  # NoQA: S102
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jchatelain/git/banzai-floyds/docs/conf.py", line 79, in <module>
    import_module(setup_cfg['name'])
  File "/usr/lib64/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'banzai_floyds'

make: *** [Makefile:44: html] Error 2

I'm not sure what the import_module() method does, but it seems to be having some issues here or I'm doing something wrong.

jchate6 · 2025-01-29T20:02:23Z

docs/README.rst

 ----------------------------------------

 .. image:: http://img.shields.io/badge/powered%20by-AstroPy-orange.svg?style=flat
    :target: http://www.astropy.org
    :alt: Powered by Astropy Badge

+BANZAI-FLOYDS is the Las Cumbres Observatory pipeline to process `FLOYDS <https://lco.global/observatory/instruments/floyds/>` data. We currently have two FLOYDS insturments in operation: one on Haleakala in Hawaii and one at Siding Springs in Australia. The FLOYDS spectrographs are low-resolution, workhorse spectrographs for supernovae and other explosive transients. FLOYDS has a resolution of R~500 and covers from 320nm - 1000nm. FLOYDS is double dispersed so the images of the slit are curved on the detector (similar to eschelle spectrographs). The sky lines are also tilted to better sample the line spread funciton to enable better sky subtraction.


you need a "_" after the link for it to work:

`FLOYDS <https://lco.global/observatory/instruments/floyds/>`_ data

jchate6 · 2025-01-29T20:05:32Z

docs/README.rst

 ----------------------------------------

 .. image:: http://img.shields.io/badge/powered%20by-AstroPy-orange.svg?style=flat
    :target: http://www.astropy.org
    :alt: Powered by Astropy Badge

+BANZAI-FLOYDS is the Las Cumbres Observatory pipeline to process `FLOYDS <https://lco.global/observatory/instruments/floyds/>` data. We currently have two FLOYDS insturments in operation: one on Haleakala in Hawaii and one at Siding Springs in Australia. The FLOYDS spectrographs are low-resolution, workhorse spectrographs for supernovae and other explosive transients. FLOYDS has a resolution of R~500 and covers from 320nm - 1000nm. FLOYDS is double dispersed so the images of the slit are curved on the detector (similar to eschelle spectrographs). The sky lines are also tilted to better sample the line spread funciton to enable better sky subtraction.


I don't think we should limit ourselves to explosive transients. A non-negligible portion of Floyds spectra go towards classifying asteroids, for example.

jchate6 · 2025-01-29T20:10:54Z

docs/README.rst

+
+The dataflow and infrastructure of this pipeline relies heavily on `BANZAI
+<https://github.com/lcogt/banzai>`_, enabling this repo to focus on analysis that is specific to slit spectroscopy.
+
 Installation
 ------------
 ```


This is improperly formatted.
Use

.. code-block:: sh pip install .

Instead (Make sure you have an empty space after too.)

You can also get away with just

:: pip install .

I think...

Below this in the license, if you want to include the actual copyright symbol, you can do

This project is Copyright |copy| cmccully and licensed under ... .. |copy| unicode:: 0xA9 .. copyright sign

jchate6 · 2025-01-29T21:22:44Z

docs/banzai_floyds/data-products.rst

+Data Products
+=============
+The banzai-floyds data products are split into a variety of files so that users only need to download the files
+they need for their analysis. All intermediate products are available to users to enable debugging of any reduction


" the files required for their analysis." sounds better

jchate6 · 2025-01-29T21:53:18Z

docs/banzai_floyds/data-products.rst

+Extracted spectra are in files with the '1d' suffix and 'SPECTRUM' OBSTYPE, following a naming convention like
+"ogg2m001-en06-20250111-0056-e91-1d.fits.fz". The extracted files are multi-extension fits files.
+
+- 'SPECTRUM' Extension: This extension includes what is considered the final reduction of the spectrum.


I would emphasize the extension name by making it bold

jchate6 · 2025-01-29T22:41:37Z

docs/banzai_floyds/data-products.rst

+
+Fringe Frames
+-------------
+Combined (stacked) lamp flat exposures are used to correct for fringing and have the '-lampflat' filename suffix.


The fringe frames have the -lampflat filename suffix?

jchate6 · 2025-01-29T22:44:38Z

docs/banzai_floyds/data-products.rst

+----------
+HgAr exposures are used for wavelength calibration. These frames have an 'a91' filename suffix and an OBSTYPE of ARC. The
+'SCI', 'BPM', 'ERR', and 'ORDER_COEFFS' are the same as the 2-D spectroscopic images. The only difference is that the
+'WAVELENGTHS' extension is derived from this frame rather than being copied in.  are the same as the 2-D spectroscopic images. Science frames reference the arc that provided the WAVELENGTH extension via the L1IDARC header keyword. The


Remove "are the same as the 2-D spectroscopic images."

jchate6 · 2025-01-29T22:52:31Z

docs/banzai_floyds/processing.rst

+We use a HgAr arc lamp frame to find the wavelength solution. We use the arc frame taken closest to the science
+exposure, preferably in the same block. If an arc in the same block does not exist, we fall back to other arcs taken
+under the same program, and finally to any arc taken in the same configuration that is now public. 
+We employ a matched filter model that uses lines in banzai_floyds.arc_lines as the reference spectrum. 


`banzai_floyds.arc_lines`

jchate6 · 2025-01-29T22:56:11Z

docs/banzai_floyds/processing.rst

+
+Flux Calibration and Telluric Correction
+----------------------------------------
+Flux standards are observed periodically and are calibrated with known reference fluxes. The README in the 


This 'README' should probably be a link.

jchate6 · 2025-01-29T22:57:02Z

docs/banzai_floyds/processing.rst

+Flux Calibration and Telluric Correction
+----------------------------------------
+Flux standards are observed periodically and are calibrated with known reference fluxes. The README in the 
+banzai_floyds/data/standards directory provides more details. The telluric features are treated as a multiplicative 


`banzai_floyds/data/standards`

Added data product documentation

28b5fe6

cmccully requested review from sfoale and jchate6 January 28, 2025 15:53

cmccully changed the base branch from main to faint-trace January 28, 2025 15:54

cmccully changed the base branch from faint-trace to order-positions January 28, 2025 15:54

sfoale reviewed Jan 29, 2025

View reviewed changes

sfoale approved these changes Jan 29, 2025

View reviewed changes

Updated for upstream fix to utcnow datetimes.

365729c

jchate6 requested changes Jan 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Data product docs #37

Data product docs #37

cmccully commented Jan 28, 2025

sfoale Jan 29, 2025 •

edited

Loading

sfoale Jan 29, 2025

sfoale Jan 29, 2025

sfoale Jan 29, 2025

sfoale left a comment

jchate6 left a comment

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

jchate6 Jan 29, 2025

Data product docs #37

Are you sure you want to change the base?

Data product docs #37

Conversation

cmccully commented Jan 28, 2025

sfoale Jan 29, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfoale left a comment

Choose a reason for hiding this comment

jchate6 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sfoale Jan 29, 2025 •

edited

Loading