Merge pull request #1834 from pypeit/aat_uhrf

Adding support for the (decommissioned) spectrograph: AAT/UHRF
pypeit · Aug 30, 2024 · bd77c38 · bd77c38
2 parents 745f36c + 905d775
commit bd77c38
Show file tree

Hide file tree

Showing 60 changed files with 883 additions and 361 deletions.
diff --git a/deprecated/arc_old.py b/deprecated/arc_old.py
@@ -855,3 +855,53 @@ def saturation_mask(a, satlevel):
 
     return mask.astype(int)
 
+def mask_around_peaks(spec, inbpm):
+    """
+    Find peaks in the input spectrum and mask pixels around them.
+
+    All pixels to the left and right of a peak is masked until
+    a pixel has a lower value than the adjacent pixel. At this
+    point, we assume that spec has reached the noise level.
+
+    Parameters
+    ----------
+    spec: `numpy.ndarray`_
+        Spectrum (1D array) in counts
+    inbpm: `numpy.ndarray`_
+        Input bad pixel mask
+
+    Returns
+    -------
+    outbpm: `numpy.ndarray`_
+        Bad pixel mask with pixels around peaks masked
+    """
+    # Find the peak locations
+    pks = detect_peaks(spec)
+
+    # Initialise some useful variables and the output bpm
+    xarray = np.arange(spec.size)
+    specdiff = np.append(np.diff(spec), 0.0)
+    outbpm = inbpm.copy()
+
+    # Loop over the peaks and mask pixels around them
+    for i in range(len(pks)):
+        # Find all pixels to the left of the peak that are above the noise level
+        wl = np.where((xarray <= pks[i]) & (specdiff > 0.0))[0]
+        ww = (pks[i]-wl)[::-1]
+        # Find the first pixel to the left of the peak that is below the noise level
+        nmask = np.where(np.diff(ww) > 1)[0]
+        if nmask.size != 0 and nmask[0] > 5:
+            # Mask all pixels to the left of the peak
+            mini = max(0,wl.size-nmask[0]-1)
+            outbpm[wl[mini]:pks[i]] = True
+        # Find all pixels to the right of the peak that are above the noise level
+        ww = np.where((xarray >= pks[i]) & (specdiff < 0.0))[0]
+        # Find the first pixel to the right of the peak that is below the noise level
+        nmask = np.where(np.diff(ww) > 1)[0]
+        if nmask.size != 0 and nmask[0] > 5:
+            # Mask all pixels to the right of the peak
+            maxi = min(nmask[0], ww.size)
+            outbpm[pks[i]:ww[maxi]+2] = True
+    # Return the output bpm
+    return outbpm
+
diff --git a/doc/api/pypeit.scripts.extract_datacube.rst b/doc/api/pypeit.scripts.extract_datacube.rst
@@ -0,0 +1,8 @@
+pypeit.scripts.extract\_datacube module
+=======================================
+
+.. automodule:: pypeit.scripts.extract_datacube
+   :members:
+   :private-members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/doc/api/pypeit.scripts.rst b/doc/api/pypeit.scripts.rst
@@ -26,6 +26,7 @@ Submodules
    pypeit.scripts.compare_sky
    pypeit.scripts.compile_wvarxiv
    pypeit.scripts.edge_inspector
+   pypeit.scripts.extract_datacube
    pypeit.scripts.flux_calib
    pypeit.scripts.flux_setup
    pypeit.scripts.identify

diff --git a/doc/api/pypeit.spectrographs.aat_uhrf.rst b/doc/api/pypeit.spectrographs.aat_uhrf.rst
@@ -0,0 +1,8 @@
+pypeit.spectrographs.aat\_uhrf module
+=====================================
+
+.. automodule:: pypeit.spectrographs.aat_uhrf
+   :members:
+   :private-members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/doc/api/pypeit.spectrographs.rst b/doc/api/pypeit.spectrographs.rst
@@ -7,6 +7,7 @@ Submodules
 .. toctree::
    :maxdepth: 4
 
+   pypeit.spectrographs.aat_uhrf
    pypeit.spectrographs.bok_bc
    pypeit.spectrographs.gemini_flamingos
    pypeit.spectrographs.gemini_gmos

diff --git a/doc/coadd3d.rst b/doc/coadd3d.rst
@@ -155,7 +155,7 @@ There are several recommended steps of the coadd3d process that can be run separ
 
         pypeit_sensfunc spec1d_StandardStarName.fits -o sens_StandardStarName.fits
 
-    For further details, see :doc:`_sensitivity_function`.
+    For further details, see :ref:`sensitivity_function`.
 
 #. Step 4 - Generate a datacube of the science exposures. This is done by running the following command:
 
@@ -164,7 +164,7 @@ There are several recommended steps of the coadd3d process that can be run separ
         pypeit_coadd_datacube ScienceName.coadd3d -o
 
     Note that you will need to specify the sensitivity function file using the ``sensfile`` option in the
-    :doc:`coadd3d_file` file. For further details, see :ref:`coadd3d_fluxing`.
+    :ref:`coadd3d_file` file. For further details, see :ref:`coadd3d_fluxing`.
 
 Combination options
 ===================
@@ -239,7 +239,7 @@ If you would like to flux calibrate your datacube, you need to
 produce your standard star datacube first. Then extract the spectrum
 of the standard star using the ``pypeit_extract_datacube`` script. This
 will produce a ``spec1d`` file that you will need to use to generate a
-sensitivity function in the usual way (see :doc:`_sensitivity_function`).
+sensitivity function in the usual way (see :ref:`sensitivity_function`).
 Then, when generating the datacube of the science frame you must include
 the name of the sensitivity function in your ``coadd3d`` file as follows:
 
@@ -289,7 +289,7 @@ then you can specify the ``skysub_frame`` in the ``spec2d`` block of the
 above. If you have dedicated sky frames, then it is generally
 recommended to reduce these frames as if they are regular science
 frames, but add the following keyword arguments at the top of your
-:doc:`coadd3d_file`:
+:ref:`coadd3d_file`:
 
 .. code-block:: ini
 

diff --git a/doc/cookbook.rst b/doc/cookbook.rst
@@ -70,7 +70,8 @@ what we recommend:
  - We will refer to that folder as ``RAWDIR``
 
 The raw images can be gzip-compressed, although this means opening files will be
-slower.
+slower.  See :ref:`setup-file-searching` for specific comments about the files
+in your raw directory.
 
 A word on calibration data
 --------------------------

diff --git a/doc/help/pypeit_cache_github_data.rst b/doc/help/pypeit_cache_github_data.rst
@@ -9,7 +9,7 @@
     Script to download/cache PypeIt github data
     
     positional arguments:
-      spectrograph          A valid spectrograph identifier: bok_bc,
+      spectrograph          A valid spectrograph identifier: aat_uhrf, bok_bc,
                             gemini_flamingos1, gemini_flamingos2,
                             gemini_gmos_north_e2v, gemini_gmos_north_ham,
                             gemini_gmos_north_ham_ns, gemini_gmos_south_ham,

diff --git a/doc/help/pypeit_chk_for_calibs.rst b/doc/help/pypeit_chk_for_calibs.rst
@@ -13,7 +13,7 @@
     options:
       -h, --help            show this help message and exit
       -s SPECTROGRAPH, --spectrograph SPECTROGRAPH
-                            A valid spectrograph identifier: bok_bc,
+                            A valid spectrograph identifier: aat_uhrf, bok_bc,
                             gemini_flamingos1, gemini_flamingos2,
                             gemini_gmos_north_e2v, gemini_gmos_north_ham,
                             gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
@@ -35,8 +35,11 @@
                             vlt_xshooter_nir, vlt_xshooter_uvb, vlt_xshooter_vis,
                             wht_isis_blue, wht_isis_red (default: None)
       -e EXTENSION, --extension EXTENSION
-                            File extension; compression indicators (e.g. .gz) not
-                            required. (default: .fits)
+                            File extension to use. Must include the period (e.g.,
+                            ".fits") and it must be one of the allowed extensions
+                            for this spectrograph. If None, root directory will be
+                            searched for all files with any of the allowed
+                            extensions. (default: None)
       --save_setups         If not toggled, remove setup_files/ folder and its
                             files. (default: False)
     
diff --git a/doc/help/pypeit_extract_datacube.rst b/doc/help/pypeit_extract_datacube.rst
@@ -0,0 +1,29 @@
+.. code-block:: console
+
+    $ pypeit_extract_datacube -h
+    usage: pypeit_extract_datacube [-h] [-e EXT_FILE] [-s SAVE] [-o]
+                                   [-b BOXCAR_RADIUS] [-v VERBOSITY]
+                                   file
+    
+    Read in a datacube, extract a spectrum of a point source,and save it as a spec1d
+    file.
+    
+    positional arguments:
+      file                  spec3d.fits DataCube file
+    
+    options:
+      -h, --help            show this help message and exit
+      -e EXT_FILE, --ext_file EXT_FILE
+                            Configuration file with extraction parameters (default:
+                            None)
+      -s SAVE, --save SAVE  Output spec1d filename (default: None)
+      -o, --overwrite       Overwrite any existing files/directories (default:
+                            False)
+      -b BOXCAR_RADIUS, --boxcar_radius BOXCAR_RADIUS
+                            Radius of the circular boxcar (in arcseconds) to use for
+                            the extraction. (default: None)
+      -v VERBOSITY, --verbosity VERBOSITY
+                            Verbosity level between 0 [none] and 2 [all]. Default:
+                            1. Level 2 writes a log with filename
+                            extract_datacube_YYYYMMDD-HHMM.log (default: 1)
+    
diff --git a/doc/help/pypeit_obslog.rst b/doc/help/pypeit_obslog.rst
@@ -10,7 +10,7 @@
     using PypeItMetaData.
     
     positional arguments:
-      spec                  A valid spectrograph identifier: bok_bc,
+      spec                  A valid spectrograph identifier: aat_uhrf, bok_bc,
                             gemini_flamingos1, gemini_flamingos2,
                             gemini_gmos_north_e2v, gemini_gmos_north_ham,
                             gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
@@ -75,8 +75,11 @@
       -s SORT, --sort SORT  Metadata keyword (pypeit-specific) to use to sort the
                             output table. (default: mjd)
       -e EXTENSION, --extension EXTENSION
-                            File extension; compression indicators (e.g. .gz) not
-                            required. (default: .fits)
+                            File extension to use. Must include the period (e.g.,
+                            ".fits") and it must be one of the allowed extensions
+                            for this spectrograph. If None, root directory will be
+                            searched for all files with any of the allowed
+                            extensions. (default: None)
       -d OUTPUT_PATH, --output_path OUTPUT_PATH
                             Path to top-level output directory. (default: current
                             working directory)

diff --git a/doc/help/pypeit_ql.rst b/doc/help/pypeit_ql.rst
@@ -17,7 +17,7 @@
     Script to produce quick-look PypeIt reductions
     
     positional arguments:
-      spectrograph          A valid spectrograph identifier: bok_bc,
+      spectrograph          A valid spectrograph identifier: aat_uhrf, bok_bc,
                             gemini_flamingos1, gemini_flamingos2,
                             gemini_gmos_north_e2v, gemini_gmos_north_ham,
                             gemini_gmos_north_ham_ns, gemini_gmos_south_ham,

diff --git a/doc/help/pypeit_sensfunc.rst b/doc/help/pypeit_sensfunc.rst
@@ -1,9 +1,9 @@
 .. code-block:: console
 
     $ pypeit_sensfunc -h
-    usage: pypeit_sensfunc [-h] [--algorithm {UVIS,IR}] [--multi MULTI] [-o OUTFILE]
-                           [-s SENS_FILE] [-f FLATFILE] [--debug]
-                           [--par_outfile PAR_OUTFILE] [-v VERBOSITY]
+    usage: pypeit_sensfunc [-h] [--extr {OPT,BOX}] [--algorithm {UVIS,IR}]
+                           [--multi MULTI] [-o OUTFILE] [-s SENS_FILE] [-f FLATFILE]
+                           [--debug] [--par_outfile PAR_OUTFILE] [-v VERBOSITY]
                            spec1dfile
     
     Compute a sensitivity function
@@ -14,6 +14,16 @@
     
     options:
       -h, --help            show this help message and exit
+      --extr {OPT,BOX}      Override the default extraction method used for
+                            computing the sensitivity function.  Note that it is not
+                            possible to set --extr and simultaneously use a .sens
+                            file with the --sens_file option. If you are using a
+                            .sens file, set the algorithm there via:
+                             
+                                [sensfunc]
+                                     extr = BOX
+                             
+                            The extraction options are: OPT or BOX
       --algorithm {UVIS,IR}
                             Override the default algorithm for computing the
                             sensitivity function.  Note that it is not possible to
@@ -62,11 +72,13 @@
       -s SENS_FILE, --sens_file SENS_FILE
                             Configuration file with sensitivity function parameters
       -f FLATFILE, --flatfile FLATFILE
-                            Use the flat file for computing the sensitivity
-                            function.  Note that it is not possible to set
-                            --flatfile and simultaneously use a .sens file with the
-                            --sens_file option. If you are using a .sens file, set
-                            the flatfile there via e.g.:
+                            Use a flat calibration file to compute the blaze
+                            function when generating the sensitivity function.  This
+                            is helpful to account for small scale undulations in the
+                            sensitivity function.  Note that it is not possible to
+                            set --flatfile and simultaneously use a .sens file with
+                            the --sens_file option. If you are using a .sens file,
+                            set the flatfile there via e.g.:
                              
                                 [sensfunc]
                                      flatfile = Calibrations/Flat_A_0_DET01.fits

diff --git a/doc/help/pypeit_setup.rst b/doc/help/pypeit_setup.rst
@@ -11,7 +11,7 @@
     options:
       -h, --help            show this help message and exit
       -s SPECTROGRAPH, --spectrograph SPECTROGRAPH
-                            A valid spectrograph identifier: bok_bc,
+                            A valid spectrograph identifier: aat_uhrf, bok_bc,
                             gemini_flamingos1, gemini_flamingos2,
                             gemini_gmos_north_e2v, gemini_gmos_north_ham,
                             gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
@@ -39,8 +39,11 @@
                             --extension option to set the types of files to search
                             for. (default: current working directory)
       -e EXTENSION, --extension EXTENSION
-                            File extension; compression indicators (e.g. .gz) not
-                            required. (default: .fits)
+                            File extension to use. Must include the period (e.g.,
+                            ".fits") and it must be one of the allowed extensions
+                            for this spectrograph. If None, root directory will be
+                            searched for all files with any of the allowed
+                            extensions. (default: None)
       -d OUTPUT_PATH, --output_path OUTPUT_PATH
                             Path to top-level output directory. (default: current
                             working directory)

diff --git a/doc/help/pypeit_trace_edges.rst b/doc/help/pypeit_trace_edges.rst
@@ -26,16 +26,16 @@
                             default mosaic. (default: None)
       -s SPECTROGRAPH, --spectrograph SPECTROGRAPH
                             A valid spectrograph identifier, which is only used if
-                            providing files directly: bok_bc, gemini_flamingos1,
-                            gemini_flamingos2, gemini_gmos_north_e2v,
-                            gemini_gmos_north_ham, gemini_gmos_north_ham_ns,
-                            gemini_gmos_south_ham, gemini_gnirs_echelle,
-                            gemini_gnirs_ifu, gtc_maat, gtc_osiris, gtc_osiris_plus,
-                            jwst_nircam, jwst_nirspec, keck_deimos, keck_esi,
-                            keck_hires, keck_kcrm, keck_kcwi, keck_lris_blue,
-                            keck_lris_blue_orig, keck_lris_red, keck_lris_red_mark4,
-                            keck_lris_red_orig, keck_mosfire, keck_nires,
-                            keck_nirspec_high, keck_nirspec_high_old,
+                            providing files directly: aat_uhrf, bok_bc,
+                            gemini_flamingos1, gemini_flamingos2,
+                            gemini_gmos_north_e2v, gemini_gmos_north_ham,
+                            gemini_gmos_north_ham_ns, gemini_gmos_south_ham,
+                            gemini_gnirs_echelle, gemini_gnirs_ifu, gtc_maat,
+                            gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
+                            keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
+                            keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
+                            keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire,
+                            keck_nires, keck_nirspec_high, keck_nirspec_high_old,
                             keck_nirspec_low, lbt_luci1, lbt_luci2, lbt_mods1b,
                             lbt_mods1r, lbt_mods2b, lbt_mods2r, ldt_deveny,
                             magellan_fire, magellan_fire_long, magellan_mage,

diff --git a/doc/help/pypeit_view_fits.rst b/doc/help/pypeit_view_fits.rst
@@ -9,7 +9,7 @@
     View FITS files with ginga
     
     positional arguments:
-      spectrograph         A valid spectrograph identifier: bok_bc,
+      spectrograph         A valid spectrograph identifier: aat_uhrf, bok_bc,
                            gemini_flamingos1, gemini_flamingos2,
                            gemini_gmos_north_e2v, gemini_gmos_north_ham,
                            gemini_gmos_north_ham_ns, gemini_gmos_south_ham,

diff --git a/doc/help/run_pypeit.rst b/doc/help/run_pypeit.rst
@@ -7,11 +7,11 @@
     ##  [1;37;42mPypeIt : The Python Spectroscopic Data Reduction Pipeline v1.16.1.dev109+g885cb1823[0m
     ##  
     ##  Available spectrographs include:
-    ##   bok_bc, gemini_flamingos1, gemini_flamingos2, gemini_gmos_north_e2v,
-    ##   gemini_gmos_north_ham, gemini_gmos_north_ham_ns,
-    ##   gemini_gmos_south_ham, gemini_gnirs_echelle, gemini_gnirs_ifu,
-    ##   gtc_maat, gtc_osiris, gtc_osiris_plus, jwst_nircam, jwst_nirspec,
-    ##   keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
+    ##   aat_uhrf, bok_bc, gemini_flamingos1, gemini_flamingos2,
+    ##   gemini_gmos_north_e2v, gemini_gmos_north_ham,
+    ##   gemini_gmos_north_ham_ns, gemini_gmos_south_ham, gemini_gnirs_echelle,
+    ##   gemini_gnirs_ifu, gtc_maat, gtc_osiris, gtc_osiris_plus, jwst_nircam,
+    ##   jwst_nirspec, keck_deimos, keck_esi, keck_hires, keck_kcrm, keck_kcwi,
     ##   keck_lris_blue, keck_lris_blue_orig, keck_lris_red,
     ##   keck_lris_red_mark4, keck_lris_red_orig, keck_mosfire, keck_nires,
     ##   keck_nirspec_high, keck_nirspec_high_old, keck_nirspec_low, lbt_luci1,

diff --git a/doc/include/class_datamodel_pypeitimage.rst b/doc/include/class_datamodel_pypeitimage.rst
@@ -10,6 +10,7 @@ Attribute         Type
 ``det_img``       `numpy.ndarray`_                                                                                     `numpy.integer`_   If a detector mosaic, this image provides the detector that contributed to each pixel.                                                                                                  
 ``detector``      :class:`~pypeit.images.detector_container.DetectorContainer`, :class:`~pypeit.images.mosaic.Mosaic`                     The detector (see :class:`~pypeit.images.detector_container.DetectorContainer`) or mosaic (see :class:`~pypeit.images.mosaic.Mosaic`) parameters                                        
 ``exptime``       int, float                                                                                                              Effective exposure time (s)                                                                                                                                                             
+``filename``      str                                                                                                                     Filename for the image                                                                                                                                                                  
 ``fullmask``      :class:`~pypeit.images.imagebitmask.ImageBitMaskArray`                                                                  Image mask                                                                                                                                                                              
 ``image``         `numpy.ndarray`_                                                                                     `numpy.floating`_  Primary image data                                                                                                                                                                      
 ``img_scale``     `numpy.ndarray`_                                                                                     `numpy.floating`_  Image count scaling applied (e.g., 1/flat-field)