Merge pull request #2473 from cta-observatory/irf-maker

Irf maker and cut optimiser

docs/api-reference/irf/benchmarks.rst

@@ -0,0 +1,12 @@
+.. _benchmarks:
+
+**********
+Benchmarks
+**********
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.benchmarks
+    :no-inheritance-diagram:

docs/api-reference/irf/binning.rst

@@ -0,0 +1,12 @@
+.. _binning:
+
+*******
+Binning
+*******
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.binning
+    :no-inheritance-diagram:

docs/api-reference/irf/index.rst

@@ -0,0 +1,44 @@
+.. _irf:
+
+**********************************************
+Instrument Response Functions (`~ctapipe.irf`)
+**********************************************
+
+.. currentmodule:: ctapipe.irf
+
+This module contains functionalities for generating instrument response functions.
+The simulated events used for this have to be selected based on their associated "gammaness"
+value and (optionally) their reconstructed angular offset from their point of origin.
+The code for doing this can found in :ref:`cut_optimization` and is intended for use via the
+`~ctapipe.tools.optimize_event_selection.EventSelectionOptimizer` tool.
+
+The generation of the irf components themselves is implemented in :ref:`irfs` and is intended for
+use via the `~ctapipe.tools.compute_irf.IrfTool` tool.
+This tool can optionally also compute some common benchmarks, which are implemented in :ref:`benchmarks`.
+
+The cut optimization as well as the calculations of the irf components and the benchmarks
+are done using the `pyirf <https://pyirf.readthedocs.io/en/stable/>`_ package.
+
+:ref:`binning`, :ref:`preprocessing`, and :ref:`spectra` contain helper functions and classes used by many of the
+other components in this module.
+
+
+Submodules
+==========
+
+.. toctree::
+    :maxdepth: 1
+
+    optimize
+    irfs
+    benchmarks
+    binning
+    preprocessing
+    spectra
+
+
+Reference/API
+=============
+
+.. automodapi:: ctapipe.irf
+    :no-inheritance-diagram:

docs/api-reference/irf/irfs.rst

@@ -0,0 +1,12 @@
+.. _irfs:
+
+**************
+IRF components
+**************
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.irfs
+    :no-inheritance-diagram:

docs/api-reference/irf/optimize.rst

@@ -0,0 +1,12 @@
+.. _cut_optimization:
+
+********************************
+G/H (and Theta) Cut Optimization
+********************************
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.optimize
+    :no-inheritance-diagram:

docs/api-reference/irf/preprocessing.rst

@@ -0,0 +1,12 @@
+.. _preprocessing:
+
+*******************************
+Event Loading and Preprocessing
+*******************************
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.preprocessing
+    :no-inheritance-diagram:

docs/api-reference/irf/spectra.rst

@@ -0,0 +1,12 @@
+.. _spectra:
+
+*************************************
+Spectra definitions for event weights
+*************************************
+
+
+Reference/ API
+==============
+
+.. automodapi:: ctapipe.irf.spectra
+    :no-inheritance-diagram:

docs/api-reference/tools/index.rst

@@ -94,7 +94,13 @@ Reference/API
     :no-inheritance-diagram:
 
 .. automodapi:: ctapipe.tools.train_disp_reconstructor
-   :no-inheritance-diagram:
+    :no-inheritance-diagram:
 
 .. automodapi:: ctapipe.tools.apply_models
     :no-inheritance-diagram:
+
+.. automodapi:: ctapipe.tools.optimize_event_selection
+    :no-inheritance-diagram:
+
+.. automodapi:: ctapipe.tools.compute_irf
+    :no-inheritance-diagram:

docs/changes/2473.feature.rst

@@ -0,0 +1,17 @@
+Add a ``ctapipe-optimize-event-selection`` tool to produce cut-selection files,
+based on a gamma, and optionally a proton and an electron DL2 file.
+Two components for calculating G/H and optionally theta cuts are added:
+``PercentileCuts`` keeps a certain percentage of gamma events in each bin and
+``PointSourceSensitivityOptimizer`` optimizes G/H cuts for maximum point source sensitivity and
+optionally calculates percentile theta cuts.
+
+Add a ``ctapipe-compute-irf`` tool to produce irfs given a cut-selection file, a gamma,
+and optionally a proton, and an electron DL2 input file.
+Given only a gamma file, the energy dispersion, effective area, and point spread function are calculated.
+Optionally, the bias and resolution of the energy reconstruction and the angular resolution can be calculated
+and saved in a separate output file.
+If a proton or a proton and an electron file is also given, a background model can be calculated,
+as well as the point source sensitivity.
+
+Irfs can be calculated with and without applying a direction cut.
+Only radially symmetric parameterizations of the irf components are implemented so far.

docs/conf.py

@@ -119,6 +119,7 @@ def setup(app):
     ("py:class", "t.Type"),
     ("py:class", "t.List"),
     ("py:class", "t.Tuple"),
+    ("py:class", "t.Sequence"),
     ("py:class", "Config"),
     ("py:class", "traitlets.config.configurable.Configurable"),
     ("py:class", "traitlets.traitlets.HasTraits"),
@@ -133,35 +134,34 @@ def setup(app):
     ("py:class", "traitlets.config.application.Application"),
     ("py:class", "traitlets.utils.sentinel.Sentinel"),
     ("py:class", "traitlets.traitlets.ObserveHandler"),
+    ("py:class", "traitlets.traitlets.T"),
+    ("py:class", "traitlets.traitlets.G"),
+    ("py:class", "Sentinel"),
+    ("py:class", "ObserveHandler"),
     ("py:class", "dict[K, V]"),
     ("py:class", "G"),
     ("py:class", "K"),
     ("py:class", "V"),
-    ("py:class", "t.Sequence"),
     ("py:class", "StrDict"),
     ("py:class", "ClassesType"),
-    ("py:class", "traitlets.traitlets.G"),
+    ("py:class", "re.Pattern"),
+    ("py:class", "re.Pattern[t.Any]"),
+    ("py:class", "astropy.coordinates.baseframe.BaseCoordinateFrame"),
+    ("py:class", "astropy.table.table.Table"),
+    ("py:class", "eventio.simtel.simtelfile.SimTelFile"),
+    ("py:class", "ctapipe.compat.StrEnum"),
+    ("py:class", "ctapipe.compat.StrEnum"),
+    ("py:obj", "traitlets.traitlets.T"),
     ("py:obj", "traitlets.traitlets.G"),
     ("py:obj", "traitlets.traitlets.S"),
-    ("py:obj", "traitlets.traitlets.T"),
-    ("py:class", "traitlets.traitlets.T"),
-    ("py:class", "re.Pattern[t.Any]"),
-    ("py:class", "re.Pattern"),
-    ("py:class", "Sentinel"),
-    ("py:class", "ObserveHandler"),
     ("py:obj", "traitlets.config.boolean_flag"),
     ("py:obj", "traitlets.TraitError"),
     ("py:obj", "-v"),  # fix for wrong syntax in a traitlets docstring
+    ("py:obj", "cls"),
+    ("py:obj", "name"),
     ("py:meth", "MetaHasDescriptors.__init__"),
     ("py:meth", "HasTraits.__new__"),
     ("py:meth", "BaseDescriptor.instance_init"),
-    ("py:obj", "cls"),
-    ("py:obj", "name"),
-    ("py:class", "astropy.coordinates.baseframe.BaseCoordinateFrame"),
-    ("py:class", "astropy.table.table.Table"),
-    ("py:class", "eventio.simtel.simtelfile.SimTelFile"),
-    ("py:class", "ctapipe.compat.StrEnum"),
-    ("py:class", "ctapipe.compat.StrEnum"),
 ]
 
 # Sphinx gallery config
@@ -405,6 +405,7 @@ def setup(app):
     "numpy": ("https://numpy.org/doc/stable", None),
     "pandas": ("https://pandas.pydata.org/pandas-docs/stable", None),
     "psutil": ("https://psutil.readthedocs.io/en/stable", None),
+    "pyirf": ("https://pyirf.readthedocs.io/en/stable/", None),
     "pytables": ("https://www.pytables.org", None),
     "pytest": ("https://docs.pytest.org/en/stable", None),
     "python": ("https://docs.python.org/3", None),

docs/user-guide/tools.rst

@@ -14,13 +14,15 @@ You can get a list of all available command-line tools by typing
 Data Processing Tools
 =====================
 
-* ``ctapipe-quickstart``: create some default analysis configurations and a working directory
+* ``ctapipe-quickstart``: Create some default analysis configurations and a working directory.
 * ``ctapipe-process``: Process event data in any supported format from R0/R1/DL0 to DL1 or DL2 HDF5 files.
 * ``ctapipe-apply-models``: Tool to apply machine learning models in bulk (as opposed to event by event).
 * ``ctapipe-calculate-pixel-statistics``: Tool to aggregate statistics and detect outliers from pixel-wise image data.
 * ``ctapipe-train-disp-reconstructor`` : Train the ML models for the  `ctapipe.reco.DispReconstructor` (monoscopic reconstruction)
 * ``ctapipe-train-energy-regressor``:  Train the ML models for the `ctapipe.reco.EnergyRegressor` (energy estimation)
 * ``ctapipe-train-particle-classifier``: Train the ML models for the  `ctapipe.reco.ParticleClassifier` (gamma-hadron separation)
+* ``ctapipe-optimize-event-selection``: Calculate gamma/hadron and direction cuts (e.g. for IRF calculation).
+* ``ctapipe-compute-irf``: Calculate an IRF with or without applying a direction cut and optionally benchmarks.
 
 File Management Tools:
 ======================

environment.yml

@@ -25,6 +25,7 @@ dependencies:
   - pypandoc
   - pre-commit
   - psutil
+  - pyirf
   - pytables
   - pytest
   - pytest-cov

pyproject.toml

@@ -55,6 +55,7 @@ all = [
     "eventio >=1.9.1,<2.0.0a0",
     "iminuit >=2",
     "matplotlib ~=3.0",
+    "pyirf ~=0.12.0"
 ]
 
 tests = [
@@ -105,6 +106,8 @@ ctapipe-dump-instrument = "ctapipe.tools.dump_instrument:main"
 ctapipe-display-dl1 = "ctapipe.tools.display_dl1:main"
 ctapipe-process = "ctapipe.tools.process:main"
 ctapipe-merge = "ctapipe.tools.merge:main"
+ctapipe-optimize-event-selection = "ctapipe.tools.optimize_event_selection:main"
+ctapipe-compute-irf = "ctapipe.tools.compute_irf:main"
 ctapipe-fileinfo = "ctapipe.tools.fileinfo:main"
 ctapipe-quickstart = "ctapipe.tools.quickstart:main"
 ctapipe-calculate-pixel-statistics = "ctapipe.tools.calculate_pixel_stats:main"