Rework tutorials (#161)

* First version

* Close to complete version

* Finished advanced-tutorials and corrected average precision

* Adressed review comments

* typo on which notebook has the advanced features

* add note on how to pip install galsim_hub

* whitespace

* Added zeropoint airmass definition

Co-authored-by: Ismael Mendoza <[email protected]>

.pre-commit-config.yaml

@@ -8,6 +8,8 @@ repos:
         args:
           - --markdown-linebreak-ext=md
       - id: check-added-large-files
+        args:
+          - "--maxkb=5000"
       - id: check-merge-conflict
       - id: end-of-file-fixer
         exclude: ^.*fits

btk/metrics.py

@@ -95,13 +95,14 @@ def meas_ksb_ellipticity(image, additional_params):
     psf_image = additional_params["psf"].drawImage(psf_image)
     pixel_scale = additional_params["pixel_scale"]
     meas_band_num = additional_params["meas_band_num"]
+    verbose = additional_params["verbose"]
     gal_image = galsim.Image(image[meas_band_num, :, :])
     gal_image.scale = pixel_scale
     shear_est = "KSB"
 
     res = galsim.hsm.EstimateShear(gal_image, psf_image, shear_est=shear_est, strict=False)
     result = [res.corrected_g1, res.corrected_g2, res.observed_shape.e]
-    if res.error_message != "":
+    if res.error_message != "" and verbose:
         print(
             f"Shear measurement error : '{res.error_message }'. \
             This error may happen for faint galaxies or inaccurate detections."
@@ -687,6 +688,7 @@ def __init__(
         save_path=None,
         f_distance=distance_center,
         distance_threshold_match=5.0,
+        verbose=False,
     ):
         """Initialize metrics generator.
 
@@ -712,6 +714,7 @@ def __init__(
                 By default the distance is the euclidean distance from center to center.
             distance_threshold_match (float): Maximum distance for matching a detected and a
                 true galaxy in pixels.
+            verbose (bool): Indicates whether errors in the target_meas should be printed or not.
         """
         self.measure_generator: MeasureGenerator = measure_generator
         self.use_metrics = use_metrics
@@ -721,6 +724,7 @@ def __init__(
         self.save_path = save_path
         self.f_distance = f_distance
         self.distance_threshold_match = distance_threshold_match
+        self.verbose = verbose
 
     def __next__(self):
         """Returns metric results calculated on one batch."""
@@ -730,6 +734,7 @@ def __next__(self):
             "psf": blend_results["psf"][self.meas_band_num],
             "pixel_scale": survey.pixel_scale,
             "meas_band_num": self.meas_band_num,
+            "verbose": self.verbose,
         }
         target_meas = {}
         for k in self.target_meas.keys():
@@ -801,15 +806,15 @@ def auc(metrics_results, measure_name, n_meas, plot=False, ax=None):
     # The integral is from zero to one
     recalls = np.insert(recalls, 0, 0)
     precisions = np.insert(precisions, 0, precisions[0])
-    recalls = np.insert(recalls, -1, 1)
-    precisions = np.insert(precisions, -1, precisions[-1])
+    recalls = np.append(recalls, 1)
+    precisions = np.append(precisions, precisions[-1])
 
     for i in range(1, n_meas):
         average_precision += precisions[i] * (recalls[i] - recalls[i - 1])
 
     if plot:
         ax = plt.gca() if ax is None else ax
-        ax.scatter(recalls, precisions, label=measure_name)
+        ax.plot(recalls, precisions, label=measure_name, marker="x")
         ax.set_xlabel("Recall")
         ax.set_ylabel("Precision")
 

btk/sampling_functions.py

@@ -140,7 +140,7 @@ def __init__(self, max_number=2, stamp_size=24.0, maxshift=None):
     @property
     def compatible_catalogs(self):
         """Defined in parent class."""
-        return "CatsimCatalog", "CosmosCatalog"
+        return "CosmosCatalog"
 
     def __call__(self, table, shifts=None, indexes=None):
         """Method returning a blend sample from table.

docs/source/tutorials.rst

@@ -447,48 +447,23 @@ We can now create the corresponding instance of DrawBlendsGenerator. There is an
 Galsim_Hub tutorial
 --------------------
 
-BTK supports galaxy image generation with galsim_hub ; please refer to :ref:`this page<Galsim_Hub>` for more details on galsim_hub. The steps for using the galsim_hub generation are very similar to those from the previous section. Before starting this tutorial, you must install galsim_hub, which can be done using pip. NOTE: galsim_hub only works with python 3.7
+BTK supports galaxy image generation with ``galsim_hub`` ; please refer to `this page <https://github.com/McWilliamsCenter/galsim_hub>`_ for more details on ``galsim_hub``. Please note that ``galsim_hub`` only works with ``python 3.7``.
 
-You can find a notebook version of this tutorial in the notebooks folder.
-
-First, you should use the ``CosmosCatalog`` catalog instead of the Catsim one. While galsim_hub only require parameters for the image generation, we have chosen to use COSMOS as the source of those parameters so as to get a realistic distribution of those parameters. We have included a small sample of the catalog in BTK, and advise you to download the full catalog (see:ref:`COSMOS`) for better results.
-
-.. jupyter-execute::
+The steps for using the galsim_hub generation are very similar to those from the previous section. Before starting this tutorial, you must install ``galsim_hub``, which can be done using pip.
 
-  COSMOS_CATALOG_PATHS = [
-    "../data/cosmos/real_galaxy_catalog_23.5_example.fits",
-    "../data/cosmos/real_galaxy_catalog_23.5_example_fits.fits",
-  ]
-  catalog = btk.catalog.CosmosCatalog.from_file(COSMOS_CATALOG_PATHS)
+```
+pip install galsim_hub
+```
 
-We then instantiate the sampling function ; you should use the one specific for galsim_hub, which includes a cut on the size of the galaxies, as artifacts tend to appear when trying to generate galaxies which are too big.
+Alternatively, you can optionally install ``galsim_hub`` along with BTK:
 
-.. jupyter-execute::
+```
+pip install btk[galsim-hub]
+```
 
-  stamp_size = 24.0
-  sampling_function = btk.sampling_functions.DefaultSamplingGalsimHub(stamp_size=stamp_size)
-
-Then we can instantiate the ``DrawBlendsGenerator`` with the survey of your choice. Please bear in mind that while BTK will draw the images in any band you desire, galsim_hub does not generate a SED for the galaxy ; this means that the magnitude will be inacurrate in any other band than the one generated by the galsim_hub model you use (by default ``"hub:Lanusse2020"``).
-
-.. .. jupyter-execute::
-
-..   draw_generator = btk.draw_blends.GalsimHubGenerator(
-..       catalog,
-..       sampling_function,
-..       [Rubin],
-..       batch_size=8,
-..       stamp_size=stamp_size,
-..       shifts=None,
-..       indexes=None,
-..       cpus=1,
-..       add_noise=True,
-..       galsim_hub_model="hub:Lanusse2020", #May be replaced by any model compatible with galsim_hub
-..       param_names=["flux_radius", "mag_auto", "zphot"], #Name of the parameters ; they must match with the model you provide
-..   )
+You can find a notebook version of this tutorial in the notebooks folder.
 
-.. jupyter-execute::
+Advanced features
+------------------
 
-  batch = next(draw_generator)
-  blend_images = batch['blend_images']
-  blend_list = batch['blend_list']
-  btk.plot_utils.plot_blends(blend_images, blend_list, limits=(30,90))
+You can find a detailed tutorial on several more advanced feature of BTK in this `notebook <https://github.com/LSSTDESC/BlendingToolKit/blob/main/notebooks/advanced-features.ipynb>`_, including how to write your own measure function which should be interesting to most user. You will also see how to use the multiresolution, write your own sampling function or survey, use several measure functions and use of the measure_kwargs feature.