Merge pull request #816 from danforthcenter/interactive_roi

Annotation sub-package and point annotation tool (a.k.a. interactive CustomROI)

docs/Points.md

@@ -0,0 +1,47 @@
+## Interactive Point Annotation Tool
+
+Using [Jupyter Notebooks](jupyter.md) it is possible to interactively click to collect coordinates from an image, which can be used in various downstream applications. Left click on the image to collect a point. Right click removes the
+closest collected point.
+
+**plantcv.Points**(*img, figsize=(12, 6)*)
+
+**returns** interactive image class
+
+- **Parameters:**
+    - img - Image data
+    - figsize - Interactive plot figure size (default = (12,6))
+
+- **Attributes:**
+    - points - Coordinates (x,y) of the collected points as a list of tuples
+
+- **Context:**
+    - Used to define a list of coordinates of interest.
+    - For example the [`pcv.roi.custom`](roi_custom.md) function defines a polygon Region of Interest based on a list of vertices, which can be labor intensive to define but is streamlined with the ability to click for point collection.
+    - The list of vertices output has also shown to be helpful while using [pcv.roi.multi](roi_multi.md) in cases where centers are defined with a custom list of vertices.    
+- **Example use:**
+    - Below
+
+
+```python
+from plantcv import plantcv as pcv
+
+# Create an instance of the Points class
+marker = pcv.Points(img=img, figsize=(12,6))
+
+# Click on the plotted image to collect coordinates
+
+# Use the identified coordinates to create a custom polygon ROI
+roi_contour, roi_hierarchy = pcv.roi.custom(img=img, vertices=marker.points)
+
+```
+
+**Selecting Coordinates**
+
+![screen-gif](img/documentation_images/annotate_Points/custom_roi.gif)
+
+**Resulting ROI**
+
+![Screenshot](img/documentation_images/annotate_Points/custom_roi.jpg)
+
+
+**Source Code:** [Here](https://github.com/danforthcenter/plantcv/blob/master/plantcv/plantcv/classes.py)

docs/updating.md

@@ -35,7 +35,7 @@ conda list plantcv
 ### Updating from the source code
 
 The general procedure for updating PlantCV if you are using the `master` branch
-cloned from the `danforthcenter/plantcv` repository is to update your local 
+cloned from the `danforthcenter/plantcv` repository is to update your local
 repository and reinstall the package.
 
 With GitHub Desktop you can [synchronize](https://docs.github.com/en/free-pro-team@latest/desktop/contributing-and-collaborating-using-github-desktop/syncing-your-branch)
@@ -51,7 +51,7 @@ automatically. Alternatively, you can run `python setup.py install` to reinstall
 
 The setuptools installation method was not available in PlantCV v1, so users
 put the `plantcv/lib` directory in their custom `PYTHONPATH`. In PlantCV v2, the
-plantcv library directory is no longer in the lib directory, now it is in the 
+plantcv library directory is no longer in the lib directory, now it is in the
 main repository folder (`plantcv/plantcv`). If you want to continue to have
 plantcv in your `PYTHONPATH` you will need to update by simply removing `lib`
 from the path. You can also remove the lib folder after pulling the new version.
@@ -60,7 +60,7 @@ Git will automatically remove the `*.py` files but because we do not track the
 cause confusion.
 
 For Linux/Unix, `PYTHONPATH` can be edited in `~/.bash_profile`, `~/.bashrc`,
-`~/.profile`, `~/.cshrc`, `~/.zshrc`, etc. For Windows, right-click on My 
+`~/.profile`, `~/.cshrc`, `~/.zshrc`, etc. For Windows, right-click on My
 Computer/This PC and select Properties > Advanced system settings >
 Environmental Variables... and edit the User variables entry for `PYTHONPATH`.
 
@@ -76,7 +76,7 @@ an issue on GitHub or contact us directly.
 
 In order to support the installation of optional add-on subpackages, we converted
 PlantCV to a [namespace package](https://packaging.python.org/guides/packaging-namespace-packages/).
-To achieve this new functionality, existing functions had to be moved into a 
+To achieve this new functionality, existing functions had to be moved into a
 subpackage to maintain easy importing. To maintain previous behavior, PlantCV
 analysis scripts simply need to have updated PlantCV import syntax. So if you were
 previously doing something like:
@@ -96,14 +96,14 @@ package API. The goal is to make each PlantCV function easier to use by reducing
 the number of inputs and outputs that need to be configured (without losing
 functionality) and by making input parameters more consistently named and clearly
 defined where input types matter (e.g. instead of just `img` it could be `rgb_img`,
-`gray_img`, or `bin_img` for RGB, grayscale, or binary image, respectively). 
+`gray_img`, or `bin_img` for RGB, grayscale, or binary image, respectively).
 
-In PlantCV v3.0dev2 onwards, all functions were redesigned to utilize a global 
-parameters class to inherit values for standard inputs like `debug` and `device` 
-so that these values will not need to be explicitly input or output to/from each 
+In PlantCV v3.0dev2 onwards, all functions were redesigned to utilize a global
+parameters class to inherit values for standard inputs like `debug` and `device`
+so that these values will not need to be explicitly input or output to/from each
 function. An instance of the class [`Params`](params.md) as `params` is created automatically
-when PlantCV is imported and it can be imported to set global defaults. For example, 
-to change debug from `None` to 'plot' or 'print' you can now just add one line to 
+when PlantCV is imported and it can be imported to set global defaults. For example,
+to change debug from `None` to 'plot' or 'print' you can now just add one line to
 the top of your script or notebook to change the behavior of all subsequent function
 calls:
 
@@ -216,13 +216,13 @@ pages for more details on the input and output variable types.
 
 * pre v3.0dev2: device, masked_img = **plantcv.apply_mask**(*img, mask, mask_color, device, debug=None*)
 * post v3.0dev2: masked_img = **plantcv.apply_mask**(*rgb_img, mask, mask_color*)
-* post v3.7: masked_img = **plantcv.apply_mask**(*img, mask, mask_color*) 
+* post v3.7: masked_img = **plantcv.apply_mask**(*img, mask, mask_color*)
 
 #### plantcv.auto_crop
 
 * pre v3.0dev2: device, cropped = **plantcv.auto_crop**(*device, img, objects, padding_x=0, padding_y=0, color='black', debug=None*)
 * post v3.0dev2: cropped = **plantcv.auto_crop**(*img, objects, padding_x=0, padding_y=0, color='black'*)
-* post v3.2: cropped = **plantcv.auto_crop**(*img, obj, padding_x=0, padding_y=0, color='black'*) 
+* post v3.2: cropped = **plantcv.auto_crop**(*img, obj, padding_x=0, padding_y=0, color='black'*)
 
 #### plantcv.background_subtraction
 
@@ -234,7 +234,7 @@ pages for more details on the input and output variable types.
 * pre v3.0dev2: device, bin_img = **plantcv.binary_threshold**(*img, threshold, maxValue, object_type, device, debug=None*)
 * post v3.0dev2: Deprecated, see:
     * bin_img = **plantcv.threshold.binary**(*gray_img, threshold, max_value, object_type="light"*)
-    
+
 #### plantcv.canny_edge_detect
 
 * pre v3.2: NA
@@ -466,20 +466,20 @@ pages for more details on the input and output variable types.
 #### plantcv.morphology.find_tips
 
 * pre v3.3: NA
-* post v3.3: tip_img = **plantcv.morphology.find_tips**(*skel_img, mask=None*) 
-* post v3.11: tip_img = **plantcv.morphology.find_tips**(*skel_img, mask=None, label="default"*) 
+* post v3.3: tip_img = **plantcv.morphology.find_tips**(*skel_img, mask=None*)
+* post v3.11: tip_img = **plantcv.morphology.find_tips**(*skel_img, mask=None, label="default"*)
 
 #### plantcv.morphology.prune
 
 * pre v3.3: NA
-* post v3.3: pruned_img = **plantcv.morphology.prune**(*skel_img, size*) 
-* post v3.4: pruned_skeleton, segmented_img, segment_objects = **plantcv.morphology.prune**(*skel_img, size=0, mask=None*) 
+* post v3.3: pruned_img = **plantcv.morphology.prune**(*skel_img, size*)
+* post v3.4: pruned_skeleton, segmented_img, segment_objects = **plantcv.morphology.prune**(*skel_img, size=0, mask=None*)
 
 #### plantcv.morphology.segment_angle
 
 * pre v3.3: NA
-* post v3.3: labeled_img = **plantcv.morphology.segment_angle**(*segmented_img, objects*) 
-* post v3.11: labeled_img = **plantcv.morphology.segment_angle**(*segmented_img, objects, label="default"*) 
+* post v3.3: labeled_img = **plantcv.morphology.segment_angle**(*segmented_img, objects*)
+* post v3.11: labeled_img = **plantcv.morphology.segment_angle**(*segmented_img, objects, label="default"*)
 
 #### plantcv.morphology.segment_curvature
 
@@ -490,15 +490,15 @@ pages for more details on the input and output variable types.
 #### plantcv.morphology.segment_euclidean_length
 
 * pre v3.3: NA
-* post v3.3: labeled_img = **plantcv.morphology.segment_euclidean_length**(*segmented_img, objects*) 
-* post v3.11: labeled_img = **plantcv.morphology.segment_euclidean_length**(*segmented_img, objects, label="default"*) 
+* post v3.3: labeled_img = **plantcv.morphology.segment_euclidean_length**(*segmented_img, objects*)
+* post v3.11: labeled_img = **plantcv.morphology.segment_euclidean_length**(*segmented_img, objects, label="default"*)
 
 #### plantcv.morphology.segment_id
 
 * pre v3.3: NA
-* post v3.3: segmented_img, labeled_img = **plantcv.morphology.segment_id**(*skel_img, objects, mask=None*) 
+* post v3.3: segmented_img, labeled_img = **plantcv.morphology.segment_id**(*skel_img, objects, mask=None*)
 
-#### plantcv.morphology.segment_path_length 
+#### plantcv.morphology.segment_path_length
 
 * pre v3.3: NA
 * post v3.3: labeled_img = **plantcv.morphology.segment_path_length**(*segmented_img, objects*)
@@ -507,7 +507,7 @@ pages for more details on the input and output variable types.
 #### plantcv.morphology.segment_skeleton
 
 * pre v3.3: NA
-* post v3.3: segmented_img, segment_objects = **plantcv.morphology.segment_skeleton**(*skel_img, mask=None*) 
+* post v3.3: segmented_img, segment_objects = **plantcv.morphology.segment_skeleton**(*skel_img, mask=None*)
 
 #### plantcv.morphology.segment_sort
 
@@ -523,7 +523,7 @@ pages for more details on the input and output variable types.
 #### plantcv.morphology.skeletontize
 
 * pre v3.3: NA
-* post v3.3: skeleton = **plantcv.morphology.skeletonize**(*mask*) 
+* post v3.3: skeleton = **plantcv.morphology.skeletonize**(*mask*)
 
 #### plantcv.naive_bayes_classifier
 
@@ -608,6 +608,11 @@ pages for more details on the input and output variable types.
 * pre v3.0dev2: **plantcv.plot_image**(*img, cmap=None*)
 * post v3.0dev2: **plantcv.plot_image**(*img, cmap=None*)
 
+#### plantcv.Points
+
+* pre v4.0: NA
+* post v4.0: marker = **plantcv.Points**(*img, figsize=(6,12)*)
+
 #### plantcv.print_image
 
 * pre v3.0dev2: **plantcv.print_image**(*img, filename*)

mkdocs.yml

@@ -28,6 +28,8 @@ nav:
       - 'Analyze NIR': analyze_NIR_intensity.md
       - 'Analyze Shape': analyze_shape.md
       - 'Analyze Thermal': analyze_thermal_values.md
+      - 'Annotation Tools':
+          - 'Points': Points.md
       - 'Apply Mask': apply_mask.md
       - 'Auto Crop': auto_crop.md
       - 'Background Subtraction': background_subtraction.md

plantcv/plantcv/__init__.py

@@ -3,6 +3,8 @@
 from plantcv.plantcv.classes import Outputs
 from plantcv.plantcv.classes import Spectral_data
 from plantcv.plantcv.classes import PSII_data
+from plantcv.plantcv.classes import Points
+
 # Initialize an instance of the Params and Outputs class with default values
 # params and outputs are available when plantcv is imported
 params = Params()
@@ -85,14 +87,15 @@
 from plantcv.plantcv.stdev_filter import stdev_filter
 from plantcv.plantcv.spatial_clustering import spatial_clustering
 from plantcv.plantcv import photosynthesis
+from plantcv.plantcv import annotate
 # add new functions to end of lists
 
 # Auto versioning
 from ._version import get_versions
 __version__ = get_versions()['version']
 del get_versions
 
-__all__ = ['fatal_error', 'Params', 'Outputs', 'Spectral_data', 'PSII_data', 'deprecation_warning', 'print_image', 
+__all__ = ['fatal_error', 'Params', 'Outputs', 'Spectral_data', 'PSII_data', 'Points', 'deprecation_warning', 'print_image',
            'plot_image', 'color_palette', 'apply_mask', 'gaussian_blur', 'transform', 'hyperspectral', 'readimage', 'readbayer',
            'laplace_filter', 'sobel_filter', 'scharr_filter', 'hist_equalization', 'erode', 'image_add', 
            'image_subtract', 'dilate', 'watershed', 'rectangle_mask', 'rgb2gray_hsv', 'rgb2gray_lab', 'rgb2gray_cmyk',
@@ -106,4 +109,4 @@
            'background_subtraction', 'naive_bayes_classifier', 'distance_transform', 'params',
            'cluster_contour_mask', 'analyze_thermal_values', 'opening',
            'closing', 'within_frame', 'fill_holes', 'get_kernel', 'crop', 'stdev_filter',
-           'spatial_clustering', 'photosynthesis', 'homology']
+           'spatial_clustering', 'photosynthesis', 'homology', 'annotate']

plantcv/plantcv/annotate/points.py

@@ -0,0 +1,23 @@
+# Point/vertice annotation tool(s)
+
+import numpy as np
+from scipy.spatial import distance
+
+## INTERACTIVE ROI TOOLS ##
+
+
+def _find_closest_pt(pt, pts):
+    """ Given coordinates of a point and a list of coordinates of a bunch of points, find the point that has the
+    smallest Euclidean to the given point
+
+    :param pt: (tuple) coordinates of a point
+    :param pts: (a list of tuples) coordinates of a list of points
+    :return: index of the closest point and the coordinates of that point
+    """
+    if pt in pts:
+        idx = pts.index(pt)
+        return idx, pt
+
+    dists = distance.cdist([pt], pts, 'euclidean')
+    idx = np.argmin(dists)
+    return idx, pts[idx]

plantcv/plantcv/classes.py

@@ -1,7 +1,11 @@
 # PlantCV classes
 import os
+import cv2
 import json
 from plantcv.plantcv import fatal_error
+import matplotlib.pyplot as plt
+from math import floor
+from plantcv.plantcv.annotate.points import _find_closest_pt
 
 
 class Params:
@@ -182,7 +186,7 @@ def save_results(self, filename, outformat="json"):
 
 class Spectral_data:
     """PlantCV Hyperspectral data class"""
-    
+
     def __init__(self, array_data, max_wavelength, min_wavelength, max_value, min_value, d_type, wavelength_dict,
                  samples, lines, interleave, wavelength_units, array_type, pseudo_rgb, filename, default_bands):
         # The actual array/datacube
@@ -235,10 +239,50 @@ def __repr__(self):
             if v is not None:
                 mvars.append(k)
         return "PSII variables defined:\n" + '\n'.join(mvars)
-    
+
     def add_data(self, protocol):
         """
         Input:
             protocol: xr.DataArray with name equivalent to initialized attributes
         """
         self.__dict__[protocol.name] = protocol
+
+
+class Points(object):
+    """Point annotation/collection class to use in Jupyter notebooks. It allows the user to
+    interactively click to collect coordinates from an image. Left click collects the point and
+    right click removes the closest collected point
+    """
+
+    def __init__(self, img, figsize=(12, 6)):
+        """
+        Initialization
+        :param img: image data
+        :param figsize: desired figure size, (12,6) by default
+        :attribute points: list of points as (x,y) coordinates tuples
+        """
+
+        self.fig, self.ax = plt.subplots(1, 1, figsize=figsize)
+        self.ax.imshow(cv2.cvtColor(img, cv2.COLOR_BGR2RGB))
+
+        self.points = []
+        self.events = []
+
+        self.fig.canvas.mpl_connect('button_press_event', self.onclick)
+
+    def onclick(self, event):
+        """ Handle mouse click events
+        """
+        self.events.append(event)
+        if event.button == 1:
+
+            self.ax.plot(event.xdata, event.ydata, 'x', c='red')
+            self.points.append((floor(event.xdata), floor(event.ydata)))
+
+        else:
+            idx_remove, _ = _find_closest_pt((event.xdata, event.ydata), self.points)
+            # remove the closest point to the user right clicked one
+            self.points.pop(idx_remove)
+            ax0plots = self.ax.lines
+            self.ax.lines.remove(ax0plots[idx_remove])
+        self.fig.canvas.draw()

tests/tests.py

@@ -5194,6 +5194,48 @@ def test_plantcv_roi_custom_bad_input():
         _ = pcv.roi.custom(img=img, vertices=[[226, -1], [3130, 1848], [2404, 2029], [2205, 2298], [1617, 1761]])
 
 
+def test_plantcv_annotate_Points_interactive():
+    # Read in a test grayscale image
+    img = cv2.imread(os.path.join(TEST_DATA, TEST_INPUT_COLOR), -1)
+
+    # initialize interactive tool
+    drawer_rgb = pcv.Points(img, figsize=(12, 6))
+
+    # simulate mouse clicks
+    # event 1, left click to add point
+    e1 = matplotlib.backend_bases.MouseEvent(name="button_press_event", canvas=drawer_rgb.fig.canvas,
+                                             x=0, y=0, button=1)
+    point1 = (200, 200)
+    e1.xdata, e1.ydata = point1
+    drawer_rgb.onclick(e1)
+
+    # event 2, left click to add point
+    e2 = matplotlib.backend_bases.MouseEvent(name="button_press_event", canvas=drawer_rgb.fig.canvas,
+                                             x=0, y=0, button=1)
+    e2.xdata, e2.ydata = (300, 200)
+    drawer_rgb.onclick(e2)
+
+    # event 3, left click to add point
+    e3 = matplotlib.backend_bases.MouseEvent(name="button_press_event", canvas=drawer_rgb.fig.canvas,
+                                             x=0, y=0, button=1)
+    e3.xdata, e3.ydata = (50, 50)
+    drawer_rgb.onclick(e3)
+
+    # event 4, right click to remove point with exact coordinates
+    e4 = matplotlib.backend_bases.MouseEvent(name="button_press_event", canvas=drawer_rgb.fig.canvas,
+                                             x=0, y=0, button=3)
+    e4.xdata, e4.ydata = (50, 50)
+    drawer_rgb.onclick(e4)
+
+    # event 5, right click to remove point with coordinates close but not equal
+    e5 = matplotlib.backend_bases.MouseEvent(name="button_press_event", canvas=drawer_rgb.fig.canvas,
+                                             x=0, y=0, button=3)
+    e5.xdata, e5.ydata = (301, 200)
+    drawer_rgb.onclick(e5)
+
+    assert drawer_rgb.points[0] == point1
+
+
 # ##############################
 # Tests for the transform subpackage
 # ##############################