Muveto + hitlet fixes

docs/source/advanced/plugin_dev.rst

@@ -43,12 +43,66 @@ You can specify defaults in several ways:
 - ``default_per_run``: Specify a list of 2-tuples: ``(start_run, default)``. Here start_run is a numerized run name (e.g 170118_1327; note the underscore is valid in integers since python 3.6) and ``default`` the option that applies from that run onwards.
 - The ``strax_defaults`` dictionary in the run metadata. This overrides any defaults specified in the plugin code, but take care -- if you change a value here, there will be no record anywhere of what value was used previously, so you cannot reproduce your results anymore!
 
+Plugin inheritance
+----------------------
+It is possible to inherit the `compute()` method of an already existing plugin with another plugin. We call these types of plugins child plugins. Child plugins are recognized by strax when the `child_plugin` attribute of the plugin is set to `True`. Below you can find a simple example of a child plugin with its parent plugin:
+
+.. code-block:: python
+    @strax.takes_config(
+    strax.Option('by_child_overwrite_option', type=int, default=5,
+                 help="Option we will overwrite in our child plugin"),
+    strax.Option('parent_unique_option', type=int, default=2,
+                 help='Option which is not touched by the child and '
+                      'therefore the same for parent and child'),
+                      )
+    class ParentPlugin(strax.Plugin):
+        provides = 'odd_peaks'
+        depends_on = 'peaks'
+        __version__ = '0.0.1'
+        dtype = parent_dtype
+
+        def compute(self, peaks):
+            peaks['area'] *= self.config['parent_unique_option']
+            peaks['time'] *= self.config['by_child_overwrite_option']
+            return res
+
+
+    # Child:
+    @strax.takes_config(
+        strax.Option('by_child_overwrite_option_child',
+                     default=3,
+                     child_option=True,
+                     parent_option_name='by_child_overwrite_option',
+                     help="Option we will overwrite in our child plugin"),
+        strax.Option('option_unique_child',
+                      default=10,
+                      help="Option we will overwrite in our child plugin"),
+    )
+    class ChildPlugin(ParentPlugin):
+        provides = 'odd_peaks_child'
+        depends_on = 'peaks'
+        __version__ = '0.0.1'
+        child_plugin = True
+
+        def compute(self, peaks):
+            res = super().compute(peaks)
+            res['width'] = self.config['option_unique_child']
+            return res
+
+The `super().compute()` statement in the `compute` method of `ChildPlugin` allows us to execute the code of the parent's compute method without duplicating it. Additionally, if needed, we can extend the code with some for the child-plugin unique computation steps.
+
+To allow for the child plugin to have different settings then its parent (e.g. `'by_child_overwrite_option'` in `self.config['by_child_overwrite_option']` of the parent's `compute` method), we have to use specific child option. These options will be recognized by strax and overwrite the config values of the parent parameter during the initialization of the child-plugin. Hence, these changes only affect the child, but not the parent.
+
+An option can be flagged as a child option if the corresponding option attribute is set `child_option=True`. Further, the option name which should be overwritten must be specified via the option attribute `parent_option_name`.
+
+The lineage of a child plugin contains in addition to its options the name and version of the parent plugin.
+
+
 Plugin types
 ----------------------
 
 There are several plugin types:
    * `Plugin`: The general type of plugin. Should contain at least `depends_on = <datakind>`, `provides = <datatype>`, `def compute(self, <datakind>)`, and `dtype = <dtype> ` or `def infer_dtype(): <>`.
-   * `ChildPlugin`: Plugin which is inherited from an already existing `strax.Plugin` (e.g. in order to use the same `compute()` method). This plugin must have `child_ends_with  = str` defined to be recognized as a child plugin. The options of the child plugin are inherited from the parent. The option values of the parent can be changed by specifying new options for the child plugins. These options must obey the following two rules in order to overwrite the parent value: 1. They are tagged as `child_option=True` 2. the option name ends with the same string as in `child_ends_with`. A child plugin can also take additional new options.
    * `OverlapWindowPlugin`: Allows a plugin to only
    * `LoopPlugin`: Allows user to loop over a given datakind and find the corresponding data of a lower datakind using for example `def compute_loop(self, events, peaks)` where we loop over events and get the corresponding peaks that are within the time range of the event. Currently the second argument (`peaks`) must be fully contained in the first argument (`events` ).
    * `CutPlugin`: Plugin type where using `def cut_by(self, <datakind>)` inside the plugin a user can return a boolean array that can be used to select data.

requirements.txt

@@ -1,4 +1,4 @@
-numpy<=1.19
+numpy==1.19
 pandas
 numba>=0.43.1
 blosc

strax/config.py

@@ -61,6 +61,7 @@ def __init__(self,
                  default_factory: ty.Callable = OMITTED,
                  default_by_run=OMITTED,
                  child_option: bool = False,
+                 parent_option_name: str = None,
                  track: bool = True,
                  help: str = ''):
         """
@@ -72,9 +73,12 @@ def __init__(self,
          - Callable. Will be called with run_id, must return value for run.
          - List [(start_run_id, value), ..,] for values specified by range of
            runs.
-        :param child_option: If true option is marked as a child_option. In case of
-            plugin which is inherited from an already existing plugin we only store the
-            child options in the lineage of the plugin.
+        :param child_option: If true option is marked as a child_option. All
+            options which are marked as a child overwrite the corresponding parent
+            option. Removes also the corresponding parent option from the lineage.
+        :param parent_option_name: Name of the parent option of child option.
+            Required to find the key of the parent option so it can be overwritten
+            by the value of the child option.
         :param track: If True (default), option value becomes part of plugin
         lineage (just like the plugin version).
         :param help: Human-readable description of the option.
@@ -86,7 +90,18 @@ def __init__(self,
         self.default_factory = default_factory
         self.track = track
         self.help = help
+
+        # Options required for inherited child plugins:
+        # Require both to be more explicit and reduce errors by the user
         self.child_option = child_option
+        self.parent_option_name = parent_option_name
+        if (self.child_option and not self.parent_option_name) \
+                or (not self.child_option and self.parent_option_name):
+            raise ValueError('You have to specify both, "child_option"=True and '
+                             'the name of the parent option which should be '
+                             'overwritten by the child. Options which are unique '
+                             'to the child should not be marked as a child option.'
+                             f'Please update {self.name} accordingly.')
 
         # if self.default_by_run is not OMITTED:
         #     warnings.warn(f"The {self.name} option uses default_by_run,"

strax/context.py

@@ -402,29 +402,28 @@ def _set_plugin_config(self, p, run_id, tolerant=True):
         p.config = {k: v for k, v in config.items()
                     if k in p.takes_config}
 
-        if p.child_ends_with:
-            # This plugin is a child of another plugin and has some different
-            # Checking if the parent plugin is registered, this does not have to be the case
-            # but enables some useful checks.
-            parent_class = self._plugin_class_registry[p.provides[-1]].__bases__[0]
-            is_parent_reg = parent_class in self._plugin_class_registry.values()
-            # options to pass. So update parent config according to child:
-            for k, opt in p.takes_config.items():
-                if k.endswith(p.child_ends_with):
-                    if opt.child_option:
-                        v = config[k]
-                        kparent = k[:-len(p.child_ends_with)]
-
-                        if is_parent_reg:
-                            mes = f'Option {kparent} is not taken by parent plugin.'
-                            assert kparent in parent_class.takes_config.keys(), mes
+        if p.child_plugin:
+            # This plugin is a child of another plugin. This means we have to overwrite
+            # the registered option settings in p.config with the options specified by the
+            # child. This is required since the super().compute() method in a child plugins
+            # will still point to the option names of the parent (e.g. self.config['parent_name']).
 
-                        p.config[kparent] = v
-                    else:
-                        raise ValueError(f'You specified plugin {p.__class__.__name__} as a child plugin.'
-                                         f' Found the option {k} with the ending {p.child_ends_with}'
-                                         ' which was not specified as a child option.'
-                                         ' Was this intended? If yes, please change the ending')
+            # options to pass. So update parent config according to child:
+            for option_name, opt in p.takes_config.items():
+                # Now loop again overall options for this plugin (parent + child)
+                # and get all child options:
+                if opt.child_option:
+                    # See if option is tagged as a child option. In that case replace the
+                    # config value of the parent with the value of the child
+                    option_value = config[option_name]
+                    parent_name = opt.parent_option_name
+
+                    mes = (f'Cannot find "{parent_name}" among the options of the parent.'
+                           f' Either you specified by accident {option_name} as child option' 
+                           f' or you specified the wrong parent_option_name. Have you specified '
+                           'the correct parent option name?')
+                    assert parent_name in p.config, mes
+                    p.config[parent_name] = option_value
 
 
     def _get_plugins(self,
@@ -469,17 +468,29 @@ def get_plugin(data_kind):
 
             last_provide = d_provides
 
-            if p.child_ends_with:
+            if p.child_plugin:
                 # Plugin is a child of another plugin, hence we have to
                 # drop the parents config from the lineage
                 configs = {}
-                for q, v in p.config.items():
-                    if q + p.child_ends_with in p.takes_config:
-                        continue
-                    elif p.takes_config[q].track:
-                        configs[q] = v
-                # Adding parent information to the lineage:
+
+                # Getting information about the parent:
                 parent_class = p.__class__.__bases__[0]
+                # Get all parent options which are overwritten by a child:
+                parent_options = [option.parent_option_name for option in p.takes_config.values()
+                                  if option.child_option]
+
+                for option_name, v in p.config.items():
+                    # Looping over all settings, option_name is either the option name of the
+                    # parent or the child.
+                    if option_name in parent_options:
+                        # In case it is the parent we continue
+                        continue
+
+                    if p.takes_config[option_name].track:
+                        # Add all options which should be tracked:
+                        configs[option_name] = v
+
+                # Also adding name and version of the parent to the lineage:
                 configs[parent_class.__name__] = parent_class.__version__
 
                 p.lineage = {last_provide: (p.__class__.__name__,

strax/plugin.py

@@ -54,8 +54,9 @@ class Plugin:
     provides: tuple
     input_buffer: typing.Dict[str, strax.Chunk]
 
-    # Needed for plugins which are inherited from an already eliciting plugin:
-    child_ends_with = None
+    # Needed for plugins which are inherited from an already existing plugins,
+    # indicates such an inheritance.
+    child_plugin = False
 
     compressor = 'blosc'
 

strax/processing/hitlets.py

@@ -218,6 +218,13 @@ def get_single_hitlet_data(hitlet, records, prev_r, next_r):
             r = records[prev_r_i]
             data, (p_start_i, end) = _get_thing_data(hitlet, r)
             if not end:
+                # If end is zero this means we have not found any
+                # overlap which should not have happened. In case of an
+                # overlap start and end should reflect the start and end
+                # sample of the hitlet for which we found data.
+                print('Data found for this record:', data,
+                      'Start index', p_start_i,
+                      'End index:', end)
                 raise ValueError('This is odd found previous record, but no'
                                  ' overlapping indices.')
             temp_data[p_start_i:data_start] = data
@@ -244,6 +251,13 @@ def get_single_hitlet_data(hitlet, records, prev_r, next_r):
             r = records[next_r_i]
             data, (start, p_end_i) = _get_thing_data(hitlet, r)
             if not start:
+                # If start is zero this means we have not found any
+                # overlap which should not have happened. In case of an
+                # overlap start and end should reflect the start and end
+                # sample of the hitlet for which we found data.
+                print('Data found for this record:', data,
+                      'Start index', start,
+                      'End index:', p_end_i)
                 raise ValueError('This is odd found the next record, but no'
                                  ' overlapping indicies.')
             temp_data[data_end:p_end_i] = data
@@ -343,29 +357,30 @@ def hitlet_properties(hitlets):
     Computes additional hitlet properties such as amplitude, FHWM, etc.
     """
     for h in hitlets:
-
         dt = h['dt']
         data = h['data'][:h['length']]
-        # Compute amplitude
-        amp_ind = np.argmax(data)
-        amp_time = int(amp_ind * dt)
-        height = data[amp_ind]
+
+        if np.any(data):
+            # Compute amplitude
+            amp_ind = np.argmax(data)
+            amp_time = int(amp_ind * dt)
+            height = data[amp_ind]
 
-        h['amplitude'] = height
-        h['time_amplitude'] = amp_time
+            h['amplitude'] = height
+            h['time_amplitude'] = amp_time
 
-        # Computing FWHM:
-        left_edge, right_edge = get_fwxm(h, 0.5)
-        width = right_edge - left_edge
+            # Computing FWHM:
+            left_edge, right_edge = get_fwxm(h, 0.5)
+            width = right_edge - left_edge
 
-        # Computing FWTM:
-        left_edge_low, right_edge = get_fwxm(h, 0.1)
-        width_low = right_edge - left_edge_low
+            # Computing FWTM:
+            left_edge_low, right_edge = get_fwxm(h, 0.1)
+            width_low = right_edge - left_edge_low
 
-        h['fwhm'] = width
-        h['left'] = left_edge
-        h['low_left'] = left_edge_low
-        h['fwtm'] = width_low
+            h['fwhm'] = width
+            h['left'] = left_edge
+            h['low_left'] = left_edge_low
+            h['fwtm'] = width_low
 
 
 @export
@@ -388,31 +403,36 @@ def get_fwxm(hitlet, fraction=0.5):
         right last sample + 1.
     """
     data = hitlet['data'][:hitlet['length']]
-    max_val = hitlet['amplitude'] * fraction
 
     index_maximum = np.argmax(data)
-    pre_max = data[:index_maximum]
-    post_max = data[1 + index_maximum:]
-
-    # First the left edge:
-    lbi, lbs = _get_fwxm_boundary(pre_max, max_val)  # coming from the left
-    if lbi == NO_FWXM:
-        # We have not found any sample below:
-        left_edge = 0.
-    else:
-        # We found a sample below so lets compute
-        # the left edge:
+    max_val = data[index_maximum] * fraction
+    if np.all(data > max_val) or np.all(data == 0):
+        # In case all samples are larger, FWXM is not definition.
+        return np.nan, np.nan
+
+    pre_max = data[:index_maximum]  # Does not include maximum
+    post_max = data[1 + index_maximum:]  # same
+
+    if len(pre_max) and np.any(pre_max <= max_val):
+        # First the left edge:
+
+        lbi, lbs = _get_fwxm_boundary(pre_max[::-1], max_val)  # Reversing data starting at sample
+        # before maximum and go left
+        lbi = (index_maximum - 1) - lbi  # start sample minus samples we went to the left
         m = data[lbi + 1] - lbs  # divided by 1 sample
-        left_edge = lbi + (max_val - lbs) / m + 0.5  # .5 to start from bin center
+        left_edge = lbi + (max_val - lbs) / m + 0.5
+    else:
+        # There is no data before the maximum:
+        left_edge = 0
 
+    if len(post_max) and np.any(post_max <= max_val):
         # Now the right edge:
-    rbi, rbs = _get_fwxm_boundary(post_max[::-1], max_val)  # coming from the right
-    if rbi == NO_FWXM:
-        right_edge = len(data)
+        rbi, rbs = _get_fwxm_boundary(post_max, max_val)  # Starting after maximum and go right
+        rbi += 1 + index_maximum  # sample to the right plus start
+        m = data[rbi - 1] - rbs
+        right_edge = rbi - (max_val - rbs) / m + 0.5
     else:
-        rbi = len(data) - rbi
-        m = data[rbi - 2] - rbs
-        right_edge = rbi - (max_val - data[rbi - 1]) / m - 0.5
+        right_edge = len(data)
 
     left_edge = left_edge * hitlet['dt']
     right_edge = right_edge * hitlet['dt']
@@ -423,15 +443,22 @@ def get_fwxm(hitlet, fraction=0.5):
 def _get_fwxm_boundary(data, max_val):
     """
     Returns sample position and height for the last sample which
-    amplitude is below the specified value
+    amplitude is below the specified value.
+
+    If no sample can be found returns position and value of last sample
+    seen.
+
+    Note:
+        For FWHM we assume that we start at the maximum.
     """
-    i = NO_FWXM
-    s = NO_FWXM
-    for ind, d in enumerate(data):
+    ind = None
+    s = None
+    for i, d in enumerate(data):
         if d <= max_val:
-            i = ind
+            ind = i
             s = d
-    return i, s
+            return ind, s
+    return len(data)-1, data[-1]
 
 @export
 def conditional_entropy(hitlets, template='flat', square_data=False):