Debugging relativity on numba_montecarlo (#1136)

* removing kwarg from MonteCarloConfiguration, as jitclass does not support them

* need to pass enable_full_relativity to configuration_initialize, as removed kwarg in MonteCarloConfiguration

* move_r_packet and trace_packet both require montecarlo_configuration arg

* r_packet object used in distance calculation in place of r_packet.r value

* the full_relativity bool was being passed as a different kwarg

* added montecarlo_configuration to args of trace_vpacket_volley and trace_vpacket, so that relativity attribute continues to get passed down

* making sure the vpacket funcs that now require montecarlo_configuration get called with this arg

* Added random seed; does not work with numba

* Changed to numpy random seed

* Commented out MonteCarloException block; will test how this impacts output

* Need the MonteCarloException block to be run, otherwise large discrepancies between numba branch and master branch

* Adding custom seeds to vpackets; few line adjustments

* changed from njit to jit to ease debugging

* More descriptive montecarloexception, with optional printing beforehand

* Changed njit to jit in single_loop to ease debugging

* Added single-packet debugging capabilities to MonteCarloRunner.

* Removing plotting funcs from montecarlo

* Added basic logging capabilities to montecarlo_numba

* Generalized logger to *args

* Tweaking configuration of logging in module

* Logger can now handle kwargs

* Renamed logger

* Make logging more in line with other TARDIS loggers.

* Share global variable DEBUG_MODE across modules with montecarlo init.

* No longer calling removed plot_single_packet; also referring to __init__ DEBUG_MODE

* Now referring to montecarlo __init__ DEBUG_MODE global

* Remove dangling  else statement

* no longer need plotting in montecarlo_numba base.py

* Remove import of function that no longer exists.

* No longer holding global variables in montecarlo init

* Importing mc_logger for global variables now

* Moving log_decorator from base.py

* Adding logger to new montecarlo_logger file

* Add new logger to, remove print statements from r_packet.py

* Ensure that DEBUG_MODE is referenced within the montecarlo_logger file

* Allow the function being decorated to take its kwargs.

* Apply wraps correctly

* Make else block logic more apparent

* Get rid of extra blank line

* Got rid of a few todos

* Add note about why decorator behavior will not change according to config

* removing reference to logger in base

* Added crude buffer

* Increase buffer; check DEBUG_MODE checked during call, not init

* Set default buffer in montecarlo logger to stated default

* Allow specification of buffer in montecarlo yml

* Make sure logger_buffer is passed through to montecarlo logger

* Remove todo, question from log_decorator

* Adding better docstrings to log_decorator

* Add newline at end of montecarlo_logger file

* Make logger config happen outside of decorator

* Moved print statement to exception;

* Added logging to file

* No longer profiling calculate_distance_line

* Add functionality to profile incomplete packet runs

* Rename previous debug mode, as it was not for single packets

* Added single packet seed to yml

* Propagate single_packet_seed throughout configurations

* Reference single packet seed from montecarloconfiguration

* Add bool type to single packet seed type

* Catch exception better, catch extra random seed

* Set whole index, including energies, not just random seed

* Change where loop is broken.

* Move close_line_threshold check past montecarloexception

* Add single-packet debug documentation and script

* Add jitclass arg for single packet seed in montecarloconfiguration

* Now the default for single_packet_seed cannot be a bool

* Added new debug page to developer part of documentation

* Now the single_packet_seed has to be not 1, not not False

* Pause using log_decorator, remove try/except for nopython

* Vpacket jitclass now allows for an int64 index

* Exceptions args must be compile-time constants in nopython mode

* Allow 0 nu_diff to not throw exception.

* v_packets should have the same random seed as their r_packet

* Fix MonteCarloException throw

* Fix estimator typos

* Make sure return is not made before relativity block

* Rename esitmators to keep them consistent

* Also rename the estimators in montecarlo.pyx

* Rename estimators in numba_interface

* Clean up estimator typos; change back to njit

* Include Doppler factor in energy calculation for full_relativity

* Add back not statement

* Added relativistic Doppler factor

* Add in Doppler factor relativity for interaction

* Add angle aberration

* add angle aberration to interactions

* Update numba version

* Remove relativity branching

* Logger no longer prints to console if printing to file

* Initialize configuration with external module, not jitclass

* Remove extra repr of logger.handlers

* Ensure the angle aberration is called on vpacket, remove jitclass reference

* Convert montecarloconfiguration refs from jitclass to module

* Add more references to global module, complete relativistic branching

* Pass r_packet.mu to angle aberration calculation

* Clarify angle aberration calc, include doppler factor to distance calc

* Remove reference to MonteCarloConfiguration

* Trying to delete C tests

* Remove C montecarlo

* Remove references to C module

* Alter formal integral tests for python version

* Rewrite formal integral in python

docs/development/debug_numba.rst

@@ -0,0 +1,14 @@
+**************************
+Debugging numba_montecarlo
+**************************
+To facilitate more in-depth debugging when interfacing with the `montecarlo_numba`
+module, we provide a set of debugging configurations. PyCharm debugging
+configurations, in addition to related scripts and .yml files, are contained in
+`tardis.scripts.debug`. Currently, these include the ability to run TARDIS
+in asingle-packet mode, with the packet seed identified at debug time.
+`tardis_example_single.yml` is the configuration filethat is used to set up the
+single-packet TARDIS run; `run_numba_single.py` is thePython script that runs
+this .yml file; `run_numba_single.xml` is the PyCharmdebug configuration file
+that can be used in conjunction with the above files.
+
+

docs/development/index.rst

@@ -13,6 +13,7 @@ to the Astropy team for designing it.
 
     running_tests
     issues
+    debug_numba
 
 
 

tardis/io/schemas/montecarlo.yml

@@ -68,6 +68,19 @@ properties:
     default: false
     description: Enables a more complete treatment of relativitic effects. This includes
       angle aberration as well as use of the fully general Doppler formula.
+  debug_packets:
+    type: boolean
+    default: false
+    description: Decide whether to go into debugging mode.
+  logger_buffer:
+    type: number
+    default: 1
+    description: Provides option to not log every line.
+  single_packet_seed:
+    type:
+      - number
+    default: -1
+    description: If debug_packets is true, this is the seed for the only packet.
 
 required:
 - no_of_packets

tardis/montecarlo/__init__.py

@@ -1 +1 @@
-from tardis.montecarlo.base import *
+from tardis.montecarlo.base import *

tardis/montecarlo/base.py

@@ -13,10 +13,13 @@
 
 from tardis.util.base import quantity_linspace
 from tardis.io.util import HDFWriterMixin
-from tardis.montecarlo import montecarlo, packet_source as source
+from tardis.montecarlo import packet_source as source
 from tardis.montecarlo.formal_integral import FormalIntegrator
+from tardis.montecarlo import montecarlo_configuration as mc_config_module
+
 
 from tardis.montecarlo.montecarlo_numba import montecarlo_radial1d
+from tardis.montecarlo.montecarlo_numba import montecarlo_logger as mc_logger
 from tardis.montecarlo.montecarlo_numba.numba_interface import (
     configuration_initialize)
 
@@ -62,7 +65,8 @@ def __init__(self, seed, spectrum_frequency, virtual_spectrum_range,
                  enable_full_relativity, inner_boundary_albedo,
                  line_interaction_type, integrator_settings,
                  v_packet_settings, spectrum_method,
-                 packet_source=None):
+                 packet_source=None, debug_packets=False,
+                 logger_buffer=1, single_packet_seed=None):
 
         self.seed = seed
         if packet_source is None:
@@ -77,11 +81,17 @@ def __init__(self, seed, spectrum_frequency, virtual_spectrum_range,
         self.inner_boundary_albedo = inner_boundary_albedo
         self.enable_full_relativity = enable_full_relativity
         self.line_interaction_type = line_interaction_type
+        self.single_packet_seed = single_packet_seed
         self.integrator_settings = integrator_settings
         self.v_packet_settings = v_packet_settings
         self.spectrum_method = spectrum_method
         self._integrator = None
         self._spectrum_integrated = None
+
+        # set up logger based on config
+        mc_logger.DEBUG_MODE = debug_packets
+        mc_logger.BUFFER = logger_buffer
+
         if self.spectrum_method == 'integrated':
             self.optional_hdf_properties.append('spectrum_integrated')
 
@@ -100,8 +110,9 @@ def _initialize_estimator_arrays(self, tau_sobolev_shape):
         # Estimators
         self.j_estimator = np.zeros(tau_sobolev_shape[1], dtype=np.float64)
         self.nu_bar_estimator = np.zeros(tau_sobolev_shape[1], dtype=np.float64)
-        self.j_b_lu_estimator = np.zeros(tau_sobolev_shape)
-        self.edot_lu_estimator = np.zeros(tau_sobolev_shape)
+        self.j_blue_estimator = np.zeros(tau_sobolev_shape)
+        self.Edotlu_estimator = np.zeros(tau_sobolev_shape)
+        # TODO: this is the wrong attribute naming style.
 
     def _initialize_geometry_arrays(self, model):
         """
@@ -227,10 +238,9 @@ def run(self, model, plasma, no_of_packets,
         self._initialize_packets(model.t_inner.value,
                                  no_of_packets)
 
-        montecarlo_configuration = configuration_initialize(
-            self, no_of_virtual_packets)
-
-        montecarlo_radial1d(model, plasma, self, montecarlo_configuration)
+        montecarlo_configuration = configuration_initialize(self,
+                                                            no_of_virtual_packets)
+        montecarlo_radial1d(model, plasma, self)
         #montecarlo.montecarlo_radial1d(
         #    model, plasma, self,
         #    virtual_packet_flag=no_of_virtual_packets,
@@ -450,4 +460,7 @@ def from_config(cls, config, packet_source=None):
                    integrator_settings=config.spectrum.integrated,
                    v_packet_settings=config.spectrum.virtual,
                    spectrum_method=config.spectrum.method,
-                   packet_source=packet_source)
+                   packet_source=packet_source,
+                   debug_packets=config.montecarlo.debug_packets,
+                   logger_buffer=config.montecarlo.logger_buffer,
+                   single_packet_seed=config.montecarlo.single_packet_seed)