Skip to content

Commit

Permalink
Merge pull request #78 from CHIMEFRB/docs-dev
Browse files Browse the repository at this point in the history 
Docs dev
  • Loading branch information
@emmanuelfonseca
emmanuelfonseca authored Nov 13, 2023
2 parents 77b0136 + 451e73a commit dfb74f5
Show file tree
Hide file tree
Showing 21 changed files with 309 additions and 3,397 deletions.
31 changes: 9 additions & 22 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
fitburst
========

This repo contains functions and objects for modeling dynamic spectra of dispersed astrophysical signals at radio frequencies.
This repository contains functions, objects, and scripts for modeling dynamic spectra of dispersed astrophysical signals at radio frequencies.

## Installation

Expand All @@ -13,28 +13,15 @@ pc> cd fitburst
pc/fitburst> pip install . # add the --user option if you're looking to install in your local environment.
```

## Usage
## Usage and Documentation
Please refer to the documentation linked above to find desciptions on the codebase and examples for interacting with it.

Once installed, the `fitburst` functionality can be imported as a Python package for various custom purposes. For example, if you wish to simply read in CHIME/FRB metadata only (e.g., parameter estimates made by the various online pipelines) for a specific event, you could do the following:
## Publication
The theory behind the modeling and analysis routines is presented in a paper currently under review, but available on the arXiv. This paper includes a variety of fitting examples and discussions on the treatment of biasing effects (e.g., intra-channel smearing from pulse dispersion) that can be accounted for within `fitburst`. If you use this codebase and publish results obtained with it, we ask that you cite the `fitburst` paper using the following BibTex citation:

```
pc> python
>>> from fitburst.backend.chimefrb import DataReader
>>> data = DataReader(48851362)
....
>>> print(data.burst_parameters) # a dictionary of pipeline-specific parameters
>>> print(data.files) # a list of filenames for the total-intensity data set
```

There are also several example scripts available in the `fitburst/pipelines` section of the repo that utilize the `fitburst` package in slightly different ways. The current script used to analyze CHIME/FRB data (`fitburst/pipelines/fitburst_example_chimefrb.py`) comes with a variety of options for interacting with the full algorithm at the command line. For example, if you wish to run the full `fitburst` pipeline on an event (i.e., data I/O and pre-processing, setup and fitting of model against pre-processed intensity data) using the intensity-DM pipeline parameters as your initial guess, and ignoring the fit of scattering parameters, you should run:
```
pc> python /path/to/fitburst_example_chimefrb.py 48851362 --pipeline dm
```

If you wish to change the size of the windowed spectrum, fit for a scattering timescale of a thin-screen model, and toggle its initial guess to a value of your choosing, you should instead run:
... COMING SOON! :sweat-smile

```
pc> python /path/to/fitburst_example_chimefrb.py 48851362 --pipeline dm --window 0.12 --fit scattering_timescale --scattering_timescale 0.05
```
## Credits
All authors of the `fitburst` papers are the founding developers of `fitburst`, with Emmanuel Fonseca leading the development team. We welcome novel and meaningful contributions from interested users!

Use the `-h` option in the above script to see all available options and units for various numerical quantities.
This package was built using [namoona](https://github.com/CHIMEFRB/namoona).
32 changes: 19 additions & 13 deletions docs/installation.md
View file
Original file line number Diff line number Diff line change
@@ -1,22 +1,28 @@
# fitburst - Installation

## Prerequisites
* Python 3.7 or greater
* [astropy](https://docs.astropy.org/en/stable/install.html)
* [numpy](https://numpy.org/install/)
* [matplotlib](https://matplotlib.org/users/installing.html)
* [mkdocs](https://www.mkdocs.org/#installation)
* [PyYAML](https://pyyaml.org/wiki/PyYAMLDocumentation)
* [scipy](https://www.scipy.org/install.html)

## Using `pip`
## Building from `PyPI`

One day, `fitburst` will be retrievable from `pip`! Believe it!
`fitburst` will soon be retrievable from the Python Package Index ([PyPI](https://pypi.org)) via [pip](https://pypi.org/project/pip/).

## Building from Source

The current `fitburst` codebase uses `setuptools` for building the distribution and grabbing the above dependencies. To build from source, run the following commands:
The `fitburst` codebase uses either `pip` or [poetry](https://python-poetry.org) for building the distribution and grabbing dependencies. The easiest route to building `fitburst` is to use `pip`:

> git clone https://github.com/CHIMEFRB/fitburst.git
> cd fitburst/
> pip install -e .
> pip install .

## Dependencies
For out-of-the-box use, `fitburst` currently depends on the external Python packages listed below. We encourage interested developers to contribute software and/or replace existing functionality with new dependencies; however, we request that any additional dependency be open-source, meaningfully used, and accessible via `pip`.

* Python 3.8 or greater
* [matplotlib](https://matplotlib.org/users/installing.html)
* [mkdocs](https://www.mkdocs.org/#installation)
* [numpy](https://numpy.org/install/)
* [pandas](https://pandas.pydata.org/docs/index.html)
* [pytz](https://pythonhosted.org/pytz/)
* [pyyaml](https://pyyaml.org/wiki/PyYAMLDocumentation)
* [scipy](https://www.scipy.org/install.html)

### Developer Dependencies
There are additional dependencies for folks who wish to contribute and build code, tests, and/or documentation. These dependenices are listed in the `pyproject.toml` file under the `tools.poetry.group.*.dependencies` attributes. However, these dependencies currently can only be installed using `poetry` due to way in which `pip` understands the `pyproject.toml` file.
122 changes: 86 additions & 36 deletions docs/usage/creating_models.md
View file
Original file line number Diff line number Diff line change
@@ -1,70 +1,120 @@
We have developed a Python class, called the `SpectrumModeler`, for generating data models in a manner suitable for interaction with downstreaming fitting routines. The simplest version of a call to the model class is,
We have developed a Python class, called the `SpectrumModeler`, for generating models of dynamic spectra. The `SpectrumModeler` is designed for interaction with downstream fitting routines; however, it can nonetheless be used as a standalone object. The simplest version of a call to the `SpectrumModeler` is given here:

``` python
>>> from fitburst.analysis.model import SpectrumModeler
>>> model = SpectrumModeler(num_freq, num_time)
from fitburst.analysis.model import SpectrumModeler
freqs = ... # array of frequency labels, in MHz
times = ... # array of timestamps, in seconds
model = SpectrumModeler(freqs, times)
```

where the quantities (`num_freq`, `num_time`) define the dimensions of the model spectrum.
where (`freqs`, `times`) define the centers of frequency channels and time bins, respectively.

## Parameters of the Model Object
The exact list of parameters will depend on the assumed shape of the spectral energy distribution (SED). By default, the `SpectrumModeler` assumes a "running power-law" model for the SED, and so the spectral parameters will be the `spectral_index` and `spectral_running`. The full list of parameters can be retrieved as follows:

``` python
>>> print(model.parameters)
['amplitude', 'arrival_time', 'burst_width', 'dm', 'dm_index', 'scattering_timescale', 'scattering_index', 'spectral_index', 'spectral_running']
```

If a Gaussian SED is instead desired, then you can instantiate the `SpectrumModeler` and set the correct option to indicate this choice:
As described in the `fitburst` paper, the `SpectrumModeler` is a function of nine fittable parameters. A tenth parameter, called `ref_freq` cannot be fitted as it serves as a frequency to which the amplitude and SED parameters are referenced. The full list of parameters can be retrieved as follows:

``` python
>>> model = SpectrumModeler(num_freq, num_time, freq_model = "gaussian")
>>> print(model.parameters)
['amplitude', 'arrival_time', 'burst_width', 'dm', 'dm_index', 'scattering_timescale', 'scattering_index', 'freq_mean', 'freq_width']
['amplitude', 'arrival_time', 'burst_width', 'dm', 'dm_index', 'ref_freq', 'scattering_timescale', 'scattering_index', 'spectral_index', 'spectral_running']
```

Notice that all but two parameters have changed, and that the `freq_mean` and `freq_width` now characterize the (Gaussian) shape of the SED. So far, only the `powerlaw` and `gaussian` SED models are available in `fitburst`.
Please refer to Section 2 and Table 1 of the `fitburst` paper for a description of these parameters.

## Loading Parameter Values into the Model Object

The above calls to the `SpectrumModeler` object yield an "empty" model object; the object is configured but contains no information on the model parameters. In order to load parameter values, we use the `update_parameters()` method of the `SpectrumModeler`:
In order to load parameter values, we use the `update_parameters()` method of the `SpectrumModeler`:

``` python
# define dictiomary containing parameter values.
burst_parameters = {
"amplitude" : [0.0],
"arrival_time" : [0.5],
"burst_width" : [0.005],
"dm" : [557.0],
"dm_index" : [-2.0],
"ref_freq" : [600.0],
"scattering_index" : [-4.0],
"scattering_timescale" : [0.0],
"freq_mean" : [450.0],
"freq_width" : [43.0],
"amplitude" : [0.],
"arrival_time" : [0.04],
"burst_width" : [0.003],
"dm" : [349.5],
"dm_index" : [-2.],
"ref_freq" : [1400.],
"scattering_index" : [-4.],
"scattering_timescale" : [0.],
"spectral_index" : [10.],
"spectral_running" : [-100.],
}

# now update Gaussian-SED model object to use the above values.
# now instantiate the SpectrumModeler
model = SpectrumModeler(freqs, times)

# update the SpectrumModeler to use the above values.
model.update_parameters(burst_parameters)

# adjust the DM value while leaving all others unchanged in the model object.
model.update_parameters({"DM": [557.5]})
# slightly adjust the DM only, leaving all others unchanged in the model object.
model.update_parameters({"dm": [348.95]})
```

The `update_parameters()` method is able to receive a dictionary that contains only one or a partial set of the full parameter list, as shown in the second method call above. This feature is important when fitting models to data where one or more model parameters are desired to be fixed to pre-determined values.
The `update_parameters()` method receives a dictionary with one or more parameters with values loaded in Python lists, as shown in the second method call above. This feature exists to allow for flexibility in generating models for fitting where one or more parameters are fixed to pre-determined values.

## Generating Mulit-Component Models

The `SpectrumModeler` is also capable of generating models of a multi-component spectrum, i.e., a dynamic spectrum with $N$ distinct pulses. Such models can be created with the same code above, but with values of the `burst_parameters` dicitionary that are lists of length $N$. For example, the following code will overwrite the above parameters to instantiate a model with 3 components:

```python
# define dictiomary containing parameter values.
burst_parameters = {
"amplitude" : [0., 0., 0.],
"arrival_time" : [0.03, 0.04, 0.05],
"burst_width" : [0.001, 0.003, 0.0005],
"dm" : [349.5, 349.5, 349.5],
"dm_index" : [-2., -2., -2.],
"ref_freq" : [1400., 1400., 1400.],
"scattering_index" : [-4., -4., -4.],
"scattering_timescale" : [0., 0., 0.],
"spectral_index" : [10., 0., -10.],
"spectral_running" : [-100., -100., -100.],
}

# now instantiate the SpectrumModeler for a three-component model.
num_components = len(burst_parameters["dm"])
model = SpectrumModeler(freqs, times, num_components = num_components)

# now update Gaussian-SED model object to use the above values.
model.update_parameters(burst_parameters)

The `SpectrumModeler` is also capable of generating models of a multi-component spectrum. The only changed needed for this to occur is for the values of the above `burst_parameters` dicitionary to be lists of length greater than 1, where the $i$th element for each list corresponds to parameters of "sub-burst" $i$.
```

## Creating Models for De-dispersed Data
Once the above configuration is done, we can then compute a model spectrum with the `compute_mode()` method within the `SpectrumModeler`.
Users will typically want to fit models of dynamic spectra against data that are already de-dispersed. The `SpectrumModeler` can be used as shown above, but with one caveat: the `dm` parameter is treated as a "DM offset" for de-dispersed spectra, instead of the "full" DM whose values are supplied in the above examples. Once this configuration is done, we can then compute a model spectrum with the `compute_mode()` method within the `SpectrumModeler`.

The following code with use the latest example above and perform the adjustment needed for generating a de-dispersed dynamic spectum:

``` python
freqs = ... # array of frequency labels, in MHz
times = ... # array of timestamps, in seconds
# indicate whether the spectrum is de-dispersed or not.
is_dedispersed = True

# define dictiomary containing parameter values.
burst_parameters = {
"amplitude" : [0., 0., 0.],
"arrival_time" : [0.03, 0.04, 0.05],
"burst_width" : [0.001, 0.003, 0.0005],
"dm" : [349.5, 349.5, 349.5],
"dm_index" : [-2., -2., -2.],
"ref_freq" : [1400., 1400., 1400.],
"scattering_index" : [-4., -4., -4.],
"scattering_timescale" : [0., 0., 0.],
"spectral_index" : [10., 0., -10.],
"spectral_running" : [-100., -100., -100.],
}

# adjust DM value to zero offset, if necessary.
num_components = len(burst_parameters["dm"])

if is_dedispersed:
burst_parameters["dm"] = [0.] * num_components

# now instantiate the SpectrumModeler for a three-component model.
model = SpectrumModeler(freqs, times, num_components = num_components)

spectrum_model = model.compute_model(times, freqs)
# grab the model spectrum.
spectrum_model = model.compute_model()
```

If you're using a `DataReader` to load and prepare data, then the above arrays can be accessed through the `freqs` and `times` attributes.
The above call with return a NumPy `ndarray` with shape `(num_freq, num_time)`.

## Creating Models for Dispersed Data
In rare or simulation cases, it may be desired to create a dispersed dynamic spectrum. This spectrum can be generated using the latest example above, but instead setting `is_dedispersed = False`.
2 changes: 1 addition & 1 deletion docs/usage/formatting_data_generic.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
We have defined a `fitburst`-compliant ("generic") data format for loading all required data into the `fitburst` data-reading object. Users can adopt this generic format to ensure initialization of required variables and arrays. The generic-format data are stored in and read from a Python3 Numpy `.npz` file.
For ease of use, we have defined a `fitburst`-compliant ("generic") data format for loading all required data into the `fitburst` data-reading object. Users can adopt this generic format to ensure initialization of required variables and arrays used within `fitburst`. The generic-format data are stored in and read from a Python3 Numpy `.npz` file.

## Concept of Generic Format
A generic-compatible data file, e.g., "input\_data.npz", is assumed to contain three entries:
Expand Down
Loading

0 comments on commit dfb74f5

Please sign in to comment.