SindyAutoencoders

Code for the paper "Data-driven discovery of coordinates and governing equations" by Kathleen Champion, Bethany Lusch, J. Nathan Kutz, and Steven L. Brunton.

The paper contains results for three example problems based on the Lorenz system, a reaction-diffusion system, and the nonlinear pendulum. Code for each example can be found in the respective subfolder in the examples folder. For each example, there are jupyter notebooks for (1) running the training procedure and (2) analyzing the resulting models.

Our training procedure can be replicated using the jupyter notebooks in each folder, which will run ten instances of the training and produce ten models. Running the notebook will also produce summary output, saved in a Pandas dataframe. The information saved in the dataframe can be used to compare among models. In the paper, we perform a model selection among the resulting models and select a subset of the models to highlight. Our model selection procedure is described in the appendix, along with a detailed description of the training procedure.

For each example, we also include jupyter notebooks to analyze the resulting models. These notebooks produce plots of the results and print out summary statistics on test data. The models analyzed in the paper are included in the repository.

Creating the network architecture and running the training procedure requires the specification of several parameters. A description of the parameters is as follows:

• input_dim - dimension of each sample of the input data

• latent_dim - dimension of the latent space

• model_order - either 1 or 2; determines whether the SINDy model predicts first or second order derivatives

• poly_order - maximum polynomial order to which to build the SINDy library; integer from 1-5

• include_sine - boolean, whether or not to include sine functions in the SINDy library

• library_dim - total number of library functions; this is determined based on the latent_dim, model_order, poly_order, and include_sine parameters and can be calculated using the function library_side in sindy_utils.py

• sequential_thresholding - boolean, whether or not to perform sequential thresholding on the SINDy coefficient matrix

• coefficient_threshold - float, minimum magnitude of coefficients to keep in the SINDy coefficient matrix when performing thresholding

• threshold_frequency - integer, number of epochs after which to perform thresholding

• coefficient_mask - matrix of ones and zeros that determines which coefficients are still included in the SINDy model; typically initialized to all ones and will be modified by the sequential thresholding procedure

• coefficient_initialization - how to initialize the SINDy coefficient matrix; options are 'constant' (initialize as all 1s), 'xavier' (initialize using the xavier initialization approach), 'specified' (pass in an additional parameter init_coefficients that has the values to use to initialize the SINDy coefficient matrix)

• loss_weight_decoder - float, weighting of the autoencoder reconstruction in the loss function (should keep this at 1.0 and adjust the other weightings proportionally)

• loss_weight_sindy_z- float, weighting of the SINDy prediction in the latent space in the loss function

• loss_weight_sindy_x - float, weighting of the SINDy prediction passed back to the input space in the loss function

• loss_weight_sindy_regularization - float, weighting of the L1 regularization on the SINDy coefficients in the loss function

• activation - activation function to be used in the network; options are 'sigmoid', 'relu', 'linear', or 'elu'

• widths - list of ints specifying the number of units for each layer of the encoder; decoder widths will be the reverse order of these widths

• epoch_size - number of training samples in an epoch

• batch_size - number of samples to use in a batch of training

• learning rate - float; learning rate passed to the adam optimizer

• data_path - path specifying where to save the resulting models

• print_progress - boolean, whether or not to print updates during training

• print_frequency - print progress at intervals of this many epochs

• max_epochs - how many epochs to run the training procedure for

• refinement_epochs - how many epochs to run the refinement training for (see paper for description of the refinement procedure)