Project Meeting 2022.04.05

Agenda Items

• Continued sharrow presentation and discussion

Action Items

• Jeff will update the code to write logsums as outputs if they are not being currently done in sharrow.
• Sharrow will be made compatible with the 2-zone/3-zone structure as part of the current phase task.

Notes

Multi-Dimensional Analysis

• Sharrow is using Xarray data format underneath which does not have dimension restrictions as in the pandas dataframe. xarray can have multiple dimensions of data.
• Above feature is useful in destination choice models.
• Currently, activity sim joins the table of workers and land use data to create a massive dataframe that is a combination of all destination zones for each worker. Other attributes are added to this big dataframe as needed for further processing.
• In the new approach, a base dataset object will have native references to different datasets as needed. For example, for the destination choice model, references to indexes of workers and land use data can be provided to the dataset object instead of merging them together.
• Expressions can also be configured in the “spec” file or also defined as a python dictionary for computation

Gotchas

• Default outputs after running the compiler are ordered in lexicographic order if the order is not specified explicitly.\

Data Encoding

• Focuses on handling the data in skims and delivering the same to numba efficiently.
• Values in the skims can be encoded to lower precision as required. For example, float 32 can be converted to int 16.

Other comments

• Jeff confirmed that sharrow can be used to generate utilities and the user can use those utilities outside of sharrow
• Any activity sim component can be run on any other novel code compatible with underlying data. There is no commitment to use sharrow in every single model component. It is just available for the model components.
• Sharrow isn’t implemented in any of the pre-processors of activity sim model components.