Project Meeting 2023.05.11

Agenda

• Admin Items
  • Sharrow implementation and questions
  • Maintenance of examples
• Continued discussion about input checker approach

Key Decisions / Action Items

• Review and accept current Pull Requests for SEMCOG and MWCOG example models.
• It is likely that the data model/pydantic approach to input checker is the preferred approach. RSG will open an issue on Github for discussion to provide sortium members an opportunity to comment further in written form. Next Thursday will be the deadline for providing comments - at which time, if consensus remains that the data model plan is superior, we will begin the process of formally changing RSG’s scope of work.

Meeting Notes

Admin Items

Sharrow

• Questions about sharrow implementation
  • SANDAG to provide list of questions/concerns regarding sharrow in their development of ABM3. SANDAG to ensure that their code review requests are specific to sharrow and not ABM3 implementation.
  • PSRC is testing sharrow implementation. They have posted some bugs in GitHub issues and are coordinating with SANDAG in what they find in order to understand whether the issues are with sharrow or the data, for example.
• General questions about sharrow
  • Noted that some expressions can be sharrow-ed and some cannot and would like some more information.
  • Sharrow, under the hood, is numba. There is lots of documentation for numba itself, which says what it can and cannot do. For about 90% of the expressions that have been put in the UECs, you can take that expression and wrap it in a numba compiler and it’s fine. For the remaining 10% of things, numba can’t handle arbitrary python expressions that don’t comply with the things than numba can do (most things that can be done in numpy) – but things that are implemented in panda and not numpy, the support is a little weaker. Jeff went through every line of every expression for every example, except for a few complicated things, and those have been set to skip sharrow.
  • Long-term considerations for future versions/examples: how to design the UECs efficiently. Think about how the expressions that go into UECs are written and make them numba-compatible so there isn’t a need to do hack-y things in sharrow for these complicated expression.
  • There is no documentation the lists out each instance of hack-y things that needed to be done in sharrow.
  • PSRC has converted all their UECs to be fully compliant with sharrow.

Maintenance of Examples

• The question is whether or not to review and accept pull requests for additional model examples / updates.
• Current pull requests
  • SEMCOG model and data
  • MWCOG model
• Prior related decisions: we will not presently expend effort on maintenance/updates of 3-zone code, nor on component validation testing against TM2 results.
• We need the data to test these examples to make sure any ActivitySim changes are compliant with these examples. Testing infrastructure is pretty large at this point, running a lot of tests and examples. If we accept these, it’ll add to run time for these compliance tests. This assumes that the PRs have been rigorously tested by those owning that example. If they have things compliant with an older version of ActivitySim, then it could fall to the consortium to debug implementations to understand if it’s a problem in the changes made to ActivitySim or the example itself.
• For these PRs in particular, the changes are minor and no issues are anticipated. Accepting them is probably a low risk but want to discuss with the group for future instances like this.
• DECISION: Review and accept current Pull Requests for SEMCOG and MWCOG example models. Discussion bigger picture of example maintenance later.

Other

• MWCOG has sent a list of potential future agenda items to Joe C. Joe to coordinate with CS on scheduling topics at future meetings

Status Update on Current Tasks

Continued discussion about Input Checker (RSG)

• See last week's meeting notes and powerpoint for overview of options and their pros/cons.
• Two approach options for the input checker
  • Pydantic-like implementation
    • Based on the data_model template created by WSP
    • Enums list enumerated variables and set input variables (also set output variables and specific parameters, but those aren’t very relevant to the input checker)
    • You can write custom validator functions, which are more flexible but add a little more overhead since everything has to be written in python (less user-friendly for staff without those skills)
  • Original proposal (more like UECs and has been implemented in Oregon and elsewhere)
    • Set of input tables, severity of check options, write expressions
• If RSG went with the data model / pydantic implementation – what are the schedule/budget implications?
  • No schedule changes needed
  • No additional budget needed but would do less on the effort to generalize it to a number of examples / trim back on functionality. WSP has already set up the framework for the data model, which helps.
• Decision: we expect we will implement the data model/pydantic version
  • Data model approach is cleaner / simpler and a good first step towards its implementation
  • RSG will open an issue on GitHub for discussion and to provide consortium members an opportunity to comment further in written format.
  • Next Thursday is the deadline for written comments, at which time if consensus remains that the data model plan is superior, we will begin the process of formally changing RSG’s scope of work.