Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sharrow Cache Dir Setting #893

Merged
merged 2 commits into from
Oct 24, 2024
Merged

Sharrow Cache Dir Setting #893

merged 2 commits into from
Oct 24, 2024

Conversation

dhensle
Copy link
Contributor

@dhensle dhensle commented Oct 1, 2024

Fix for #890. Also fixes the same issue for profile_dir and pipeline_file_name settings.

@dhensle dhensle requested review from jpn-- and JoeJimFlood October 3, 2024 16:44
@@ -478,15 +478,12 @@ def load_settings(self) -> State:
include_stack=False,
)

# the settings can redefine the cache directories.
cache_dir = raw_settings.pop("cache_dir", None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We used "pop" here, this takes this setting off the raw_settings. The revision leaves the setting in place... should we be popping all these values?

I think we need to write a test [or tests] to make sure this is solving the problem well.

@jpn--
Copy link
Member

jpn-- commented Oct 24, 2024

This appears to work. We need to have a test to confirm that, but writing one is some effort and in the meantime consortium members will benefit from having this fixed. I opened #902 to record that we want this tested, and will merge this for now.

@jpn-- jpn-- merged commit cecb570 into ActivitySim:main Oct 24, 2024
17 checks passed
@dhensle dhensle deleted the sharrow_cache_fix branch November 7, 2024 23:54
dhensle added a commit to RSGInc/activitysim that referenced this pull request Dec 9, 2024
* pydantic for estimation settings

* allow df as type in config

* fix table_info

* auto ownership

* repair for pydantic

* update for ruff

* updated for simple models

* repair for Pydantic

* simple simulate and location choice

* df is attribute

* scheduling

* stop freq

* test locations

* cdap

* nonmand_and_joint_tour_dest_choice

* nonmand_tour_freq

* fix ci to stop using mamba

* test updates

* use larch6 from pip

* use numba for stop freq

* fix for pandas 1.5

* fix stop freq test for numba

* Sharrow Cache Dir Setting (ActivitySim#893)

* setting necessary filesystem changes from settings file

* set for multiprocessing

* repair github actions

* github action updates (ActivitySim#903)

* script to make data

* unified script for making data

* remove older

* bug

* doc note

* load from parquet if available

* add original alt ids to EDB output when using compact

* fix MP race

* script arg to skip to EDB

* clean up CDAP and blacken

* refactor model_estimation_table_types

change to estimation_table_types, to avoid pydantic namespace clash

* repair drop_dupes

* blacken

* location choice with compact

* choice_def for compact

* spec changes for simple-simulate

* re-estimation demo for auto ownership

* clean up status messages

* change name to stop pydantic warnings

* edit configs

* default estimation sample size is same as regular sample size

* allow location alts not in cv format

* dummy zones for location choice

* update scheduling model estimation

* various cleanup

* stop freq

* tidy build script

* update 02 school location for larger example

* update notebook 04

* editable model re-estimation for location choice

* fix test names

* update notebooks

* cdap print filenames as loading

* notebook 07

* tests thru 07

* notebooks 08 09

* build the data first

* runnable script

* change larch version dependency

* keep pandas<2

* notebooks 10 11

* notebook 12

* remove odd print

* add matplotlib

* notebook 13 14

* test all the notebooks

* add xlsxwriter to tests

* notebook 15

* CDAP revise model spec demo

* notebook 16

* notebook 17

* longer timeout

* notebook 18

* notebook 19

* notebook 20

* smaller notebook 15

* configurable est mode setup

* notebook 21

* notebook 22

* config sample size in GA

* notebook 23

* updates for larch and graphviz

* change default to compact

* compare model 03

* test updates

* rename test targets

* repair_av_zq

* move doctor up

* add another repair

* oops

---------

Co-authored-by: David Hensle <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants