Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev accessibilities #614

Merged
merged 138 commits into from
Jan 28, 2023
Merged
Show file tree
Hide file tree
Changes from 128 commits
Commits
Show all changes
138 commits
Select commit Hold shift + click to select a range
b3df946
initial commit with basic working proto pop gen
nick-fournier-rsg Jun 29, 2022
6441c9a
fixed bad dependency range
nick-fournier-rsg Jun 29, 2022
1b8511f
fixed vary_on method to vary all with mapped fields after
nick-fournier-rsg Jul 2, 2022
96fca8e
added check if already dataframe
nick-fournier-rsg Jul 20, 2022
e92ed4a
added dataframe check if the table passed is already a dataframe
nick-fournier-rsg Jul 25, 2022
68ff25e
working proto pop gen. still debugging mandatory work/school model
nick-fournier-rsg Jul 25, 2022
5189fc8
check all table is df
nick-fournier-rsg Jul 25, 2022
a8c2376
workplace runs, next testing school
nick-fournier-rsg Jul 25, 2022
c82504f
began setting up to get all logsums, not just chosen ones
nick-fournier-rsg Jul 27, 2022
b9fa41b
restructured to run as either one-off or model step
nick-fournier-rsg Jul 28, 2022
7838d2f
revert to last merge. Removed dataframe check in iterate, no longer n…
nick-fournier-rsg Aug 24, 2022
286b5c9
extracted logsums, need to inject into pipeline or save output?
nick-fournier-rsg Aug 24, 2022
c835590
working standalone or step model
nick-fournier-rsg Aug 26, 2022
a73e6a4
extractable accessibilities for fixed and nonmandatory
nick-fournier-rsg Aug 30, 2022
194d327
extractable accessibilities for fixed and nonmandatory
nick-fournier-rsg Aug 30, 2022
a237cfd
cleanup and add documentation
nick-fournier-rsg Aug 31, 2022
d5a7bdd
resolve minor conflicts in model __init__ and conda env yml
nick-fournier-rsg Aug 31, 2022
2e993d9
resolved duplicates of old 'example' folders
nick-fournier-rsg Aug 31, 2022
3bb82b1
model uses write_tables functionality
nick-fournier-rsg Sep 1, 2022
6791a43
disaggregate accessibility runs as model step or standalone. Runs as …
nick-fournier-rsg Sep 2, 2022
3022f67
working model test. table initialization not yet working
nick-fournier-rsg Sep 2, 2022
256b778
added logsum short circuit to get logsums and avoid drawing unneccesa…
nick-fournier-rsg Sep 7, 2022
ee3a1f1
working model plus major cleanup
nick-fournier-rsg Sep 7, 2022
69efa7b
working model plus major cleanup
nick-fournier-rsg Sep 7, 2022
c6f26b5
override sample size
nick-fournier-rsg Sep 8, 2022
a26902c
fixed trace output files
nick-fournier-rsg Sep 8, 2022
c877a46
skip_choice default to False
nick-fournier-rsg Sep 8, 2022
765fd74
fixed empty logsums bug
nick-fournier-rsg Sep 8, 2022
5b27210
cleanup redundent parameters
nick-fournier-rsg Sep 8, 2022
a2137c3
fixed list index instead of set
nick-fournier-rsg Sep 8, 2022
df97dd1
coordinated model run files
nick-fournier-rsg Sep 8, 2022
2ba4249
added default skip to false
nick-fournier-rsg Sep 8, 2022
a66366e
added multiprocessing line
nick-fournier-rsg Sep 9, 2022
120c760
began setup for multiprocessing
nick-fournier-rsg Sep 12, 2022
9a663dd
fixed sampling problem. Pipeline for multi processing, NOT WORKING YET
nick-fournier-rsg Sep 12, 2022
753fa0b
deleted run file, not needed anymore
nick-fournier-rsg Sep 12, 2022
9dd261b
added helper functions for handling suffix args
nick-fournier-rsg Sep 14, 2022
13df8ed
to enable multiprocessing, overhauled pipeline method to include prot…
nick-fournier-rsg Sep 14, 2022
141bb64
working multiprocess, but requires debugging. fails on other models
nick-fournier-rsg Sep 14, 2022
ac3edd3
working multiprocess, but requires debugging. fails on other models
nick-fournier-rsg Sep 14, 2022
6b2181d
cleanup of mp table registering
nick-fournier-rsg Sep 15, 2022
005307d
fixed tracing and slicing issue
nick-fournier-rsg Sep 15, 2022
2c195f0
removed old 'run' model
nick-fournier-rsg Sep 15, 2022
5198916
setup example 2 zones
nick-fournier-rsg Sep 26, 2022
98efe5c
minimum working 2 zone
nick-fournier-rsg Sep 26, 2022
4771bf1
multizone basic working
nick-fournier-rsg Sep 26, 2022
be5d2ca
fixed buggy settings
nick-fournier-rsg Sep 26, 2022
26c57a7
multiprocessing working, but empty final tables
dhensle Sep 27, 2022
749c65c
fixed blank output bug and cleaned up settings to use verbose table n…
nick-fournier-rsg Sep 27, 2022
01b0e5b
Merge branch 'dev_accessibilities' of github.com:nick-fournier-rsg/ac…
nick-fournier-rsg Sep 27, 2022
0902f88
cleanup run scripts a bit
nick-fournier-rsg Sep 27, 2022
00ba5f9
fixed missing base path for full run
nick-fournier-rsg Sep 27, 2022
ea82c39
fixed missing path
nick-fournier-rsg Sep 27, 2022
570b45c
fixed typo
nick-fournier-rsg Sep 27, 2022
9364cce
fixed 1 per taz sampling
nick-fournier-rsg Sep 27, 2022
25d01c7
fixing person merge bug and filtering workers
dhensle Sep 27, 2022
263cca6
Merge branch 'dev_accessibilities' of https://github.com/nick-fournie…
dhensle Sep 27, 2022
9e5053c
fixed duplicate tours!
nick-fournier-rsg Sep 28, 2022
8739d56
updated settings for initialize disagg
nick-fournier-rsg Sep 29, 2022
f41c769
removed obsolete file
nick-fournier-rsg Sep 29, 2022
b36de0f
cleaned up file and moved initialize to tables/...
nick-fournier-rsg Sep 29, 2022
3013f7c
added find nearest node function
nick-fournier-rsg Sep 29, 2022
4a179a9
added initialize disaggregate_accessibility to initialize_households
nick-fournier-rsg Sep 29, 2022
06e7178
moved initialize steps to 'tables/disaggregate_accessibility'
nick-fournier-rsg Sep 30, 2022
8b1d529
updated settings to include initialize disaggregate accessibilities a…
nick-fournier-rsg Sep 30, 2022
991b2c6
moved initialize disagg accessibilities to tables. Created working me…
nick-fournier-rsg Sep 30, 2022
17b3fea
add mp back in
nick-fournier-rsg Sep 30, 2022
cc97d5f
PEP formatting revisions
nick-fournier-rsg Sep 30, 2022
7b4b780
fixed logsums merge on households instead of persons
nick-fournier-rsg Sep 30, 2022
895438e
fixed _accessibility suffix
nick-fournier-rsg Sep 30, 2022
d9c465c
fixed conflict with persons_merged
nick-fournier-rsg Sep 30, 2022
e1b1b0c
updated yaml to use simpler join method
nick-fournier-rsg Sep 30, 2022
ca1c130
Merge remote-tracking branch 'origin/develop' into dev_accessibilities
nick-fournier-rsg Sep 30, 2022
ba27eb3
PEP formatting fixes
nick-fournier-rsg Oct 1, 2022
9c295f9
refreshed example folder from develop
nick-fournier-rsg Oct 1, 2022
56184ad
added missing line at end of file
nick-fournier-rsg Oct 1, 2022
1e78698
black fixes
nick-fournier-rsg Oct 1, 2022
301ac23
black fixes to disaggregate accessibility changes
nick-fournier-rsg Oct 1, 2022
7be7dd8
Merge branch 'ActivitySim:main' into dev_accessibilities
nick-fournier-rsg Oct 3, 2022
188a05f
fixed missing column pipeline error
nick-fournier-rsg Oct 3, 2022
0da4af9
merged disagg accessibilities into mtc_extended and added doc section
nick-fournier-rsg Oct 3, 2022
a2a743c
ran black on disaggregate_accessibility.py
nick-fournier-rsg Oct 3, 2022
ff51013
updated dependencies
nick-fournier-rsg Oct 3, 2022
25ddd57
removing sklearn imports
dhensle Oct 3, 2022
4298141
blacken
dhensle Oct 3, 2022
9fa6d0b
add if none catch
nick-fournier-rsg Oct 3, 2022
4123c86
fixed None suffix default
nick-fournier-rsg Oct 3, 2022
0941d01
moved order of get_table(persons_merged) to avoid pulling prematurely…
nick-fournier-rsg Oct 3, 2022
40a8170
tested and cleaned up rng channels
nick-fournier-rsg Oct 4, 2022
079dcfb
setup injectable suffixes to allow add_size_table as model step not j…
nick-fournier-rsg Oct 4, 2022
2187a4b
removed accessibility output from test
nick-fournier-rsg Oct 4, 2022
d52666e
re blacken py scripts
nick-fournier-rsg Oct 4, 2022
8ab2f65
fixed tracing typo
nick-fournier-rsg Oct 4, 2022
29a9ddc
added variable index name suffix to pass optionally
nick-fournier-rsg Oct 4, 2022
d4886ee
pipeline housekeeping to cleanup any tables, traceables, or channels …
nick-fournier-rsg Oct 5, 2022
5406541
added multiprocess testing
nick-fournier-rsg Oct 5, 2022
6521f56
blacken updates
nick-fournier-rsg Oct 5, 2022
82c36c7
updated test scripts to include MP, problem with vehicle model in mp …
nick-fournier-rsg Oct 5, 2022
ecd53a6
added improved origin sampling, resolved issue with merging with samp…
nick-fournier-rsg Oct 7, 2022
4e01bf2
added sci-kit learn to test depends
nick-fournier-rsg Oct 7, 2022
b6ac7ef
fixed person merging error causing pytest fail, uses inject method to…
nick-fournier-rsg Oct 7, 2022
f53ec77
cleanup comments
nick-fournier-rsg Oct 7, 2022
35d0a5d
fixed pytest to include accessibility table in regress
nick-fournier-rsg Oct 7, 2022
d5042f5
setup for mp test, but needs debugging
nick-fournier-rsg Oct 7, 2022
d5132fc
'blacken'
nick-fournier-rsg Oct 7, 2022
ca317fc
'blacken'
nick-fournier-rsg Oct 7, 2022
935b42d
cleanup example folder
nick-fournier-rsg Oct 7, 2022
b351eb0
fixed pipeline NoneType bug for disagg accessibility table
nick-fournier-rsg Oct 7, 2022
5ab73ed
add mp test for prototype_mtc_extended
dhensle Oct 11, 2022
a52bb48
merging mp_testing
dhensle Oct 11, 2022
e41fe66
fixed pytest fail on mp, due to exept:True on mp_simulate
nick-fournier-rsg Oct 11, 2022
34e08b3
'blacken'
nick-fournier-rsg Oct 11, 2022
87c251f
added weighted k-means method
nick-fournier-rsg Oct 13, 2022
aeff8e8
created run script for multiple sampling scenarios
nick-fournier-rsg Oct 13, 2022
93544fd
blacken changes
nick-fournier-rsg Oct 13, 2022
47d1e6b
blacken
nick-fournier-rsg Oct 13, 2022
9a9c9cf
fixed copy script
nick-fournier-rsg Oct 17, 2022
a291027
blacken sampling script
nick-fournier-rsg Oct 17, 2022
8180d7d
fixed n_zone when integer
nick-fournier-rsg Oct 17, 2022
5960f40
fixed typo
nick-fournier-rsg Oct 17, 2022
6584f7b
update sampling script
nick-fournier-rsg Oct 25, 2022
dada5cd
'blacken'
nick-fournier-rsg Oct 25, 2022
1096cfb
fixed replacement sample bug
nick-fournier-rsg Oct 25, 2022
0d0a7c3
updated documentation
nick-fournier-rsg Oct 25, 2022
17c6b21
more flexible scenario testing
nick-fournier-rsg Oct 26, 2022
7dde020
blacken
nick-fournier-rsg Oct 26, 2022
73264ab
added test runs to __main__ in test_mtc_extended
dhensle Oct 27, 2022
f5f4a2e
additional logging, random seed option, landuse sample weight column
dhensle Nov 2, 2022
bb813c0
revisions per Sijia's comments, except for the proto-table-template i…
nick-fournier-rsg Dec 7, 2022
f738aa3
added template pop option
nick-fournier-rsg Dec 8, 2022
ce78cfb
blacken
nick-fournier-rsg Dec 8, 2022
f02c466
set back to create_tables to pass test against old tables
nick-fournier-rsg Dec 8, 2022
57e726a
updated proto-pop test data
nick-fournier-rsg Dec 8, 2022
f0b6524
resolved bad test data issue
nick-fournier-rsg Dec 8, 2022
6a845e4
optimize nearest zone determination
dhensle Dec 15, 2022
902cb81
Merge branch 'develop' into dev_accessibilities
dhensle Dec 15, 2022
d132ac2
adding sandag 2-zone test
dhensle Dec 17, 2022
9db0029
Merge branch 'develop' into dev_accessibilities
jpn-- Dec 30, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions activitysim/abm/models/__init__.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
# ActivitySim
# See full license in LICENSE.txt.

from . import (
accessibility,
disaggregate_accessibility,
atwork_subtour_destination,
atwork_subtour_frequency,
atwork_subtour_mode_choice,
Expand Down
660 changes: 660 additions & 0 deletions activitysim/abm/models/disaggregate_accessibility.py

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions activitysim/abm/models/initialize.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import pandas as pd

from activitysim.abm.tables import shadow_pricing
from activitysim.abm.tables import shadow_pricing, disaggregate_accessibility
from activitysim.core import chunk, config, expressions, inject, mem, pipeline, tracing
from activitysim.core.steps.output import (
track_skim_usage,
Expand Down Expand Up @@ -141,7 +141,8 @@ def initialize_households():
add_size_tables = model_settings.get("add_size_tables", True)
if add_size_tables:
# warnings.warn(f"Calling add_size_tables from initialize will be removed in the future.", FutureWarning)
shadow_pricing.add_size_tables()
suffixes = inject.get_injectable("disaggregate_suffixes")
shadow_pricing.add_size_tables(suffixes)

# - preload person_windows
person_windows = inject.get_table("person_windows").to_frame()
Expand Down
4 changes: 4 additions & 0 deletions activitysim/abm/models/location_choice.py
Original file line number Diff line number Diff line change
Expand Up @@ -524,6 +524,7 @@ def run_location_simulate(
chunk_size,
chunk_tag,
trace_label,
skip_choice=False,
):
"""
run location model on location_sample annotated with mode_choice logsum
Expand Down Expand Up @@ -594,6 +595,7 @@ def run_location_simulate(
trace_label=trace_label,
trace_choice_name=model_settings["DEST_CHOICE_COLUMN_NAME"],
estimator=estimator,
skip_choice=skip_choice,
)

if not want_logsums:
Expand All @@ -618,6 +620,7 @@ def run_location_choice(
chunk_tag,
trace_hh_id,
trace_label,
skip_choice=False,
):
"""
Run the three-part location choice algorithm to generate a location choice for each chooser
Expand Down Expand Up @@ -716,6 +719,7 @@ def run_location_choice(
trace_label=tracing.extend_trace_label(
trace_label, "simulate.%s" % segment_name
),
skip_choice=skip_choice,
)

if estimator:
Expand Down
31 changes: 24 additions & 7 deletions activitysim/abm/models/util/tour_destination.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,9 +119,12 @@ def _destination_sample(
trace_label=trace_label,
)

# if special person id is passed
chooser_id_column = model_settings.get("CHOOSER_ID_COLUMN", "person_id")

# remember person_id in chosen alts so we can merge with persons in subsequent steps
# (broadcasts person_id onto all alternatives sharing the same tour_id index value)
choices["person_id"] = choosers.person_id
choices[chooser_id_column] = choosers[chooser_id_column]

return choices

Expand Down Expand Up @@ -519,14 +522,18 @@ def run_destination_sample(

# FIXME - MEMORY HACK - only include columns actually used in spec (omit them pre-merge)
chooser_columns = model_settings["SIMULATE_CHOOSER_COLUMNS"]

# if special person id is passed
chooser_id_column = model_settings.get("CHOOSER_ID_COLUMN", "person_id")

persons_merged = persons_merged[
[c for c in persons_merged.columns if c in chooser_columns]
]
tours = tours[
[c for c in tours.columns if c in chooser_columns or c == "person_id"]
[c for c in tours.columns if c in chooser_columns or c == chooser_id_column]
]
choosers = pd.merge(
tours, persons_merged, left_on="person_id", right_index=True, how="left"
tours, persons_merged, left_on=chooser_id_column, right_index=True, how="left"
)

# interaction_sample requires that choosers.index.is_monotonic_increasing
Expand Down Expand Up @@ -576,7 +583,7 @@ def run_destination_sample(

# remember person_id in chosen alts so we can merge with persons in subsequent steps
# (broadcasts person_id onto all alternatives sharing the same tour_id index value)
choices["person_id"] = tours.person_id
choices[chooser_id_column] = tours[chooser_id_column]

return choices

Expand Down Expand Up @@ -612,6 +619,8 @@ def run_destination_logsums(
"""

logsum_settings = config.read_model_settings(model_settings["LOGSUM_SETTINGS"])
# if special person id is passed
chooser_id_column = model_settings.get("CHOOSER_ID_COLUMN", "person_id")

chunk_tag = "tour_destination.logsums"

Expand All @@ -624,7 +633,7 @@ def run_destination_logsums(
choosers = pd.merge(
destination_sample,
persons_merged,
left_on="person_id",
left_on=chooser_id_column,
right_index=True,
how="left",
)
Expand Down Expand Up @@ -662,6 +671,7 @@ def run_destination_simulate(
estimator,
chunk_size,
trace_label,
skip_choice=False,
):
"""
run destination_simulate on tour_destination_sample
Expand All @@ -678,14 +688,18 @@ def run_destination_simulate(

# FIXME - MEMORY HACK - only include columns actually used in spec (omit them pre-merge)
chooser_columns = model_settings["SIMULATE_CHOOSER_COLUMNS"]

# if special person id is passed
chooser_id_column = model_settings.get("CHOOSER_ID_COLUMN", "person_id")

persons_merged = persons_merged[
[c for c in persons_merged.columns if c in chooser_columns]
]
tours = tours[
[c for c in tours.columns if c in chooser_columns or c == "person_id"]
[c for c in tours.columns if c in chooser_columns or c == chooser_id_column]
]
choosers = pd.merge(
tours, persons_merged, left_on="person_id", right_index=True, how="left"
tours, persons_merged, left_on=chooser_id_column, right_index=True, how="left"
)

# interaction_sample requires that choosers.index.is_monotonic_increasing
Expand Down Expand Up @@ -743,6 +757,7 @@ def run_destination_simulate(
trace_label=trace_label,
trace_choice_name="destination",
estimator=estimator,
skip_choice=skip_choice,
)

if not want_logsums:
Expand All @@ -764,6 +779,7 @@ def run_tour_destination(
chunk_size,
trace_hh_id,
trace_label,
skip_choice=False,
):

size_term_calculator = SizeTermCalculator(model_settings["SIZE_TERM_SELECTOR"])
Expand Down Expand Up @@ -839,6 +855,7 @@ def run_tour_destination(
estimator=estimator,
chunk_size=chunk_size,
trace_label=tracing.extend_trace_label(segment_trace_label, "simulate"),
skip_choice=skip_choice,
)

choices_list.append(choices)
Expand Down
1 change: 1 addition & 0 deletions activitysim/abm/tables/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
# See full license in LICENSE.txt.
from . import (
accessibility,
disaggregate_accessibility,
households,
landuse,
persons,
Expand Down
Loading