-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Convert notebooks to scripts #23
Conversation
|
||
# temporary reduction of the dataset for quick analysis | ||
# TODO: check if this should be present? | ||
spc = spc.head(15000) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps can be an input parameter for the script to determine if run on the whole population?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the readme! Everything else looks good to me, except I can't run the 1_prep_synthpop.py. I think this is an issue in my setup so can revisit it in the weekly meeting. Happy to merge this if @BZ-BowenZhang is ok with it also
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no oa11cd
and msoa11cd
found in the region_people_hh.parquet
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I checked the output file from scripts/1_prep_synthpop.py
, the column names are oa
and msoa
, I think that is the reason for this error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The acbm
package automatically installed from poetry does not have the processing
sub-module, I had to manually move the processing.py
into the folder to fix this issue
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This error seems fixed when I ran poetry update
again
This PR closes #21 converting the preprocessing notebooks to scripts so that the downstream data is constructed for notebooks 3 and onwards.
Outstanding areas to look at refining: