Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Some miscellaneous column mapping cleanup, mostly for the entity-specific columns which get turned into
operating_date
values. Things I came across while looking into a better fix for #423.operating_year
andoperating_month
columns in the EIA860M retired generators consistently -- these were getting silently lost because the other EIA860 generator operating date columns are namegenerator_operating_date
.*_(operating|retired)_(year|month)
columns have specific names in column mapping metadata CSVs, and remove the manual renames that were happening elsewhere to ensure that there aren't a bunch of different operating_date and retirement_date columns floating around in the transforms which actually apply to different entities. This change affected the operating month/year/date columns for cooling_system, emission_control, fgd, and fgp.convert_to_date()
andmonth_year_to_date()
helper functions next to each other in their module... they really need to be combined!obstacle_num_faa
toobstacle_id_faa
and update field description to provide more context on what those IDs refer to.core_eia860__yearly_generators_wind
tocore_eia860__scd_generators_wind
, as it's not a data table, but a generator attribute table.Testing
make pytest-validate
on the fully populated database and it passed.make pytest-integration-full
on the fully populated database and it passed.To-do list