Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DEV] Use geofabrik .json for countries and regions for internal processing (from former constants.py and wahoomc/resources/json) #182

Merged
merged 22 commits into from
Feb 19, 2023

Conversation

treee111
Copy link
Owner

@treee111 treee111 commented Feb 19, 2023

This PR…

  • gets rid of the locally defined countries, continents (regions) and country to geofabrik + county to continents mapping for wahoomc internal processing
  • accesses the at all time downloaded geofabrik .json file for these types of checks via a own wahoomc/geofabrik_json.py file and class

Considerations and implementations

This PR takes away local hardcoded values in constants.py and wahoomc/resources/json/ by using fresh downloaded geofabrik .json therefore.
This ensures

  • up-to-date constants
  • less translation calls (country to geofabrik country etc.)
  • dismantling of workarounds
  • better readable code

and most important:

The local wahoomc/resources/json/ files are still there and the -co as well as -xy coding uses it. The reason is that X/Y processing at the time of writing is not integrated using geofabrik .json. For comparison of local json/ and geofabrik .json output, the files are still there for now.
In the next PR, all that will be done.

How to test

  1. do usual tests
  2. run unittests

Pull Request Checklist

should make it clearer like this
- less cluttered (access)
- only one read location of the .json file
- download might be placed to another place or changed
- dont use function translate_country_input_to_geofabrik as it make things harder to understand
- calc relevant countries in setUp. these should be valid for the whole class
- remove all countries which have no id in geofabrik file, makes most sense here
- for better access don't read the whole geofabrik and loop through it
- refactor get functions to reflect that and easier reading
did break test_generated_files and test_osm_maps

This reverts commit 03f31a2.
- tiles X/Y are mostly different
- latitude longitude are different even if X/Y are equal
not per each instantiation
- get_osm_pbf_filepath_url might be fully removed later on
- translating input to valid geofabrik ident no in class GeofabrikJson

- return of get_geofabrik_parent changed --> adjust test_reading_geofabrik_parent
- comparing via test_gofabrik_url_against_built_url makes no sense anymore
- superseeded by fully relying on geofabrik
- and adjust/ delete functions + test
@treee111 treee111 force-pushed the switch-to-geofabrik-constants branch from 9270b91 to 8a1ed3e Compare February 19, 2023 03:19
@treee111 treee111 changed the base branch from develop to release-v4.0.0 February 19, 2023 03:19
@treee111 treee111 changed the title [DEV] Switch to geofabrik constants [DEV] Switch from local to geofabrik .json constants for countries and regions Feb 19, 2023
@treee111 treee111 changed the title [DEV] Switch from local to geofabrik .json constants for countries and regions [DEV] Use geofabrik .json constants for countries and regions (from former constants.py) Feb 19, 2023
@treee111 treee111 changed the title [DEV] Use geofabrik .json constants for countries and regions (from former constants.py) [DEV] Use geofabrik .json constants for countries and regions as input and during processing (from former constants.py) Feb 19, 2023
@treee111 treee111 force-pushed the switch-to-geofabrik-constants branch from 7981dde to 6a5d00c Compare February 19, 2023 03:29
@treee111 treee111 marked this pull request as ready for review February 19, 2023 03:37
@treee111 treee111 changed the title [DEV] Use geofabrik .json constants for countries and regions as input and during processing (from former constants.py) [DEV] Use geofabrik .json for countries and regions as input and during processing (from former constants.py and wahoomc/resources/json) Feb 19, 2023
@treee111 treee111 added enhancement New feature or request refactoring labels Feb 19, 2023
- helper function equals the real function
- not needed commented out test
- autoformatting

test
@treee111 treee111 force-pushed the switch-to-geofabrik-constants branch from 6a5d00c to 9c3e7f2 Compare February 19, 2023 10:38
@treee111 treee111 changed the title [DEV] Use geofabrik .json for countries and regions as input and during processing (from former constants.py and wahoomc/resources/json) [DEV] Use geofabrik .json for countries and regions for internal processing (from former constants.py and wahoomc/resources/json) Feb 19, 2023
@treee111 treee111 merged commit 8632fbf into release-v4.0.0 Feb 19, 2023
@treee111 treee111 deleted the switch-to-geofabrik-constants branch February 19, 2023 20:50
treee111 added a commit that referenced this pull request May 5, 2023
…essing (from former `constants.py` and `wahoomc/resources/json`) (#182)

* variables are empty at that time

should make it clearer like this

* unittests for existing functionality

* geofabrik .json file as own class, move gefabrik functions

- less cluttered (access)
- only one read location of the .json file
- download might be placed to another place or changed

* correct json file to geofabrik country comparison

- dont use function translate_country_input_to_geofabrik as it make things harder to understand
- calc relevant countries in setUp. these should be valid for the whole class
- remove all countries which have no id in geofabrik file, makes most sense here

* unittest: compare built URL with geofabrik URL

* refactor geofabrik class

- for better access don't read the whole geofabrik and loop through it
- refactor get functions to reflect that and easier reading

* Revert "variables are empty at that time"

did break test_generated_files and test_osm_maps

This reverts commit 03f31a2.

* check input for valid geofabrik id

* unittest: compare regions

* unittests make no sense

- tiles X/Y are mostly different
- latitude longitude are different even if X/Y are equal

* only read geofabrik .json file once

not per each instantiation

* fetch geofabrik URL instead of composing

- get_osm_pbf_filepath_url might be fully removed later on
- translating input to valid geofabrik ident no in class GeofabrikJson

- return of get_geofabrik_parent changed --> adjust test_reading_geofabrik_parent

* reduce function to make it readabler

- comparing via test_gofabrik_url_against_built_url makes no sense anymore

* additional unittests

* remove unnused functions and constants

- superseeded by fully relying on geofabrik

* exchange continents and regions with geofabrik content

- and adjust/ delete functions + test

* after running vulture

* changes during coding review

- helper function equals the real function
- not needed commented out test
- autoformatting

test

* move geofabrik json class to own file

* Bump to version v4.0.0a0

* delete maps/ content version when last run version < v4.0.0a0
alfh pushed a commit to alfh/wahooMapsCreator that referenced this pull request Nov 4, 2023
…essing (from former `constants.py` and `wahoomc/resources/json`) (treee111#182)

* variables are empty at that time

should make it clearer like this

* unittests for existing functionality

* geofabrik .json file as own class, move gefabrik functions

- less cluttered (access)
- only one read location of the .json file
- download might be placed to another place or changed

* correct json file to geofabrik country comparison

- dont use function translate_country_input_to_geofabrik as it make things harder to understand
- calc relevant countries in setUp. these should be valid for the whole class
- remove all countries which have no id in geofabrik file, makes most sense here

* unittest: compare built URL with geofabrik URL

* refactor geofabrik class

- for better access don't read the whole geofabrik and loop through it
- refactor get functions to reflect that and easier reading

* Revert "variables are empty at that time"

did break test_generated_files and test_osm_maps

This reverts commit 03f31a2.

* check input for valid geofabrik id

* unittest: compare regions

* unittests make no sense

- tiles X/Y are mostly different
- latitude longitude are different even if X/Y are equal

* only read geofabrik .json file once

not per each instantiation

* fetch geofabrik URL instead of composing

- get_osm_pbf_filepath_url might be fully removed later on
- translating input to valid geofabrik ident no in class GeofabrikJson

- return of get_geofabrik_parent changed --> adjust test_reading_geofabrik_parent

* reduce function to make it readabler

- comparing via test_gofabrik_url_against_built_url makes no sense anymore

* additional unittests

* remove unnused functions and constants

- superseeded by fully relying on geofabrik

* exchange continents and regions with geofabrik content

- and adjust/ delete functions + test

* after running vulture

* changes during coding review

- helper function equals the real function
- not needed commented out test
- autoformatting

test

* move geofabrik json class to own file

* Bump to version v4.0.0a0

* delete maps/ content version when last run version < v4.0.0a0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request refactoring
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant