Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch exception to skip invalid file #27

Merged
merged 1 commit into from
Dec 30, 2022

Conversation

hugovk
Copy link
Collaborator

@hugovk hugovk commented Dec 29, 2022

Fixes #23.

I have a GPX file with a first <trkpt> element like this:

      <trkpt>
        <ele>31.2</ele>
        <time>2019-08-31T09:02:36Z</time>
        <extensions>
          <gpxdata:temp>0</gpxdata:temp>
        </extensions>
      </trkpt>

It's missing lon and lat elements. The rest look like this:

      <trkpt lon="24.12345" lat="60.12345">
        <ele>31.2</ele>
        <time>2019-08-31T09:02:37Z</time>
        <extensions>
          <gpxdata:temp>29</gpxdata:temp>
        </extensions>
      </trkpt>

Running on a directory containing it causes an exception: gpxpy.gpx.GPXException: latitude is mandatory in None (got None). Also, we don't know which file causes the problem.

Full traceback
Processing data...
Processing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
                    ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/stravavis/process_data.py", line 13, in process_file
    return process_gpx(fpath)
           ^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/stravavis/process_data.py", line 21, in process_gpx
    activity = gpxpy.parse(open(gpxfile))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/__init__.py", line 39, in parse
    return parser.parse(version)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/parser.py", line 154, in parse
    mod_gpxfield.gpx_fields_from_xml(self.gpx, root, version)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 590, in gpx_fields_from_xml
    value = gpx_field.from_xml(current_node, version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 241, in from_xml
    result.append(gpx_fields_from_xml(self.classs, child,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 590, in gpx_fields_from_xml
    value = gpx_field.from_xml(current_node, version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 241, in from_xml
    result.append(gpx_fields_from_xml(self.classs, child,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 590, in gpx_fields_from_xml
    value = gpx_field.from_xml(current_node, version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 241, in from_xml
    result.append(gpx_fields_from_xml(self.classs, child,
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 590, in gpx_fields_from_xml
    value = gpx_field.from_xml(current_node, version)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/gpxpy/gpxfield.py", line 197, in from_xml
    raise mod_gpx.GPXException(f'{self.name} is mandatory in {self.tag} (got {result})')
gpxpy.gpx.GPXException: latitude is mandatory in None (got None)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/bin/stravavis", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/stravavis/cli.py", line 51, in main
    df = process_data(args.path)
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/stravavis/process_data.py", line 108, in process_data
    processed = list(it)
                ^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/rich/progress.py", line 168, in track
    yield from progress.track(
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/rich/progress.py", line 1215, in track
    for value in sequence:
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 873, in next
    raise value
gpxpy.gpx.GPXException: latitude is mandatory in None (got None)

Let's catch this exception and instead skip the problematic file. Let's also print the filename so the user can check what the problem is (in my case I can manually delete the bad <trkpt> and try again).

$ stravavis tests/gpx
Processing data...
Processing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━   0% -:--:--
Skipping tests/gpx/invalid-lon-lat-missing.gpx: GPXException: latitude is mandatory in None (got None)
Processing ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Plotting facets...
Saved to strava-facets.png
Plotting map...
Plotting activities ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
Saved to strava-map.png
...

@hugovk hugovk merged commit 71d6d16 into marcusvolz:main Dec 30, 2022
@hugovk hugovk deleted the skip-invalid-file branch December 30, 2022 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Getting: "gpxpy.gpx.GPXException: latitude is mandatory in None (got None)"
1 participant