Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: move CI to GHA #36

Merged
merged 2 commits into from
Mar 5, 2021
Merged

CI: move CI to GHA #36

merged 2 commits into from
Mar 5, 2021

Conversation

martinfleis
Copy link
Member

Closes #35

I have removed python 3.6 and added 3.9, which is run on Linux, windows and mac. Dev env uses master of geopandas, pygeos and dask.

It should also start using codecov for coverage.

@martinfleis
Copy link
Member Author

Oh, yeah. It won't activate until merged since there's no GHA workflow yet.

source activate test
pytest -v -r s --color=yes --cov=dask_geopandas --cov-append --cov-report term-missing --cov-report xml .

- uses: codecov/codecov-action@v1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

identation is off here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, fixed

@jorisvandenbossche
Copy link
Member

Let's test it in master

@jorisvandenbossche jorisvandenbossche merged commit 78f7c63 into geopandas:master Mar 5, 2021
@martinfleis martinfleis deleted the gha branch March 5, 2021 20:01
@martinfleis
Copy link
Member Author

Well, the good news is that CI seems to work. The bad that it fails.

@jorisvandenbossche
Copy link
Member

It's strange, it's the same test that is failing as on travis, but with a different error ...

@jorisvandenbossche
Copy link
Member

But now it seems as something we should be able to solve

@jorisvandenbossche
Copy link
Member

It might be a change in pyproj. If I update pyproj to the latest 3.0.1, then I can reproduce the error.

But given it's a strange decoding error, that might be the thread-safety issue that came up recently.

@jorisvandenbossche
Copy link
Member

Very short term "solution" might be to pin pyproj to an older version in our CI to get that green again.

A more proper short term solution might be to ensure that each partition gets a copy of the CRS, while now we set the same CRS object on each geopandas geodataframe (which are accessed from different threads):

def set_crs(self, value):
"""Set the value of the crs on a new object"""
return self.map_partitions(_set_crs, value, enforce_metadata=False)

def _set_crs(df, crs):
"""Return a new object with crs set to ``crs``"""
df = df.copy(deep=False)
df.crs = crs
return df

Eventually this will be fixed in pyproj as well. @snowman2 just opened a PR for that!

@snowman2
Copy link

snowman2 commented Mar 6, 2021

This is likely the issue: pyproj4/pyproj#589

I would be surprised if there is an older version of pyproj that worked. The issue appears to be kind of random.

You could test out this fix: pyproj4/pyproj#793

@jorisvandenbossche
Copy link
Member

You could test out this fix: pyproj4/pyproj#793

That's what I was planning. And indeed, with pyproj master I can reproduce the failure, with that branch it's fixed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI: expand CI and move it to GHA
3 participants