Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the conversion error for pandas.Series with missing values in pandas<=2.1 #3505

Merged
merged 4 commits into from
Oct 11, 2024

Conversation

seisman
Copy link
Member

@seisman seisman commented Oct 10, 2024

Description of proposed changes

See #2844 for the issue report.

It turns out that in pandas <= 2.1, pd.Series with integer values and pd.NA are converted to object type, which can't be recognized later, but in pandas >= 2.2, they are converted to float type.

In this PR, we add a workaround for pandas<=2.1 by casting the pd.Series to float type explicitly if it contains pd.NA.

Closes #2844.

As per SPEC 0 policy, support of pandas 2.1 should be dropped in 2025 Q3, so we expect to remove the workaround for PyGMT v0.17.0.

pygmt/clib/conversion.py Outdated Show resolved Hide resolved
@seisman seisman added bug Something isn't working needs review This PR has higher priority and needs review. labels Oct 11, 2024
@seisman seisman added this to the 0.14.0 milestone Oct 11, 2024
@seisman seisman changed the title POC: Add workaround for pandas.NA support with pandas<=2.1 Fix the conversion error for pandas.Series with missing values in pandas<=2.1 Oct 11, 2024
@seisman seisman marked this pull request as ready for review October 11, 2024 01:58
@seisman seisman requested a review from weiji14 October 11, 2024 02:15
pygmt/clib/conversion.py Outdated Show resolved Hide resolved
@seisman seisman added final review call This PR requires final review and approval from a second reviewer and removed needs review This PR has higher priority and needs review. labels Oct 11, 2024
@seisman seisman removed the final review call This PR requires final review and approval from a second reviewer label Oct 11, 2024
@seisman seisman merged commit 8e04d00 into main Oct 11, 2024
5 checks passed
@seisman seisman deleted the pandas/NA branch October 11, 2024 05:59
seisman added a commit that referenced this pull request Oct 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error when passing pd.NA value in a data column with pandas nullable int dtype
3 participants