Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failure when attempting to parse pendulum.DateTime into DataFrames #20544

Open
2 tasks done
danieljaffeovo opened this issue Jan 3, 2025 · 1 comment · May be fixed by #21003
Open
2 tasks done

Failure when attempting to parse pendulum.DateTime into DataFrames #20544

danieljaffeovo opened this issue Jan 3, 2025 · 1 comment · May be fixed by #21003
Assignees
Labels
A-temporal Area: date/time functionality bug Something isn't working P-medium Priority: medium python Related to Python Polars

Comments

@danieljaffeovo
Copy link

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

The script below shows how polars (1.18.0) has expected behaviour for datetime datetime objects and fails to parse the pendulum counterpart.

The error given by the script is:

polars.exceptions.ComputeError: unable to parse time zone: 'FixedTimezone(0, name="+00:00")'. Please check the Time Zone Database for a list of available time zones
import datetime as dt
import pendulum as pm
import polars as pl

# Set arbitrary string timestamp
dt_string = "2025-01-03 10:06:00.232323+00:00"

# Parse string as pendulum.DateTime
pm_dt = pm.from_format(dt_string, fmt="YYYY-MM-DD HH:mm:ss.SSSSSSZ")

# Parse string as datetime.datetime
dt_dt = dt.datetime.strptime(dt_string, "%Y-%m-%d %H:%M:%S.%f%z")

# Empty DataFrame #
df = pl.DataFrame()

# Add new column
# datetime_col works
# pendulum_col breaks
df = df.with_columns([
        pl.lit(pm_dt).alias("pendulum_col"),
        # pl.lit(dt_dt).alias("datetime_col"),
    ])

if __name__ == "__main__":
    print(df)

Log output

No response

Issue description

Polars does not parse pendulum.DateTime objects when adding to DataFrame with the with_columns method.

Issue appears to be with how timezone is encoded, although polars works fine with a timezone-aware datetime.datetime object.

Expected behavior

Polars to parse the pendulum.DateTime column and add as column to DataFrame with type polars.Datetime.

Installed versions

--------Version info---------
Polars:              1.18.0
Index type:          UInt32
Platform:            macOS-15.2-arm64-arm-64bit
Python:              3.11.9 (main, Aug 19 2024, 09:48:03) [Clang 15.0.0 (clang-1500.3.9.4)]
LTS CPU:             False
@danieljaffeovo danieljaffeovo added bug Something isn't working needs triage Awaiting prioritization by a maintainer python Related to Python Polars labels Jan 3, 2025
@alexander-beedie alexander-beedie added the A-temporal Area: date/time functionality label Jan 30, 2025
@alexander-beedie
Copy link
Collaborator

alexander-beedie commented Jan 30, 2025

Looks like we aren't properly identifying fixed-offset timezones for datetime objects passed to Python-side lit (note that the same "pendulum" datetime values work fine if passed directly into the DataFrame constructor).

Seems to be a more generic issue than just pendulum, as I can recreate the error with dateutil too. I'll take a look 👍

@alexander-beedie alexander-beedie added P-medium Priority: medium and removed needs triage Awaiting prioritization by a maintainer labels Jan 30, 2025
@alexander-beedie alexander-beedie self-assigned this Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-temporal Area: date/time functionality bug Something isn't working P-medium Priority: medium python Related to Python Polars
Projects
None yet
2 participants