Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] [Follow on] Follow on issues for the checking non UTC test cases. #9627

Open
3 tasks
res-life opened this issue Nov 3, 2023 · 1 comment
Open
3 tasks
Labels
test Only impacts tests

Comments

@res-life
Copy link
Collaborator

res-life commented Nov 3, 2023

Describe the bug

After #9482 is merged, should further finish the following items:

  • Test permerge with a time zone Env variable, by default it's UTC.
import os
def get_time_zone_for_testing():
  tz = os.getenv('TEST_SPARK_SESSION_TIMEZONE')
  if tz is None:
    tz = "UTC"
  else:
    tz = tz.strip()
  return tz

# spark_session.py
tz = get_time_zone_for_testing()
_spark.conf.set("spark.sql.session.timeZone", tz)

# conftest.py
    if (item.get_closest_marker('disable_timezone_test') and 
        get_time_zone_for_testing() != "UTC"):
        // for cases disable non-utc test and current timezone for testing is non-utc, skip them.
        pytest.skip('Skip because this case is not ready for non UTC time zone')
  • Update CI to testing non UTC for some Spark versions, e.g.: only for Spark 311 and Spark 350

  • For the cases with 'disable_timezone_test' mark, these cases was skipped when timezone is non UTC. We should confirm they fallbacks to CPU when timezone is non UTC either by manually way or write corresponding case for each case that has 'disable_timezone_test' mark

Additional context
Finally we should remove the fallback cases that testing non UTC.

@res-life res-life added bug Something isn't working ? - Needs Triage Need team to review and classify labels Nov 3, 2023
@res-life res-life self-assigned this Nov 7, 2023
@res-life res-life changed the title [BUG] [Follow on] Follow on issues for the checking non UTC test cases. [FEA] [Follow on] Follow on issues for the checking non UTC test cases. Nov 7, 2023
@mattahrens mattahrens added test Only impacts tests and removed ? - Needs Triage Need team to review and classify bug Something isn't working labels Nov 7, 2023
@res-life
Copy link
Collaborator Author

Not planing work on this for release 24.04, unassign it from me.

@res-life res-life removed their assignment Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

No branches or pull requests

2 participants