Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow parsing timezone without fully provided time #50178

Merged
merged 10 commits into from
Jan 8, 2020

Conversation

pgomulka
Copy link
Contributor

@pgomulka pgomulka commented Dec 13, 2019

strict_date_optional_time in Joda allowed to provide just a timezone without fully provided time. So dates like 2018-01-01T00+01 , 2018-01-01T00:00+01 , 2018-01-01T00:00:00+01 , 2018-01-01T00:00:00.000+01 were allowed.
This should be fixed in java.time implementation too.
However, Joda was incorrectly parsing 2018-01-01T+01 when no time part was provided. It is because +01 which should be considered an offset, is parsed as a signed hour part.
java.time implementation does not suffer from this and correctly parses this date. We are not allowing this though, as this is not allowed be iso8601 spec.
In essence this PR is only fixing parsing for 2018-01-01T00+01. Other formats were allowed already.
closes #49351

@pgomulka pgomulka added :Core/Infra/Core Core issues without another label v8.0.0 v7.6.0 labels Dec 13, 2019
@pgomulka pgomulka requested a review from rjernst December 13, 2019 13:43
@pgomulka pgomulka self-assigned this Dec 13, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Core)

@rjernst
Copy link
Member

rjernst commented Dec 13, 2019

I'm not sure this is something we want to do. I don't think ISO 8601 allows for timezone without a time. I thought we were moving towards deprecating joda specific functionality? This especially seems a weird feature to support since based on your description it appears joda doesn't actually support this as it intended, hence why the dueling tests are commented out.

@mark-vieira
Copy link
Contributor

FYI, you'll need to merge in the latest changes from master to fix these CI failure due to the new Java 13 build requirement.

@pgomulka
Copy link
Contributor Author

@elasticmachine update branch


public void testTimezoneParsing() {
//with Timezone
// assertSameDateAs("2016-11-30T+01", "strict_date_optional_time", "strict_date_optional_time");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we perhaps want a comment here about Joda as well?

@jamshid
Copy link

jamshid commented Dec 16, 2019

Thank you for fixing this!
Both 2016-11-30T12+01 and 2016-11-30T+01 were parsed as valid dates with 6.8.x.
But only the former will continue to parse in 7.6.0?

@pgomulka
Copy link
Contributor Author

@jamshid as per my comment here https://github.com/elastic/elasticsearch/pull/50178/files#diff-216a9f905247015f48298c3b6b1bb3a0R68
our strict_date_optional_time in Joda incorrectly parses 2016-11-30T+01 as 2016-11-30T01:00:00Z when it should be 2016-11-29T23:00:00Z. So even though you were not getting any exceptions, it was parsing these dates incorrectly
date_optional_time does not suffer from this

please bear in mind, I need to confirm with ISO8601 if this should be merged at all.

@pgomulka
Copy link
Contributor Author

@elasticmachine update branch

@pgomulka
Copy link
Contributor Author

@rjernst it looks that this should indeed be allowed as ISO/WD 8601-1 allows time representation with reduced accuracy
draft iso spec:
http://www.loc.gov/standards/datetime/iso-tc154-wg5_n0038_iso_wd_8601-1_2016-02-16.pdf
points 4.2.2.3 - 4.2.5.2.

@pgomulka
Copy link
Contributor Author

pgomulka commented Jan 3, 2020

I actually re-read the spec and it looks that timezone can only by applied with time.

Expressions	 of	 the	 difference	 between	 local	 time	 and	 UTC	 of	 day	 are	 a	 component	 in	 the	
representations	defined	in	4.2.5.2;	they	shall	not	be	used	as	self-standing	expressions.

where

4.2.5.2 Local	time	and	the	difference	from	UTC

allows reduced accuracy of local time. This is defined in

4.2.2.3 Representations	with	reduced	accuracy

and allows only hh:mm or hh

Therefore 2016-11-30T12+01 is correct
but 2016-11-30T+01 is incorrect.
You can also see a discussion here https://stackoverflow.com/questions/52215961/how-can-i-write-an-iso-8601-date-with-a-timezone-but-no-time-component
I understand this was parsed successfully in 6.8 but I would consider this to be a bug in joda.

@pgomulka
Copy link
Contributor Author

pgomulka commented Jan 7, 2020

@elasticmachine update branch

@pgomulka pgomulka merged commit cf933b1 into elastic:master Jan 8, 2020
pgomulka added a commit to pgomulka/elasticsearch that referenced this pull request Jan 8, 2020
strict_date_optional_time changes to have optional minute part.
It already allowed optional second and fraction of second part.
This allows parsing 2018-01-01T00+01 , 2018-01-01T00:00+01 , 2018-01-01T00:00:00+01 , 2018-01-01T00:00:00.000+01
It won't allow parsing a timezone without an hour part as this is not allowed by iso8601 spec

closes elastic#49351
pgomulka added a commit that referenced this pull request Jan 8, 2020
…50740)

strict_date_optional_time changes to have optional minute part.
It already allowed optional second and fraction of second part.
This allows parsing 2018-01-01T00+01 , 2018-01-01T00:00+01 , 2018-01-01T00:00:00+01 , 2018-01-01T00:00:00.000+01
It won't allow parsing a timezone without an hour part as this is not allowed by iso8601 spec

closes #49351
SivagurunathanV pushed a commit to SivagurunathanV/elasticsearch that referenced this pull request Jan 23, 2020
strict_date_optional_time changes to have optional minute part. It already allowed optional second and fraction of second part. This allows parsing 2018-01-01T00+01 , 2018-01-01T00:00+01 , 2018-01-01T00:00:00+01 , 2018-01-01T00:00:00.000+01
It won't allow parsing a timezone without an hour part as this is not allowed by iso8601 spec

closes elastic#49351
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Regression in date parsing ("2016-11-30T12+01" accepted before ES 7
8 participants