Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JodaTime support month offsets are incorrect and cannot existing data with March 30 dates #37

Closed
Freaky-namuH opened this issue Feb 28, 2014 · 4 comments
Labels

Comments

@Freaky-namuH
Copy link
Contributor

Months in java.util.Calendar are '0' based where as in joda-time months are '1' based.

http://docs.oracle.com/javase/7/docs/api/java/util/Calendar.html#set(int, int, int, int, int, int)

month - the value used to set the MONTH calendar field. Month value is 0-based. e.g., 0 for January.

http://joda-time.sourceforge.net/apidocs/org/joda/time/LocalDate.html#LocalDate(int, int, int)

monthOfYear - the month of the year, from 1 to 12

This can be seen by reading a row that has an existing March 30th (2014-03-30) date in it, i.e. not one that has been inserted using the joda-time mapper.

It fails with:

org.joda.time.IllegalFieldValueException: Value 30 for dayOfMonth must be in the range [1,28]
    at org.joda.time.field.FieldUtils.verifyValueBounds(FieldUtils.java:236)
    at org.joda.time.chrono.BasicChronology.getDateMidnightMillis(BasicChronology.java:614)
    at org.joda.time.chrono.BasicChronology.getDateTimeMillis(BasicChronology.java:159)
    at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:120)
    at org.joda.time.LocalDate.<init>(LocalDate.java:457)
    at org.joda.time.LocalDate.<init>(LocalDate.java:436)
    at com.github.tminglei.slickpg.PgDateSupportJoda$class.com$github$tminglei$slickpg$PgDateSupportJoda$$sqlDate2jodaDate(PgDateSupportJoda.scala:53)
    at com.github.tminglei.slickpg.PgDateSupportJoda$DateTimeImplicits$$anonfun$1.apply(PgDateSupportJoda.scala:23)
    at com.github.tminglei.slickpg.PgDateSupportJoda$DateTimeImplicits$$anonfun$1.apply(PgDateSupportJoda.scala:23)
    at scala.Option.map(Option.scala:145)
    at com.github.tminglei.slickpg.date.PgDateJavaTypes$DateJdbcType.nextValue(PgDateJavaTypes.scala:33)
    at scala.slick.driver.JdbcTypesComponent$JdbcType$class.nextValueOrElse(JdbcTypesComponent.scala:30)
    at com.github.tminglei.slickpg.date.PgDateJavaTypes$DateJdbcType.nextValueOrElse(PgDateJavaTypes.scala:17)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:23)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:20)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:243)
    at scala.slick.driver.JdbcInvokerComponent$QueryInvoker.extractValue(JdbcInvokerComponent.scala:44)
    at scala.slick.jdbc.StatementInvoker$$anon$1.extractValue(StatementInvoker.scala:36)
    at scala.slick.jdbc.PositionedResultIterator.foreach(PositionedResult.scala:211)
    at scala.slick.jdbc.Invoker$class.foreach(Invoker.scala:98)
    at scala.slick.jdbc.StatementInvoker.foreach(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.build(Invoker.scala:69)
    at scala.slick.jdbc.StatementInvoker.build(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.list(Invoker.scala:59)
    at scala.slick.jdbc.StatementInvoker.list(StatementInvoker.scala:9)
    at scala.slick.jdbc.UnitInvoker$class.list(Invoker.scala:157)
    at scala.slick.driver.JdbcInvokerComponent$UnitQueryInvoker.list(JdbcInvokerComponent.scala:50)
    at Foo$$anonfun$run$1.apply(<console>:21)

For example, the mapping from a calendar to LocalDate is incorrect
https://github.com/tminglei/slick-pg/blob/master/addons/joda-time/src/main/scala/com/github/tminglei/slickpg/PgDateSupportJoda.scala#L55 as it does not add '1' to the MONTH.

The opposite is also true. https://github.com/tminglei/slick-pg/blob/master/addons/joda-time/src/main/scala/com/github/tminglei/slickpg/PgDateSupportJoda.scala#L55 Does not subtract '1' to the month.

@tminglei
Copy link
Owner

tminglei commented Mar 1, 2014

@Freaky-namuH can you provide your code snippet? I want to know more details.

(I remember I took into account for it in the codes.)

@Freaky-namuH
Copy link
Contributor Author

I've create a test project that highlight the issue using version 0.5.1.1.

To create the data:

create table datetest(id serial, a_date date);
insert into datetest(a_date) values('2014-03-30');
select * from datetest;

Run the au.com.myanaesthetic.slick.test.Main class using:

mvn scala:run

https://docs.google.com/file/d/0B-OQSZyNiKpIM1ppbTE1blFhbXM/edit

When you run it you'll see

Exception in thread "main" org.joda.time.IllegalFieldValueException: Value 30 for dayOfMonth must be in the range [1,28]
    at org.joda.time.field.FieldUtils.verifyValueBounds(FieldUtils.java:236)
    at org.joda.time.chrono.BasicChronology.getDateMidnightMillis(BasicChronology.java:614)
    at org.joda.time.chrono.BasicChronology.getDateTimeMillis(BasicChronology.java:159)
    at org.joda.time.chrono.AssembledChronology.getDateTimeMillis(AssembledChronology.java:120)
    at org.joda.time.LocalDate.<init>(LocalDate.java:457)
    at org.joda.time.LocalDate.<init>(LocalDate.java:436)
    at com.github.tminglei.slickpg.PgDateSupportJoda$class.com$github$tminglei$slickpg$PgDateSupportJoda$$sqlDate2jodaDate(PgDateSupportJoda.scala:53)
    at com.github.tminglei.slickpg.PgDateSupportJoda$DateTimeImplicits$$anonfun$1.apply(PgDateSupportJoda.scala:23)
    at com.github.tminglei.slickpg.PgDateSupportJoda$DateTimeImplicits$$anonfun$1.apply(PgDateSupportJoda.scala:23)
    at scala.Option.map(Option.scala:145)
    at com.github.tminglei.slickpg.date.PgDateJavaTypes$DateJdbcType.nextValue(PgDateJavaTypes.scala:33)
    at scala.slick.driver.JdbcTypesComponent$JdbcType$class.nextValueOrElse(JdbcTypesComponent.scala:30)
    at com.github.tminglei.slickpg.date.PgDateJavaTypes$DateJdbcType.nextValueOrElse(PgDateJavaTypes.scala:17)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:23)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:20)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:243)
    at scala.slick.profile.RelationalMappingCompilerComponent$TypeMappingResultConverter.read(RelationalProfile.scala:262)
    at scala.slick.driver.JdbcInvokerComponent$QueryInvoker.extractValue(JdbcInvokerComponent.scala:44)
    at scala.slick.jdbc.StatementInvoker$$anon$1.extractValue(StatementInvoker.scala:36)
    at scala.slick.jdbc.PositionedResultIterator.foreach(PositionedResult.scala:211)
    at scala.slick.jdbc.Invoker$class.foreach(Invoker.scala:98)
    at scala.slick.jdbc.StatementInvoker.foreach(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.build(Invoker.scala:69)
    at scala.slick.jdbc.StatementInvoker.build(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.list(Invoker.scala:59)
    at scala.slick.jdbc.StatementInvoker.list(StatementInvoker.scala:9)
    at scala.slick.jdbc.UnitInvoker$class.list(Invoker.scala:157)
    at scala.slick.driver.JdbcInvokerComponent$UnitQueryInvoker.list(JdbcInvokerComponent.scala:50)
    at au.com.myanaesthetic.slick.test.Db$$anonfun$testDatetimeFunctions$1.apply(Db.scala:35)
    at au.com.myanaesthetic.slick.test.Db$$anonfun$testDatetimeFunctions$1.apply(Db.scala:34)
    at scala.slick.backend.DatabaseComponent$DatabaseDef$class.withSession(DatabaseComponent.scala:31)
    at scala.slick.jdbc.JdbcBackend$DatabaseFactoryDef$$anon$4.withSession(JdbcBackend.scala:61)
    at au.com.myanaesthetic.slick.test.Db.testDatetimeFunctions(Db.scala:34)
    at au.com.myanaesthetic.slick.test.Main$.run(Main.scala:10)
    at au.com.myanaesthetic.slick.test.Main$.main(Main.scala:6)
    at au.com.myanaesthetic.slick.test.Main.main(Main.scala)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

I rewrote the type conversion using the hibernate joda time project as a base (https://github.com/JodaOrg/joda-time-hibernate/tree/master/src/main/java/org/joda/time/contrib/hibernate) and it works ok. There are some unit test failures that I don't understand (specifically an '-' operator test).

Here are the test results I get

[date] '+' sql = select x2."date" + x2."time" from "Datetime2Test" x2 where x2."id" = ?
[date] '+' sql = select x2."time" + x2."date" from "Datetime2Test" x2 where x2."id" = ?
[date] '+++' sql = select x2."date" + x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '+++' sql = select x2."time" + x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '+' sql = select x2."date" + x2."time" from "DatetimeJodaTest" x2 where x2."id" = ?
[date] '+++' sql = select x2."datetime" + x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '++' sql = select x2."date" + ? from "Datetime2Test" x2 where x2."id" = ?
[date] '--' sql = select x2."date" - ? from "Datetime2Test" x2 where x2."id" = ?
[date] '--' sql = select x2."datetime" - x2."time" from "Datetime2Test" x2 where x2."id" = ?
[date] '-' sql = select x2."datetime" - x2."date" from "Datetime2Test" x2 where x2."id" = ?
[error] Test com.github.tminglei.slickpg.PgDateSupportJodaTest.testDatetimeFunctions failed: expected:<2010-11-03T12:33:01.000> but was:<2010-11-03T22:33:01.000>
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]     com.github.tminglei.slickpg.PgDateSupportJodaTest
[date] '-' sql = select cast(x2."date" as timestamp) - x2."datetime" from "Datetime2Test" x2 where x2."id" = ?
[date] '-' sql = select x2."date" - {d '2009-08-05'} from "Datetime2Test" x2 where x2."id" = ?
[date] '-' sql = select x2."time" - ? from "Datetime2Test" x2 where x2."id" = ?
[date] '---' sql = select x2."datetime" - x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '---' sql = select x2."time" - x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '---' sql = select x2."date" - x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] 'age' sql = select age(x2."datetime") from "Datetime2Test" x2 where x2."id" = ?
[date] 'age' sql1 = select age(cast({fn curdate()} as timestamp),x2."datetime") from "Datetime2Test" x2 where x2."id" = ?
[date] 'part' sql = select date_part('year',x2."datetime") from "Datetime2Test" x2 where x2."id" = ?
[date] 'part' sql = select date_part('year',x2."duration") from "Datetime2Test" x2 where x2."id" = ?
[date] 'trunc' sql = select date_trunc('day',x2."datetime") from "Datetime2Test" x2 where x2."id" = ?
[date] '+' sql = select x2."duration" + ? from "Datetime2Test" x2 where x2."id" = ?
[date] 'unary_-' sql = select - x2."duration" from "Datetime2Test" x2 where x2."id" = ?
[date] '-' sql = select x2."duration" - ? from "Datetime2Test" x2 where x2."id" = ?
[date] '*' sql = select x2."duration" * 3.5 from "Datetime2Test" x2 where x2."id" = ?
[date] '*' sql = select x2."duration" / 5.0 from "Datetime2Test" x2 where x2."id" = ?
[date] 'justifyDays' sql = select justify_days(x2."duration") from "Datetime2Test" x2 where x2."id" = ?
[date] 'justifyHours' sql = select justify_hours(x2."duration") from "Datetime2Test" x2 where x2."id" = ?
[date] 'justifyInterval' sql = select justify_interval(x2."duration") from "Datetime2Test" x2 where x2."id" = ?
[date] 'age' sql = select age(x2."dateTimetz") from "Datetime2Test" x2 where x2."id" = ?
[date] 'age' sql1 = select age(cast({fn curdate()} as timestamptz),x2."dateTimetz") from "Datetime2Test" x2 where x2."id" = ?
[date] 'part' sql = select date_part('year',x2."dateTimetz") from "Datetime2Test" x2 where x2."id" = ?
[date] 'trunc' sql = select date_trunc('day',x2."dateTimetz") from "Datetime2Test" x2 where x2."id" = ?
[error] Test com.github.tminglei.slickpg.PgDate2SupportTest.testDatetimeFunctions failed: expected:<2001-01-03T00:00+08:00> but was:<2001-01-03T00:00+11:00>
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error]     com.github.tminglei.slickpg.PgDate2SupportTest

This is the final class I came up with

package com.github.tminglei.slickpg

import scala.slick.driver.PostgresDriver
import org.joda.time._
import org.joda.time.format.DateTimeFormat
import java.sql.{Timestamp, Time, Date}
import scala.slick.lifted.Column
import org.postgresql.util.PGInterval

trait PgDateSupportJoda extends date.PgDateExtensions with date.PgDateJavaTypes with utils.PgCommonJdbcTypes {
  driver: PostgresDriver =>

  type DATE = LocalDate
  type TIME = LocalTime
  type TIMESTAMP = LocalDateTime
  type INTERVAL = Period

  type TIMESTAMP_TZ = DateTime

  trait DateTimeImplicits {
    val tzDateTimeFormatter = DateTimeFormat.forPattern("yyyy-MM-dd HH:mm:ssZ")

    implicit val jodaDateTypeMapper = new DateJdbcType(sqlDate2jodaDate, jodaDate2sqlDate)
    implicit val jodaTimeTypeMapper = new TimeJdbcType(sqlTime2jodaTime, jodaTime2sqlTime)
    implicit val jodaDateTimeTypeMapper = new TimestampJdbcType(sqlTimestamp2jodaDateTime, jodaDateTime2sqlTimestamp)
    implicit val jodaPeriodTypeMapper = new GenericJdbcType[Period]("interval", pgIntervalStr2jodaPeriod, hasLiteralForm = false)
    implicit val timestampTZTypeMapper = new GenericJdbcType[DateTime]("timestamptz",
      DateTime.parse(_, tzDateTimeFormatter), _.toString(tzDateTimeFormatter), hasLiteralForm = false)

    ///
    implicit def dateColumnExtensionMethods(c: Column[LocalDate]) = new DateColumnExtensionMethods(c)

    implicit def dateOptColumnExtensionMethods(c: Column[Option[LocalDate]]) = new DateColumnExtensionMethods(c)

    implicit def timeColumnExtensionMethods(c: Column[LocalTime]) = new TimeColumnExtensionMethods(c)

    implicit def timeOptColumnExtensionMethods(c: Column[Option[LocalTime]]) = new TimeColumnExtensionMethods(c)

    implicit def timestampColumnExtensionMethods(c: Column[LocalDateTime]) = new TimestampColumnExtensionMethods(c)

    implicit def timestampOptColumnExtensionMethods(c: Column[Option[LocalDateTime]]) = new TimestampColumnExtensionMethods(c)

    implicit def intervalColumnExtensionMethods(c: Column[Period]) = new IntervalColumnExtensionMethods(c)

    implicit def intervalOptColumnExtensionMethods(c: Column[Option[Period]]) = new IntervalColumnExtensionMethods(c)

    implicit def timestampTZColumnExtensionMethods(c: Column[DateTime]) = new TimestampTZColumnExtensionMethods(c)

    implicit def timestampTZOptColumnExtensionMethods(c: Column[Option[DateTime]]) = new TimestampTZColumnExtensionMethods(c)
  }

  //--------------------------------------------------------------------

  /// sql.Date <-> joda LocalDate
  private def sqlDate2jodaDate(date: Date): LocalDate = {
    new LocalDate(date)
  }

  private def jodaDate2sqlDate(date: LocalDate): Date = {
    new Date(date.toDateTimeAtStartOfDay().toDate().getTime())
  }

  /// sql.Time <-> joda LocalTime
  private def sqlTime2jodaTime(time: Time): LocalTime = {
    new LocalTime(time, DateTimeZone.UTC)
  }

  private def jodaTime2sqlTime(time: LocalTime): Time = {
    new Time(time.getMillisOfDay)
  }

  /// sql.Timestamp <-> joda LocalDateTime
  private def sqlTimestamp2jodaDateTime(ts: Timestamp): LocalDateTime = {
    new LocalDateTime(ts)
  }

  private def jodaDateTime2sqlTimestamp(dt: LocalDateTime): Timestamp = {
    new Timestamp(dt.toDateTime.toDate.getTime)
  }

  /// pg interval string <-> joda Duration
  private def pgIntervalStr2jodaPeriod(intervalStr: String): Period = {
    val pgInterval = new PGInterval(intervalStr)
    val seconds = Math.floor(pgInterval.getSeconds).asInstanceOf[Int]
    val millis = ((pgInterval.getSeconds - seconds) * 1000).asInstanceOf[Int]

    new Period(
      pgInterval.getYears,
      pgInterval.getMonths,
      0, // weeks
      pgInterval.getDays,
      pgInterval.getHours,
      pgInterval.getMinutes,
      seconds, millis
    )
  }
}

I hope this helps.

@tminglei
Copy link
Owner

tminglei commented Mar 2, 2014

Well, I see. I've ever submitted a fix change for this problem to slick1 edition of slick-pg, but forgot to do it for slick2 edition.

But your way is better, can you send me a pull request?

Thanks for your nice help!

@tminglei
Copy link
Owner

tminglei commented Mar 5, 2014

Hi @Freaky-namuH I just published a new version of slick-pg, which included the new patch.

Pls update slick-pg dependencies as below to use it:

libraryDependencies += "com.github.tminglei" % "slick-pg_2.10" % "0.5.1.2"
libraryDependencies += "com.github.tminglei" % "slick-pg_joda-time_2.10" % "0.5.1.2"

@tminglei tminglei added the bug label Mar 24, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants