Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array of custom types deserialization problem #44

Closed
anakryiko opened this issue Mar 27, 2014 · 9 comments
Closed

Array of custom types deserialization problem #44

anakryiko opened this issue Mar 27, 2014 · 9 comments
Labels

Comments

@anakryiko
Copy link

Hi,

I have a column which type is array of my custom types:

bom bom_entry[]

where bom_entry is

                Composite type "public.bom_entry"
  Column  |       Type       | Modifiers | Storage  | Description
----------+------------------+-----------+----------+-------------
 ipn      | text             |           | extended |
 quantity | double precision |           | plain    |
 seq_num  | integer          |           | plain    |

If the ipn contains a text that has spaces inside, I get the following error (see at the bottom).
If I change the value to ABC_ABC everything works. It seems like slick-pg has problem with deserializaing this representation of array (as presented in psql). Is the problem in escaping?

                           bom
---------------------------------------------------------
 {"(\"ABC ABC\",1,0)","(\"DEF DEF\",1,0)"}
(1 row)

I think the problem is in escaping quotes and quotes are added by PostgreSQL only if there are spaces. With ABC_ABC value, the output is as following:

                       bom
-------------------------------------------------
 {"(ABC_ABC,1,0)","(DEF_DEF,1,0)"}
(1 row)

I created custom type mapper as per your samples.

Can this be fixed somehow? Thanks!

Exception in thread "main" java.lang.IllegalArgumentException: unsupported token CTString(List(SingleQuote, Chunk(ABC ABC), SingleQuote))
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$PGTokenReducer$.com$github$tminglei$slickpg$utils$PGObjectTokenizer$PGTokenReducer$$mergeString$1(PGObjectTokenizer.scala:58)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$PGTokenReducer$$anonfun$1.applyOrElse(PGObjectTokenizer.scala:87)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$PGTokenReducer$$anonfun$1.applyOrElse(PGObjectTokenizer.scala:84)
    at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
    at scala.collection.TraversableLike$$anonfun$collect$1.apply(TraversableLike.scala:278)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.TraversableLike$class.collect(TraversableLike.scala:278)
    at scala.collection.AbstractTraversable.collect(Traversable.scala:105)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$PGTokenReducer$.com$github$tminglei$slickpg$utils$PGObjectTokenizer$PGTokenReducer$$mergeComposite$1(PGObjectTokenizer.scala:84)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$PGTokenReducer$.compose(PGObjectTokenizer.scala:100)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$$anonfun$tokenParser$2.apply(PGObjectTokenizer.scala:333)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$$anonfun$tokenParser$2.apply(PGObjectTokenizer.scala:333)
    at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:136)
    at scala.util.parsing.combinator.Parsers$Success.map(Parsers.scala:135)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$map$1.apply(Parsers.scala:242)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:239)
    at scala.util.parsing.combinator.Parsers$Parser$$anonfun$flatMap$1.apply(Parsers.scala:239)
    at scala.util.parsing.combinator.Parsers$$anon$3.apply(Parsers.scala:222)
    at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
    at scala.util.parsing.combinator.Parsers$$anon$2$$anonfun$apply$14.apply(Parsers.scala:891)
    at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
    at scala.util.parsing.combinator.Parsers$$anon$2.apply(Parsers.scala:890)
    at scala.util.parsing.combinator.RegexParsers$class.parse(RegexParsers.scala:144)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer.parse(PGObjectTokenizer.scala:9)
    at scala.util.parsing.combinator.RegexParsers$class.parseAll(RegexParsers.scala:156)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer.parseAll(PGObjectTokenizer.scala:9)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer.tokenize(PGObjectTokenizer.scala:337)
    at com.github.tminglei.slickpg.utils.PGObjectTokenizer$.tokenize(PGObjectTokenizer.scala:356)
    at com.github.tminglei.slickpg.utils.TypeConverters$Util$$anon$4.apply(TypeConverters.scala:123)
    at com.github.tminglei.slickpg.utils.TypeConverters$Util$$anon$4.apply(TypeConverters.scala:117)
    at scala.Option.map(Option.scala:145)
    at com.github.tminglei.slickpg.array.PgArrayJavaTypes$ArrayListJavaType.nextValue(PgArrayJavaTypes.scala:29)
    at com.github.tminglei.slickpg.array.PgArrayJavaTypes$ArrayListJavaType.nextValue(PgArrayJavaTypes.scala:12)
    at scala.slick.driver.JdbcTypesComponent$JdbcType$class.nextValueOrElse(JdbcTypesComponent.scala:30)
    at com.github.tminglei.slickpg.array.PgArrayJavaTypes$ArrayListJavaType.nextValueOrElse(PgArrayJavaTypes.scala:12)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:23)
    at scala.slick.jdbc.JdbcMappingCompilerComponent$MappingCompiler$$anon$1.read(JdbcMappingCompilerComponent.scala:20)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter$$anonfun$read$1.apply(RelationalProfile.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
    at scala.collection.AbstractTraversable.map(Traversable.scala:105)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:244)
    at scala.slick.profile.RelationalMappingCompilerComponent$ProductResultConverter.read(RelationalProfile.scala:243)
    at scala.slick.profile.RelationalMappingCompilerComponent$TypeMappingResultConverter.read(RelationalProfile.scala:262)
    at scala.slick.driver.JdbcInvokerComponent$QueryInvoker.extractValue(JdbcInvokerComponent.scala:44)
    at scala.slick.jdbc.StatementInvoker$$anon$1.extractValue(StatementInvoker.scala:36)
    at scala.slick.jdbc.PositionedResultIterator.foreach(PositionedResult.scala:211)
    at scala.slick.jdbc.Invoker$class.foreach(Invoker.scala:98)
    at scala.slick.jdbc.StatementInvoker.foreach(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.build(Invoker.scala:69)
    at scala.slick.jdbc.StatementInvoker.build(StatementInvoker.scala:9)
    at scala.slick.jdbc.Invoker$class.list(Invoker.scala:59)
    at scala.slick.jdbc.StatementInvoker.list(StatementInvoker.scala:9)
    at scala.slick.jdbc.UnitInvoker$class.list(Invoker.scala:157)
    at scala.slick.driver.JdbcInvokerComponent$UnitQueryInvoker.list(JdbcInvokerComponent.scala:50)
@tminglei
Copy link
Owner

Hi @anakryiko I can't understand your problem, pls check my pg composite support tests here, which is similar to your case, but looks good.

@anakryiko
Copy link
Author

Hmm.. that's strange, I use exactly the same custom type mappers as in your tests.
I will experiment with the order of fields in custom types, I can't think anything else.

But either way, what can cause this error in your code?
java.lang.IllegalArgumentException: unsupported token CTString(List(SingleQuote, Chunk(ABC ABC), SingleQuote))

Also, changing spaces to underscores fixes the problem, so it is definitely the problem with spaces.

@anakryiko
Copy link
Author

Ok, I just confirmed why your tests are passing and my data causes errors.

I created another custom type, where first field is not string. Then it doesn't fail. Can you try to change the order of your fields so that the first field is String (with spaces)?

So in my case, this fails:

{"("ABC ABC",1,0)"}

and this doesn't:

{"(1, "ABC ABC")"}

Can this be fixed fast or should I alter all my custom data types?

@tminglei
Copy link
Owner

Well, I'll check and fix it ASAP. Thanks for your pointing it out!

@tminglei
Copy link
Owner

Hi @anakryiko I added one more test for composite support, which is very similar to your case.
And it still looks ok. pls check it here.

@anakryiko
Copy link
Author

Ok, so I checked your test on my Db -- it worked, so I started to dig deeper.
As it turns out, the test doesn't fail because of structure of test.
If you have composite3(a text, b integer) -- everything works
But if you have composite3(a text, b integer, c integer) -- it breaks.

Can you check with the following changes to your tests:

  case class Composite3(
                         a: String,
                         b: Int,
                         c: Int
                         )

val rec13 = TestBean1(123, List(Composite3("ABC ABC", 0, 0)))

The only difference (though it shouldn't matter) is that I created types and tables and put the data into table manually, not through your code.

So, it turns out to be some very specific combination of fields that triggers this error. I hope that's helpful in finding bug.

@tminglei
Copy link
Owner

Hi @anakryiko I published slick-pg v0.5.2.2, which included changes to fix the bug.

Pls update your slick-pg dependency as below to use it:

libraryDependencies += "com.github.tminglei" % "slick-pg_2.10" % "0.5.2.2"

Thanks for your nice help :)

@anakryiko
Copy link
Author

Hi,

I've updated to 05.2.2 and verified that the bug was fixed!
Thank you for your fast responses!

@tminglei
Copy link
Owner

You're welcome :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants