Fix the simpleString used in error messages #18667
Conversation
|
Can one of the admins verify this patch? |
| @@ -43,7 +43,7 @@ class LongType private() extends IntegralType { | |||
| */ | |||
| override def defaultSize: Int = 8 | |||
|
|
|||
| override def simpleString: String = "bigint" | |||
| override def simpleString: String = "long" | |||
There was a problem hiding this comment.
I don't think so. bigint is the SQL type for an 8-byte integer, right?
There was a problem hiding this comment.
When you try to read a csv and map to a case class with a Long your get a message like this one:
EXCEPTION:org.apache.spark.sql.AnalysisException: Cannot up cast linked_docs.MR_NUMBER_OF_DOCS_UPLOADED from string to bigint as it may truncate
The type path of the target object is:
- field (class: "scala.Long", name: "MR_NUMBER_OF_DOCS_UPLOADED")
Getting a message that talks about bigint while you are trying to cast a String to a Long looks confusing to me. I thought this was a typo.
There was a problem hiding this comment.
You're renaming the whole type, when the issue is at best that some intermediate stage of the plan does cast to bigint first. At the least, this is not the fix, and I'm not sure it's really a problem, even if the error is from an implementation detail.
What changes were proposed in this pull request?
Fix the simpleString used in error messages
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Please review http://spark.apache.org/contributing.html before opening a pull request.