Support legacy behavior of parameterless count

[razajafri]

This PR returns a Long col with a single row with a value of 0

closes #1737

Signed-off-by: Raza Jafri [email protected]

[razajafri]

@abellina can you take a look since you originally worked on aggregates?

[razajafri]

build

[razajafri]

I have added tests. I assumed the count was tested but it wasn't the parameterless count obviously, should've known better.

@abellina have I answered your questions?
@revans2 PTAL

[razajafri]

build

[razajafri]

@abellina are you OK with this PR?

[abellina]

@razajafri just thought of an edge case. Two count() "aggs" back to back. Does your code work with this?

scala> spark.sql("select count(),count() from foo").explain(true)
== Parsed Logical Plan ==
'Project [unresolvedalias('count(), None), unresolvedalias('count(), None)]
+- 'UnresolvedRelation [foo], [], false

== Analyzed Logical Plan ==
count(): bigint, count(): bigint
Aggregate [count() AS count()#255L, count() AS count()#256L]
+- SubqueryAlias foo
   +- SerializeFromObject [input[0, int, false] AS value#2]
      +- ExternalRDD [obj#1]

== Optimized Logical Plan ==
Aggregate [0 AS count()#255L, 0 AS count()#256L]
+- SerializeFromObject
   +- ExternalRDD [obj#1]

== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=false
+- HashAggregate(keys=[], functions=[], output=[count()#255L, count()#256L])
   +- Exchange SinglePartition, ENSURE_REQUIREMENTS, [id=#983]
      +- HashAggregate(keys=[], functions=[], output=[])
         +- SerializeFromObject
            +- Scan[obj#1]

scala> spark.sql("select count(),count() from foo").collect
res32: Array[org.apache.spark.sql.Row] = Array([0,0])

[abellina]

build

[razajafri]

@abellina PTAL

[abellina]

@razajafri thanks for adding tests. I am still not clear on how this works.

My theory is that it works because there is a projection at the end that just sets 0s (i.e. the scalar you are generating is not used). A quick test would be to use something other than 0 for your scalar, to see if that makes a difference.

If the above is true, I don't think it changes the impl. It would just be good to fully understand how this is propagating to the result.

[razajafri]

My theory is that it works because there is a projection at the end that just sets 0s (i.e. the scalar you are generating is not used). A quick test would be to use something other than 0 for your scalar, to see if that makes a difference.

You are right that's how its working. I did a quick test by using a scalar value of 2 and that had no effect on the result

[abellina]

@razajafri thanks for the changes and for the testing. I think this makes sense and I can't think of a simpler way at this point.

[razajafri]

build