Document reduce_agg aggregate function #12195

wenleix · 2019-01-08T19:56:54Z

No description provided.

findepi · 2019-01-08T21:52:28Z

presto-docs/src/main/sphinx/functions/aggregate.rst

+    ``combineFunction`` takes two states and returns the new state.
+    The final state is returned.
+
+    ``redcue_agg`` supports state with native container type long, double or boolean.


does it mean that S must be one of these?

(just curious, why so?)

typo: redcue_agg -> reduce_agg

Could you add an example of usage?

@findepi : Correct, see the comments in code:

presto/presto-main/src/main/java/com/facebook/presto/operator/aggregation/ReduceAggregationFunction.java

Lines 124 to 127 in 79f480b

// State with Slice or Block as native container type is intentionally not supported yet,

// as it may result in excessive JVM memory usage of remembered set.

// See JDK-8017163.

throw new UnsupportedOperationException();

Also see #9553

findepi · 2019-01-08T21:52:40Z

presto-docs/src/main/sphinx/functions/aggregate.rst

+.. function:: reduce_agg(inputValue T, initialState S, inputFunction(S, T, S), combineFunction(S, S, S)) -> S
+
+    Returns a single value reduced from input values. ``inputFunction`` and ``combineFunction`` will be invoked.
+    ``inputFunction`` takes the current state (initially ``initialState``) and the input value, and returns the new state.


does it mean reduce_agg runs on single node?
the wording suggests so; but existence of combineFunction suggests it is parallel

No it runs in parallel. Any thoughts about how to revise the wording? -- Since when I read it, I didn't see it will run on single node, but I am obviously biased...

@findepi, I don't think the name combineFunction implies parallelism.

@dain not the name -- the existence of it.
It it was single-threaded, there would be just initialSate and inputFunction (aka "accumulator" or "accumulatingFunction").
The need for combineFunction arises only when you want to merge two branches of processing (ie parallel computations).

presto-docs/src/main/sphinx/functions/aggregate.rst

wenleix · 2019-01-09T06:07:39Z

Thanks @mbasmanova and @findepi for reviewing. I added examples, here is how it looks like:

dain · 2019-01-09T19:12:17Z

presto-docs/src/main/sphinx/functions/aggregate.rst

+        -- (1, 12)
+        -- (2, 30)
+
+    ``reduce_agg`` supports state with native container type long, double or boolean.


I don't think any end user will understand what this means. Maybe state this as reduce_agg currently does not support variable width types, container types, or inexact types.

@dain : Maybe I will just say reduce_agg currently supports boolean, integer (TINYINT, SMALLINT, INTEGER, BIGINT) and floating-point (REAL, DOUBLE) types .

It also supports dates, times, timestamps, right?

nezihyigitbasi · 2019-01-19T00:05:07Z

@findepi @dain @mbasmanova Is there anything else that's needed from this PR? We need to merge this and update the release notes PR (#12194) with a link to this new doc.

presto-docs/src/main/sphinx/functions/aggregate.rst

wenleix · 2019-01-22T19:16:52Z

Thanks @electrum for reviewing. Comments addressed.

Extracted from prestodb/presto#12195

mbasmanova · 2019-01-22T22:31:46Z

@electrum David, is this ready to merge? Could you approve?

mbasmanova · 2019-01-19T00:10:06Z

presto-docs/src/main/sphinx/functions/aggregate.rst

+    Returns a single value reduced from input values. ``inputFunction`` and ``combineFunction`` will be invoked.
+    ``inputFunction`` takes the current state (initially ``initialState``) and the input value, and returns the new state.
+    ``combineFunction`` takes two states and returns the new state.
+    The value of final state is returned::


typo: there should be only one colon, e.g. "returned: "

@mbasmanova : This is Sphinx syntax, e.g. https://github.com/prestodb/presto/blame/5c6bf5f8893a3ac7080b199667f23b480ff54983/presto-docs/src/main/sphinx/functions/array.rst#L122

wenleix assigned electrum and mbasmanova Jan 8, 2019

facebook-github-bot added the CLA Signed label Jan 8, 2019

wenleix force-pushed the reduce_agg_doc branch from 92e00bd to 1d2a324 Compare January 8, 2019 19:57

findepi reviewed Jan 8, 2019

View reviewed changes

wenleix force-pushed the reduce_agg_doc branch from 1d2a324 to 49d0764 Compare January 9, 2019 06:06

dain reviewed Jan 9, 2019

View reviewed changes

wenleix force-pushed the reduce_agg_doc branch from 49d0764 to d111ce9 Compare January 11, 2019 21:13

nezihyigitbasi mentioned this pull request Jan 19, 2019

Add release notes for 0.216 #12194

Merged

electrum reviewed Jan 19, 2019

View reviewed changes

presto-docs/src/main/sphinx/functions/aggregate.rst Outdated Show resolved Hide resolved

electrum reviewed Jan 19, 2019

View reviewed changes

presto-docs/src/main/sphinx/functions/aggregate.rst Show resolved Hide resolved

electrum reviewed Jan 19, 2019

View reviewed changes

presto-docs/src/main/sphinx/functions/aggregate.rst Outdated Show resolved Hide resolved

Document reduce_agg aggregate function

d4de05b

wenleix force-pushed the reduce_agg_doc branch from d111ce9 to d4de05b Compare January 22, 2019 19:13

electrum added a commit to electrum/trino that referenced this pull request Jan 22, 2019

Document reduce_agg aggregate function

82b7fff

Extracted from prestodb/presto#12195

mbasmanova approved these changes Jan 22, 2019

View reviewed changes

wenleix merged commit 559aaa9 into prestodb:master Jan 22, 2019

wenleix deleted the reduce_agg_doc branch January 22, 2019 22:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document reduce_agg aggregate function #12195

Document reduce_agg aggregate function #12195

wenleix commented Jan 8, 2019

findepi Jan 8, 2019

mbasmanova Jan 8, 2019

wenleix Jan 9, 2019

findepi Jan 8, 2019

wenleix Jan 9, 2019 •

edited

Loading

dain Jan 9, 2019

findepi Jan 9, 2019

wenleix commented Jan 9, 2019

dain Jan 9, 2019

wenleix Jan 9, 2019

findepi Jan 9, 2019

nezihyigitbasi commented Jan 19, 2019 •

edited

Loading

wenleix commented Jan 22, 2019

mbasmanova commented Jan 22, 2019

mbasmanova Jan 19, 2019

wenleix Jan 22, 2019

	// State with Slice or Block as native container type is intentionally not supported yet,
	// as it may result in excessive JVM memory usage of remembered set.
	// See JDK-8017163.
	throw new UnsupportedOperationException();

Document reduce_agg aggregate function #12195

Document reduce_agg aggregate function #12195

Conversation

wenleix commented Jan 8, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenleix Jan 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenleix commented Jan 9, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nezihyigitbasi commented Jan 19, 2019 • edited Loading

wenleix commented Jan 22, 2019

mbasmanova commented Jan 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wenleix Jan 9, 2019 •

edited

Loading

nezihyigitbasi commented Jan 19, 2019 •

edited

Loading