Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify mapreduce_impl #16217

Merged
merged 1 commit into from
Jun 3, 2016
Merged

simplify mapreduce_impl #16217

merged 1 commit into from
Jun 3, 2016

Conversation

vtjnash
Copy link
Member

@vtjnash vtjnash commented May 5, 2016

this is largely now possible since Functors are not needed for performance
fixes #16185

@KristofferC
Copy link
Member

I don't think we should close #16185 until the reduced example I posted there is fixed. Alternatively a new issue could be made for that example.

@JeffBezanson
Copy link
Member

The reduced example in that issue might not get fixed. The extra code allows dynamic dispatch to a version of g specialized for f. The method in the example is probably not performance-critical, since all it does is call another function. This is part of the cost of faster higher-order functions.

@StefanKarpinski
Copy link
Member

From triage call:

@StefanKarpinski
Copy link
Member

TODO: there should be a note in NEWS about the changed reduction order.

@JeffBezanson
Copy link
Member

We should definitely keep both pairwise summation and the 0.4 level of performance. If necessary we should manually specialize sum, or use a reduce_impl that has the mapping function removed.

this is possible now that Functors are not needed for performance
fixes #16185
@vtjnash vtjnash force-pushed the jn/mapreduce-fast-again branch from fbaa2c0 to cbd77c2 Compare June 2, 2016 20:06
@vtjnash vtjnash merged commit b0bc3f7 into master Jun 3, 2016
@vtjnash vtjnash deleted the jn/mapreduce-fast-again branch June 3, 2016 03:54
@tkelman
Copy link
Contributor

tkelman commented Jun 4, 2016

Apparently DataArrays was using sum_pairwise_blocksize

@vtjnash
Copy link
Member Author

vtjnash commented Jun 4, 2016

it can define sum_pairwise_blocksize(T) = pairwise_blocksize(T, +) for compatibility

@nalimilan
Copy link
Member

That's what I did in JuliaStats/DataArrays.jl#192. If that was the only use of it outside of Base, I guess that's OK.

@tkelman
Copy link
Contributor

tkelman commented Jun 5, 2016

NullableArrays was also using it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

performance regression in sum(a)
6 participants