Use `⨟` instead of `|>` (was: Use `∘` instead of `|>`?) #67

tkf · 2019-10-28T14:34:32Z

Ever since I learnt that transducers are adjoint of iterator transforms #13, I've been wanting to rollback the decision to use |> instead of ∘. We'd then have this nice property (e.g., like we have with matrices (ABC)' = C'B'A'):

(Map(f) ∘ Filter(p) ∘ Cat())'
    = Cat()' ∘ Filter(p)' ∘ Map(f)'
    = flatten ∘ filter(p, _) ∘ Generator(f, _)

Other reasons to avoid |> are

We might have an ASCII alias for ∘: Add ASCII alias compose of ∘ JuliaLang/julia#33573 (one of the motivation behind |> was to provide ASCII-based API)
There was (is?) a plan to deprecate overloading of |>: Define what is and is not covered by SemVer JuliaLang/julia#33684 (comment) / Deprecate |> JuliaLang/julia#24886 (comment)

This does not mean that the original reason for using |> is solved; i.e., the usual order of evaluation due to ∘ (left-to-right) is different from how elements in a collection are processed (right-to-left). But hiding this from users may just introduce more confusion.

To solve this "flipped ∘" problem, a better solution may be to use iterator transforms as surface API by

creating a "shim" iterator transform library which just wraps transducers
resurrecting Make higher-order functions composable JuliaLang/julia#33027 (maybe very hard)
writing Transducers.jl "backend" for Query.jl (?)

The text was updated successfully, but these errors were encountered:

jw3126 · 2020-01-09T14:51:31Z

I think there is a way to resolve this problem, by chaning slightly how we think about the category of transducers and iterator transforms.

There is a single category of Transforms. It has a right action on ReductionFunctoins and a left action on Iterators
To emphasise the right action on can call it Transducers, to emphasise the left action one can call it IteratorTransforms. But really there is only a single such category, with two actions.
This way of thinking leads to very nice notation. We can write
fold(rf, Filter(pred) ∘ Map(f) ∘ Take(10), itr)
and it is mathematically unambiguous.

To move closer to this picture in julia, we should:

Deprecate transducer(rf) and instead introduce act(rf, transducer). With a nice unicode infix symbol if we find one.
Deprecate |>
Switch the order of transducer1 ∘ transducer2

jw3126 · 2020-01-09T14:58:32Z

↫ and ↬ are unicode symbols, that people sometimes use for right/left actions.

tkf · 2020-01-09T20:37:31Z

Thanks a lot for the comments! Very interesting!

We can write
fold(rf, Filter(pred) ∘ Map(f) ∘ Take(10), itr)
and it is mathematically unambiguous.

So you mean

fold(rf, Filter(pred) ∘ Map(f) ∘ Take(10), itr)
== fold(((rf ↫ Filter(pred)) ↫ Map(f)) ↫ Take(10), itr)
== fold(rf, Filter(pred) ↬ Map(f) ↬ Take(10) ↬ itr)
== fold(rf, Filter(pred)(Map(f)(Take(10)(itr))))
== fold(rf, itr |> Take(10) |> Map(f) |> Filter(pred))  # `|>` in the usual sense

? This is beautiful!

In addition to this, I think it'd still be nice to have the "opposite" composition operator where the elements flow left to right. Do you know any symbols used for this purpose? Category (mathematics) - Wikipedia mentions "diagrammatic order" composition with ; (so g ; f = f ∘ g) but obviously we can't use ; as an operator in Julia. Maybe use : as it looks close enough?

Take(10) : Map(f) : Filter(pred)
=== Filter(pred) ∘ Map(f) ∘ Take(10)

or maybe |.

It'd be nice if we have other pipe-like ASCII operators in Julia. I wonder if it is feasible to add \> and <\ to the parser. We then can have

fold(rf, itr |> Take(10) |> Map(f) |> Filter(pred))
= fold(rf, itr |> (Take(10) \> Map(f) \> Filter(pred)))
= fold(rf, Take(10) \> Map(f) \> Filter(pred), itr)
= fold(rf ↫ Take(10) \> Map(f) \> Filter(pred), itr)
= fold(rf, (Filter(pred) <\ Map(f) <\ Take(10)) <| itr)
= fold(rf, (Filter(pred) <\ Map(f) <\ Take(10)) ↬ itr)

jw3126 · 2020-01-09T21:25:09Z

So you mean

fold(rf, Filter(pred) ∘ Map(f) ∘ Take(10), itr)
== fold(((rf ↫ Filter(pred)) ↫ Map(f)) ↫ Take(10), itr)
== fold(rf, Filter(pred) ↬ Map(f) ↬ Take(10) ↬ itr)
== fold(rf, Filter(pred)(Map(f)(Take(10)(itr))))
== fold(rf, itr |> Take(10) |> Map(f) |> Filter(pred))  # `|>` in the usual sense

Jup, this is what I meant.

In addition to this, I think it'd still be nice to have the "opposite" composition operator where the elements flow left to right. Do you know any symbols used for this purpose? Category (mathematics) - Wikipedia mentions "diagrammatic order" composition with ; (so g ; f = f ∘ g) but obviously we can't use ; as an operator in Julia. Maybe use : as it looks close enough?

I also often wished that julia, had an operator for opposite composition. The only math notation I am aware of is ∘ₒₚ . But it looks a bit ugly in unicode. Maybe there are other programming languages with opposite composition operator?

\> and <\ they are a cool pair of operators. But I don't really like them for composition. I think ∘ is so widely used, it should be one of the composition operators, it just needs a partner. (I would not like to have an infix alias for ∘)
: it is already used for ranges, so using it for this totally different purpose is undesirable
| is taken for or and I associate it more with |> then with ∘

tkf · 2020-01-09T22:32:36Z

Maybe there are other programming languages with opposite composition operator?

A quick googling told me that Haskell's Control.Arrow has >>> (= ∘ₒₚ) and <<< (= ∘). I think \> and <\ are similar in that they clarifies the directions.

I think ∘ is so widely used, it should be one of the composition operators

I agree with this part. I wasn't suggesting to drop ∘ (although I thought <\ was a nice ASCII alias of ∘).

To stick with something like ∘, maybe something like ~∘ or -∘ makes sense? Or even '∘ or ∘' although parsing them sounds hard.

tkf · 2020-01-09T22:53:10Z

obviously we can't use ; as an operator in Julia

I was wrong 😆

julia> Base.vcat(xfs::Transducer...) = foldl(Base.:|>, xfs)

julia> [Map(sin); Map(cos); Map(tan)]
Map(sin) |>
    Map(cos) |>
    Map(tan)

(No I'm not serious)

tkf · 2020-01-10T04:36:57Z

if it is feasible to add \> and <\ to the parser

It was super easy tkf/julia@56d2556

jw3126 · 2020-01-10T11:05:07Z

I just realized, that one can do right now in julia:

julia> ∘ₒₚ(g,f) = f∘g
∘ₒₚ (generic function with 1 method)

Maybe use ∘, ∘ₒₚ with <\, \> as ASCII aliases?

tkf · 2020-01-10T22:24:59Z

That sounds good.

I opened a PR to request adding the syntax JuliaLang/julia#34340. I'm not requesting adding the functions yet as I thought it may slow down the reviewing.

tkf · 2020-01-11T04:01:31Z

I'm thinking to create a super simple package (say Compositions.jl) with

const compose = ∘
flip(f) = (args...) -> f(reverse(args)...)

If we have JuliaLang/julia#34340 but ∘ₒₚ etc. are rejected then it can also contain

∘ₒₚ(args...) = ∘(reverse(args)...)
const var"</" = ∘
const var"/>" = ∘ₒₚ

I think it's nice to have ASCII functions compose and flip(compose) even in old julias. What do you think?

Also, is filp better be called op? Though then I guess we'd want op to be a functor. But that's impossible unless we can write op(::Morphism) and op(::Object) or something like that.

jw3126 · 2020-01-12T07:26:57Z

Compositions.jl sounds nice. Yeah, from op I would expect that it is a functor. (Which would require that Base.∘ returns a callable struct and not a lambda. Which would also enable better e.g show(isfinite ∘ maximum).

tkf · 2020-01-12T07:55:28Z

BTW I've been wanting to define show for f ∘ g for a while now too.

jw3126 · 2020-02-10T21:00:56Z

https://discourse.julialang.org/t/add-composition-operator-unicode-2a1f-for-flipped/34436

tkf · 2020-05-31T04:11:31Z

So, I've been wondering what the API of the "right action" of transducers (action on reducing functions) should be. It's a bit unfortunate that the left action is so easy (xf(itr)) while the right action is a bit clumsy (reducingfunction(xf, rf)). I think the cleanest solution might be to use something like

adjoint(xf::Transducer) = rf -> reducingfunction(xf, rf)

This way, both Map(f)(itr) and Map(f)'(rf) work nicely and cleanly.

(adjoint probably should return ReducingFunctionTransform object or something so that xf'' is xf and the composition works in this category too)

tkf · 2020-06-29T06:54:45Z

This is now implemented by #319. The new documentation (preview) is here: https://juliafolds.github.io/Transducers.jl/previews/PR319/

In particular, it's somewhat unfortunate (though expected) now that the notion of "transducer" is diverged from the rest of world: https://juliafolds.github.io/Transducers.jl/previews/PR319/explanation/glossary/#Transducer-1 (glossary). But I think practicality of itr |> xf1 |> xf2 is a win overall.

@jw3126 Want to have a look at it?

jw3126 · 2020-06-30T09:36:34Z

Cool thanks, I will take a look this week.

This patch implements the idea discussed in #67. That is to say, ```julia collect(Map(f) |> Filter(g), xs) foldl(+, Map(f) |> Filter(g), xs) ``` is now written as ```julia xs |> Map(f) |> Filter(g) |> collect foldl(+, xs |> Map(f) |> Filter(g)) ``` or ```julia collect(opcompose(Map(f), Filter(g)), xs) foldl(+, opcompose(Map(f), Filter(g)), xs) collect(Map(f) ⨟ Filter(g), xs) foldl(+, Map(f) ⨟ Filter(g), xs) ``` or even (not recommended) ```julia collect(Filter(g)((Map(f)(xs)))) # Julia >= 1.3 collect(Filter(g) ∘ Map(f), xs) foldl(+, Filter(g) ∘ Map(f), xs) collect((Filter(g)' ⨟ Map(f)')', xs) foldl(+, (Filter(g)' ⨟ Map(f)')', xs) collect((Map(f)' ∘ Filter(g)')', xs) foldl(+, (Map(f)' ∘ Filter(g)')', xs) ``` Above syntax are all compatible with the view that `xf(itr)` is an iterator transformation. OTOH, `xf'` (`adjoint(::Transducer)`) is now the "classic" transducer; i.e., reducing function transformation. This makes it easy to use transducers with other "reducing function combinators" like `TeeRF`: ```julia rf = TeeRF(Filter(iseven)'(min), Filter(isodd)'(max)) reduce(rf, Map(identity), xs) # => (even minimum, odd maximum) ``` This PR only deprecates `::Transducer |> ::Transducer` to help migration from the old syntax.

tkf changed the title ~~Use |> instead of ∘?~~ Use ∘ instead of |>? Oct 28, 2019

tkf added this to the 0.5 milestone Jan 10, 2020

tkf mentioned this issue Jan 11, 2020

RFC: Add new pipe-like operator syntaxes </, |>>, etc. JuliaLang/julia#34340

Open

tkf mentioned this issue Feb 3, 2020

Include in examples non-piping versions #199

Closed

tkf mentioned this issue May 30, 2020

Use JuliaFunctional/CompositionsBase.jl? jw3126/Setfield.jl#139

Open

tkf changed the title ~~Use ∘ instead of |>?~~ Use ⨟ instead of |> (was: Use ∘ instead of |>?) Jun 1, 2020

tkf pinned this issue Jun 1, 2020

This was referenced Jun 23, 2020

Make mapreduce(..., withprogress(...)) etc. work without transducers #72

Merged

Treat transducers as iterator transforms at surface syntax #319

Merged

mergify bot closed this as completed in #319 Jul 4, 2020

tkf unpinned this issue Jul 5, 2020

MasonProtter mentioned this issue Sep 21, 2023

making things cleaner, more transparent, more consistent: road to 1.0 JuliaFolds2/Transducers.jl#22

Open

MasonProtter mentioned this issue Nov 18, 2024

What to do about composition of Transducers? JuliaFolds2/TransducersNext.jl#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `⨟` instead of `|>` (was: Use `∘` instead of `|>`?) #67

Use `⨟` instead of `|>` (was: Use `∘` instead of `|>`?) #67

tkf commented Oct 28, 2019

jw3126 commented Jan 9, 2020

jw3126 commented Jan 9, 2020

tkf commented Jan 9, 2020

jw3126 commented Jan 9, 2020

tkf commented Jan 9, 2020

tkf commented Jan 9, 2020

tkf commented Jan 10, 2020

jw3126 commented Jan 10, 2020

tkf commented Jan 10, 2020

tkf commented Jan 11, 2020

jw3126 commented Jan 12, 2020 •

edited

Loading

tkf commented Jan 12, 2020

jw3126 commented Feb 10, 2020

tkf commented May 31, 2020

tkf commented Jun 29, 2020

jw3126 commented Jun 30, 2020

Use ⨟ instead of |> (was: Use ∘ instead of |>?) #67

Use ⨟ instead of |> (was: Use ∘ instead of |>?) #67

Comments

tkf commented Oct 28, 2019

jw3126 commented Jan 9, 2020

jw3126 commented Jan 9, 2020

tkf commented Jan 9, 2020

jw3126 commented Jan 9, 2020

tkf commented Jan 9, 2020

tkf commented Jan 9, 2020

tkf commented Jan 10, 2020

jw3126 commented Jan 10, 2020

tkf commented Jan 10, 2020

tkf commented Jan 11, 2020

jw3126 commented Jan 12, 2020 • edited Loading

tkf commented Jan 12, 2020

jw3126 commented Feb 10, 2020

tkf commented May 31, 2020

tkf commented Jun 29, 2020

jw3126 commented Jun 30, 2020

Use `⨟` instead of `|>` (was: Use `∘` instead of `|>`?) #67

Use `⨟` instead of `|>` (was: Use `∘` instead of `|>`?) #67

jw3126 commented Jan 12, 2020 •

edited

Loading