Feature Request: De/Attachable Subgraphs

[SimonDanisch]

Hi,
I've been speaking about this with Shashi, but I think so far I haven't had good examples, so the conclusion was mostly, that I'm using react wrong.
I did some more thinking, and I came to the conclusion, that we really need some mechanisms, to connect different react graphs, as you can't always construct the whole graph.
Example:
In an API I want to offer, to make a shader out of some string signal. But at shader creation, I don't want to force the user to decide, from which string signal he wants to create the shader.

function makeshader(defaultsource::String)
   sourcecode = ascii(defaultsource)
   dummyInput = Input(sourcecode)
   lift(dummyInput) do sourcecode
      ... Make new shader
      ... Delete old shader
      return shader
   end
   return dummyInput # return "socket" to which one can connect
end
# At initializsation, by some API, somewhere, where the user has no access
shader = makeshader(SomeShaderSource)

# At a different time point, user wants to interact with the shader, namely updating it
newsource = lift(x -> readall(open("filename")), every(1.0)) # this is just exemplary and a really wasteful way of doing this

# Now, this doesn't work in the current API:
lift(x-> push!(shader, x), newsource)
# So something like this is needed!?
connect(shader.dummyInput, newsource)

Another example:
I'm working on widgets for my plots.
The data for a plot in GLPlot looks a little like this:

[
:attributename => Input(someValue)
:attributname => SomeLift
....
]

Now, at the part where I initialize this datastructure, I don't even remotely know, if the person wants to change the value via a my glwidgets, via the commandline with push!(), or if someone builds, for example, some tk wrapper.

Best,
Simon

[ryaninvents]

The following feels like it should work, but I get ERROR: push! called when another signal is still updating. I'm not sure if this is an implementation issue or a conceptual issue; I'll dig deeper tonight when I have more time.

julia> using Reactive

julia> shader = Input(Input("first shader"))
"first shader"

julia> dummyInput = foldl(Input("initial value"), shader) do main, newSignal
           lift((val) -> push!(main, val), newSignal)
       end
"initial value"

julia> lift(dummyInput) do v
           println(string("Value! ", v.value))
       end
Value! initial value
nothing

julia> mySignal = Input("bar")
"bar"

julia> push!(shader, mySignal)
ERROR: push! called when another signal is still updating.
 in push! at /Users/mullery1/.julia/v0.3/Reactive/src/Reactive.jl:257

A similar (but much more complicated) library for JavaScript called Bacon.js has a Bus class which, along with flatMap, would completely get the job done. I think Reactive.jl would benefit greatly from a flatten() function or similar, so you could simply write:

using Reactive
shader = Input(Input("first shader"))
dummyInput = flatten(shader)
lift(shader) do value
  println(string("Got a value: ", value))
end
# now prints "Got a value: first shader"
myNewSignal = Input("second shader")
push!(shader, myNewSignal) # prints "Got a value: second shader"
push!(myNewSignal, "third shader") # prints "Got a value: third shader"

A large part of the power of reactive programming is that you can have signals of signals (or streams of streams, or however you want to think about them). I'd love to see flatten() be added to Reactive.jl.

The only issue that I could see is that whenever you plug in a new signal, it will "hijack" the shader stream. For instance, say you had an initial shader stored in the sourcecode variable in your above example, and then your widget initialized. When the widget plugged its stream into shader its (probably useless) initial value would override the value of sourcecode. You could probably fix this with some kind of dummy initial value that gets dropped, but that feels a bit hacky.

[SimonDanisch]

ERROR: push! called when another signal is still updating
That's supposed to happen, as it's simply not allowed to push inside lifts. You can get around it by sleeping for a while, but it's extremely hacky as it just works in 80% of the time.

Yeah initial values are a bit cumbersome, especially when you work with events, that not yet exist.
I sometimes use an array, which is either empty or holds the value ;) (has quite some overhead though, when I think about it... Better would be the Option type, which I saw flying around somewhere)

push!(signala, signalb) seems to be a lot better than connect! =)

[shashi]

@SimonDanisch I think what you are looking for is to merge two signals? merge(::Signal...,) gives a signal which updates when any of the inputs updates.

I still think the right way to do this is not have Reactive as a dependency at all in GLPlot. Maybe only for setting up input signals from the window.

makeshader should just be the do block that is inside.

function useshader( sourcecode)
      #... Make new shader
      #... Delete old shader
      # also tell OpenGL to use the new shader
   end

And then the user could do:

using Reactive
newsource = lift(useshader, lift(x -> readall(open("filename")), every(1.0)))

if they want to update the shaders at all. If they don't want to, they need not even bother learning about Reactive. As for the plot object:

[
:attributename => Input(someValue)
:attributname => SomeLift
....
]

Should definitely be

[ :attributename => someValue, :attributname => someValue2]

And then all you need to take care of is rendering a signal of plots if at all that's what the user wants to render. They can use the input signals from the window to create a signal of plots to render.

What Reactive implements is loosely first order FRP almost identical to Elm. This means the graph can only grow by adding child nodes but cannot change in connectivity. This recent talk explains the tradeoffs when using signals of signals or reconfigurable signal graphs. I quite like the convenience Reactive provides right now without being too complicated.

@BaconScript Thanks for the examples. flatten seems like a nice way to allow someone to attach or detach signals. But merge usually gets the job done in the case of one-off interactive UIs such as those in Interact and GLPlot. I'll look into what problems we might face if we introduce flatten. There is also the chance that we could implement Automaton instead as Evan describes it in the talk.

[SimonDanisch]

I need to think about the others, but one quick comment:
one-off interactive UIs such as those in Interact and GLPlot
That's probably exactly where I don't agree... I simply want something more flexible than one off UIs.
I want to have an API, that prepares you a subgraph, which is already set-up to solve a certain sub-problem, but the user can decide how and when to integrate it in the bigger picture.
I think, again, about the other suggestion.

[SimonDanisch]

After watching the video I think it is quite clear:
It's not weather we want this feature, but who will implement it and when...
Automatons, if I understand them correctly, might be what I need, to make things work as intended.

Why do I still use the dicts of Signals?!
It's the simplest way for me, right now, and offers me to switch out signals dynamically, already. So this is my work around at the moment. It doesn't hurt much, as I'm not at a point yet, where I really render in a reactive way.
To get my whole render pipeline working in the real sense of Reactive Programming is on my list. But it will mean huge amounts of work for me, as it's not very easy to elegantly work together with OpenGL, while keeping performance and a straight forward API.
It basically means, to build a reactive 3D scene graph, which will be very challenging.

[shashi]

@SimonDanisch I just PR'd an implementation of flatten (#32) which turns out works exactly the way @r24y describes :)

However, I am of the opinion that this must be used sparingly for simplicity.

[shashi]

Now that we have flatten and even bind! and unbind!. I guess we can close this...