Testing

[paldepind]

Testing FRP code can sometimes be tricky and cumbersome.

To solve the problem we are working on a way to FRP code in a way that is declarative and convenient.

Here is a simple example. Let's say one want's to test this function:

function foobar(stream1, stream2) {
  const a = stream1.filter(isEven).map((n) => n * n);
  const b = stream1.filter((n) => !isEven(n)).map(Math.sqrt);
  return combine(a, b);
}

Currently, such a function can be tested like this:

const a = testStreamFromObject({ 0: 1, 2: 4, 4: 6 });
const b = testStreamFromObject({ 1: 9, 3: 8 });
const result = foobar(a, b);
const expected = testStreamFromObject({ 1: 3, 2: 16, 4: 36 });
assert.deepEqual(result.semantic(), expected.semantic());

The testing feature is currently quite incomplete. And we completely lack a way to test code that uses Now. I think testing Now could be achieved by combining elements from how testing IO works and how testing behavior/streams work.

This issue is for discussion ideas and problems related to testing.

[paldepind]

I'm interested in what you think about this @jayrbolton. Do you think this is useful for testing FRP code? How do you normally test such code?

[jayrbolton]

I've always tested flyd code with manual pushes and reads. The way I've thought about before to do it less imperatively is very similar to your example there, where you have a special test stream constructor where you can manually order each value. I wonder if this could also be good for documentation examples. In documentation, it's nice to show how the values change over time with real values without resorting to imperative push/read. I also wonder: if you treat the keys as milliseconds, could you test time-based stream logic declaratively? I think I need more experience with Hareactive and more intuition about Now to be of any help though.

One thought: I've always wanted to try to implement QuickCheck from Haskell in javascript. I think something like that could be applied here. For example, the resulting stream has these properties that could be automatically tested:

• If a number in the result stream is odd, its square must be odd
• If a number in the result stream is even, it must have an integer square root that is even

[paldepind]

I've always tested flyd code with manual pushes and reads. The way I've thought about before to do it less imperatively is very similar to your example there, where you have a special test stream constructor where you can manually order each value.

Then we've had the same thought process. The goal here is exactly to make tests more compact and pure.

I wonder if this could also be good for documentation examples. In documentation, it's nice to show how the values change over time with real values without resorting to imperative push/read.

Yes! I think that is a very good idea. It will make examples much more compact and thus more easy to read. Maybe it can also be used to power diagrams like (RxMarbles)[http://rxmarbles.com/).

I also wonder: if you treat the keys as milliseconds, could you test time-based stream logic declaratively?

Yes. We support that in delay already.

One thought: I've always wanted to try to implement QuickCheck from Haskell in javascript. I think something like that could be applied here. For example, the resulting stream has these properties that could be automatically tested:

• If a number in the result stream is odd, its square must be odd
• If a number in the result stream is even, it must have an integer square root that is even

That would be very cool! And probably also very useful. Besides supporting more thorough testing, one would also avoid creating the test streams and have the quickcheck library do it.

I'm aware that jsverify exists but I've never used it.

[dmitriz]

Does .semantic() return some kind of plain object representation of the stream?

[paldepind]

@dmitriz

Yes, indeed. On a behavior it returns (time: Time) => A and on a stream it returns SemanticStream defined as follows:

export type Occurrence<A> = {
  time: Time,
  value: A
};

export type SemanticStream<A> = Occurrence<A>[]

For testing the basic idea is this: put semantic streams and behaviors into your system/function and you will get semantic streams and behaviors out.

You declare your semantic streams/behaviors by giving their full history and the semantic streams/behaviors you get out in the end also contain a full history. You can then test that the results match the expected.

This makes is possible to test FRP code in a way that is completely declarative, synchronous and free from side effects. I'm quite excited about the feature as reactive libraries can often be hard to test.

I do however think that we still need some more helper functions in order to make the testing is easy as possible.

[dmitriz]

@paldepind
Thank you for the explanation,
yes, it is very nice indeed, comparing with the imperative pushing and waiting.
Even if the latter can feel more intuitive, being closer to how you think about streams.
But once the idea is explained, it feels easy.
And while not "realistic", feels just right for testing purposes ;)

It may be just me, but I felt quite confused by the name semantic,
looking at it first, I had no idea what it was.
For me, the "meaning" of the stream is the sequence of events occurring in time,
not the model representing their entire history with timestamps at once.

I find the analogy with Physics as mentioned in
#29 (comment)
the easiest way to understand it.
But would you call the function describing the physical particle motion "semantic"?
Perhaps something like "history" would be an intuitive name
as it captures the idea of the unfolding the stream into the list.
"Unfolding" can also be a cool name here, at the peril of introducing more jargon ;)

The type Occurrence is exactly the same as the Future?

I do however think that we still need some more helper functions in order to make the testing is easy as possible.

I probably would call the testStreamFromObject something like fromHistoryObject.