how to handle rest with invalid state argument?

[mschauer]

As the rest of an iterator can contain more elements then the original one, e.g.

collect(rest(1:5,0))

maybe from(iter, state) would be a better name.

[nalimilan]

Probably better use a name which is common in other languages. Would you make a short survey?

[yuyichao]

Shouldn't iterating on rest(1:5, 0) raise a bounds error?

[mschauer]

@nalimilan I think this might be julia specific, python and java do not allow this and mandate the use of tee and others only "restart" at the "beginning" but that is not terribly well defined.

@yuyichao This behaviour is only possible in specific cases. In general you cannot test if a certain state is ever reached from an iterator.

[toivoh]

Isn't the interpretation of the iterator state meant to be an entirely private matter to the type being iterated over? This function strikes me as dangerous, as it encourages people to make assumptions about such implementation details, which might actually change at any time. Perhaps there are good reasons that make it needed in Base, but instead of renaming it I suggest we stop exporting it.

[yuyichao]

I can see this being useful when you are calling start, next etc yourself. I don't think supplying a random state should be supported for exactly the reason @toivoh mentioned though.

[JeffBezanson]

Yes, we're assuming the state is valid. It's definitely a general issue, e.g. UTF-8 string indexes come to mind. I don't see anything we can do about it without a performance disaster.

[JeffBezanson]

However, we could have a function to check whether a state is valid, and call it when constructing a Rest iterator.

[toivoh]

But would it be a performance disaster to limit the usage to Base? I doubt that a lot of code outside of Base is using it anyway.

[mschauer]

Datapoint: In @samoconnor called the same functionality in #14805 "resume"

[samoconnor]

Isn't the interpretation of the iterator state meant to be an entirely private matter to the type being iterated over?

Yes. state should be treated as an opaque token.

This function strikes me as dangerous, as it encourages people to make assumptions about such implementation details, which might actually change at any time.

The function isn't dangerous. Passing hand constructed state values is dangerous. But surely there are lots of functions that do unpredictable things when passed random values.
It seems to me that rest(x, state) is no different to done(x, state) and next(x, state).
The only valid state to pass to any of these is a value returned by start or next.

Is there anything in the documentation to suggest that constructing a state object by hand is an ok thing to do?

[mschauer]

Yes, and I like a name derived from "resume" because that conveys the notion better that the argument is a state which was obtained earlier using legal means.

[JeffBezanson]

There doesn't seem to be enough clarity on what (if anything) to do here for this to make 0.6.