You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
People trying to commit to this project and create recipes are very likely to look up expected behavior of pipeline transformations (PTransforms) and they are very likely to get some nasty surprises as things are currently set up. It is extremely frustrating that Beam's pipeline definition DSL is so heavily dependent upon mutation of hidden state, but that appears to be a fundamental limitation of the pipeline API and a likely source for future bugs.
LET IT BE KNOWN: if you're ever writing a DSL for DAG construction, there's no good reason to avoid immutability. This doesn't need to be high performance! (I'm looking at you, beam devs...)
Anyway, we should probably torch anything that's going to introduce bugs and violate expectations of the underlying Beam libraries. The strategy I've proposed can be found here #168 and a compliant recipe can be found here:
The text was updated successfully, but these errors were encountered:
moradology
changed the title
Deprecate PTransform recipes (>=0.10pfr style)
Deprecate PTransform recipes (<=0.10pfr style)
Feb 12, 2024
People trying to commit to this project and create recipes are very likely to look up expected behavior of pipeline transformations (
PTransform
s) and they are very likely to get some nasty surprises as things are currently set up. It is extremely frustrating that Beam's pipeline definition DSL is so heavily dependent upon mutation of hidden state, but that appears to be a fundamental limitation of the pipeline API and a likely source for future bugs.LET IT BE KNOWN: if you're ever writing a DSL for DAG construction, there's no good reason to avoid immutability. This doesn't need to be high performance! (I'm looking at you, beam devs...)
Anyway, we should probably torch anything that's going to introduce bugs and violate expectations of the underlying Beam libraries. The strategy I've proposed can be found here #168 and a compliant recipe can be found here:
The text was updated successfully, but these errors were encountered: