You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now short term serialization (for executors) uses __get/setstate__ and handles everything, while long term (file storage) serialization uses a bespoke solution and only handles nodes that live in a .py file.
Since we'd like so support (with a warning maybe) long-term storage of in-notebook nodes, we'll need to bring in some element of the pickle-style storage in there anyhow. Thus, there is probably room to get these two styles of serialization using the same infrastructure -- namely __get/setstate__. This was my initial attack prior to #160, and has since been requested by @jan-janssen, so it's worth taking another crack at this approach.
The text was updated successfully, but these errors were encountered:
Everything except for the while-loop is now fully pickle-compliant (as long as you don't stick anything un-pickleable as data on the node; #293#303 etc), and as long as all the nodes are somewhere importable (i.e. not in main), you can even use the standard concurrent.futures.ProcessPoolExecutor (#304).
So this is actually just waiting on a pickle-compliant long-term storage solution and I will close it.
Right now short term serialization (for executors) uses
__get/setstate__
and handles everything, while long term (file storage) serialization uses a bespoke solution and only handles nodes that live in a .py file.Since we'd like so support (with a warning maybe) long-term storage of in-notebook nodes, we'll need to bring in some element of the pickle-style storage in there anyhow. Thus, there is probably room to get these two styles of serialization using the same infrastructure -- namely
__get/setstate__
. This was my initial attack prior to #160, and has since been requested by @jan-janssen, so it's worth taking another crack at this approach.The text was updated successfully, but these errors were encountered: