All the tutorials: we need motivating (advanced) examples in our docs #175
Comments
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
|
Wow, some of these, particularly the stream sorting may do well as their own separate repos/libraries! Another possible example is to implement one of the function-as-a-service protocols with some tuning goodies. |
@parity3 sweet idea. I'd welcome a PR for something that too 😉
Also check #182, I think the caps based RPC is going to be here to stay the more I've thought about it. |
|
Another piece of code that may be nice to see is for a tractor controller app to run as a daemon that some other sync or polyglot app could call into, and be used as a glorified task queue implementation. You could make it compatible with celery or dramatiq or whatever. |
I was gonna say, IMO if you pair That might be good further down the line to demonstrate how with the right primitives from the outset these specialized message pattern designs like This was kind of always the intention from the outset: make the |
|
Hmm probably worth linking celery/ceps#27 |
|
@parity3 yeah maybe a "micro-service design" tutorial is in order as well. I think we'll end up picking the simplest / hippest of these examples and toss the rest in a "parallelism tutorial" in the docs. |
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
This is a draft of the `tractor` way to implement the example from the "processs pool" in the stdlib's `concurrent.futures` docs: https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example Our runtime is of course slower to startup but once up we of course get the same performance, this confirms that we need to focus some effort not on warm up and teardown times. The mp forkserver method definitely improves startup delay; rolling our own will likely be a good hot spot to play with. What's really nice is our implementation is done in approx 10th the code ;) Also, do we want offer and interface that yields results as they arrive? Relates to #175
goodboy
changed the title
goodboy
changed the title
Clearly, we need some fancier and more immediately practical examples in our readme (says every person who's glanced ever), so I figured I'd start list of what I think would be coolio.
A "worker pool" example copying
concurrent.futures's prime number calculatorProcessPoolExecutor(ugh) so we'll need to probably mock up the executor api and show the SC way of doing thingsThe
multiprocessing.Poolexamples:A "controlled fork blast" example
A real-time streaming sorting algorithm
trio-ized version of an Rxmergeto serialize results.Target tutorial set:
this is the back of hand list I can think of:
tractor.extrasLurker opinions always welcome.
We might actually need to pick the best of these and then stick the rest in a parallelism tutorial.
Actually, we might want a streaming, shared state, and parallelism tutorial in the new docs.
The text was updated successfully, but these errors were encountered: