Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Asynchronous remote server #20

Open
alans09 opened this issue Jun 16, 2014 · 6 comments
Open

Asynchronous remote server #20

alans09 opened this issue Jun 16, 2014 · 6 comments

Comments

@alans09
Copy link

alans09 commented Jun 16, 2014

Hi,

it is possible to create asynchronous remote server? Using SocketServer.ThreadingMixIn or any other possible way?

It would be nice to have server that can accept more than one command in one time. It would definitely lowers the time to run keywords (eg. vmware connector to work with more than one machine at the time)

@pekkaklarck
Copy link
Member

Sounds good to me. I don't have too much experience from asynchronous servers nor time to look at this in the foreseeable future, though. Anyone interested to create a PR?

@pekkaklarck
Copy link
Member

I tested using ThreadingMixin and ForkingMixin simply inheriting them in addition to SimpleXMLRPCServer. Here are findings:

  1. Allows server to process multiple requests simultaneously as expected. Basically this means that one long lasting keyword doesn't block using other keywords.

  2. Somewhat surprisingly there were no performance benefits whatsoever when running tests with quick keywords in parallel. My guess is that the overhead of threads/processes is too big compared to benefits when keywords are run really fast.

  3. The current code for intercepting stdout/stderr isn't threadsafe. Currently we just do sys.stdout = StringIO() and later sys.stdout.getvalue() and sys.stdout = sys.__stdout__, but that doesn't work if multiple threads are changing sys.stdout. Results included AttributeError when using sys.stdout.getvalue() after another thread had already restored sys.stdout and also segfault with error Fatal Python error: GC object already tracked. We somehow needed to have own sys.stdout for each thread.

@pekkaklarck
Copy link
Member

It seems that the thread safety problem could be fixed by using slightly modified version of the answers to this SO question:
http://stackoverflow.com/questions/3029816/how-do-i-get-a-thread-safe-print-in-python-2-6

pekkaklarck added a commit that referenced this issue Apr 11, 2017
Added ThreadingMixIn at some point when testing different approaches
to run server on background. It would add async support which would be
awesome, but unfortunately it doesn't work well with the current
system so intercept stdout/stderr. See issue #20 that requests this
async support for more information about the problems. The issue also
contains a potential solution, but it is too big task for 1.1 release.
@mohan-barathi
Copy link

Hi @pekkaklarck ...!
I have implemented this a couple of years back.

To overcome the thread safety problem, I implemented a mechanism in the robotframework library, where I keep track of stdout and stderr of each and every thread.

Also, I created separate output and logger objects, so that a single process can produce multiple output logs [and] reports in the same time, when multiple invocations come from different threads.

This also includes an implementation, where I authenticate the callee using IP
Hence, this also handles the issue-51

I can give a try to port that implementation to the latest robotframework library and PythonRemoteServer, if you are still interested..!

Best Regards 😃

@DanielPenning
Copy link

Hi @mohan-barathi This sounds really interesting to me! Your comment dates a few years, are you by any chance still able to give it a try? If not, can you briefly lay out your thoughts about authentication?

@SX91
Copy link

SX91 commented Jul 7, 2023

Hello! I've made a (very) simple alternative remote library implementation with async support via aiohttp and aiohttp_xmlrpc: https://github.com/SX91/aioroboremote
It lacks proper (any) docs and some useful features of this library, yet supports basic type annotations and get_keyword_types. To get started you could follow the example in repo.

It's using the same stream interception approach, thus it's not multiclient safe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants