Changes to turn scrcpy into a library that can be embedded... #2591
Conversation
…y another process.
|
In principle, libscrcpy is a long-term goal I have in mind: #707 (comment). With just an important nuance from what people could expect from a library: I don't want it to be stable, i.e. any scrcpy version could change the API, for the same reasons the protocol is not "stable" (I don't want to restrict possible evolutions/refactors to preserve backward compatibility: that would constrain scrcpy developement, and the scrcpy app is the priority). It not clear how to design the API yet, but probably the client should start the server, register some callbacks / plug some sinks (what you did), and interact with the device. But first, I need to refactor some things, like how the
Note that if you just want the video stream, you could use V4L2 sink, and use any V4L2 API in your program (which could be "independent" of scrcpy). But granted, that would just provide the video, not the control part.
IIUC, it's called from your Java app (via JavaCPP), but it's in the same process (otherwise you won't be able to call functions without some form of IPC). (Your commit is squashed with some other code already in scrcpy, so it's sometimes difficult to follow. Also, you should write it on |
|
Hello, Thanks for your input. Ok, let me switch to dev branch. I'll also try to clean up the Java code and put it on github so that you can see that side of things, even though it doesn't affect scrcpy directly. With regards to API, the way I see it there should be ways to:
I have implemented 1, 2 and 4 in my PR by splitting scrcpy() function. 3 is still on my TODO list. I thought about v4l and rejected that, because I also need a way to send events, and v4l is not available on Windows. For now the only hook you can add is the AVFrame sink. In terms of API- what I did is quite simple. start() gives you a struct which needs to be passed into any other function calls, including stop(). AVFrame hook just takes callback function pointers. The structs we're passing around could probably use improvement though. But overall, I think the splitting of scrcpy() function into 3 parts could be merged pretty much as-is with minimum changes. Oh, and personally, at this point I'm happy to have a fluid API which will change and break a lot. We could start with a "pre-alpha" libscrcpy version and go from there. Sincerely, |
|
I opened #2595 which pulls into dev branch |
... and used by another process.
Hello,
Do not rush to merge this PR. At this point I'm more interested in your feedback and ideas how to improve my code.
My intent is to integrate scrcpy with SikuliX library to enable testing and automation of Android phones. https://github.com/RaiMan/SikuliX1 RaiMan/SikuliX1#473
To do that, I am attempting to modify scrcpy to be a shared library that can be invoked. Since SikuliX is in Java, I'm using https://github.com/bytedeco/javacpp/ for integration. I needed to make following changes to scrcpy:
These changes are sufficient for me to get a video feed inside Java process and copy frames into a BufferedImage. I can provide Java code tomorrow if you are interested- it needs a bit of cleaning up before I can make it public.
My question is if you would consider accepting changes like this upstream and merging them. It is my honest belief that adding better ability to integrate it with other software would boost scrcpy's popularity and usefulness.
Next steps- I intend to add ability to send events (mouse clicks, key presses, etc.) to Android next. If you have any comments on how to improve the changes I have made, please let me know and I'll do my best to accommodate you.
P.S. I'm doing this as a hobby, not a commercial project.
Sincerely,
Teofilis Martisius