Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix turn screen off on Android 14 #4446

Closed
wants to merge 10 commits into from
Closed

Fix turn screen off on Android 14 #4446

wants to merge 10 commits into from

Conversation

rom1v
Copy link
Collaborator

@rom1v rom1v commented Nov 21, 2023

This PR is on top of #4441, so just consider the last commits.

On Android 14, execute a separate process with a different classpath and LD_PRELOAD to execute the methods required to turn the device screen off.

Fixes #3927
Refs #3927 (comment)

Tested on Pixel 8 with Android 14.

Binary for Windows:

rom1v and others added 10 commits November 21, 2023 00:01
Deprecate the option --rotation and introduce a new option
--display-orientation with the 8 possible orientations (0, 90, 180, 270,
flip0, flip90, flip180 and flip270).

New shortcuts MOD+Shift+(arrow) dynamically change the display
(horizontal or vertical) flip.

Fixes #1380 <#1380>
Fixes #3819 <#3819>
For consistency with the new --display-orientation option, express the
--lock-video-orientation in degrees clockwise:

 * --lock-video-orientation=0 -> --lock-video-orientation=0
 * --lock-video-orientation=3 -> --lock-video-orientation=90
 * --lock-video-orientation=2 -> --lock-video-orientation=180
 * --lock-video-orientation=1 -> --lock-video-orientation=270
Add an option to store the orientation to apply in a recorded file.

Only rotations are supported (not flip), and most players correctly
handle it only for MP4 files (not MKV files).
Add a shortcut to set both the display and record orientations.
When running ./release.sh:

> DEPRECATION: "pkgconfig" entry is deprecated and should be replaced by
> "pkg-config"
The path can be retrieved from the classpath.

PR #4416 <#4416>

Co-authored-by: Romain Vimont <[email protected]>
Signed-off-by: Romain Vimont <[email protected]>
When scrcpy is run, a server is pushed to
/data/local/tmp/scrcpy-server.jar.

Running simultaneous scrcpy instances on the same device was not a
problem, because the file was unlinked (removed) almost immediately once
it started, avoiding any conflicts.

In order to support executing new process using the scrcpy-server.jar at
any time (to change the display power mode from a separate process), the
server file must not be unlinked, so using different names are necessary
to avoid conflicts.

Reuse the scid mechanism already used for generating device socket
names.

Refs 4315be1
Refs 439a1fd
The server was unlinked (removed) just after it started.

In order to execute a new process using the server jarfile at any time
(typically to set the display power mode from another process), keep the
file until the server closes.
On Android 14, execute a separate process with a different classpath and
LD_PRELOAD to execute the methods required to turn the device screen
off.

Fixes #3927 <#3927>
Refs #3927 comment <#3927 (comment)>

Co-authored-by: Simon Chan <[email protected]>
@Sy-CoDe
Copy link

Sy-CoDe commented Nov 21, 2023

screen off "mod o" works on my pixel 6 Android 14

@f870103
Copy link

f870103 commented Nov 21, 2023

Confirm worked on Poco F5 (Redmi Note 12 Turbo)
Android 14 Evolution X

@f870103
Copy link

f870103 commented Nov 21, 2023

but command line flags "-Swf" don't work

I don't have problem with -Swf flags.

@Sy-CoDe
Copy link

Sy-CoDe commented Nov 21, 2023

but command line flags "-Swf" don't work

I don't have problem with -Swf flags.

correction, flags do work for me as well, mistake on my end

@corentin-c
Copy link

Fixed on Pixel 6 Pro Android 14

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 22, 2023

It takes few hundred milliseconds to execute a new java process.

If I add services.jar to the classpath and libandroid_servers.so to LD_PRELOAD for the main scrcpy-server process, and execute the code directly, it is slightly faster to turn the screen off an on when using shortcuts (it is noticeable). But it fails to run scrcpy on older devices:

CANNOT LINK EXECUTABLE: library "/system/lib64/libandroid_servers.so" not found
page record for 0xb6f4f00c was not found (block_size=64)

Maybe I could start the process once (the first time the device screen is turned off), and keep it alive for further requests.

Or just accept the few hundred milliseconds latency when turning the screen on and off.

rom1v added a commit that referenced this pull request Nov 23, 2023
On Android 14, execute a separate process with a different classpath and
LD_PRELOAD to execute the methods required to turn the device screen
off.

Fixes #3927 <#3927>
Refs #3927 comment <#3927 (comment)>
PR #4446 <#4446>

Co-authored-by: Simon Chan <[email protected]>
rom1v added a commit that referenced this pull request Nov 23, 2023
On Android 14, a separate process was spawn on every display mode
request (to turn the screen off or on).

But starting a java process takes time (a few hundred milliseconds),
causing a noticeable latency between the request to turn the screen off
(MOD+o) or on (MOD+Shift+o) and the actual power mode change.

To minimize this latency, keep the process alive between requests, so
that only the first one will have to spawn the process.

It would be possible to spawn the process in advance, so that even the
first request would be immediate, but any problem would impact all
Android 14 users even without using the "turn screen off" feature.

PR #4446 <#4446>
rom1v added a commit that referenced this pull request Nov 23, 2023
On Android 14, a separate process was spawn on every display mode
request (to turn the screen off or on).

But starting a java process takes time (a few hundred milliseconds),
causing a noticeable latency between the request to turn the screen off
(MOD+o) or on (MOD+Shift+o) and the actual power mode change.

To minimize this latency, keep the process alive between requests, so
that only the first one will have to spawn the process.

It would be possible to spawn the process in advance, so that even the
first request would be immediate, but any problem would impact all
Android 14 users even without using the "turn screen off" feature.

PR #4446 <#4446>
@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2023

Maybe I could start the process once (the first time the device screen is turned off), and keep it alive for further requests.

I just did this: c5c2734

Please test the new binary :)

@yume-chan
Copy link
Contributor

yume-chan commented Nov 23, 2023

This might work (I only tested in emulator) without new processes, LD_PRELOAD and multiple CLASSPATH

var classLoaderFactoryClass = Class.forName("com.android.internal.os.ClassLoaderFactory");
var createClassLoaderMethod = classLoaderFactoryClass.getDeclaredMethod("createClassLoader", String.class, String.class, String.class, ClassLoader.class, int.class, boolean.class, String.class);
var classLoader = (PathClassLoader) createClassLoaderMethod.invoke(null, "/system/framework/services.jar", "", "", ClassLoader.getSystemClassLoader(), 10000, true, null);

var displayControlClass = classLoader.loadClass("com.android.server.display.DisplayControl");

var loadLibraryMethod = Runtime.class.getDeclaredMethod("loadLibrary0", Class.class, String.class);
loadLibraryMethod.setAccessible(true);
loadLibraryMethod.invoke(Runtime.getRuntime(), displayControlClass, "android_servers");

@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2023

@yume-chan Wow, that works! Thanks 🎉

I will rework my PR without a separate process then 😄 (even if this adds a lot of private calls)

Could you please detail why this particular ClassLoader with these arguments? (I will investigate later)


Just a note to match the parameter names:

    public static ClassLoader createClassLoader(String dexPath,
            String librarySearchPath, String libraryPermittedPath, ClassLoader parent,
            int targetSdkVersion, boolean isNamespaceShared, String classLoaderName) {

https://github.com/aosp-mirror/platform_frameworks_base/blob/0e28d56b1493b67ed0e1524065c07408f7608f99/core/java/com/android/internal/os/ClassLoaderFactory.java#L110-L112

@yume-chan
Copy link
Contributor

yume-chan commented Nov 23, 2023

First I found Android has its own dynamic library linker (libdl) in bionic, and it has a feature called linker namespace, that can limit which libraries a caller can load.

Android Runtime utilizes this feature by associating class loaders with linker namespaces. app_process will create a class loader for services.jar with a shared linker namespace, but create a class loader for user code (for example scrcpy-server.jar) with an isolated and limited linker namespace. That's why calling System.loadLibrary("android_servers") in our code will fail. (by using LD_PRELOAD, the library is loaded immediately using the "default" linker namespace, so it succeeds)

A shared namespace means it can load all libraries from its parent namespace. For app_process, the parent namespace should be "default", which can load libraries from any locations.

In Android Runtime, the call stack is System.loadLibrary -> JavaVMExt::LoadNativeLibrary -> android::OpenNativeLibrary. The last method finds the associated linker namespace for class_loader parameter (or create an isolated one if it doesn't have one, this is the case for the class loader that loads user code), then calls android_dlopen_ext in bionic to load the library using that linker namespace.

So we need a class loader that 1) has a shared linker namespace so it can load libandroid_servers.so and its dependencies in /apex folder and 2) has Java classes from services.jar so JNI initialization in libandroid_servers.so can find the classes it wants.

The dexPath argument includes "/system/framework/services.jar" to satisfy 2), and isNamespaceShared argument is true to satisfy 1). Other arguments don't seem to matter, with parent: null and targetSdkVersion: 0 it still works.

We also need to actually load libandroid_servers.so using that class loader. com.android.server.SystemServer.main contains System.loadLibrary("android_servers");, but calling it using reflection fails with initializing system services.System.loadLibrary (internally Runtime.loadLibrary) uses caller class's class loader, so I have to call Runtime.loadLibrary0 using reflection, as it accepts a custom class (and class loader).


IMO a much cleaner implementation is re-creating the communication with SurfaceFlinger in Java (not sure if it's possible). At least com.android.shell package has the system permission to do that.

rom1v added a commit that referenced this pull request Nov 23, 2023
On Android 14, the methods to access the display have been moved to
DisplayControl, which is not in the core framework. Use a specific
ClassLoader to access this class and its native dependencies.

Fixes #3927 <#3927>
Refs #3927 comment <#3927 (comment)>
Refs #4446 comment <#4446 (comment)>
PR #4456 <#4456>

Co-authored-by: Simon Chan <[email protected]>
@rom1v
Copy link
Collaborator Author

rom1v commented Nov 23, 2023

Superseded by #4456

@rom1v rom1v closed this Nov 23, 2023
rom1v added a commit that referenced this pull request Nov 25, 2023
On Android 14, the methods to access the display have been moved to
DisplayControl, which is not in the core framework. Use a specific
ClassLoader to access this class and its native dependencies.

Fixes #3927 <#3927>
Refs #3927 comment <#3927 (comment)>
Refs #4446 comment <#4446 (comment)>
PR #4456 <#4456>

Co-authored-by: Simon Chan <[email protected]>
Signed-off-by: Romain Vimont <[email protected]>
@yume-chan
Copy link
Contributor

FYI, using direct communication with SurfaceFlinger

var serviceManagerClass = Class.forName("android.os.ServiceManager");
var getServiceMethod = serviceManagerClass.getDeclaredMethod("getService", String.class);
var surfaceComposer = (IBinder) getServiceMethod.invoke(null, "SurfaceFlingerAIDL");

long[] displayIds;
{
    var data = Parcel.obtain();
    var reply = Parcel.obtain();
    try {
        data.writeInterfaceToken("android.gui.ISurfaceComposer");
        surfaceComposer.transact(IBinder.FIRST_CALL_TRANSACTION + 5, data, reply, 0);
        reply.readException();
        var size = reply.readInt();
        displayIds = new long[size];
        for (var i = 0; i < size; i += 1) {
            displayIds[i] = reply.readLong();
        }
    } finally {
        data.recycle();
        reply.recycle();
    }
}

var surfaceControlClass = Class.forName("android.view.SurfaceControl");
var setDisplayPowerModeMethod = surfaceControlClass.getDeclaredMethod("setDisplayPowerMode", IBinder.class, int.class);

for (long displayId : displayIds) {
    IBinder token;
    {
        var data = Parcel.obtain();
        var reply = Parcel.obtain();
        try {
            data.writeInterfaceToken("android.gui.ISurfaceComposer");
            data.writeLong(displayId);
            surfaceComposer.transact(IBinder.FIRST_CALL_TRANSACTION + 6, data, reply, 0);
            reply.readException();
            token = reply.readStrongBinder();
        } finally {
            data.recycle();
            reply.recycle();
        }
    }

    setDisplayPowerModeMethod.invoke(null, token, 0);
}

@rom1v
Copy link
Collaborator Author

rom1v commented Dec 1, 2023

Thank you for the sample. It's great to have several ways to access it 👍

But I'm afraid that it would be too "fragile" (and will fail on some devices without useful error messages if the transaction id doesn't match).

Moreover, this commit says:

Additonal service, named as "SurfaceFlingerAIDL", is added to surfaceflinger during the process of migrating ISurfaceComposer interface to AIDL. New changes are put into namespace, android::gui. Once migration is complete, this service will be deleted.

So IIUC, this solution may not be permanent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants