From 4a8fce114a6ea05db6180b019807b6c3aea7ac3e Mon Sep 17 00:00:00 2001 From: GiviMAD Date: Mon, 20 Dec 2021 19:21:11 +0100 Subject: [PATCH] [androiddebugbridge] Add channels for record events, open urls and doc improvements (#11692) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Add channels for record events, open urls and doc improvements Signed-off-by: Miguel Álvarez Díez Signed-off-by: Michael Schmidt --- .../README.md | 30 +++++- .../AndroidDebugBridgeBindingConstants.java | 4 +- .../AndroidDebugBridgeConfiguration.java | 4 + .../internal/AndroidDebugBridgeDevice.java | 98 ++++++++++++++++++- .../AndroidDebugBridgeDiscoveryService.java | 2 +- .../internal/AndroidDebugBridgeHandler.java | 54 +++++++++- .../resources/OH-INF/thing/thing-types.xml | 26 +++++ 7 files changed, 208 insertions(+), 10 deletions(-) diff --git a/bundles/org.openhab.binding.androiddebugbridge/README.md b/bundles/org.openhab.binding.androiddebugbridge/README.md index cc44815071c3f..e8cd54748c2e1 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/README.md +++ b/bundles/org.openhab.binding.androiddebugbridge/README.md @@ -1,7 +1,9 @@ # Android Debug Bridge Binding -This binding allows to connect to android devices through the adb protocol. -The device needs to have **usb debugging enabled** and **allow debugging over tcp**, some devices allow to enable this in the device options but others need a previous connection through adb or even be rooted. +This binding allows to connect to android devices through the adb protocol. + +The device needs to have **usb debugging enabled** and **allow debugging over tcp**, some devices allow to enable this in the device options but others need a previous connection through adb or even be rooted. + If you are not familiar with adb I suggest you to search "How to enable adb over wifi on \" or something like that. ## Supported Things @@ -10,7 +12,11 @@ This binding was tested on the Fire TV Stick (android version 7.1.2, volume cont ## Discovery -As I can not find a way to identify android devices in the network the discovery will try to connect through adb to all the reachable ip in the defined range, you could customize the discovery process through the binding options. **Your device will prop a message requesting you to authorize the connection, you should check the option "Always allow connections from this device" (or something similar) and accept**. +As I can not find a way to identify android devices in the network the discovery will try to connect through adb to all the reachable ip in the defined range. + +You could customize the discovery process through the binding options. + +**Your device will prompt a message requesting you to authorize the connection, you should check the option "Always allow connections from this device" (or something similar) and accept**. ## Binding Configuration @@ -33,6 +39,7 @@ As I can not find a way to identify android devices in the network the discovery | port | int | Device port listening to adb connections (default: 5555) | | refreshTime | int | Seconds between device status refreshes (default: 30) | | timeout | int | Command timeout in seconds (default: 5) | +| recordDuration | int | Record input duration in seconds | | mediaStateJSONConfig | String | Expects a JSON array. Allow to configure the media state detection method per app. Described in the following section | ## Media State Detection @@ -52,6 +59,18 @@ This is a sample of the mediaStateJSONConfig thing configuration: `[{"name": "com.amazon.tv.launcher", "mode": "idle"},{"name": "org.jellyfin.androidtv", "mode": "wake_lock", "wakeLockPlayStates": [2,3]},{"name": "com.amazon.firetv.youtube", "mode": "wake_lock", "wakeLockPlayStates": [2]}]` +## Record/Send input events +As the execution of key events takes a while, you can use input events as an alternative way to control your device. + +They are pretty device specific, so you should use the record-input and recorded-input channels to store/send those events. + +An example of what you can do: +* You can send the command `UP` to the `record-input` channel the binding will then capture the events you send through your remote for the defined recordDuration config for the thing, so press once the UP key on your remote and wait a while. +* Now that you have recorded your input, you can send the `UP` command to the `recorded-input` event and it will send the recorded event to the android device. + +Please note that events could fail if the input method is removed, for example it could fail if you clone the events of a bluetooth controller and the remote goes offline. This is happening for me when recording the Fire TV remote events but not for my Xiaomi TV which also has a bt remote controller. + + ## Channels | channel | type | description | @@ -59,12 +78,17 @@ This is a sample of the mediaStateJSONConfig thing configuration: | key-event | String | Send key event to android device. Possible values listed below | | text | String | Send text to android device | | tap | String | Send tap event to android device (format x,y) | +| url | String | Open url in browser | | media-volume | Dimmer | Set or get media volume level on android device | | media-control | Player | Control media on android device | | start-package | String | Run application by package name | | stop-package | String | Stop application by package name | +| stop-current-package | String | Stop current application | | current-package | String | Package name of the top application in screen | +| record-input | String | Capture events, generate the equivalent command and store it under the provided name | +| recorded-input | String | Emulates previously captured input events by name | | shutdown | String | Power off/reboot device (allowed values POWER_OFF, REBOOT) | +| awake-state | OnOff | Awake state value. | | wake-lock | Number | Power wake lock value | | screen-state | Switch | Screen power state | diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeBindingConstants.java b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeBindingConstants.java index 26343ecc8c873..79116fb1e91cc 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeBindingConstants.java +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeBindingConstants.java @@ -37,6 +37,7 @@ public class AndroidDebugBridgeBindingConstants { public static final String KEY_EVENT_CHANNEL = "key-event"; public static final String TEXT_CHANNEL = "text"; public static final String TAP_CHANNEL = "tap"; + public static final String URL_CHANNEL = "url"; public static final String MEDIA_VOLUME_CHANNEL = "media-volume"; public static final String MEDIA_CONTROL_CHANNEL = "media-control"; public static final String START_PACKAGE_CHANNEL = "start-package"; @@ -47,7 +48,8 @@ public class AndroidDebugBridgeBindingConstants { public static final String WAKE_LOCK_CHANNEL = "wake-lock"; public static final String SCREEN_STATE_CHANNEL = "screen-state"; public static final String SHUTDOWN_CHANNEL = "shutdown"; - + public static final String RECORD_INPUT_CHANNEL = "record-input"; + public static final String RECORDED_INPUT_CHANNEL = "recorded-input"; // List of all Parameters public static final String PARAMETER_IP = "ip"; public static final String PARAMETER_PORT = "port"; diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeConfiguration.java b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeConfiguration.java index 17f23029d8cbd..62717beb574fd 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeConfiguration.java +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeConfiguration.java @@ -38,6 +38,10 @@ public class AndroidDebugBridgeConfiguration { * Command timeout seconds. */ public int timeout = 5; + /** + * Record input duration in seconds. + */ + public int recordDuration = 5; /** * Configure media state detection behavior by package */ diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDevice.java b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDevice.java index 06ee5ff2367fb..debd31e62a287 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDevice.java +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDevice.java @@ -22,6 +22,7 @@ import java.nio.charset.StandardCharsets; import java.security.NoSuchAlgorithmException; import java.security.spec.InvalidKeySpecException; +import java.util.ArrayList; import java.util.Arrays; import java.util.Base64; import java.util.concurrent.*; @@ -55,6 +56,10 @@ public class AndroidDebugBridgeDevice { private static final Pattern TAP_EVENT_PATTERN = Pattern.compile("(?\\d+),(?\\d+)"); private static final Pattern PACKAGE_NAME_PATTERN = Pattern .compile("^([A-Za-z]{1}[A-Za-z\\d_]*\\.)+[A-Za-z][A-Za-z\\d_]*$"); + private static final Pattern URL_PATTERN = Pattern.compile( + "https?:\\/\\/(www\\.)?[-a-zA-Z0-9@:%._\\+~#=]{2,256}\\.[a-z]{2,4}\\b([-a-zA-Z0-9@:%_\\+.~#?&//=]*)$"); + private static final Pattern INPUT_EVENT_PATTERN = Pattern + .compile("/(?\\S+): (?\\S+) (?\\S+) (?\\S+)$", Pattern.MULTILINE); private static @Nullable AdbCrypto adbCrypto; @@ -78,6 +83,7 @@ public class AndroidDebugBridgeDevice { private String ip = "127.0.0.1"; private int port = 5555; private int timeoutSec = 5; + private int recordDuration; private @Nullable Socket socket; private @Nullable AdbConnection connection; private @Nullable Future commandFuture; @@ -86,10 +92,11 @@ public class AndroidDebugBridgeDevice { this.scheduler = scheduler; } - public void configure(String ip, int port, int timeout) { + public void configure(String ip, int port, int timeout, int recordDuration) { this.ip = ip; this.port = port; this.timeoutSec = timeout; + this.recordDuration = recordDuration; } public void sendKeyEvent(String eventCode) @@ -111,18 +118,68 @@ public void sendTap(String point) runAdbShell("input", "mouse", "tap", match.group("x"), match.group("y")); } + public void openUrl(String url) + throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { + var match = URL_PATTERN.matcher(url); + if (!match.matches()) { + throw new AndroidDebugBridgeDeviceException("Unable to parse url"); + } + runAdbShell("am", "start", "-a", url); + } + public void startPackage(String packageName) throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { + if (packageName.contains("/")) { + startPackageWithActivity(packageName); + return; + } if (!PACKAGE_NAME_PATTERN.matcher(packageName).matches()) { logger.warn("{} is not a valid package name", packageName); return; } var out = runAdbShell("monkey", "--pct-syskeys", "0", "-p", packageName, "-v", "1"); if (out.contains("monkey aborted")) { + startTVPackage(packageName); + } + } + + private void startTVPackage(String packageName) + throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { + // https://developer.android.com/training/tv/start/start + String result = runAdbShell("monkey", "--pct-syskeys", "0", "-c", "android.intent.category.LEANBACK_LAUNCHER", + "-p", packageName, "1"); + if (result.contains("monkey aborted")) { throw new AndroidDebugBridgeDeviceException("Unable to open package"); } } + public void startPackageWithActivity(String packageWithActivity) + throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { + var parts = packageWithActivity.split("/"); + if (parts.length != 2) { + logger.warn("{} is not a valid package", packageWithActivity); + return; + } + var packageName = parts[0]; + var activityName = parts[1]; + if (!PACKAGE_NAME_PATTERN.matcher(packageName).matches()) { + logger.warn("{} is not a valid package name", packageName); + return; + } + if (!PACKAGE_NAME_PATTERN.matcher(activityName).matches()) { + logger.warn("{} is not a valid activity name", activityName); + return; + } + var out = runAdbShell("am", "start", "-n", packageWithActivity); + if (out.contains("usage: am")) { + out = runAdbShell("am", "start", packageWithActivity); + } + if (out.contains("usage: am") || out.contains("Exception")) { + logger.warn("open {} fail; retrying to open without activity info", packageWithActivity); + startPackage(packageName); + } + } + public void stopPackage(String packageName) throws AndroidDebugBridgeDeviceException, InterruptedException, TimeoutException, ExecutionException { if (!PACKAGE_NAME_PATTERN.matcher(packageName).matches()) { @@ -160,7 +217,7 @@ public boolean isScreenOn() throws InterruptedException, AndroidDebugBridgeDevic var state = devicesResp.split("=")[1].trim(); return state.equals("ON"); } catch (NumberFormatException e) { - logger.debug("Unable to parse device wake lock: {}", e.getMessage()); + logger.debug("Unable to parse device screen state: {}", e.getMessage()); } } throw new AndroidDebugBridgeDeviceReadException("Unable to read screen state"); @@ -258,6 +315,36 @@ private VolumeInfo getVolume(int stream) throws AndroidDebugBridgeDeviceExceptio return volumeInfo; } + public String recordInputEvents() + throws AndroidDebugBridgeDeviceException, InterruptedException, TimeoutException, ExecutionException { + String out = runAdbShell(recordDuration * 2, "getevent", "&", "sleep", Integer.toString(recordDuration), "&&", + "exit"); + var matcher = INPUT_EVENT_PATTERN.matcher(out); + var commandList = new ArrayList(); + try { + while (matcher.find()) { + String inputPath = matcher.group("input"); + int n1 = Integer.parseInt(matcher.group("n1"), 16); + int n2 = Integer.parseInt(matcher.group("n2"), 16); + int n3 = Integer.parseInt(matcher.group("n3"), 16); + commandList.add(String.format("sendevent /%s %d %d %d", inputPath, n1, n2, n3)); + } + } catch (NumberFormatException e) { + logger.warn("NumberFormatException while parsing events, aborting"); + return ""; + } + return String.join(" && ", commandList); + } + + public void sendInputEvents(String command) + throws AndroidDebugBridgeDeviceException, InterruptedException, TimeoutException, ExecutionException { + String out = runAdbShell(command.split(" ")); + if (out.length() != 0) { + logger.warn("Device event unexpected output: {}", out); + throw new AndroidDebugBridgeDeviceException("Device event execution fail"); + } + } + public void rebootDevice() throws AndroidDebugBridgeDeviceException, InterruptedException, TimeoutException, ExecutionException { try { @@ -313,6 +400,11 @@ public void connect() throws AndroidDebugBridgeDeviceException, InterruptedExcep private String runAdbShell(String... args) throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { + return runAdbShell(timeoutSec, args); + } + + private String runAdbShell(int commandTimeout, String... args) + throws InterruptedException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { var adb = connection; if (adb == null) { throw new AndroidDebugBridgeDeviceException("Device not connected"); @@ -337,7 +429,7 @@ private String runAdbShell(String... args) return byteArrayOutputStream.toString(StandardCharsets.US_ASCII); }); this.commandFuture = commandFuture; - return commandFuture.get(timeoutSec, TimeUnit.SECONDS); + return commandFuture.get(commandTimeout, TimeUnit.SECONDS); } finally { var commandFuture = this.commandFuture; if (commandFuture != null) { diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDiscoveryService.java b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDiscoveryService.java index 837716b21b082..ef0de0dcd0b9a 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDiscoveryService.java +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeDiscoveryService.java @@ -129,7 +129,7 @@ protected void startScan() { private void discoverWithADB(String ip, int port) throws InterruptedException, AndroidDebugBridgeDeviceException, AndroidDebugBridgeDeviceReadException, TimeoutException, ExecutionException { var device = new AndroidDebugBridgeDevice(scheduler); - device.configure(ip, port, 10); + device.configure(ip, port, 10, 0); try { device.connect(); logger.debug("connected adb at {}:{}", ip, port); diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeHandler.java b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeHandler.java index 39d18c33295f8..14dea1ada80b5 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeHandler.java +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/java/org/openhab/binding/androiddebugbridge/internal/AndroidDebugBridgeHandler.java @@ -20,6 +20,7 @@ import java.util.concurrent.ScheduledFuture; import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; +import java.util.regex.Pattern; import org.eclipse.jdt.annotation.NonNullByDefault; import org.eclipse.jdt.annotation.Nullable; @@ -61,6 +62,7 @@ public class AndroidDebugBridgeHandler extends BaseThingHandler { private static final String SHUTDOWN_POWER_OFF = "POWER_OFF"; private static final String SHUTDOWN_REBOOT = "REBOOT"; private static final Gson GSON = new Gson(); + private static final Pattern RECORD_NAME_PATTERN = Pattern.compile("^[A-Za-z0-9_]*$"); private final Logger logger = LoggerFactory.getLogger(AndroidDebugBridgeHandler.class); private final AndroidDebugBridgeDevice adbConnection; private int maxMediaVolume = 0; @@ -116,6 +118,9 @@ private void handleCommandInternal(ChannelUID channelUID, Command command) case TAP_CHANNEL: adbConnection.sendTap(command.toFullString()); break; + case URL_CHANNEL: + adbConnection.openUrl(command.toFullString()); + break; case MEDIA_VOLUME_CHANNEL: handleMediaVolume(channelUID, command); break; @@ -170,9 +175,51 @@ private void handleCommandInternal(ChannelUID channelUID, Command command) updateStatus(ThingStatus.OFFLINE, ThingStatusDetail.GONE, "Rebooting"); break; } + break; + case RECORD_INPUT_CHANNEL: + recordDeviceInput(command); + break; + case RECORDED_INPUT_CHANNEL: + String recordName = getRecordPropertyName(command); + var inputCommand = this.getThing().getProperties().get(recordName); + if (inputCommand != null) { + adbConnection.sendInputEvents(inputCommand); + } + break; + } + } + + private void recordDeviceInput(Command recordNameCommand) + throws AndroidDebugBridgeDeviceException, InterruptedException, TimeoutException, ExecutionException { + var recordName = recordNameCommand.toFullString(); + if (!RECORD_NAME_PATTERN.matcher(recordName).matches()) { + logger.warn("Invalid record name, accepts alphanumeric values with '_'."); + return; + } + String recordPropertyName = getRecordPropertyName(recordName); + logger.debug("RECORD: {}", recordPropertyName); + var eventCommand = adbConnection.recordInputEvents(); + if (eventCommand.isEmpty()) { + logger.debug("No events recorded"); + if (this.getThing().getProperties().containsKey(recordPropertyName)) { + this.getThing().setProperty(recordPropertyName, null); + updateProperties(editProperties()); + logger.debug("Record {} deleted", recordName); + } + } else { + updateProperty(recordPropertyName, eventCommand); + logger.debug("New record {}: {}", recordName, eventCommand); } } + private String getRecordPropertyName(String recordName) { + return String.format("input-record:%s", recordName); + } + + private String getRecordPropertyName(Command recordNameCommand) { + return getRecordPropertyName(recordNameCommand.toFullString()); + } + private void handleMediaVolume(ChannelUID channelUID, Command command) throws InterruptedException, AndroidDebugBridgeDeviceReadException, AndroidDebugBridgeDeviceException, TimeoutException, ExecutionException { @@ -264,7 +311,8 @@ public void initialize() { if (mediaStateJSONConfig != null && !mediaStateJSONConfig.isEmpty()) { loadMediaStateConfig(mediaStateJSONConfig); } - adbConnection.configure(currentConfig.ip, currentConfig.port, currentConfig.timeout); + adbConnection.configure(currentConfig.ip, currentConfig.port, currentConfig.timeout, + currentConfig.recordDuration); updateStatus(ThingStatus.UNKNOWN); connectionCheckerSchedule = scheduler.scheduleWithFixedDelay(this::checkConnection, 0, currentConfig.refreshTime, TimeUnit.SECONDS); @@ -331,7 +379,8 @@ private void refreshStatus() throws InterruptedException, AndroidDebugBridgeDevi awakeState = adbConnection.isAwake(); deviceAwake = awakeState; } catch (TimeoutException e) { - logger.warn("Unable to refresh awake state: Timeout"); + // happen a lot when device is sleeping; abort refresh other channels + logger.debug("Unable to refresh awake state: Timeout; aborting channels refresh"); return; } var awakeStateChannelUID = new ChannelUID(this.thing.getUID(), AWAKE_STATE_CHANNEL); @@ -339,6 +388,7 @@ private void refreshStatus() throws InterruptedException, AndroidDebugBridgeDevi updateState(awakeStateChannelUID, OnOffType.from(awakeState)); } if (!awakeState && !prevDeviceAwake) { + // abort refresh channels while device is sleeping, throws many timeouts logger.debug("device {} is sleeping", config.ip); return; } diff --git a/bundles/org.openhab.binding.androiddebugbridge/src/main/resources/OH-INF/thing/thing-types.xml b/bundles/org.openhab.binding.androiddebugbridge/src/main/resources/OH-INF/thing/thing-types.xml index 3641be49d2b87..df8d9a26d417a 100644 --- a/bundles/org.openhab.binding.androiddebugbridge/src/main/resources/OH-INF/thing/thing-types.xml +++ b/bundles/org.openhab.binding.androiddebugbridge/src/main/resources/OH-INF/thing/thing-types.xml @@ -11,6 +11,9 @@ + + + @@ -44,6 +47,11 @@ Command timeout seconds. 5 + + + How much time the record-input channel wait for events to record. + 5 + JSON config that allows to modify the media state detection strategy for each app. Refer to the binding @@ -363,6 +371,24 @@ Send tap event to android device + + String + + Open url in the browser + + + + String + + Record input events under provided name + + + + String + + Send previous recorded input events by name + + String