Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System tray state machine #290

Open
wants to merge 49 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
011d8de
build app.py
jesicasusanto Jun 19, 2023
dcd56b6
Merge branch 'MLDSAI:main' into windows-widget
jesicasusanto Jun 20, 2023
d8c465d
add button
jesicasusanto Jun 20, 2023
4ca59a8
add button
jesicasusanto Jun 20, 2023
dbb9979
implement widget with tkinter
jesicasusanto Jun 23, 2023
cf04900
implement replay inprogress and replay paused
jesicasusanto Jun 23, 2023
323e829
resize image
jesicasusanto Jun 23, 2023
81f6eca
Merge branch 'MLDSAI:main' into windows-widget
jesicasusanto Jun 25, 2023
2f5ecd0
Merge branch 'MLDSAI:main' into windows-widget
jesicasusanto Jun 26, 2023
238ce1d
add kivy widget app
jesicasusanto Jun 26, 2023
ad2c104
position widget above active window for windows
jesicasusanto Jun 26, 2023
b071ec3
add widget macos
jesicasusanto Jun 26, 2023
f765d48
format code with black
jesicasusanto Jun 26, 2023
6096a8f
reusing openadapt.window
jesicasusanto Jun 27, 2023
ecf0e65
split proc to replay_proc and record_proc
jesicasusanto Jun 27, 2023
aa5d1bc
remove unused imports
jesicasusanto Jun 27, 2023
8c99bf3
work in progress
jesicasusanto Jun 27, 2023
59fd3e9
add show_data flag to get_window_state()
jesicasusanto Jun 28, 2023
5e28b26
handle NoneType Error when running stateful
jesicasusanto Jun 28, 2023
6abdee7
Merge branch 'MLDSAI:main' into windows-widget
jesicasusanto Jun 29, 2023
e89ee68
add SCREEN_SCALE to pos widget above active window
jesicasusanto Jun 29, 2023
ca130bc
undo strategies. __init__
jesicasusanto Jun 30, 2023
2d51ff9
make widget move faster
jesicasusanto Jun 30, 2023
23fb1ad
resize logo and delete unsude logo
jesicasusanto Jun 30, 2023
d74385d
fix SHOW_DATA parameter
jesicasusanto Jun 30, 2023
17c545c
fix SHOW_DATA
jesicasusanto Jun 30, 2023
1de4930
fix widget keep moving to top
jesicasusanto Jun 30, 2023
b6d7bc1
remove log
jesicasusanto Jun 30, 2023
895d738
fix widget post on fullscreen & on diff platforms
jesicasusanto Jun 30, 2023
ddead1e
add docstrings and comments
jesicasusanto Jun 30, 2023
ec3f672
add kivy
jesicasusanto Jun 30, 2023
74254f3
fix position_above_active_window args error
jesicasusanto Jun 30, 2023
8d6f8c2
add WIDGET_HEIGHT and WIDGET_WIDTH
jesicasusanto Jun 30, 2023
f962f73
update app1.py with PySide6
jesicasusanto Jul 4, 2023
de071c1
Merge branch 'OpenAdaptAI:main' into windows-widget
jesicasusanto Jul 5, 2023
2da733b
modify tray
jesicasusanto Jul 7, 2023
a2c2d27
Merge branch 'windows-widget' of https://github.com/jesicasusanto/PAT…
jesicasusanto Jul 7, 2023
0c6283b
Merge branch 'OpenAdaptAI:main' into windows-widget
jesicasusanto Jul 12, 2023
1027546
black
jesicasusanto Jul 18, 2023
49d3d32
remove is not None
jesicasusanto Jul 18, 2023
bb45d40
fix merge conflict
jesicasusanto Jul 18, 2023
19afebd
Merge branch 'OpenAdaptAI:main' into windows-widget
jesicasusanto Jul 18, 2023
3a66d88
remove kivy and add pyside6
jesicasusanto Jul 18, 2023
038470e
rename app.py to widget.py
jesicasusanto Jul 20, 2023
e746bc6
add poetry run widget
jesicasusanto Jul 21, 2023
dbed721
black
jesicasusanto Jul 21, 2023
e3afd5a
fix merge conflict
jesicasusanto Jul 25, 2023
443150a
typo
jesicasusanto Jul 25, 2023
7a5e401
fix models
jesicasusanto Aug 2, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 128 additions & 0 deletions openadapt/widget/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
import kivy
from subprocess import Popen
import signal
from kivy.app import App
from kivy.uix.button import Button
from kivy.uix.floatlayout import FloatLayout
from kivy.core.window import Window
import pygetwindow as gw
from loguru import logger
import ctypes
from ctypes import wintypes
from kivy.clock import Clock
import sys
import pywinauto
import Quartz


class OpenAdaptWidget(FloatLayout):
def __init__(self, **kwargs):
super(OpenAdaptWidget, self).__init__(**kwargs)
self.window = Window
self.window.borderless = True
self.window.size = (50, 50)
self.window.clearcolor = (255, 255, 255)
self.window.always_on_top = True
self.PROC = None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this isn't a constant, what do you think about naming it e.g. process? Even better, how about splitting it into replay_process and record_process?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved in ecf0e65

self.button = Button(background_normal="assets/logo.png")
self.button.bind(on_press=self.callback)
self.current_state = "default"
self.add_widget(self.button)
# Check for active window changes every 0.5 seconds
self.active_window = None
self._active_window_checker = Clock.schedule_interval(
self.position_above_active_window, 0.5
)

def position_above_active_window(self, *args):
if sys.platform == "win32":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jesicasusanto ! What do you think about re-using openadapt.window here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're welcome @abrichr resolved in 6096a8f

try:
app = pywinauto.application.Application(backend="win32").connect(
active_only=True
)
window = app.active()
active_window_properties = window.get_properties()
self.window.top, self.window.right = (
active_window_properties["rectangle"].top,
active_window_properties["rectangle"].right,
)
except:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned elsewhere I think it would be preferable to re-use openadapt.window here.

In general, however, swallowing exceptions like this is not recommended -- logging is preferable.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolved in 6096a8f

elif sys.platform == "darwin":
windows = Quartz.CGWindowListCopyWindowInfo(
(
Quartz.kCGWindowListExcludeDesktopElements
| Quartz.kCGWindowListOptionOnScreenOnly
),
Quartz.kCGNullWindowID,
)
active_windows_info = [win for win in windows if win["kCGWindowLayer"] == 0]
meta = active_windows_info[0]
bounds = meta["kCGWindowBounds"]
left = bounds["X"]
self.window.top = bounds["Y"] + 10
width = bounds["Width"]
self.window.left = left
if (self.window.top, self.window.right) != self.prev_active_window_position:
self.prev_active_window_position = (self.window.top, self.window.right)
if self.current_state == "replay_in_progress":
self.current_state == "replay_paused"

def callback(self, instance):
if self.current_state == "default":
self.button.background_normal = "assets/recording_inprogress.png"
self.current_state = "recording_in_progress"
self.start_recording()
elif self.current_state == "recording_in_progress":
self.button.background_normal = "assets/replay_available.png"
self.current_state = "replay_available"
self.stop_recording()
elif self.current_state == "replay_available":
self.button.background_normal = "assets/replay_inprogress.png"
self.current_state = "replaying_in_progress"
self.replay_recording()
elif self.current_state == "replaying_in_progress":
self.button.background_normal = "assets/replay_paused"
self.current_state = "replaying_paused"
self.pause_replay()
elif self.current_state == "replaying_paused":
self.button.background_normal = "assets/replay_inprogress.png"
self.current_state = "replaying_in_progress"
self.resume_replay()

def start_recording(self):
self.PROC = Popen(
"python -m openadapt.record " + "test",
shell=True,
)

def stop_recording(self):
try:
self.PROC.send_signal(signal.CTRL_C_EVENT)
# Wait for process to terminate
self.PROC.wait()
except KeyboardInterrupt:
# Catch the KeyboardInterrupt exception to prevent termination
pass
self.PROC = None

def replay_recording(self):
self.PROC = Popen(
"python -m openadapt.replay " + "NaiveReplayStrategy",
shell=True,
)

def pause_replay(self):
self.PROC.send_signal(signal.SIGSTOP)

def resume_replay(self):
self.PROC.send_signal(signal.SIGCONT)


class OpenAdapt(App):
def build(self):
return OpenAdaptWidget()


if __name__ == "__main__":
OpenAdapt().run()
Binary file added openadapt/widget/assets/available.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/logo.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/recording_available.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/recording_inprogress.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/replay_available.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/replay_inprogress.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added openadapt/widget/assets/replay_paused.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.