Concept of Assist on Apple Watch #2400
Conversation
Codecov Report
@@ Coverage Diff @@
## master #2400 +/- ##
==========================================
+ Coverage 28.05% 28.17% +0.12%
==========================================
Files 273 277 +4
Lines 30232 30337 +105
==========================================
+ Hits 8481 8548 +67
- Misses 21751 21789 +38
|
|
Also, another todo item that just popped on my mind is to try to use Apple Watch Ultra action button to open Assist directly |
|
I had great progress on this concept, now you are able to use just voice even though watchOS can't transcribe locally, it sends the audio file to iPhone and then it is processed there. The performance is great, I will add a video below. IMG_0524-low.mov |
|
I did one more iteration, now it is possible to launch assist via complication and also it automatically stops recording on silence. 123-low-2.mov |
|
I have moved all watch actions that were handled in App Delegate to a specific service for it's type, sharing the same protocol. |
|
Hallo, kan ik deze app ergens testen voor Apple Watch? Dit is erg handig! |
Hey, if you know how to compile an iOS app you can try right away using this branch, otherwise I still need to find time to wrap it up and check with Nabu Casa if the feature matches their expectations (since Android already has it, I would need to align the UI and behavior) |
bgoncal
force-pushed
the
assist-apple-watch
branch
from
3e7f414 to
bdfeed7
Compare
bgoncal
force-pushed
the
assist-apple-watch
branch
from
51ee6c2 to
67ac37c
Compare
|
Update on it's speed launching from complication. Next step: launch from action button (Apple Watch Ultra) and allow selecting pipeline wanted to use. |
|
I'm closing this draft PR, I will open another one based on this but using HA Assist pipeline directly through HA Websocket API |
Summary
Goal:
Achieved on this PR:
Challenge:
Since SFSpeechRecognizer is not available on the watchOS I have use (for demonstration purposes) the keyboard dictation, which is not good since you need extra taps. One alternative could be record the audio in the Apple Watch and process over the iPhone, but performance would need to be tested.-> I am now recording the audio on watchOS and processing it on iOS
Also pending:
provide a complication that directly opens Assist. (Done ✔️ check third demo video)~Why not demo it on a real Apple Watch?
Currently my devices are running iOS and watchOS beta's and I couldn't make them quickly work with home assistant project yet.-> I have added a new demo video showing it working on a real device + fully by voice (currently you still need to "end" the session manually)
Screenshots
assist-low.mov
IMG_0524-low.mov
123-low-2.mov