Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat:media #9

Merged
merged 2 commits into from
Oct 30, 2024
Merged

feat:media #9

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
25c0a32
feat:media
JarbasAl Oct 30, 2024
18be8d5
Update hivemind_mic_sat/__init__.py
JarbasAl Oct 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 9 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,15 @@ Voice relay is built on top of [ovos-plugin-manager](https://github.com/OpenVoic

Supported plugins:

| Plugin Type | Description | Required | Link |
|------------------|-------------------------------------------------|----------|---------------------------------------------------------------------------------------------------------|
| Microphone | Captures voice input | Yes | [Microphone](https://openvoiceos.github.io/ovos-technical-manual/mic_plugins/) |
| VAD | Voice Activity Detection | Yes | [VAD](https://openvoiceos.github.io/ovos-technical-manual/vad_plugins/) |
| PHAL | Platform/Hardware Abstraction Layer | No | [PHAL](https://openvoiceos.github.io/ovos-technical-manual/PHAL/) |
| TTS Transformers | Mutate TTS Audio before playback | No | [TTS Transformers](https://openvoiceos.github.io/ovos-technical-manual/audio_service/#transformer-plugins) |
| G2P | Generate visemes (mouth movements), eg. for Mk1 | No | [G2P](https://openvoiceos.github.io/ovos-technical-manual/g2p_plugins/) |

> NOTE: the mic satellite can not (yet) play media, if you ask OVOS to "play XXX" nothing will happen as the mic-satellite will ignore the received uri
| Plugin Type | Description | Required | Link |
|------------------------|----------------------------------------------------|----------|------------------------------------------------------------------------------------------------------------|
| Microphone | Captures voice input | Yes | [Microphone](https://openvoiceos.github.io/ovos-technical-manual/mic_plugins/) |
| VAD | Voice Activity Detection | Yes | [VAD](https://openvoiceos.github.io/ovos-technical-manual/vad_plugins/) |
| PHAL | Platform/Hardware Abstraction Layer | No | [PHAL](https://openvoiceos.github.io/ovos-technical-manual/PHAL/) |
| TTS Transformers | Mutate TTS Audio before playback | No | [TTS Transformers](https://openvoiceos.github.io/ovos-technical-manual/audio_service/#transformer-plugins) |
| G2P | Generate visemes (mouth movements), eg. for Mk1 | No | [G2P](https://openvoiceos.github.io/ovos-technical-manual/g2p_plugins/) |
| Media Playback Plugins | Enables media playback (e.g., "play Metallica") | No | [Media Playback Plugins](https://openvoiceos.github.io/ovos-technical-manual/media_plugins/) |
| OCP Plugins | Provides playback support for URLs (e.g., YouTube) | No | [OCP Plugins](https://openvoiceos.github.io/ovos-technical-manual/ocp_plugins/) |

The regular voice satellite is built on top of [ovos-dinkum-listener](https://github.com/OpenVoiceOS/ovos-dinkum-listener) and is full featured supporting all plugins

Expand All @@ -46,5 +46,3 @@ This repo needs less resources but it is also **missing** some features
- Multiple WakeWords
- Audio Transformers plugins
- Dialog Transformers plugins (TODO - support in the future)
- Media Playback plugins (TODO - support in the future)
- OCP Stream plugins (TODO - support in the future)
13 changes: 10 additions & 3 deletions hivemind_mic_sat/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
from hivemind_bus_client.client import HiveMessageBusClient, BinaryDataCallbacks
from hivemind_bus_client.message import HiveMessage, HiveMessageType
from hivemind_bus_client.serialization import HiveMindBinaryPayloadType
from ovos_audio.audio import AudioService
from ovos_audio.playback import PlaybackThread as _PT
from ovos_plugin_manager.microphone import OVOSMicrophoneFactory, Microphone
from ovos_plugin_manager.utils.tts_cache import hash_sentence
Expand Down Expand Up @@ -53,20 +54,26 @@ def handle_receive_tts(self, bin_data: bytes,

class HiveMindMicrophoneClient:

def __init__(self, prefer_b64=False):
def __init__(self, prefer_b64=False, enable_media=True):
self.prefer_b64 = prefer_b64
internal = FakeBus()

self.playback: PlaybackThread = PlaybackThread(bus=internal,
queue=Queue())

self.hm_bus = HiveMessageBusClient(bin_callbacks=TTSHandler(self.playback),
internal_bus=internal)
self.hm_bus.connect(FakeBus())
self.hm_bus.connected_event.wait()
LOG.info("== connected to HiveMind")
self.mic: Microphone = OVOSMicrophoneFactory.create()
self.vad: VADEngine = OVOSVADFactory.create()
self.audio: Optional[AudioService] = None
if enable_media:
try:
self.audio = AudioService(bus=internal, validate_source=False)
LOG.info("Media playback support enabled")
except Exception as e:
LOG.error(f"Failed to initialize AudioService: {e}")
LOG.warning("Media playback support will be disabled")
self.running = False
self.hm_bus.on_mycroft("recognizer_loop:wakeword", self.handle_ww)
self.hm_bus.on_mycroft("recognizer_loop:record_begin", self.handle_rec_start)
Expand Down
Loading