Skip to content

Commit

Permalink
[TASK] Add processing indication for websocket clients
Browse files Browse the repository at this point in the history
  • Loading branch information
@Sharrnah
Sharrnah authored Oct 23, 2022
1 parent 3133352 commit 71a1f0b
Show file tree
Hide file tree
Showing 7 changed files with 267 additions and 43 deletions.
2 changes: 1 addition & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ _(because of the 2 GB Limit, no direct release files on GitHub)_
3. Set the Browser to allow Transparency.
4. Attach the Browser to your VR-Headset.

voilà, you have live translated subtitles of other people speaking (or videos playing) in VR which automatically disappear after 25 seconds.
Voilà, you have live translated subtitles in VR of other people speaking (or videos playing) which automatically disappear after 25 seconds.

<img src=images/vr_subtitles.gif>

Expand Down
2 changes: 2 additions & 0 deletions audioWhisper.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,8 @@ def main(devices, device_index, sample_rate, task, model, english, condition_on_
# set typing indicator for VRChat
if osc_ip != "0":
VRC_OSCLib.Bool(True, "/chatbox/typing", IP=osc_ip, PORT=osc_port)
# send start info for processing indicator in websocket client
websocket.BroadcastMessage(json.dumps({"type": "processing_start", "data": True}))

data = io.BytesIO(audio.get_wav_data())
audio_clip = AudioSegment.from_file(data)
Expand Down
17 changes: 9 additions & 8 deletions documentation/websocket-clients.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ You have build a nice Client and would like to share?, let me know so i can add

Is the simplest implementation.

Mostly intended to show how to use it without much around it, to make it easier to build your own Overlay.
Mostly intended to show how to use it without much additional logic around it, to make it easier to build your own Overlay.

_Supports only the `ws_server` [URL argument](#all-possible-configuration-url-arguments)._

Expand Down Expand Up @@ -39,13 +39,14 @@ _(first option is added with a `?`, all following with a `&`)_

## All possible Configuration URL arguments

| Arguments | Default Value | Description |
|:---------------------:|:-----------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| `ws_server` | ws://127.0.0.1:5000 | Sets the Websocket Server IP and Port to connect to. |
| `no_scroll` | None | Removes the Scrollbar if set. |
| `auto_hide_message` | 0 | Sets the Seconds after which Transcriptions are hidden, but not removed. (Can be shown again by clicking in the lower part where the Boxes appear) 0 = Disabled |
| `auto_rm_message` | 0 | Sets the Seconds after which Transcriptions are removed. (makes it impossible to access them again) 0 = Disabled |
| `show_transl_results` | websocket-remote=`both`<br/>streaming-overlay=`auto` | Sets what is shown as translation / transcription result. Can be:<br/>`both` = shows both whisper and text translations<br/>`whisper_only` = shows only Whisper results<br/>`auto` = shows text translation if available, otherwise Whisper result |
| Arguments | Default Value | Description |
|:---------------------:|:----------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
| `ws_server` | ws://127.0.0.1:5000 | Sets the Websocket Server IP and Port to connect to. |
| `no_scroll` | None | Removes the Scrollbar if set. |
| `no_loader` | False | Disables the audio processing indicator |
| `auto_hide_message` | 0 | Sets the Seconds after which Transcriptions are hidden, but not removed. (Can be shown again by clicking in the lower part where the Boxes appear) 0 = Disabled |
| `auto_rm_message` | 0 | Sets the Seconds after which Transcriptions are removed. (makes it impossible to access them again) 0 = Disabled |
| `show_transl_results` | websocket-remote=`both`<hr/>streaming-overlay=`auto` | Sets what is shown as translation / transcription result. Can be:<br/>`both` = shows both whisper and text translations.<br/>`whisper_only` = shows only Whisper results.<br/>`auto` = shows text translation if available, otherwise Whisper result. |


## Other Use-Cases
Expand Down
78 changes: 61 additions & 17 deletions websocket_clients/streaming-overlay-01/index.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@
}
return undefined;
}
function clearAllTimeouts(timeoutList) {
for (var i = 0; i < timeoutList.length; i++) {
clearTimeout(timeoutList[i]);
}
}

ws_server = getParameter("ws_server");
if (typeof ws_server != 'undefined') {
Expand All @@ -53,13 +58,31 @@
show_transl_results = "auto";
}

// no_loader argument. If set to "true", the loader is not shown
no_loader = typeof getParameter("no_loader") != 'undefined';

no_scroll = getParameter("no_scroll");

document.addEventListener("DOMContentLoaded", () => {
if (typeof no_scroll != 'undefined') {
document.querySelector('body').classList.add("noscroll");
}

// loader animation
var loaderTimeouts = [];
function showLoader(hideTime = 10) {
document.querySelector('#loader').classList.remove("hidden");
loaderTimeouts.push(
setTimeout(() => {
document.querySelector('#loader').classList.add("hidden");
}, hideTime * 1000)
);
}
function hideLoader() {
clearAllTimeouts(loaderTimeouts);
document.querySelector('#loader').classList.add("hidden");
}

// possible alert types: alert-primary, alert-success, alert-info, alert-warning, alert-danger
function addMessage(message, type, deleteTime = 0, hideTime = 0) {

Expand Down Expand Up @@ -126,11 +149,6 @@
);
});
}
function clearAllTimeouts(timeoutList) {
for (var i = 0; i < timeoutList.length; i++) {
clearTimeout(timeoutList[i]);
}
}
document.querySelector('#sse').addEventListener("click", showAllTranscriptions);
}

Expand Down Expand Up @@ -171,13 +189,20 @@
break;
}

hideLoader();
addMessage(
messageHtml,
"alert-primary",
auto_remove_msg_time,
auto_hide_msg_time
);
}

// receiving other messages
if (!no_loader && typeof received_msg.type != 'undefined' && received_msg.type === "processing_start") {
// show loader for 10 seconds
showLoader(10);
}
};

ws.onclose = function() {
Expand All @@ -204,17 +229,36 @@
</div>
</template>

<body>
<div id="sse">
<section>
<!--<div class="square_box box_three"></div>
<div class="square_box box_four"></div>-->
<div class="container mt-5">
<div class="row transcriptions">

<body>
<div id="loader" class="hidden">
<section>
<div class="container mt-2">
<div class="row center">
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
<div class="wave"></div>
</div>
</div>
</div>
</section>
</div>
</body>
</section>
</div>

<div id="sse">
<section>
<!--<div class="square_box box_three"></div>
<div class="square_box box_four"></div>-->
<div class="container mt-3">
<div class="row transcriptions">

</div>
</div>
</section>
</div>
</body>
</html>
69 changes: 67 additions & 2 deletions websocket_clients/streaming-overlay-01/main.css
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ body.noscroll {
font-size: 22pt;
line-height: 30pt;
font-weight: 400;
animation: appear-box-animation 1s cubic-bezier(0.16, 1, 0.3, 1) 0s 1 forwards;
animation: appear-box-animation 2s cubic-bezier(0.16, 1, 0.3, 1) 0s 1 forwards;
}

.transcript.hide-transcript {
Expand Down Expand Up @@ -261,4 +261,69 @@ body.noscroll {
100% {
opacity: 0;
}
}
}

/* loader animation */
#loader .center {
display: flex;
justify-content: center;
align-items: center;
/*background: #000;*/
background-color: rgba(7, 73, 149, 0.60);

box-shadow: 0px 0px 2px #0396ff;

white-space:nowrap;
overflow:hidden;
margin: 0 20vw;
min-width: 250px;

animation: appear-box-animation 1s cubic-bezier(0.16, 1, 0.3, 1) 0s 1 forwards;
}
#loader .wave {
width: 5px;
height: 54px;
background: linear-gradient(45deg, cyan, #fff);
margin: 10px;
animation: wave 1s linear infinite;
border-radius: 20px;
}
#loader .wave:nth-child(2) {
animation-delay: 0.1s;
}
#loader .wave:nth-child(3) {
animation-delay: 0.2s;
}
#loader .wave:nth-child(4) {
animation-delay: 0.3s;
}
#loader .wave:nth-child(5) {
animation-delay: 0.4s;
}
#loader .wave:nth-child(6) {
animation-delay: 0.5s;
}
#loader .wave:nth-child(7) {
animation-delay: 0.6s;
}
#loader .wave:nth-child(8) {
animation-delay: 0.7s;
}
#loader .wave:nth-child(9) {
animation-delay: 0.8s;
}
#loader .wave:nth-child(10) {
animation-delay: 0.9s;
}

@keyframes wave {
0% {
transform: scale(0);
}
50% {
transform: scale(1);
}
100% {
transform: scale(0);
}
}
Loading

0 comments on commit 71a1f0b

Please sign in to comment.