New JavaScript api for TTS

[mikunimaru]

Purpose / Description

New JavaScript api

Added an api that can call TTS from JavaScript.

Fixes

Fixes #8794

Approach

The following API has been added.

ankiTtsSpeak // Speak a string
ankiTtsSetLanguage // Set the language for speaking
ankiTtsSetPitch // Set the pitch when speaking
ankiTtsSetSpeechRate //Set the speed when speak
ankiTtsStop //Stop speaking

How Has This Been Tested?

Confirmed the operation of all apis on physical devices.

Checklist

Please, go through these checks before submitting the PR.

• You have not changed whitespace unnecessarily (it makes diffs hard to read)
• You have a descriptive commit message with a short title (first line, max 50 chars).
• Your code follows the style of the project (e.g. never omit braces in if statements)
• You have commented your code, particularly in hard-to-understand areas
• You have performed a self-review of your own code
• UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
• UI Changes: You have tested your change using the Google Accessibility Scanner

[welcome]

First PR! 🚀 We sincerely appreciate that you have taken the time to propose a change to AnkiDroid! Please have patience with us as we are all volunteers - we will get to this as soon as possible.

[krmanik]

This features looks good. Can you add how to test it in How Has This Been Tested? PR template? What should be added in front/back to use these features?

It will be added after merge of this PR to https://github.com/ankidroid/Anki-Android/wiki/AnkiDroid-Javascript-API

[mikunimaru]

This is a sample template.
You can try it with the template on the back of a regular card.

{{FrontSide}}

<hr id=answer>

<div class='jsTts'>
{{BackSide}}
</div>

<script> 
// Select the text for TTS.
const jsTts = document.querySelector(".jsTts"); 
const text = jsTts.textContent;

// Select TTS language
AnkiDroidJS.ankiTtsSetLanguage('en-US');
// Select the speed of TTS.
AnkiDroidJS.ankiTtsSetSpeechRate(1.5);

 // Speak
AnkiDroidJS.ankiTtsSpeak(text);

// Change TTS pitch and speed.
AnkiDroidJS.ankiTtsSetPitch(1.1)
AnkiDroidJS.ankiTtsSetSpeechRate(0.8);

// Speak
// If you add "1" to the second argument, it will wait until the previous Speak ends before speaking.
AnkiDroidJS.ankiTtsSpeak(text,1);

// Restore the pitch and speed of the TTS.
AnkiDroidJS.ankiTtsSetSpeechRate(1);
AnkiDroidJS.ankiTtsSetPitch(1)
</script>

[krmanik]

Looks good to me. Wait for review from maintainers.

[david-allison]

EDIT: This comment no longer applies

Hi, thanks for the PR, and sorry for the slow review.

JavaScript API changes are very delicate: if we make the wrong step, we'd be forced to break clients. We have three variables to worry about, and each of these can cause pain for the API clients if we don't handle them correctly:

• AnkiDroid version (under our control: defines the available APIs). Defined by both Android version (we do stop supporting old Androids) and user choice (users can downgrade and upgrade AnkiDroid).
• Deck API version (deck creators control this).
• Android version (user control, but typically immutable and not well informed WRT AnkiDroid).

This adds a dependency between Android version and Deck API version: there's a potential for breakage if we don't code this well, and a deck using an old API is used with a newer Android version.

I don't think we've introduced a dependency like this, and there'll be some discussion to ensure that we handle it with as much care for our downstream API clients as possible. "I upgraded my phone and AnkiDroid broke" is a terrible place to be in, especially if they depend on a deck which both uses JavaScript and isn't maintained

My main comment requires a second look from @mikehardy,

Most comments are quick nitpicks which can be solved quickly

[mikunimaru]

I tried to modify the code according to the polite review comments.

Regarding the practicality of the API, I have already imported it into my ankidroid and verified it, so I will give an example.
Realization of the function that tts is played again no matter where I tap on the screen. (It's convenient because I don't always have to move my finger near the answer button)
Realization of the behavior that the app speaks not only the cloz part but the entire line where the cloz existed when I opened the cloz.
(The content of the speech changes depending on whether cloz is on the first line or the second line)

In addition, since the introduction of the JS add-on to Ankidroid has been proposed, I feel that sooner or later an api that handles TTS on JS will be required.

[AbdelrahmanMahmoudMD]

Amazing!!! can't wait for this to added!! thank you all for your great work!

[mikehardy]

There was a tiny conflict from recent work in the codebase and the PR was not rebase-merge-able anyway, so I fixed the conflict to get a CI run in with current code

[mikehardy]

OK - tagged this up for review, and unrelated to anything @mikunimaru I was looking at a different issue (hermes vm sort not stable, related to a react-native repo I manage) and if I'm not mistaken I saw you in there! Small world :-)

[david-allison]

LGTM, thanks!

[mikehardy]

Let's do it! Thanks @mikunimaru

[krmanik]

@mikunimaru
Can you update the wiki page for this API?
https://github.com/ankidroid/Anki-Android/wiki/AnkiDroid-Javascript-API

[mikehardy]

Hi there!

Apologies for the delay, and if you're already submitted something to OpenCollective for PRs merged in August 2021, you may feel free to ignore this and it will be processed shortly

Just a friendly notice that we try to process OpenCollective payments monthly - it's time for August 2021 submissions

If you are interested in compensation for this work, the process with details is here:

https://github.com/ankidroid/Anki-Android/wiki/OpenCollective-Payment-Process#how-to-get-paid

(I only post one comment per person to avoid spamming you, regardless of the number of PRs merged, but this note applies to all PRs merged for in the month of August)

Thanks!

[MagTun]

I just want to say a big thank you to all of you for this feature! I subscribed to the beta to get it as soon as possible. For an unrelated problem, I had to install the alpha version AnkiDroid-2.16. Out of curiosity I tested the script given above and it worked! This is super giga top cool!

[mikehardy]

Yeah, @infinyte7 has been doing great work on the JS APIs in general and then @mikunimaru stepped in here with this 💎 - it is pretty cool that any of this is possible in my opinion

[AbdelrahmanMahmoudMD]

@mikunimaru thanks for you great work!!! Been waiting for this very long time , can you please simplify what to add in front and back card tempelate to get TTS work on ankidroid and ankidesktop ?

[MagTun]

@AbdelrahmanHamdy1998

The simplest TTS script is :

AnkiDroidJS.ankiTtsSetLanguage('en-US'); 
AnkiDroidJS.ankiTtsSpeak("some text here");

The "some text here" can be loaded from a variable that you retrieve from the HTML via regular javascript function like document.getElementById or document.getElementsByClassName or as in the example document.querySelector

You don't need to set the pitch and the Speech rate.

I don't know about ankidestop but I guess AnkiDroidJS would probably not work

[krmanik]

• To make it speak only cloze part then add this to back side of the card template.

<script>
var jsTts = document.querySelector(".cloze"); 
var text = jsTts.textContent;
AnkiDroidJS.ankiTtsSpeak(text,1);
</script>

• To make it speak cloze part and all text in the field containing it then add this to the front/back side of card template

<div id="text_id">{{cloze:Text}}</div>

<script>
var text = document.getElementById("text_id").textContent;
AnkiDroidJS.ankiTtsSpeak(text,1);
</script>

@mikunimaru
It will be nice to have the TTS API usage in https://github.com/ankidroid/Anki-Android/wiki/AnkiDroid-Javascript-API

[mikunimaru]

@infinyte7
Great sample code.

I'm not good at English, so the difficulty of creating a document is quite high.

// Select TTS language
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#setLanguage(java.util.Locale)
// @return  
// 0 Denotes the language is available for the language by the locale, but not the country and variant.
// 1 Denotes the language is available for the language and country specified by the locale, but not the variant.
// 2 Denotes the language is available exactly as specified by the locale.
// -1 Denotes the language data is missing.
// -2 Denotes the language is not supported.
AnkiDroidJS.ankiTtsSetLanguage('en-US');

// Speak
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#speak(java.lang.CharSequence,%20int,%20android.os.Bundle,%20java.lang.String)
// @return ERROR(-1) SUCCESS(0)
AnkiDroidJS.ankiTtsSpeak(text);
// If you add "1" to the second argument, it will wait until the previous Speak ends before speaking.
AnkiDroidJS.ankiTtsSpeak(text,1);

// Change TTS pitch and speed.
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#setPitch(float)
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#setSpeechRate(float)
// @return ERROR(-1) SUCCESS(0)
AnkiDroidJS.ankiTtsSetPitch(1.1);
AnkiDroidJS.ankiTtsSetSpeechRate(0.8);
// Int is also available
AnkiDroidJS.ankiTtsSetSpeechRate(1);
AnkiDroidJS.ankiTtsSetPitch(1)

// Whether the app is currently speaking (boolean)
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#isSpeaking()
let isSpeaking = AnkiDroidJS.ankiTtsIsSpeaking();

// Stop speech
// https://developer.android.com/reference/android/speech/tts/TextToSpeech#stop()
// @return ERROR(-1) SUCCESS(0)
AnkiDroidJS.ankiTtsStop();

The above is a brief description of all the APIs implemented this time.

Since the api has a one-to-one correspondence with the android tts api, I think it would be better to quote the android api page or paste a link for the exact operation explanation. For the time being, I have pasted the link to the corresponding android api in the above brief description.

[krmanik]

Thanks @mikunimaru
I will update the wiki page when I get time.

To make cloze work on AnkiDesktop and AnkiDroid

<div id="text_id">{{cloze:Text}}</div>

{{Extra}}

<div id="anki_tts">{{tts en_US voices=Apple_Samantha,Microsoft_Zira speed=1.0:cloze:Text}}</div>

<!-- play button -->
<a class="replaybutton" href="#" onclick="playAnkiDroidTts();"><span><svg viewBox="0 0 32 32"><polygon points="11,25 25,16 11,7"></polygon>Replay</svg></span></a>

<script>
 if (document.documentElement.classList.contains("android")) {
    document.getElementById("anki_tts").innerHTML = "";
    document.querySelector(".replaybutton").style.display = "unset";
    playAnkiDroidTts(); 
 } else {
    document.querySelector(".replaybutton").style.display = "none";
 }

function playAnkiDroidTts() {
    var text = document.getElementById("text_id").textContent;
    AnkiDroidJS.ankiTtsSpeak(text,1);
}
</script>

[MagTun]

@infinyte7 , didn't you miss the second } of {{tts en_US voices=Apple_Samantha,Microsoft_Zira speed=1.0:cloze:Text}?

[krmanik]

@infinyte7 , didn't you miss the second } of {{tts en_US voices=Apple_Samantha,Microsoft_Zira speed=1.0:cloze:Text}?

I have updated it. Thanks

[srghma]

Getting on Android with AnkiDroid 2.15.6 downloaded from Google Play

ankiTtsSetLanguage is not a function

[MagTun]

@srghma , You need to use AnkiDroidJS.ankiTtsSetLanguage(). You can see the functions in the wiki

[srghma]

@MagTun

[MagTun]

@srghma , this is working for me:

Which Anki version do you have? I think you still need to use the alpha version for it to work (I don't know if this TTS has already been implemented in the beta):
https://ankidroid.org/docs/manual.html#betaTesting (this link also explains how to install alpha)
https://github.com/ankidroid/Anki-Android/releases (if you want to do it manually)

[gowhari]

AnkiDroidJS.ankiTtsSetLanguage is undefined on version 2.15.6

ankiGetCardDid: ƒ ()
ankiGetCardDue: ƒ ()
ankiGetCardFactor: ƒ ()
ankiGetCardFlag: ƒ ()
ankiGetCardId: ƒ ()
ankiGetCardInterval: ƒ ()
ankiGetCardLapses: ƒ ()
ankiGetCardLeft: ƒ ()
ankiGetCardMark: ƒ ()
ankiGetCardMod: ƒ ()
ankiGetCardNid: ƒ ()
ankiGetCardODid: ƒ ()
ankiGetCardODue: ƒ ()
ankiGetCardQueue: ƒ ()
ankiGetCardReps: ƒ ()
ankiGetCardType: ƒ ()
ankiGetDeckName: ƒ ()
ankiGetETA: ƒ ()
ankiGetLrnCardCount: ƒ ()
ankiGetNewCardCount: ƒ ()
ankiGetNextTime1: ƒ ()
ankiGetNextTime2: ƒ ()
ankiGetNextTime3: ƒ ()
ankiGetNextTime4: ƒ ()
ankiGetRevCardCount: ƒ ()
ankiIsActiveNetworkMetered: ƒ ()
ankiIsDisplayingAnswer: ƒ ()
ankiIsInFullscreen: ƒ ()
ankiIsInNightMode: ƒ ()
ankiIsTopbarShown: ƒ ()
init: ƒ ()

var jsApi = {"version" : "0.0.1", "developer" : "[email protected]"}
apiStatus = AnkiDroidJS.init(JSON.stringify(jsApi))
'{"markCard":true,"toggleFlag":true}'

[MagTun]

@gowhari , you need to use the alpha version cf my last post

[gowhari]

@MagTun, Yes, Thanks it's available on 2.16alpha92

By the way I couldn't find HTML / Javascript Debugging to enable remote WebView connection in this new alpha version.
It was available on 2.15.6

[MagTun]

It's working without having to enable the debugging (you just have to enable the USB debugging in your phone settings).