Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

複数エンジン対応: すべてのエンジンからキャラクター情報を取得する #743

Closed
wants to merge 107 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
a135eff
add runEngineAll
aoirint Mar 8, 2022
f81a774
fix log
aoirint Mar 8, 2022
22e0168
fix log
aoirint Mar 8, 2022
c5202dd
add killEngineAll
aoirint Mar 8, 2022
6e46ad9
fix log
aoirint Mar 8, 2022
31bfe9f
fix log
aoirint Mar 8, 2022
eb919bd
add restartEngineAll
aoirint Mar 8, 2022
39b81df
call killEngineAll
aoirint Mar 8, 2022
c8a7c22
fix log unit
aoirint Mar 8, 2022
9778831
npm run fmt
aoirint Mar 8, 2022
7d7b41c
add START_WAITING_ENGINE_ALL, RESTART_ENGINE_ALL
aoirint Mar 8, 2022
e0198fe
npm run fmt
aoirint Mar 8, 2022
5f33a07
DETECTED_ENGINE_ERROR engineKey
aoirint Mar 8, 2022
e23e9fb
npm run fmt
aoirint Mar 8, 2022
9d593e0
fix check all ready logic
aoirint Mar 8, 2022
4f0c2ee
fix engineState
aoirint Mar 8, 2022
c9a2272
fix test
aoirint Mar 8, 2022
23bdd06
type with undefined
aoirint Mar 8, 2022
8272997
fix allEngineState logic
aoirint Mar 8, 2022
63c5816
fix test
aoirint Mar 8, 2022
4754d9f
add comment
aoirint Mar 8, 2022
f3777c8
multiple engine: SET_CHARACTER_INFOS, USER_ORDERED_CHARACTER_INFOS
aoirint Mar 8, 2022
67591d6
multiple engine: LOAD_CHARACTER_ALL
aoirint Mar 8, 2022
a97a3c6
fix buildFileName logic
aoirint Mar 8, 2022
d0bad7b
fix default style id
aoirint Mar 8, 2022
9b3defa
fix characterInfos from vue
aoirint Mar 9, 2022
579f08b
pass engineInfos to vue
aoirint Mar 9, 2022
d963590
fix log
aoirint Mar 9, 2022
b1c6fda
fix changeStyleId
aoirint Mar 9, 2022
7333e5a
fix synthesis to use engineKey/Id
aoirint Mar 9, 2022
825a0fb
fix character portrait to use engineId
aoirint Mar 9, 2022
2f35f0a
fix audiocell icon to use engineId
aoirint Mar 9, 2022
b76e2b8
fix character voice testplay
aoirint Mar 9, 2022
f7c8746
wip fix voicesample
aoirint Mar 9, 2022
a02fd17
fix test
aoirint Mar 9, 2022
46c1228
use <0.12 for project migration
aoirint Mar 10, 2022
e21c23e
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 10, 2022
cc83cd9
use semver for migration
aoirint Mar 10, 2022
2102710
assume to be killed
aoirint Mar 10, 2022
c24c7e4
add memo comment
aoirint Mar 10, 2022
5c1cf7c
fix comment
aoirint Mar 10, 2022
086bd30
call onAllKilled after final process kill errored
aoirint Mar 10, 2022
9399dd1
fix log
aoirint Mar 10, 2022
db81c2b
fix log
aoirint Mar 10, 2022
26e4b2d
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 11, 2022
b681ec4
fix infinite loop when using external engine
aoirint Mar 11, 2022
abe3453
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
5aa3eef
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
cb968e6
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
334f77f
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
19aca50
add logs
aoirint Mar 13, 2022
d33a330
call IS_ENGINE_READY from IS_ALL_ENGINE_READY
aoirint Mar 13, 2022
858f650
fix comment
aoirint Mar 13, 2022
521252e
call IS_ENGINE_READY with 0th engineKey
aoirint Mar 13, 2022
88d9fc9
fixme to todo
aoirint Mar 13, 2022
a75322e
fix comment
aoirint Mar 13, 2022
f5204a7
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 13, 2022
a6f1922
assert engineKey, engineId, styleId is not undefined
aoirint Mar 13, 2022
110a762
fix comment
aoirint Mar 14, 2022
2c754de
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
da2e26d
remove unused import
aoirint Mar 14, 2022
6d30f34
Merge branch 'patch-multiple-engine-api-log' into patch-multiple-engi…
aoirint Mar 14, 2022
af19eae
fix CharacterOrderDialog
aoirint Mar 14, 2022
a719ae5
CharacterOrderDialog: change icon based on selected style
aoirint Mar 14, 2022
7175447
remove .value in template
aoirint Mar 14, 2022
7cac6e5
DefaultStyleSelectDialog: multiple engine (allow duplicated styleId)
aoirint Mar 14, 2022
8319276
LibraryPolicy: multiple engine
aoirint Mar 14, 2022
bf31b88
fix log
aoirint Mar 14, 2022
4453c8e
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
222792d
CharacterOrderDialog: allow duplication of styleId when speakerUuid i…
aoirint Mar 14, 2022
695ef6f
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 14, 2022
8261eb6
fix comment
aoirint Mar 14, 2022
a240b63
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
8c2b543
split icon change pr
aoirint Mar 14, 2022
ac6310b
fix comment
aoirint Mar 14, 2022
09332c1
state.engineInfos to Record<string, EngineInfo> from EngineInfo[]
aoirint Mar 24, 2022
5d0ee83
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 24, 2022
7031f3d
fix dictionary actions to use state.engineKeys instead of engineInfos
aoirint Mar 24, 2022
c37757c
npm run fmt
aoirint Mar 24, 2022
f7e2bdf
initialize engineStates with STARTING
aoirint Mar 24, 2022
a075309
npm run fmt
aoirint Mar 25, 2022
8c24a7e
GENERATE_AUDIO_ITEM: add engineKey undefined check
aoirint Mar 25, 2022
966f013
use strict equality for engineKey undefined check
aoirint Mar 25, 2022
f86a7d5
add comment to describe loop behavior
aoirint Mar 25, 2022
4e745d5
add STARTING assert test
aoirint Mar 25, 2022
b152e36
remove unneeded undefined check
aoirint Mar 26, 2022
1d15f02
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 26, 2022
d4db483
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 26, 2022
56063cb
DictionaryManageDialog: temporarily use first engineKey
aoirint Mar 26, 2022
51f19f1
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 26, 2022
571d980
flatMap engineInfos -> engineKeys
aoirint Mar 26, 2022
d610af5
loop engineInfos -> engineKeys
aoirint Mar 26, 2022
8d6e974
add characterInfos assert.isObject
aoirint Mar 26, 2022
d30dfbb
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 22, 2022
2fa8690
fix DictionaryManageDialog working for first engine
aoirint Jun 22, 2022
c0af3b5
Merge remote-tracking branch 'origin/patch-multiple-engine-api' into …
aoirint Jun 22, 2022
7e5e1e1
SETUP_ENGINE_SPEAKER: add engineKey (fix character init)
aoirint Jun 22, 2022
b7759cd
fix assertion compare
aoirint Jun 22, 2022
f137521
fix assert message
aoirint Jun 22, 2022
4afe467
project migration version 0.12 -> 0.13
aoirint Jun 22, 2022
0a34c0d
add comment
aoirint Jun 22, 2022
a0a812a
LOAD_USER_DICT: add engineKey
aoirint Jun 22, 2022
70c863c
getFlattenCharacterInfos func
aoirint Jun 22, 2022
2c4bed5
GENERATE_AUDIO_ITEM: (engineId, styleId) assertion
aoirint Jun 22, 2022
34fc2dc
fix message
aoirint Jun 22, 2022
7747878
add comment
aoirint Jun 26, 2022
bf28087
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 34 additions & 7 deletions src/components/AudioCell.vue
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
"
@click="
changeStyleId(
characterInfo.metas.speakerUuid,
getDefaultStyle(characterInfo.metas.speakerUuid).styleId
)
"
Expand Down Expand Up @@ -89,7 +90,12 @@
v-close-popup
active-class="selected-character-item"
:active="style.styleId === selectedStyle.styleId"
@click="changeStyleId(style.styleId)"
@click="
changeStyleId(
characterInfo.metas.speakerUuid,
style.styleId
)
"
>
<q-avatar rounded size="2rem" class="q-mr-md">
<q-img
Expand Down Expand Up @@ -160,6 +166,7 @@ import { computed, watch, defineComponent, ref } from "vue";
import { useStore } from "@/store";
import { AudioItem } from "@/store/type";
import { QInput, debounce } from "quasar";
import { getEngineIdByEngineKey } from "@/store/audio";

export default defineComponent({
name: "AudioCell",
Expand Down Expand Up @@ -187,11 +194,11 @@ export default defineComponent({

const selectedCharacterInfo = computed(() =>
userOrderedCharacterInfos.value !== undefined &&
audioItem.value.engineId !== undefined &&
audioItem.value.styleId !== undefined
? userOrderedCharacterInfos.value.find((info) =>
info.metas.styles.find(
(style) => style.styleId === audioItem.value.styleId
)
? store.getters.CHARACTER_INFO(
audioItem.value.engineId,
audioItem.value.styleId
)
: undefined
);
Expand Down Expand Up @@ -245,9 +252,22 @@ export default defineComponent({
}
};

const changeStyleId = (styleId: number) => {
const changeStyleId = (speakerUuid: string, styleId: number) => {
const engineKey = store.state.engineKeys.find((_engineKey) =>
(store.state.characterInfos[_engineKey] ?? []).some(
(characterInfo) => characterInfo.metas.speakerUuid === speakerUuid
)
);
if (engineKey === undefined)
throw new Error(
`No engineKey for target character style (speakerUuid == ${speakerUuid}, styleId == ${styleId})`
);

const engineId = getEngineIdByEngineKey(store.state, engineKey);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

たしかにengineKeyとengineIdの使い分けがなかなか難しそうな気がしました!
ちょっと何度も聞いてしまっていて申し訳ないのですが、どういう使い分けが想定されていたんでしたっけ・・・🙇

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

engineIdは、エンジンの種類(VOICEVOX ENGINEやCOEIROINK ENGINE)に対して、エンジンの提供者が割り振るIDです。バージョンが変わっても同じIDを使うことを想定しています。

engineKeyは、エディタに登録されたエンジンの登録項目(ユーザ登録エンジン1、ユーザ登録エンジン2のような)に対して、エディタが自動的に割り振るIDです。
例えば、以下を区別するためのものです。

  • ローカルで実行されているVOICEVOX ENGINE 0.12.3
  • ローカルで実行されているVOICEVOX ENGINE 0.7.5
  • VPSで実行されているVOICEVOX ENGINE 0.7.5

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ただ製品版VOICEVOX(エディタとエンジンが同梱配布)では、エディタ初回実行時にengineKeyを決定するような仕組みを入れるのは複雑かなと思ったので、配布時に既に割り振られている(.env.productionに記述されている)形を想定しています(DEFAULT_ENGINE_INFOSでは決め打ちの値ということ)。

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VOICEVOX 0.12.3では、キャラクター+スタイルの情報は、AudioCellがstyleIdとして持っています。
AudioCellの内容はプロジェクトファイルに保存されます。
複数エンジン対応後、キャラクター+スタイルを一意に識別するには、少なくとも(engineId, styleId)または(speakerUuid, styleId)が必要です(エンジンバージョンによる違いは無視)。
#609 (comment) で(engineId, styleId)がよさそうということだったと思うので、このPRではAudioCell(プロジェクトファイル)が(engineId, styleId)をもつように変更しています。

Copy link
Member Author

@aoirint aoirint Jun 26, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

engineKeyからengineIdを引く処理、またはengineIdからengineKeyを引く処理が必要になっていて、この変換が挟まること、どの関数でどちらを受け取るべきなのかを決めること、が複雑(わかりにくい)と思っています。

このPRでは、実装を簡単にするためにengineKey == engineIdを想定していて、このロジックが分散しないようにgetEngineIdByEngineKeygetDefaultEngineKeyByEngineIdで共通化しています(インタフェースは暫定的なものです)。

例えば、エンジン設定UIでengineIdに対するデフォルトのengineKeyを設定できるようにして、それをgetDefaultEngineKeyByEngineIdが返すことで、シンプルなエンジン切り替え(バージョン切り替え)の仕組みが作れると思っています(いろいろな初期化が必要そうなので再起動を促すと楽そう)。

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ありがとうございます!!非常によくわかりました!!!

AudioCellでengineIdを使うのは、現行と同じ挙動になって、少なくとも使い勝手が悪くなることは無く、かつ実装もそこそこシンプルなので、やっぱり良い方法だと感じました。

将来的にはAudioCell(よいうよりAudioInfo)にengineKeyも保存する形を目指す感じになる感じでしょうか。
(過去バージョンのエンジンと今のエンジンを統一的に使えるように)

engineIdとengineKeyを分けたいのは賛成なのですが、今は同一に扱っているため、潜在的なバグに気づきにくく、かつ実装方針もなかなか定まりにくいのかなと感じています。
もうAudioInfoにengineKeyも持たせちゃうのはどうでしょう 👀

getDefaultEngineKeyByEngineIdgetEngineIdByEngineKeyを用意するのは賛成です!
でももしかしたらgetDefaultEngineKeyByEngineIdgetEngineIdByEngineKeyを呼ぶべき機会はとても少なくて、
前者は「AudioInfoにあるengineKeyがengineInfosにないので仕方なくデフォルトのを使うとき」のみ、
後者は「AudioInfoに登録するためにengineIdを得るとき」のみにできるのかなと思っています。

なんか見落としや気づいてない点がありそうなので、ご指摘頂けると。。。🙇

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

もしくは、engineId周りを一旦コードから外しちゃうというのはどうでしょう 👀

一旦外すことを提案している理由はもう2つあります。
理由の1つは、上述のような、engineIdをどう扱うか・どう実装するかの議論が発生してしまうためです。
結構大きな変更コード量なので、差分を明確にしたいという気持ちがあります。

もう1つの理由は、このプルリクエストが、エディタ側の複数エンジン対応に必須な最後のコードだからです。
この最後のキャラクター統合さえ完了すれば、他のコードを一気に組み込んで行けるため、ぜひ早く取り込みたいという気持ちがあります・・・!!!

もしよければ・・・!!


store.dispatch("COMMAND_CHANGE_STYLE_ID", {
audioKey: props.audioKey,
engineId,
styleId,
});
};
Expand Down Expand Up @@ -305,10 +325,17 @@ export default defineComponent({
await pushAudioText();
}

const engineId = audioItem.value.engineId;
if (engineId === undefined)
throw new Error("assert engineId !== undefined");

const styleId = audioItem.value.styleId;
if (styleId == undefined) throw new Error("styleId == undefined");
if (styleId === undefined)
throw new Error("assert styleId !== undefined");

const audioKeys = await store.dispatch("COMMAND_PUT_TEXTS", {
texts,
engineId,
styleId,
prevAudioKey,
});
Expand Down
11 changes: 5 additions & 6 deletions src/components/CharacterPortrait.vue
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,17 @@ export default defineComponent({
const store = useStore();

const characterInfo = computed(() => {
const characterInfos = store.state.characterInfos || [];
const activeAudioKey: string | undefined = store.getters.ACTIVE_AUDIO_KEY;
const audioItem = activeAudioKey
? store.state.audioItems[activeAudioKey]
: undefined;

const engineId = audioItem?.engineId;
const styleId = audioItem?.styleId;

return styleId !== undefined
? characterInfos.find((info) =>
info.metas.styles.find((style) => style.styleId === styleId)
)
: undefined;
if (engineId === undefined || styleId === undefined) return undefined;

return store.getters.CHARACTER_INFO(engineId, styleId);
});

const characterName = computed(() => {
Expand Down
28 changes: 27 additions & 1 deletion src/components/DictionaryManageDialog.vue
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ import {
import AudioAccent from "@/components/AudioAccent.vue";
import { QInput, useQuasar } from "quasar";
import { AudioItem } from "@/store/type";
import { getEngineIdByEngineKey } from "@/store/audio";

export default defineComponent({
name: "DictionaryManageDialog",
Expand All @@ -237,6 +238,8 @@ export default defineComponent({
const store = useStore();
const $q = useQuasar();

const engineKeyComputed = computed(() => store.state.engineKeys[0]); // TODO: 複数エンジン対応

const dictionaryManageDialogOpenedComputed = computed({
get: () => props.modelValue,
set: (val) => emit("update:modelValue", val),
Expand All @@ -256,10 +259,16 @@ export default defineComponent({
};

const loadingDictProcess = async () => {
const engineKey = engineKeyComputed.value;
if (engineKey === undefined)
throw new Error(`assert engineKey !== undefined`);

loadingDict.value = true;
try {
userDict.value = await createUILockAction(
store.dispatch("LOAD_USER_DICT")
store.dispatch("LOAD_USER_DICT", {
engineKey,
})
);
} catch {
$q.dialog({
Expand Down Expand Up @@ -330,6 +339,10 @@ export default defineComponent({
surface.value = convertHankakuToZenkaku(text);
};
const setYomi = async (text: string, changeWord?: boolean) => {
const engineKey = engineKeyComputed.value;
if (engineKey === undefined)
throw new Error(`assert engineKey !== undefined`);

// テキスト長が0の時にエラー表示にならないように、テキスト長を考慮する
isOnlyHiraOrKana.value = !text.length || kanaRegex.test(text);
// 読みが変更されていない場合は、アクセントフレーズに変更を加えない
Expand All @@ -352,6 +365,7 @@ export default defineComponent({
await createUILockAction(
store.dispatch("FETCH_ACCENT_PHRASES", {
text: text + "ガ'",
engineKey,
styleId: styleId.value,
isKana: true,
})
Expand All @@ -370,12 +384,17 @@ export default defineComponent({
};

const changeAccent = async (_: number, accent: number) => {
const engineKey = engineKeyComputed.value;
if (engineKey === undefined)
throw new Error(`assert engineKey !== undefined`);

if (accentPhrase.value) {
accentPhrase.value.accent = accent;
accentPhrase.value = (
await createUILockAction(
store.dispatch("FETCH_MORA_DATA", {
accentPhrases: [accentPhrase.value],
engineKey,
styleId: styleId.value,
})
)
Expand All @@ -387,6 +406,12 @@ export default defineComponent({
audioElem.pause();

const play = async () => {
const engineKey = engineKeyComputed.value;
if (engineKey === undefined)
throw new Error(`assert engineKey !== undefined`);

const engineId = getEngineIdByEngineKey(store.state, engineKey);

if (!accentPhrase.value) return;
nowGenerating.value = true;
const query: AudioQuery = {
Expand All @@ -403,6 +428,7 @@ export default defineComponent({

const audioItem: AudioItem = {
text: yomi.value,
engineId,
styleId: styleId.value,
query,
};
Expand Down
7 changes: 5 additions & 2 deletions src/components/LibraryPolicy.vue
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,16 @@
import { useStore } from "@/store";
import { computed, defineComponent, ref } from "@vue/runtime-core";
import { useMarkdownIt } from "@/plugins/markdownItPlugin";
import { getFlattenCharacterInfos } from "@/store/audio";

export default defineComponent({
setup() {
const store = useStore();
const md = useMarkdownIt();

const characterInfos = computed(() => store.state.characterInfos);
const flattenCharacterInfos = computed(() =>
getFlattenCharacterInfos(store.state)
);

const convertMarkdown = (text: string) => {
return md.render(text);
Expand All @@ -62,7 +65,7 @@ export default defineComponent({
};

return {
characterInfos,
characterInfos: flattenCharacterInfos,
convertMarkdown,
selectCharacterInfIndex,
detailIndex,
Expand Down
Loading