Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

複数エンジン対応: すべてのエンジンからキャラクター情報を取得する #743

Closed
wants to merge 107 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
107 commits
Select commit Hold shift + click to select a range
a135eff
add runEngineAll
aoirint Mar 8, 2022
f81a774
fix log
aoirint Mar 8, 2022
22e0168
fix log
aoirint Mar 8, 2022
c5202dd
add killEngineAll
aoirint Mar 8, 2022
6e46ad9
fix log
aoirint Mar 8, 2022
31bfe9f
fix log
aoirint Mar 8, 2022
eb919bd
add restartEngineAll
aoirint Mar 8, 2022
39b81df
call killEngineAll
aoirint Mar 8, 2022
c8a7c22
fix log unit
aoirint Mar 8, 2022
9778831
npm run fmt
aoirint Mar 8, 2022
7d7b41c
add START_WAITING_ENGINE_ALL, RESTART_ENGINE_ALL
aoirint Mar 8, 2022
e0198fe
npm run fmt
aoirint Mar 8, 2022
5f33a07
DETECTED_ENGINE_ERROR engineKey
aoirint Mar 8, 2022
e23e9fb
npm run fmt
aoirint Mar 8, 2022
9d593e0
fix check all ready logic
aoirint Mar 8, 2022
4f0c2ee
fix engineState
aoirint Mar 8, 2022
c9a2272
fix test
aoirint Mar 8, 2022
23bdd06
type with undefined
aoirint Mar 8, 2022
8272997
fix allEngineState logic
aoirint Mar 8, 2022
63c5816
fix test
aoirint Mar 8, 2022
4754d9f
add comment
aoirint Mar 8, 2022
f3777c8
multiple engine: SET_CHARACTER_INFOS, USER_ORDERED_CHARACTER_INFOS
aoirint Mar 8, 2022
67591d6
multiple engine: LOAD_CHARACTER_ALL
aoirint Mar 8, 2022
a97a3c6
fix buildFileName logic
aoirint Mar 8, 2022
d0bad7b
fix default style id
aoirint Mar 8, 2022
9b3defa
fix characterInfos from vue
aoirint Mar 9, 2022
579f08b
pass engineInfos to vue
aoirint Mar 9, 2022
d963590
fix log
aoirint Mar 9, 2022
b1c6fda
fix changeStyleId
aoirint Mar 9, 2022
7333e5a
fix synthesis to use engineKey/Id
aoirint Mar 9, 2022
825a0fb
fix character portrait to use engineId
aoirint Mar 9, 2022
2f35f0a
fix audiocell icon to use engineId
aoirint Mar 9, 2022
b76e2b8
fix character voice testplay
aoirint Mar 9, 2022
f7c8746
wip fix voicesample
aoirint Mar 9, 2022
a02fd17
fix test
aoirint Mar 9, 2022
46c1228
use <0.12 for project migration
aoirint Mar 10, 2022
e21c23e
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 10, 2022
cc83cd9
use semver for migration
aoirint Mar 10, 2022
2102710
assume to be killed
aoirint Mar 10, 2022
c24c7e4
add memo comment
aoirint Mar 10, 2022
5c1cf7c
fix comment
aoirint Mar 10, 2022
086bd30
call onAllKilled after final process kill errored
aoirint Mar 10, 2022
9399dd1
fix log
aoirint Mar 10, 2022
db81c2b
fix log
aoirint Mar 10, 2022
26e4b2d
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Mar 11, 2022
b681ec4
fix infinite loop when using external engine
aoirint Mar 11, 2022
abe3453
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
5aa3eef
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
cb968e6
Merge branch 'patch-multiple-engine-process-container' into patch-mul…
aoirint Mar 11, 2022
334f77f
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 11, 2022
19aca50
add logs
aoirint Mar 13, 2022
d33a330
call IS_ENGINE_READY from IS_ALL_ENGINE_READY
aoirint Mar 13, 2022
858f650
fix comment
aoirint Mar 13, 2022
521252e
call IS_ENGINE_READY with 0th engineKey
aoirint Mar 13, 2022
88d9fc9
fixme to todo
aoirint Mar 13, 2022
a75322e
fix comment
aoirint Mar 13, 2022
f5204a7
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 13, 2022
a6f1922
assert engineKey, engineId, styleId is not undefined
aoirint Mar 13, 2022
110a762
fix comment
aoirint Mar 14, 2022
2c754de
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
da2e26d
remove unused import
aoirint Mar 14, 2022
6d30f34
Merge branch 'patch-multiple-engine-api-log' into patch-multiple-engi…
aoirint Mar 14, 2022
af19eae
fix CharacterOrderDialog
aoirint Mar 14, 2022
a719ae5
CharacterOrderDialog: change icon based on selected style
aoirint Mar 14, 2022
7175447
remove .value in template
aoirint Mar 14, 2022
7cac6e5
DefaultStyleSelectDialog: multiple engine (allow duplicated styleId)
aoirint Mar 14, 2022
8319276
LibraryPolicy: multiple engine
aoirint Mar 14, 2022
bf31b88
fix log
aoirint Mar 14, 2022
4453c8e
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
222792d
CharacterOrderDialog: allow duplication of styleId when speakerUuid i…
aoirint Mar 14, 2022
695ef6f
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 14, 2022
8261eb6
fix comment
aoirint Mar 14, 2022
a240b63
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 14, 2022
8c2b543
split icon change pr
aoirint Mar 14, 2022
ac6310b
fix comment
aoirint Mar 14, 2022
09332c1
state.engineInfos to Record<string, EngineInfo> from EngineInfo[]
aoirint Mar 24, 2022
5d0ee83
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 24, 2022
7031f3d
fix dictionary actions to use state.engineKeys instead of engineInfos
aoirint Mar 24, 2022
c37757c
npm run fmt
aoirint Mar 24, 2022
f7e2bdf
initialize engineStates with STARTING
aoirint Mar 24, 2022
a075309
npm run fmt
aoirint Mar 25, 2022
8c24a7e
GENERATE_AUDIO_ITEM: add engineKey undefined check
aoirint Mar 25, 2022
966f013
use strict equality for engineKey undefined check
aoirint Mar 25, 2022
f86a7d5
add comment to describe loop behavior
aoirint Mar 25, 2022
4e745d5
add STARTING assert test
aoirint Mar 25, 2022
b152e36
remove unneeded undefined check
aoirint Mar 26, 2022
1d15f02
Merge remote-tracking branch 'upstream/main' into patch-multiple-engi…
aoirint Mar 26, 2022
d4db483
Merge branch 'patch-multiple-engine-start-waiting-engine-all' into pa…
aoirint Mar 26, 2022
56063cb
DictionaryManageDialog: temporarily use first engineKey
aoirint Mar 26, 2022
51f19f1
Merge branch 'patch-multiple-engine-character-order-dialog' into patc…
aoirint Mar 26, 2022
571d980
flatMap engineInfos -> engineKeys
aoirint Mar 26, 2022
d610af5
loop engineInfos -> engineKeys
aoirint Mar 26, 2022
8d6e974
add characterInfos assert.isObject
aoirint Mar 26, 2022
d30dfbb
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 22, 2022
2fa8690
fix DictionaryManageDialog working for first engine
aoirint Jun 22, 2022
c0af3b5
Merge remote-tracking branch 'origin/patch-multiple-engine-api' into …
aoirint Jun 22, 2022
7e5e1e1
SETUP_ENGINE_SPEAKER: add engineKey (fix character init)
aoirint Jun 22, 2022
b7759cd
fix assertion compare
aoirint Jun 22, 2022
f137521
fix assert message
aoirint Jun 22, 2022
4afe467
project migration version 0.12 -> 0.13
aoirint Jun 22, 2022
0a34c0d
add comment
aoirint Jun 22, 2022
a0a812a
LOAD_USER_DICT: add engineKey
aoirint Jun 22, 2022
70c863c
getFlattenCharacterInfos func
aoirint Jun 22, 2022
2c4bed5
GENERATE_AUDIO_ITEM: (engineId, styleId) assertion
aoirint Jun 22, 2022
34fc2dc
fix message
aoirint Jun 22, 2022
7747878
add comment
aoirint Jun 26, 2022
bf28087
Merge remote-tracking branch 'origin/main' into patch-multiple-engine…
aoirint Jun 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
multiple engine: SET_CHARACTER_INFOS, USER_ORDERED_CHARACTER_INFOS
aoirint committed Mar 8, 2022
commit f3777c83cf5af5238b9c13ada7959a173f765378
170 changes: 95 additions & 75 deletions src/store/audio.ts
Original file line number Diff line number Diff line change
@@ -188,12 +188,22 @@ export const audioStore: VoiceVoxStoreOptions<
: undefined;
},
USER_ORDERED_CHARACTER_INFOS: (state) => {
const characterInfos = state.characterInfos?.slice();
return characterInfos?.sort(
(a, b) =>
state.userCharacterOrder.indexOf(a.metas.speakerUuid) -
state.userCharacterOrder.indexOf(b.metas.speakerUuid)
);
let characterInfoList: CharacterInfo[] = [];
for (const engineInfo of state.engineInfos) {
const engineCharacterInfos: CharacterInfo[] | undefined =
state.characterInfos[engineInfo.key];
if (engineCharacterInfos === undefined) continue;

characterInfoList = characterInfoList.concat(engineCharacterInfos);
}

return characterInfoList.length !== 0
? characterInfoList.sort(
(a, b) =>
state.userCharacterOrder.indexOf(a.metas.speakerUuid) -
state.userCharacterOrder.indexOf(b.metas.speakerUuid)
)
: undefined;
},
},

@@ -209,9 +219,12 @@ export const audioStore: VoiceVoxStoreOptions<
},
SET_CHARACTER_INFOS(
state,
{ characterInfos }: { characterInfos: CharacterInfo[] }
{
engineKey,
characterInfos,
}: { engineKey: string; characterInfos: CharacterInfo[] }
) {
state.characterInfos = characterInfos;
state.characterInfos[engineKey] = characterInfos;
},
SET_ACTIVE_AUDIO_KEY(state, { audioKey }: { audioKey?: string }) {
state._activeAudioKey = audioKey;
@@ -539,84 +552,91 @@ export const audioStore: VoiceVoxStoreOptions<
}
}
),
LOAD_CHARACTER: createUILockAction(async ({ state, commit, dispatch }) => {
const engineInfo = state.engineInfos[0]; // TODO: 複数エンジン対応
if (!engineInfo)
throw new Error(`No such engineInfo registered: index == 0`);

const speakers = await dispatch("INVOKE_ENGINE_CONNECTOR", {
engineKey: engineInfo.key,
action: "speakersSpeakersGet",
// 連想配列が第一引数になければ失敗する
payload: [{}],
})
.then(toDispatchResponse("speakersSpeakersGet"))
.catch((error) => {
window.electron.logError(error, `Failed to get speakers.`);
throw error;
});
const base64ToUrl = function (base64: string, type: string) {
const buffer = Buffer.from(base64, "base64");
const iconBlob = new Blob([buffer.buffer], { type: type });
return URL.createObjectURL(iconBlob);
};
const getStyles = function (speaker: Speaker, speakerInfo: SpeakerInfo) {
const styles: StyleInfo[] = new Array(speaker.styles.length);
speaker.styles.forEach((style, i) => {
const styleInfo = speakerInfo.styleInfos.find(
(styleInfo) => style.id === styleInfo.id
);
if (!styleInfo)
throw new Error(
`Not found the style id "${style.id}" of "${speaker.name}". `
);
const voiceSamples = styleInfo.voiceSamples.map((voiceSample) => {
return base64ToUrl(voiceSample, "audio/wav");
});
styles[i] = {
styleName: style.name,
styleId: style.id,
iconPath: base64ToUrl(styleInfo.icon, "image/png"),
voiceSamplePaths: voiceSamples,
};
});
return styles;
};
const getSpeakerInfo = async function (speaker: Speaker) {
const engineInfo = state.engineInfos[0]; // TODO: 複数エンジン対応
LOAD_CHARACTER: createUILockAction(
async ({ state, commit, dispatch }, { engineKey }) => {
const engineInfo = state.engineInfos.find(
(engineInfo) => engineInfo.key === engineKey
);
if (!engineInfo)
throw new Error(`No such engineInfo registered: index == 0`);

const speakerInfo = await dispatch("INVOKE_ENGINE_CONNECTOR", {
const speakers = await dispatch("INVOKE_ENGINE_CONNECTOR", {
engineKey: engineInfo.key,
action: "speakerInfoSpeakerInfoGet",
payload: [{ speakerUuid: speaker.speakerUuid }],
action: "speakersSpeakersGet",
// 連想配列が第一引数になければ失敗する
payload: [{}],
})
.then(toDispatchResponse("speakerInfoSpeakerInfoGet"))
.then(toDispatchResponse("speakersSpeakersGet"))
.catch((error) => {
window.electron.logError(error, `Failed to get speakers.`);
throw error;
});
const styles = getStyles(speaker, speakerInfo);
const characterInfo: CharacterInfo = {
portraitPath: base64ToUrl(speakerInfo.portrait, "image/png"),
metas: {
speakerUuid: speaker.speakerUuid,
speakerName: speaker.name,
styles: styles,
policy: speakerInfo.policy,
},
const base64ToUrl = function (base64: string, type: string) {
const buffer = Buffer.from(base64, "base64");
const iconBlob = new Blob([buffer.buffer], { type: type });
return URL.createObjectURL(iconBlob);
};
return characterInfo;
};
const characterInfos: CharacterInfo[] = await Promise.all(
speakers.map(async (speaker) => {
return await getSpeakerInfo(speaker);
})
);
const getStyles = function (
speaker: Speaker,
speakerInfo: SpeakerInfo
) {
const styles: StyleInfo[] = new Array(speaker.styles.length);
speaker.styles.forEach((style, i) => {
const styleInfo = speakerInfo.styleInfos.find(
(styleInfo) => style.id === styleInfo.id
);
if (!styleInfo)
throw new Error(
`Not found the style id "${style.id}" of "${speaker.name}". `
);
const voiceSamples = styleInfo.voiceSamples.map((voiceSample) => {
return base64ToUrl(voiceSample, "audio/wav");
});
styles[i] = {
styleName: style.name,
styleId: style.id,
iconPath: base64ToUrl(styleInfo.icon, "image/png"),
voiceSamplePaths: voiceSamples,
};
});
return styles;
};
const getSpeakerInfo = async function (speaker: Speaker) {
const engineInfo = state.engineInfos[0]; // TODO: 複数エンジン対応
if (!engineInfo)
throw new Error(`No such engineInfo registered: index == 0`);

commit("SET_CHARACTER_INFOS", { characterInfos });
}),
const speakerInfo = await dispatch("INVOKE_ENGINE_CONNECTOR", {
engineKey: engineInfo.key,
action: "speakerInfoSpeakerInfoGet",
payload: [{ speakerUuid: speaker.speakerUuid }],
})
.then(toDispatchResponse("speakerInfoSpeakerInfoGet"))
.catch((error) => {
window.electron.logError(error, `Failed to get speakers.`);
throw error;
});
const styles = getStyles(speaker, speakerInfo);
const characterInfo: CharacterInfo = {
portraitPath: base64ToUrl(speakerInfo.portrait, "image/png"),
metas: {
speakerUuid: speaker.speakerUuid,
speakerName: speaker.name,
styles: styles,
policy: speakerInfo.policy,
},
};
return characterInfo;
};
const characterInfos: CharacterInfo[] = await Promise.all(
speakers.map(async (speaker) => {
return await getSpeakerInfo(speaker);
})
);

commit("SET_CHARACTER_INFOS", { engineKey, characterInfos });
}
),
GENERATE_AUDIO_KEY() {
const audioKey = uuidv4();
audioElements[audioKey] = new Audio();
10 changes: 7 additions & 3 deletions src/store/type.ts
Original file line number Diff line number Diff line change
@@ -73,7 +73,7 @@ export type QuasarDialog = QVueGlobals["dialog"];

export type AudioStoreState = {
engineStates: Record<string, EngineState>;
characterInfos?: CharacterInfo[];
characterInfos: Record<string, CharacterInfo[]>;
audioItems: Record<string, AudioItem>;
audioKeys: string[];
audioStates: Record<string, AudioState>;
@@ -131,12 +131,16 @@ type AudioStoreTypes = {
mutation: { engineKey: string; engineState: EngineState };
};

LOAD_CHARACTER: {
LOAD_CHARACTER_ALL: {
action(): void;
};

LOAD_CHARACTER: {
action(payload: { engineKey: string }): void;
};

SET_CHARACTER_INFOS: {
mutation: { characterInfos: CharacterInfo[] };
mutation: { engineKey: string; characterInfos: CharacterInfo[] };
};

USER_ORDERED_CHARACTER_INFOS: {