Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

@remotion/install-whisper-cpp: Add support for large-v3-turbo #4350

Merged
merged 1 commit into from
Oct 2, 2024
Merged

@remotion/install-whisper-cpp: Add support for large-v3-turbo #4350

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions packages/docs/docs/install-whisper-cpp/download-whisper-model.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
image: /generated/articles-docs-install-whisper-cpp-download-whisper-model.png
title: downloadWhisperModel()
crumb: "@remotion/install-whisper-cpp"
crumb: '@remotion/install-whisper-cpp'
---

# downloadWhisperModel()<AvailableFrom v="4.0.115"/>
Expand All @@ -12,12 +12,12 @@ You should first install Whisper.cpp, for example through [`installWhisperCpp()`
```tsx twoslash title="install-whisper.mjs"
// @module: esnext
// @target: es2022
import path from "path";
import { downloadWhisperModel } from "@remotion/install-whisper-cpp";
import path from 'path';
import {downloadWhisperModel} from '@remotion/install-whisper-cpp';

const { alreadyExisted } = await downloadWhisperModel({
model: "medium.en",
folder: path.join(process.cwd(), "whisper.cpp"),
const {alreadyExisted} = await downloadWhisperModel({
model: 'medium.en',
folder: path.join(process.cwd(), 'whisper.cpp'),
});
```

Expand All @@ -29,14 +29,14 @@ The folder to download the model to. The model will be downloaded into this fold

### `model`

The model to download. Possible values: `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large-v1`, `large-v2`, `large-v3`.
The model to download. Possible values: `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large-v1`, `large-v2`, `large-v3`, `large-v3-turbo`.

### `onProgress?`

Act upon download progress. This is the function signature:

```tsx twoslash
import type { OnProgress } from "@remotion/install-whisper-cpp";
import type {OnProgress} from '@remotion/install-whisper-cpp';

const onProgress: OnProgress = (
downloadedBytes: number,
Expand Down
18 changes: 9 additions & 9 deletions packages/docs/docs/install-whisper-cpp/transcribe.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
image: /generated/articles-docs-install-whisper-cpp-transcribe.png
title: transcribe()
crumb: "@remotion/install-whisper-cpp"
crumb: '@remotion/install-whisper-cpp'
---

# transcribe()<AvailableFrom v="4.0.131 "/>
Expand All @@ -16,13 +16,13 @@ This function only works with Whisper.cpp 1.5.5 or later, unless `tokenLevelTime
```tsx twoslash title="transcribe.mjs"
// @module: esnext
// @target: es2022
import path from "path";
import { transcribe } from "@remotion/install-whisper-cpp";
import path from 'path';
import {transcribe} from '@remotion/install-whisper-cpp';

const { transcription } = await transcribe({
inputPath: "/path/to/audio.wav",
whisperPath: path.join(process.cwd(), "whisper.cpp"),
model: "medium.en",
const {transcription} = await transcribe({
inputPath: '/path/to/audio.wav',
whisperPath: path.join(process.cwd(), 'whisper.cpp'),
model: 'medium.en',
tokenLevelTimestamps: true,
});

Expand Down Expand Up @@ -67,7 +67,7 @@ _default: `base.en`_

Specify a specific Whisper model for the transcription.

Possible values: `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large-v1`, `large-v2`, `large-v3`.
Possible values: `tiny`, `tiny.en`, `base`, `base.en`, `small`, `small.en`, `medium`, `medium.en`, `large-v1`, `large-v2`, `large-v3`, `large-v3-turbo`.

Make sure the model you want to use exists in your `whisper.cpp/models` folder. You can ensure a specific model is available locally by utilizing the [downloadWhisperModel()](/docs/install-whisper-cpp/download-whisper-model) API.

Expand Down Expand Up @@ -129,7 +129,7 @@ Listen for progress updates from the transcription process.
The progress is a number between `0` and `1`.

```tsx twoslash
import type { TranscribeOnProgress } from "@remotion/install-whisper-cpp";
import type {TranscribeOnProgress} from '@remotion/install-whisper-cpp';

const onProgress: TranscribeOnProgress = (progress) => {
console.log(`Transcription progress: ${progress * 100}%`);
Expand Down
2 changes: 2 additions & 0 deletions packages/install-whisper-cpp/src/download-whisper-model.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ const models = [
'large-v1',
'large-v2',
'large-v3',
'large-v3-turbo',
] as const;

const modelSizes: {[key in WhisperModel]: number} = {
Expand All @@ -22,6 +23,7 @@ const modelSizes: {[key in WhisperModel]: number} = {
'large-v1': 3094623691,
'large-v2': 3094623691,
'large-v3': 3095033483,
'large-v3-turbo': 1624555275,
small: 487601967,
tiny: 77691713,
'small.en': 487614201,
Expand Down
2 changes: 1 addition & 1 deletion packages/install-whisper-cpp/src/transcribe.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ export type TranscribeOnProgress = (progress: number) => void;
// https://github.com/ggerganov/whisper.cpp/blob/fe36c909715e6751277ddb020e7892c7670b61d4/examples/main/main.cpp#L989-L999
// https://github.com/remotion-dev/remotion/issues/4168
export const modelToDtw = (model: WhisperModel): string => {
if (model === 'large-v3') {
if (model === 'large-v3' || model === 'large-v3-turbo') {
return 'large.v3';
}

Expand Down
Loading