Performance depends on options

[ArtyomZemlyak]

Tried different models with beam_size = 1.

beam_size 1

• 10 seconds audio

model	T, s	CPU/GPU
tiny	108	CPU
base	3	CPU
small	8	CPU
medium	58	CPU
large	126	CPU

• 200 seconds (7 diff audio files with diff quality)

model	T, s	CPU/GPU
base	320	CPU
small	310	CPU
tiny	43	GPU
base	29	GPU
small	19	GPU
medium	105	GPU
large	-	GPU

beam_size 5

• 10 seconds audio
• with TorchDynamo (Inductor)

model	T, s	CPU/GPU
tiny	11	CPU
base	7	CPU
small	22	CPU
medium	165	CPU
large	-	CPU

• 200 seconds (7 diff audio files with diff quality)

model	T, s	CPU/GPU
tiny	488	CPU
base	564	CPU
small	*3	CPU
medium	*20	CPU
large	*30	CPU
tiny	24	GPU
base	29	GPU
small	41	GPU
medium	89	GPU
large	-	GPU

[ArtyomZemlyak]

GreedyDecoder (beam_size None)

• 10 seconds audio

model	T, s	CPU/GPU
tiny	10	CPU
base	2	CPU
small	7	CPU
medium	55	CPU
large	119	CPU

• 200 seconds (7 diff audio files with diff quality)

model	T, s	CPU/GPU
base	122	CPU
small	260	CPU
tiny	27	GPU
base	20	GPU
small	14	GPU
medium	57	GPU
large	-	GPU

[kafan1986]

@ArtyomZemlyak Can you explain what does this T, s means? As per my understanding greedy decoding means beam size = 1 but you have mentioned it None.