Skip to content
Benchmark

grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) #412

Sign in to view logs

grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses)

grammars: 1.5x faster inference w/ complex grammars (vector reserves / reuses) #412

Run time

Learn about OS pricing on GitHub Actions
Job Run time
bench-server-baseline (phi-2, q4_0)
13m 38s
bench-server-baseline (phi-2, q8_0)
1s
bench-server-baseline (phi-2, f16)
5m 4s
18m 43s