feat: Add new Drain tokenizer that splits on most punctuation #3668
Job | Run time |
---|---|
3m 19s | |
3m 37s | |
10s | |
3m 30s | |
1m 14s | |
4m 31s | |
1m 14s | |
55s | |
2m 30s | |
3m 37s | |
1m 59s | |
1m 49s | |
2m 15s | |
2m 34s | |
1m 53s | |
1m 17s | |
1m 28s | |
2m 29s | |
1m 40s | |
1m 9s | |
1m 56s | |
41s | |
1m 28s | |
2m 18s | |
1m 15s | |
2m 1s | |
1m 26s | |
1m 20s | |
1m 55s | |
1m 17s | |
1m 9s | |
1m 59s | |
2m 3s | |
1m 48s | |
2m 44s | |
1m 13s | |
2m 10s | |
1m 55s | |
47s | |
2m 56s | |
1m 56s | |
2m 5s | |
1m 26s | |
45s | |
1m 56s | |
1m 24s | |
2m 0s | |
9s | |
1h 29m 12s |