Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move Tokenizer API to own crate #1766

Closed
PSeitz opened this issue Jan 8, 2023 · 0 comments · Fixed by #1767
Closed

Move Tokenizer API to own crate #1766

PSeitz opened this issue Jan 8, 2023 · 0 comments · Fixed by #1767

Comments

@PSeitz
Copy link
Contributor

PSeitz commented Jan 8, 2023

As of now with every tantivy version released, all tokenizers need also release a new version even if nothing changed. That causes unnecessary churn and incompatibilities.

Moving the tantivy tokenizer API to an own crate would solve that issue.
@PSeitz PSeitz mentioned this issue Jan 8, 2023
Merged
PSeitz added a commit that referenced this issue Jan 8, 2023
@PSeitz
move tokenizer API to seperate crate
fce7dae 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
PSeitz added a commit that referenced this issue Jan 8, 2023
@PSeitz
move tokenizer API to seperate crate
e5e8722 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
PSeitz added a commit that referenced this issue Jan 8, 2023
@PSeitz
move tokenizer API to seperate crate
c34a9e3 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
PSeitz added a commit that referenced this issue Jan 8, 2023
@PSeitz
move tokenizer API to seperate crate
8ab60c1 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
PSeitz added a commit that referenced this issue Jan 8, 2023
@PSeitz
move tokenizer API to seperate crate
d7525a8 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
PSeitz added a commit that referenced this issue Jan 9, 2023
@PSeitz
move tokenizer API to seperate crate (#1767)
514d23a 
closes #1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
Hodkinson pushed a commit to Hodkinson/tantivy that referenced this issue Jan 30, 2023
@PSeitz
move tokenizer API to seperate crate (quickwit-oss#1767)
5cb45f7 
closes quickwit-oss#1766

Finding tantivy tokenizers is a frustrating experience currently, since
they need be updated for each tantivy version. That's unnecessary since
the API is rather stable anyway.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

move tokenizer API to seperate crate quickwit-oss/tantivy
1 participant
@PSeitz