Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Align TransformersReader with FARMReader #319

Merged
merged 3 commits into from
Aug 18, 2020
Merged

Align TransformersReader with FARMReader #319

merged 3 commits into from
Aug 18, 2020

Conversation

bogdankostic
Copy link
Contributor

@bogdankostic bogdankostic commented Aug 17, 2020 •
edited by Timoeller
Loading

The TransformersReader is adapted such that there is no need to provide a tokenizer as it can be inferred by the model name.
Also, max_seq_len and doc_stride can now be set by the user.

@bogdankostic bogdankostic requested a review from Timoeller August 17, 2020 15:07
@bogdankostic
Copy link
Contributor Author

@brandenchan
I adapted the TransformersReader such that there is no need to provide a tokenizer as it can be inferred by the model name. Also, max_seq_len and doc_stride can now be set by the user.

Timoeller
Timoeller approved these changes Aug 18, 2020
View reviewed changes
Copy link
Contributor

@Timoeller Timoeller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good.

The context window calculation needs to change in FARM, so we can use it for Transformers as well.

@Timoeller Timoeller merged commit 3a95fe2 into master Aug 18, 2020
@julian-risch julian-risch deleted the align_transformers_reader branch November 15, 2021 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Timoeller Timoeller Timoeller approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@bogdankostic @Timoeller