Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/ DBLP import - support importing >1000 papers #2175

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

xkopenreview
Copy link
Collaborator

currently there's a limit on dblp import to import at most 1000 papers

it looks like there's not really a reason to put his limit now (through max replies) with the changes implemented in the past
and it's causing issue with api performance

this pr should remove the 1000 import limit

@emily-grabowski
Copy link
Contributor

When I tested this I was able to import 1000 publications, then I got the message 'Import quota reached':
Screenshot 2024-11-15 at 1 18 59 PM

@xkopenreview
Copy link
Collaborator Author

When I tested this I was able to import 1000 publications, then I got the message 'Import quota reached': Screenshot 2024-11-15 at 1 18 59 PM

you can remove the max reply of dblp record invitation
that's how Carlos fixed the slow query caused by dblp import

Copy link
Member

@carlosmondra carlosmondra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected.
Be sure to remove the 1000 from maxReplies in http://localhost:3030/invitation/edit?id=DBLP.org/-/Record before testing

@enrubio
Copy link
Member

enrubio commented Jan 3, 2025

Does this mean allowing you to import 1000+ papers at once? When I try to test this it says 500 publications is the max allowed.

@xkopenreview
Copy link
Collaborator Author

@carlosmondra is it possible to remove the rate limiting when

Does this mean allowing you to import 1000+ papers at once? When I try to test this it says 500 publications is the max allowed.

no.
because the user will probably encounter error with rate limiting

@carlosmondra
Copy link
Member

@carlosmondra is it possible to remove the rate limiting when

Does this mean allowing you to import 1000+ papers at once? When I try to test this it says 500 publications is the max allowed.

no. because the user will probably encounter error with rate limiting

Is this the rate limit when POSTing note edits?

@xkopenreview
Copy link
Collaborator Author

@carlosmondra is it possible to remove the rate limiting when

Does this mean allowing you to import 1000+ papers at once? When I try to test this it says 500 publications is the max allowed.

no. because the user will probably encounter error with rate limiting

Is this the rate limit when POSTing note edits?

yes

@xkopenreview
Copy link
Collaborator Author

does not require change for next.js upgrade

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants