Tweaks to new citation lookup tool

[mlissner]

First, I just tried clicking here:

https://www.courtlistener.com/api/rest/v3/citation-lookup/

And got a crash: COURTLISTENER-6ZE

[mlissner]

Second, looks like we didn't do any auth. It should use the same auth as other areas of the API at least at first, though we may want to tighten it down the road, if performance is an issue.

[sentry-io]

I tossed a really big blob of text at it and got this:

Sentry Issue: COURTLISTENER-6ZF

[ERosendo]

I tossed a really big blob of text at it and got this:

Sentry Issue: COURTLISTENER-6ZF

@mlissner It seems like Eyecite was able to identify the reporter and page number in a citation but could not extract the volume. The citation object is:

FullCaseCitation('Thompson, 394', groups={'volume': None, 'reporter': 'Thompson', 'page': '394'}, metadata=FullCaseCitation.Metadata(parenthetical=None, pin_cite=None, year='1969', court=None, plaintiff='Shapiro', defendant='', extra='U. S. 618, 634'))

should we ignore this citation in the response? or should we include it with an error message?

[mlissner]

Interesting. This fails too:

s2 = 'Shapiro v. Thompson, 394 U. S. 618'
r = requests.post('https://www.courtlistener.com/api/rest/v3/citation-lookup/', data={"text":s2 })
r.status_code
500

But the front end looks fine:

[ERosendo]

I tried "Shapiro v. Thompson, 394 U. S. 618" in the web citation tool and it's also failing. 🤔

[mlissner]

APIs, man, they never fail to find problems in other places! Good news, we get to fix both!

[ERosendo]

COURTLISTENER-6ZE is fixed:

[mlissner]

Let's not forget here that we also need to add auth to this endpoint before anybody can start using it. Want to suggest something, @ERosendo?

[ERosendo]

we also need to add auth to this endpoint before anybody can start using it.

@mlissner You're right. We can add auth to this endpoint using two approaches:

Option 1: Simple Authentication

The IsAuthenticated class provides a simple way to restrict access to this endpoint. Users only need to include the authentication token found in their profile's developer tab within their request. This ensures only registered users can access the tool.

Option 2: Granular Permissions

We use a custom class inheriting from DjangoModelPermissions. This approach offers finer control over who can access the endpoint because it allows us to define specific user permissions that grant access. However, users would need to request API access beforehand, and permissions are assigned manually through the admin interface. We're using this approach for some endpoints like the one to retrieve/list docket entries(the RECAPUsersReadOnly class uses the perms_map property to define the custom model permissions)

Let me know what you think

[mlissner]

Simple auth seems fine. What about throttling?

[ERosendo]

What about throttling?

Considering that users can send large payloads, I believe that the ratelimiter_all_2_per_m limiter is appropriate. This would allow users to send up to 2 request per minute. Does this seem reasonable?

[mlissner]

I was thinking more about a throttle_class: https://www.django-rest-framework.org/api-guide/throttling/.

We'll need more than 2/m though. Maybe what we really need are citations/s. I don't care how much empty text you send at me. I care how many citations I have to look up. It looks like making a custom throttle to do that should be pretty easy using DRF, if you look at the SimpleRateThrottle's thottle_success method.

But assuming we can pull that off, how many do we want people to be able to look up per second? I'm not sure. I think these queries are very well optimized. Maybe 1/s?