fix Pleroma search API incompatibility with empty limit param

[snarfed]

Pleroma supports the Mastodon API, so a number of people have already signed their Pleroma accounts up for Bridgy via the Mastodon support. And it works!

...at least, mostly. Publish seems pretty functional, and listen (aka backfeed) has worked before, but right now it's hitting the API incompatibility below. Evidently Mastodon's /api/v2/search endpoint handles a blank value for the limit query param ok, but Pleroma doesn't. cc @jk-na etc.

I 2020-09-27 22:40:01.151421 requests.get https://toot.icyphox.sh/api/v2/search {'params': {'q': 'icyphox.sh', 'resolve': True, 'offset': 0, 'limit': ''}, 'headers': {'Authorization': '...'}}
I 2020-09-27 22:40:02.334777 Redirected to https://toot.icyphox.sh/api/v2/search?q=icyphox.sh&resolve=True&offset=0&limit=
E 2020-09-27 22:40:02.338004 Error 400, response body: '{"error":"limit - Invalid integer. Got: string.","errors":[{"message":"Invalid integer. Got: string","source":{"pointer":"/limit"},"title":"Invalid value"}]}'
Traceback (most recent call last):
...
  File "/srv/tasks.py", line 86, in post
    self.poll(source)
  File "/srv/tasks.py", line 134, in poll
    links = source.search_for_links()
  File "/srv/mastodon.py", line 148, in search_for_links
    return self.get_activities(
  File "/srv/models.py", line 393, in get_activities
    return self.get_activities_response(**kwargs)['items']
  File "/srv/models.py", line 387, in get_activities_response
    resp = self.gr_source.get_activities_response(**kwargs)
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 197, in get_activities_response
    statuses = self._get(API_SEARCH, params={
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 120, in _get
    return self._api(util.requests_get, *args, **kwargs)
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 135, in _api
    resp.raise_for_status()
  File "/layers/google.python.pip/pip/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://toot.icyphox.sh/api/v2/search?q=icyphox.sh&resolve=True&offset=0&limit=

[jk-na]

I only just noticed earlier today that I wasn't seeing feedback via pleroma reflected on my site since I updated pleroma to the 2.1.1 release a few weeks ago. (Edit: possibly 2.1.0 as there seems to have been many api updates there: https://git.pleroma.social/pleroma/pleroma/-/blob/stable/CHANGELOG.md )

I'll track down some working and non-working URLs tomorrow and raise a pleroma ticket.

[snarfed]

cc @lambadalambda @rinpatch @feld, just FYI, feel free to redirect to whoever on the Pleroma team works on the API!

[jk-na]

This is the log from the last successful interaction from pleroma to my server:
D 2020-09-08 09:42:07.925561 Params: [('response_key', 'agdicmlk...')] I 2020-09-08 09:42:08.044517 Source: @[email protected] (Mastodon) @[email protected], https://brid.gy/mastodon/@[email protected] I 2020-09-08 09:42:08.580631 Source: @[email protected] (Mastodon) @[email protected], https://brid.gy/mastodon/@[email protected] I 2020-09-08 09:42:08.648537 Created by this poll: https://brid.gy/log?start_time=1599558066&key=agdicmlkLWd5cigLEghNYXN0b2RvbiIaQGprQHNvY2lhbC5uaXBwb25hbGJhLnNjb3QM>https://brid.gy/log?start_time=1599558066&Created by this poll: https://brid.gy/log?start_time=1599558066&key=agdicmlkLWd5cigLEghNYXN0b2RvbiIaQGprQHNvY2lhbC5uaXBwb25hbGJhLnNjb3QM/a> I 2020-09-08 09:42:08.649158 Starting Response like tag:social.nipponalba.scot,2013Starting Response like tag:social.nipponalba.scot,2013:9ywa42OcvvmGCeUDVw_favorited_by_9qEkWb1d76feMQOdRw https://social.nipponalba.scot/notice/9ywa42OcvvmGCeUDVw#favorited-by-9qEkWb1d76feMQOdRwhttps://social.nipponalba.scot/notice/9ywa42OcvvmGCeUDVw#favorited-by-9qEkWb1d76feMQOdRw I 2020-09-08 09:42:08.651036 Webmention from https://brid-gy.appspot.com/like/mastodon/@[email protected]/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw to https://jk.nipponalba.scot/note/5f574716/ I 2020-09-08 09:42:08.651310 Sending... I 2020-09-08 09:42:09.619104 Sent! {'request': 'POST https://webmention.io/jk.nipponalba.scot/webmention (with source=https://brid-gy.appspot.com/like/mastodon/@[email protected]/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw, target=https://jk.nipponalba.scot/note/5f574716/)';, 'http_status': 201, 'body': '{"status":"queued","summary":"Webmention was queued for processing","location":"https://webmention.io/jk.nipponalba.scot/webmention/o8JcupAF5h7gryInLVO2";,"source":"https://brid-gy.appspot.com/like/mastodon/@[email protected]/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw";,"target":"https://jk.nipponalba.scot/note/5f574716/";}'} I 2020-09-08 09:42:09.671017 Setting last_webmention_sent

This was before I updated from 2.0.7 to 2.1.0 - I saw the list of breaking changes in the change notes and so waited until I had the time to work through them before updating.

Since then I have successfully posted several times via brid.gy and have had various interactions on pleroma to the posts but no logs of any activity on my brid.gy page in the last 18 days.

[feld]

Evidently Mastodon's /api/v2/search endpoint handles a blank value for the limit query param ok, but Pleroma doesn't.

Is this believed to be the primary source of your issues with Pleroma?

[snarfed]

@feld i haven't done a thorough survey or test yet...but yes, this seems like the only thing blocking us right now.

having said that, you all deserve a lot of credit for how much feature parity you've achieved with the Mastodon API, at least the parts i'm using! this is obviously a trivial difference, and it will be easy to handle on my end, without even special casing Pleroma, by just omitting the limit param when i'm not providing a value. i'm happy to do that. just wanted to raise it on your radar in case you're intereseted.

[feld]

Our API is very strict now and invalid parameters throw an error instead of being ignored. I suspect that limit is not supposed to ever permit an empty value but Mastodon doesn't validate parameters as closely...

[snarfed]

sgtm! thanks for the info.

[feld]

Complementary issue over here now: https://git.pleroma.social/pleroma/pleroma/-/issues/2198

[snarfed]

woo, fixed! https://brid.gy/mastodon/@[email protected] looks good now, sent a bunch of wms.

[ScootRay]

I noticed this error seems to have snuck back for Pleroma. It's reporting permission errors per my log:

https://brid.gy/log?start_time=1614298087&key=agdicmlkLWd5cmYLEg1QdWJsaXNoZWRQYWdlIj9odHRwczovL2Fsb25ndGhlcmF5LmNvbS9waG90b3Mvc3Vuc2V0LW9mZi1vZi1jYW1wb2JlbGxvLWlzbGFuZC8MCxIHUHVibGlzaBiAgIi52oaJCQw

I also looked at the logs for:

woo, fixed! https://brid.gy/mastodon/@[email protected] looks good now, sent a bunch of wms.

...and I see the same errors, whoops.

Is there anything I can do to help fix this?

Thanks,
Ray

[snarfed]

Thanks for the nudge! You're actually seeing a different error than this. I'll follow up on #1015.