Add Gitea as VCS provider

[steko]

Details

• Read the Docs project URL: https://readthedocs.org/projects/iosacal/
• Build URL (if applicable):
• Read the Docs username (if applicable): steko

Expected Result

The source repository of iosacal is hosted on Codeberg, a community-run Gitea instance with more than 10.000 active repositories. I would like Read the Docs to do automated builds when I push updates to the repository, just like with GitHub repositories. Ideally this would apply to any Gitea instance.

Actual Result

The generic webhook that can be created from the Read the Docs admin dashboard doesn't work, because there is no way to configure a single repository on Gitea with the secret token. The issue is described in more detail on the Codeberg community issue tracker with regard to the current webhooks available in Gitea.

[astrojuanlu]

Hello @steko, thanks for opening this issue. I just read https://codeberg.org/Codeberg/Community/issues/486 and go-gitea/gitea#7487 as well.

We got a similar request a few months ago for Launchpad #8137 and, although we are trying not to support specific platforms, we are definitely happy to make our generic webhook more useful.

As you already saw, at the moment it expects a token parameter, but as far as I understand, both Launchpad, Gitea, and Gogs provide a HMAC-signed secret, and all have standardized for X-Hub-Signature as the header name, since it comes from a W3 standard https://www.w3.org/TR/websub/#conformance-classes 🏆 In fact, even GitHub is now using X-Hub-Signature as well https://docs.github.com/en/developers/webhooks-and-events/webhooks/securing-your-webhooks#validating-payloads-from-github

We will comment on this issue when we have more updates.

[astrojuanlu]

(Edit: forgot to add the links to the external issues)

[humitos]

Hi! What would be the changes required to the Read the Docs generic webhook to be compatible with what Gitea sends/expects?

[astrojuanlu]

From my very limited understanding, the X-Hub-Signature header works in a similar way as the one we already use from GitHub:

readthedocs.org/readthedocs/api/v2/views/integrations.py

Lines 329 to 353 in bc3e147

	def is_payload_valid(self):
	"""
	GitHub use a HMAC hexdigest hash to sign the payload.
	It is sent in the request's header.
	See https://developer.github.com/webhooks/securing/.
	"""
	signature = self.request.META.get(GITHUB_SIGNATURE_HEADER)
	secret = self.get_integration().secret
	if not secret:
	log.info(
	'Skipping payload signature validation. project=%s',
	self.project.slug,
	)
	return True
	if not signature:
	return False
	msg = self.request.body.decode()
	digest = GitHubWebhookView.get_digest(secret, msg)
	result = hmac.compare_digest(
	b'sha1=' + digest.encode(),
	signature.encode(),
	)
	return result

[humitos]

From my very limited understanding, the X-Hub-Signature header works in a similar way as the one we already use from GitHub

👍🏼 That is what I understood from your research (*), but I'm not sure why it's not working currently if it works in the same way. That's why I was asking what's what we need to modify in our generic webhook to make Gitea work.

(*) I haven't into the code or deeper in the problem yet.

[stsewd]

Apart from the token, we need to handle the payload as well, which is different between providers, that's the main issue here I think, not the secret token (we only generate/use one when your account is connected)

The generic webhook that can be created from the Read the Docs admin dashboard doesn't work, because there is no way to configure a single repository on Gitea with the secret token.

Does gitea use the same payload as github? If so, you can just create a github type of integration on rtd and that should work.

[steko]

Does gitea use the same payload as github? If so, you can just create a github type of integration on rtd and that should work.

Gitea does use the same payload as GitHub with X-Hub-Signature (according to the detailed comments by @astrojuanlu above and feedback from the Gitea developers) and, as absurd as it may seem, creating a GitHub webhook and feeding it to Gitea will work perfectly. Read the Docs will complain when creating the webhook since there is no GitHub integration setup for the project, but after the first successful run the warning message disappears.

The request headers are like the following and there is no X-Hub-Signature that I can see but I know very little about how this is supposed to work.

Host:
    readthedocs.org
Connection:
    close
X-Amzn-Trace-Id:
    Root=1-60ff1c91-2c42073b1ac752b940ada12b
Accept-Encoding:
    gzip
Cf-Ipcountry:
    DE
Cf-Ray:
    67506a28aa364de8-FRA
Cf-Visitor:
    {"scheme":"https"}
User-Agent:
    Go-http-client/1.1
X-Github-Delivery:
    800db08e-50f6-416e-9410-474a03955139
X-Github-Event:
    push
X-Gitea-Delivery:
    800db08e-50f6-416e-9410-474a03955139
X-Gitea-Event:
    push
X-Gitea-Signature:
X-Gogs-Delivery:
    800db08e-50f6-416e-9410-474a03955139
X-Gogs-Event:
    push
X-Gogs-Signature:
Cf-Connecting-Ip:
    2a03:4000:4c:e24:85e:10ff:fef8:a405
Cdn-Loop:
    cloudflare
Content-Type:
    application/json 

I'm really happy about the outcome and I would like to thank all developers who chimed in with their experience and suggestions ✨

I think a better UI for Gitea (and Launchpad) would be for the best!

[astrojuanlu]

go-gitea/gitea#16176 was merged a few days ago, perhaps Codeberg is not at the latest version?

In any case, glad it works! I think this shows that, with supposedly little work, we can improve the experience for a bunch of providers at once.

[6543]

mentioned pull is not in latest stable (it's in v1.15.0 witch has it's second rc at the moment)

[stsewd]

We can probably mention in the docs that you can create a github integration and use it for gitea as a workaround. We also have been talking about exposing the secret token for manually created integrations.

[steko]

We can probably mention in the docs that you can create a github integration and use it for gitea as a workaround.

Can I prepare a pull request for this? Should Launchpad be mentioned as well?

[stsewd]

Can I prepare a pull request for this? Should Launchpad be mentioned as well?

Sure, about launchpad I'm not sure if they share them same payload as github/gitea

[cjwatson]

Sure, about launchpad I'm not sure if they share them same payload as github/gitea

No, Launchpad uses a different payload, so the same approach won't work.

[6543]

mentioned pull is not in latest stable (it's in v1.15.0 witch has it's second rc at the moment)

v1.15.2 is out 🚀

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[humitos]

Unfortunately, we are not going to implement full support for Gitea soon. We weren't able to prioritize this and put it into our roadmap. Besides, the work suggested to update our documentation was done in #8402 and, at least, there is a workaround to set up a Gitea repository on Read the Docs.

I'm closing this issue for now but we can come back in the future if there is more people interested in fully supporting Gitea and if we can find some time to prioritize it.

Thanks you all for helping debugging this and working towards a workaround 😄

[Turk571]

finansbank