Push the built site to a docs branch

[marko-bekhta]

To help move forward the indexing of Quarkiverse guides and then provide search capabilities over them (quarkusio/search.quarkus.io#260) we should push the built HTML pages to some branch in this repository.
This way search.quarkus.io will be able to fetch the repository and work with the local filesystem to scrape the data it needs rather than relying on fetching each page separately over HTTP.

[holly-cummins]

I thought it wasn't even possible to deploy to pages without pushing to a branch first, but looking at the workflows, the official actions for pages do it by an upload instead. If we didn't want to fuss with the workflow here, the CI artifacts (such as https://github.com/quarkiverse/quarkiverse-docs/actions/runs/9991399209/artifacts/1714955609) could be used ... but it's probably easier to just add an extra step in the workflow than it is to fuss with getting the ID of the latest CI run and then browsing its artifacts.

[yrodiere]

but it's probably easier to just add an extra step in the workflow than it is to fuss with getting the ID of the latest CI run and then browsing its artifacts.

+1. Both are possible, but pushing to a branch is much easier; be it only for testing.

[gastaldi]

-1 to pushing to a branch to deploy to GH Pages: The existing workflow was migrated from that to prevent messing around with the generated content.

+1 to add an extra step in the workflow to do whatever is necessary (including pushing the generated content to search.quarkus.io if necessary)

[yrodiere]

-1 to pushing to a branch to deploy to GH Pages: The existing workflow was migrated from that to prevent messing around with the generated content.

Fine.

+1 to add an extra step in the workflow to do whatever is necessary (including pushing the generated content to search.quarkus.io if necessary)

Pushing to search.quarkus.io is not a good idea, as the service is currently "stateless" -- in the sense that it can be dropped and redeployed at will, and it will reinitialize itself. I wouldn't want to push state there that cannot easily be re-generated.

If we can change the workflow though, maybe we could have it publish a GitHub "release" or "package". I imagine there are ways to cache these on the search.quarkus.io side -- using SHAs or IDs or whatever.

[holly-cummins]

If we can change the workflow though, maybe we could have it publish a GitHub "release" or "package". I imagine there are ways to cache these on the search.quarkus.io side -- using SHAs or IDs or whatever.

I think there's three options.

• We can use the normal publish-artifact action to publish a zip to the individual workflow, and then the search.quarkus.io can look up the most recent green build and grab the artifact using download-artifact and the run-id. That action now works across repos, which is handy.
• We can try and do some cross-repo build triggering so that search doesn't have to do the hevay lifting of finding the artifact? I haven't thought through how it would work, though
• We use release mechanisms for the artifact sharing. The downside of this is that it makes a lot of notification noise. The upside is that it's easy for the consumer to get the latest artifact. See https://github.com/quarkiverse/antora-ui-quarkiverse and https://github.com/quarkiverse/quarkiverse-docs for a pair of repositories that have this kind of linkage. Note the ridiculously high release count in antora-ui-quarkiverse 😬

[yrodiere]

We can use the normal publish-artifact action to publish a zip to the individual workflow, and then the search.quarkus.io can look up the most recent green build and grab the artifact

I mentioned releases because I think build artifacts expire after some time... but I suppose this doesn't matter if we're sure we'll rebuild quarkiverse-docs nightly.

search.quarkus.io already has the GitHub API in the classpath, and we've already set up a GitHub account for it (so that it can report about indexing at quarkusio/search.quarkus.io#130). So it could definitely figure out what to download.

using download-artifact and the run-id. That action now works across repos, which is handy.

Note search.quarkus.io is deployed as a simple Quarkus application, without data. Indexing happens at runtime (on startup, and every day), not at build time. So GitHub actions are not an option here.

Anyway, thanks all for your suggestions. I think we have all the information we need, now we'll need to find time to work on this :)

[holly-cummins]

Note search.quarkus.io is deployed as a simple Quarkus application, without data. Indexing happens at runtime (on startup, and every day), not at build time. So GitHub actions are not an option here.

There's a world of automation beyond GitHub Actions and build-time? 🤯

(Er, joke, in case it wasn't obvious. 😀 ... but I actually do sort out so much of my professional life through the mechanism of GH Actions workflows it is genuinely useful to be reminded there are other ways.)