Create exportEvents function in plugins that abstracts away all the queueing, batching, retrying logic

[mariusandra]

We're now manually writing a lot of queueing and retrying/jobs logic into events.

Queuing: https://github.com/PostHog/posthog-plugin-replicator/blob/974854579a7a51f815f9d3601e9c5b0f075200d8/index.ts#L19-L27

Retrying: https://github.com/PostHog/s3-export-plugin/blob/91e4813e84237e88142107c00929dc6b147963b7/index.ts#L37-L41

This is super powerful stuff, but our export plugins could be better. Something like:

export async function exportEvents (events, { global, config }) {
    try {
        await fetch(`https://${config.host}/e`, {
            method: 'POST',
            body: JSON.stringify(events),
            headers: { 'Content-Type': 'application/json' },
        })
    } catch (error) {
         throw new RetryError() // ask to retry
    }
}

It would just be an abstraction over existing tools, and it would take care of 1) retry queues, 2) batching, 3) async processing .

You literally write the data upload call and we take care of the rest.

Plus, this will unlock one really strong feature: "start from the beginning". More on that later. :D

Open questions: I'm not sure what's the best way to configure this (e.g. some queues can take 1MB of buffer, some want 100kb, some go just one by one)