Allow API keys to be forwarded with events

[nblumhardt]

Individual applications on a machine may use different API keys to drive their logging level, filtering, and attached properties on ingestion.

If an application uses an API key when connecting to the forwarder, it would be nice to send the events under that API key in their own batch, overriding any API key configured directly in the forwarder.

From: http://docs.getseq.net/v3/discuss/575ac49ea083950e0044871a

[EvanMulawski]

Any update on this? We would like to use one shared instance of Seq Forwarder on each of our hosts to capture and send all events from all apps running on a host to the configured Seq server.

[nblumhardt]

Thanks for the nudge, Evan. We haven't made progress on this one, though I hope that after the coming Seq 4 release we'll be able to focus some energies on the forwarder. I'll update this ticket when things get moving :-)

[mason-mcglothlin]

Any update on the status of this?

[nblumhardt]

@mason-mcglothlin thanks for the nudge. While not strictly a part of the overall Seq release train, I've added this to our 4.2 milestone and will aim to have a solution at or around the time we get Seq 4.2 out the door. Expecting ~September at this stage. Thanks again!

[danjohnso]

Definitely interested in seeing this function. We have ~6 individual applications that all forward logs through this.

[nblumhardt]

Making progress on this one finally 🎉 ... should have a pre-release build out in the next day or so to try.

The overall design is for the forwarder to be transparent as far as API keys are concerned: if the client app provides one, its events will be sent to Seq using that key. If no API key is provided by the client app, we'll fall back to the API key configured in SeqForwarder.json, if there is one. This should make the upgrade path reasonably smooth, as long as apps are connecting to the forwarder with either no key, or a valid key.

The major challenge is that, if the forwarder acts transparently and groups calls to Seq by API key provided by the application, multiple log buffers need to be maintained: calls with one API key may succeed, while others may fail, with the net result being we need to manage a cursor and storage per client API key.

The plan at this stage is just to create a buffer and shipper per API key - manageable for up to 20 or so API keys per instance, I think.

[nblumhardt]

All - there's now an MSI at:

https://ci.appveyor.com/project/datalust/seq-forwarder/build/artifacts

It's built from the dev branch, with this feature merged. If you're able to check it out, any feedback on how the feature has shaped up would be fantastic. Thanks!

[danjohnso]

Working as advertised so far in our dev environment. Will roll it out to prod in our next release cycle.

[nblumhardt]

Brilliant, thanks @danjohnso