Add macaroon based RPC middleware interceptor

[guggero]

With this PR we introduce the concept of RPC middleware: A mechanism
similar to the existing channel or HTLC interceptors but this time for
gRPC messages themselves.
An RPC middleware can register itself to the main RPC server to get
notified each time a new gRPC request comes in, a gRPC response is sent
back or a streaming RPC is connected. The middleware can
validate/inspect incoming requests and modify/overwrite outgoing
responses.

The new flow for gRPC requests with this PR looks like this:

//        |
//        | gRPC request from client
//        |
//    +---v--------------------------------+
//    |   InterceptorChain                 |
//    +-+----------------------------------+
//      | Log Interceptor                  |
//      +----------------------------------+
//      | RPC State Interceptor            |
//      +----------------------------------+
//      | Macaroon Interceptor             |
//      +----------------------------------+--------> +---------------------+
//      | RPC Macaroon Middleware Handler  |<-------- | External Middleware |
//      +----------------------------------+          |   - approve request |
//      | Prometheus Interceptor           |          +---------------------+
//      +-+--------------------------------+
//        | validated gRPC request from client
//    +---v--------------------------------+
//    |   main gRPC server                 |
//    +---+--------------------------------+
//        |
//        | original gRPC request to client
//        |
//    +---v--------------------------------+--------> +---------------------+
//    |   RPC Macaroon Middleware Handler  |<-------- | External Middleware |
//    +---+--------------------------------+          |   - modify response |
//        |                                           +---------------------+
//        | edited gRPC request to client
//        v

Motivation

This PR was initially motivated by #291. Adding all that business logic to lnd itself just didn't seem like the reasonable way to implement accounts.

Instead a similar mechanism to the channel and HTLC acceptors would make more sense as it would also allow other developers to implement custom business logic outside of lnd, remotely similar to how plugins work in c-lightning for example.
The main advantage of the approach in this PR is that the custom business logic would not live in the same process (or even a sub-process) of lnd. The main disadvantage lies in the added request/response latency that intercepting each message can add.

An example of how this can be used to implement an account system on top of lnd can be seen here, where the functionality is added to faraday.

Security

Since this also opens the door for malicious software to interfere with
lnd in a negative way, we bind everything to macaroons with custom
caveat conditions: A middleware declares upon registration which custom
caveat name it can handle. Only client requests that send a macaroon
with that custom caveat will then be given to the middleware for
inspection. The alternative is for a middleware to register for
read-only access. In that case it will get access to all requests and
responses but isn't allowed to modify anything.
Therefore requests with the default, unencumbered macaroons
can never be modified by any middleware.

Fixes #4383.

[alexbosworth]

How do you pass extra data to the middleware in the request? Modify the proto to include the additional data?

[guggero]

This isn't supported with the current version, really. Modifying the proto would kind of defeat the purpose of having a generic interceptor/middleware that allows you to use a stock lnd.
It could be done in a number of different ways though. What kind of additional information did you have in mind?
In the use case I built this for, the only extra information required is a caveat in the macaroon.

[alexbosworth]

This isn't supported with the current version, really. Modifying the proto would kind of defeat the purpose of having a generic interceptor/middleware that allows you to use a stock lnd.
It could be done in a number of different ways though. What kind of additional information did you have in mind?
In the use case I built this for, the only extra information required is a caveat in the macaroon.

I don't have any specific information in mind, but since you can mutate the macaroon I guess you can add more request data there?

I'm curious why the interceptor doesn't see the mutated request data if it has access to the raw protobuf request blob?

[gkrizek]

Would registering your middleware to the main RPC require a macaroon?

Once your middleware is registered, it only see requests from macaroons with the custom caveat?

[guggero]

Yes, to register the middleware you need a macaroon with the macaroon:write permission which only the admin default macaroon has. You also tell lnd which custom macaroon caveat your middleware handles and lnd will then only let you intercept any requests that have a macaroon with that caveat.

[gkrizek]

Got it. I think this is a great idea.

Do you think there would ever be a situation to allow all macaroons to go through the middleware or to use a macaroon other than macaroon:write to register? I'll explain an idea I had;

What if I wanted to have a readonly middleware that could keep an access logs of the commands ran on the node? In that scenario it would be great to use a macaroon with macaroon:read permissions to register the RPC middleware. And when I use the macaroon:read permission to register that means the middleware could never take any action on a request. Just view it. This would also be a good case to see all macaroon actions, not just custom caveats. Just a thought I had...

[guggero]

Yeah, I think with the mechanism in place where the middleware has to "register" itself after connecting, we could definitely add some feature toggles. Where you could say "I want read access but to all requests".

And maybe we could add a new macaroon permission as well, for example middleware:read and middleware:write. With the read permission you would only be allowed to set the above mentioned flag to true anyway.
I think that could be very useful!

[Roasbeef]

Woah, this is gonna be soooo powerful

The implementation is also rather streamlined IMO, first review pass, left a few initial comments. Will also check out that Faraday example so I can get a better idea w.r.t how this can be used in practice

[guggero]

Rebased and addressed commits.

The following things still need to be added before the WIP state can be removed:

• Add unit tests
• Add integration tests
• Add flag to enable interceptor globally

[guggero]

Update: only one TODO remaining, removing WIP state now:

• Add itest case for overwriting a response.

[Kixunil]

My last question was not related to read/write permissions.

Basically I can see at least two use cases:

• Have multiple users with budgets, each gets a unique macaroon and budgets are tracked by the middleware.
• Let an application only see invoices it created - so if an application is compromised the privacy leak is limited.

In order to achieve this the middleware has to be able to distinguish macaroons somehow. What's the best way?

[guggero]

Just need to review itest + do some regtest testing now, testing with faraday example best way to do so I assume?

@carlaKC I rebased the lndclient and faraday example branches but didn't have time to fully test them again... I hope they still work. But yes, the faraday example is probably the best one to test this out.

In order to achieve this the middleware has to be able to distinguish macaroons somehow. What's the best way?

I'm not sure I understand the problem. The middleware gets the full, raw macaroon in the interception request. So it can look at all the caveats and decide what needs to be done based on that.

[Kixunil]

I'm asking whether putting the client identifier into custom condition is a good way to do it or if there's a better way.

[guggero]

I'm asking whether putting the client identifier into custom condition is a good way to do it or if there's a better way.

I think so, yes. With that construction you can safely give the macaroon to a user and they cannot remove the restriction.
If you want multiple middlewares to be able to interact with requests of the same user, you can do that as well. For example the accounting middleware would add the custom macaroon caveat account <acct_ID> and another , let's say custom logging middleware would add a "marker" caveat, for example mylogger.
A user would then get a macaroon that has both caveats applied. So any request from that user would be intercepted by both middlewares. And the logger middleware could still extract the account ID from that other caveat, even if it didn't specifically register for it.

[Roasbeef]

Just needs a rebase and this can land, also we likely want to land this soon to avoid rebases since it touches so much in the RPC layer.

[carlaKC]

tACK! What an awesome change to lnd 🥇 can't wait to see what people build out on top of this.

Didn't test the full scope of this (had to timebox my fun), but managed to cover the following with a regtest interceptor stub:

• Readonly interceptor
• Custom interceptor w/ modification
• Custom interceptor with + without caveats
• Mandatory interceptor enforcement

Only thing that I noticed while testing is that it's difficult to use our lndclient with mandatory interceptors, since we do a bunch of GetInfo calls etc to get started, but should be easy enough to just add middle ware registration before those calls? More of a note to self than an issue with the PR.

Just an unhappy linter then we're gg 🚀

[guggero]

Oops, looks like I broke something in the itests with the latest rebase. Will fix on Monday.