Support for JWT Token based Node Attestation

[snanjundaswamy]

Problem

The current join token-based node attestation plugin identifies only a single node. Unlike the k8s/aws_iid node attestation, the join token does not expose any selectors, which limits us in certain aspects:

1. We need to create a join token request for every single node.
2. If we need to create a node alias from join tokens, we must track the initial SPIFFE ID assigned to each node in a separate database and then create a node alias for each of those nodes. Although one might argue that specifying a SPIFFE ID during the join token creation could act as an alias, it is challenging to create multiple node aliases for a subset of these nodes in the same node pool.
3. Creating and managing multiple aliases for nodes in a dynamic environment becomes operationally challenging especially when new nodes are added/removed.

Potential Solution

Create a generic plugin for JWT-based node attestation. A trusted entity that bootstraps these nodes would inject a JWT token with a specific set of labels in the claims. The agent bootstrapped with this JWT token would presented this to the server. The plugin would then verify the JWT token and expose all the claims as selectors. This enables us to create node aliases based on the selectors exposed by the plugin.

This would function similar to a Kubernetes PSAT node attestor, except that the issuer of the JWT token would be a trusted entity responsible for verifying the node before issuing the JWT tokens. To restrict arbitrary claims, we can add a template attributes on the plugin side to validate (Or we can totally ignore this and let folks write then restrictions on selectors when they create node alias entries)

We're happy to contribute back to the community if this is a good idea.

Open Questions

1. Are there any alternatives to solve this problem without implementing a new node attestor plugin?
2. Should we provide an option for the JWT token to be valid for a single attestation to ensure consistent behavior with a join token?

Related Discussions

#2612
#671

Edit 1 : Address comment from @amartinezfayo

Lets say we have just started fresh with tens of thousands nodes on-prem (this is our usecase too). The goal for us is to run multiple sets of services on these machines internal systemd services, telemetry agents and orchestrators themselves. At this layer on baremetal, nothing else works other than join_tokens.

Copying my own example from the spire slack,

Why join_token is not a good fit
For instance consider this arbitrary usecase, Join Token has no way to solve this elegantly and we need to store the mapping of nodes to cluster(lack of a better terminology) in a different database and management of such entries could get out of hand quickly.

spiffe://foo.com/node1 => spiffe://foo.com/cluster1, spiffe://foo.com/cluster2, spiffe://foo.com/cluster3
spiffe://foo.com/node2 => spiffe://foo.com/cluster1, spiffe://foo.com/cluster3
spiffe://foo.com/node3 => spiffe://foo.com/cluster1, spiffe://foo.com/cluster2

We could create a join token with a SpiffeId, but it would be a flat alias across all nodes spiffe://foo.com/nodegroup1 , spiffe://foo.com/nodegroup2 etc.
How do we create an identity that spans a subset of hosts across the two nodegroups 1&2?

Like k8s psat attestors, the powerful weapon we get with JWTs attestor is the ability to extract selectors. With plugin selectors, we could easily map them to a node-alias eliminating the need for additional persistent storage to map overlapping nodes to different clusters.

In a desperate attempt, we could get by using K8s_psat attestor as a generic JWT based node attestor by mimicking the APIs, but it wouldn't an elegant solution

[amartinezfayo]

Thank you @snanjundaswamy for opening this issue.
Have you considered other node attestors rather than join_token? It would be useful to have some more details about the platform(s) where SPIRE is deployed, and the reasons why you think that current node attestors are not a good option.
Also, many node attestors support using an agent path template, that's a way of customizing the format of generated SPIFFE IDs for agents. Maybe that can help with your node alias problem?

[snanjundaswamy]

Thank you @snanjundaswamy for opening this issue. Have you considered other node attestors rather than join_token? It would be useful to have some more details about the platform(s) where SPIRE is deployed, and the reasons why you think that current node attestors are not a good option. Also, many node attestors support using an agent path template, that's a way of customizing the format of generated SPIFFE IDs for agents. Maybe that can help with your node alias problem?

@amartinezfayo I have added the response to the issue description

[amartinezfayo]

Thank you @snanjundaswamy for the details.
Is there a reason why you can't use the x509pop node attestor?

[rturner3]

Expanding on @amartinezfayo's question, the x509pop node attestor has a field called agent_path_template that can be used to extract information out of the node attestation certificate attributes to generate a SPIFFE ID for an agent, similar to how you are thinking about extracting data out of JWT claims. Is that functionality satisfactory for your use case?

[snanjundaswamy]

Thank you for the response folks!

There are two distinct problems here.

1. Construction of the Spiffe ID during node attestation- Could be easily done via Join token's SpiffeID/X509Pop Agent Template. Both of them satisfy our requirements.
2. How to perform node attestation for nodes that could be part of multiple clusters(again for lack of better terminology, refer above example) - X509 pop node attestor exposes one parameter in the form of common_name and can only be mapped to one single Spiffe ID by default. This limitations spans other plugins like join_token/http_challenge etc. On one hand, lets look at plugins like K8s_psat or aws_iid; they expose multiple selectors such as aws_iid:tag:name:blog in aws_iid and k8s_psat:agent_pod_label:key:value in k8s_psat which allows us to use the labels to map them to different clusters. This kind of flexibility is not offered by any other plugins as they do not expose any flex selectors. A JWT token on the other hand enables us to create multiple clusters on the same set of nodes, each one matching a specific claim (labels) within the JWT. Refer to the example I provided above.
   Note: If new labels are added to a node's JWT, we should be okay absorbing the complexity of re-attestation vs us having to create component to map nodes to cluster.

[rturner3]

Thanks for the context, @snanjundaswamy, this is helpful. If it suits your use case, I think it's feasible to have the x509pop produce new selectors corresponding to attributes in the X.509 attestation certificate, depending on which attribute you're interested in.

If that's still not adequate because you need to key your node attestation off of an arbitrary number of selectors or off selector values that don't cleanly map into existing X.509 attributes, then I think introducing a JWT node attestor plugin sounds reasonable. It's a bit more work to implement a new node attestor plugin, so I just wanted to be sure there wasn't a potentially simpler solution to enable your use case with the existing set of node attestation plugins before going down the road of adding a new one.

[snanjundaswamy]

@rturner3 The scaffolding that needs to be in place to get issue X509 certs and JWTs are similar.
I also like the idea incorporating this functionality within X509 node attestor as well. The way we could implement this is by stuffing the required selector attributes into a single/multiple Subject Alternative Names on the leaf cert and get away without needing to implement a new plugin.

Theoretically, X509 does not discourage such augmentation of SANs, just that it feels like an unnatural fit. There are some un-intended side effects of solving this with arbitrary SANs.

1. For instance, if the same cert is used for TLS elsewhere, it could pass the authentication checks for multiple services if one of the SAN (containing the selector) is a DNS resolvable name
2. If the initial cert is issued by a 3rd part service and cannot be customized, we still end up in the same situation.

Directionally managing fewer things makes sense and my current inclination is to implement this with X509 node attestor provided that we are okay with the above two shortcomings.

[rturner3]

1. For instance, if the same cert is used for TLS elsewhere, it could pass the authentication checks for multiple services if one of the SAN (containing the selector) is a DNS resolvable name

It's definitely something to consider in how you manage host identity in your infrastructure. If the need arises, you could provision multiple certs onto a host for different purposes (e.g. one for SPIRE node attestation and another for TLS communication between host processes and remote services).

2. If the initial cert is issued by a 3rd part service and cannot be customized, we still end up in the same situation.

Sure, but I assume that to do this with JWTs, you would still have to set up some internal JWT issuer that sets the proper claims in tokens that you provision onto your hosts? If you already had this JWT infrastructure set up and didn't have an internal PKI, then I can understand why JWT-based attestation might seem like a more appealing option though.

We would welcome a PR that provides SANs as node selectors and agent path template parameters (if not already available) in the x509pop NodeAttestor plugin.

[rturner3]

@snanjundaswamy Would you mind creating a new issue for the proposed changes in the x509pop plugin for adding SAN-based selectors?