-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add granular privileges for API keys #42020
Conversation
In the current implementation of API keys, to create/get/invalidate API keys one needs to be super user which limits the usage of API keys. We would want to have fine grained privileges rather than system wide privileges for using API keys. - `manage_api_key` cluster privilege which allows users to create, retrieve and invalidate any API keys in the system. This allows for limited access than `manage_security` or `all` privileges. To support scenario's where we want to give granular privileges this commit adds support for conditional cluster privileges where we can create different combinations of `create`, `get`, `invalidate` API key privileges for restricting actions to API keys: - owned by the user - owned by group of users - owned by group of users under specified realms - any user from any realm This commit does not:- - define any new API for ease, this will be taken up later after we decide if this approach works - HLRC changes
Pinging @elastic/es-security |
this.users = (users == null) ? Collections.emptySet() : Set.copyOf(users); | ||
this.usersPredicate = Automatons.predicate(this.users); | ||
|
||
this.requestPredicate = (request, authentication) -> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This takes care of enforcing the authorization, makes use of Authentication
details containing the principal
or API key id from authentication metadata.
For owner API keys enforcement, the request must contain either (username
, realmName
) or apiKeyId
*/ | ||
public void getApiKeyForApiKeyName(String apiKeyName, ActionListener<GetApiKeyResponse> listener) { | ||
public void getApiKeys(String realmName, String username, String apiKeyName, String apiKeyId, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ApiKeyService
has two APIs for retrieving and invalidating API keys.
They do not enforce any authorization but with validation that atleast one of the parameters must be specified.
getApiKeys(String realmName, String username, String apiKeyName, String apiKeyId);
invalidateApiKeys(String realmName, String username, String apiKeyName, String apiKeyId);
return sameUsername; | ||
} else if (request instanceof GetApiKeyRequest) { | ||
GetApiKeyRequest getApiKeyRequest = (GetApiKeyRequest) request; | ||
if (authentication.getAuthenticatedBy().getType().equals("_es_api_key")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is required for case where the API key is used for authentication and it should be able to get the details and it does not have any API key privileges in the assigned role.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this accounted for in ManageApiKeyConditionalClusterPrivilege#checkIfUserIsOwnerOfApiKeys
:
if (authentication.getAuthenticatedBy().getType().equals("_es_api_key")) {
// API key id from authentication must match the id from request
String authenticatedApiKeyId = (String) authentication.getMetadata().get("_security_api_key_id");
if (Strings.hasText(apiKeyId)) {
return apiKeyId.equals(authenticatedApiKeyId);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code in the ManageApiKeyConditionalClusterPrivilege
would be used only when the user has manage_own_api_key
privilege assigned. In the case where an API key was created but we did not give it any permission related to API key management then, at the least it should be able to access its own details. This code checks for that condition.
import java.util.function.BiPredicate; | ||
import java.util.function.Predicate; | ||
|
||
public final class ManageApiKeyConditionalPrivileges implements ConditionalClusterPrivilege { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With conditional privileges for API keys, the Rest APIs remain same but when conditional privileges restrict access of API keys to owner only then there is inherent requirement that requests must either contain (username
& realmName
) or (apiKeyId
) or else the request fails.
* ability to execute actions related to the management of application privileges (Get, Put, Delete) for a subset | ||
* of applications (identified by a wildcard-aware application-name). | ||
*/ | ||
public static class ManageApplicationPrivileges implements ConditionalClusterPrivilege { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has been moved to its own file.
@elasticmachine run elasticsearch-ci/1 |
What's the rationale for adding this level of complexity? My primary concern is that the Kibana security management UI doesn't support "global" privileges, so they have to be edited manually, so I feel like we've made this much more complex for ourselves as well as for security admins, without know what use case that we're trying to solve. |
this.usersPredicate = Automatons.predicate(this.users); | ||
|
||
this.requestPredicate = (request, authentication) -> { | ||
if (request instanceof CreateApiKeyRequest && privilege.predicate().test(CREATE_API_KEY_PATTERN)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think you need to check the privilege here do you? ConditionalClusterPermission
always checks the predicate against the action being executed, so why do we need to check this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, we need not, removed the check. Thank you.
final GetApiKeyRequest getApiKeyRequest = (GetApiKeyRequest) request; | ||
if (this.realms.contains("_self") && this.users.contains("_self")) { | ||
return checkIfUserIsOwnerOfApiKeys(authentication, getApiKeyRequest.getApiKeyId(), getApiKeyRequest.getUserName(), | ||
getApiKeyRequest.getRealmName()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem right - it means if you have _self
then you can only ever get your own keys (it never falls through to the checkIfAccessAllowed
condition to allow you to get other keys that you might have permission over).
But nothing in this class enforces that _self
must be the only item in the list, so "realms": [ "_self", "native" ], "users": [ "_self", "*"]
wouldn't actually let me manage API keys for the native realm, but nor will it give me an error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you, I think I had this test case to enforce that when _self
is used it must be the only item but missed it while writing the code. I have enforced the check.
} | ||
|
||
private boolean checkIfUserIsOwnerOfApiKeys(Authentication authentication, String apiKeyId, String username, String realmName) { | ||
if (authentication.getAuthenticatedBy().getType().equals("_es_api_key")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we move these string checks to somewhere central, like a method on ApiKeyService
:
public static boolean isApiKeyAuthentication(Authentication auth) { ... }
or even better
public static String getAuthenticatedApiKey(Authentication auth) { ... }
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am with you on adding the check in ApiKeyService
but the current code structure does not allow it to do so. ManageApiKeyConditionalPrivileges
is in core
due to it being used when we parse conditional privileges in role descriptor. It feels like we need some refactoring here to push the classes to security but unsure of what all things should we move. Do you want to tackle this here with this PR or as a separate PR for refactoring?
String authenticatedUserRealm = authentication.getAuthenticatedBy().getName(); | ||
if (Strings.hasText(username) && Strings.hasText(realmName)) { | ||
return username.equals(authenticatedUserPrincipal) && realmName.equals(authenticatedUserRealm); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this behaviour is what we'd want when run_as
is used.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, you are right. This does not take run_as
scenario into consideration as we currently do not support run_as
for token based schemes.
As I understand if we were to then the condition needs to be updated to check the realm name against the lookedUpBy
realm instead of authenticatedBy
realm.
Do we want to handle the scenario here with this PR as right now we do not support run_as
for API keys?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't follow why run_as
for API keys would come in to play here.
The question is What should the behaviour be if I authenticate as elastic
and run-as bizybot
and then try to delete one of bizybot
's keys?
…y _self when restricting API keys to owner
I agree the current proposal is verbose. As in we need to specify: cluster actions, realms, users and create the conditional cluster privilege. I started this to see if we can construct conditional privilege, for example, creation and retrieval of API keys limited to self. I must admit that I did not consider Kibana UI, though I feel that UI should not drive the decision here. The next step that I thought was to create named conditional cluster privileges similar to the cluster privilege names and use these names when constructing the role. I have restricted the model to be limited to manage API keys with this PR, but we could enable this conditional cluster privileges generically so any combination of cluster actions could be used giving flexibility in the field to construct role where we miss some cluster privileges (Ref: #30078) I agree specifying the cluster permissions and creation of conditional cluster can be complex for security admins and so I think we would like to have some notion of named conditional privileges which come by default and also allow the flexibility to do combinations of cluster actions as an advanced capability. Also, this does not require us to add any extra APIs and is solely driven by the user's role. |
Why would this be the case? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall, I think this tries too hard to hide that manage_own_api_key
is a ConditionalClusterPrivilege
. For example:
- The
ClusterPrivilegeResolver
combinesGlobalClusterPrivileges
withClusterPrivileges
. The ability to name certainGlobalClusterPrivileges
and to be able to refer to them by their name as aClusterPrivilege
is interesting but it complicates everything. I would suggest we have a separate PR for this. What do you think? Is there a subtler issue behind all this, in which case I completely missed it? ManageApiKeyConditionalClusterPrivilege
is not aGlobalClusterPrivilege
because then it will be grouped as such in the role descriptor.ManageApiKeyConditionalClusterPrivilege
is configurable withrestrictActionsToAuthenticatedUser
, presumably so thatmanage_api_key
andmanage_own_api_key
names as cluster privileges instantiate this conditional cluster privilege with different parameters.
I think I have an idea on how to simplify this, but I need to try and code it first so that we minimize the noise here.
return sameUsername; | ||
} else if (request instanceof GetApiKeyRequest) { | ||
GetApiKeyRequest getApiKeyRequest = (GetApiKeyRequest) request; | ||
if (authentication.getAuthenticatedBy().getType().equals("_es_api_key")) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Isn't this accounted for in ManageApiKeyConditionalClusterPrivilege#checkIfUserIsOwnerOfApiKeys
:
if (authentication.getAuthenticatedBy().getType().equals("_es_api_key")) {
// API key id from authentication must match the id from request
String authenticatedApiKeyId = (String) authentication.getMetadata().get("_security_api_key_id");
if (Strings.hasText(apiKeyId)) {
return apiKeyId.equals(authenticatedApiKeyId);
}
}
What I had in mind is likewise nasty. I wanted to move I will spend some more time tomorrow, but I would very much enjoy smaller PRs, because this is difficult to review. |
I agree, but I see it in the reverse.
Based on those things, I feel pretty strongly that we can't tell users that I think the problem you're seeing is more a case that we're trying to hold on to the plain old |
The question is how we get there. I'm worried about how long thing PR has been floating about and where we find the boundary for what it should deliver. From the beginning I've been pushing for this to provide the bare minimum of features, but to implement them correctly. And I think it's close to doing that - except that the In my opinion, I also have some ideas on how to clean that up, but I don't know if we can solve them completely in this PR. Like @albertzaharovits, I'd need to go away and actually play with the code and propose a set of refactorings that get us where we need to go. So, my proposal would be to get this PR to the point where we can merge it into a feature branch, so that we can then have separate PRs to do that cleanup. (*) Or, probably correct - we can throw away/re-write a feature branch if we decide to, but let's at least avoid going down paths that we don't believe will work. |
Excellent write-up Tim! I agree with all your points.
In principle I agree, but practically it still needs iterations. For example |
I agree that a big part of the solution for clearing this up will be to find a common base class / interface for the things that the resolver returns. |
PR build failed due to existing intermittent issue #43315 |
@elasticmachine run elasticsearch-ci/1 |
Thank you @albertzaharovits and @tvernum. Note: As suggested we can merge into a branch and raise the refactoring PR which will be easier to review. I can revert and raise another PR if you feel the same. Thanks. |
public List<Tuple<ClusterPrivilege, ConditionalClusterPrivilege>> privileges() { | ||
return Collections.singletonList(new Tuple<>(super.privilege, null)); | ||
public List<ClusterPrivilege> privileges() { | ||
return Collections.singletonList(super.privilege); | ||
} | ||
} | ||
|
||
/** | ||
* A permission that makes use of both cluster privileges and request inspection | ||
*/ | ||
public static class ConditionalClusterPermission extends ClusterPermission { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ConditionalClusterPermission
and SimpleClusterPermission
seem to overlap in scope.
|
||
import static java.util.Map.entry; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ClusterPrivilege
got barren, we should probably remove it.
INVALIDATE_API_KEY_PATTERN); | ||
|
||
private final boolean restrictActionsToAuthenticatedUser; | ||
private final BiPredicate<TransportRequest, Authentication> requestPredicate; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is not accounted for in hashCode
and equals
.
return username.equals(authenticatedUserPrincipal) && realmName.equals(authenticatedUserRealm); | ||
} | ||
} | ||
return false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not very comfortable with the option of interpreting values for user and realm based on the privilege it has as we do not know the intent of the user invoking the API. For eg. when the user does not specify user and realm, and the user has manage_own_api_key we interpret it as self and continue with the operation.
The API is designed to query by id
, name
, user
and realm
and to have visibility over the whole pool of keys.
If the API can query by user
and we want to limit it, then we either forbid the parameter in a new API or authorize the parameter value. If we authorize the parameter value, what happens if the parameter is not specified; we either reject it or fill in a default value that will not reject (un-authorize) the request.
I honestly don't know what to do in this case, I would suggest we discuss this as a team.
Sorry @bizybot I think this touches too much of the authz process at once for me to be confident to Okay it. This is my suggestion, what do you think? I can help, and take parts of it. If you or others feel otherwise I could give it another thorough round. |
Given where we're at with this, and the limited availability we've had for people to work on it, I think the first step we can take right now is to split It won't be enough, but it's a step that we can take. I'll see if I can get that up in the next few hours. The other option we can look at in the short term is to add a |
Thanks, @tvernum for raising a separate PR for the short term fix. I was going to suggest the same. I think the next step would be to split this PR into two:
|
Closing this in favor of splitting this into refactoring and API key privilege PRs. |
In the current implementation of API keys, to create/get/invalidate
API keys one needs to be a superuser which limits the usage of API keys.
We would want to have fine-grained privileges rather than system-wide
privileges for using API keys.
manage_api_key
cluster privilege which allows users to create, retrieveand invalidate any API keys in the system. This allows for limited access than
manage_security
orall
privileges.manage_own_api_key
is a conditional cluster privilege which allows creating,retrieving and invalidation of API keys owned by the currently authenticated user.
This commit adds support for conditional cluster privileges for API keys so we can create privileges like
manage_own_api_key
. The conditional cluster privilege names can be used to specify cluster privileges.This commit does not:-