[FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

cwperks · 2022-12-09T14:57:09Z

Is your feature request related to a problem?

New reserved roles are often added between releases when a new plug-in is released and associated roles are created to provision access to features of the new plugin. Reserved roles are added to the security repo roles.yml like is done in this PR: #2192 which introduces 3 new reserved roles between 2.3.0 and 2.4.0.

When starting up a cluster with the upgraded version of OpenSearch I receive the following line:

Index .opendistro_security already contains doc with id roles, skipping update

indicating that the new reserved roles in roles.yml will need to be manually added either via the API by an admin or via securityadmin tool connecting via clientcert as a super admin.

This becomes additionally complicated when users of OpenSearch add custom roles or modify the security index in any way via the OSD UI or API as it is destructive to run securityadmin.sh because it can delete custom configuration. In order to get around this I have:

Ran a backup of the security index using security admin: https://opensearch.org/docs/latest/security-plugin/configuration/security-admin/#a-word-of-caution
Reconcile the exported roles.yml with new reserved roles in the repo's roles.yml: https://github.com/opensearch-project/security/blob/main/config/roles.yml
Run securityadmin.sh with the coalesced exported roles.yml with custom roles and the repo's roles.yml of the corresponding version of opensearch

This is not an idea process and there can be a better solution to source new reserved roles during an upgrade

The text was updated successfully, but these errors were encountered:

peternied · 2022-12-12T20:21:59Z

@opensearch-project/security-analytics This scenario might be impacting y'all, it might be worthwhile to invest in this scenario. Here is where I believe there is a lot of overlap, after upgrading a cluster that does not have the new roles needed for the security-analytics plugin when it is 'upgraded-in' on at a later time.

cwperks · 2022-12-12T20:22:00Z

[Triage] In the short-term this can be a documentation fix.

@peternied to follow-up on this issue.

eirsep · 2023-02-16T18:00:40Z

One possible solution:
Have 2 security config indices - one for pre-defined roles (internal to Opensearch and its plugins) and one for execlusively for user defined roles
On every node restart/bootup (or a new admin API invocation), we should reload roles/permissions into the pre-defined roles index from the yml file and leave the user security roles as is.

This would remove some of the static-ness from our design and solve the problem for upgrading domains getting the latest roles and permissions from yml file without user action/documentation.

peterzhuamazon · 2023-05-01T18:27:40Z

This is observed on a cluster where the cluster was upgrade from 1.3 to 2.3 and missing CCR perms and roles. This does not resolve after redeploy.

peternied · 2023-05-01T22:40:38Z

This is observed on a cluster where the cluster was upgrade from 1.3 to 2.3 and missing CCR perms and roles. This does not resolve after redeploy.

This is the expected behavior, the security plugin does not alter the security configuration on established clusters. Cluster administrators can add this functionality via the security admin tool https://opensearch.org/docs/latest/security/configuration/security-admin

This feature request is about adding this functionality which some administrators might be comfortable with, whereas others might not want this behavior. Its a complex scenario based on the expectations of the administrator, we would be open for pull requests around this space.

peterzhuamazon · 2023-05-02T22:34:28Z

This is observed on a cluster where the cluster was upgrade from 1.3 to 2.3 and missing CCR perms and roles. This does not resolve after redeploy.

This is the expected behavior, the security plugin does not alter the security configuration on established clusters. Cluster administrators can add this functionality via the security admin tool https://opensearch.org/docs/latest/security/configuration/security-admin

This feature request is about adding this functionality which some administrators might be comfortable with, whereas others might not want this behavior. Its a complex scenario based on the expectations of the administrator, we would be open for pull requests around this space.

Got it, it would be nice if it can prompt a windows to let me know new roles in 2.3 version after upgrade. This would be very handy and easy for the users.

Without this, I was fairly confused for half a day trying to figure out is my cluster have enough permission to run ccr, with multiple b/g and upgrade to latest version, without using the all_access role.

Thanks.

peternied · 2023-05-02T22:37:31Z

@peterzhuamazon Roger that - must have been frustrating. Good idea about the warning.

peternied · 2024-02-26T19:18:05Z

Proposal - Check/Upgrade reserved roles from config

OpenSearch's security defaults are controlled by the ./config/.* files, including roles. This file contains many reserved roles that are updated by plugins for in OpenSearch distribution. By checking the reserved roles against the roles of the cluster we could detect that an update to the reserved roles might be needed and also apply an update resync these roles.

Updated eligible check on startup

On the security plugin start - for only the master node, read the config/roles.yml file into memory, compare that all roles on disk exist, if not then suggest an upgrade. Then check that all roles have the same permissions, if they do not then suggest an upgrade. Do this as a single warning message that includes the REST API to use to perform this upgrade - [WARN] Reserved roles do not match the defaults from ./config/roles.yml. Resync these roles by calling "POST /_plugins/_security/api/_reserved/_upgrade_apply".

Upgrade eligible check via API

Create an API "GET /_plugins/_security/api/_reserved/_upgrade_check" that can be used by tools to determine the upgrade state with the following responses.

200 OK
{
   "canUpgrade": true
   "roles": {
       "missingRoles": [
            // Full role definition
           "{ROLE-NAME}: {
              "cluster_permissions": ...
           }
       ],
       "updatedRoles": [
            ...
       ]
    }
}

Upgrade action via API

Create an API "POST /_plugins/_security/api/_reserved/_upgrade_apply" that triggers an upgrade, overrides all permissions on reserved roles and replaces them with what is found in the config/roles.yml file. Internally this will be handled the same as a PATCH request to the roles API

200 OK
{
  "status": "OK",
  "message": "Reserved roles '<role>,...' added, '<role>,...' updated."
}

Security admin tool

Add additional options to the security admin tool to include options to check this API and perform the upgrade.

cwperks · 2024-02-27T19:41:29Z

@peternied This approach looks good to me. I think it makes sense to create an API where you can check to see if your current security config has the latest version of all reserved rolls.

I know there is no current mapping between role -> plugin, but if its possible, it would also be interesting to take into account whether a specific plugin is installed and that the roles associated with the plugins exist in the security index.

peternied · 2024-03-05T01:17:32Z

FYI @cwperks draft PR willyborankin#1

…4102) ### Description This adds a new API that allows for checking and updating configurations from the default configurations on disk. Initial feature supports only Roles. #### Response when no upgrade is available ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : false } ``` #### Response when a new role is available ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ] } } } ``` #### Response when a new role + existing role were updated ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` #### Perform an upgrade ``` POST _plugins/_security/api/_upgrade_perform 200 { "status" : "OK", "upgrades" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` #### Perform an upgrade when unneeded ``` POST _plugins/_security/api/_upgrade_perform 400 { "status": "BAD_REQUEST", "message": "Unable to upgrade, no differences found in 'roles' config" } ``` ### Issues Resolved - #2316 ### Testing New unit test and integration test cases ### Check List - [X] New functionality includes testing - [ ] New functionality has been documented - [X] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). --------- Signed-off-by: Peter Nied <[email protected]> Signed-off-by: Andrey Pleskach <[email protected]> Signed-off-by: Peter Nied <[email protected]>

…pensearch-project#4102) This adds a new API that allows for checking and updating configurations from the default configurations on disk. Initial feature supports only Roles. ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : false } ``` ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ] } } } ``` ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` ``` POST _plugins/_security/api/_upgrade_perform 200 { "status" : "OK", "upgrades" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` ``` POST _plugins/_security/api/_upgrade_perform 400 { "status": "BAD_REQUEST", "message": "Unable to upgrade, no differences found in 'roles' config" } ``` - opensearch-project#2316 New unit test and integration test cases - [X] New functionality includes testing - [ ] New functionality has been documented - [X] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). --------- Signed-off-by: Peter Nied <[email protected]> Signed-off-by: Andrey Pleskach <[email protected]> Signed-off-by: Peter Nied <[email protected]> (cherry picked from commit fa877ba)

…pensearch-project#4102) ### Description This adds a new API that allows for checking and updating configurations from the default configurations on disk. Initial feature supports only Roles. #### Response when no upgrade is available ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : false } ``` #### Response when a new role is available ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ] } } } ``` #### Response when a new role + existing role were updated ``` GET _plugins/_security/api/_upgrade_check 200 { "status": "ok", "upgradeAvailable" : true, "upgradeActions" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` #### Perform an upgrade ``` POST _plugins/_security/api/_upgrade_perform 200 { "status" : "OK", "upgrades" : { "roles" : { "add" : [ "flow_framework_full_access" ], "modify" : [ "flow_framework_read_access" ] } } } ``` #### Perform an upgrade when unneeded ``` POST _plugins/_security/api/_upgrade_perform 400 { "status": "BAD_REQUEST", "message": "Unable to upgrade, no differences found in 'roles' config" } ``` ### Issues Resolved - opensearch-project#2316 ### Testing New unit test and integration test cases ### Check List - [X] New functionality includes testing - [ ] New functionality has been documented - [X] Commits are signed per the DCO using --signoff By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). --------- Signed-off-by: Peter Nied <[email protected]> Signed-off-by: Andrey Pleskach <[email protected]> Signed-off-by: Peter Nied <[email protected]>

peternied · 2024-05-02T18:50:51Z

This feature was added in [Feature] Check for and perform upgrades on security configurations #4102

peternied self-assigned this Feb 23, 2024

peternied mentioned this issue Mar 5, 2024

Security Config Upgrade willyborankin/security#1

Closed

3 tasks

This was referenced Mar 8, 2024

[Feature] Check for and perform upgrades on security configurations #4102

Merged

[DOC] Add documentation for security configuration upgrade feature opensearch-project/documentation-website#6633

Closed

peternied closed this as completed May 2, 2024

cwperks mentioned this issue Jan 6, 2025

[FEATURE] Automatically upgrade default security configs (static + reserved) on cluster upgrade #5010

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

[FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

cwperks commented Dec 9, 2022

peternied commented Dec 12, 2022

cwperks commented Dec 12, 2022

eirsep commented Feb 16, 2023 •

edited

Loading

peterzhuamazon commented May 1, 2023

peternied commented May 1, 2023

peterzhuamazon commented May 2, 2023 •

edited

Loading

peternied commented May 2, 2023

peternied commented Feb 26, 2024

cwperks commented Feb 27, 2024

peternied commented Mar 5, 2024

peternied commented May 2, 2024

[FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

[FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

Comments

cwperks commented Dec 9, 2022

peternied commented Dec 12, 2022

cwperks commented Dec 12, 2022

eirsep commented Feb 16, 2023 • edited Loading

peterzhuamazon commented May 1, 2023

peternied commented May 1, 2023

peterzhuamazon commented May 2, 2023 • edited Loading

peternied commented May 2, 2023

peternied commented Feb 26, 2024

Proposal - Check/Upgrade reserved roles from config

Updated eligible check on startup

Upgrade eligible check via API

Upgrade action via API

Security admin tool

cwperks commented Feb 27, 2024

peternied commented Mar 5, 2024

peternied commented May 2, 2024

eirsep commented Feb 16, 2023 •

edited

Loading

peterzhuamazon commented May 2, 2023 •

edited

Loading