[Feature] Check for and perform upgrades on security configurations

[peternied]

Description

This adds a new API that allows for checking and updating configurations from the default configurations on disk. Initial feature supports only Roles.

Response when no upgrade is available

GET _plugins/_security/api/_upgrade_check
200 {
  "status": "ok",
  "upgradeAvailable" : false
}

Response when a new role is available

GET _plugins/_security/api/_upgrade_check
200 {
  "status": "ok",
  "upgradeAvailable" : true,
  "upgradeActions" : {
    "roles" : {
      "add" : [ "flow_framework_full_access" ]
    }
  }
}

Response when a new role + existing role were updated

GET _plugins/_security/api/_upgrade_check
200 {
  "status": "ok",
  "upgradeAvailable" : true,
  "upgradeActions" : {
    "roles" : {
      "add" : [ "flow_framework_full_access" ],
      "modify" : [ "flow_framework_read_access" ]
    }
  }
}

Perform an upgrade

POST _plugins/_security/api/_upgrade_perform
200 {
  "status" : "OK",
  "upgrades" : {
    "roles" : {
      "add" : [ "flow_framework_full_access" ],
      "modify" : [ "flow_framework_read_access" ]
    }
  }
}

Perform an upgrade when unneeded

POST _plugins/_security/api/_upgrade_perform
400 {
   "status": "BAD_REQUEST",
   "message": "Unable to upgrade, no differences found in 'roles' config"
}

Issues Resolved

• [FEATURE] New reserved roles are not sourced to security index on an upgrade #2316

Testing

New unit test and integration test cases

Check List

• New functionality includes testing
• New functionality has been documented
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[cwperks]

Thank you @peternied. This PR looks good to me. I left one minor comment on the integration test about strengthening the assertions post upgrade.

[stephen-crawford]

Overall looks good. Just two minor comments

[cwperks]

Thank you for adding an additional assertion. This looks good to me.

[peternied]

@DarshitChanpura @cwperks @scrawfor99 I've resolved all comment threads, could you take another look?

[opensearch-trigger-bot]

The backport to 2.13 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/security/backport-2.13 2.13
# Navigate to the new working tree
pushd ../.worktrees/security/backport-2.13
# Create a new branch
git switch --create backport/backport-4102-to-2.13
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 fa877babe3dac13c30bcc2bcbe4d484bcdb6101f
# Push it to GitHub
git push --set-upstream origin backport/backport-4102-to-2.13
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/security/backport-2.13

Then, create a pull request where the base branch is 2.13 and the compare/head branch is backport/backport-4102-to-2.13.

[opensearch-trigger-bot]

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/security/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/security/backport-2.x
# Create a new branch
git switch --create backport/backport-4102-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 fa877babe3dac13c30bcc2bcbe4d484bcdb6101f
# Push it to GitHub
git push --set-upstream origin backport/backport-4102-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/security/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4102-to-2.x.