Fix data-stream name resolution for wild-cards

[sandeshkr419]

Signed-off-by: Sandesh Kumar [email protected]

Description

[Describe what this change achieves]

• Category: Bug fix

• Security plugin is not able to invalidate non-permitted data-stream (get/delete/stats) requests sent using wild-card expressions.
  This is because the list of 'allIndices' in 'ResolvedIndicesProvider' class does not has the resolved names of data streams. If the 'allIndices' variable when resolved, is empty -> leads the authorization to succeed as there are no eligible index patterns to block the request.
  In this change, we add the the names of resolved data streams to 'allIndices' so data stream names also can get resolved.

• Additional Information: This PR (code changes) was already reviewed and merged in 1.3 branch but was reverted back because 1.3 branch was finalized.
  Refer: Fix data-stream name resolution for wild-cards #1716

• What is the old behavior before changes and new behavior after changes?

Suppose the indices are as follows in a OS cluster.

kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_cat/indices?v' --insecure -u 'admin:admin'
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_cat/indices?v' --insecure -u 'admin:admin'
health status index                        uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   .ds-logs-nginx1-000001       3V1VOJ4NQ0KEqtrJ51el8g   1   1          0            0       208b           208b
yellow open   logs2                        AwHqtjeITBGumadYv5GNuQ   1   1          0            0       208b           208b
yellow open   .ds-logs-nginx11-000001      lnMe0uUPToKbQKJCrMUoSw   1   1          0            0       208b           208b
yellow open   logs1                        oR5D0gWFT_u5lbpvSi-SOA   1   1          0            0       208b           208b
yellow open   security-auditlog-2022.03.28 YknIcv-GQLGtdocWUA6-UA   1   1         31            0     94.8kb         94.8kb
yellow open   .ds-logs-nginx3-000001       E2fpfxHSSMarY84WMIjhhQ   1   1          0            0       208b           208b
green  open   .opendistro_security         h6L4aW-QR5KMrCui6BL12g   1   0          9            0     87.9kb         87.9kb

Assume the user sandesh to have the following data-streams related permissions:

kusandes@a483e789b681 ~ % curl -XPUT https://localhost:9200/_plugins/_security/api/roles/ds_all -u 'admin:admin' --insecure -H 'Content-Type: application/json' -d '
{
  "cluster_permissions": [ ],
  "index_permissions": [{
    "index_patterns": [
      "logs-nginx1*"
    ],
    "dls": "",
    "fls": [],
    "masked_fields": [],
    "allowed_actions": [
      "indices:admin/data_stream/get",
      "indices:admin/data_stream/delete",
      "indices:monitor/data_stream/stats"
    ]
  }],
  "tenant_permissions": [{
    "tenant_patterns": [
      "human_resources"
    ],
    "allowed_actions": [
    ]
  }]
}
'

The ideal (expected) behavior is that user sandesh should not be able to access data-streams other than logs-nginx1 & logs-nginx11.

Old Behaviour:

kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_data_stream/logs-nginx*?pretty' --insecure -u 'sandesh:kumar'
{
  "data_streams" : [
    {
      "name" : "logs-nginx1",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx1-000001",
          "index_uuid" : "3V1VOJ4NQ0KEqtrJ51el8g"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    },
    {
      "name" : "logs-nginx11",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx11-000001",
          "index_uuid" : "lnMe0uUPToKbQKJCrMUoSw"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    },
    {
      "name" : "logs-nginx3",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx3-000001",
          "index_uuid" : "E2fpfxHSSMarY84WMIjhhQ"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    }
  ]
}

New Behaviour:

kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_data_stream/logs-nginx*?pretty' --insecure -u 'sandesh:kumar'
{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "no permissions for [indices:admin/data_stream/get] and User [name=sandesh, backend_roles=[], requestedTenant=null]"
      }
    ],
    "type" : "security_exception",
    "reason" : "no permissions for [indices:admin/data_stream/get] and User [name=sandesh, backend_roles=[], requestedTenant=null]"
  },
  "status" : 403
}

kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_data_stream/logs-nginx1*?pretty' --insecure -u 'sandesh:kumar'
{
  "data_streams" : [
    {
      "name" : "logs-nginx1",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx1-000001",
          "index_uuid" : "3V1VOJ4NQ0KEqtrJ51el8g"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    },
    {
      "name" : "logs-nginx11",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx11-000001",
          "index_uuid" : "lnMe0uUPToKbQKJCrMUoSw"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    }
  ]
}

{
  "error" : {
    "root_cause" : [
      {
        "type" : "security_exception",
        "reason" : "no permissions for [indices:admin/data_stream/get] and User [name=sandesh, backend_roles=[], requestedTenant=null]"
      }
    ],
    "type" : "security_exception",
    "reason" : "no permissions for [indices:admin/data_stream/get] and User [name=sandesh, backend_roles=[], requestedTenant=null]"
  },
  "status" : 403
}

kusandes@a483e789b681 ~ % curl -XGET 'https://localhost:9200/_data_stream/logs-nginx1,logs-nginx11?pretty' --insecure -u 'sandesh:kumar'
{
  "data_streams" : [
    {
      "name" : "logs-nginx1",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx1-000001",
          "index_uuid" : "3V1VOJ4NQ0KEqtrJ51el8g"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    },
    {
      "name" : "logs-nginx11",
      "timestamp_field" : {
        "name" : "@timestamp"
      },
      "indices" : [
        {
          "index_name" : ".ds-logs-nginx11-000001",
          "index_uuid" : "lnMe0uUPToKbQKJCrMUoSw"
        }
      ],
      "generation" : 1,
      "status" : "YELLOW",
      "template" : "logs-template"
    }
  ]
}

Issues Resolved

#1498

Is this a backport? If so, please add backport PR # and/or commits #

Testing

• Tested the above behaviour/fix manually with get/delete/stats data stream api.
• Added integration tests for the same to verify.

Check List

• [Y] New functionality includes testing
• [Y] New functionality has been documented
• [Y] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1723 (f1404eb) into main (978c71a) will decrease coverage by 0.00%.
The diff coverage is 60.00%.

@@             Coverage Diff              @@
##               main    #1723      +/-   ##
============================================
- Coverage     62.88%   62.88%   -0.01%     
+ Complexity     3265     3264       -1     
============================================
  Files           253      253              
  Lines         18097    18102       +5     
  Branches       3246     3247       +1     
============================================
+ Hits          11380    11383       +3     
- Misses         5063     5064       +1     
- Partials       1654     1655       +1     

Impacted Files	Coverage Δ
...earch/security/resolver/IndexResolverReplacer.java	65.52% <60.00%> (+0.49%)	⬆️
...a/org/opensearch/security/tools/SecurityAdmin.java	45.55% <0.00%> (-0.25%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 978c71a...f1404eb. Read the comment docs.