Document security scope for extensions

[peternied]

Description

While working on extensions there are many considerations for security, this document captures what those areas are as a way to discuss design approaches and codify the current state of affairs.

To approve this document means that we have agreement of areas and 'as-is' state of OpenSearch Plugins and Extensions. There are more details as to the phases of this document over time within the doc itself

Issues Resolved

• Related [META] Migrate Anomaly Detector plugin to work as an Extension #24
• Related [FEATURE] Security support for extensions  security#1895

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[peternied]

@opensearch-project/security @CEHENKLE @dblock @saratvemulapalli @dbwiddis What do you think of using a document like this for extensions security related features?

[dbwiddis]

Good first start. I like the distinction between plugin and extension up front but then I'm starting to miss the usage later on as everything seems to say "plugin". Is it documenting current state?

It may be good to organize the sections into "current state" (plugin) and "future state" (extension + select plugins) to make clear what we are scoping for in the future.

[dbwiddis]

This needs an "As is" and "To Be" summary. We are planning to do something else with User, aren't we? You're also mixing extension and plugin here, not sure if this is documenting current vs. future. I would like to put RBAC as a goal.

[peternied]

I would like to put RBAC as a goal.

Me too, I think there is a version of this document that includes lists of issues to be implemented/decisions to make. Before we come to the possible implementations I'd like to lay out the surface area for prioritization. How would you feel about holding off on individual features until we codify this framework?

[dbwiddis]

As mentioned in earlier comment, we do need an individual feature list somewhere. We can limit this particular document to the overall framework/scope but identifying at least major categories of features is needed for any actual planning and scoping of future work

[peternied]

Lets revisit this in a follow up PR to make sure we've got the detail we need

[dbwiddis]

@opensearch-project/security @CEHENKLE @dblock @saratvemulapalli @dbwiddis What do you think of using a document like this for extensions security related features?

I think a template with a subset of important decision points similar to a document like this is a good idea once this document is ironed out. This document can serve more as a guiding principle while extensions should complete the template with implementation specifics for that extension.

[peternied]

Good first start. I like the distinction between plugin and extension up front but then I'm starting to miss the usage later on as everything seems to say "plugin". Is it documenting current state?

Great idea - I'm reshaping the doc to be focused on the 'as is' as aspects, there was a couple of details that were basically recommendations in disguise.

[peternied]

@dbwiddis I have respond or made updates to all of the comments you created if you wouldn't mind taking another look.

Following the document phases, for this revision I'm trying to concentrate on landing Agreement of areas and 'as-is' state of OpenSearch Plugins and Extensions. But happy to take feedback on these phases as well. I've also updated the description to capture this

[davidlago]

This is only true for out-of-process extensions. How about the ones that can run in-process? I think we are still open to having those supported down the line

[peternied]

AFAIK in process extensions are not being designed or implemented at this time. How do you feel about not including it until this effort is more concrete?

[dbwiddis]

I think defining "REST API" is too specific for the potential application. The key point to make here is that interaction is limited to a defined set of API locations. For external extensions, they will use REST. If REST is not needed (e.g., in-process) there still should be another method of using the same API point.

[peternied]

If extensions are available in-process, it seems like we are recreating the existing plugin feature with its same architectural weaknesses. With a solid security boundary between OpenSearch and its extensions, we can build extensibility in alignment with trustworthy functionality. I think there is a discussion to be had for in-process extensions and I would prefer to wait to add those considerations to this document until we are committing more thought in this space, does this seem like a worthwhile trade-off in the near term?

I am only away of using REST APIs for extensions, if we want to broaden the language do you have a suggestion?

[dbwiddis]

Looked into this further and I'm fine with leaving this as REST API for simplicity.

[davidlago]

Do we also use OpenSearch indexes to store configuration elements too?

[peternied]

Good point, I know the security plugin built functionality for this pattern, but I'm not sure if other plugins do the same. I'll follow up on this.

[dbwiddis]

I think defining "REST API" is too specific for the potential application. The key point to make here is that interaction is limited to a defined set of API locations. For external extensions, they will use REST. If REST is not needed (e.g., in-process) there still should be another method of using the same API point.

[dbwiddis]

I've been doing some thinking about the integration of security, extensions, and versioning and think they all inter-relate a bit. Tossing the following diagram and content which you might consider including in the scope document in some way. Presently an outline, but each of the below bullet points should be fleshed out. @peternied @saratvemulapalli interested in your thoughts on this.

API Controls

All of the below interactions will use the API/SDK interaction points as the primary means of enabling/enforcing the points below.

• SDK will ensure Extensions identify themselves with a unique version which is permitted, Users authenticate appropriately, and requests are rejected if they would potentially harm the cluster

Security & Identity

• The OpenSearch cluster is protected from improper use of its resources
• Only authorized users have accesess to data that they need in their current role (RBAC)

Versioning

• All components of the OpenSearch ecosystem (core, extensions) can be uniquely identified in a way that represents their features as well as code, dependencies, etc.

Extensibility

• Code external to OpenSearch can communicate with OpenSearch via a defined API/SDK

Security + Versioning

• Individual versions of extensions can be declared unsecure (alternately, extensions must be "approved" to be allowed) due to vulnerabilities

Security + Extensibility

• Extensions need a common framework to communicate user identities to the API, including certificate stores, identity server, etc.

Extensiblity + Versioning

• Semantic Versioning and feature sets should combine to provide compatible version ranges

[saratvemulapalli]

Thanks for writing this up @peternied.

Couple of more thoughts/questions:

• How do we protect inter-extension communication?
• How can we enable extensions to expose resources, perform authentication and authorization?
• I see across the docs, the assumption is that the communication based on REST API's. So far extensions are just built on transport(TCP). For security, if REST APIs are preferred we could weigh in pros and cons.
• I see this doc scope as security for plugins and extensions. What are we thinking for Security for OpenSearch?
• When we define scope, i believe it would be the time to define tenets we would want to stick to.
• We did close: [FEATURE] Security support for extensions  security#1895 in favor for this PR. I dont see the tactical items on the plate to execute for supporting AD as an extension. How do you want to take on this?

[peternied]

Great questions @saratvemulapalli thanks for the feedback.

How do we protect inter-extension communication?

This sounds like a future-feature request. For the moment, proxied via OpenSearch seems like the path, but there is much potential in this space. When we consider adding this functionality I think it warrants an in-depth discussion. Should we file an issue on this now to track, or is there another document that captures this? I'm leaning on having this come back up organically instead of creating an issue we forget about.

How can we enable extensions to expose resources, perform authentication and authorization?

I'm not sure that we would implement all of these in this SDK. For example in the identity space there is Open ID, I do not think that functionality would below in this SDK, as implementing the standard in a native plugin might be the better way.

I see across the docs, the assumption is that the communication based on REST API's. So far extensions are just built on transport(TCP). For security, if REST APIs are preferred we could weigh in pros and cons.

If we want to be security focused with extensions, I think we need to strongly consider moving to only communicate via REST API. While there are likely trade-offs from existing functionality/perf, there are huge benefits to quickly onboard simple extensions written in python, rust, or golang. Additionally, by limited communicates with an ephemeral connections we can be sure that authorization is granted at the time of a given request. It creates a well defined trust boundary. I've created #34 for additional discussion, as well as updating this doc in the COMSEC section.

I see this doc scope as security for plugins and extensions. What are we thinking for Security for OpenSearch?

This is a good callout, I'll start an effort to connect OpenSearch / Security as well as this doc together.

When we define scope, i believe it would be the time to define tenets we would want to stick to.

I like this in principle, but I am struggling to see how we could roll this out. I don't think we have existing engineering tenets in OpenSearch, this might be exactly the time to create them. This said I think that those will be documented in the OpenSearch codebase and I'd like to see about merging this change without them, how does that occur to you?

We did close: opensearch-project/security#1895 in favor for this PR. I dont see the tactical items on the plate to execute for supporting AD as an extension. How do you want to take on this?

Thanks for bringing this up, after I merge this version of the document, there should be a fast follow for calling out the investigations and work that needs to be done to support Anomaly Detections feature set. I'll reopen that issue and close it when this has been done.

[peternied]

@dbwiddis @saratvemulapalli how does this look in its current form?

[dbwiddis]

@dbwiddis @saratvemulapalli how does this look in its current form?

I'm happy with it in it's current form. We may tweak a few things in the future but it's a good working document at this point.

Good questions regarding REST and auth/security. I think it's a solvable problem, and all these are implementation details rather than scope which the doc outlines.

• We can use SSL/TLS for the REST APIs to encrypt and then use (initially) the existing security mechanisms (user/password, certificate based) while also enabling (future) identity/RBAC capabilities.
  • May consider whether to make this user configurable, e.g., force https by default/allow insecure opt-in.
• In-process comms can still use the REST API; localhost avoids any network traffic
• Inter-process comms we'll need to figure out how to register availability, push vs. pull, robustness to failures, etc.