Add Security by Default #1029
Comments
setiah
added
the
enhancement
setiah
changed the title
|
I like this very much, very good! Some comments/questions.
|
|
The OpenSearch project was initially designed for people who are using or building on top of OpenSearch. By adding the security plugin this becomes incompatible for those who were previously using the open-source ElasticSearch distributions. This breaks things for users, which is problematic. You couldn't replace ElasticSearch with OpenSearch in projects which are open source like Jaeger, Skywalking, and probably breaks things for other folks embedding the technology. I strongly suggest against this or using a flag to disable it. It's really going to be an issue for compatibility and against the main goals of this project. The plugins for ODFE should remain plugins and not become part of the base vanilla distribution. This is also how other foundational open-source projects are designed as well (OpenTelemetry for example). |
|
@jkowall @AmiStrn I think backwards compatibility and careful deprecation are a must, so let's not assume that "enabling security by default" leads to "breaks everything". I would very much want all the above-mentioned products to continue working "as is". That said, I'm curious whether you have thoughts on the competing desires to enable security by default to start preventing that long list of breaches at the source and the desire to preserve the status quo? |
|
I can guarantee you that it will break our use of the project, and the fact that there isn't a bare minimum distribution will create a lot of challenges. Similarly, Jaeger's default configuration will not work with security enabled unless you make configuration changes. I agree there are breaches when folks do not configure proper security around a database, but forcing users to adopt specific security functionality that is not part of the open-source distributions which existed before seems like a bad idea. |
|
An opt out flag (for this entire proposed module functionality) in the cluster settings is basically the only way to ensure that disregarding security would be done on purpose rather than accidentaly. If a company opts out they are then liable for any data breach due to cluster misconfiguration. Regarding auditing every use of the REST API: this is odd imo, the more users use the API the more data is stored even if they are just running searches? It sounds like that bit should be only in the security plugin since it is not required for keeping the cluster actually secure, and it is adding load to the cluster. I think the long list of breaches is not the responsibility of the maintainers. However, if an opt-out flag was added to the design (making it really easy to have no security by CHOICE) I think the list would slow down a lot. |
I did not say it would "break everything", I merely stated that it's forcing functionality on an audience who would need to figure out how to remove it which is not a good approach for open source projects. Similar to other projects (like OpenTelemetry for example) we should have a small lightweight distribution (core), and then a larger distribution (enhanced). The core distribution is designed for people embedding or building on top of the database. The enhanced distribution is designed for end users wishing to deploy it. I think both are valid use cases for the technology, and if projects like Opensearch Dashboards are better suited to run on the enhanced distribution they can specify that in the getting started docs. |
|
Thanks @jkowall, @AmiStrn for your comments. I'll try to clarify some of your concerns/questions and hope to seek some more feedback.
Minor clarification here- this is not adding security plugin to the core. The plugin would still be a separate entity that can be installed on top of min distribution.
Overridability is the key here. The inbuilt security module will be overridable (from day 1) with custom implementations to ensure a specific security functionality is not forced. (see #1039) The intention is to protect users by providing basic security features built-in, while also allowing easy controls for advanced users to override for their specific use case.
Agree that the core distribution should be lightweight and for the same reason, the proposal is to add only bare minimum security features to keep it lightweight yet secure by default. There would still be two distributions, a small lightweight (core) and larger distribution (enhanced) that comes with full security plugin.
Fair point. But would you mind shedding some light on the use-cases that seem broken with this? |
Thanks @dblock. These are some great questions which can be included in the FAQs. Let me get back on these in the FAQ section. |
|
Maybe we should be talking about adding a security interface when we talk about enabling security by default? I believe that we definitely want the full default bundle distribution to have security support for RBAC or TLS, and other standard stuff that secure products have, so people don't deploy insecure clusters by default. At the same time we don't want to make it harder to get started. I think we should add a strong security interface, and all calls should go through it, to the minimal artifact. Then you should be able to configure authentication, authorization and auditing to wildcard-allow all the things, or use a pass-through implementation. Finally, you should be able to have competing implementations of the security plugin that work together. For example, one should be able to develop and install an authentication plugin that uses ActiveDirectory, an authorization plugin that implements RBAC based on ActiveDirectory policies and groups, and auditing plugin that sends telemetry somewhere else. @AmiStrn @jkowall Another consideration for the long term - I think we want to enable such things as a cluster administrator installing a third-party plugin that advertises certain functions, get prompted to explicitly grant the plugin permission to do X, Y, and Z, but nothing else, and being confident that a plugin is not allowed to use its long hands to reach into data it's not allowed to see. |
What I was saying with that point is: if there is a simple (single) flag that I can set to The use cases that are broken with this proposal:
imo, Keeping the minimal artifact clean and simple is essential for the community to adopt this project. Lastly, Both projects are open-source so there is less margin for error: Edit: removed irrelevant argument. |
I hear what you’re suggesting and I think as long as defaults are w/ security, with a configurable way to turn it off, it should be okay. IMO what we are striving for is a mindset shift towards security in OpenSearch. Today it is "no default security, with an option to add security", while what we should to move towards is "security by default, with an option to turn it off explicitly". Interested in seeing how others think about it. Also, to your point on Audit log, the audit logging can be provided as a configurable option to only audit specific requests (say only writes, for example). This should further reduce any additional stress on cluster resources. |
Yes, the cluster administrator when overriding inbuilt-security module with their own custom implementation will get additional prompts to explicitly confirm their action. This will help safeguard against any malicious attempts by third party plugins to override security, when it was intended for some other purpose. This can be added as part of #1039 . If a plugin tries to override inbuilt-security, it would look something like - |
|
@setiah said, "the proposal is to add only bare minimum security features to keep it lightweight" IMO, auditing is not a "bare minimum". Having literally worked with 100s of organizations deploying some form of Elasticsearch/ODFE, only ONE ever cared about auditing. Those that truly have an auditing requirement can use the full security plugin. |
|
The security breaches happen for many reasons - a weak password, lack of password rotation, having a user/password not removed when the user is removed from org, are just few examples. I wonder if get into extending the basic core to keep meeting those objectives, will this take its own shape. Building a builtin security model versus a simpler plugin (or plugins for authn, EIT and logging with event filtering) is a choice the developer should consider. The existing security plugin has its complexities and those don't lend well to managing the security across a fleet of products whether managing many instances of Opensearch or many different tools and services that a security admin needs to manage. The security policy consistency is hard to achieve if the 'databases' like user db' are builtin, the built in roles change/evolve (and will evolve at different speeds across many instantiations). I would prefer if the 'security interfaces' were defined to achieve some of the goals in RFC and we have a reference set of pluggable implementations of those interfaces. Then, both the existing Security Plugin and a simpler security plugin version of the proposal could use the same interfaces. Opensearch is a tool and to make a product or service out of it requires certain scaffolding around it to manage the lifecycle, orchestration, security and monitoring. When it is all in one box appliance, it is easy to use things as a monolith. If the Openseach was used in an environment where the orchestration is cloud native (thinking of K8S, istio), and needs to scale, then the local implementations of authentication, logging streams and encryption-in-transit will need to be switched over to the ecosystem's way. Certificate expiry, automated renewal, checking for revocation, are all part of the TLS scaling and there are CNCF tools for the same but they separate the security aspect from the workload design. So, there should be a consideration for such separation. May be the existing security plugin could be changed to allow easier management and consumption? At present it is not a first class cloud native solution (thinking of SAML implementation, User DB there) and this RFC suggests it is not a best choice for an easy to use single installation. I hope the 'basic' core plugin/feature doesn't take life of its own and then we have competition between this and existing plugin. Like another comment mentioned about LDAP. Incremental changes to basic core will keep dividing the user community as well as developer community around when to switch to existing security plugin for that one extra small feature. |
💯 |
|
Apologies for getting back late on this @vikansal.
I think this is applicable not only to security, but any other feature that goes into core. Maintainers have the responsibility of keeping the core right to its purpose.
The user management APIs provide a way to manage users. But I'm not sure if I understood your concern correctly here.
Absolutely and thanks for the callout. Will lay out these details in the design (coming next) |
|
Let me summarize this discussion in key points raised so far
Thanks everyone for the feedback. We will keep this RFC issue open for one more week (until 08/24/2021) and then close this discussion. A new issue will be opened next, which would cover the design aspects and help answer some questions raised above around it. |
|
Another insecure by default cluster: https://link.medium.com/fonnHXaoSib |
This is exactly how FGAC(fain-grained access control) works.
And index-level(ILS), document-level(DLS) and field-level access control(FLS) come together with and dependent on FGAC authc and authz, and user can configure those layers of authc without having to explicitly enable them. It might make more sense to have all these built-in in OpenSearch core engine too. In comparison, external Identity provider based authentication(e.g. SAML) and compliance audit logging need users to explicitly enabled, which can be left in security plugin for users to opt in. But these details can be further discussed when this proposal goes to design stage. |
I like the idea being proposed above to secure OpenSearch minimum distribution, but this upgrade would be a huge challenge! Have any potential solutions/migration plans been under evaluated? I don't see how this would work. I'm really glad to hear if any feasible suggestions on this.
It would be nice to offer users flexibilities to configure security posture based on use cases. Having an external security plugin overriding build-in security module looks lowering the value of having build-in security module. It might cause additional configuration/setup overheads when build-in security module functionalities do not meet the requirements of use cases. In this case, users would have to do some extra configurations to overwrite the default security module. |
dblock
changed the title
|
I decided to reopen this feature and rename it to "Add Security by Default". In short, this proposal consists in shifting responsibility of securing data, including DLS/FLS into OpenSearch core, therefore removing the need for plugins to perform security functions altogether, and to eventually deprecate the security plugin. Also, check out #5834 that is working towards these goals. |
|
Thanks for re-openning this discussion. We absolutely need a better security posture for OpenSearch and as has been called out there are big concerns with how migration into this posture. I propose that we build in new primitive objects in OpenSearch that are responsible for identity related functionality, then we can start building up features that tap into these security scenarios. I have a blog post for the project website link that outlines these ideas. During the Jan 17th community meeting link I am on the agenda to introduce Identity and solicit feedback. |
|
Recognizing that moving security (or the security plumbing with pluggable specifics) into core would be a major breaking change, how receptive are folks to making that kind of change in 3.0? @reta and I had a good chat this morning in the Search backlog and triage meeting about the idea of letting I feel like "security in core" could be a big, bold 3.0 change. |
I believe adding more security related features to core is definitely a step in right direction, and some of those have been added already (like identity, secure transports, etc) |
|
A major step in the direction of making this more palatable to the community is to introduce security as a non breaking change in v2 this way in v3 we can make it so the only change is that the optional settings are no longer optional. this may require going through 2 major versions, i am unfamiliar with the details of transitioning smoothly from no security plugin to using a security plugin. |
|
I agree with @reta on a Phased approach. Secure transports in core is a good example of taking something from the security plugin and moving it into core. I can see other opportunities in the short-to-medium term as well:
|
OpenSearch inbuilt security request for comments
This document proposes to build an inbuilt security module within OpenSearch core that enforces authentication, auditing and encryption-in-transit, while making security a first-class citizen of OpenSearch 2.0. In this request for comments, we outline the problem statement, proposed solution, and areas we would like your feedback on. Please add your comments on this issue.
Problem Statement
OpenSearch is an open source search and analytics engine. Its lightning fast search, powerful analytics and data visualization capabilities have made it a great choice for a data store. It is used by enterprises and developers for their search and log analytics workloads. However, it does not come with any inbuilt access control mechanism to protect access to your confidential or business critical data. This leaves it vulnerable to security breaches and malicious cyber attacks. Also there is no trail of who accessed the cluster or what actions were executed. In fact, from a quick internet search we were able to find at least 10 critical Elasticsearch related security breaches in the past one year (see Appendix A). The frequency and severity of data breaches continues to escalate. According to a report by CyberSecurity Ventures, cybercrime will cost the global economy $10.5 trillion annually by the end of 2025. Thus, security is an absolute must have.
Today, users can enable security in OpenSearch with the security plugin. However, the security plugin comes with a full suite of advanced security features that are not required by everyone. Users need to invest time in understanding and configuring the right level of security for their use case. This could create a barrier in adoption for users and developers, who only need a basic security. Also since security is only available with plugin, this leaves out minimum distribution vulnerable with no security by default.
Proposed Solution
We propose an inbuilt security module in OpenSearch 2.0 that provides basic security features including basic authentication, auditing and encryption-in-transit. It will provide
username/password based authenticationon all REST APIs and log all user actions in anaudit log. It will enforce TLS encryption at the REST and transport layers, ensuring all client to node and node to node communication is encrypted.The inbuilt security module will include a built-in user database. It will provide user management APIs for CRUD operations on users via OpenSearch Dashboards and REST clients. Default but configurable strict password rules will ensure strong passwords. Every REST API call in OpenSearch will be authenticated against the user database and audited for record.
OpenSearch 2.0 with the inbuilt security module, will offer a simple getting started experience. It will provide tools for TLS certificate generation, setting up users and passwords, and configuring OpenSearch settings, so that users can set up their clusters in production with minimal installation steps. It will also support a developer mode for new users to run OpenSearch and use the APIs, without configuring certificates and passwords.
The inbuilt security features will be available to all OpenSearch plugins, which will make integration with security much easier. Developers can extend the basic security features for advanced use cases using a well defined set of interfaces. This will further help with a better plugin integration experience with security features leading to secure clusters.
Advanced security features such as external Identity provider based authentication, compliance audit logging, fine grained access control, Kerberos based authentication, index-level, document-level, field-level access control, etc will continue to be a part of the security plugin that can be installed on demand. This will help to keep the inbuilt security module lightweight, secure and applicable for everyone.
The inbuilt security module will use the same REST specification as OpenSearch security plugin, so we do not expect any breaking changes for your existing clients other than requiring to use HTTPS as the default communication protocol. Inbuilt security module will use a similar but lightweight authentication, auditing and encryption in transit model as used in the optional security plugin. For users who want advanced security features, there will be a seamless upgrade mechanism from the inbuilt security to the OpenSearch security plugin.
We understand that security is being handled in different ways across different organizations, and having an enforced inbuilt security model might disrupt usability with existing security plugins developed by other individuals or enterprises. To prevent this from breaking your existing security plugin, OpenSearch will support hooks for overriding the inbuilt security module with an external security plugin.
Feedback
While inbuilt security makes OpenSearch secure by default, it is a significant change to the OpenSearch minimum distribution. Please provide any feedback or comments on this issue. Some specific areas we would like your feedback on are
FAQs
What security features will the inbuilt security module provide for users?
The inbuilt security module will provide authentication, auditing and encryption in transit.
For authentication, it will support basic authentication (username/password based auth) and certificate based authentication.
For auditing, it will support auditing of all REST API actions, which means all user triggered API actions will be audited. This includes actions that are triggered through OpenSearch Dashboards.
With encryption in transit, it will support TLS encryption at both REST layer and Transport layer, which means all node to node and client to node communication will be encrypted.
These features will provide a basic layer of security to the OpenSearch minimum distribution and protect the cluster against accidental public access. We do not intend to add a basic authorization support at this point, however, it will be open for discussion and can be added in future versions if needed. Users can still enable authorization support by installing the optional OpenSearch security plugin.
What security features will the inbuilt security module provide for developers?
OpenSearch plugin developers will get basic security features, including authentication, auditing and encryption in transit, integrated with their plugin out of the box with inbuilt security module.
What is the difference between Inbuilt Security and OpenSearch security plugin? Are they two compatible if a user wants to move from one to the other?
Inbuilt security will be a core module in OpenSearch 2.0 that provides basic security, with mandatory authentication, auditing and encryption in transit support, in all OpenSearch distributions.
OpenSearch security plugin will provide more advanced security features for OpenSearch. The plugin comes pre-installed in full distribution, along with other plugins, however, it is not part of the minimum distribution.
Security plugin will be fully compatible with inbuilt security and the transition experience, from default inbuilt security to the security plugin, will be seamless.
I have my own security plugin and don’t need inbuilt security module to interfere w/ its security posture. How can I continue using it the same way as in OpenSearch 1.0?
OpenSearch will provide a mechanism to override the inbuilt security module with a custom security plugin and that would make your plugin work in the same way as it works for OpenSearch 1.0.
Can I uninstall inbuilt security?
No, inbuilt security will be a core module in OpenSearch and cannot be uninstalled. However, it will have hooks to override with a custom security plugin.
How would the out of box installation experience look different for OpenSearch 2.0?
OpenSearch 2.0 with the inbuilt security module, will offer a simple getting started experience. It will provide tools for TLS certificate generation, setting up users and passwords, and configuring OpenSearch settings, so that users can setup their clusters in production with minimal installation steps. It will also support a developer mode for new users to run OpenSearch and use the APIs, without configuring certificates and passwords.
We are open to feedback on the out of box experience you would like to see on this RFC github issue.
What actions will get logged in audit logs? Is there a default audit log truncation policy?
All REST actions will be logged in audit logs. Yes, there will be a default and configurable log rotation policy.
Appendix
Appendix A: What have been some prominent attacks/breaches on Elasticsearch?
The term Elasticsearch is never far away from the news headlines and usually for the wrong reasons. Seemingly every week that goes by brings a new story about an Elasticsearch server that has been breached, often resulting in troves of data being exposed.
Recent stories
The text was updated successfully, but these errors were encountered: