Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature/sandboxing branch]Add initial doc for plugin sandboxing #1582

Closed
wants to merge 54 commits into from
Closed

[feature/sandboxing branch]Add initial doc for plugin sandboxing #1582

wants to merge 54 commits into from

Conversation

saratvemulapalli
Copy link
Member

Description

Adding initial doc for plugin sandboxing.

Issues Resolved

Part of #1422

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

adnapibar and others added 30 commits October 26, 2021 09:45
#1404)

This commit adds a method that can be used to provide a custom TranslogDeletionPolicy
from within plugins that implement the EnginePlugin interface. This enables plugins to
provide a custom deletion policy with the current limitation that only one plugin can
override the policy. An exception will be thrown if more than one plugin overrides the
policy.

Signed-off-by: Nicholas Walter Knize <[email protected]>
When creating the second instance of an InternalEngine using the same
translog config of the default InternalEngine instance, the second
instance will attempt to delete all the existing translog files. I found
a deterministic test failure when running with the seed
`E3E6AAD95ABD299B`.

As opposed to creating a second engine instance with a different
translog location, just close the first one before creating the second.

Signed-off-by: Andrew Ross <[email protected]>
This commit upgrades to the latest release of lucene 8.10

Signed-off-by: Nicholas Walter Knize <[email protected]>
…n lease. (#1416)

The settings and the corresponding logic for translog pruning by retention lease which were added as part of #1100 have been deprecated. This commit removes those deprecated code in favor of an extension point for providing a custom TranslogDeletionPolicy.

Signed-off-by: Rabi Panda <[email protected]>
This commit upgrades the BWC 1.2 lucene dependency to version 8.10.1

Signed-off-by: Nicholas Walter Knize <[email protected]>
As part of the commit 2ebd0e0, we added a new method to the EnginePlugin to provide a 
custom TranslogDeletionPolicy. This commit makes minTranslogGenRequired method 
abstract in this class for implementation by child classes. The default implementation 
is provided by DefaultTranslogDeletionPolicy.

Signed-off-by: Rabi Panda <[email protected]>
This change updates the logic in RunTask to use deterministic port values for all nodes rather than just the first node in a cluster. The local address of the first node is also used as the seed_hosts value to allow the cluster to form correctly. The seed_hosts setting is also allow-listed as an overrideable value.

Signed-off-by: Kartik Ganesh <[email protected]>
* Enabling spotless, disabling checkstyle on below modules

:plugins:mapper-annotated-text
:plugins:mapper-murmur3
:plugins:mapper-size
:plugins:repository-azure
:plugins:repository-gcs
:plugins:repository-hdfs
:plugins:repository-s3
:plugins:store-smb
:plugins:transport-nio
:qa:die-with-dignity

Signed-off-by: Himanshu Setia <[email protected]>

* Enabling spotless for more plugins

Signed-off-by: Himanshu Setia <[email protected]>

* Fixing error in merge conflict

Signed-off-by: Himanshu Setia <[email protected]>
* Added logic to allow {dot} files on startup

Signed-off-by: Ryan Bogan <[email protected]>

* Ensures that only plugin directories are returned by findPluginDirs()

Signed-off-by: Ryan Bogan <[email protected]>

* Prevents . files from being returned as plugins

Signed-off-by: Ryan Bogan <[email protected]>
Signed-off-by: Nicholas Walter Knize <[email protected]>
I observed a [test failure][1] that I believe was caused by the fact
that an OpenSearch server happened to be running at localhost:9200 when
these unit tests were executed. The code under test has logic to try to
connect to localhost:9200 and then fall back to defaults if that port is
not open. The tests expect these defaults and will fail if different
data is returned. The change here is to use a non-default port to make
it very unlikely that a real instance will be running at the non-default
port. I don't know why an OpenSearch service happened to be running
during the linked test failure, but I think making these unit tests more
independent and isolated is helpful no matter the underlying cause in
this case.

[1]: https://fork-jenkins.searchservices.aws.dev/job/OpenSearch_CI/job/PR_Checks/job/Gradle_Check/975/artifact/gradle_check_975.log/*view*/

Signed-off-by: Andrew Ross <[email protected]>
* Made changes.

Signed-off-by: Megha Sai Kavikondala <[email protected]>

* Signed-off-by: Megha Sai Kavikondala <[email protected]>

Changes made by deleting the TestSettingsIT file and adding new lines in FullClusterRestartSettingsUpgradeIT.java
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 1cc7268
Log 1129

Reports 1129

@owaiskazi19
Copy link
Member

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 1cc7268
Log 1132

Reports 1132

@saratvemulapalli
Copy link
Member Author

Looks like its because of changes missing in feature branch vs main.
I'll merge the changes first from main.

andrross and others added 13 commits November 19, 2021 13:37
I observed a test failure with the message
'Attempted to append to non-started appender mock' from an assertion in
`OpenSearchTestCase::after`. I believe this indicates that a
MockLogAppender (which is named "mock") was added as an appender to the
static logging context and some other test in the same JVM happened to
cause a logging statement to hit that appender and cause an error, which
then caused an unrelated test to fail (because they share static state
with the logger). Almost all usages of MockLogAppender start it
immediately after creation. I found a few that did not and fixed those.
I also made a static helper in MockLogAppender to start it upon
creation.

Signed-off-by: Andrew Ross <[email protected]>
The 1.1 release notes are missing a set of commits. This updates the release notes
to the commit specified in our relase 15e9f13.

Signed-off-by: Marc Handalian <[email protected]>
…ndex and reindex requests (#1533)

Signed-off-by: Jan Baudisch <[email protected]>

Co-authored-by: Jan Baudisch <[email protected]>
…#1568)

* Signed-off-by: Megha Sai Kavikondala <[email protected]>

Changes related to Informative error messages.

* Signed-off-by: Megha Sai Kavikondala <[email protected]>

Spotless changes

* Signed-off-by: Megha Sai Kavikondala <[email protected]>

naming and message changes.

Signed-off-by: Megha Sai Kavikondala <[email protected]>

* Signed-off-by: Megha Sai Kavikondala <[email protected]>

Deleting the log message.
* Adding Github action for auto backport PR creation

Signed-off-by: Vacha <[email protected]>

* Adding details to DEVELOPER_GUIDE

Signed-off-by: Vacha <[email protected]>
* Rename field_masking_span to span_field_masking

Signed-off-by: Xue Zhou <[email protected]>

* Update SearchModuleTests.java

Signed-off-by: Xue Zhou <[email protected]>

* Rename field_masking_span to span_field_masking

Signed-off-by: Xue Zhou <[email protected]>
Signed-off-by: Andriy Redko <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Wrapper Validation failure d5de152d6cfd0911deb7ae18c2bd23165de32bc0

:alert: Gradle Wrapper integrity has been altered

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success 4749793

@saratvemulapalli
Copy link
Member Author

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success d5de152d6cfd0911deb7ae18c2bd23165de32bc0

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success 4749793

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure d5de152d6cfd0911deb7ae18c2bd23165de32bc0
Log 1291

Reports 1291

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 4749793
Log 1292

Reports 1292

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4749793
Log 1293

Reports 1293

@saratvemulapalli
Copy link
Member Author

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 4749793
Log 1294

Reports 1294

@saratvemulapalli
Copy link
Member Author

@owaiskazi19 DCO is failing because the number of commits is high and out of bounds for the check.
Once you review it, i'll merge it. Sequential commits should be fine.

@saratvemulapalli
Copy link
Member Author

Cancelling this and creating a new feature branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.