-
Notifications
You must be signed in to change notification settings - Fork 58
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rename everything from opensearch-sdk to opensearch-sdk-java #96
Changes from 10 commits
43f176e
707402d
a0ce958
7e270b2
ce5c1f2
39f476f
fc9c63d
a0a7d02
ab15318
0e54394
46cd1ca
01dcca3
884119e
59a47a1
463cdba
6f7084d
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,2 @@ | ||
# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams | ||
* @opensearch-project/opensearch-sdk | ||
* @opensearch-project/opensearch-sdk-java | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
- [Contributing to OpenSearch-SDK](#contributing-to-opensearch-sdk) | ||
- [Contributing to OpenSearch-SDK-JAVA](#contributing-to-opensearch-sdk-java) | ||
- [First Things First](#first-things-first) | ||
- [Ways to Contribute](#ways-to-contribute) | ||
- [Bug Reports](#bug-reports) | ||
|
@@ -7,13 +7,13 @@ | |
- [Developer Certificate of Origin](#developer-certificate-of-origin) | ||
- [Review Process](#review-process) | ||
|
||
# Contributing to OpenSearch-SDK | ||
# Contributing to OpenSearch-SDK-JAVA | ||
|
||
OpenSearch-SDK is a community project that is built and maintained by people just like **you**. We're glad you're interested in helping out. There are several different ways you can do it, but before we talk about that, let's talk about how to get started. | ||
OpenSearch-SDK-JAVA is a community project that is built and maintained by people just like **you**. We're glad you're interested in helping out. There are several different ways you can do it, but before we talk about that, let's talk about how to get started. | ||
|
||
## First Things First | ||
|
||
1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk/issues). | ||
1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues). | ||
|
||
2. **Only submit your own work** (or work you have sufficient rights to submit) - Please make sure that any code or documentation you submit is your work or you have the rights to submit. We respect the intellectual property rights of others, and as part of contributing, we'll ask you to sign your contribution with a "Developer Certificate of Origin" (DCO) that states you have the rights to submit this work and you understand we'll use your contribution. There's more information about this topic in the [DCO section](#developer-certificate-of-origin). | ||
|
||
|
@@ -23,17 +23,17 @@ OpenSearch-SDK is a community project that is built and maintained by people jus | |
|
||
Ugh! Bugs! | ||
|
||
A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk). | ||
A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk-java). | ||
|
||
Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk/issues). | ||
Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk-java/issues). | ||
|
||
### Feature Requests | ||
|
||
If you've thought of a way that OpenSearch-SDK could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work. | ||
If you've thought of a way that OpenSearch-SDK-JAVA could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work. | ||
|
||
### Contributing Code | ||
|
||
As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue. | ||
As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue. | ||
|
||
Once you've opened an issue, check out our [Developer Guide](./DEVELOPER_GUIDE.md) for instructions on how to get started. | ||
|
||
|
@@ -68,7 +68,7 @@ By making a contribution to this project, I certify that: | |
consistent with this project or the open source license(s) | ||
involved. | ||
``` | ||
We require that every contribution to OpenSearch-SDK is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms. | ||
We require that every contribution to OpenSearch-SDK-JAVA is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "Developer Certificate of Origin" isn't clear to new contributors. This needs to be linked to better documentation. I'd suggest linking to the main project |
||
|
||
Each commit must include a DCO which looks like this | ||
|
||
|
@@ -79,7 +79,7 @@ You may type this line on your own when writing your commit messages. However, i | |
|
||
## Review Process | ||
|
||
We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction. | ||
We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction. | ||
|
||
It is necessary to request at least two contributors to review your pull request. It is required to have two approvals in order for your pull request to be accepted. | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -9,7 +9,7 @@ i.e critical workloads like ingestion/search traffic would be impacted because o | |
This problem is exponentially grows when we would like to run a 3rd Party plugin from the community. | ||
As OpenSearch and plugins run in the same process, it brings in security risk, dependency conflicts and reduces the velocity of releases. | ||
|
||
Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK. | ||
Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK JAVA. | ||
|
||
Meta Issue: [Steps to make OpenSearch extensible](https://github.com/opensearch-project/OpenSearch/issues/2447) | ||
Sandboxing: [Step towards modular architecture in OpenSearch](https://github.com/opensearch-project/OpenSearch/issues/1422) | ||
|
@@ -32,7 +32,7 @@ During the bootstrap of OpenSearch node, it class loads all the code under `~/pl | |
|
||
![](Docs/Extensions.png) | ||
|
||
Extensions are independent processes which are built using `opensearch-sdk`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes. | ||
Extensions are independent processes which are built using `opensearch-sdk-java`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes. | ||
Extensions are designed to extend features via transport APIs which are exposed using extension points of OpenSearch. | ||
|
||
### Discovery | ||
|
@@ -41,8 +41,8 @@ Here is an example extension configuration `extensions.yml`: | |
|
||
``` | ||
extensions: | ||
- name: opensearch-sdk // extension name | ||
uniqueId: opensearch-sdk-1 // identifier for the extension | ||
- name: opensearch-sdk-java // extension name | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can we please make this There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. fixed |
||
uniqueId: opensearch-sdk-java-1 // identifier for the extension | ||
hostName: 'sdk_host' // name of the host where extension is running | ||
hostAddress: '127.0.0.1' // host to reach | ||
port: '4532' // port to reach | ||
|
@@ -55,7 +55,7 @@ extensions: | |
### Communication | ||
As we are running extensions on the port defined in the `extensions.yml`, the communication between OpenSearch and Extensions happens using a ServerSocket which binds the port and the host address. OpenSearch will initialize the extensions during the bootstrap by making a request to all the extensions running on different ports and thus creating a medium for the future requests. | ||
|
||
### OpenSearch SDK | ||
### OpenSearch SDK JAVA | ||
Currently, plugins relies on extension points to communicate with OpenSearch. To turn plugins into extensions, all the extension points should be converted into Transport APIs which will be present in the SDK. Plugins need to integrate SDK, call those APIs, and later SDK will take care of the communication and the required attributes from OpenSearch. | ||
|
||
### Settings | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,11 @@ | ||
|
||
# OpenSearch SDK Developer Guide | ||
# OpenSearch SDK JAVA Developer Guide | ||
- [Introduction](#introduction) | ||
- [Getting Started](#getting-started) | ||
- [Git Clone OpenSearch-SDK Repo](#git-clone-OpenSearch-SDK-repo) | ||
- [Git Clone OpenSearch-SDK-JAVA Repo](#git-clone-OpenSearch-SDK-JAVA-repo) | ||
- [Git Clone OpenSearch Repo](#git-clone-opensearch-repo) | ||
- [Publish OpenSearch Feature/Extensions branch to Maven local](#publish-opensearch-feature/extensions-branch-to-maven-local) | ||
- [Run OpenSearch-SDK](#run-opensearch-sdk) | ||
- [Run OpenSearch-SDK-JAVA](#run-opensearch-sdk-java) | ||
- [Create extensions.yml file](#create_extensions_file) | ||
- [Run OpenSearch](#run-opensearch) | ||
- [Run Tests](#run-tests) | ||
|
@@ -18,14 +18,14 @@ Read more about extensibility [here](https://github.com/opensearch-project/OpenS | |
|
||
## Getting Started | ||
|
||
### Git Clone OpenSearch SDK Repo | ||
Fork [OpenSearch SDK](https://github.com/opensearch-project/opensearch-sdk) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk.git`. | ||
### Git Clone OpenSearch SDK JAVA Repo | ||
Fork [OpenSearch SDK JAVA](https://github.com/opensearch-project/opensearch-sdk-java) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk-java.git`. | ||
|
||
### Git Clone OpenSearch Repo | ||
Fork [OpenSearch](https://github.com/opensearch-project/OpenSearch/), checkout feature/extensions branch, and clone locally, e.g. `git clone https://github.com/[your username]/OpenSearch.git`. | ||
|
||
## Publish OpenSearch feature/extensions Branch to Maven local | ||
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running the OpenSearch SDK on a seperate process. | ||
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running the OpenSearch SDK JAVA on a seperate process. | ||
|
||
- First navigate to the directory that OpenSearch has been cloned to | ||
- Checkout the correct branch, e.g. `git checkout feature/extensions`. | ||
|
@@ -34,9 +34,9 @@ The work done to support the extensions framework is located on the `feature/ext | |
|
||
It is necessary to publish dependencies to a local maven repository until this branch is merged to `main`, at which point all dependencies will be published to Maven central. | ||
|
||
## Run OpenSearch SDK | ||
## Run OpenSearch SDK JAVA | ||
|
||
Navigate to the directory that OpenSearch-SDK has been cloned to and run main script using `./gradlew run`. | ||
Navigate to the directory that OpenSearch-SDK-JAVA has been cloned to and run main script using `./gradlew run`. | ||
|
||
``` | ||
./gradlew run | ||
|
@@ -73,8 +73,8 @@ Sample extensions.yml file: | |
|
||
``` | ||
extensions: | ||
- name: opensearch-sdk | ||
uniqueId: opensearch-sdk-1 | ||
- name: opensearch-sdk-java | ||
uniqueId: opensearch-sdk-java-1 | ||
hostName: 'sdk_host' | ||
hostAddress: '127.0.0.1' | ||
port: '4532' | ||
|
@@ -83,7 +83,7 @@ extensions: | |
opensearchVersion: '3.0.0' | ||
javaVersion: '14' | ||
className: ExtensionsRunner | ||
customFolderName: opensearch-sdk | ||
customFolderName: opensearch-sdk-java | ||
hasNativeController: false | ||
``` | ||
|
||
|
@@ -92,7 +92,7 @@ extensions: | |
- Return to the OpenSearch directory by using `cd ..`. | ||
- Start OpenSearch feature/extensions branch using `./bin/opensearch`. | ||
|
||
During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to the OpenSearch SDK and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on. | ||
During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to the OpenSearch SDK JAVA and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on. | ||
|
||
``` | ||
[2022-06-16T21:30:18,857][INFO ][o.o.t.TransportService ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} | ||
|
@@ -102,7 +102,7 @@ During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the ext | |
[2022-06-16T21:30:19,000][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] received PluginResponse{examplepluginname} | ||
``` | ||
|
||
OpenSearch SDK terminal will also log all requests and responses it receives from OpenSearch : | ||
OpenSearch SDK JAVA terminal will also log all requests and responses it receives from OpenSearch : | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Java There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed |
||
|
||
TCP HandShake Request : | ||
``` | ||
|
@@ -156,7 +156,7 @@ MESSAGE RECEIVED:ES-ǣ!internal:discovery/extensionsnode_extensionQSt9oKXFTSWqgX | |
21:30:18.999 [opensearch[extension][transport_worker][T#6]] TRACE org.opensearch.transport.TransportService.tracer - [3][internal:discovery/extensions] sent response | ||
``` | ||
|
||
It is important that the OpenSearch SDK is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK will now be able to comminicate. | ||
It is important that the OpenSearch SDK JAVA is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK JAVA is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK JAVA will now be able to comminicate. | ||
|
||
## Run Tests | ||
|
||
|
@@ -165,7 +165,7 @@ Run tests : | |
./gradlew clean build test | ||
``` | ||
## Generate Artifact | ||
In opensearch-sdk navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one: | ||
In opensearch-sdk-java navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should be a |
||
``` | ||
./gradlew clean && ./gradlew build | ||
``` | ||
|
@@ -174,11 +174,11 @@ Look for tar ball in /build/distributions. To run the artifact i.e. tar ball, ru | |
``` | ||
tar -xvf opensearch-sdk-1.0.0-SNAPSHOT.tar | ||
``` | ||
TODO https://github.com/opensearch-project/opensearch-sdk/issues/52 | ||
Navigate to opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/ | ||
TODO https://github.com/opensearch-project/opensearch-sdk-java/issues/52 | ||
Navigate to opensearch-sdk-java/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/ | ||
- Check if src folder exists in using `ls`. | ||
- If the src folder does not exist, create it using `mkdir src && cd src && mkdir test && cd test && mkdir resources && cd resources`. | ||
The above command will generate a path opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/src/test/resources | ||
The above command will generate a path opensearch-sdk-java/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/src/test/resources | ||
- Manually create a file titled `extension.yml` within the resources directory using an IDE or an in-line text editor. Below is the sample of extension.yml | ||
dbwiddis marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Sample extensions.yml file: | ||
|
@@ -187,7 +187,7 @@ Sample extensions.yml file: | |
hostAddress: 127.0.0.1 | ||
hostPort: 4532 | ||
``` | ||
- After extension.yml is generated then start opensearch-sdk by ./bin/opensearch-sdk | ||
- After extension.yml is generated then start opensearch-sdk-java by ./bin/opensearch-sdk-java | ||
|
||
## Submitting Changes | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,4 @@ | ||
# OpenSearch SDK | ||
# OpenSearch SDK JAVA | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No need to SHOUT :) Java There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed |
||
- [Introduction](#introduction) | ||
- [Contributing](#contributing) | ||
- [Code of Conduct](#code-of-conduct) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did change the name of the team from
opensearch-sdk
toopensearch-sdk-java