Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename everything from opensearch-sdk to opensearch-sdk-java #96

Merged
merged 16 commits into from
Aug 23, 2022
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
* @opensearch-project/opensearch-sdk
* @opensearch-project/opensearch-sdk-java
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did change the name of the team from opensearch-sdk to opensearch-sdk-java

20 changes: 10 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
- [Contributing to OpenSearch-SDK](#contributing-to-opensearch-sdk)
- [Contributing to OpenSearch-SDK-JAVA](#contributing-to-opensearch-sdk-java)
- [First Things First](#first-things-first)
- [Ways to Contribute](#ways-to-contribute)
- [Bug Reports](#bug-reports)
Expand All @@ -7,13 +7,13 @@
- [Developer Certificate of Origin](#developer-certificate-of-origin)
- [Review Process](#review-process)

# Contributing to OpenSearch-SDK
# Contributing to OpenSearch-SDK-JAVA

OpenSearch-SDK is a community project that is built and maintained by people just like **you**. We're glad you're interested in helping out. There are several different ways you can do it, but before we talk about that, let's talk about how to get started.
OpenSearch-SDK-JAVA is a community project that is built and maintained by people just like **you**. We're glad you're interested in helping out. There are several different ways you can do it, but before we talk about that, let's talk about how to get started.

## First Things First

1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk/issues).
1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues).

2. **Only submit your own work** (or work you have sufficient rights to submit) - Please make sure that any code or documentation you submit is your work or you have the rights to submit. We respect the intellectual property rights of others, and as part of contributing, we'll ask you to sign your contribution with a "Developer Certificate of Origin" (DCO) that states you have the rights to submit this work and you understand we'll use your contribution. There's more information about this topic in the [DCO section](#developer-certificate-of-origin).

Expand All @@ -23,17 +23,17 @@ OpenSearch-SDK is a community project that is built and maintained by people jus

Ugh! Bugs!

A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk).
A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk-java).

Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk/issues).
Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk-java/issues).

### Feature Requests

If you've thought of a way that OpenSearch-SDK could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work.
If you've thought of a way that OpenSearch-SDK-JAVA could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work.

### Contributing Code

As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue.
As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue.

Once you've opened an issue, check out our [Developer Guide](./DEVELOPER_GUIDE.md) for instructions on how to get started.

Expand Down Expand Up @@ -68,7 +68,7 @@ By making a contribution to this project, I certify that:
consistent with this project or the open source license(s)
involved.
```
We require that every contribution to OpenSearch-SDK is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms.
We require that every contribution to OpenSearch-SDK-JAVA is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Developer Certificate of Origin" isn't clear to new contributors. This needs to be linked to better documentation.

I'd suggest linking to the main project CONTRIBUTING doc:
https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin


Each commit must include a DCO which looks like this

Expand All @@ -79,7 +79,7 @@ You may type this line on your own when writing your commit messages. However, i

## Review Process

We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction.
We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction.

It is necessary to request at least two contributors to review your pull request. It is required to have two approvals in order for your pull request to be accepted.

Expand Down
10 changes: 5 additions & 5 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ i.e critical workloads like ingestion/search traffic would be impacted because o
This problem is exponentially grows when we would like to run a 3rd Party plugin from the community.
As OpenSearch and plugins run in the same process, it brings in security risk, dependency conflicts and reduces the velocity of releases.

Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK.
Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK JAVA.

Meta Issue: [Steps to make OpenSearch extensible](https://github.com/opensearch-project/OpenSearch/issues/2447)
Sandboxing: [Step towards modular architecture in OpenSearch](https://github.com/opensearch-project/OpenSearch/issues/1422)
Expand All @@ -32,7 +32,7 @@ During the bootstrap of OpenSearch node, it class loads all the code under `~/pl

![](Docs/Extensions.png)

Extensions are independent processes which are built using `opensearch-sdk`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes.
Extensions are independent processes which are built using `opensearch-sdk-java`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes.
Extensions are designed to extend features via transport APIs which are exposed using extension points of OpenSearch.

### Discovery
Expand All @@ -41,8 +41,8 @@ Here is an example extension configuration `extensions.yml`:

```
extensions:
- name: opensearch-sdk // extension name
uniqueId: opensearch-sdk-1 // identifier for the extension
- name: opensearch-sdk-java // extension name
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please make this sample-extension? Extension names are not and never should be this project's name. I have a request in #74 to change this, and changing it here will avoid creating a merge conflict that I'll need to handle. :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

uniqueId: opensearch-sdk-java-1 // identifier for the extension
hostName: 'sdk_host' // name of the host where extension is running
hostAddress: '127.0.0.1' // host to reach
port: '4532' // port to reach
Expand All @@ -55,7 +55,7 @@ extensions:
### Communication
As we are running extensions on the port defined in the `extensions.yml`, the communication between OpenSearch and Extensions happens using a ServerSocket which binds the port and the host address. OpenSearch will initialize the extensions during the bootstrap by making a request to all the extensions running on different ports and thus creating a medium for the future requests.

### OpenSearch SDK
### OpenSearch SDK JAVA
Currently, plugins relies on extension points to communicate with OpenSearch. To turn plugins into extensions, all the extension points should be converted into Transport APIs which will be present in the SDK. Plugins need to integrate SDK, call those APIs, and later SDK will take care of the communication and the required attributes from OpenSearch.

### Settings
Expand Down
38 changes: 19 additions & 19 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@

# OpenSearch SDK Developer Guide
# OpenSearch SDK JAVA Developer Guide
- [Introduction](#introduction)
- [Getting Started](#getting-started)
- [Git Clone OpenSearch-SDK Repo](#git-clone-OpenSearch-SDK-repo)
- [Git Clone OpenSearch-SDK-JAVA Repo](#git-clone-OpenSearch-SDK-JAVA-repo)
- [Git Clone OpenSearch Repo](#git-clone-opensearch-repo)
- [Publish OpenSearch Feature/Extensions branch to Maven local](#publish-opensearch-feature/extensions-branch-to-maven-local)
- [Run OpenSearch-SDK](#run-opensearch-sdk)
- [Run OpenSearch-SDK-JAVA](#run-opensearch-sdk-java)
- [Create extensions.yml file](#create_extensions_file)
- [Run OpenSearch](#run-opensearch)
- [Run Tests](#run-tests)
Expand All @@ -18,14 +18,14 @@ Read more about extensibility [here](https://github.com/opensearch-project/OpenS

## Getting Started

### Git Clone OpenSearch SDK Repo
Fork [OpenSearch SDK](https://github.com/opensearch-project/opensearch-sdk) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk.git`.
### Git Clone OpenSearch SDK JAVA Repo
Fork [OpenSearch SDK JAVA](https://github.com/opensearch-project/opensearch-sdk-java) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk-java.git`.

### Git Clone OpenSearch Repo
Fork [OpenSearch](https://github.com/opensearch-project/OpenSearch/), checkout feature/extensions branch, and clone locally, e.g. `git clone https://github.com/[your username]/OpenSearch.git`.

## Publish OpenSearch feature/extensions Branch to Maven local
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running the OpenSearch SDK on a seperate process.
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running the OpenSearch SDK JAVA on a seperate process.

- First navigate to the directory that OpenSearch has been cloned to
- Checkout the correct branch, e.g. `git checkout feature/extensions`.
Expand All @@ -34,9 +34,9 @@ The work done to support the extensions framework is located on the `feature/ext

It is necessary to publish dependencies to a local maven repository until this branch is merged to `main`, at which point all dependencies will be published to Maven central.

## Run OpenSearch SDK
## Run OpenSearch SDK JAVA

Navigate to the directory that OpenSearch-SDK has been cloned to and run main script using `./gradlew run`.
Navigate to the directory that OpenSearch-SDK-JAVA has been cloned to and run main script using `./gradlew run`.

```
./gradlew run
Expand Down Expand Up @@ -73,8 +73,8 @@ Sample extensions.yml file:

```
extensions:
- name: opensearch-sdk
uniqueId: opensearch-sdk-1
- name: opensearch-sdk-java
uniqueId: opensearch-sdk-java-1
hostName: 'sdk_host'
hostAddress: '127.0.0.1'
port: '4532'
Expand All @@ -83,7 +83,7 @@ extensions:
opensearchVersion: '3.0.0'
javaVersion: '14'
className: ExtensionsRunner
customFolderName: opensearch-sdk
customFolderName: opensearch-sdk-java
hasNativeController: false
```

Expand All @@ -92,7 +92,7 @@ extensions:
- Return to the OpenSearch directory by using `cd ..`.
- Start OpenSearch feature/extensions branch using `./bin/opensearch`.

During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to the OpenSearch SDK and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on.
During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to the OpenSearch SDK JAVA and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on.

```
[2022-06-16T21:30:18,857][INFO ][o.o.t.TransportService ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
Expand All @@ -102,7 +102,7 @@ During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the ext
[2022-06-16T21:30:19,000][INFO ][o.o.e.ExtensionsOrchestrator] [runTask-0] received PluginResponse{examplepluginname}
```

OpenSearch SDK terminal will also log all requests and responses it receives from OpenSearch :
OpenSearch SDK JAVA terminal will also log all requests and responses it receives from OpenSearch :
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed


TCP HandShake Request :
```
Expand Down Expand Up @@ -156,7 +156,7 @@ MESSAGE RECEIVED:ES-ǣ!internal:discovery/extensionsnode_extensionQSt9oKXFTSWqgX
21:30:18.999 [opensearch[extension][transport_worker][T#6]] TRACE org.opensearch.transport.TransportService.tracer - [3][internal:discovery/extensions] sent response
```

It is important that the OpenSearch SDK is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK will now be able to comminicate.
It is important that the OpenSearch SDK JAVA is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK JAVA is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK JAVA will now be able to comminicate.

## Run Tests

Expand All @@ -165,7 +165,7 @@ Run tests :
./gradlew clean build test
```
## Generate Artifact
In opensearch-sdk navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one:
In opensearch-sdk-java navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be a -java in the tarball name.

```
./gradlew clean && ./gradlew build
```
Expand All @@ -174,11 +174,11 @@ Look for tar ball in /build/distributions. To run the artifact i.e. tar ball, ru
```
tar -xvf opensearch-sdk-1.0.0-SNAPSHOT.tar
```
TODO https://github.com/opensearch-project/opensearch-sdk/issues/52
Navigate to opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/
TODO https://github.com/opensearch-project/opensearch-sdk-java/issues/52
Navigate to opensearch-sdk-java/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/
- Check if src folder exists in using `ls`.
- If the src folder does not exist, create it using `mkdir src && cd src && mkdir test && cd test && mkdir resources && cd resources`.
The above command will generate a path opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/src/test/resources
The above command will generate a path opensearch-sdk-java/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/src/test/resources
- Manually create a file titled `extension.yml` within the resources directory using an IDE or an in-line text editor. Below is the sample of extension.yml
dbwiddis marked this conversation as resolved.
Show resolved Hide resolved

Sample extensions.yml file:
Expand All @@ -187,7 +187,7 @@ Sample extensions.yml file:
hostAddress: 127.0.0.1
hostPort: 4532
```
- After extension.yml is generated then start opensearch-sdk by ./bin/opensearch-sdk
- After extension.yml is generated then start opensearch-sdk-java by ./bin/opensearch-sdk-java

## Submitting Changes

Expand Down
2 changes: 1 addition & 1 deletion MAINTAINERS.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# OpenSearch-SDK Maintainers
# OpenSearch-SDK-JAVA Maintainers

## Maintainers
| Maintainer | GitHub ID | Affiliation |
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# OpenSearch SDK
# OpenSearch SDK JAVA
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to SHOUT :) Java

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

- [Introduction](#introduction)
- [Contributing](#contributing)
- [Code of Conduct](#code-of-conduct)
Expand Down
Loading