Skip to content

Commit

Permalink
Rename everything from opensearch-sdk to opensearch-sdk-java (#96)
Browse files Browse the repository at this point in the history 
* issue #28

Signed-off-by: mloufra <[email protected]>

* Update the lastest coomit

Signed-off-by: mloufra <[email protected]>

* fix merge conflict

Signed-off-by: mloufra <[email protected]>

* Rename the method and fix the conflict

Signed-off-by: mloufra <[email protected]>

* Rename everything from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

* Rename everything from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

* Rename everythingg from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

* Rename extensionName to avoid merge conflict

Signed-off-by: mloufra <[email protected]>

* Renanme everything from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

* Rename everything from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

* Rename everything from opensearch-sdk to opensearch-sdk-java

Signed-off-by: mloufra <[email protected]>

Signed-off-by: mloufra <[email protected]>
Signed-off-by: Frank Lou <[email protected]>
  • Loading branch information
@mloufra
mloufra authored Aug 23, 2022
1 parent 367c762 commit 9e60bea
Show file tree
Hide file tree
Showing 7 changed files with 42 additions and 42 deletions.
2 changes: 1 addition & 1 deletion .github/CODEOWNERS
View file
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
# This should match the owning team set up in https://github.com/orgs/opensearch-project/teams
* @opensearch-project/opensearch-sdk
* @opensearch-project/opensearch-sdk-java
16 changes: 8 additions & 8 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
- [Contributing to OpenSearch-SDK](#contributing-to-opensearch-sdk)
- [Contributing to OpenSearch-SDK-Java](#contributing-to-opensearch-sdk-java)
- [First Things First](#first-things-first)
- [Ways to Contribute](#ways-to-contribute)
- [Bug Reports](#bug-reports)
Expand All @@ -13,7 +13,7 @@ OpenSearch-SDK is a community project that is built and maintained by people jus

## First Things First

1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk/issues).
1. **When in doubt, open an issue** - For almost any type of contribution, the first step is opening an issue. Even if you think you already know what the solution is, writing down a description of the problem you're trying to solve will help everyone get context when they review your pull request. If it's truly a trivial change (e.g. spelling error), you can skip this step -- but as the subject says, when in doubt, [open an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues).

2. **Only submit your own work** (or work you have sufficient rights to submit) - Please make sure that any code or documentation you submit is your work or you have the rights to submit. We respect the intellectual property rights of others, and as part of contributing, we'll ask you to sign your contribution with a "Developer Certificate of Origin" (DCO) that states you have the rights to submit this work and you understand we'll use your contribution. There's more information about this topic in the [DCO section](#developer-certificate-of-origin).

Expand All @@ -23,17 +23,17 @@ OpenSearch-SDK is a community project that is built and maintained by people jus

Ugh! Bugs!

A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk).
A bug is when software behaves in a way that you didn't expect and the developer didn't intend. To help us understand what's going on, we first want to make sure you're working from the latest version. Please make sure you're testing against the [latest version](https://github.com/opensearch-project/opensearch-sdk-java).

Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk/issues).
Once you've confirmed that the bug still exists in the latest version, you'll want to check to make sure it's not something we already know about on the [open issues GitHub page](https://github.com/opensearch-project/opensearch-sdk-java/issues).

### Feature Requests

If you've thought of a way that OpenSearch-SDK could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work.
If you've thought of a way that OpenSearch-SDK-Java could be better, we want to hear about it. We track feature requests using GitHub, so please feel free to open an issue which describes the feature you would like to see, why you need it, and how it should work.

### Contributing Code

As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue.
As with other types of contributions, the first step is to [**open an issue on GitHub**](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose). Opening an issue before you make changes makes sure that someone else isn't already working on that particular problem. It also lets us all work together to find the right approach before you spend a bunch of time on a PR. So again, when in doubt, open an issue.

Once you've opened an issue, check out our [Developer Guide](./DEVELOPER_GUIDE.md) for instructions on how to get started.

Expand Down Expand Up @@ -68,7 +68,7 @@ By making a contribution to this project, I certify that:
consistent with this project or the open source license(s)
involved.
```
We require that every contribution to OpenSearch-SDK is signed with a Developer Certificate of Origin. Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms.
We require that every contribution to OpenSearch-SDK-Java is signed with a [Developer Certificate of Origin](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin). Additionally, please use your real name. We do not accept anonymous contributors nor those utilizing pseudonyms.

Each commit must include a DCO which looks like this

Expand All @@ -79,7 +79,7 @@ You may type this line on your own when writing your commit messages. However, i

## Review Process

We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction.
We deeply appreciate everyone who takes the time to make a contribution. We will review all contributions as quickly as possible. As a reminder, [opening an issue](https://github.com/opensearch-project/opensearch-sdk-java/issues/new/choose) discussing your change before you make it is the best way to smooth the PR process. This will prevent a rejection because someone else is already working on the problem, or because the solution is incompatible with the architectural direction.

It is necessary to request at least two contributors to review your pull request. It is required to have two approvals in order for your pull request to be accepted.

Expand Down
8 changes: 4 additions & 4 deletions DESIGN.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ i.e critical workloads like ingestion/search traffic would be impacted because o
This problem is exponentially grows when we would like to run a 3rd Party plugin from the community.
As OpenSearch and plugins run in the same process, it brings in security risk, dependency conflicts and reduces the velocity of releases.

Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK.
Introducing extensions, a simple and easy way to extend features of OpenSearch. It would support all plugin features and enable them to run in a seperate process or on another node via OpenSearch SDK Java.

Meta Issue: [Steps to make OpenSearch extensible](https://github.com/opensearch-project/OpenSearch/issues/2447)
Sandboxing: [Step towards modular architecture in OpenSearch](https://github.com/opensearch-project/OpenSearch/issues/1422)
Expand All @@ -32,7 +32,7 @@ During the bootstrap of OpenSearch node, it class loads all the code under `~/pl

![](Docs/Extensions.png)

Extensions are independent processes which are built using `opensearch-sdk`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes.
Extensions are independent processes which are built using `opensearch-sdk-java`. They communicate with OpenSearch via [transport](https://github.com/opensearch-project/OpenSearch/tree/main/modules/transport-netty4) protocol which today is used to communicate between OpenSearch nodes.
Extensions are designed to extend features via transport APIs which are exposed using extension points of OpenSearch.

### Discovery
Expand All @@ -41,7 +41,7 @@ Here is an example extension configuration `extensions.yml`:

```
extensions:
- name: opensearch-sdk // extension name
- name: sample-extension // extension name
uniqueId: opensearch-sdk-1 // identifier for the extension
hostName: 'sdk_host' // name of the host where extension is running
hostAddress: '127.0.0.1' // host to reach
Expand All @@ -55,7 +55,7 @@ extensions:
### Communication
As we are running extensions on the port defined in the `extensions.yml`, the communication between OpenSearch and Extensions happens using a ServerSocket which binds the port and the host address. OpenSearch will initialize the extensions during the bootstrap by making a request to all the extensions running on different ports and thus creating a medium for the future requests.

### OpenSearch SDK
### OpenSearch SDK Java
Currently, plugins relies on extension points to communicate with OpenSearch. To turn plugins into extensions, all the extension points should be converted into Transport APIs which will be present in the SDK. Plugins need to integrate SDK, call those APIs, and later SDK will take care of the communication and the required attributes from OpenSearch.

### Settings
Expand Down
38 changes: 19 additions & 19 deletions DEVELOPER_GUIDE.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@

# OpenSearch SDK Developer Guide
# OpenSearch SDK for Java Developer Guide
- [Introduction](#introduction)
- [Getting Started](#getting-started)
- [Git Clone OpenSearch-SDK Repo](#git-clone-OpenSearch-SDK-repo)
- [Git Clone OpenSearch-SDK-Java Repo](#git-clone-OpenSearch-SDK-Java-repo)
- [Git Clone OpenSearch Repo](#git-clone-opensearch-repo)
- [Publish OpenSearch Feature/Extensions branch to Maven local](#publish-opensearch-featureextensions-branch-to-maven-local)
- [Run OpenSearch-SDK](#run-opensearch-sdk)
- [Run OpenSearch-SDK-Java](#run-opensearch-sdk-java)
- [Create extensions.yml file](#create-extensionsyml-file)
- [Run OpenSearch](#run-opensearch)
- [Publish OpenSearch-SDK to Maven Local](#publish-openSearch-sdk-to-maven-local)
Expand All @@ -19,14 +19,14 @@ Read more about extensibility [here](https://github.com/opensearch-project/OpenS

## Getting Started

### Git Clone OpenSearch SDK Repo
Fork [OpenSearch SDK](https://github.com/opensearch-project/opensearch-sdk) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk.git`.
### Git Clone OpenSearch SDK for Java Repo
Fork [OpenSearch SDK for Java](https://github.com/opensearch-project/opensearch-sdk-java) and clone locally, e.g. `git clone https://github.com/[your username]/opensearch-sdk-java.git`.

### Git Clone OpenSearch Repo
Fork [OpenSearch](https://github.com/opensearch-project/OpenSearch/), checkout feature/extensions branch, and clone locally, e.g. `git clone https://github.com/[your username]/OpenSearch.git`.

## Publish OpenSearch feature/extensions Branch to Maven local
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running the OpenSearch SDK on a seperate process.
The work done to support the extensions framework is located on the `feature/extensions` branch of the OpenSearch project. It is necessary to publish the dependencies of this branch to your local maven repository prior to running OpenSearch SDK for Java on a seperate process.

- First navigate to the directory that OpenSearch has been cloned to
- Checkout the correct branch, e.g. `git checkout feature/extensions`.
Expand All @@ -35,9 +35,9 @@ The work done to support the extensions framework is located on the `feature/ext

It is necessary to publish dependencies to a local maven repository until this branch is merged to `main`, at which point all dependencies will be published to Maven central.

## Run OpenSearch SDK
## Run OpenSearch SDK for Java

Navigate to the directory that OpenSearch-SDK has been cloned to and run main script using `./gradlew run`.
Navigate to the directory that OpenSearch-SDK-Java has been cloned to and run main script using `./gradlew run`.

```
./gradlew run
Expand Down Expand Up @@ -79,7 +79,7 @@ Sample `extensions.yml` file (the name must match the `extensionName` field in t
```
extensions:
- name: sample-extension
uniqueId: opensearch-sdk-1
uniqueId: opensearch-sdk-java-1
hostName: 'sdk_host'
hostAddress: '127.0.0.1'
port: '4532'
Expand All @@ -88,7 +88,7 @@ extensions:
opensearchVersion: '3.0.0'
javaVersion: '14'
className: ExtensionsRunner
customFolderName: opensearch-sdk
customFolderName: opensearch-sdk-java
hasNativeController: false
```

Expand All @@ -97,7 +97,7 @@ extensions:
- Return to the OpenSearch directory by using `cd ..`.
- Start OpenSearch feature/extensions branch using `./bin/opensearch`.

During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to the OpenSearch SDK and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on.
During OpenSearch bootstrap, `ExtensionsOrchestrator` will then discover the extension listenening on a pre-defined port and execute the TCP handshake protocol to establish a data transfer connection. A request will be sent to OpenSearch SDK for Java and upon acknowledgment, the extension will respond with its name which will be logged onto terminal that OpenSearch is running on.

```
[2022-06-16T21:30:18,857][INFO ][o.o.t.TransportService ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
Expand Down Expand Up @@ -161,7 +161,7 @@ MESSAGE RECEIVED:ES-ǣ!internal:discovery/extensionsnode_extensionQSt9oKXFTSWqgX
21:30:18.999 [opensearch[extension][transport_worker][T#6]] TRACE org.opensearch.transport.TransportService.tracer - [3][internal:discovery/extensions] sent response
```

It is important that the OpenSearch SDK is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK will now be able to comminicate.
It is important that the OpenSearch SDK for Java is already up and running on a seperate process prior to starting OpenSearch, since extension discovery occurs only if the OpenSearch SDK for Java is already listening on a pre-defined port. Once discovery is complete and the data transfer connection between both nodes has been established, OpenSearch and the OpenSearch SDK for Java will now be able to comminicate.

## Run Tests

Expand All @@ -170,29 +170,29 @@ Run tests :
./gradlew clean build test
```
## Generate Artifact
In opensearch-sdk navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one:
In opensearch-sdk-java navigate to build/distributions. Look for tar ball in the form `opensearch-sdk-java-1.0.0-SNAPSHOT.tar`. If not found follow the below steps to create one:
```
./gradlew clean && ./gradlew build
```
Once the tar ball is generated navigate to /src/test/resources and look for extension.yml. Create one if not present
Look for tar ball in /build/distributions. To run the artifact i.e. tar ball, run the below command
```
tar -xvf opensearch-sdk-1.0.0-SNAPSHOT.tar
tar -xvf opensearch-sdk-java-1.0.0-SNAPSHOT.tar
```
TODO https://github.com/opensearch-project/opensearch-sdk/issues/52
Navigate to opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/
TODO https://github.com/opensearch-project/opensearch-sdk-java/issues/52
Navigate to opensearch-sdk-java/build/distributions/opensearch-sdk-java-1.0.0-SNAPSHOT/
- Check if src folder exists in using `ls`.
- If the src folder does not exist, create it using `mkdir src && cd src && mkdir test && cd test && mkdir resources && cd resources`.
The above command will generate a path opensearch-sdk/build/distributions/opensearch-sdk-1.0.0-SNAPSHOT/src/test/resources
The above command will generate a path opensearch-sdk-java/build/distributions/opensearch-sdk-java-1.0.0-SNAPSHOT/src/test/resources
- Manually create a file titled `extension.yml` within the resources directory using an IDE or an in-line text editor. Below is the sample of extension.yml

Sample extensions.yml file:
```
extensionName: extension
extensionName: sample-extension
hostAddress: 127.0.0.1
hostPort: 4532
```
- After extension.yml is generated then start opensearch-sdk by ./bin/opensearch-sdk
- After extension.yml is generated then start opensearch-sdk-java by ./bin/opensearch-sdk-java

## Submitting Changes

Expand Down
2 changes: 1 addition & 1 deletion MAINTAINERS.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# OpenSearch-SDK Maintainers
# OpenSearch-SDK-Java Maintainers

## Maintainers
| Maintainer | GitHub ID | Affiliation |
Expand Down
Loading

0 comments on commit 9e60bea

Please sign in to comment.