Skip to content

Commit

Permalink
Updated user guide with a complete example.
Browse files Browse the repository at this point in the history
Signed-off-by: dblock <[email protected]>
  • Loading branch information
dblock committed Feb 1, 2023
1 parent 3be033e commit c4b603e
Show file tree
Hide file tree
Showing 6 changed files with 170 additions and 90 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
### Added
- Adds Github workflow for changelog verification ([#89](https://github.com/opensearch-project/opensearch-rs/pull/89))
- Adds Github workflow for unit tests ([#112](https://github.com/opensearch-project/opensearch-rs/pull/112))
- Updates users guide ([#114](https://github.com/opensearch-project/opensearch-rs/pull/114))

### Dependencies
- Bumps `simple_logger` from 2.3.0 to 4.0.0
Expand Down
115 changes: 59 additions & 56 deletions DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,18 @@
- [Install Prerequisites](#install-prerequisites)
- [Development](#development)
- [Prerequisites](#prerequisites)
- [Cargo make](#cargo-make)
- [Run Tests](#run-tests)
- [Cargo Make](#cargo-make)
- [Packages](#packages)
- [Design principles](#design-principles)
- [Coding style guide](#coding-style-guide)
- [Coding Style Guide](#coding-style-guide)
- [Formatting](#formatting)
- [Clippy](#clippy)
- [Running MSVC debugger in VS Code](#running-msvc-debugger-in-vs-code)
- [Running MSVC Debugger in Visual Studio Code](#running-msvc-debugger-in-visual-studio-code)

# Developer Guide

So you want to contribute code to the OpenSearch Rust Client? Excellent! We're glad you're here. Here's what you need to do:
So you want to contribute code to the OpenSearch Rust Client? Excellent! We're glad you're here.

## Getting Started

Expand All @@ -26,42 +28,54 @@ e.g. `git clone https://github.com/[your username]/opensearch-rs.git`.

## Development

The following information will help in getting up and running:
The following information will help in getting up and running.

### Prerequisites

The project makes use of the following, which should be installed

- [**Docker**](https://www.docker.com/)
- [**Rust**](https://www.rust-lang.org/tools/install)

Docker is used to start instances of OpenSearch by using
[OpenSearch docker image](https://hub.docker.com/r/opensearchproject/opensearch).
```
$ rustc --version
rustc 1.65.0
```

- [**Cargo make**](https://sagiegurari.github.io/cargo-make/)

Cargo make is used to define and configure a set of tasks, and run them as a flow. This helps with performing actions
such as starting an OpenSearch instance for integration tests

Cargo make can be installed with
Cargo make is used to define and configure a set of tasks, and run them as a flow. This helps with performing actions such as starting an OpenSearch instance for integration tests. Install as follows.

```sh
cargo install --force cargo-make
```

- [**Docker**](https://www.docker.com/)

Docker is used to start instances of OpenSearch required for integration tests using [OpenSearch docker images](https://hub.docker.com/r/opensearchproject/opensearch).

[Set `vm.max_map_count` for your platform](https://opensearch.org/docs/latest/opensearch/install/important-settings) to allow OpenSearch to start.

### Run Tests

If you are running the tests in Docker, [set `vm.max_map_count` for your platform](https://opensearch.org/docs/latest/opensearch/install/important-settings) to allow OpenSearch to start.
Run unit tests.

### Cargo make
```sh
cargo test
```

To include optional features, such as AWS auth use the following.

```sh
cargo test --features "aws-auth"
```

### Cargo Make

Cargo make is used to define and configure a set of tasks, and run them as a flow. To see all of the OpenSearch
category tasks defined
Cargo make is used to define and configure a set of tasks, and run them as a flow. To see all of the OpenSearch category tasks defined.

```sh
cargo make
```

The `OpenSearch` category of steps are specifically defined for this project and are defined in
[Makefile.toml](Makefile.toml).
The `OpenSearch` category of steps are specifically defined for this project and are defined in [Makefile.toml](Makefile.toml).

- Build all packages

Expand Down Expand Up @@ -90,8 +104,7 @@ The `OpenSearch` category of steps are specifically defined for this project and

Optionally pass

- `STACK_VERSION`: OpenSearch version like `1.0.0` or can be
a snapshot release like `1.x-SNAPSHOT`
- `STACK_VERSION`: OpenSearch version, e.g. `1.0.0`, or can be a snapshot release, e.g. `1.x-SNAPSHOT`.

```sh
STACK_VERSION=1.2.4 cargo make test-yaml
Expand All @@ -103,66 +116,57 @@ The workspace contains the following packages:

- #### `opensearch`

The client package crate. The client exposes all OpenSearch APIs as associated functions, either on
the root client, `OpenSearch`, or on one of the _namespaced clients_, such as `Cat`, `Indices`, etc. The _namespaced clients_
are based on the grouping of APIs within the [OpenSearch](https://github.com/opensearch-project/OpenSearch/tree/main/rest-api-spec) REST API specs from which much of the client is generated.
All API functions are `async` only, and can be `await`ed.
The client package crate. The client exposes all OpenSearch APIs as associated functions, either on the root client, `OpenSearch`, or on one of the _namespaced clients_, such as `Cat`, `Indices`, etc. The _namespaced clients_ are based on the grouping of APIs within the [OpenSearch](https://github.com/opensearch-project/OpenSearch/tree/main/rest-api-spec) REST API specs from which much of the client is generated. All API functions are `async` only, and can be `await`ed.

- #### `api_generator`

A small executable that downloads REST API specs from GitHub and generates much of the client package from the specs.
The minimum REST API spec version compatible with the generator is `v7.4.0`.
A small executable that downloads REST API specs from GitHub and generates much of the client package from the specs. The minimum REST API spec version compatible with the generator is `v7.4.0`.

The `api_generator` package makes heavy use of the [`syn`](https://docs.rs/syn/1.0.5/syn/) and [`quote`](https://docs.rs/quote/1.0.2/quote/) crates to generate Rust code from the REST API specs.
The `quote!` macro is particularly useful as it accepts Rust code that can include placeholder tokens (prefixed with `#`)
that will be interpolated during expansion. Unlike procedural macros, the token stream returned by the `quote!` macro
can be `to_string()`'ed and written to disk, and this is used to create much of the client scaffolding.
The `api_generator` package makes heavy use of the [`syn`](https://docs.rs/syn/1.0.5/syn/) and [`quote`](https://docs.rs/quote/1.0.2/quote/) crates to generate Rust code from the REST API specs. The `quote!` macro is particularly useful as it accepts Rust code that can include placeholder tokens (prefixed with `#`) that will be interpolated during expansion. Unlike procedural macros, the token stream returned by the `quote!` macro can be `to_string()`'ed and written to disk, and this is used to create much of the client scaffolding.

- #### `yaml_test_runner`

A small executable that downloads YAML tests from GitHub and generates client tests from the YAML tests. The
version of YAML tests to download are determined from the commit hash of a running Elasticsearch instance.
A small executable that downloads YAML tests from GitHub and generates client tests from the YAML tests. The v0ersion of YAML tests to download are determined from the commit hash of a running Elasticsearch instance.

The `yaml_test_runner` package can be run with `cargo make test-yaml` to run the generated client tests and we can pass environment variable `STACK_VERSION` to control the distribution and version.

### Design principles

1. Generate as much of the client as feasible from the REST API specs

The REST API specs contain information about
- the URL parts e.g. `{index}/_search` and variants
- accepted HTTP methods e.g. `GET`, `POST`
- the URL query string parameters
- whether the API accepts a body
The REST API specs contain information about the following:

- The URL parts e.g. `{index}/_search` and variants
- Accepted HTTP methods e.g. `GET`, `POST`
- The URL query string parameters
- Whether the API accepts a body

2. Prefer generation methods that produce ASTs and token streams over strings.
The `quote` and `syn` crates help
2. Prefer generation methods that produce ASTs and token streams over strings. The `quote` and `syn` crates help.

3. Get it working, then refine/refactor
3. Get it working, then refine/refactor.

- Start simple and iterate
- Design of the API is conducive to ease of use
- Asynchronous only
- Control API invariants through arguments on API function. For example
- Control API invariants through arguments on API function, .e.g

```no_run
client.delete_script(DeleteScriptParts::Id("script_id"))
.send()
.await?;
```
An id must always be provided for a delete script API call, so the `delete_script()` function
must accept it as a value.
An id must always be provided for a delete script API call, so the `delete_script()` function must accept it as a value.
### Coding style guide
### Coding Style Guide
The repository adheres to the styling enforced by `rustfmt`.
#### Formatting
Rust code can be formatted using [`rustfmt`](https://github.com/rust-lang/rustfmt) through cargo make.
Rust code can be formatted using [`rustfmt`](https://github.com/rust-lang/rustfmt) through `cargo make`.
To format all packages in a workspace, from the workspace root
To format all packages in a workspace, from the workspace root run the following.
```sh
cargo make format
Expand All @@ -172,22 +176,21 @@ It is strongly recommended to run this before opening a PR.

#### Clippy

[Clippy](https://github.com/rust-lang/rust-clippy) is a bunch of lints to catch common mistakes and improve your Rust code!
[Clippy](https://github.com/rust-lang/rust-clippy) is a bunch of lints to catch common mistakes and improve your Rust code.

Run clippy before opening a PR
Run clippy before opening a PR as follows.

```sh
cargo make clippy
```

### Running MSVC debugger in VS Code
### Running MSVC Debugger in Visual Studio Code

From [Bryce Van Dyk's blog post](https://www.brycevandyk.com/debug-rust-on-windows-with-visual-studio-code-and-the-msvc-debugger/),
if wishing to use the MSVC debugger with Rust in VS code, which may be preferred on Windows
Inspired by [Bryce Van Dyk's blog post](https://www.brycevandyk.com/debug-rust-on-windows-with-visual-studio-code-and-the-msvc-debugger/), use the MSVC debugger with Rust in Visual Studio Code as follows.

1. Install [C/C++ VS Code extensions](https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools)
1. Install [C/C++ VS Code extensions](https://marketplace.visualstudio.com/items?itemName=ms-vscode.cpptools).

2. Place the following in `.vscode/launch.json` in the project root
2. Place the following in `.vscode/launch.json` in the project root.

```json
{
Expand All @@ -208,4 +211,4 @@ if wishing to use the MSVC debugger with Rust in VS code, which may be preferred
}
```

3. Add `"debug.allowBreakpointsEverywhere": true` to VS code settings.json
3. Add `"debug.allowBreakpointsEverywhere": true` to VS code settings.json.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
OpenSearch Rust Client

- [Welcome!](#welcome)
- [User Guide](#user-guide)
- [Compatibility with OpenSearch](#compatibility-with-opensearch)
- [Code of Conduct](#code-of-conduct)
- [Security](#security)
Expand All @@ -16,7 +17,11 @@ OpenSearch Rust Client

## Welcome!

**opensearch-rs** is [a community-driven, open source fork](https://aws.amazon.com/blogs/opensource/introducing-opensearch/) of elasticsearch-rs licensed under the [Apache v2.0 License](LICENSE.txt). For more information, see [opensearch.org](https://opensearch.org/).
The opensearch-rs client is [a community-driven, open source fork](https://aws.amazon.com/blogs/opensource/introducing-opensearch/) of elasticsearch-rs, licensed under the [Apache v2.0 License](LICENSE.txt). For more information, see [opensearch.org](https://opensearch.org/).

## User Guide

See [User Guide](USER_GUIDE.md).

## Compatibility with OpenSearch

Expand Down
111 changes: 90 additions & 21 deletions USER_GUIDE.md
Original file line number Diff line number Diff line change
@@ -1,54 +1,123 @@
- [User Guide](#user-guide)
- [Example](#example)
- [Create a client](#create-a-client)
- [Making API calls](#making-api-calls)
- [Create a Client](#create-a-client)
- [Display Server Version](#display-server-version)
- [Create an Index](#create-an-index)
- [Add a Document to the Index](#add-a-document-to-the-index)
- [Search for a Document](#search-for-a-document)
- [Delete the Index](#delete-the-index)
- [Amazon OpenSearch Service](#amazon-opensearch-service)
- [Create a Client](#create-a-client-1)

# User Guide

## Example
In the example below, we create a client, an index with non-default settings, insert a document to the index,
search for the document, delete the document and finally delete the index.

In the example below, we create a client, an index, insert a document to the index, search for the document, and finally delete the index.

#### Create a client
### Create a Client

To create a client to make API calls to OpenSearch running on `http://localhost:9200`
To create a client to make API calls to OpenSearch running on `http://localhost:9200`.

```rust
let client = OpenSearch::default();
```

Alternatively, you can create a client to make API calls against OpenSearch running on a
specific `url::Url`
specific `url::Url`.

```rust
let transport = Transport::single_node("https://example.com")?;
let client = OpenSearch::new(transport);
```

#### Making API calls
### Display Server Version

```rust
let info: Value = client.info().send().await?.json().await?;
println!(
"{}: {}",
info["version"]["distribution"].as_str().unwrap(),
info["version"]["number"].as_str().unwrap()
);
```

### Create an Index

```rust
client
.indices()
.create(opensearch::indices::IndicesCreateParts::Index("movies"))
.send()
.await?;
```

### Add a Document to the Index

```rust
client
.index(opensearch::IndexParts::Index("movies"))
.body(serde_json::json!({
"title": "Moneyball",
"director": "Bennett Miller",
"year": 2011
}
))
.send()
.await?;
```

### Search for a Document

The following makes an API call to `tweets/_search` with the json body
`{"query":{"match":{"message":"OpenSearch"}}}`
Make a query and display the response body and the search results.

```rust
let response = client
.search(SearchParts::Index(&["tweets"]))
.from(0)
.size(10)
.body(json!({
"query": {
"match": {
"message": "OpenSearch rust"
.search(opensearch::SearchParts::Index(&["movies"]))
.body(serde_json::json!({
"query": {
"match": {
"director": "miller"
}
}
}
}))
))
.send()
.await?;

let response_body = response.json::<Value>().await?;
let took = response_body["took"].as_i64().unwrap();
println!("{}", serde_json::to_string_pretty(&response_body).unwrap());

for hit in response_body["hits"]["hits"].as_array().unwrap() {
// print the source document
println!("{:?}", hit["_source"]);
}
```
```

### Delete the Index

```rust
client
.indices()
.delete(opensearch::indices::IndicesDeleteParts::Index(&["movies"]))
.send()
.await?;
```

## Amazon OpenSearch Service

This library supports [Amazon OpenSearch Service](https://aws.amazon.com/opensearch-service/).

### Create a Client

Create a client with AWS credentials as follows. Make sure to specify the correct service name and signing region.

```rust
let url = Url::parse("https://...");
let conn_pool = SingleNodeConnectionPool::new(url?);
let region_provider = RegionProviderChain::default_provider().or_else("us-east-1");
let aws_config = aws_config::from_env().region(region_provider).load().await.clone();
let transport = TransportBuilder::new(conn_pool)
.auth(aws_config.clone().try_into()?)
.build()?;
let client = OpenSearch::new(transport);
```
Loading

0 comments on commit c4b603e

Please sign in to comment.