Skip to content

Commit

Permalink
Add page about bulk:write and bulk:mwrite (#300)
Browse files Browse the repository at this point in the history
## What does this PR do?

Documents bulk:write and bulk:mWrite.  
See kuzzleio/kuzzle#1302

### How should this be manually tested?


  - Step 1 :
  - Step 2 :
  - Step 3 :  
  ...

### Other changes

Add references to these methods in data validation, kuzzle metadata and real-time guides
  • Loading branch information
Aschen authored and scottinet committed Jun 4, 2019
1 parent 8814264 commit 03f361b
Show file tree
Hide file tree
Showing 4 changed files with 264 additions and 4 deletions.
147 changes: 147 additions & 0 deletions src/core/1/api/api-reference/controller-bulk/m-write/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@
---
code: true
type: page
title: mWrite
---

# mWrite

<SinceBadge version="1.8.0" />


Create or replace multiple documents directly into the storage engine.

This is a low level route intended to bypass Kuzzle actions on document creation, notably:
- check [document validity](/core/1/guide/guides/essentials/data-validation),
- add [kuzzle metadata](/core/1/guide/guides/essentials/document-metadata),
- trigger [realtime notifications](/core/1/guide/guides/essentials/real-time) (unless asked otherwise)

---

## Query Syntax

### HTTP

```http
URL: http://kuzzle:7512/<index>/<collection>/_mWrite[?refresh=wait_for][&notify]
Method: POST
Body:
```

```js
{
"documents": [
{
"_id": "<documentId>",
"body": {
// document content
}
},
{
"_id": "<anotherDocumentId>",
"body": {
// document content
}
}
]
}
```

### Other protocols

```js
{
"index": "<index>",
"collection": "<collection>",
"controller": "bulk",
"action": "mWrite",

"notify": "<boolean>",
"body": {
"documents": [
{
"_id": "<documentId>",
"body": {
// document content
}
},
{
"_id": "<anotherDocumentId>",
"body": {
// document content
}
}
]
}
}
```

---

## Arguments

- `collection`: data collection
- `index`: data index

### Optional:

- `notify`: if set to true, Kuzzle will trigger realtime notifications
- `refresh`: if set to `wait_for`, Kuzzle will not respond until the created/replaced documents are indexed

---

## Body properties

- `documents`: an array of object. Each object describes a document to create or replace, by exposing the following properties:
- `_id`: document unique identifier
- `body`: document content

---

## Response

Returns a `hits` array, containing the list of created documents, in the same order than the one provided in the query.

Each created document is an object with the following properties:

- `_id`: created document unique identifier
- `_source`: document content
- `_version`: version number of the document
- `created`: a boolean telling whether a document is created

If one or more document creations fail, the response status is set to `206`, and the `error` object contains a [partial error](/core/1/api/essentials/errors/#partialerror) error.

### Example

```javascript
{
"status": 200,
"error": null,
"index": "<index>",
"collection": "<collection>",
"action": "mWrite",
"controller": "bulk",
"requestId": "<unique request identifier>",
"result": {
"hits": [
{
"_id": "<documentId>",
"_source": {
// document content
},
"_version": 2,
"created": false
},
{
"_id": "<anotherDocumentId>",
"_source": {
// document content
},
"_version": 1,
"created": true
}
],
"total": 2
}
}
```
101 changes: 101 additions & 0 deletions src/core/1/api/api-reference/controller-bulk/write/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
---
code: true
type: page
title: write
---

# write

<SinceBadge version="1.8.0" />

Create or replace a document directly into the storage engine.

This is a low level route intended to bypass Kuzzle actions on document creation, notably:
- check [document validity](/core/1/guide/guides/essentials/data-validation),
- add [kuzzle metadata](/core/1/guide/guides/essentials/document-metadata),
- trigger [realtime notifications](/core/1/guide/guides/essentials/real-time) (unless asked otherwise).

---

## Query Syntax

### HTTP

```http
URL: http://kuzzle:7512/<index>/<collection>/_write[?refresh=wait_for][&notify][&_id=<document ID>]
Method: POST
Body:
```

```js
{
// document content
}
```

### Other protocols

```js
{
"index": "<index>",
"collection": "<collection>",
"controller": "bulk",
"action": "write",

"_id": "<documentId>",
"notify": "<boolean>",
"body": {
// document content
}
}
```

---

## Arguments

- `collection`: data collection
- `index`: data index

### Optional:

- `documentId`: set the document unique ID to the provided value, instead of auto-generating a random ID
- `notify`: if set to true, Kuzzle will trigger realtime notifications
- `refresh`: if set to `wait_for`, Kuzzle will not respond until the newly created document is indexed

---

## Body properties

Document content to create.

---

## Response

Returns an object with the following properties:

- `_id`: created document unique identifier
- `_source`: document content
- `_version`: version of the created document (should be `1`)
- `created`: a boolean telling if a new document has been created

```javascript
{
"status": 200,
"error": null,
"index": "<index>",
"collection": "<collection>",
"controller": "bulk",
"action": "write",
"requestId": "<unique request identifier>",
"result": {
"_id": "<documentId>",
"_version": 1,
"created": true,
"_source": {
// ...
},
}
}
```
4 changes: 4 additions & 0 deletions src/core/1/guide/guides/essentials/data-validation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ With Kuzzle, instead of programming the validation logic yourself, you can pick

For a detailed look at data validation, please refer to our [Data Validation Reference](/core/1/guide/datavalidation).

::: info
You can bypass data validation by using [bulk:write](/core/1/api/api-reference/bulk-controller/write) or [bulk:mWrite](/core/1/api/api-reference/bulk-controller/m-write) actions.
:::

---

## Basic Validation
Expand Down
16 changes: 12 additions & 4 deletions src/core/1/guide/guides/essentials/document-metadata/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,11 @@ order: 450

# Document Metadata

Whenever a document gets created, updated or deleted, Kuzzle will add or update the document's metadata. This metadata provides information about the document's lifecycle.
Whenever a document is created, updated or deleted, Kuzzle will add or update the document's metadata. This metadata provides information about the document's lifecycle.

::: info
You can bypass metadata automatic creation by using [bulk:write](/core/1/api/api-reference/bulk-controller/write) or [bulk:mWrite](/core/1/api/api-reference/bulk-controller/m-write) actions.
:::

---

Expand All @@ -23,8 +27,8 @@ Metadata can be viewed in the document's `_kuzzle_info` field and contains the f
- `createdAt`: Timestamp of document creation (create or replace), in epoch-milliseconds format.
- `updatedAt`: Timestamp of last document update in epoch-milliseconds format, or `null` if no update has been made.
- `updater`: The [unique identifier](/core/1/guide/guides/essentials/user-authentication/#kuzzle-user-identifier-kuid) of the user that updated the document, or `null` if the document has never been updated.
- `active`: The status of the document. `true` if the document is active and `false` if the document has been put in the trashcan.
- `deletedAt`: Timestamp of document deletion in epoch-milliseconds format, or `null` if the document has not been deleted.
- `active` <DeprecatedBadge version="1.2.0" />: The status of the document. `true` if the document is active and `false` if the document has been put in the trashcan.
- `deletedAt`<DeprecatedBadge version="1.2.0" />: Timestamp of document deletion in epoch-milliseconds format, or `null` if the document has not been deleted.

Here is an example of a Kuzzle response, containing a document's `_id`, `_source` and `_meta` fields:

Expand Down Expand Up @@ -78,7 +82,7 @@ Here is an example of a Kuzzle response, containing a document's `_id` and `_sou

## How metadata are physically stored

Documents metadata are managed by Kuzzle and cannot be changed using the API.
Documents metadata are managed by Kuzzle and cannot be changed using the API (except with [bulk:write](/core/1/api/api-reference/bulk-controller/write) and [bulk:mWrite](/core/1/api/api-reference/bulk-controller/m-write)).
Metadata are stored in the `_kuzzle_info` field of each document in Elasticsearch.

Elasticsearch might contain documents that don't have metadata. This can be the case for documents that were not inserted through Kuzzle. Such documents will automatically obtain metadata when they are updated through Kuzzle.
Expand Down Expand Up @@ -107,6 +111,8 @@ For example, to query by a document's creation timestamp, we can use the followi

## Documents Deletion

<DeprecatedBadge version="1.2.0" />

When a document gets deleted, Kuzzle first isolates it from other active documents by placing it in the `trashcan`.

Documents in the `trashcan` cannot be accessed, searched or counted, unless the `includeTrash` flag is set to `true` when invoking the API route.
Expand All @@ -115,6 +121,8 @@ Documents in the `trashcan` cannot be accessed, searched or counted, unless the

## Garbage Collection

<DeprecatedBadge version="1.2.0" />

Kuzzle will routinely search and permanently delete the oldest documents in the `trashcan`. This garbage collecting can be configured using the `services.garbageCollector` property in the Kuzzle [configuration file](/core/1/guide/guides/essentials/configuration/). In general, garbage collection works as follows:

- When Kuzzle is started, it will check the `services.garbageCollector` property and wait the configured delay before running the garbage collection for the first time.
Expand Down

0 comments on commit 03f361b

Please sign in to comment.