Skip to content

Commit

Permalink
Address comment
Browse files Browse the repository at this point in the history
  • Loading branch information
GuanLuo committed Oct 12, 2023
1 parent fb950bd commit 538693f
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion docs/protocol/extension_generate.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,8 @@

# Generate Extension

**Note:** The Generate Extension is *provisional* and likely to change in future versions.
> [!NOTE]
> The Generate Extension is *provisional* and likely to change in future versions.
This document describes Triton's generate extension. The generate
extension provides a simple text-oriented endpoint schema for interacting with
Expand Down Expand Up @@ -85,6 +86,10 @@ return an error.
generate request expressed as key/value pairs. See
[Parameters](#parameters) for more information.

> [!NOTE]
> Any additional properties in the request object are passed either as
> parameters or tensors based on model specification.
#### Parameters

The *$parameters* JSON describes zero or more “name”/”value” pairs,
Expand All @@ -106,6 +111,8 @@ specification to set the parameters.
Below is an example to send generate request with additional model parameters `stream` and `temperature`.

```
$ curl -X POST localhost:8000/v2/models/mymodel/generate -d '{"text_input": "client input", "parameters": {"stream": false, "temperature": 0}}'
POST /v2/models/mymodel/generate HTTP/1.1
Host: localhost:8000
Content-Type: application/json
Expand Down

0 comments on commit 538693f

Please sign in to comment.