Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

Commit

Permalink
Merge pull request #196 from janhq/pena-patch
Browse files Browse the repository at this point in the history
Update docs
  • Loading branch information
dan-menlo authored Sep 16, 2024
2 parents 225e257 + d39c98e commit ae25ff5
Show file tree
Hide file tree
Showing 67 changed files with 522 additions and 389 deletions.
2 changes: 1 addition & 1 deletion docs/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: "architecture"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

## Introduction
Expand Down
2 changes: 1 addition & 1 deletion docs/basic-usage/command-line.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: "command-line"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex has a [Docker](https://docs.docker.com/engine/reference/commandline/cli/) and [Ollama](https://ollama.com/)-inspired [CLI syntax](/docs/cli) for running model operations.
Expand Down
12 changes: 6 additions & 6 deletions docs/basic-usage/integration/js-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ slug: "ts-library"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex can be used in a Typescript application with the `cortex.js` library. Cortex provides a Typescript client library as a **fork of OpenAI's [Typescript library](https://github.com/openai/openai-node)** with additional methods for Local AI.
Cortex.cpp can be used in a Typescript application with the `cortex.js` library. Cortex.cpp provides a Typescript client library as a **fork of OpenAI's [Typescript library](https://github.com/openai/openai-node)** with additional methods for Local AI.

## Installation

Expand All @@ -18,19 +18,19 @@ npm install @janhq/cortexso-node

## Usage

1. Replace the OpenAI import with Cortex in your application:
1. Replace the OpenAI import with Cortex.cpp in your application:

```diff
- import OpenAI from 'openai';
+ import Cortex from '@janhq/cortexso-node';
```

2. Modify the initialization of the client to use Cortex:
2. Modify the initialization of the client to use Cortex.cpp:

```diff
- const openai = new OpenAI({
+ const cortex = new Cortex({
baseURL: "BASE_URL", // The default base URL for Cortex is 'http://localhost:1337'
baseURL: "BASE_URL", // The default base URL for Cortex is 'http://localhost:3928'
apiKey: "OPENAI_API_KEY", // This can be omitted if using the default
});

Expand All @@ -43,7 +43,7 @@ import Cortex from "@janhq/cortexso-node";

async function inference() {
const cortex = new Cortex({
baseURL: "http://localhost:1337/v1",
baseURL: "http://localhost:3928/v1",
apiKey: "",
});

Expand Down
10 changes: 5 additions & 5 deletions docs/basic-usage/integration/py-library.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ slug: "py-library"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::
Cortex can be used in a Python application with the `cortex.py` library. Cortex provides a Python client library as a **fork of OpenAI's [Python library](https://github.com/openai/openai-python)** with additional methods for Local AI.
Cortex.cpp can be used in a Python application with the `cortex.py` library. Cortex.cpp provides a Python client library as a **fork of OpenAI's [Python library](https://github.com/openai/openai-python)** with additional methods for Local AI.
## Installation

```py
Expand All @@ -16,14 +16,14 @@ pip install @janhq/cortex-python

## Usage

1. Replace the OpenAI import with Cortex in your application:
1. Replace the OpenAI import with Cortex.cpp in your application:

```diff
- from openai import OpenAI
+ from @janhq/cortex-python import Cortex
```

2. Modify the initialization of the client to use Cortex:
2. Modify the initialization of the client to use Cortex.cpp:

```diff
- client = OpenAI(api_key='your-api-key')
Expand All @@ -36,7 +36,7 @@ pip install @janhq/cortex-python
```py
from @janhq/cortex-python import Cortex

client = OpenAI(base_url="http://localhost:1337", api_key="cortex")
client = OpenAI(base_url="http://localhost:3928", api_key="cortex")

model = "TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF"
client.models.start(model=model)
Expand Down
18 changes: 9 additions & 9 deletions docs/basic-usage/server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex has an [API server](https://cortex.so/api-reference) that runs at `localhost:1337`.
Cortex has an [API server](https://cortex.so/api-reference) that runs at `localhost:3928`.


## Usage
Expand All @@ -28,10 +28,10 @@ cortex --dataFolder <dataFolderPath>
```bash
# Pull a model
curl --request POST \
--url http://localhost:1337/v1/models/mistral/pull
--url http://localhost:3928/v1/models/mistral/pull
# Start the model
curl --request POST \
--url http://localhost:1337/v1/models/mistral/start \
--url http://localhost:3928/v1/models/mistral/start \
--header 'Content-Type: application/json' \
--data '{
"prompt_template": "system\n{system_message}\nuser\n{prompt}\nassistant",
Expand All @@ -47,19 +47,19 @@ curl --request POST \
"flash_attn": true,
"cache_type": "f16",
"use_mmap": true,
"engine": "cortex.llamacpp"
"engine": "llamacpp"
}'
```
### Show the Model State
```bash
# Check the model status
curl --request GET \
--url http://localhost:1337/v1/system/events/model
--url http://localhost:3928/v1/system/events/model
```
### Chat with Model
```bash
# Invoke the chat completions endpoint
curl http://localhost:1337/v1/chat/completions \
curl http://localhost:3928/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "",
Expand All @@ -85,11 +85,11 @@ curl http://localhost:1337/v1/chat/completions \
```bash
# Stop a model
curl --request POST \
--url http://localhost:1337/v1/models/mistral/stop
--url http://localhost:3928/v1/models/mistral/stop
```
### Pull Model
```bash
# Pull a model
curl --request POST \
--url http://localhost:1337/v1/models/mistral/pull
--url http://localhost:3928/v1/models/mistral/pull
```
2 changes: 1 addition & 1 deletion docs/benchmarking-architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ slug: "benchmarking-architecture"


:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

## Architecture
Expand Down
4 changes: 2 additions & 2 deletions docs/built-in-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,10 @@ import TabItem from "@theme/TabItem";


:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex maintains a collection of built-in models that cover the most popular open-source models.
Cortex.cpp maintains a collection of built-in models that cover the most popular open-source models.

## Cortex Model Repos
Built-in models are [Cortex Model Repositories](/docs/hub/cortex-hub) hosted on HuggingFace and pre-compiled for different engines, allowing one model to have multiple branches in various formats.
Expand Down
12 changes: 6 additions & 6 deletions docs/chat-completions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

Cortex's Chat API is compatible with OpenAI’s [Chat Completions](https://platform.openai.com/docs/api-reference/chat) endpoint. It is a drop-in replacement for local inference.
Expand All @@ -32,7 +32,7 @@ cortex chat --model mistral
<Tabs>
<TabItem value="single" label="Single Request Example">
```bash
curl http://localhost:1337/v1/chat/completions \
curl http://localhost:3928/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "",
Expand All @@ -58,7 +58,7 @@ cortex chat --model mistral
</TabItem>
<TabItem value="dialogue" label="Dialogue Request Example">
```bash
curl http://localhost:1337/v1/chat/completions \
curl http://localhost:3928/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"messages": [
Expand Down Expand Up @@ -131,9 +131,9 @@ To configure each engine, refer to the [`cortex engines init`](/docs/cli/engines
Learn more about our engine architecture:

- cortex.cpp
- [cortex.llamacpp](/docs/cortex-llamacpp)
- cortex.tensorrt-llm
- [cortex.onnx](/docs/cortex-onnx)
- [llamacpp](/docs/cortex-llamacpp)
- tensorrt-llm
- [onnx](/docs/cortex-onnx)

### Multiple Remote APIs

Expand Down
2 changes: 1 addition & 1 deletion docs/cli/benchmark.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ For example, it will return the following:
},
model: {
modelId: 'tinyllama',
engine: 'cortex.llamacpp',
engine: 'llamacpp',
status: 'running',
duration: '2h 38m 44s',
ram: '-',
Expand Down
8 changes: 4 additions & 4 deletions docs/cli/chat.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: "chat"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex chat`
Expand All @@ -20,7 +20,7 @@ This command starts a chat session with a specified model, allowing you to inter
## Usage

```bash
cortex chat <model_id> [options]
cortex chat [options] <model_id> <message>
```
:::info
This command uses a `model_id` from the model that you have downloaded or available in your file system.
Expand All @@ -30,8 +30,8 @@ This command uses a `model_id` from the model that you have downloaded or availa

| Option | Description | Required | Default value | Example |
| ----------------------------- | ----------------------------------------------------------------------------------------------- | -------- | ------------- | ----------------------------- |
| `model_id` | Model ID to chat with. | No | - | `mistral` |
| `-m`, `--message <message>` | Message to send to the model | No | - | `-m "Hello, model!"` |
| `model_id` | Model ID to chat with. | Yes | - | `mistral` |
| `-m`, `--message <message>` | Message to send to the model | Yes | - | `-m "Hello, model!"` |
| `-h`, `--help` | Display help information for the command. | No | - | `-h` |

<!-- | `-t`, `--thread <thread_id>` | Thread ID. If not provided, will create new thread | No | - | `-t 98765` | -->
Expand Down
2 changes: 1 addition & 1 deletion docs/cli/configs/get.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Cortex configs subcommands.
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex configs get`
Expand Down
2 changes: 1 addition & 1 deletion docs/cli/configs/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Cortex Configs
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex configs`
Expand Down
2 changes: 1 addition & 1 deletion docs/cli/configs/list.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Cortex configs subcommands.
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex configs list`
Expand Down
2 changes: 1 addition & 1 deletion docs/cli/configs/set.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Cortex configs subcommands.
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex configs set`
Expand Down
17 changes: 7 additions & 10 deletions docs/cli/cortex.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@ slug: /cli
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# Cortex
:::info
This is the initial command you need to run to start using Cortex.
This is the initial command you need to run to start using Cortex.cpp.
:::

This command starts the Cortex process and the API server, which runs on port `1337` by default.
This command starts the Cortex.cpp process and the API server, which runs on port `1337` by default.

## Usage

Expand All @@ -25,11 +25,11 @@ cortex [command] [options]

| Option | Description | Required | Default value | Example |
| ---------------------------- | ----------------------------------------- | -------- | ------------- | ----------------------------- |
| `-a`, `--address <address>` | Address to use. | No | - | `-a 192.168.1.1` |
| `-p`, `--port <port>` | Port to serve the application. | No | - | `-p 1337` |
| `-v`, `--version` | Show version. | No | - | `-v` |
| `-h`, `--help` | Display help information for the command. | No | - | `-h` |
| `--verbose` | Show the detailed command logs | No | - | `--verbose` |
<!-- | `-a`, `--address <address>` | Address to use. | No | - | `-a 192.168.1.1` |
| `-p`, `--port <port>` | Port to serve the application. | No | - | `-p 1337` | -->
<!--| `--dataFolder <dataFolder>` | Set the data folder directory | No | - | `--dataFolder /path/to/data` | -->


Expand All @@ -46,11 +46,8 @@ For example:
- [cortex models](/docs/cli/models): Manage and configure models.
- [cortex chat](/docs/cli/chat): Send a chat request to a model.
- [cortex ps](/docs/cli/ps): Display active models and their operational status.
- [cortex presets](/docs/cli/presets): Show all the available model presets within Cortex.
- [cortex embeddings](/docs/cli/embeddings): Create an embedding vector representing the input text.
- [cortex benchmark](/docs/cli/benchmark): Benchmark and analyze the performance of a specific AI model using your system.
- [cortex engines](/docs/cli/engines): Manage Cortex engines.
- [cortex engines](/docs/cli/engines): Manage Cortex.cpp engines.
- [cortex pull|download](/docs/cli/pull): Download a model.
- [cortex run](/docs/cli/run): Shortcut to start a model and chat.
- [cortex telemetry](/docs/cli/telemetry): Retrieve telemetry logs for monitoring and analysis.
- [cortex stop](/docs/cli/stop): Stop the API server.
- [cortex update](/docs/cli/stop): Update the Cortex.cpp version.
2 changes: 1 addition & 1 deletion docs/cli/embeddings.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: "embeddings"
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex embeddings`
Expand Down
6 changes: 3 additions & 3 deletions docs/cli/engines/get.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ description: Cortex engines subcommands.
---

:::warning
🚧 Cortex is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
🚧 Cortex.cpp is currently under development. Our documentation outlines the intended behavior of Cortex, which may not yet be fully implemented in the codebase.
:::

# `cortex engines get`
Expand All @@ -23,7 +23,7 @@ For example, it returns the following:
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ (index) β”‚ Values β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ name β”‚ 'cortex.onnx' β”‚
β”‚ name β”‚ 'onnx' β”‚
β”‚ description β”‚ 'This extension enables chat completion API calls using the Cortex engine' β”‚
β”‚ version β”‚ '0.0.1' β”‚
β”‚ productName β”‚ 'Cortex Inference Engine' β”‚
Expand All @@ -38,6 +38,6 @@ To get an engine name, run the [`engines list`](/docs/cli/engines/list) command

| Option | Description | Required | Default value | Example |
|-------------------|-------------------------------------------------------|----------|---------------|-----------------|
| `name` | The name of the engine that you want to retrieve. | Yes | - | `cortex.llamacpp`|
| `name` | The name of the engine that you want to retrieve. | Yes | - | `llamacpp`|
| `-h`, `--help` | Display help information for the command. | No | - | `-h` |

Loading

0 comments on commit ae25ff5

Please sign in to comment.