Skip to content

Commit

Permalink
docs: Update README.md (#767)
Browse files Browse the repository at this point in the history
  • Loading branch information
freelerobot authored and irfanpena committed Jun 26, 2024
1 parent ce95358 commit 2f12165
Showing 1 changed file with 16 additions and 72 deletions.
88 changes: 16 additions & 72 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Cortex - CLI
# Cortex
<p align="center">
<img alt="cortex-cpplogo" src="https://raw.githubusercontent.com/janhq/cortex/dev/assets/cortex-banner.png">
</p>
Expand All @@ -11,86 +11,27 @@
> ⚠️ **Cortex is currently in Development**: Expect breaking changes and bugs!
## About
Cortex is an openAI-compatible local AI server that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and a Typescript client library. It can be used as a standalone server, or imported as a library.
Cortex is an OpenAI-compatible AI engine that developers can use to build LLM apps. It is packaged with a Docker-inspired command-line interface and client libraries. It can be used as a standalone server or imported as a library.

Cortex currently supports two inference engines:
Cortex currently supports 3 inference engines:

- Llama.cpp
- ONNX Runtime
- TensorRT-LLM

> Read more about Cortex at https://jan.ai/cortex
## Quicklinks
**Cortex**:
- [Website](https://jan.ai/)
- [GitHub](https://github.com/janhq/cortex)
- [User Guides](https://jan.ai/cortex)
- [API reference](https://jan.ai/api-reference)

## Prerequisites

### **Dependencies**

Before installation, ensure that you have installed the following:

- **Node.js**: version 18 and above is required to run the installation.
- **NPM**: Needed to manage packages.
- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page.


>💡 The **CPU instruction sets** are not required for the initial installation of Cortex. This dependency will be automatically installed during the Cortex initialization if they are not already on your system.


### **Hardware**

Ensure that your system meets the following requirements to run Cortex:

- **OS**:
- MacOSX 13.6 or higher.
- Windows 10 or higher.
- Ubuntu 12.04 and later.
- **RAM (CPU Mode):**
- 8GB for running up to 3B models.
- 16GB for running up to 7B models.
- 32GB for running up to 13B models.
- **VRAM (GPU Mode):**

- 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.
- 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU.

- **Disk**: At least 10GB for app and model download.
- [Homepage](https://cortex.jan.ai/)
- [Docs](https://cortex.jan.ai/docs/)

## Quickstart
To install Cortex CLI, follow the steps below:
1. Install the Cortex NPM package globally:
``` bash
npm i -g @janhq/cortex
```
> Cortex automatically detects your CPU and GPU, downloading the appropriate CPU instruction sets and required dependencies to optimize GPU performance.
2. Download a GGUF model from Hugging Face:
``` bash
# Pull a model most compatible with your hardware
cortex pull llama3

# Pull a specific variant with `repo_name:branch`
cortex pull llama3:7b

# Pull a model with the HuggingFace `model_id`
cortex pull microsoft/Phi-3-mini-4k-instruct-gguf
```
3. Load the model:
``` bash
cortex models start llama3:7b
```
Visit [Quickstart](https://cortex.jan.ai/docs/quickstart) to get started.

4. Start chatting with the model:
``` bash
cortex chat tell me a joke
npm i -g @janhq/cortex
cortex run llama3
```


## Run as an API server
To run Cortex as an API server:
```bash
cortex serve
Expand Down Expand Up @@ -120,9 +61,10 @@ chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js'
npm link
```

## Cortex CLI Command
The following CLI commands are currently available:
> ⚠️ **Cortex is currently in Development**: More commands will be added soon!
## Cortex CLI Commands

The following CLI commands are currently available.
See [CLI Reference Docs](https://cortex.jan.ai/docs/cli) for more information.

```bash

Expand All @@ -142,6 +84,7 @@ The following CLI commands are currently available:
models stop Stop a specified model.
models update Update the configuration of a specified model.
```
## Uninstall Cortex
Run the following command to uninstall Cortex globally on your machine:
Expand All @@ -150,7 +93,8 @@ Run the following command to uninstall Cortex globally on your machine:
# Uninstall globally using NPM
npm uninstall -g @janhq/cortex
```
## Contact Support
- For support, please file a GitHub ticket.
- For questions, join our Discord [here](https://discord.gg/FTk2MvZwJH).
- For long-form inquiries, please email [[email protected]](mailto:[email protected]).
- For long-form inquiries, please email [[email protected]](mailto:[email protected]).

0 comments on commit 2f12165

Please sign in to comment.