diff --git a/README.md b/README.md index 7bbc5ceab..aef3ae290 100644 --- a/README.md +++ b/README.md @@ -70,64 +70,150 @@ This monorepo contains two projects: CortexJS and CortexCPP. └── ... ``` -## Installation +# Install +## Prerequisites -### NPM Install +### **Dependencies** + +Before installation, ensure that you have installed the following: + +- **Node.js**: Required for running the installation. +- **NPM**: Needed to manage packages. +- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page. + + + +### **Hardware** + +Ensure that your system meets the following requirements to run Cortex: + +- **OS**: + - MacOSX 13.6 or higher. + - Windows 10 or higher. + - Ubuntu 12.04 and later. +- **RAM (CPU Mode):** + - 8GB for running up to 3B models. + - 16GB for running up to 7B models. + - 32GB for running up to 13B models. +- **VRAM (GPU Mode):** + - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + +- **Disk**: At least 10GB for app and model download. + +## Cortex Installation + +To install Cortex, follow the steps below: + +### Step 1: Install Cortex + +Run the following command to install Cortex globally on your machine: + +```bash +# Install using NPM globally +npm i -g @janhq/cortex +``` + +### Step 2: Verify the Installation + +After installation, you can verify that Cortex is installed correctly by getting help information. + +```bash +# Get the help information +cortex -h +``` + +### Step 3: Initialize Cortex + +Once verified, you need to initialize the Cortex engine. + +1. Initialize the Cortex engine: + +``` +cortex init +``` + +1. Select between `CPU` and `GPU` modes. + +```bash +? Select run mode (Use arrow keys) +> CPU + GPU +``` + +2. Select between GPU types. -* Pre-install script: ```bash -npm pre-install script; platform specific (MacOS / Windows / Linux) +? Select GPU types (Use arrow keys) +> Nvidia + Others (Vulkan) ``` -* Tag based: -```json -npm install @janhq/cortex -npm install @janhq/cortex#cuda -npm install @janhq/cortex#cuda-avx512 -npm install @janhq/cortex#cuda-avx +3. Select CPU instructions (will be deprecated soon). + +```bash +? Select CPU instructions (Use arrow keys) +> AVX2 + AVX + AVX-512 ``` -### CLI Install Script +1. Cortex will download the required CPU instruction sets if you choose `CPU` mode. If you choose `GPU` mode, Cortex will download the necessary dependencies to use your GPU. +2. Once downloaded, Cortex is ready to use! + +### Step 4: Pull a model +From HuggingFace +```bash +cortex pull janhq/phi-3-medium-128k-instruct-GGUF +``` +From Jan Hub (TBD) ```bash -cortex init (AVX2 + Cuda) +cortex pull llama3 +``` -Enable GPU Acceleration? -1. Nvidia (default) - detected -2. AMD -3. Mac Metal +### Step 5: Chat +```bash +cortex run janhq/phi-3-medium-128k-instruct-GGUF +``` -Enter your choice: +## Run as an API server +```bash +cortex serve +``` -CPU Instructions -1. AVX2 (default) - Recommend based on what the user has -2. AVX (old CPU) -3. AVX512 +## Build from Source -Enter your choice: +To install Cortex from the source, follow the steps below: -Downloading cortex-cuda-avx.so........................25% +1. Clone the Cortex repository [here](https://github.com/janhq/cortex/tree/dev). +2. Navigate to the `cortex-js` folder. +3. Open the terminal and run the following command to build the Cortex project: -Cortex is ready! +```bash +npx nest build +``` -It seems like you have installed models from other applications. Do you want to import them? -1. Import from /Users/HOME/jan/models -2. Import from /Users/HOME/lmstudio/models -3. Import everything +1. Make the `command.js` executable: -Importing from /Users/HOME/jan/models..................17% +```bash +chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js' ``` -## Backend (jan app) +1. Link the package globally: -```json -POST /settings -{ - "gpu_enabled": true, - "gpu_family": "Nvidia", - "cpu_instructions": "AVX2" -} +```bash +npm link ``` -## Client Library Configuration +## Uninstall Cortex -TBD \ No newline at end of file +Run the following command to uninstall Cortex globally on your machine: + +``` +# Uninstall globally using NPM +npm uninstall -g @janhq/cortex +``` diff --git a/cortex-js/README.md b/cortex-js/README.md index 83729419a..62a9ae278 100644 --- a/cortex-js/README.md +++ b/cortex-js/README.md @@ -1,73 +1,147 @@ -

- Nest Logo -

- -[circleci-image]: https://img.shields.io/circleci/build/github/nestjs/nest/master?token=abc123def456 -[circleci-url]: https://circleci.com/gh/nestjs/nest - -

A progressive Node.js framework for building efficient and scalable server-side applications.

-

-NPM Version -Package License -NPM Downloads -CircleCI -Coverage -Discord -Backers on Open Collective -Sponsors on Open Collective - - Support us - -

- - -## Description - -[Nest](https://github.com/nestjs/nest) framework TypeScript starter repository. - -## Installation +# Install +## Prerequisites + +### **Dependencies** + +Before installation, ensure that you have installed the following: + +- **Node.js**: Required for running the installation. +- **NPM**: Needed to manage packages. +- **CPU Instruction Sets**: Available for download from the [Cortex GitHub Releases](https://github.com/janhq/cortex/releases) page. + + + +### **Hardware** + +Ensure that your system meets the following requirements to run Cortex: + +- **OS**: + - MacOSX 13.6 or higher. + - Windows 10 or higher. + - Ubuntu 12.04 and later. +- **RAM (CPU Mode):** + - 8GB for running up to 3B models. + - 16GB for running up to 7B models. + - 32GB for running up to 13B models. +- **VRAM (GPU Mode):** + - 6GB can load the 3B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 8GB can load the 7B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + - 12GB can load the 13B model (int4) with `ngl` at 120 ~ full speed on CPU/ GPU. + +- **Disk**: At least 10GB for app and model download. + +## Cortex Installation + +To install Cortex, follow the steps below: + +### Step 1: Install Cortex + +Run the following command to install Cortex globally on your machine: ```bash -$ yarn install +# Install using NPM globally +npm i -g @janhq/cortex ``` -## Running the app +### Step 2: Verify the Installation + +After installation, you can verify that Cortex is installed correctly by getting help information. ```bash -# development -$ yarn run start +# Get the help information +cortex -h +``` + +### Step 3: Initialize Cortex -# watch mode -$ yarn run start:dev +Once verified, you need to initialize the Cortex engine. -# production mode -$ yarn run start:prod +1. Initialize the Cortex engine: + +``` +cortex init ``` -## Test +1. Select between `CPU` and `GPU` modes. ```bash -# unit tests -$ yarn run test +? Select run mode (Use arrow keys) +> CPU + GPU +``` -# e2e tests -$ yarn run test:e2e +2. Select between GPU types. -# test coverage -$ yarn run test:cov +```bash +? Select GPU types (Use arrow keys) +> Nvidia + Others (Vulkan) ``` +3. Select CPU instructions (will be deprecated soon). -## Support +```bash +? Select CPU instructions (Use arrow keys) +> AVX2 + AVX + AVX-512 +``` -Nest is an MIT-licensed open source project. It can grow thanks to the sponsors and support by the amazing backers. If you'd like to join them, please [read more here](https://docs.nestjs.com/support). +1. Cortex will download the required CPU instruction sets if you choose `CPU` mode. If you choose `GPU` mode, Cortex will download the necessary dependencies to use your GPU. +2. Once downloaded, Cortex is ready to use! -## Stay in touch +### Step 4: Pull a model +From HuggingFace +```bash +cortex pull janhq/phi-3-medium-128k-instruct-GGUF +``` -- Author - [Kamil Myśliwiec](https://kamilmysliwiec.com) -- Website - [https://nestjs.com](https://nestjs.com/) -- Twitter - [@nestframework](https://twitter.com/nestframework) +From Jan Hub (TBD) +```bash +cortex pull llama3 +``` + +### Step 5: Chat +```bash +cortex run janhq/phi-3-medium-128k-instruct-GGUF +``` + +## Run as an API server +```bash +cortex serve +``` + +## Build from Source + +To install Cortex from the source, follow the steps below: + +1. Clone the Cortex repository [here](https://github.com/janhq/cortex/tree/dev). +2. Navigate to the `cortex-js` folder. +3. Open the terminal and run the following command to build the Cortex project: + +```bash +npx nest build +``` + +1. Make the `command.js` executable: + +```bash +chmod +x '[path-to]/cortex/cortex-js/dist/src/command.js' +``` + +1. Link the package globally: -## License +```bash +npm link +``` + +## Uninstall Cortex -Nest is [MIT licensed](LICENSE). +Run the following command to uninstall Cortex globally on your machine: + +``` +# Uninstall globally using NPM +npm uninstall -g @janhq/cortex +```