Skip to content
This repository has been archived by the owner on Oct 14, 2024. It is now read-only.

docs: README edits #198

Closed
dan-menlo opened this issue Sep 17, 2024 · 1 comment
Closed

docs: README edits #198

dan-menlo opened this issue Sep 17, 2024 · 1 comment
Assignees
Labels
type: documentation Improvements or additions to documentation

Comments

@dan-menlo
Copy link
Contributor

dan-menlo commented Sep 17, 2024

Goal

  • Simplify README
  • Make corrections
  • Link to Docs where appropriate

README Feedback

Structure

I would like to make a quick modification to the structure

## Overview

- Combine "About" and "Installation"

## Quickstart

## Built-in Model Library

## Alternate Installation Methods

### Beta & Nightly 

### Build from Source

About Section

  • Let's combine the "About" and "Installation" sections
  • We should include links only for the Stable release
  • We prefer "Download" link (vs. "cortexcpp.exe" filename)

You will need to ask @hiento09 for the exact Installers we are providing:

  • Mac might be universal
  • There might be a Network installer (i.e. pulls at install-time)

Quickstart

I would like to have a more comprehensive Quickstart, that covers our most common scenario:

# Start server
cortex start

# Pulls models
cortex pull <huggingface>
cortex pull llama3.1         # From our curated built-in model library

# Run and use model
cortex run llama3.1 
cortex chat llama3.1

Guide the user to use the API server

  • Cortex server is running at localhost:3928
  • make API request to cortex.cpp
  • Guide user that they can use this with many integrations
curl /chat/completions

Walk the user to stop the model, and the server

cortex stop model
cortex stop

Possible additional things to weave in (we can discuss in comments):

  • .cortexrc
  • Model folder (after pull, see where files are stored)
  • Editing a model.yaml

Built-in Model Library

I would like to combine all 3 tables into one, simplified version

Model llama.cpp
:gguf
TensorRT
:tensorrt
ONNXRuntime
:onnx
Command
llama3.1 cortex run llama3.1
mistral cortex run mistral
qwen2 cortex run qwen2

CLI Commands

REST API

Uninstall

  • We should have uninstall instructions here
  • Check with @hiento09 to make sure all details are captured

Alternate Installation

  • We should move Beta, and Nightly to this section (can re-use the table)
  • We should move the Build from Source to this section
@dan-menlo dan-menlo added this to Menlo Sep 17, 2024
@dan-menlo dan-menlo converted this from a draft issue Sep 17, 2024
@dan-menlo dan-menlo moved this to Scheduled in Menlo Sep 17, 2024
@dan-menlo dan-menlo added the type: documentation Improvements or additions to documentation label Sep 17, 2024
@irfanpena
Copy link
Contributor

@dan-homebrew I think for the following we should just detail them in the docs rather than the readme, as I think this requires a lot of explanation as the readme should just be a simple docs that really introduces what is Cortex.cpp:

  • .cortexrc
  • Editing a model.yaml

@irfanpena irfanpena closed this as completed by moving to Completed in Menlo Sep 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: documentation Improvements or additions to documentation
Projects
Archived in project
Development

No branches or pull requests

2 participants