Code

Add LLM sample for DirectML#1082

Code

Merged

guotuofeng merged 57 commits intomainmicrosoft/Olive:mainfrom user/pavignol/llm-combined-cleanmicrosoft/Olive:user/pavignol/llm-combined-cleanCopy head branch name to clipboardApr 18, 2024

+2,744-3,659

Commits

Commits on Dec 7, 2023

• Add GQA support (batch version still looks buggy?)

  PatriceVignola

  committed
  0b55774

• Fix batching

  PatriceVignola

  committed
  b3dba4c

• Fix chat app pipeline

  PatriceVignola

  committed
  9a17f08

Commits on Dec 8, 2023

• Remove argmax sampling model

  PatriceVignola

  committed
  4e3b686

Commits on Dec 17, 2023

• Merge branch 'main' of https://github.com/microsoft/Olive into user/pavignol/add-llama-gqa-support

  PatriceVignola

  committed
  acbff0d

• Use HF version of LLaMA

  PatriceVignola

  committed
  0cbf150

• Use json config from huggingface instead of tokenizer.model

  PatriceVignola

  committed
  b7c7f3e

Commits on Dec 18, 2023

• Add kv broadcasting (not working yet)

  PatriceVignola

  committed
  bc92843

• Change key permutate location to enable MHA fusion

  PatriceVignola

  committed
  69ab184

• Onboard mistral

  PatriceVignola

  committed
  de3d8d0

• Rename the llama_v2 example to llm

  PatriceVignola

  committed
  16002a5

• Fix chat app

  PatriceVignola

  committed
  e73a376

• Remove non-chat llama from list of supported models

  PatriceVignola

  committed
  8fdbc74

• Allow multiple models to be generated at the same time

  PatriceVignola

  committed
  9e13431

• Remove llama-2-7b from the list of models

  PatriceVignola

  committed
  ba3d44c

Commits on Dec 19, 2023

• Fix models

  PatriceVignola

  committed
  3df03ed

Commits on Jan 4, 2024

• Remove onnxruntime-directml 1.16.2 dependency

  PatriceVignola

  committed
  b9d2b6e

Commits on Jan 17, 2024

• Perf improvements

  PatriceVignola

  committed
  f06a339

Commits on Jan 21, 2024

• Use torch LayerNorm instead of custom one

  PatriceVignola

  committed
  79c1a56

• Add residual connection post layernorm option

  PatriceVignola

  committed
  884e5ad

• Add option to keep LayerNorm nodes in fp32

  PatriceVignola

  committed
  2852811

• Include model_type parameter

  PatriceVignola

  committed
  ccc3e97

• Fix layer norm

  PatriceVignola

  committed
  a28a01e

• Add default chat template option

  PatriceVignola

  committed
  212b060

Commits on Jan 28, 2024

• Add LLaVA support

  PatriceVignola

  committed
  b37949b

• Fix chat app

  PatriceVignola

  committed
  437b504

Commits on Jan 30, 2024

• Add codellama

  PatriceVignola

  committed
  69130ba

• Add OpenOrca Mistral

  PatriceVignola

  committed
  f141cbc

Commits on Feb 1, 2024

• Add tiiuae/falcon-7b-instruct model support to llm-combined (#907)

  Show description for f7dbe6d
  aamajumder

  authored
  f7dbe6d

• Onboard more models

  PatriceVignola

  committed
  04223f9

• Merge branch 'user/pavignol/llm-combined' of https://github.com/microsoft/Olive into user/pavignol/llm-combined

  PatriceVignola

  committed
  9aa5acd

• WIP

  PatriceVignola

  committed
  6947392

Commits on Feb 20, 2024

• Add AWQ (WIP)

  PatriceVignola

  committed
  5381d24

• Cleanup

  PatriceVignola

  committed
  7577f72

Commits on Mar 5, 2024

• Add Phi support

  PatriceVignola

  committed
  f29a123

Commits on Mar 21, 2024

• Fix opset issue

  PatriceVignola

  committed
  496f977

Commits on Apr 14, 2024

• WIP

  PatriceVignola

  committed
  3961bd2

• Merge branch 'main' of https://github.com/microsoft/Olive into user/pavignol/llm-combined-clean

  PatriceVignola

  committed
  92dcd23

• Fix Phi-2

  PatriceVignola

  committed
  5d54bda

• Update README and delete phi/dolly-v2 folders

  PatriceVignola

  committed
  7e10835

Commits on Apr 15, 2024

• Add block_size and bit_size options

  PatriceVignola

  committed
  4c5e9e9

• Add local model support

  PatriceVignola

  committed
  6ef9bd6

• Merge branch 'main' of https://github.com/microsoft/Olive into user/pavignol/llm-combined-clean

  PatriceVignola

  committed
  c73f55d

• Revert optimum_merging changes

  PatriceVignola

  committed
  1e0f8d0

• Address PR comments

  PatriceVignola

  committed
  25fdc62

• Fix lint errors

  PatriceVignola

  committed
  f2846a5

• Use sys.exit(1) instead of exit(1)

  PatriceVignola

  committed
  04b46c2

• Fix lint errors

  PatriceVignola

  committed
  4346041

• Update examples.md

  PatriceVignola

  committed
  b7637a9

• Fixes

  PatriceVignola

  committed
  fa18cab

• Remove dolly from example.md

  PatriceVignola

  committed
  6225f08

• Make NUM_KEY_VALUE_HEADS_NAMES a tuple

  PatriceVignola

  committed
  01341af

Commits on Apr 16, 2024

• Address PR comments

  PatriceVignola

  committed
  424dbcd

• Address PR comments

  PatriceVignola

  committed
  b90f9f4

• Fix dataloader and update neural-compressor dependency

  PatriceVignola

  committed
  eafe308

• Merge branch 'main' of https://github.com/microsoft/Olive into user/pavignol/llm-combined-clean

  PatriceVignola

  committed
  0c70dae

Commits on Apr 18, 2024

• Address PR comments

  PatriceVignola

  committed
  be09c7a