Skip to content

Olive-ai 0.7.0
Compare
Choose a tag to compare
@shaahji shaahji released this 16 Oct 23:00
v0.7.0
2e77d71

Command Line Interface (CLI)

Introducing new command line interface for Olive with support to execute well-defined concrete workflows without user having to ever create or edit a config manually. CLI workflow commands can be chained i.e. output of one execution can be fed as input to the next, to facilitate ease of operations for the entire pipeline. Below is a list of few CLI workflow commands -

  • finetune - Fine-tune a model on a dataset using peft and optimize the model for ONNX Runtime
  • capture-onnx-graph: Capture ONNX graph for a Huggingface model.
  • auto-opt: Automatically optimize a model for performance.
  • quantize: Quantize model using given algorithm for desired precision and target.
  • tune-session-params: Automatically tune the session parameters for a ONNX model.
  • generate-adapter: Generate ONNX model with adapters as inputs.

Improvements

  • Added support for yaml based workflow config
  • Streamlined DataConfig management
  • Simplified workflow configuration
  • Added shared cache support for intermediate models and supporting data files
  • Added QuaRoT quantization pass for PyTorch models
  • Added support to evaluate generative PyTorch models
  • Streamlined support for user-defined evaluators
  • Enabled use of llm-evaluation-harness for generative model evaluations

Examples

  • Llama
    • Updated multi-lora example to use ORT genreate() API
    • Updated to demonstrate use of shared cache
  • Phi3
    • Updated to demonstrate evaluation using lm-eval harness
    • Updated to showcase search across three different QLoRA ranks
    • Added Vision tutorial
Assets 3
Loading