This is a plugin for llm that uses GitHub Models via the Azure AI Inference SDK.
$ llm install llm-github-models
To set the API key, use the llm keys set github command or use the GITHUB_MODELS_KEY environment variable.
To get an API key, create a personal access token (PAT) inside GitHub Settings.
Learn about rate limits here
All model names are affixed with github/ to distinguish the OpenAI ones from the builtin models.
$ llm prompt 'top facts about cheese' -m github/mistral-large
Sure, here are some interesting facts about cheese:
1. There are over 2000 types of cheese: The variety of cheese is vast, with different flavors, textures, and aromas. This is due to factors like the type of milk used, the aging process, and the specific bacteria and mold cultures involved.
2. Cheese is an ancient food: The earliest evidence of cheese-making dates back to around 6000 BC, found in ancient Polish sites.
Multi-modal vision models can accept image attachments using the LLM attachments options:
llm -m github/Llama-3.2-11B-Vision-Instruct "Describe this image" -a https://static.simonwillison.net/static/2024/pelicans.jpgProduces
This image depicts a dense gathering of pelicans, with the largest birds situated in the center, showcasing their light brown plumage and long, pointed beaks. The pelicans are standing on a rocky shoreline, with a serene body of water behind them, characterized by its pale blue hue and gentle ripples. In the background, a dark, rocky cliff rises, adding depth to the scene.
The overall atmosphere of the image exudes tranquility, with the pelicans seemingly engaging in a social gathering or feeding activity. The photograph's clarity and focus on the pelicans' behavior evoke a sense of observation and appreciation for the natural world.