API access to Google's Gemini models
Install this plugin in the same environment as LLM.
llm install llm-gemini
Configure the model by setting a key called "gemini" to your API key:
llm keys set gemini
<paste key here>
You can also set the API key by assigning it to the environment variable LLM_GEMINI_KEY
.
Now run the model using -m gemini-1.5-pro-latest
, for example:
llm -m gemini-1.5-pro-latest "A joke about a pelican and a walrus"
A pelican walks into a seafood restaurant with a huge fish hanging out of its beak. The walrus, sitting at the bar, eyes it enviously.
"Hey," the walrus says, "That looks delicious! What kind of fish is that?"
The pelican taps its beak thoughtfully. "I believe," it says, "it's a billfish."
Other models are:
gemini-1.5-flash-latest
gemini-1.5-flash-8b-latest
- the least expensivegemini-exp-1114
- recent experimental #1gemini-exp-1121
- recent experimental #2gemini-exp-1206
- recent experimental #3gemini-2.0-flash-exp
- Gemini 2.0 Flash
Gemini models are multi-modal. You can provide images, audio or video files as input like this:
llm -m gemini-1.5-flash-latest 'extract text' -a image.jpg
Or with a URL:
llm -m gemini-1.5-flash-8b-latest 'describe image' \
-a https://static.simonwillison.net/static/2024/pelicans.jpg
Audio works too:
llm -m gemini-1.5-pro-latest 'transcribe audio' -a audio.mp3
And video:
llm -m gemini-1.5-pro-latest 'describe what happens' -a video.mp4
The Gemini prompting guide includes extensive advice on multi-modal prompting.
Use -o json_object 1
to force the output to be JSON:
llm -m gemini-1.5-flash-latest -o json_object 1 \
'3 largest cities in California, list of {"name": "..."}'
Outputs:
{"cities": [{"name": "Los Angeles"}, {"name": "San Diego"}, {"name": "San Jose"}]}
Gemini models can write and execute code - they can decide to write Python code, execute it in a secure sandbox and use the result as part of their response.
To enable this feature, use -o code_execution 1
:
llm -m gemini-1.5-pro-latest -o code_execution 1 \
'use python to calculate (factorial of 13) * 3'
To chat interactively with the model, run llm chat
:
llm chat -m gemini-1.5-pro-latest
The plugin also adds support for the text-embedding-004
embedding model.
Run that against a single string like this:
llm embed -m text-embedding-004 -c 'hello world'
This returns a JSON array of 768 numbers.
This command will embed every README.md
file in child directories of the current directory and store the results in a SQLite database called embed.db
in a collection called readmes
:
llm embed-multi readmes --files . '*/README.md' -d embed.db -m text-embedding-004
You can then run similarity searches against that collection like this:
llm similar readmes -c 'upload csvs to stuff' -d embed.db
See the LLM embeddings documentation for further details.
To set up this plugin locally, first checkout the code. Then create a new virtual environment:
cd llm-gemini
python3 -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
llm install -e '.[test]'
To run the tests:
pytest
This project uses pytest-recording to record Gemini API responses for the tests.
If you add a new test that calls the API you can capture the API response like this:
PYTEST_GEMINI_API_KEY="$(llm keys get gemini)" pytest --record-mode once
You will need to have stored a valid Gemini API key using this command first:
llm keys set gemini
# Paste key here