Submit LLM to Homebrew

[simonw]

Getting it into homebrew-core would mean that they would build the Bottle files, which would hopefully resolve the issues with brew install simonw/llm/llm taking a long time and sometimes breaking.

[simonw]

https://docs.brew.sh/Adding-Software-to-Homebrew

I ran this:

brew audit --strict --new-formula --online llm

And got no errors.

[simonw]

https://docs.brew.sh/How-To-Open-a-Homebrew-Pull-Request is complicated.

[simonw]

I tried brew test:

% brew test llm
Error: Testing requires the latest version of simonw/llm/llm

So I'm upgrading now:

brew upgrade llm

...
==> Upgrading 1 outdated package:
simonw/llm/llm 0.5 -> 0.6
==> Fetching simonw/llm/llm

[simonw]

Tried this:

brew install --build-from-source llm

Got this error:

Last 15 lines from /Users/simon/Library/Logs/Homebrew/llm/17.pip:
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /opt/homebrew/Cellar/llm/0.6/libexec/bin/python3.11 /opt/homebrew/Cellar/llm/0.6/libexec/lib/python3.11/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /private/tmp/pip-build-env-rlv85tao/overlay --no-warn-script-location --no-binary :all: --only-binary :none: -i https://pypi.org/simple -- 'maturin>=1,<2' 'typing-extensions >=4.6.0,!=4.7.0'
  cwd: [inherit]
  Installing build dependencies: finished with status 'error'
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[notice] A new release of pip is available: 23.0.1 -> 23.2.1
[notice] To update, run: python3.11 -m pip install --upgrade pip

Here's the full content of/Users/simon/Library/Logs/Homebrew/llm/17.pip: https://gist.github.com/simonw/5af1bf2501b8a71ac77ead46f0ff686d

Key snippets:

Processing /private/tmp/llm--pydantic-core-20230723-77664-1qubplp/pydantic_core-2.3.0
  Installing build dependencies: started
  Running command pip subprocess to install build dependencies
  DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
  Collecting maturin<2,>=1
    Downloading maturin-1.1.0.tar.gz (181 kB)
...
  Building wheels for collected packages: maturin, typing-extensions
    Building wheel for maturin (pyproject.toml): started
    Building wheel for maturin (pyproject.toml): finished with status 'error'
    error: subprocess-exited-with-error

    × Building wheel for maturin (pyproject.toml) did not run successfully.
    │ exit code: 1
    ╰─> [35 lines of output]
        running bdist_wheel
        running build
        running build_py
        creating build
        creating build/lib.macosx-13-arm64-cpython-311
        creating build/lib.macosx-13-arm64-cpython-311/maturin
        copying maturin/__init__.py -> build/lib.macosx-13-arm64-cpython-311/maturin
        copying maturin/import_hook.py -> build/lib.macosx-13-arm64-cpython-311/maturin
        copying maturin/__main__.py -> build/lib.macosx-13-arm64-cpython-311/maturin
        running egg_info
        creating maturin.egg-info
        writing maturin.egg-info/PKG-INFO
        writing dependency_links to maturin.egg-info/dependency_links.txt
        writing requirements to maturin.egg-info/requires.txt
        writing top-level names to maturin.egg-info/top_level.txt
        writing manifest file 'maturin.egg-info/SOURCES.txt'
        reading manifest file 'maturin.egg-info/SOURCES.txt'
        reading manifest template 'MANIFEST.in'
        warning: no files found matching '*.json' under directory 'src/python_interpreter'
        writing manifest file 'maturin.egg-info/SOURCES.txt'
        warning: build_py: byte-compiling is disabled, skipping.

        running build_ext
        running build_rust
        error: can't find Rust compiler
...
  Failed to build maturin
  ERROR: Could not build wheels for maturin, which is required to install pyproject.toml-based projects

[simonw]

I'm using this to open the formula for editing:

brew edit llm

[simonw]

OK this should hopefully fix that: simonw/homebrew-llm@5a69a7c

Now running this again:

brew install --build-from-source llm

[simonw]

Here's what "new formula" PRs to Homebrew look like: https://github.com/Homebrew/homebrew-core/pulls?q=is%3Apr+label%3A%22new+formula%22

Looks like you submit one without the bottle bits: Homebrew/homebrew-core@af9e6a4

And a bot then adds them for you: Homebrew/homebrew-core@477fd6f

[simonw]

brew install --build-from-source llm worked!

[simonw]

brew test llm

That passed.

brew audit --strict --online llm

Got three failures:

llm
  * Dependency 'python3' is an alias; use the canonical name '[email protected]'.
  * line 11, col 3: dependency "rust" (line 11) should be put before dependency "python3" (line 9)
  * line 13, col 1: Extra blank line detected.
Error: 3 problems in 1 formula detected.

[simonw]

Fixing that by editing the first few lines of llm.rb to:

class Llm < Formula
  include Language::Python::Virtualenv

  desc "Access large language models from the command-line"
  homepage "https://github.com/simonw/llm"
  url "https://files.pythonhosted.org/packages/64/a3/24218c5cdada7e9eac5425bb702d734605d2f16bf8450c3a81571dd34558/llm-0.6.tar.gz"
  sha256 "6226bcbb4c79affd85870a69a09a5b47aec6bc549c65b620f086253424e3410a"

  depends_on "rust" => :build
  depends_on "[email protected]"

[simonw]

brew audit --strict --online llm passes now.

[simonw]

I have a https://github.com/simonw/homebrew-core fork already but it's wildly out of date. I'm going to destroy and recreate it.

[simonw]

One more audit:

brew audit --new

[simonw]

It failed!

llm
  * Formulae in homebrew/core must specify a license.
Error: 1 problem in 1 formula detected.

[simonw]

Looks like I need to add license "Apache-2.0"

[simonw]

That audit passes as well now. I force pushed that change to my llm branch.

[simonw]

Opened my PR against homebrew-core:

• llm 0.6 (new formula) Homebrew/homebrew-core#137353

[simonw]

Sequence of commands:

brew tap --force homebrew/core
cd "$(brew --repository homebrew/core)"\n
pwd
git remote add simonw https://github.com/simonw/homebrew-core.git
git checkout master
brew update
git checkout -b llm origin/master
git add Formula/llm.rb
git commit
git push https://github.com/simonw/homebrew-core/ llm
brew audit --new llm
brew edit llm
git status
git diff
git commit --amend -a
git push https://github.com/simonw/homebrew-core/ llm --force
brew audit --new llm

[simonw]

Now blocked waiting to see if the PR gets accepted.

[simonw]

Just spotted this in https://docs.brew.sh/Acceptable-Formulae#niche-or-self-submitted-stuff

We frown on authors submitting their own work unless it is very popular.

Fingers crossed that doesn't block this.

[simonw]

Got feedback that the test wasn't good enough, so I built a dummy server that can run a fake prompt like this:

OPENAI_API_KEY='x' \
OPENAI_API_BASE='https://openai-canned-completion.vercel.app/v1' \
llm 'hello'

Always outputs:

Hello! How can I assist you today?

[simonw]

That Vercel app looks like this:

import base64
import json

lines = """
{"type": "http.response.start", "status": 200, "headers": [[{"$base64": "ZGF0ZQ=="}, {"$base64": "U3VuLCAyMyBKdWwgMjAyMyAxOToyMzoxNiBHTVQ="}], [{"$base64": "Y29udGVudC10eXBl"}, {"$base64": "dGV4dC9ldmVudC1zdHJlYW0="}], [{"$base64": "dHJhbnNmZXItZW5jb2Rpbmc="}, {"$base64": "Y2h1bmtlZA=="}], [{"$base64": "Y29ubmVjdGlvbg=="}, {"$base64": "a2VlcC1hbGl2ZQ=="}], [{"$base64": "YWNjZXNzLWNvbnRyb2wtYWxsb3ctb3JpZ2lu"}, {"$base64": "Kg=="}], [{"$base64": "Y2FjaGUtY29udHJvbA=="}, {"$base64": "bm8tY2FjaGUsIG11c3QtcmV2YWxpZGF0ZQ=="}], [{"$base64": "b3BlbmFpLW9yZ2FuaXphdGlvbg=="}, {"$base64": "dXNlci1yM2U2MWZwYWswNGNiYW9rcDVidW9hZTQ="}], [{"$base64": "b3BlbmFpLXByb2Nlc3NpbmctbXM="}, {"$base64": "OQ=="}], [{"$base64": "b3BlbmFpLXZlcnNpb24="}, {"$base64": "MjAyMC0xMC0wMQ=="}], [{"$base64": "c3RyaWN0LXRyYW5zcG9ydC1zZWN1cml0eQ=="}, {"$base64": "bWF4LWFnZT0xNTcyNDgwMDsgaW5jbHVkZVN1YkRvbWFpbnM="}], [{"$base64": "eC1yYXRlbGltaXQtbGltaXQtcmVxdWVzdHM="}, {"$base64": "MzUwMA=="}], [{"$base64": "eC1yYXRlbGltaXQtbGltaXQtdG9rZW5z"}, {"$base64": "OTAwMDA="}], [{"$base64": "eC1yYXRlbGltaXQtcmVtYWluaW5nLXJlcXVlc3Rz"}, {"$base64": "MzQ5OQ=="}], [{"$base64": "eC1yYXRlbGltaXQtcmVtYWluaW5nLXRva2Vucw=="}, {"$base64": "ODk5ODA="}], [{"$base64": "eC1yYXRlbGltaXQtcmVzZXQtcmVxdWVzdHM="}, {"$base64": "MTdtcw=="}], [{"$base64": "eC1yYXRlbGltaXQtcmVzZXQtdG9rZW5z"}, {"$base64": "MTJtcw=="}], [{"$base64": "eC1yZXF1ZXN0LWlk"}, {"$base64": "NDhjZmI5MDI3ZDc2ZTYyYzczOTJkNTEyZjViMzAwNTY="}], [{"$base64": "Y2YtY2FjaGUtc3RhdHVz"}, {"$base64": "RFlOQU1JQw=="}], [{"$base64": "c2VydmVy"}, {"$base64": "Y2xvdWRmbGFyZQ=="}], [{"$base64": "Y2YtcmF5"}, {"$base64": "N2ViNjRiZmIxOTBmN2NjZS1MQVg="}], [{"$base64": "YWx0LXN2Yw=="}, {"$base64": "aDM9Ijo0NDMiOyBtYT04NjQwMA=="}]]}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsicm9sZSI6ImFzc2lzdGFudCIsImNvbnRlbnQiOiIifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IkhlbGxvIn0sImZpbmlzaF9yZWFzb24iOm51bGx9XX0KCg=="}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiEifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiBIb3cifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiBjYW4ifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiBJIn0sImZpbmlzaF9yZWFzb24iOm51bGx9XX0KCg=="}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiBhc3Npc3QifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiB5b3UifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6IiB0b2RheSJ9LCJmaW5pc2hfcmVhc29uIjpudWxsfV19Cgo="}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnsiY29udGVudCI6Ij8ifSwiZmluaXNoX3JlYXNvbiI6bnVsbH1dfQoK"}, "more_body": true}
{"type": "http.response.body", "body": {"$base64": "ZGF0YTogeyJpZCI6ImNoYXRjbXBsLTdmWXhneTZRaTJnMlltTjBPb2hDTXFjYkxibFpMIiwib2JqZWN0IjoiY2hhdC5jb21wbGV0aW9uLmNodW5rIiwiY3JlYXRlZCI6MTY5MDE0MDE5NiwibW9kZWwiOiJncHQtMy41LXR1cmJvLTA2MTMiLCJjaG9pY2VzIjpbeyJpbmRleCI6MCwiZGVsdGEiOnt9LCJmaW5pc2hfcmVhc29uIjoic3RvcCJ9XX0KCmRhdGE6IFtET05FXQoK"}, "more_body": true}
{"type": "http.response.body", "more_body": false}
""".strip().split("\n")

def as_base64(dct):
    if '$base64' in dct:
        return base64.b64decode(dct["$base64"])
    return dct

messages = []
for line in lines:
    messages.append(json.loads(line, object_hook=as_base64))

async def app(scope, receive, send):
    if scope["type"] != "http":
        return
    for message in messages:
        await send(message)

[simonw]

Here's the code for that Vercel app: https://github.com/simonw/openai-canned-completion

[simonw]

It's in:

• Homebrew/homebrew-core@a67dd99
• https://github.com/Homebrew/homebrew-core/blob/master/Formula/llm.rb