Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local #503

Closed
gavinmclelland opened this issue Sep 24, 2023 · 0 comments · Fixed by #505
Closed

ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local #503

gavinmclelland opened this issue Sep 24, 2023 · 0 comments · Fixed by #505
Labels
Bug Something isn't working

Comments

@gavinmclelland
Copy link
Contributor

Describe the bug

When following the "Code-Llama on MacOS (Apple Silicon)" steps as described in the MACOS.md, the llama-cpp-python library installs as an x86_64 version instead of ARM64 on an Apple Silicon machine.

Reproduce

  1. Navigate to the MACOS.md guide.
  2. Follow the steps under "Code-Llama on MacOS (Apple Silicon)."
  3. Execute the command lipo -info [/path/to/]libllama.dylib.
  4. Non-fat file: /Users/[user]/miniconda3/envs/openinterpreter/lib/python3.11/site-packages/llama_cpp/libllama.dylib is architecture: x86_64

Expected behavior

After following the installation steps, I expected the libllama.dylib file to be of the arm64 architecture. Upon running:

interpreter --local

I anticipated that after the model is set, the Local LLM interface package would be found automatically. I did not expect to be prompted to install llama-cpp-python again.

Screenshots

Non-fat file llama_cpp libllama dylib is architecture x86_64

Local_LLM_interfacePackageNotFound_Install_llama-cpp-python

Open Interpreter version

0.1.5

Python version

Python 3.11.4

Operating System name and version

macOS 13.5.1

Additional context

The issue created by following the MACOS.md guide was resolved by running the following commands:

pip uninstall llama-cpp-python
CMAKE_ARGS="-DCMAKE_OSX_ARCHITECTURES=arm64 -DLLAMA_METAL=on" FORCE_CMAKE=1 pip install --upgrade --force-reinstall llama-cpp-python --no-cache-dir

llama_cpp_python arm64 metal support

issue resolved through CMAKE args
@gavinmclelland gavinmclelland added the Bug Something isn't working label Sep 24, 2023
gavinmclelland added a commit to gavinmclelland/open-interpreter that referenced this issue Sep 24, 2023
@gavinmclelland
Fix ARM64 llama-cpp-python Install on Apple Silicon
a3d39cc 
This commit updates the `MACOS.md` documentation to include detailed steps for correctly installing `llama-cpp-python` with ARM64 architecture support on Apple Silicon-based macOS systems. The update provides:

- A prerequisite check for Xcode Command Line Tools.
- Step-by-step installation instructions for `llama-cpp-python` with ARM64 and Metal support.
- A verification step to confirm the correct installation of `llama-cpp-python` for ARM64 architecture.
- An additional step for installing server components for `llama-cpp-python`.

This commit resolves the issue described in `ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local OpenInterpreter#503`.
@gavinmclelland gavinmclelland mentioned this issue Sep 24, 2023
Merged
11 tasks
unaidedelf8777 added a commit to unaidedelf8777/open-interpreter that referenced this issue Sep 29, 2023
@unaidedelf8777
Modified to comply with generator update, and so that no changes to t…
6ae3f20 
…he base interpreter class or anything in the core folder was needed.

Update README from base/main

merge rebased branch to main. (#2)

* fix: stop overwriting boolean config values

Without the default set to None, any boolean CLI flag that isn't passed reverts to its default state even if it is configured in the config.yaml file.

* The Generator Update (English docs)

* Improved --conversations, --config

---------

quality of life and error messages

errors and stuff again

re-add readline method because doc formatting removed it somehow

fix readline method of wrapper

added file upload and download functionality

finalized upload and download commands. tested stuff

visual

Improved --conversations, --config

The Generator Update (English docs)

fix: stop overwriting boolean config values

Without the default set to None, any boolean CLI flag that isn't passed reverts to its default state even if it is configured in the config.yaml file.

Update WINDOWS.md

Warns the user to re-launch cmd windows after installing llama locally

Fix ARM64 llama-cpp-python Install on Apple Silicon

This commit updates the `MACOS.md` documentation to include detailed steps for correctly installing `llama-cpp-python` with ARM64 architecture support on Apple Silicon-based macOS systems. The update provides:

- A prerequisite check for Xcode Command Line Tools.
- Step-by-step installation instructions for `llama-cpp-python` with ARM64 and Metal support.
- A verification step to confirm the correct installation of `llama-cpp-python` for ARM64 architecture.
- An additional step for installing server components for `llama-cpp-python`.

This commit resolves the issue described in `ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local OpenInterpreter#503`.

Broken empty message response

fix crash on unknwon command on call to display help message

removed unnecessary spaces

Update get_relevant_procedures.py

Fixed a typo in the instructions to the model

The Generator Update

The Generator Update

The Generator Update - Azure fix

The Generator Update - Azure function calling

The Generator Update - Azure fix

Better debugging

Better debugging

Proper TokenTrimming for new models

Generator Update Fixes (Updated Version)

Generator Update Quick Fixes

Added example JARVIS Colab Notebook

Added example JARVIS Colab Notebook

Skip wrap_in_trap on Windows

fix: allow args to have choices and defaults

This allows non-boolean args to define possible options and default values, which were ignored previously.

feat: add semgrep code scanning via --safe flag

This reintroduces the --safe functionality from OpenInterpreter#24.

--safe has 3 possible values auto, ask, and off

Code scanning is opt-in.

fix: default to 'off' for scan_code attribute

fix: toggle code_scan based on auto_run setting; update --scan docs

revert: undo default and choices change to cli.py

This is being removed from this PR in favor of a standalone fix in OpenInterpreter#511

feat: cleanup code scanning and convert to safe mode

docs: fix naming of safe_mode flag in README

fix: pass debug_mode flag into file cleanup for code scan

fix: remove extra tempfile import from scan_code util

Fixed first message inturruption error

Holding `--safe` docs for pip release

fix: stop overwriting safe_mode config.yaml setting with default in args

Fixed `%load` magic command

But I think we should deprecate it in favor of `--conversations`.

Generalized API key error message

Better model validation, better config debugging

Better config debugging

Better config debugging

Better config debugging

Better --config

Cleaned up initial message

Generator Update Quick Fixes II

Force then squashing (#3)
joshuavial pushed a commit to joshuavial/open-interpreter that referenced this issue Nov 16, 2023
@gavinmclelland
Fix ARM64 llama-cpp-python Install on Apple Silicon
de4fd81 
This commit updates the `MACOS.md` documentation to include detailed steps for correctly installing `llama-cpp-python` with ARM64 architecture support on Apple Silicon-based macOS systems. The update provides:

- A prerequisite check for Xcode Command Line Tools.
- Step-by-step installation instructions for `llama-cpp-python` with ARM64 and Metal support.
- A verification step to confirm the correct installation of `llama-cpp-python` for ARM64 architecture.
- An additional step for installing server components for `llama-cpp-python`.

This commit resolves the issue described in `ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local OpenInterpreter#503`.


Former-commit-id: a3d39cc
Former-commit-id: 47e5b5edf07cca168c654b43d54b25b37a7e5f4f
Former-commit-id: 83dabf0a1e17ca7e31f03e41fc0c3cbd4c38f8d7 [formerly 50cab63a8cde8aac50b5de75d93f77dd9c039ace]
Former-commit-id: 1d18d82f473e5087e5d8fe857b649c856d259b0d
joshuavial pushed a commit to joshuavial/open-interpreter that referenced this issue Nov 16, 2023
@gavinmclelland
Fix ARM64 llama-cpp-python Install on Apple Silicon
7645635 
This commit updates the `MACOS.md` documentation to include detailed steps for correctly installing `llama-cpp-python` with ARM64 architecture support on Apple Silicon-based macOS systems. The update provides:

- A prerequisite check for Xcode Command Line Tools.
- Step-by-step installation instructions for `llama-cpp-python` with ARM64 and Metal support.
- A verification step to confirm the correct installation of `llama-cpp-python` for ARM64 architecture.
- An additional step for installing server components for `llama-cpp-python`.

This commit resolves the issue described in `ARM64 Installation Issue with llama-cpp-python on Apple Silicon Macs for interpreter --local OpenInterpreter#503`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix ARM64 llama-cpp-python Install on Apple Silicon gavinmclelland/open-interpreter
1 participant
@gavinmclelland