Local Voice Chatbot | Seeed Studio Wiki

[giscus[bot]]

Local Voice Chatbot | Seeed Studio Wiki

Deploy Riva and Llama2 on reComputer

https://wiki.seeedstudio.com/Local_Voice_Chatbot/

[jasonthenderson]

Getting failures trying to launch the text-generation-inference container - it says it can't find BitsAndBytes and a number of other libraries.

Any ideas?

Namespace(packages=['text-generation-inference'], prefer=['local', 'registry', 'build'], disable=[''], user='dustynv', output='/tmp/autotag', quiet=False, verbose=False)
-- L4T_VERSION=36.2.0 JETPACK_VERSION=6.0 CUDA_VERSION=12.2
-- Finding compatible container image for ['text-generation-inference']
fatal: invalid reference: origin/dev
ERROR:root:failed to update container registry cache from GitHub (/home/jason/jetson-containers/data/containers.json)
ERROR:root:return code 128 > cd /home/jason/jetson-containers && git fetch origin dev --quiet && git checkout --quiet origin/dev -- data/containers.json

Couldn't find a compatible container for text-generation-inference, would you like to build it? [y/N] y
-- Package text-generation-inference has missing dependencies, disabling... ("couldn't find package: bitsandbytes")
-- Package l4t-tensorflow:tf1 has missing dependencies, disabling... ("couldn't find package: tensorflow")
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/jason/jetson-containers/jetson_containers/tag.py", line 58, in
image = find_container(args.packages[0], prefer_sources=args.prefer, disable_sources=args.disable, user=args.user, quiet=args.quiet)
File "/home/jason/jetson-containers/jetson_containers/container.py", line 532, in find_container
return build_container('', package) #, simulate=True)
File "/home/jason/jetson-containers/jetson_containers/container.py", line 66, in build_container
packages = resolve_dependencies(packages)
File "/home/jason/jetson-containers/jetson_containers/packages.py", line 316, in resolve_dependencies
packages, changed = add_depends(packages)
File "/home/jason/jetson-containers/jetson_containers/packages.py", line 288, in add_depends
for dependency in find_package(package).get('depends', []):
File "/home/jason/jetson-containers/jetson_containers/packages.py", line 174, in find_package
raise KeyError(f"couldn't find package: {package}")
KeyError: "couldn't find package: text-generation-inference"
-- Error: return code 1

• docker run --runtime nvidia -it --rm --network host --volume /tmp/argus_socket:/tmp/argus_socket --volume /etc/enctune.conf:/etc/enctune.conf --volume /etc/nv_tegra_release:/etc/nv_tegra_release --volume /tmp/nv_jetson_model:/tmp/nv_jetson_model --volume /var/run/dbus:/var/run/dbus --volume /var/run/avahi-daemon/socket:/var/run/avahi-daemon/socket --volume /var/run/docker.sock:/var/run/docker.sock --volume /home/jason/jetson-containers/data:/data --device /dev/snd --device /dev/bus/usb
  "docker run" requires at least 1 argument.
  See 'docker run --help'.

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Create and run a new container from an image
-bash: text-generation-launcher: command not found