Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--main_gpu does not select specified gpu - Intel Arc GPUs #9

Open
Chadwicksracing opened this issue Oct 7, 2023 · 1 comment
Open
Labels
bounty Will pay for someone to get this done

Comments

@Chadwicksracing
Copy link

C:\Users\rtm\Documents>gputopia-worker-opencl-win-64.exe --ln_url [email protected] --force_layers 33 --test_model TheBloke/CodeLlama-7B-Instruct-GGUF:Q4_K_M --debug --main_gpu 0 --tensor_split 0,1,0,1,0,1,0,1,0,1
2023-10-07 10:52:12,482 - ai_worker.main - DEBUG - no nvidia: NVML Shared Library Not Found
{'auth_key': '',
'cl_driver_version': 'OpenCL 3.0 ',
'cl_gpus': [{'clock': 2400,
'clock_unit': 'mhz',
'memory': 8319.0,
'name': 'Intel(R) Arc(TM) A750 Graphics',
'uuid': None},
{'clock': 2400,
'clock_unit': 'mhz',
'memory': 8319.0,
'name': 'Intel(R) Arc(TM) A750 Graphics',
'uuid': None},
{'clock': 2400,
'clock_unit': 'mhz',
'memory': 8319.0,
'name': 'Intel(R) Arc(TM) A750 Graphics',
'uuid': None},
{'clock': 2400,
'clock_unit': 'mhz',
'memory': 8319.0,
'name': 'Intel(R) Arc(TM) A750 Graphics',
'uuid': None},
{'clock': 2400,
'clock_unit': 'mhz',
'memory': 8319.0,
'name': 'Intel(R) Arc(TM) A750 Graphics',
'uuid': None}],
'cpu_count': 16,
'disk_space': 428351,
'ln_address': '[email protected]',
'ln_url': '[email protected]',
'nv_driver_version': None,
'nv_gpu_count': None,
'nv_gpus': [],
'vram': 5996646400,
'web_gpus': [],
'worker_id': '',
'worker_version': '0.1.9'}
2023-10-07 10:52:13,051 - ai_worker.main - DEBUG - loading model: TheBloke/CodeLlama-7B-Instruct-GGUF:Q4_K_M
ggml_opencl: selecting platform: 'Intel(R) OpenCL Graphics'
ggml_opencl: selecting device: 'Intel(R) Arc(TM) A750 Graphics'
ggml_opencl: device FP16 support: true
Traceback (most recent call last):
File "ai_worker_main_.py", line 4, in
File "ai_worker\main.py", line 385, in main
File "asyncio\runners.py", line 190, in run
File "asyncio\runners.py", line 118, in run
File "asyncio\base_events.py", line 653, in run_until_complete
File "ai_worker\main.py", line 144, in run
File "ai_worker\main.py", line 114, in test_model
File "ai_worker\main.py", line 201, in load_model
File "llama_cpp\server\app.py", line 342, in create_app
File "llama_cpp\llama.py", line 312, in init
IndexError: invalid index
[12344] Failed to execute script 'main' due to unhandled exception!

image

--tensor_split not working either

@earonesty
Copy link
Contributor

agreed, needs to be looked into.

@earonesty earonesty added the bounty Will pay for someone to get this done label Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bounty Will pay for someone to get this done
Projects
None yet
Development

No branches or pull requests

2 participants