Add vision support

[ErikBjare]

Since the OpenAI API now has vision in beta, and we could use LLaVa locally.

Might be a lot of work, or might be super easy.

Question is, what would it be useful for?

• Add screenshot tool, integrate with vision #51: Xvfb to understand display/output and make a E2E desktop agent
• Add browser screenshot tool, integrate with vision #52: Screenshot with browser tool
  • Can be used to take screenshots of developed webapps for visually-aided autodebugging
• Have it review plot outputs for correctness and to inspect results
  • Could be useful for data science, but reading a good plain text output might still be superior

[ErikBjare]

Now works after adding the vision tool: 597c66c

Also merged the screenshot tool: #92

[ErikBjare]

Anthropic just announced their computer use stuff: https://www.anthropic.com/news/3-5-models-and-computer-use

Basically same thing lol, although I should implement clicks and input (with pynput?), and let it run in Docker at 1280x800 resolution.

[ErikBjare]

I saw that @simonw had implemented support for not only images, but also video and audio in https://github.com/simonw/llm/releases/tag/0.17: https://x.com/simonw/status/1851280825662521370

That would be a good continuation on this issue