📖 Add Ollama to VSCode Offering

[Gary-H9]

User Story

As a… AP product engineer
I need/want/expect to… offer LLMs to our users
To satisfy user needs

Value / Purpose

No response

Useful Contacts

No response

User Types

No response

Hypothesis

If we... add ollama
We will provide functionality our users desire

Proposal

We are about to launch a vscode tool as part of the AP offering, a long-requested feature. As proposed by @Gary-H9, we could set up a restricted release with ollama or similar tool rolled into the vscode environment. That said, as this is a sandbox, it would be highly undesirable to allow access to actual AP data. Therefore, the proposal is as following:

• A restricted vscode release that deploys either in the AP or the DPAT EKS cluster with
• Custom role that allows access to only one bucket
• All users of the restricted release have access to the same bucket and everyone is able to see its contents (to prevent people loading up restricted data in it..
• With minimal support

Additional Information

No response

Definition of Done

• Add ollama to the visual studio code release
• Test locally

TBC:

• Add ollama to a vscode release
• Release in development
• Test functionality

[AntFMoJ]

Container for VS Code running locally as expected.
Added installation of Ollama to Dockerfile in analytical-platform-visual-studio-code.
Tested ollama serve and ollama run llama2, working correctly.

Next steps:

• Generate SHA256 checksum for ollama binary and add check to Dockerfile in order. There isn't one available from Ollama currently, Jacob W has requested this from them.
• Add link and description of ollama to analytical-platform-visual-studio-code documentation, along with command for getting SHA256 for ollama binary.
• Add test for ollama version to container-structure-test.yml
• Update AP docs to include link to Ollama website/Github repo, and include instructions for ollama serve and ollama run <model_name>.
• Raise PRs.

[BrianEllwood]

User documentation updated and PR created

[AntFMoJ]

Steps above completed, ticket it done