Codeium AI autocomplete integration #343

1egoman · 2024-10-09T19:19:38Z

Introduces codeium ai autocomplete into the srcbook editor.

A few high level notes:

There is no way to enable / disable this feature currently.
The other cells are used as context when performing code generation, leading to more contextually relevant suggestions.
By default, the EDITOR_API_KEY constant token is used. If a user authenticates with codeium, a token linked to their user account is used instead.

1egoman · 2024-10-10T18:57:33Z

packages/web/src/lib/ai-autocomplete/index.ts

+import protobuf from 'protobufjs';
+import Long from 'long';
+
+import languageServerProto from './language-server-proto';
+
+// NOTE: this EDITOR_API_KEY value was just included as a raw string in
+// @codeium/react-code-editor. This seems to not be a secret? See here:
+// https://github.com/Exafunction/codeium-react-code-editor/blob/768e1b231c00e078c86bc19c8ede697a1e37ec75/src/components/CodeiumEditor/CompletionProvider.ts#L48
+const EDITOR_API_KEY = 'd49954eb-cfba-4992-980f-d8fb37f0e942';
+
+// NOTE: The below logic has been adapted from codeium's `@codeium/react-code-editor package. See here:
+// https://github.com/Exafunction/codeium-react-code-editor/blob/768e1b231c00e078c86bc19c8ede697a1e37ec75/src/components/CodeiumEditor/CompletionProvider.ts#L147-L159
+export async function runCodiumAiAutocomplete(
+  optionalApiKey: string | null,
+  source: string,
+  sourceLanguage: 'javascript' | 'typescript',
+  cursorOffset: number,
+): Promise<CodiumCompletionResult> {
+  const protos = protobuf.Root.fromJSON(languageServerProto as protobuf.INamespace);
+  const GetCompletionsRequest = protos.lookupType('exa.language_server_pb.GetCompletionsRequest');
+  const Metadata = protos.lookupType('exa.codeium_common_pb.Metadata');
+  const DocumentInfo = protos.lookupType('exa.language_server_pb.Document');
+  const EditorOptions = protos.lookupType('exa.codeium_common_pb.EditorOptions');
+  const Language = protos.lookupEnum('exa.codeium_common_pb.Language');
+  const GetCompletionsResponse = protos.lookupType('exa.language_server_pb.GetCompletionsResponse');
+
+  const sessionId = `react-editor-${crypto.randomUUID()}`;
+  const apiKey = optionalApiKey ?? EDITOR_API_KEY;
+
+  const payload = {
+    otherDocuments: [],
+    metadata: Metadata.create({
+      ideName: 'web',
+      extensionVersion: '1.0.12',
+      apiKey,
+      ideVersion: 'unknown',
+      extensionName: '@codeium/react-code-editor',
+      sessionId,
+    }),
+    document: DocumentInfo.create({
+      text: source,
+      editorLanguage: sourceLanguage,
+      language: Language.getOption(sourceLanguage === 'javascript' ? 'JAVASCRIPT' : 'TYPESCRIPT'),
+      cursorOffset: Long.fromValue(cursorOffset),
+      lineEnding: '\n',
+    }),
+    editorOptions: EditorOptions.create({
+      tabSize: Long.fromValue(4),
+      insertSpaces: true,
+    }),
+  };
+
+  const requestData = GetCompletionsRequest.create(payload);
+  const buffer = GetCompletionsRequest.encode(requestData).finish();
+
+  const response = await fetch(
+    'https://web-backend.codeium.com/exa.language_server_pb.LanguageServerService/GetCompletions',
+    {
+      method: 'POST',
+      body: buffer,
+      headers: {
+        'Connect-Protocol-Version': '1',
+        'Content-Type': 'application/proto',
+        Authorization: `Basic ${apiKey}-${sessionId}`,
+      },
+    },
+  );
+
+  const responseBodyBytes = new Uint8Array(await response.arrayBuffer());
+  const responseBody = GetCompletionsResponse.decode(responseBodyBytes);
+
+  return responseBody.toJSON() as CodiumCompletionResult;
+}


Here's what the logic to kick off the codeium autocomplete request looks like. While complex, the protobuf serialization / deserialization logic is unfortunately required the query codeium.

1egoman · 2024-10-10T19:06:32Z

packages/web/src/lib/ai-autocomplete/language_server.proto

+// NOTE: to generate a new set of json definitions, run:
+// npm run generate-codeium-proto-json
+
+// Copyright Exafunction, Inc.
+
+syntax = "proto3";
+
+package exa.language_server_pb;
+
+import "codeium_common.proto";


Here's the underlying protobuf definitions that are used to generate the codeium autocomplete request. These came from here: https://github.com/Exafunction/codeium-react-code-editor/blob/main/exa/language_server_pb/language_server.proto.

Note that to avoid including this raw text in the bundle that is sent to the client, I converted the protobuf file to json descriptiors (more about these here). These are a fair bit smaller and work just as well.

Another option I considered but ultimately decided against given the complexity of introducing was an alternate protobuf library that does code generation at build time, like buf. If protobufs were going to be used everywhere and definitions were going to be changing often, I think it would be a lot of sense, but given this is only used here that seemed like overkill.

@1egoman are we okay with the copyright on this?

1egoman · 2024-10-10T19:09:37Z

packages/web/src/routes/settings.tsx

+
+          <h3 className="text-md pb-2">Codeium AI Autocomplete</h3>
+          <div className="flex flex-col">
+            <div className="opacity-70 text-sm pb-1">
+              By default, Codeium uses a public api token with limited capabilities. Optionally,
+              sign in to remove rate limits:
+            </div>
+
+            {codeiumApiKey ? (
+              <div
+                className="flex flex-col p-3 border rounded-sm"
+                onMouseEnter={() => setCodeiumApiKeyHovering(true)}
+                onMouseLeave={() => setCodeiumApiKeyHovering(false)}
+              >
+                <div className="opacity-70 text-sm pb-2">Signed in! Codeium API Key:</div>
+                <div className="flex justify-between items-center gap-2">
+                  <div className="text-left align-middle relative w-full">
+                    <Input
+                      name="codeiumApiKey"
+                      type={codeiumApiKeyVisible ? 'text' : 'password'}
+                      value={codeiumApiKey}
+                      disabled
+                      className="disabled:opacity-100 disabled:cursor-text group-hover:border-border group-focus-within:border-border pr-8"
+                    />
+                    {codeiumApiKeyVisible ? (
+                      <EyeOffIcon
+                        size={14}
+                        className={cn(
+                          'absolute right-3 top-2.5 cursor-pointer opacity-80 bg-background',
+                          !codeiumApiKeyHovering && 'hidden',
+                        )}
+                        onClick={() => setCodeiumApiKeyVisible(false)}
+                      />
+                    ) : (
+                      <EyeIcon
+                        size={14}
+                        className={cn(
+                          'absolute right-3 top-2.5 cursor-pointer opacity-80 bg-background',
+                          !codeiumApiKeyHovering && 'hidden',
+                        )}
+                        onClick={() => setCodeiumApiKeyVisible(true)}
+                      />
+                    )}
+                  </div>
+
+                  <Button
+                    variant="secondary"
+                    onClick={() => {
+                      updateConfigContext({ codeiumApiKey: null })
+                        .then(() => {
+                          toast.success('Detached Codeium API key.');
+                        })
+                        .catch((err) => {
+                          console.error('Error detaching Codeium API key:', err);
+                          toast.error('Error detaching Codeium API key!');
+                        });
+                    }}
+                  >
+                    Detach
+                  </Button>
+                </div>
+              </div>
+            ) : (
+              <div className="flex justify-center items-center p-3 h-[64px] border rounded-sm">
+                <Button asChild variant="secondary">
+                  <Link
+                    to={`https://www.codeium.com/profile?response_type=token&redirect_uri=${codeiumCallbackUrl}&state=a&scope=openid%20profile%20email&redirect_parameters_type=query`}
+                  >
+                    Authenticate with Codeium
+                  </Link>
+                </Button>
+              </div>
+            )}
+          </div>


I added a new interface element to the settings page to facilitate a user authenticating with codeium:

Note that even if a user does not sign in, the autocomplete logic still works, because it falls back to that EDITOR_API_KEY value.

Nit: Could add a link to their sign in page

1egoman · 2024-10-10T19:12:59Z

packages/web/package.json

@@ -10,23 +10,28 @@
    "lint": "eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0",
    "format": "prettier --write .",
    "preview": "vite preview",
-    "check-types": "tsc"
+    "check-types": "tsc",
+    "generate-codeium-proto-json": "node --eval 'console.log(\"export default\", JSON.stringify(require(\"protobufjs\").loadSync(\"src/lib/ai-autocomplete/language_server.proto\").toJSON(), null, 2));' > src/lib/ai-autocomplete/languageServerProto.ts"


I added this npm task to take the language_server.proto file and convert it into the json descriptors data that is required by the client.

Note that currently this generated json descriptors file is committed. I could potentially set this up to generate fresh when starting up the dev server / building for release, but as this in practice will rarely / more likely never change, it seemed like setting up more complex build infrastructure here would be less useful than it otherwise would be.

…requests

…ursor position

As part of this, I had to build the protobuf file into a set of "json definitions" that I could more easily include in a javascript bundle vs the raw bytes of a proto file.

…e right language

benjreinhart

Nit: I would prefer the if the protos we're in their own folder named something like "generated"

benjreinhart · 2024-10-14T20:07:21Z

packages/web/src/lib/ai-autocomplete/codeium_common.proto

@@ -0,0 +1,166 @@
+// Copyright Exafunction, Inc.


@1egoman do you know if we're able to pull this in given the copyright?

https://github.com/Exafunction/codeium-react-code-editor/blob/main/LICENSE MIT license for the source

1egoman commented Oct 10, 2024

View reviewed changes

1egoman marked this pull request as ready for review October 10, 2024 19:51

1egoman added 17 commits October 10, 2024 15:52

feat: add codeium_api_key column to config table

abaebbf

feat: add MVP of codeium oauth process

aa68685

feat: add hacked together demo for codium ai autocomplete

0a191be

feat: use the user provided codeium api key if available when making …

8b9eb03

…requests

fix: address issue in cell autocomplete due to an off by one in the c…

2f43f50

…ursor position

feat: relocate codeium api request from server to client

1a8a643

As part of this, I had to build the protobuf file into a set of "json definitions" that I could more easily include in a javascript bundle vs the raw bytes of a proto file.

feat: add subcommmand to rebuild codeium proto json definitions

4f8b323

fix: run npm run format

c6d5bb1

refactor: remove dead code

317398f

feat: ensure that codeium ai autocomplete generates suggestions in th…

4c7d066

…e right language

feat: slightly clean up the codeium oauth interface on the settings page

899b136

docs: add comments on codeium logic

4c9c265

refactor: consolodate codeium types with the codeium request code

49890f1

refactor: reorder imports

2ed663a

feat: make the codeium-callback page look a little better

5afa2b3

fix: address typo in function name

5656ec5

feat: include other cells as context with ai autocomplete

d529456

1egoman force-pushed the codeium-integration branch from b3cd5d8 to d529456 Compare October 10, 2024 19:52

1egoman changed the title ~~Codeium AI authcomplete integration~~ Codeium AI autocomplete integration Oct 11, 2024

Merge branch 'main' into codeium-integration

11cca95

benjreinhart approved these changes Oct 14, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Codeium AI autocomplete integration #343

Codeium AI autocomplete integration #343

1egoman commented Oct 9, 2024 •

edited

Loading

1egoman Oct 10, 2024 •

edited

Loading

1egoman Oct 10, 2024

versecafe Oct 10, 2024

1egoman Oct 10, 2024 •

edited

Loading

nichochar Oct 15, 2024

1egoman Oct 10, 2024 •

edited

Loading

benjreinhart left a comment

benjreinhart Oct 14, 2024

versecafe Oct 14, 2024

Codeium AI autocomplete integration #343

Are you sure you want to change the base?

Codeium AI autocomplete integration #343

Conversation

1egoman commented Oct 9, 2024 • edited Loading

1egoman Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

1egoman Oct 10, 2024

Choose a reason for hiding this comment

versecafe Oct 10, 2024

Choose a reason for hiding this comment

1egoman Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

nichochar Oct 15, 2024

Choose a reason for hiding this comment

1egoman Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

benjreinhart left a comment

Choose a reason for hiding this comment

benjreinhart Oct 14, 2024

Choose a reason for hiding this comment

versecafe Oct 14, 2024

Choose a reason for hiding this comment

1egoman commented Oct 9, 2024 •

edited

Loading

1egoman Oct 10, 2024 •

edited

Loading

1egoman Oct 10, 2024 •

edited

Loading

1egoman Oct 10, 2024 •

edited

Loading