feat (provider/google-vertex): Add support for edge runtime. #3908

shaper · 2024-11-27T00:00:31Z

No description provided.

socket-security · 2024-11-27T00:01:34Z

New and removed dependencies detected. Learn more about Socket for GitHub ↗︎

Package	New capabilities	Transitives	Size	Publisher
npm/[email protected]	Transitive: environment, filesystem, network	`+20`	1.94 MB	google-wombot

🚮 Removed packages: npm/@google-cloud/[email protected]

View full report↗︎

packages/google-vertex/src/google-vertex-auth-edge.ts

packages/google-vertex/src/google-vertex-auth-google-auth-library.ts

packages/google/src/google-generative-ai-embedding-model.ts

packages/google/src/google-generative-ai-language-model.test.ts

lgrammel · 2024-11-27T08:39:23Z

packages/google/src/google-generative-ai-settings.ts

@@ -40,6 +40,7 @@ Optional. A list of unique safety settings for blocking unsafe content.
   */
  safetySettings?: Array<{
    category:
+      | 'HARM_CATEGORY_UNSPECIFIED'


are these harm categories the same in vertex & genai? i vaguely remember differences.

The only difference I see is for the older PaLM models. Here is my understanding:

GenAI Harm Categories
Vertex Harm Categories

The PaLM entries in the GenAI link are missing:

HARM_CATEGORY_DEROGATORY | PaLM - Negative or harmful comments targeting identity and/or protected attribute. HARM_CATEGORY_TOXICITY | PaLM - Content that is rude, disrespectful, or profane. HARM_CATEGORY_VIOLENCE | PaLM - Describes scenarios depicting violence against an individual or group, or general descriptions of gore. HARM_CATEGORY_SEXUAL | PaLM - Contains references to sexual acts or other lewd content. HARM_CATEGORY_MEDICAL | PaLM - Promotes unchecked medical advice. HARM_CATEGORY_DANGEROUS | PaLM - Dangerous content that promotes, facilitates, or encourages harmful acts.

I assume these have been updated over time. Based on the current specs linked above, my take is that this PR is newly updated to reflect the latest, and the only consideration is whether to add the PaLM entries:

we were missing the HARM_CATEGORY_UNSPECIFIED in GenAI but they are relevant there

we were missing HARM_CATEGORY_CIVIC_INTEGRITY in both GenAI and Vertex, and it's relevant in both

we were missing the PaLM entries above in both. GenAI spec has the PaLM entries and Vertex doesn't

https://ai.google.dev/palm_docs notes "Warning: The PaLM API is decomissioned. The Vertex AI PaLM API is scheduled to be decomissioned in October 2024. Please upgrade to the Gemini API. Learn more about this in the PaLM API deprecation guide."

It is not clear to me that the PaLM models are a forward-going concern, so I would personally omit them for now.

packages/google/src/google-generative-ai-settings.ts

lgrammel · 2024-11-27T08:40:49Z

Needs docs update (showing how to use different auth solutions) and changeset (works on edge now).

packages/google-vertex/package.json

- vertex: peer deps for google, google-auth-library - vertex: don't export experimental auth token fns - vertex: no need for "module" export for './internal' - vertex: revise edge-compatible auth (buffer, file creds) - vertex: provider incorporates user option for generateAuthToken - vertex: update export symbol for edge-compatible auth token generation fn - google: 'content' in response schema optional for safety failure parsing

This comment was marked as outdated.

Sign in to view

shaper force-pushed the shaper/pr/vertex-edge-redux branch 2 times, most recently from 6715320 to 4b9ee6f Compare November 27, 2024 00:08

vercel bot deployed to Preview November 27, 2024 00:10 View deployment

shaper commented Nov 27, 2024

View reviewed changes

packages/google-vertex/src/google-vertex-auth-edge.ts Outdated Show resolved Hide resolved

shaper requested a review from lgrammel November 27, 2024 00:14

shaper marked this pull request as ready for review November 27, 2024 00:14