Interact with Vertex Tuned Models

[LuizOVianna]

I did not find a way to interact with tuned models prediction endpoints. Is there a way?

[jochenkirstaetter]

Hello @LuizOVianna

Right now, I have to look in to the URI endpoint patterns.
The current endpoints for Google AI as well as Vertex AI are fixed.

This needs to be more flexible for Vertex AI in particular.

Thanks for pointing this out.

Cheers, JoKi

[jochenkirstaetter]

Looking quickly at the sources, it shows that the URL of Vertex AI is templated and shall be ready for other endpoints.
See here

generative-ai/src/Mscc.GenerativeAI/GenerativeModel.cs

Lines 27 to 31 in 4b0e7fe

	private const string UrlVertexAi = "https://{region}-aiplatform.googleapis.com/{version}/projects/{projectId}/locations/{region}/publishers/{publisher}/{model}:{method}";
	private readonly bool _useVertexAi;
	private readonly string _region = "us-central1";
	private readonly string _publisher = "google";

for reference.

However, there's currently no publicly exposed method to specify the publisher part.

As said, going to look into that. @LuizOVianna , I would appreciate that you could provide an example or two for Vertex AI endpoints, if you don't mind. Of course, masking sensitive information.

Thanks JoKi

[LuizOVianna]

Hi,

I could make it work using HttpClient. The Endpoint indicated by Google is a little diferent, in this following format:

baseUrl = $"https://{location}-aiplatform.googleapis.com"; 
endpointurl = $"{baseUrl}/v1/projects/{projectId}/locations/{location}/endpoints/{endpointId}:generateContent";

It needs the projectId, location/region, indpointId(not name)

See here original Google instruction:
https://cloud.google.com/vertex-ai/generative-ai/docs/models/gemini-use-supervised-tuning

Hope all these helps! ;)

[jochenkirstaetter]

Hello @LuizOVianna

That's absolutely great, thanks!
I'm currently updating the code base according to the latest changes of the API, and then I'm going to look into this.

Please bear with me, changes are on the way.

Cheers, JoKi

[LuizOVianna]

Nice,

Let me know when you got there...

Regards from Brazil ;)

[jochenkirstaetter]

@LuizOVianna

Quick info that I started working on this.
Looking forward to release quite soon.

Regards, JoKi

[jochenkirstaetter]

Hello @LuizOVianna

Kindly check whether Release v1.9.0 resolves this issue. There had been quite a bit of refactoring to incorporate tuning jobs and Vertex AI endpoints of tuned models.

Hope this helps, JoKi

[LuizOVianna]

Great!!

I´ll try as soon I get into this, and let You know... Is there any documentation of it?

[jochenkirstaetter]

Hello @LuizOVianna,

Ah, yes, documentation. Gotta have to write that (still), thanks for the reminder. ;-)

For now, using the VertexAI class, you would first specify the endpoint reference.
Then, like previously, pass in the model name of the tuned model to get an instance of the GenerativeModel type.

using Mscc.GenerativeAI;

var vertex = new VertexAI(projectId: projectId, region: region);
vertex.EndpointId = "<specify TunedModel.Endpoint>"    // eg. endpoints/1234
var model = vertex.GenerativeModel(model: "<specify TunedModel.Name>");

var response = await model.GenerateContent(prompt);
Console.WriteLine(response.Text);

There is also a new overload to pass in a TuningJob as a parameter.

using Mscc.GenerativeAI;

var vertex = new VertexAI(projectId: projectId, region: region);
var sft = vertex.SupervisedTuningJob();
var tuningJob = await sft.List().FirstOrDefault();     // or use sft.Get("<specify name of tuning job>");
var model = vertex.GenerativeModel(tuningJob);

var response = await model.GenerateContent(prompt);
Console.WriteLine(response.Text);

Which means that the package now also provides functionality to create, list, get and cancel supervised fine-tuning in Vertex AI.

Hope this helps, JoKi