feat: add assistant experiment #4

mc-jones · 2024-04-16T13:54:31Z

This PR adds an experiment for OpenAi's "assistant" feature. I just dropped in the function calls from #2 as it seems preferable to not mix our experiment code, even where we repeat ourselves.

Some general observations and notes about assistants surfaced through this experiment will be 🔒 here.

Considerations

The output is a expectedly poor here as this experiment is not trying build the best response possible, but rather spin up an assistant that is making reasonable judgements about which of the tools to utilize and which args to use.

Examples

# EXAMPLE 1
yarn tsx src/02-assistant/index.ts "I want to purchase art with a budget of 1,000,000. I like abstract art." 
yarn run v1.22.21
$ /Users/mattjones/code/quantum/node_modules/.bin/tsx src/02-assistant/index.ts 'I want to purchase art with a budget of 1,000,000. I like abstract art.'
Calling function: get_artists with args: {"size":5,"sort":"TRENDING_DESC"} # <---Function call
user > I want to purchase art with a budget of 1,000,000. I like abstract art.
assistant > I have found some trending abstract artists on Artsy that you might be interested in:

1. [Sarah Drinan](https://www.artsy.net/artist/sarah-drinan) - Abstract artist born in 1994 with 7 for sale artworks.
2. [Krishna Shamrao Kulkarni](https://www.artsy.net/artist/krishna-shamrao-kulkarni) - Indian abstract artist (1916–1994) with 12 for sale artworks.
3. [Joey Leung Ka-yin](https://www.artsy.net/artist/joey-leung-ka-yin) - Hong Kong abstract artist with 5 for sale artworks.
4. [Larry Muñoz](https://www.artsy.net/artist/larry-munoz) - Abstract artist born in 1982 with 22 for sale artworks.
5. [Simone Prouvé](https://www.artsy.net/artist/simone-prouve) - French abstract artist born in 1931 with 4 for sale artworks.

You can explore their artworks on their profile pages.
✨  Done in 13.40s.

# EXAMPLE 2
$ /Users/mattjones/code/quantum/node_modules/.bin/tsx src/02-assistant/index.ts 'I want to purchase art with a budget of 1,000,000. I don\'\''t want anything created in the last 30 years.'
Calling function: get_artists with args: {"size":5,"sort":"CREATED_AT_ASC"}  # <---Function call
user > I want to purchase art with a budget of 1,000,000. I don\'t want anything created in the last 30 years.
assistant > I have found some artists whose works may fit your criteria. You can explore their profiles to see if any artworks interest you:

1. **Emna Zghal**
   - Nationality: Tunisian
   - [View Emna Zghal's Profile](https://www.artsy.net/artist/emna-zghal)

2. **John Bradford**
   - Nationality: American
   - Birthday: 1949
   - For Sale Artworks: 4
   - [View John Bradford's Profile](https://www.artsy.net/artist/john-bradford)

3. **Karen Whitman**
   - Nationality: American
   - Birthday: 1953
   - For Sale Artworks: 4
   - [View Karen Whitman's Profile](https://www.artsy.net/artist/karen-whitman)

4. **Layla Fanucci**
   - Birthday: 1957
   - For Sale Artworks: 46
   - [View Layla Fanucci's Profile](https://www.artsy.net/artist/layla-fanucci)

5. **Richard Pantell**
   - Birthday: 1951
   - For Sale Artworks: 1
   - [View Richard Pantell's Profile](https://www.artsy.net/artist/richard-pantell)

Feel free to browse their artworks and see if any align with your preferences and budget. Let me know if you need more information or assistance!

anandaroop

Awesome.

as it seems wise to not mix our experiment code, even where we repeat ourselves

👍🏽 makes sense to go with this approach until/unless we explicitly want to build up any shared library code.

anandaroop · 2024-04-16T14:26:42Z

.env.example

@@ -1 +1,4 @@
 OPENAI_API_KEY=REPLACE
+
+# For Experiment 02-assistant
+ASSISTANT_ID=REPLACE


question (non-blocking): This is more of an identifier than a secret, yeah? Should we consider hard-coding a default here, or does it make more sense to leave this blank to allow devs to iterate in parallel without conflicts?

This is more of an identifier than a secret, yeah?

Exactly. Just config. I wasn't sure if there is any way it could be abused, and didn't want to check into the repo without being more confident (I would imagine its likely fine and doesn't work without our API key).

or does it make more sense leave this blank to allow devs to iterate in parallel without conflicts?

Yeah, this was also part of the reasoning.

Happy to remove if we would like to keep .env purely for secrets. Not strongly opinionated here.

Makes sense this way

anandaroop · 2024-04-16T14:44:51Z

src/02-assistant/index.ts

+  let run = await openai.beta.threads.runs.createAndPoll(thread.id, {
+    assistant_id: assistant.id,
+    instructions:
+      "Respond like you work at artsy.net. Always provide a list of artists and include the link to their profile. Always check artsy before making a recommendation.",


question (non-blocking): Does the "Always check artsy" bit force any kind of live web-browsing behavior? Does the assistant have that capability? (Wasn't clear to me that it did, from the create statement above.)

This is more an artifact of my testing different instructions and trying to see if I can force it to use one of the tools. There was no noticeable difference in the small number of test prompts, however, and probably can be removed.

anandaroop · 2024-04-16T20:12:59Z

src/02-assistant/index.ts

+
+  if (run.status === "completed") {
+    const messages = await openai.beta.threads.messages.list(run.thread_id)
+    for (const message of messages.data.reverse()) {


question (non-blocking): Why reverse the messages here?

This block came from their docs, so not 100% but would guess is that by default the messages come in reverse chronological order, which would be confusing when printed out.

anandaroop · 2024-04-16T20:15:03Z

src/02-assistant/index.ts

+        .arguments || "null"
+    )
+
+    console.log(`Calling function: ${name} with args: ${JSON.stringify(args)}`)


Good idea 👍🏽

mc-jones added 3 commits April 15, 2024 17:33

feat: add assistant experiment

bd0f5d2

feat: take user input and optionallyf use an existing assistant

860e6c3

chore: add empty last line

84c900c

mc-jones requested review from anandaroop and sweir27 April 16, 2024 13:54

mc-jones assigned anandaroop Apr 16, 2024

anandaroop approved these changes Apr 16, 2024

View reviewed changes

anandaroop merged commit d99a42f into main Apr 16, 2024
1 check passed

mc-jones deleted the mc-jones/test-assistant branch April 25, 2024 17:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add assistant experiment #4

feat: add assistant experiment #4

mc-jones commented Apr 16, 2024 •

edited

Loading

anandaroop left a comment

anandaroop Apr 16, 2024

mc-jones Apr 16, 2024

anandaroop Apr 16, 2024

anandaroop Apr 16, 2024

mc-jones Apr 16, 2024

anandaroop Apr 16, 2024

mc-jones Apr 16, 2024 •

edited

Loading

anandaroop Apr 16, 2024

feat: add assistant experiment #4

feat: add assistant experiment #4

Conversation

mc-jones commented Apr 16, 2024 • edited Loading

Considerations

Examples

anandaroop left a comment

Choose a reason for hiding this comment

anandaroop Apr 16, 2024

Choose a reason for hiding this comment

mc-jones Apr 16, 2024

Choose a reason for hiding this comment

anandaroop Apr 16, 2024

Choose a reason for hiding this comment

anandaroop Apr 16, 2024

Choose a reason for hiding this comment

mc-jones Apr 16, 2024

Choose a reason for hiding this comment

anandaroop Apr 16, 2024

Choose a reason for hiding this comment

mc-jones Apr 16, 2024 • edited Loading

Choose a reason for hiding this comment

anandaroop Apr 16, 2024

Choose a reason for hiding this comment

mc-jones commented Apr 16, 2024 •

edited

Loading

mc-jones Apr 16, 2024 •

edited

Loading