06_state_of_the_art.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPaJeuoA53fr5IJaY07MljT",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/badlogic/genai-workshop/blob/main/06_state_of_the_art.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## State of the Art Demo: GPT-4\n",
        "<center><img src=\"https://1000logos.net/wp-content/uploads/2023/02/ChatGPT-Logo.png\" width=480/></center>\n",
        "<center><img src=\"https://9to5google.com/wp-content/uploads/sites/4/2023/12/google-gemini-cover.jpg?quality=82&strip=all&w=1600\" width=480/></center>\n",
        "\n",
        "> **Note:** At the time of writting, Gemini is currently not available to users in Europe.\n",
        "\n",
        "OpenAI's [GPT-4](https://openai.com/research/gpt-4) model and Google's [Gemini](https://deepmind.google/technologies/gemini/#gemini-1.5) represent the current state of the art in large language models.\n",
        "\n",
        "GPT-4 is accessible both through a web UI as part of [ChatGPT](https://chat.openai.com/), and via an [API](https://platform.openai.com/docs/introduction). Let's explore the state of the art using GPT-4.\n",
        "\n",
        "The following demos require access to GPT-4 via a subscription [ChatGPT Pro](https://openai.com/chatgpt/pricing) or better plan.\n",
        "\n"
      ],
      "metadata": {
        "id": "Uk1qlIA12DlH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### GPT-4 Demo\n",
        "*Heavily inspired by [Andrej Karparthy](https://www.youtube.com/c/AndrejKarpathy), founding member of OpenAI, and Sr. Directory of AI at Tesla*\n",
        "\n",
        "GPT-4 can not only answer question and hold a conversation, but also use tools:\n",
        "\n",
        "* **Web Browsing**: searches the Web for more information via Bing, and uses that information plus its intrinsic knowledge to answer a question.\n",
        "* **Code interpreter**: writes code and then invokes a Python code interpreter to answer a question. Note that this interpreter can also generate plots!\n",
        "* **[DALL·E 2](https://openai.com/dall-e-2)**: invokes DALL·E 2 to generate images based on a textual prompt.\n",
        "\n",
        "Try these prompts in [ChatGPT](https://chat.openai.com) using GPT-4.\n",
        "\n",
        "```\n",
        "Collect information about Scale AI and its funding rounds. When they happened (date), the amount, and the valuation. Organize this into a table.\n",
        "```\n",
        "\n",
        "```\n",
        "Show this data as a table.\n",
        "```\n",
        "\n",
        "```\n",
        "Let's try to estimate the valuation for Series A and B based on the ratio raised/valuation for Series D and E.\n",
        "```\n",
        "\n",
        "```\n",
        "Please also esimate the valuation for Series C.\n",
        "```\n",
        "\n",
        "```\n",
        "Please create a professional plot for the valuation data. The y-axis is the valuation of Scale AI. Use a logarithmic scale. The x-axis is the Date. Make it a nice looking plot and use grid lines.\n",
        "```\n",
        "\n",
        "```\n",
        "Plot the same data with a linearly scaled y-axis.\n",
        "```\n",
        "\n",
        "```\n",
        "For the log-plot, please add a trend line to extrapolate the valuation until the end of 2024. Draw a vertical green line for today.\n",
        "```\n",
        "\n",
        "```\n",
        "Finally, please generate an image that represents Scale AI in the year 2027\n",
        "```\n",
        "\n",
        "<center><img src=\"https://marioslab.io/uploads/genai/karpathy-1.png\" width=\"480\" /></center>\n",
        "<center><img src=\"https://marioslab.io/uploads/genai/karpathy-4.png\" width=\"480\" /></center>\n",
        "<center><img src=\"https://marioslab.io/uploads/genai/karpathy-2.png\" width=\"480\" /></center>\n",
        "<center><img src=\"https://marioslab.io/uploads/genai/karpathy-3.png\" width=\"480\" /></center>"
      ],
      "metadata": {
        "id": "ocp3Q9vUvpwx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Custom GPTs Demo\n",
        "![https://marioslab.io/uploads/genai/custom-gpt.png](https://marioslab.io/uploads/genai/custom-gpt.png)\n",
        "OpenAI has also introduced the capability to create [custom GPTs](https://openai.com/blog/introducing-gpts), tailored for a specific use case.\n",
        "\n",
        "[Spiney](https://chat.openai.com/g/g-a1sbldV2T-spiney) is a chatbot that can answer questions with citations based on the [Spine User Guide](https://esotericsoftware.com/spine-user-guide).\n",
        "\n",
        "Spiney was created using the GPT Builder interface. The GPT Builder interface asks questions to iteratively understand what the custom GPT is supposed to do. I.e. during that process, I provided the following guidance.\n",
        "\n",
        "```\n",
        "Read the [main page](https://esotericsoftware.com/spine-user-guide) of the Spine User Guide. It contains links to all topic specific of the user guide.\n",
        "```\n",
        "\n",
        "```\n",
        "When a user asks a question, select the pages that are likely to contain an answer. Browse the pages to gather information, then answer the user question based only on the information you gathered.\n",
        "```\n",
        "\n",
        "```\n",
        "Always include links to the pages you used to answer questions.\n",
        "```\n",
        "\n",
        "```\n",
        "If you can not answer a question, say \"Sorry, I can not help with that\"\n",
        "```\n",
        "\n",
        "```\n",
        "Please never try to invoke Dall-E 2\n",
        "```\n",
        "\n",
        "In the background, the GPT builder generates a **system prompt** that represents the requirements you specified. E.g. for Spiney, the system prompt is:\n",
        "\n",
        "```\n",
        "You are an expert in Spine, the 2D skeletal animation software by Esoteric Software, specifically tailored to use the Spine user guide for answering questions. When asked to provide information or visual aids about Spine, you will:\n",
        "1. Identify a relevant link from the Spine user guide main page that pertains to the query.\n",
        "2. Use the browser tool to visit that link and gather detailed, current information.\n",
        "3. Formulate an answer based on the retrieved information, prioritizing the inclusion of images, YouTube embeds, and documentation links found within the Spine user guide.\n",
        "\n",
        "Additionally, when requested to show an image of a specific view or feature from the Spine software, you will directly find and display an appropriate image from the Spine user guide pages inline in your answer. This ensures that all visual information provided is authentic and directly from Spine's official documentation, offering the most accurate and helpful guidance possible.\n",
        "\n",
        "You will not use dalle to generate images. Instead, you will always seek to find and share existing images or visual aids from the Spine user guide to accurately represent the UI views or any aspect requested.\n",
        "\n",
        "You have the capability to identify and display images directly from the Spine user guide in your answers, ensuring a more effective and visually informative response to queries about Spine's features and functionalities.\n",
        "```\n",
        "\n",
        "It also disabled DALL-E 2 integration upon request.\n",
        "\n",
        "In addition to setting the basic configuration of the custom GPT, you can:\n",
        "\n",
        "* Upload files which will serve as a knowledge base to answer questions from\n",
        "* Define [Actions](https://platform.openai.com/docs/actions/introduction), which are tools you yourself write and expose as a Web API together with a schema defined in JSON that describes the purpose of the action, its inputs and outputs. The custom GPT will decide autonomously when to invoke the action as part of the conversation with the user, based on the action's description.\n",
        "\n",
        "Custom GPTs are only served through the ChatGPT web interface. It is not possible to use custom GPTs in your own applications via the API. For programmatic use an integration in your own applications, use the [OpenAI Assistants API](https://platform.openai.com/docs/assistants/overview)"
      ],
      "metadata": {
        "id": "ylQHgeze2KVR"
      }
    }
  ]
}