19_whisper_and_gpt.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Whisper API转录"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import openai, os\n",
    "\n",
    "openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
    "\n",
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "欢迎来到 Onboard 真实的一线经验 走新的投资思考 我是 Monica 我是高宁 我们一起聊聊软件如何改变世界 大家好 欢迎来到 Onboard 我是 Monica 自从OpenAI发布的ChatGBT 掀起了席卷世界的AI热潮 不到三个月就积累了 超过一亿的越货用户 超过1300万的日货用户 真的是展现了AI让人惊讶的 也让很多人直呼 这就是下一个互联网的未来 有不少观众都说 希望我们再做一期AI的讨论 于是这次硬核讨论就来了 这次我们请来了 Google Brain的研究员雪芝 她是Google大语言模型PALM Pathway Language Model的作者之一 要知道这个模型的参数量 是GPT-3的三倍还多 另外还有两位AI产品大牛 一位来自著名的StableDM 背后的商业公司Stability AI 另一位来自某硅谷科技大厂 也曾在吴恩达教授的Landing AI中 担任产品负责人 此外 莫妮凯还邀请到一位 一直关注AI的投资人朋友Bill 当做我的特邀共同主持嘉宾 我们主要讨论几个话题 一方面从研究的视角 最前沿的研究者在关注什么 现在技术的天花板 和未来大的变量可能会在哪里 第二个问题是 未来大的变量可能会在哪里 从产品和商业的角度 什么是一个好的AI产品 整个生态可能随着技术 有怎样的演变 更重要的 我们又能从上一波 AI的创业热潮中学到什么 最后 莫妮凯和Bill还会从投资人的视角 做一个回顾 总结和畅想 这里还有一个小的update 在本集发布的时候 Google也对爆发式增长的 Chad GPT做出了回应 正在测试一个基于Lambda 模型的聊天机器人 ApprenticeBot 正式发布后会有怎样的惊喜 我们都拭目以待 AI无疑是未来几年 最令人兴奋的变量之一 莫妮凯也希望未来能邀请到更多 一线从业者 从不同角度讨论这个话题 不论是想要做创业 研究 产品 还是投资的同学 希望这些对话 对于大家了解这些技术演进 商业的可能 甚至未来对于我们每个人 每个社会意味着什么 都能引发一些思考 提供一些启发 这次的讨论有些技术硬核 需要各位对生成式AI 大模型都有一些基础了解 讨论中涉及到的论文和重要概念 也会总结在本集的简介中 供大家复习参考 几位嘉宾在北美工作生活多年 夹杂英文在所难免 也请大家体谅了 欢迎来到未来 希望大家enjoy\n"
     ]
    }
   ],
   "source": [
    "print(transcript['text'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt增加标点符号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, \n",
    "                                     prompt=\"这是一段中文播客内容。\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "欢迎来到 Onboard,真实的一线经验,走新的投资思考。 我是 Monica。 我是高宁。我们一起聊聊软件如何改变世界。 大家好,欢迎来到 Onboard,我是 Monica。 自从 OpenAI 发布的 ChatGBT 掀起了席卷世界的 AI 热潮, 不到三个月就积累了超过一亿的越活用户,超过一千三百万的日活用户。 真的是展现了 AI 让人惊叹的能力, 也让很多人直呼这就是下一个互联网的未来。 有不少观众都说希望我们再做一期 AI 的讨论, 于是这次硬核讨论就来了。 这次我们请来了 Google Brain 的研究员雪芝, 她是 Google 大语言模型 PAMP,Pathway Language Model 的作者之一。 要知道,这个模型的参数量是 GPT-3 的三倍还多。 另外还有两位 AI 产品大牛,一位来自著名的 Stable Diffusion 背后的商业公司 Stability AI, 另一位来自某硅谷科技大厂,也曾在吴恩达教授的 Landing AI 中担任产品负责人。 此外,Monica 还邀请到一位一直关注 AI 的投资人朋友 Bill 当作我的特邀共同主持嘉宾。 我们主要讨论几个话题,一方面从研究的视角,最前沿的研究者在关注什么? 现在技术的天花板和未来大的变量可能会在哪里? 从产品和商业的角度,什么是一个好的 AI 产品? 整个生态可能随着技术有怎样的演变? 更重要的,我们又能从上一波 AI 的创业热潮中学到什么? 最后,Monica 和 Bill 还会从投资人的视角做一个回顾、总结和畅想。 这里还有一个小的 update,在本集发布的时候, Google 也对爆发式增长的ChatGPT 做出了回应, 正在测试一个基于 Lambda 模型的聊天机器人 ApprenticeBot。 正式发布后会有怎样的惊喜?我们都拭目以待。 AI 无疑是未来几年最令人兴奋的变量之一, Monica 也希望未来能邀请到更多一线从业者从不同角度讨论这个话题。 不论是想要做创业、研究、产品还是投资的同学, 希望这些对话对于大家了解这些技术演进、商业的可能, 甚至未来对于我们每个人、每个社会意味着什么, 都能引发一些思考,提供一些启发。 这次的讨论有些技术硬核,需要各位对生成式 AI 大模型都有一些基础了解。 讨论中涉及到的论文和重要概念,也会总结在本集的简介中,供大家复习参考。 几位嘉宾在北美工作生活多年,夹杂英文在所难免,也请大家体谅了。 欢迎来到未来,大家 enjoy!\n"
     ]
    }
   ],
   "source": [
    "print(transcript['text'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prompt增加中英文混排格式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "欢迎来到Onboard,真实的一线经验,走新的投资思考。 我是Monica,我是高宁,我们一起聊聊软件如何改变世界。 大家好,欢迎来到Onboard,我是Monica。 自从OpenAI发布的ChatGBT掀起了席卷世界的AI热潮, 不到三个月就积累了超过一亿的越活用户,超过1300万的日活用户。 真的是展现了AI让人惊叹的能力,也让很多人直呼这就是下一个互联网的未来。 有不少观众都说希望我们再做一期AI的讨论,于是这次硬核讨论就来了。 这次我们请来了Google Brain的研究员雪芝, 她是Google大语言模型POM,Pathway Language Model的作者之一。 要知道这个模型的参数量是GPT-3的三倍还多。 另外还有两位AI产品大牛,一位来自著名的Stable Diffusion背后的商业公司Stability AI, 另一位来自某硅谷科技大厂,也曾在吴恩达教授的Landing AI中担任产品负责人。 此外,Monica还邀请到一位一直关注AI的投资人朋友Bill,当做我的特邀共同主持嘉宾。 我们主要讨论几个话题,一方面从研究的视角,最前沿的研究者在关注什么? 现在的技术的天花板和未来大的变量可能会在哪里? 从产品和商业的角度,什么是一个好的AI产品? 整个生态可能随着技术有怎样的演变? 更重要的,我们又能从上一波AI的创业热潮中学到什么? 最后,Monica和Bill还会从投资人的视角做一个回顾、总结和畅想。 这里还有一个小的update,在本集发布的时候, Google也对爆发式增长的ChatGPT做出了回应, 正在测试一个基于Lambda模型的聊天机器人ApprenticeBot。 正式发布后会有怎样的惊喜?我们都拭目以待。 AI无疑是未来几年最令人兴奋的变量之一, Monica也希望未来能邀请到更多一线从业者从不同角度讨论这个话题。 不论是想要做创业、研究、产品还是投资的同学, 希望这些对话对于大家了解这些技术演进、商业的可能, 甚至未来对于我们每个人、每个社会意味着什么, 都能引发一些思考,提供一些启发。 这次的讨论有些技术硬核,需要各位对生成式AI、大模型都有一些基础了解。 讨论中涉及到的论文和重要概念,也会总结在本集的简介中,供大家复习参考。 几位嘉宾在北美工作生活多年,夹杂英文在所难免,也请大家体谅了。 欢迎来到未来,大家enjoy!\n"
     ]
    }
   ],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, \n",
    "                                     prompt=\"这是一段Onboard播客的内容。\")\n",
    "print(transcript['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "欢迎来到Onboard,真实的一线经验,走新的投资思考。我是Monica。 我是高宁。我们一起聊聊软件如何改变世界。 大家好,欢迎来到Onboard,我是Monica。 自从OpenAI发布的ChatGPT掀起了席卷世界的AI热潮,不到三个月就积累了超过一亿的越活用户,超过1300万的日活用户。 真的是展现了AI让人惊叹的能力,也让很多人直呼这就是下一个互联网的未来。 有不少观众都说希望我们再做一期AI的讨论,于是这次硬核讨论就来了。 这次我们请来了Google Brain的研究员雪芝,她是Google大语言模型PALM Pathways Language Model的作者之一。 要知道,这个模型的参数量是GPT-3的三倍还多。 另外还有两位AI产品大牛,一位来自著名的Stable Diffusion背后的商业公司Stability AI, 另一位来自某硅谷科技大厂,也曾在吴恩达教授的Landing AI中担任产品负责人。 此外,Monica还邀请到一位一直关注AI的投资人朋友Bill当作我的特邀共同主持嘉宾。 我们主要讨论几个话题,一方面从研究的视角,最前沿的研究者在关注什么? 现在的技术的天花板和未来大的变量可能会在哪里? 从产品和商业的角度,什么是一个好的AI产品? 整个生态可能随着技术有怎样的演变? 更重要的,我们又能从上一波AI的创业热潮中学到什么? 最后,Monica和Bill还会从投资人的视角做一个回顾、总结和畅想。 这里还有一个小的update,在本集发布的时候,Google也对爆发式增长的Chat GPT做出了回应。 正在测试一个基于Lambda模型的聊天机器人ApprenticeBot。 证实发布后会有怎样的惊喜,我们都拭目以待。 AI无疑是未来几年最令人兴奋的变量之一。 Monica也希望未来能邀请到更多一线从业者从不同角度讨论这个话题。 不论是想要做创业、研究、产品还是投资的同学, 希望这些对话对于大家了解这些技术演进、商业的可能,甚至未来对于我们每个人、每个社会意味着什么都能引发一些思考,提供一些启发。 这次的讨论有些技术硬核,需要各位对生成式AI大模型都有一些基础了解。 讨论中涉及到的论文和重要概念也会总结在本集的简介中,供大家复习参考。 几位嘉宾在北美工作生活多年,夹杂英文在所难免,也请大家体谅了。 欢迎来到未来,大家enjoy!\n"
     ]
    }
   ],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, \n",
    "                                     prompt=\"这是一段Onboard播客，里面会聊到ChatGPT以及PALM这个大语言模型。这个模型也叫做Pathways Language Model。\")\n",
    "print(transcript['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1\n",
      "00:00:01,000 --> 00:00:07,000\n",
      "欢迎来到Onboard,真实的一线经验,走新的投资思考。我是Monica。\n",
      "\n",
      "2\n",
      "00:00:07,000 --> 00:00:11,000\n",
      "我是高宁。我们一起聊聊软件如何改变世界。\n",
      "\n",
      "3\n",
      "00:00:15,000 --> 00:00:17,000\n",
      "大家好,欢迎来到Onboard,我是Monica。\n",
      "\n",
      "4\n",
      "00:00:17,000 --> 00:00:28,000\n",
      "自从OpenAI发布的ChatGBT掀起了席卷世界的AI热潮,不到三个月就积累了超过一亿的越活用户,超过1300万的日活用户。\n",
      "\n",
      "5\n",
      "00:00:28,000 --> 00:00:34,000\n",
      "真的是展现了AI让人惊叹的能力,也让很多人直呼这就是下一个互联网的未来。\n",
      "\n",
      "6\n",
      "00:00:34,000 --> 00:00:41,000\n",
      "有不少观众都说希望我们再做一期AI的讨论,于是这次硬核讨论就来了。\n",
      "\n",
      "7\n",
      "00:00:41,000 --> 00:00:49,000\n",
      "这次我们请来了Google Brain的研究员雪芝,她是Google大语言模型PALM Pathways Language Model的作者之一。\n",
      "\n",
      "8\n",
      "00:00:49,000 --> 00:00:53,000\n",
      "要知道,这个模型的参数量是GPT-3的三倍还多。\n",
      "\n",
      "9\n",
      "00:00:53,000 --> 00:01:01,000\n",
      "另外还有两位AI产品大牛,一位来自著名的Stable Diffusion背后的商业公司Surbility AI,\n",
      "\n",
      "10\n",
      "00:01:01,000 --> 00:01:08,000\n",
      "另一位来自某硅谷科技大厂,也曾在吴恩达教授的Landing AI中担任产品负责人。\n",
      "\n",
      "11\n",
      "00:01:08,000 --> 00:01:15,000\n",
      "此外,Monica还邀请到一位一直关注AI的投资人朋友Bill当作我的特邀共同主持嘉宾。\n",
      "\n",
      "12\n",
      "00:01:15,000 --> 00:01:22,000\n",
      "我们主要讨论几个话题,一方面从研究的视角,最前沿的研究者在关注什么?\n",
      "\n",
      "13\n",
      "00:01:22,000 --> 00:01:26,000\n",
      "现在的技术的天花板和未来大的变量可能会在哪里?\n",
      "\n",
      "14\n",
      "00:01:26,000 --> 00:01:30,000\n",
      "从产品和商业的角度,什么是一个好的AI产品?\n",
      "\n",
      "15\n",
      "00:01:30,000 --> 00:01:33,000\n",
      "整个生态可能随着技术有怎样的演变?\n",
      "\n",
      "16\n",
      "00:01:33,000 --> 00:01:38,000\n",
      "更重要的,我们又能从上一波AI的创业热潮中学到什么?\n",
      "\n",
      "17\n",
      "00:01:38,000 --> 00:01:44,000\n",
      "最后,Monica和Bill还会从投资人的视角做一个回顾、总结和畅想。\n",
      "\n",
      "18\n",
      "00:01:44,000 --> 00:01:51,000\n",
      "这里还有一个小的update,在本集发布的时候,Google也对爆发式增长的Chat GPT做出了回应。\n",
      "\n",
      "19\n",
      "00:01:51,000 --> 00:01:56,000\n",
      "正在测试一个基于Lambda模型的聊天机器人ApprenticeBot。\n",
      "\n",
      "20\n",
      "00:01:56,000 --> 00:02:00,000\n",
      "证实发布后会有怎样的惊喜,我们都拭目以待。\n",
      "\n",
      "21\n",
      "00:02:00,000 --> 00:02:04,000\n",
      "AI无疑是未来几年最令人兴奋的变量之一。\n",
      "\n",
      "22\n",
      "00:02:04,000 --> 00:02:10,000\n",
      "Monica也希望未来能邀请到更多一线从业者从不同角度讨论这个话题。\n",
      "\n",
      "23\n",
      "00:02:10,000 --> 00:02:15,000\n",
      "不论是想要做创业、研究、产品还是投资的同学,\n",
      "\n",
      "24\n",
      "00:02:15,000 --> 00:02:27,000\n",
      "希望这些对话对于大家了解这些技术演进、商业的可能,甚至未来对于我们每个人、每个社会意味着什么都能引发一些思考,提供一些启发。\n",
      "\n",
      "25\n",
      "00:02:27,000 --> 00:02:34,000\n",
      "这次的讨论有些技术硬核,需要各位对生成式AI大模型都有一些基础了解。\n",
      "\n",
      "26\n",
      "00:02:34,000 --> 00:02:41,000\n",
      "讨论中涉及到的论文和重要概念也会总结在本集的简介中,供大家复习参考。\n",
      "\n",
      "27\n",
      "00:02:41,000 --> 00:02:47,000\n",
      "几位嘉宾在北美工作生活多年,夹杂英文在所难免,也请大家体谅了。\n",
      "\n",
      "28\n",
      "00:02:47,000 --> 00:03:12,000\n",
      "欢迎来到未来,大家enjoy!\n",
      "\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, response_format=\"srt\",\n",
    "                                     prompt=\"这是一段Onboard播客，里面会聊到PALM这个大语言模型。这个模型也叫做Pathways Language Model。\")\n",
    "print(transcript)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WEBVTT\n",
      "\n",
      "00:00:01.000 --> 00:00:07.000\n",
      "欢迎来到Onboard,真实的一线经验,走新的投资思考。我是Monica。\n",
      "\n",
      "00:00:07.000 --> 00:00:11.000\n",
      "我是高宁。我们一起聊聊软件如何改变世界。\n",
      "\n",
      "00:00:15.000 --> 00:00:17.000\n",
      "大家好,欢迎来到Onboard,我是Monica。\n",
      "\n",
      "00:00:17.000 --> 00:00:28.000\n",
      "自从OpenAI发布的ChatGBT掀起了席卷世界的AI热潮,不到三个月就积累了超过一亿的越活用户,超过1300万的日活用户。\n",
      "\n",
      "00:00:28.000 --> 00:00:34.000\n",
      "真的是展现了AI让人惊叹的能力,也让很多人直呼这就是下一个互联网的未来。\n",
      "\n",
      "00:00:34.000 --> 00:00:41.000\n",
      "有不少观众都说希望我们再做一期AI的讨论,于是这次硬核讨论就来了。\n",
      "\n",
      "00:00:41.000 --> 00:00:49.000\n",
      "这次我们请来了Google Brain的研究员雪芝,她是Google大语言模型PALM Pathways Language Model的作者之一。\n",
      "\n",
      "00:00:49.000 --> 00:00:53.000\n",
      "要知道,这个模型的参数量是GPT-3的三倍还多。\n",
      "\n",
      "00:00:53.000 --> 00:01:01.000\n",
      "另外还有两位AI产品大牛,一位来自著名的Stable Diffusion背后的商业公司Surbility AI,\n",
      "\n",
      "00:01:01.000 --> 00:01:08.000\n",
      "另一位来自某硅谷科技大厂,也曾在吴恩达教授的Landing AI中担任产品负责人。\n",
      "\n",
      "00:01:08.000 --> 00:01:15.000\n",
      "此外,Monica还邀请到一位一直关注AI的投资人朋友Bill当作我的特邀共同主持嘉宾。\n",
      "\n",
      "00:01:15.000 --> 00:01:22.000\n",
      "我们主要讨论几个话题,一方面从研究的视角,最前沿的研究者在关注什么?\n",
      "\n",
      "00:01:22.000 --> 00:01:26.000\n",
      "现在的技术的天花板和未来大的变量可能会在哪里?\n",
      "\n",
      "00:01:26.000 --> 00:01:30.000\n",
      "从产品和商业的角度,什么是一个好的AI产品?\n",
      "\n",
      "00:01:30.000 --> 00:01:33.000\n",
      "整个生态可能随着技术有怎样的演变?\n",
      "\n",
      "00:01:33.000 --> 00:01:38.000\n",
      "更重要的,我们又能从上一波AI的创业热潮中学到什么?\n",
      "\n",
      "00:01:38.000 --> 00:01:44.000\n",
      "最后,Monica和Bill还会从投资人的视角做一个回顾、总结和畅想。\n",
      "\n",
      "00:01:44.000 --> 00:01:51.000\n",
      "这里还有一个小的update,在本集发布的时候,Google也对爆发式增长的Chat GPT做出了回应。\n",
      "\n",
      "00:01:51.000 --> 00:01:56.000\n",
      "正在测试一个基于Lambda模型的聊天机器人ApprenticeBot。\n",
      "\n",
      "00:01:56.000 --> 00:02:00.000\n",
      "证实发布后会有怎样的惊喜,我们都拭目以待。\n",
      "\n",
      "00:02:00.000 --> 00:02:04.000\n",
      "AI无疑是未来几年最令人兴奋的变量之一。\n",
      "\n",
      "00:02:04.000 --> 00:02:10.000\n",
      "Monica也希望未来能邀请到更多一线从业者从不同角度讨论这个话题。\n",
      "\n",
      "00:02:10.000 --> 00:02:15.000\n",
      "不论是想要做创业、研究、产品还是投资的同学,\n",
      "\n",
      "00:02:15.000 --> 00:02:27.000\n",
      "希望这些对话对于大家了解这些技术演进、商业的可能,甚至未来对于我们每个人、每个社会意味着什么都能引发一些思考,提供一些启发。\n",
      "\n",
      "00:02:27.000 --> 00:02:34.000\n",
      "这次的讨论有些技术硬核,需要各位对生成式AI大模型都有一些基础了解。\n",
      "\n",
      "00:02:34.000 --> 00:02:41.000\n",
      "讨论中涉及到的论文和重要概念也会总结在本集的简介中,供大家复习参考。\n",
      "\n",
      "00:02:41.000 --> 00:02:47.000\n",
      "几位嘉宾在北美工作生活多年,夹杂英文在所难免,也请大家体谅了。\n",
      "\n",
      "00:02:47.000 --> 00:03:12.000\n",
      "欢迎来到未来,大家enjoy!\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, response_format=\"vtt\",\n",
    "                                     prompt=\"这是一段Onboard播客，里面会聊到PALM这个大语言模型。这个模型也叫做Pathways Language Model。\")\n",
    "print(transcript)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 转录中顺便翻译"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"text\": \"Welcome to Onboard. Real first-line experience. New investment thinking. I am Monica. I am Gao Ning. Let's talk about how software can change the world. Hello everyone, welcome to Onboard. I am Monica. Since the release of ChatGPT by OpenAI, the world's AI has been in a frenzy. In less than three months, it has accumulated more than 100 million active users, and more than 13 million active users. It really shows the amazing ability of AI. It also makes many people say that this is the future of the next Internet. Many viewers said that they wanted us to do another AI discussion. So this discussion came. This time we invited a researcher from Google Brain, Xue Zhi. He is one of the authors of Google's large-scale model PaLM, Pathways Language Model. You should know that the number of parameters of this model is three times more than ChatGPT-3. In addition, there are two AI product big cows. One is from the famous company behind Stable Diffusion, Stability AI. The other is from a Silicon Valley technology factory. He was also the product manager in Professor Wu Wenda's Landing AI. In addition, Monica also invited a friend of AI who has been paying attention to AI, Bill, as my special guest host. We mainly discuss several topics. On the one hand, from the perspective of research, what are the most cutting-edge researchers paying attention to? Where are the cutting-edge technologies and the large variables of the future? From the perspective of products and business, what is a good AI product? What kind of evolution may the whole state follow? More importantly, what can we learn from the previous wave of AI entrepreneurship? Finally, Monica and Bill will also make a review, summary and reflection from the perspective of investors. Here is a small update. When this issue was released, Google also responded to the explosive growth of ChatGPT. We are testing an Apprentice Bot based on Lambda model. What kind of surprises will be released? We are looking forward to it. AI is undoubtedly one of the most exciting variables in the coming years. Monica also hopes to invite more first-line entrepreneurs to discuss this topic from different angles. Whether you want to do entrepreneurship, research, product or investment, I hope these conversations will help you understand the possibilities of these technical horizons and business. Even in the future, it can cause some thoughts and inspire us to think about what it means to each person and each society. This discussion is a bit technical, and requires you to have some basic understanding of the biometric AI model. The papers and important concepts involved in the discussion will also be summarized in this episode's summary, which is for your reference. You have worked in North America for many years, and you may have some English mistakes. Please understand. Welcome to the future. Enjoy. Let me give you a brief introduction. Some of your past experiences. A fun fact. Using an AI to represent the world is now palped.\"\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "audio_file= open(\"./data/podcast_clip.mp3\", \"rb\")\n",
    "translated_prompt=\"\"\"This is a podcast discussing ChatGPT and PaLM model. \n",
    "The full name of PaLM is Pathways Language Model.\"\"\"\n",
    "transcript = openai.Audio.translate(\"whisper-1\", audio_file, \n",
    "                                    prompt=translated_prompt)\n",
    "print(transcript['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Welcome to Onboard. Real first-line experience. New investment thinking. I am Monica. I am Gao Ning. Let's talk about how software can change the world. Hello everyone, welcome to Onboard. I am Monica. Since the release of ChatGPT by OpenAI, the world's AI has been in a frenzy. In less than three months, it has accumulated more than 100 million active users, and more than 13 million active users. It really shows the amazing ability of AI. It also makes many people say that this is the future of the next Internet. Many viewers said that they wanted us to do another AI discussion. So this discussion came. This time we invited a researcher from Google Brain, Xue Zhi. He is one of the authors of Google's large-scale model PaLM, Pathways Language Model. You should know that the number of parameters of this model is three times more than ChatGPT-3. In addition, there are two AI product big cows. One is from the famous company behind Stable Diffusion, Stability AI. The other is from a Silicon Valley technology factory. He was also the product manager in Professor Wu Wenda's Landing AI. In addition, Monica also invited a friend of AI who has been paying attention to AI, Bill, as my special guest host. We mainly discuss several topics. On the one hand, from the perspective of research, what are the most cutting-edge researchers paying attention to? Where are the cutting-edge technologies and the large variables of the future? From the perspective of products and business, what is a good AI product? What kind of evolution may the whole state follow? More importantly, what can we learn from the previous wave of AI entrepreneurship? Finally, Monica and Bill will also make a review, summary and reflection from the perspective of investors. Here is a small update. When this issue was released, Google also responded to the explosive growth of ChatGPT. We are testing an Apprentice Bot based on Lambda model. What kind of surprises will be released? We are looking forward to it. AI is undoubtedly one of the most exciting variables in the coming years. Monica also hopes to invite more first-line entrepreneurs to discuss this topic from different angles. Whether you want to do entrepreneurship, research, product or investment, I hope these conversations will help you understand the possibilities of these technical horizons and business. Even in the future, it can cause some thoughts and inspire us to think about what it means to each person and each society. This discussion is a bit technical, and requires you to have some basic understanding of the biometric AI model. The papers and important concepts involved in the discussion will also be summarized in this episode's summary, which is for your reference. You have worked in North America for many years, and you may have some English mistakes. Please understand. Welcome to the future. Enjoy. Let me give you a brief introduction. Some of your past experiences. A fun fact. Using an AI to represent the world is now palped.\n"
     ]
    }
   ],
   "source": [
    "print(transcript['text'])"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 分割音频转录"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install pydub"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pydub import AudioSegment\n",
    "\n",
    "podcast = AudioSegment.from_mp3(\"./data/podcast_long.mp3\")\n",
    "\n",
    "# PyDub handles time in milliseconds\n",
    "ten_minutes = 15 * 60 * 1000\n",
    "\n",
    "total_length = len(podcast)\n",
    "\n",
    "start = 0\n",
    "index = 0\n",
    "while start < total_length:\n",
    "    end = start + ten_minutes\n",
    "    if end < total_length:\n",
    "        chunk = podcast[start:end]\n",
    "    else:\n",
    "        chunk = podcast[start:]\n",
    "    with open(f\"./data/podcast_clip_{index}.mp3\", \"wb\") as f:\n",
    "        chunk.export(f, format=\"mp3\")\n",
    "    start = end\n",
    "    index += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "original_prompt = \"这是一段Onboard播客，里面会聊到ChatGPT以及PALM这个大语言模型。这个模型也叫做Pathways Language Model。\\n\\n\"\n",
    "prompt = original_prompt\n",
    "for i in range(index):\n",
    "    clip = f\"./data/podcast_clip_{i}.mp3\"\n",
    "    audio_file= open(clip, \"rb\")\n",
    "    transcript = openai.Audio.transcribe(\"whisper-1\", audio_file, \n",
    "                                     prompt=prompt)\n",
    "    # mkdir ./data/transcripts if not exists\n",
    "    if not os.path.exists(\"./data/transcripts\"):\n",
    "        os.makedirs(\"./data/transcripts\")\n",
    "    # write to file\n",
    "    with open(f\"./data/transcripts/podcast_clip_{i}.txt\", \"w\") as f:\n",
    "        f.write(transcript['text'])\n",
    "    # get last sentence of the transcript\n",
    "    sentences = transcript['text'].split(\"。\")\n",
    "    prompt = original_prompt + sentences[-1]\n",
    "    "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 用ChatGPT总结"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Created a chunk of size 2914, which is longer than the specified 2048\n",
      "WARNING:root:Created a chunk of size 3362, which is longer than the specified 2048\n",
      "WARNING:root:Created a chunk of size 2062, which is longer than the specified 2048\n",
      "WARNING:root:Created a chunk of size 2332, which is longer than the specified 2048\n",
      "WARNING:llama_index.llm_predictor.base:Unknown max input size for gpt-3.5-turbo, using defaults.\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total LLM token usage: 0 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [build_index_from_nodes] Total embedding token usage: 0 tokens\n",
      "INFO:llama_index.indices.common_tree.base:> Building index from nodes: 25 chunks\n",
      "INFO:llama_index.indices.common_tree.base:> Building index from nodes: 2 chunks\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total LLM token usage: 106312 tokens\n",
      "INFO:llama_index.token_counter.token_counter:> [query] Total embedding token usage: 0 tokens\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "这个播客讨论了人工智能和深度学习领域的高级技术和最新发展，包括稳定性人工智能、语言模型的预训练方法、图像生成模型的训练和优化，以及各种机器学习模型的比较和应用场景。同时，我们探讨了开源社区的作用和趋势，以及开源商业化的优缺点及如何应对。我们还讨论了人工智能在各个领域的应用和未来发展趋势，并强调了找到实际应用场景和解决实际问题的重要性。最后，我们提醒说，未来值得期待的AI应用将是能够真正跟人交互的产品，对于创业公司来说，需要从用户实际的痛点出发去考虑如何更好地应用AI技术。\n"
     ]
    }
   ],
   "source": [
    "from langchain.chat_models import ChatOpenAI\n",
    "from langchain.text_splitter import SpacyTextSplitter\n",
    "from llama_index import GPTListIndex, LLMPredictor, ServiceContext, SimpleDirectoryReader\n",
    "from llama_index.node_parser import SimpleNodeParser\n",
    "\n",
    "# define LLM\n",
    "llm_predictor = LLMPredictor(llm=ChatOpenAI(temperature=0, model_name=\"gpt-3.5-turbo\", max_tokens=1024))\n",
    "\n",
    "text_splitter = SpacyTextSplitter(pipeline=\"zh_core_web_sm\", chunk_size = 2048)\n",
    "parser = SimpleNodeParser(text_splitter=text_splitter)\n",
    "documents = SimpleDirectoryReader('./data/transcripts').load_data()\n",
    "nodes = parser.get_nodes_from_documents(documents)\n",
    "\n",
    "service_context = ServiceContext.from_defaults(llm_predictor=llm_predictor)\n",
    "\n",
    "list_index = GPTListIndex(nodes=nodes, service_context=service_context)\n",
    "response = list_index.query(\"请你总结一下我们的播客内容:\", response_mode=\"tree_summarize\")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "这个播客讨论了人工智能和深度学习领域的高级技术和最新发展，包括稳定性人工智能、语言模型的预训练方法、图像生成模型的训练和优化，以及各种机器学习模型的比较和应用场景。同时，我们探讨了开源社区的作用和趋势，以及开源商业化的优缺点及如何应对。我们还讨论了人工智能在各个领域的应用和未来发展趋势，并强调了找到实际应用场景和解决实际问题的重要性。最后，我们提醒说，未来值得期待的AI应用将是能够真正跟人交互的产品，对于创业公司来说，需要从用户实际的痛点出发去考虑如何更好地应用AI技术。\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "geektime",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10.9"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}