MMD_auto_motion_run_CH.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "MMD_auto_motion_run_CH",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_k0iM-tgybse"
      },
      "source": [
        "# 欢迎来到colab的MMD自动跟踪! (执行篇)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NxA42Uase5hk"
      },
      "source": [
        "# MMD自动跟踪工具包的准备"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "aCQg_tD0e9v0"
      },
      "source": [
        "该笔记本计算机准备并执行MMD自动跟踪.\n",
        "\n",
        "点击屏幕左上方的“>”。目录将打开\n",
        "\n",
        "![目次](https://drive.google.com/uc?export=view&id=1x8AdFNmsIQPrtYptBf_NXPRNBJF8ON8z)\n",
        "\n",
        "从上到下检查笔记本电脑，然后一步一步执行以下步骤。\n",
        "\n",
        "- **「检查运行时类型是否为GPU」**\n",
        "  - 确保运行时已更改为GPU\n",
        "  - 请检查前面的准备篇以了解如何进行更改\n",
        "- **「是否绑定 Google 云端硬盘」**\n",
        "  - 确保您可以使用Google云端硬盘\n",
        "  - 请检查前面的准备篇以了解如何进行更改\n",
        "- **「准备批处理执行」**\n",
        "    - 运行准备部分中的所有单元格\n",
        "      - 在此过程中，MMD自动跟踪所需的所有程序和数据都在colab上创建。\n",
        "      - 大约需要40至60分钟。\n",
        "- **「执行MMD自动跟踪工具包」**\n",
        "  - 从上到下逐一执行执行单元中的单元，这里请严格遵守，如果没有按照顺序，请重置后，重新开始\n",
        "    - 指定跟踪源视频\n",
        "    - 跟踪参数设置\n",
        "    - 跟踪处理执行\n",
        "    - 视人数而定，使用6000帧视频大约需要50至60分钟。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "F8qp5VzAWyGl"
      },
      "source": [
        "## 检查运行时类型是否为GPU"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "UNrlmW0-W1D1"
      },
      "source": [
        "在菜单栏中选择“**代码执行程序**”>“**更改运行时类型**”>“**GPU**”，并保存。\n",
        "\n",
        "如果已更改为GPU，请执行以下单元格。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "P3zDmyRDwRs0"
      },
      "outputs": [],
      "source": [
        "! nvcc --version\n",
        "! nvidia-smi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "oo-BfaAdw9cN"
      },
      "source": [
        "**【成功】**\n",
        "\n",
        "如果显示如下，则表示成功。\n",
        "\n",
        "![GPU変更成功](https://drive.google.com/uc?export=view&id=17CG697kiTLkwVOdH1wg2W3MSB-hyi9u5)\n",
        "\n",
        "---\n",
        "\n",
        "**【失败】**\n",
        "\n",
        "如果显示如下，则表明运行时更改失败，因此请再次检查运行时类型。\n",
        "\n",
        "![GPU切り替え失敗](https://drive.google.com/uc?export=view&id=1tufSuT7ocWxv3HkrmA5kwlemhu0gv6Je)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "o8OpmLpVp4qr"
      },
      "source": [
        "## 绑定 Google 云端硬盘"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FmdrRx_Tp8Bk"
      },
      "source": [
        "适用于Google云端硬盘 `autotrace` 文件夹，如果需要更换文件夹，可以自行修改下面代码.\n",
        "\n",
        "绑定流程请参照前面的**准备篇**，进行绑定google硬盘\n",
        "\n",
        "请执行以下单元格"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "_f1KFUn_qGsU"
      },
      "outputs": [],
      "source": [
        "from google.colab import drive\n",
        "import os\n",
        "\n",
        "# Google文件系统，外挂gdrive文件夹\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 设置MMD自动化初始文件路径\n",
        "base_path = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "! echo \"Contents of [autotrace] folder -----------\"\n",
        "! ls -l \"$base_path\"\n",
        "! echo \"--------------------\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "38aL3FiVWvmN"
      },
      "source": [
        "## 准备批处理执行"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6k716wWUxT4F"
      },
      "source": [
        "在此，准备部分中的单元被一起执行。\n",
        "\n",
        "在目录中选择“执行MMD自动跟踪工具包”。\n",
        "\n",
        "![执行选择](https://drive.google.com/uc?export=view&id=1pMCBS8cymVNjK_tfZtAQATqcuignRYI5)\n",
        "\n",
        "\n",
        "在菜单中选择 **代码执行程序** > **运行当前单元格之前的所有单元格** ，则准备部分的单元格代码，会自动按照顺序执行下去，请耐心等待，此时不需要任何额外的操作\n",
        "\n",
        "![运行单元格](https://drive.google.com/uc?export=view&id=1nUqciQ20099h_iymDslDwZZeG8d-R-Vy)\n",
        "\n",
        "---\n",
        "\n",
        "** 【OK】**\n",
        "\n",
        "单元格输出以下内容，表示完成。\n",
        "\n",
        "![処理成功](https://drive.google.com/uc?export=view&id=1D21xezv6QN0RQF5ZU_LR7PRnOk0Dw4Sc)\n",
        "\n",
        "It takes about 40 to 60 minutes.\n",
        "\n",
        "---\n",
        "\n",
        "**【NG】**\n",
        "\n",
        "![処理失敗](https://drive.google.com/uc?export=view&id=1t-immeF3Ji1_GBNatZOG1C07j42de4Rq)\n",
        "\n",
        "如果在最后一行输出“没有这样的文件或目录”，则失败。\n",
        "\n",
        "如果您不知道解决方案，请共享笔记本。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "E-h6RWCXnZU8"
      },
      "source": [
        "### 设置运行环境"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "IPiDvSBanScr"
      },
      "outputs": [],
      "source": [
        "# 开始时间\n",
        "import time\n",
        "start_time = time.time()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "9UMIfs3snkRm"
      },
      "outputs": [],
      "source": [
        "# Openpose版本标签\n",
        "ver_openpose = \"v1.5.1\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "iWQcbt_rnblK"
      },
      "outputs": [],
      "source": [
        "# MMD自动跟踪工具包版本标签\n",
        "ver_tag = \"work_1.03\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "EK-WUxgciv9V"
      },
      "source": [
        "### 下载已编译好的openpose"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "djxuuJjKix5B"
      },
      "outputs": [],
      "source": [
        "#https://drive.google.com/open?id=1DDdX2FgIB_7FM3GaxuRb92CIuhiiDyXo\n",
        "\n",
        "file_id = \"1DDdX2FgIB_7FM3GaxuRb92CIuhiiDyXo\"\n",
        "file_name = \"openpose.zip\"\n",
        "! cd  ./ && curl -sc ./cookie \"https://drive.google.com/uc?export=download&id=$file_id\" > /dev/null\n",
        "code = \"$(awk '/_warning_/ {print $NF}' ./cookie)\"  \n",
        "! cd  ./ && curl -Lb ./cookie \"https://drive.google.com/uc?export=download&confirm=$code&id=$file_id\" -o \"$file_name\"\n",
        "! cd  ./ && unzip openpose.zip\n",
        "\n",
        "#  Openpose 修改运行环境\n",
        "! echo '/content/content/openpose/build/caffe/lib'>> /etc/ld.so.conf\n",
        "! echo '/content/content/openpose/build/src/openpose'>> /etc/ld.so.conf\n",
        "! ldconfig "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Viqw8qJqfDyf"
      },
      "source": [
        "### Openpose 依赖库安装"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "a-fnE9kwgcfg"
      },
      "outputs": [],
      "source": [
        "# 安装依赖库文件\n",
        "\n",
        "# Basic\n",
        "! sudo apt-get --assume-yes update\n",
        "! sudo apt-get --assume-yes install build-essential\n",
        "# OpenCV\n",
        "! sudo apt-get --assume-yes install libopencv-dev\n",
        "# General dependencies\n",
        "! sudo apt-get --assume-yes install libatlas-base-dev libprotobuf-dev libleveldb-dev libsnappy-dev libhdf5-serial-dev protobuf-compiler\n",
        "! sudo apt-get --assume-yes install --no-install-recommends libboost-all-dev\n",
        "# Remaining dependencies, 14.04\n",
        "! sudo apt-get --assume-yes install libgflags-dev libgoogle-glog-dev liblmdb-dev\n",
        "# Python2 libs\n",
        "! sudo apt-get --assume-yes install python-setuptools python-dev build-essential\n",
        "! sudo easy_install pip\n",
        "! sudo -H pip install --upgrade numpy protobuf opencv-python\n",
        "# Python3 libs\n",
        "! sudo apt-get --assume-yes install python3-setuptools python3-dev build-essential\n",
        "! sudo apt-get --assume-yes install python3-pip\n",
        "! sudo -H pip3 install --upgrade numpy protobuf opencv-python\n",
        "# OpenCV 2.4 -> Added as option\n",
        "# # sudo apt-get --assume-yes install libopencv-dev\n",
        "# OpenCL Generic\n",
        "! sudo apt-get --assume-yes install opencl-headers ocl-icd-opencl-dev\n",
        "! sudo apt-get --assume-yes install libviennacl-dev"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lZLayojmhDdI"
      },
      "source": [
        "### mannequinchallenge-vmd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "rJw3lnjKhIp7"
      },
      "outputs": [],
      "source": [
        "# mannequinchallenge-vmd  clone\n",
        "\n",
        "! git clone  --depth 1 -b \"$ver_tag\" https://github.com/miu200521358/mannequinchallenge-vmd.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "sNbLo-sBhlcL"
      },
      "outputs": [],
      "source": [
        "# mannequinchallenge-vmd 识别深度模型下载\n",
        "\n",
        "! cd  ./mannequinchallenge-vmd && ./fetch_checkpoints.sh"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Gvkd3YfCiVJ8"
      },
      "source": [
        "### 3d-pose-baseline-vmd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "w3Uh4e6liYPg"
      },
      "outputs": [],
      "source": [
        "# 3d-pose-baseline-vmd  clone\n",
        "! git clone  --depth 1 -b \"$ver_tag\" https://github.com/miu200521358/3d-pose-baseline-vmd.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "Iw7kI9zhi_7k"
      },
      "outputs": [],
      "source": [
        "# 3d-pose-baseline-vmd Human3.6M 模型数据DL\n",
        "\n",
        "# 建立Human3.6M模型数据文件夹\n",
        "! mkdir -p ./3d-pose-baseline-vmd/data\n",
        "\n",
        "# 下载Human3.6M模型数据并解压\n",
        "! cd  ./3d-pose-baseline-vmd/data && wget -c \"https://www.dropbox.com/s/e35qv3n6zlkouki/h36m.zip\" && unzip h36m.zip"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "dclND00zjGdN"
      },
      "outputs": [],
      "source": [
        "# 3d-pose-baseline-vmd 训练数据\n",
        "\n",
        "# 3d-pose-baseline学习数据文件夹\n",
        "! mkdir -p ./3d-pose-baseline-vmd/experiments\n",
        "\n",
        "# 下载3d-pose-baseline训练后的数据\n",
        "file_id = \"1v7ccpms3ZR8ExWWwVfcSpjMsGscDYH7_\"\n",
        "file_name = \"experiments.zip\"\n",
        "! cd  ./3d-pose-baseline-vmd && curl -sc ./cookie \"https://drive.google.com/uc?export=download&id=$file_id\" > /dev/null\n",
        "code = \"$(awk '/_warning_/ {print $NF}' ./cookie)\"  \n",
        "! cd  ./3d-pose-baseline-vmd && curl -Lb ./cookie \"https://drive.google.com/uc?export=download&confirm=$code&id=$file_id\" -o \"$file_name\"\n",
        "! cd  ./3d-pose-baseline-vmd && unzip experiments.zip"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "jruMP1J4jLXX"
      },
      "source": [
        "### VMD-3d-pose-baseline-multi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "87ZPjj6IjPgj"
      },
      "outputs": [],
      "source": [
        "# VMD-3d-pose-baseline-multi  clone\n",
        "\n",
        "! git clone  --depth 1 -b \"$ver_tag\" https://github.com/miu200521358/VMD-3d-pose-baseline-multi.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "fnuSwMT9jW5E"
      },
      "outputs": [],
      "source": [
        "# 安装VMD-3d-pose-baseline-multi 依赖库\n",
        "\n",
        "! sudo apt-get install python3-pyqt5  \n",
        "! sudo apt-get install pyqt5-dev-tools\n",
        "! sudo apt-get install qttools5-dev-tools"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GIp8lIjZY7ih"
      },
      "source": [
        "### 确认准备环境结果"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "T8vXn_ZE6bHk"
      },
      "outputs": [],
      "source": [
        "# 执行示例确认\n",
        "! cd /content/content/openpose && ./build/examples/openpose/openpose.bin --video examples/media/video.avi --write_json ./output/ --display 0  --write_video ./output/openpose.avi"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "gZjjbPz0llYs"
      },
      "outputs": [],
      "source": [
        "import time\n",
        "\n",
        "elapsed_time = (time.time() - start_time) / 60\n",
        "\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "! echo \"■■所有处理均已完成\"\n",
        "! echo \"■■\"\n",
        "! echo \"■■处理时间：\" \"$elapsed_time\" \"分\"\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "! echo \"Openpose执行结果\"\n",
        "\n",
        "! ls -l ./content/openpose/output/openpose.avi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "IrnCtUv8XohO"
      },
      "source": [
        "# 执行MMD自动跟踪工具包"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "3g59mu0Q5fC8"
      },
      "source": [
        "## 安装完成确认"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "l5-4naWjXqUo"
      },
      "source": [
        "从这里开始，我们将实际执行该工具包。\n",
        "\n",
        "确认“运行上一个单元”中，是否已完成所有安装？\n",
        "\n",
        "有关更多信息，请查看“准备篇”部分\n",
        "\n",
        "准备就绪后，运行下面的单元格以查看安装是否完成"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "soBmKdn_KjW5"
      },
      "outputs": [],
      "source": [
        "!ls -l ./content/openpose/README.md\n",
        "!ls -l ./FCRN-DepthPrediction-vmd/README.md\n",
        "!ls -l ./3d-pose-baseline-vmd/README.md\n",
        "!ls -l ./VMD-3d-pose-baseline-multi/README.md\n",
        "!ls -l ./content/openpose/output/openpose.avi"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Dd1C8N7iL4ZJ"
      },
      "source": [
        "**【OK】**\n",
        "\n",
        "如果如下所示显示文件名和文件大小，则表明安装已完成。\n",
        "\n",
        "请继续输入视频文件.\n",
        "\n",
        "![インストール成功](https://drive.google.com/uc?export=view&id=1l13A2iF9oTpGcZSe9q8k7JyDyiABxOQT)\n",
        "\n",
        "---\n",
        "\n",
        "**【NG】**\n",
        "\n",
        "如果如下所示显示“没有这样的文件或目录”，则安装失败。\n",
        "\n",
        "![インストール失敗](https://drive.google.com/uc?export=view&id=1LuKoSMwFOzFg8NguFxqAtmQy9B1_KMXr)\n",
        "\n",
        "返回此笔记本的顶部，然后从头开始重试(最好进行重置一次)。\n",
        "\n",
        "如果尝试三遍后安装失败，请共享此次运行代码。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "4R8ogPNZXtQc"
      },
      "source": [
        "## 输入视频文件上传"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FrglOLBZXv9B"
      },
      "source": [
        "准备要处理的视频文件.\n",
        "\n",
        " - 文件名 **只能是半角字符**. opencv无法读取全角字符\n",
        " - 请将其放在Google云端硬盘的 **autotrace** 文件夹下\n",
        " - FPS 应该是 **30fps** 或 **60fps**.\n",
        " - 视频大小应为 **1280x720**.\n",
        " - 如果大小或fps未指定，则程序将重新编码。（重新编码为每秒帧数为30，大小1280x720）\n",
        " - **安装后无法正确识别或覆盖Goole驱动器上的文件.** 请使用新名称上传新文件并进行处理。\n",
        " - 跟踪多人时，如果人体从屏幕上消失，则可能会导致顺序错误，并且轨迹可能不正常。确保尽可能多地显示要获取的人员的所有成员\n",
        " - 上传完成后，依次执行下部的单元格。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "WQxj2Y6-Zutl"
      },
      "outputs": [],
      "source": [
        "#@markdown ### 【O】输入视频文件\n",
        "#@markdown 输入要分析的视频文件的名称。\n",
        "#@markdown 如果宽度不是1280或帧速率不是30fps，则会执行重新编码(这里存在BUG，如果视频帧大小不是30FPS，会导致视频动作时间错误,所以请尽量使用30FPS的视频)。\n",
        "input_video_name = \"input.mp4\"  #@param {type: \"string\"}\n",
        "\n",
        "from google.colab import drive\n",
        "import os\n",
        "import cv2\n",
        "import datetime\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "base_path = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "! echo \"自动跟踪文件夹的内容 -----------\"\n",
        "! ls -l \"$base_path\"\n",
        "! echo \"--------------------\"\n",
        "\n",
        "# 入力動画ファイル\n",
        "input_video = base_path + \"/\"+ input_video_name\n",
        "\n",
        "print(\"视频名称: \", os.path.basename(input_video))\n",
        "print(\"视频大小: \", os.path.getsize(input_video))\n",
        "\n",
        "\n",
        "video = cv2.VideoCapture(input_video)\n",
        "# 宽\n",
        "W = video.get(cv2.CAP_PROP_FRAME_WIDTH)\n",
        "# 高\n",
        "H = video.get(cv2.CAP_PROP_FRAME_HEIGHT)\n",
        "# 总帧数\n",
        "count = video.get(cv2.CAP_PROP_FRAME_COUNT)\n",
        "# fps\n",
        "fps = video.get(cv2.CAP_PROP_FPS)\n",
        "\n",
        "print(\"宽: {0}, 高: {1}, 总帧数: {2}, fps: {3}\".format(W, H, count, fps))\n",
        "\n",
        "\n",
        "\n",
        "width = 1280\n",
        "height = 720\n",
        "\n",
        "if W != 1280 or (fps != 30 and fps != 60):\n",
        "    print(\"重新编码，因为大小或fps不在范围: \"+ input_video)\n",
        "    \n",
        "    # 縮尺\n",
        "    scale = width / W\n",
        "    \n",
        "    # 高さ\n",
        "    height = int(H * scale)\n",
        "\n",
        "    # 出力ファイルパス\n",
        "    out_name = 'recode_{0}.mp4'.format(\"{0:%Y%m%d_%H%M%S}\".format(datetime.datetime.now()))\n",
        "    out_path = '{0}/{1}'.format(base_path, out_name)\n",
        "    \n",
        "    try:\n",
        "        fourcc = cv2.VideoWriter_fourcc(*\"MP4V\")\n",
        "        out = cv2.VideoWriter(out_path, fourcc, 30.0, (width, height), True)\n",
        "        # 入力ファイル\n",
        "        cap = cv2.VideoCapture(input_video)\n",
        "\n",
        "        while(cap.isOpened()):\n",
        "            # 動画から1枚キャプチャして読み込む\n",
        "            flag, frame = cap.read()  # Capture frame-by-frame\n",
        "\n",
        "            # 動画が終わっていたら終了\n",
        "            if flag == False:\n",
        "                break\n",
        "\n",
        "            # 縮小\n",
        "            output_frame = cv2.resize(frame, (width, height))\n",
        "\n",
        "            # 出力\n",
        "            out.write(output_frame)\n",
        "\n",
        "        # 終わったら開放\n",
        "        out.release()\n",
        "    except Exception as e:\n",
        "        print(\"重新编码失败\", e)\n",
        "\n",
        "    cap.release()\n",
        "    cv2.destroyAllWindows()\n",
        "    \n",
        "    print('MMD重新生成MP4文件成功', out_path)\n",
        "    input_video_name = out_name\n",
        "\n",
        "    # 入力動画ファイル再設定\n",
        "    input_video = base_path + \"/\"+ input_video_name\n",
        "    \n",
        "    video = cv2.VideoCapture(input_video)\n",
        "    # 幅\n",
        "    W = video.get(cv2.CAP_PROP_FRAME_WIDTH)\n",
        "    # 高さ\n",
        "    H = video.get(cv2.CAP_PROP_FRAME_HEIGHT)\n",
        "    # 総フレーム数\n",
        "    count = video.get(cv2.CAP_PROP_FRAME_COUNT)\n",
        "    # fps\n",
        "    fps = video.get(cv2.CAP_PROP_FPS)\n",
        "\n",
        "    print(\"【重新生成】宽: {0}, 高: {1}, 总帧数: {2}, fps: {3}\".format(W, H, count, fps))\n",
        "\n",
        "    \n",
        "!echo \"输入视频文件为\" \"$input_video_name\" \"\"\n",
        "!echo \"\"\n",
        "!echo \"如果没有出现失败，请继续。\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "LXcgFDk-YDMB"
      },
      "source": [
        "如果获得了最后一个文件名，则成功\n",
        "\n",
        "---\n",
        "**【OK】**\n",
        "\n",
        "如果文件大小和fps如指定，则按原样处理输入文件。\n",
        "\n",
        "![OK](https://drive.google.com/uc?export=view&id=1lvOhCAj99_NUNDb-wfRAxeu-o0Exth7v)\n",
        "\n",
        "----\n",
        "**【Re-encoding】**\n",
        "\n",
        "如果文件大小和fps没有指定，则将重新编码的mp4文件视为输入文件。\n",
        "\n",
        "![再エンコード](https://drive.google.com/uc?export=view&id=1xEiy-pdeHWQpt4CLZePg9YT1_7U8bEqz)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "8qMeT79QYFeC"
      },
      "source": [
        "## 参数设定"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "AFlJGTLmFLXE"
      },
      "source": [
        "Please set the parameters.\n",
        "\n",
        " - 【O】… Openpose 使用的参数\n",
        " - 【F】… mannequinchallenge-vmd 使用的参数\n",
        " - 【V】… VMD-3d-pose-baseline-multi 使用的参数"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "FQ3yCAl8YI1o"
      },
      "outputs": [],
      "source": [
        "#@markdown 输入用于跟踪图像的参数并执行单元。\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【O】视频中的最大人数\n",
        "#@markdown 请输入您希望从视频中获得的人数。\n",
        "#@markdown 请与视频中人数尽量保持一致\n",
        "number_people_max = 1  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【O】要从第几帧开始分析\n",
        "#@markdown 输入帧号以开始分析。（从0开始）\n",
        "#@markdown 请指定在视频中显示所有人的第一帧，默认为0即可，除非你需要跳过某些片段（例如片头）。\n",
        "frame_first = 0  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【O】是否开启高精度\n",
        "#@markdown 选择以开启最高精度输出，yes或no \n",
        "#@markdown ，注：此功能非必须,开启此功能会大幅增加openpose的时间，如果openpose出现错误，请关闭此开关\n",
        "high_acc_flag = \"no\"  #@param ['yes', 'no']\n",
        "is_high_acc = 1 if high_acc_flag == \"yes\" else 0\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【F】要从第几帧结束\n",
        "#@markdown 请输入要从哪一帧结束\n",
        "#@markdown （从0开始）在“FCRN-DepthPrediction-vmd”中调整反向或顺序时，可以完成过程并查看结果，默认为-1 表示执行到最后\n",
        "end_frame_no = -1  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【F】反转数据表\n",
        "#@markdown 指定由Openpose反转的帧号（从0开始），人员INDEX顺序和反转的内容。\n",
        "#@markdown 按照Openpose在 0F 识别的顺序，将INDEX分配为0,1,...。\n",
        "\n",
        "#@markdown 格式: [{帧号}: 用于指定反转的人INDEX, {反转内容}]\n",
        "#@markdown {反转内容}: R: 整体身体反转, U:上半身反转, L: 下半身反转, N: 无反转\n",
        "\n",
        "#@markdown 例如：[10:1,R]　整个人在第10帧中反转第一个人。在message.log中会记录以上述格式输出内容\n",
        "\n",
        "\n",
        "#@markdown 因此请参考与[10:1,R][30:0,U],中一样，可以在括号中指定多个项目 ps(不要带有中文标点符号))\n",
        "reverse_specific = \"\"  #@param {type: \"string\"}\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【F】输出颜色(仅参考，如果多人时，某个人序号跟别人交换或者错误，可以用此项修改)\n",
        "#@markdown 请在多人轨迹中的交点之后指定人索引顺序。如果要跟踪一个人，可以将其留为空白。\n",
        "#@markdown 按照Openpose在0F时识别的顺序分配0、1和INDEX。格式：[<帧号>：第几个人的索引，第几个人的索引，…]示例）[10:1,0]…第帧10是从左数第1人按第0个人的顺序对其进行排序。\n",
        "#@markdown message.log包含以上述格式输出的顺序，因此请参考它。可以在括号中指定多个项目，例如[10:1,0] [30:0,1]。在output_XXX.avi中，按照估计顺序为人们分配了颜色。身体的右半部分为红色，左半部分为以下颜色。\n",
        "#@markdown 0：绿色，1：蓝色，2：白色，3：黄色，4：桃红色，5：浅蓝色，6：深绿色，7：深蓝色，8：灰色，9：深黄色，​​10：深桃红色，11：深浅蓝色\n",
        "order_specific = \"\"  #@param {type: \"string\"}\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【V】骨骼结构CSV文件\n",
        "#@markdown 选择或输入跟踪目标模型的骨骼结构CSV文件的路径。请将csv文件上传到Google云端硬盘的“ autotrace”文件夹。\n",
        "#@markdown 您可以选择 \"Animasa-Miku\" 和 \"Animasa-Miku semi-standard\", 也可以输入任何模型的骨骼结构CSV文件\n",
        "#@markdown 如果要输入任何模型骨骼结构CSV文件, 请将csv文件上传到Google云端硬盘的 \"autotrace\" 文件夹下\n",
        "#@markdown 然后请输入「/gdrive/My Drive/autotrace/[csv file name]」\n",
        "born_model_csv = \"born/animasa_miku_born.csv\" #@param [\"born/animasa_miku_born.csv\", \"born/animasa_miku_semi_standard_born.csv\"] {allow-input: true}\n",
        "\n",
        "\n",
        "#@markdown --- \n",
        "\n",
        "#@markdown ### 【V】是否使用IK输出\n",
        "#@markdown 选择以IK输出，yes或no \n",
        "#@markdown 如果输入no，则以输出FK\n",
        "ik_flag = \"yes\"  #@param ['yes', 'no']\n",
        "is_ik = 1 if ik_flag == \"yes\" else 0\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】脚与地面位置校正\n",
        "#@markdown 请输入数值的鞋跟的Y轴校正值（可以为小数）\n",
        "#@markdown 输入负值会接近地面，输入正值会远离地面。\n",
        "#@markdown 尽管会自动在某种程度上自动校正，但如果无法校正，请进行设置。\n",
        "heel_position = 0.0  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】Z中心放大倍率\n",
        "#@markdown 以将放大倍数应用到Z轴中心移动（可以是小数）\n",
        "#@markdown 值越小，中心Z移动的宽度越小\n",
        "#@markdown 输入0时，不进行Z轴中心移动。\n",
        "center_z_scale =   1#@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】Z 轴深度平滑频率\n",
        "#@markdown 指定Z轴的平滑频率\n",
        "#@markdown 请仅输入大于1或者更大的整数\n",
        "#@markdown 频率越大，动作越平滑 (Z 轴方向)\n",
        "depth_smooth_times = 4  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】平滑频率\n",
        "#@markdown 指定运动的平滑频率\n",
        "#@markdown 请仅输入1或更大的整数\n",
        "#@markdown 频率越大，频率越平滑。（行为幅度会变小）\n",
        "smooth_times =   1#@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】移动稀疏量 (低于该阀值的运动宽度，不会进行输出，防抖动)\n",
        "#@markdown 用数值（允许小数）指定用于稀疏移动（IK /中心）的移动量\n",
        "#@markdown 如果在指定范围内有移动，则将稀疏。如果移动抽取量设置为0，则不执行抽取。\n",
        "#@markdown 当移动稀疏量设置为0时，不进行稀疏。\n",
        "threshold_pos = 0.5  #@param {type: \"number\"}\n",
        "\n",
        "#@markdown ---\n",
        "\n",
        "#@markdown ### 【V】旋转稀疏角 (低于该阀值的运动角度，则不会进行输出)\n",
        "#@markdown 指定用于稀疏旋转键的角度（0到180度的十进制数）\n",
        "#@markdown 如果在指定角度范围内有旋转，则稀疏旋转键。\n",
        "threshold_rot = 3  #@param {type: \"number\"}\n",
        "\n",
        "\n",
        "\n",
        "print(\" 【O】Maximum number of people in the video: \"+str(number_people_max))\n",
        "print(\" 【O】Frame number to start analysis: \"+str(frame_first))\n",
        "print(\" 【O】Whether to turn on high Accuracy : \"+str(high_acc_flag))\n",
        "print(\" 【F】Frame number to finish analysis: \"+str(end_frame_no))\n",
        "print(\" 【F】Reverse specification list: \"+str(reverse_specific))\n",
        "print(\" 【F】Ordered list: \"+str(order_specific))\n",
        "print(\" 【V】Bone structure CSV file: \"+str(born_model_csv))\n",
        "print(\" 【V】Whether to output with IK: \"+str(ik_flag))\n",
        "print(\" 【V】Heel position correction: \"+str(heel_position))\n",
        "print(\" 【V】Center Z moving magnification: \"+str(center_z_scale))\n",
        "print(\" 【V】Center-Z Smoothing frequency: \"+str(depth_smooth_times))\n",
        "print(\" 【V】Smoothing frequency: \"+str(smooth_times))\n",
        "print(\" 【V】Movement key thinning amount: \"+str(threshold_pos))\n",
        "print(\" 【V】Rotating Key Culling Angle: \"+str(threshold_rot))\n",
        "\n",
        "\n",
        "print(\"\")\n",
        "print(\"If the above is correct, please proceed to the next.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SFWBO6c0YLa4"
      },
      "source": [
        "## 自动跟踪执行（全部执行）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "Zz0bFW4CYQJ9"
      },
      "outputs": [],
      "source": [
        "#@markdown 填写上面的表格后，执行此单元格。按以下顺序执行程序。\n",
        "\n",
        "#@markdown 1. Openpose（Video→2D）(提取2D坐标，耗时最长)\n",
        "#@markdown 2. mannequinchallenge-vmd（Depth estimation）(视频深度计算,计算Z轴)\n",
        "#@markdown 3. 3d-pose-baseline-vmd（2D→3D）(生成3D坐标)\n",
        "#@markdown 4. VMD-3d-pose-baseline-multi（3D→VMD）(生成VMD数据，参数主要针对这里调整)\n",
        "\n",
        "#@markdown 尽管视频人数对分析时间有影响，但是通常6000F下大约需要50-60分钟。当Openpose启动时，本单元格会一直处于等待状态，三角形播放按钮会一直旋转。\n",
        "#@markdown 此时，请耐心等候而无需任何操作。\n",
        "#@markdown 如果尚未生成vmd文件，pos.txt的内容为空，或者只有error.txt，首先检查error.txt的内容，然后重新检查并执行，\n",
        "#@markdown 如果还是无法解决，请参照前面的 准备篇 中的错误反馈\n",
        "\n",
        "import time\n",
        "import datetime\n",
        "import cv2\n",
        "import shutil\n",
        "import glob\n",
        "from google.colab import drive\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# 出力フォルダ削除\n",
        "if os.path.exists(\"./output\"):\n",
        "    !rm -r ./output\n",
        "\n",
        "# 処理日時\n",
        "now_str = \"{0:%Y%m%d_%H%M%S}\".format(datetime.datetime.now())\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "drive_base_dir = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "output_json = \"/content/output/json\"\n",
        "output_openpose_avi = \"/content/output/openpose.avi\"\n",
        "! mkdir -p \"$output_json\"\n",
        "\n",
        "# 出力用Googleドライブフォルダ\n",
        "drive_dir_path = drive_base_dir + \"/\" + now_str \n",
        "! mkdir -p \"$drive_dir_path\"\n",
        "\n",
        "! echo ------------------------------------------\n",
        "! echo Openpose\n",
        "! echo ------------------------------------------\n",
        "\n",
        "# Openpose実行\n",
        "if is_high_acc :\n",
        "  ! echo \"高精度模式\"\n",
        "  ! cd content/openpose/ && ./build/examples/openpose/openpose.bin --video \"$input_video\" --display 0 --model_pose COCO --write_json \"$output_json\" --write_video \"$output_openpose_avi\" --frame_first \"$frame_first\" --number_people_max \"$number_people_max\" --scale_number 4 --scale_gap 0.25 --net_resolution \"1312x736\"\n",
        "else :\n",
        "  ! echo \"正常模式\"\n",
        "  ! cd content/openpose/ && ./build/examples/openpose/openpose.bin --video \"$input_video\" --display 0 --model_pose COCO --write_json \"$output_json\" --write_video \"$output_openpose_avi\" --frame_first \"$frame_first\" --number_people_max \"$number_people_max\" \n",
        "\n",
        "\n",
        "! echo ------------------------------------------\n",
        "! echo FCRN-DepthPrediction-vmd\n",
        "! echo ------------------------------------------\n",
        "\n",
        "! cd mannequinchallenge-vmd && python predict_video.py --video_path \"$input_video\" --json_path \"$output_json\" --interval 20 --reverse_specific \"$reverse_specific\" --order_specific \"$order_specific\" --verbose 1 --now \"$now_str\" --avi_output \"yes\"  --number_people_max \"$number_people_max\" --end_frame_no \"$end_frame_no\" --input single_view --batchSize 1\n",
        "\n",
        "    \n",
        "# 深度結果コピー\n",
        "depth_dir_path =  output_json + \"_\" + now_str + \"_depth\"\n",
        "\n",
        "if os.path.exists( depth_dir_path + \"/error.txt\"):\n",
        "    \n",
        "    # エラー発生\n",
        "    ! cp \"$depth_dir_path\"/error.txt \"$drive_dir_path\"\n",
        "\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "    ! echo \"■■由于发生错误，处理被中断。\"\n",
        "    ! echo \"■■\"\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "    ! echo \"$drive_dir_path\" \"请检查 error.txt 的内容。\"\n",
        "\n",
        "else:\n",
        "    \n",
        "    ! cp \"$depth_dir_path\"/*.avi \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/message.log \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/reverse_specific.txt \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/order_specific.txt \"$drive_dir_path\"\n",
        "\n",
        "    for i in range(1, number_people_max+1):\n",
        "        ! echo ------------------------------------------\n",
        "        ! echo 3d-pose-baseline-vmd [\"$i\"]\n",
        "        ! echo ------------------------------------------\n",
        "\n",
        "        target_name = \"_\" + now_str + \"_idx0\" + str(i)\n",
        "        target_dir = output_json + target_name\n",
        "\n",
        "        !cd ./3d-pose-baseline-vmd && python src/openpose_3dpose_sandbox_vmd.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 200 --load 4874200 --gif_fps 30 --verbose 1 --openpose \"$target_dir\" --person_idx 1    \n",
        "\n",
        "        ! echo ------------------------------------------\n",
        "        ! echo VMD-3d-pose-baseline-multi [\"$i\"]\n",
        "        ! echo ------------------------------------------\n",
        "\n",
        "        ! cd ./VMD-3d-pose-baseline-multi && python main.py -v 2 -t \"$target_dir\" -b \"$born_model_csv\" -c 30 -z \"$center_z_scale\" -s \"$smooth_times\" -p \"$threshold_pos\" -r \"$threshold_rot\" -k \"$is_ik\" -e \"$heel_position\" -d \"$depth_smooth_times\"\n",
        "\n",
        "        # INDEX別結果コピー\n",
        "        idx_dir_path = drive_dir_path + \"/idx0\" + str(i)\n",
        "        ! mkdir -p \"$idx_dir_path\"\n",
        "        \n",
        "        # 日本語対策でpythonコピー\n",
        "        for f in glob.glob(target_dir +\"/*.vmd\"):\n",
        "            shutil.copy(f, idx_dir_path)\n",
        "        \n",
        "        ! cp \"$target_dir\"/pos.txt \"$idx_dir_path\"\n",
        "        ! cp \"$target_dir\"/start_frame.txt \"$idx_dir_path\"\n",
        "\n",
        "    # Googleドライブ再マウント\n",
        "    drive.mount('/gdrive')\n",
        "\n",
        "    elapsed_time = (time.time() - start_time) / 60\n",
        "\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "    ! echo \"■■所有处理完成\"\n",
        "    ! echo \"■■\"\n",
        "    ! echo \"■■处理時間：\" \"$elapsed_time\" \"分\"\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "    ! echo \"\"\n",
        "    ! echo \"MMD自动跟踪执行结果\"\n",
        "\n",
        "    ! echo \"$drive_dir_path\"\n",
        "    ! ls -l \"$drive_dir_path\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6mhinYooFuj1"
      },
      "source": [
        "## 自动跟踪执行（部分执行）(主要用于优化自动化结果使用)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WKNMqF4LF3zw"
      },
      "source": [
        "**请先完成上面的自动跟踪(全部执行)，再开始下面的执行，这里主要用于优化生成结果，进行快速的调试生成报告** \n",
        "\n",
        "如果要进行修改，请根据下方提示修改，以重新生成数据\n",
        "\n",
        "1. 当您想更改跟踪源视频时\n",
        " - 上传新视频到谷歌云端硬盘到 **autotrace** 文件夹中\n",
        " - 执行 **\"输入视频文件上传\"** 单元格\n",
        " - 执行 **\"参数设定\"** 单元格\n",
        " - 执行 **\"自动跟踪执行（全部执行）\"** 单元格 \n",
        "  \n",
        "2. 如果更改【O】openpose的参数\n",
        " - 更改**\"参数设定\"**中【O】的参数\n",
        " - 执行 **\"参数设定\"** 单元格\n",
        " - 执行 **\"自动跟踪执行（全部执行）\"** 单元格 \n",
        "\n",
        "3. 如果要更改【F】 mannequinchallenge-vmd深度估计的参数\n",
        " - 更改**\"参数设定\"**中【F】的参数\n",
        " - 执行 **\"参数设定\"** 单元格\n",
        " - 执行 **\"A）自动跟踪重新执行（深度估计）单元格\"**\n",
        " - 如果需要生成VMD数据，请继续往下执行\n",
        "       - 执行 **\"B) 自动跟踪重新运行（2D→3D）\"**\n",
        "       - 执行 **\"C) 自动跟踪重新运行（3D→VMD）\"** \n",
        " \n",
        "4. 如果要更改【V】 VMD-3d-pose-baseline-multi 3D->VMD程序的参数\n",
        " - 更改 **\"参数设定\"**中【V】的参数\n",
        " - 执行 **\"参数设定\"** 单元格\n",
        " - 执行 **\"C) 自动跟踪重新运行（3D→VMD）\"** 生成VMD数据\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "EQUcMLsf3TcN"
      },
      "source": [
        "### A）自动跟踪重新执行（深度估计）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "yM2PNPpd3uai"
      },
      "outputs": [],
      "source": [
        "#@markdown 执行参数设定单元格后，执行此单元格。\n",
        "#@markdown 在人员索引重新排序过程之后执行深度估计处理\n",
        "\n",
        "\n",
        "import time\n",
        "import datetime\n",
        "import cv2\n",
        "import os\n",
        "from google.colab import drive\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# 過去深度結果\n",
        "past_depth_dir_path =  output_json + \"_\" + now_str + \"_depth\"\n",
        "\n",
        "# 処理日時\n",
        "now_str = \"{0:%Y%m%d_%H%M%S}\".format(datetime.datetime.now())\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "drive_base_dir = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "output_json = \"/content/output/json\"\n",
        "output_openpose_avi = \"/content/output/openpose.avi\"\n",
        "\n",
        "# 出力用Googleドライブフォルダ\n",
        "drive_dir_path = drive_base_dir + \"/\" + now_str \n",
        "! mkdir -p \"$drive_dir_path\"\n",
        "\n",
        "! echo ------------------------------------------\n",
        "! echo FCRN-DepthPrediction-vmd\n",
        "! echo ------------------------------------------\n",
        "    \n",
        "# 深度結果コピー\n",
        "depth_dir_path =  output_json + \"_\" + now_str + \"_depth\"\n",
        "\n",
        "\n",
        "! cd mannequinchallenge-vmd && python predict_video.py --video_path \"$input_video\" --json_path \"$output_json\" --past_depth_path \"$past_depth_dir_path\" --interval 20 --reverse_specific \"$reverse_specific\" --order_specific \"$order_specific\" --verbose 1 --now \"$now_str\" --avi_output \"yes\"  --number_people_max \"$number_people_max\" --end_frame_no \"$end_frame_no\" --input single_view --batchSize 1\n",
        "\n",
        "if os.path.exists( depth_dir_path + \"/error.txt\"):\n",
        "    \n",
        "    # エラー発生\n",
        "    ! cp \"$depth_dir_path\"/error.txt \"$drive_dir_path\"\n",
        "\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "    ! echo \"■■出现错误，中断执行。\"\n",
        "    ! echo \"■■\"\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "    ! echo \"$drive_dir_path\" \"の error.txt の中身を確認してください。\"\n",
        "\n",
        "else:\n",
        "    \n",
        "    ! cp \"$depth_dir_path\"/*.avi \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/message.log \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/reverse_specific.txt \"$drive_dir_path\"\n",
        "    ! cp \"$depth_dir_path\"/order_specific.txt \"$drive_dir_path\"\n",
        "\n",
        "    # Googleドライブ再マウント\n",
        "    drive.mount('/gdrive')\n",
        "\n",
        "    elapsed_time = (time.time() - start_time) / 60\n",
        "\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "    ! echo \"■■【深度推定】处理完成\"\n",
        "    ! echo \"■■\"\n",
        "    ! echo \"■■处理時間：\" \"$elapsed_time\" \"分\"\n",
        "    ! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "    ! echo \"\"\n",
        "    ! echo \"MMD自动跟踪执行结果\"\n",
        "\n",
        "    ! echo \"$drive_dir_path\"    \n",
        "    ! ls -l \"$drive_dir_path\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Mm0tv4ykzsLn"
      },
      "source": [
        "### B) 自动跟踪重新运行（2D→3D）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "MAGaxOsH0Daw"
      },
      "outputs": [],
      "source": [
        "#@markdown 执行参数设定单元格后，执行此单元格\n",
        "#@markdown 重新生成3D数据\n",
        "\n",
        "\n",
        "import time\n",
        "import datetime\n",
        "import cv2\n",
        "import os\n",
        "from google.colab import drive\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "drive_base_dir = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "output_json = \"/content/output/json\"\n",
        "output_openpose_avi = \"/content/output/openpose.avi\"\n",
        "\n",
        "# 出力用Googleドライブフォルダ\n",
        "drive_dir_path = drive_base_dir + \"/\" + now_str \n",
        "! mkdir -p \"$drive_dir_path\"\n",
        "\n",
        "for i in range(1, number_people_max+1):\n",
        "    ! echo ------------------------------------------\n",
        "    ! echo 3d-pose-baseline-vmd [\"$i\"]\n",
        "    ! echo ------------------------------------------\n",
        "\n",
        "    target_name = \"_\" + now_str + \"_idx0\" + str(i)\n",
        "    target_dir = output_json + target_name\n",
        "\n",
        "    !cd ./3d-pose-baseline-vmd && python src/openpose_3dpose_sandbox_vmd.py --camera_frame --residual --batch_norm --dropout 0.5 --max_norm --evaluateActionWise --use_sh --epochs 200 --load 4874200 --gif_fps 30 --verbose 1 --openpose \"$target_dir\" --person_idx 1    \n",
        "\n",
        "    # INDEX別結果コピー\n",
        "    idx_dir_path = drive_dir_path + \"/idx0\" + str(i)\n",
        "    ! mkdir -p \"$idx_dir_path\"\n",
        "    ! cp \"$target_dir\"/pos.txt \"$idx_dir_path\"\n",
        "\n",
        "# Googleドライブ再マウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "elapsed_time = (time.time() - start_time) / 60\n",
        "\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "! echo \"■■【2D→3D】处理完成\n",
        "! echo \"■■\"\n",
        "! echo \"■■处理时间：\" \"$elapsed_time\" \"分\"\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "! echo \"\"\n",
        "! echo \"MMD自动跟踪执行结果\"\n",
        "\n",
        "! echo \"$drive_dir_path\"    \n",
        "! ls -l \"$drive_dir_path\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "tQh07oDkDCz0"
      },
      "source": [
        "### C) 自动跟踪重新运行（3D→VMD）"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "FULbLzXfyJ2W"
      },
      "outputs": [],
      "source": [
        "#@markdown 执行参数设定单元格后，执行此单元格\n",
        "#@markdown 根据参数，重新生成VMD数据\n",
        "\n",
        "import time\n",
        "import datetime\n",
        "import cv2\n",
        "import shutil\n",
        "import glob\n",
        "from google.colab import drive\n",
        "\n",
        "start_time = time.time()\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "drive_base_dir = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "output_json = \"/content/output/json\"\n",
        "output_openpose_avi = \"/content/output/openpose.avi\"\n",
        "\n",
        "# 出力用Googleドライブフォルダ\n",
        "drive_dir_path = drive_base_dir + \"/\" + now_str \n",
        "! mkdir -p \"$drive_dir_path\"\n",
        "\n",
        "\n",
        "for i in range(1, number_people_max+1):\n",
        "\n",
        "    ! echo ------------------------------------------\n",
        "    ! echo VMD-3d-pose-baseline-multi [\"$i\"]\n",
        "    ! echo ------------------------------------------\n",
        "    \n",
        "    target_name = \"_\" + now_str + \"_idx0\" + str(i)\n",
        "    target_dir = output_json + target_name\n",
        "\n",
        "    ! cd ./VMD-3d-pose-baseline-multi && python main.py -v 2 -t \"$target_dir\" -b \"$born_model_csv\" -c 30 -z \"$center_z_scale\" -s \"$smooth_times\" -p \"$threshold_pos\" -r \"$threshold_rot\" -k \"$is_ik\" -e \"$heel_position\" -d \"$depth_smooth_times\"\n",
        "\n",
        "    # INDEX別結果コピー\n",
        "    idx_dir_path = drive_dir_path + \"/idx0\" + str(i)\n",
        "    ! mkdir -p \"$idx_dir_path\"\n",
        "    ! cp \"$target_dir\"/*.vmd \"$idx_dir_path\"\n",
        "    # 日本語対策でpythonコピー\n",
        "    for f in glob.glob(target_dir +\"/*.vmd\"):\n",
        "        shutil.copy(f, idx_dir_path)\n",
        "\n",
        "# Googleドライブ再マウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "elapsed_time = (time.time() - start_time) / 60\n",
        "\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "! echo \"■■【3D→VMD】处理完成\"\n",
        "! echo \"■■\"\n",
        "! echo \"■■处理时间：\" \"$elapsed_time\" \"分\"\n",
        "! echo \"■■■■■■■■■■■■■■■■■■■■■■■■\"\n",
        "\n",
        "! echo \"\"\n",
        "! echo \"MMD自动跟踪执行结果\"\n",
        "\n",
        "! echo \"$drive_dir_path\"\n",
        "! ls -l \"$drive_dir_path\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "UzsQ7HrjphVF"
      },
      "source": [
        "# 如果发生错误"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_JiZw2Ott1Rr"
      },
      "source": [
        "如果发生错误且尚未生成vmd文件，请从顶部开始逐个执行此部分，并检查右上角的会话状态是否处于**连接**状态。\n",
        "\n",
        "如果这不起作用，请按照**准备篇**的错误处理步骤共享笔记本的副本。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NCckdf_gp0F2"
      },
      "source": [
        "## 1.如果文件未添加到Google云端硬盘"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wWIauqkWq15H"
      },
      "source": [
        "如果error.txt和vmd均未输出，请检查单元格输出。\n",
        "\n",
        "如果单元格输出存在文件名的最终列表，则输出本身将成功。\n",
        "\n",
        "但是，假如已经确认即使与Google云端硬盘的链接已完成，生成的数据也无法在谷歌云端硬盘查询到的情况。\n",
        "\n",
        "详细信息正在调查中，请暂时将生成的数据下载到本地。\n",
        "\n",
        "1. 单击 **目录**旁边的\"**文件**\"列\n",
        "2. 点击 \"**刷新**\" 按钮\n",
        "3. 打开 \"**output** ＞ **json**\"\n",
        "4. xxx_depth ＞ output_XXX.avi …　2D实际跟踪AVI(MMD)\n",
        "5. xxx_idxXX ＞ output_XXX.vmd　…　VMD动作数据(MMD)\n",
        "6. xxx_idxXX ＞ pos.txt　…　3D 关键点信息数据(Unity)\n",
        "7. 如果跟踪多个人，则有多个idx。\n",
        "\n",
        "![クラウドデータ](https://drive.google.com/uc?export=view&id=1fArRyRdfs1kBLaLTpdkdJ-MYwHNe-UUq)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ASco7yEigfFn"
      },
      "source": [
        "## 4.如果要重新开始"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "X0leFRGtgoP7"
      },
      "source": [
        "如果您还没有准备好或者环境已无法恢复，并且想重做所有事情，请重置运行时。\n",
        "\n",
        "标题 >  代码执行程序 > 运行时恢复出厂设置\n",
        "\n",
        "![リセット](https://drive.google.com/uc?export=view&id=1HNOEDju8R5pTZseJ0FCOnFVDRI9ruuLC)\n",
        "\n",
        "将出现一个确认对话框，单击“确定”继续。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "iOLiarVDw_TY"
      },
      "source": [
        "# 提示"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "oiVbKfOVxByH"
      },
      "source": [
        "除上述内容外，我还将写一些我认为对您有所帮助的内容。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RS2M0r4vFH5Z"
      },
      "source": [
        "## 推荐工作流程"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ESNt5KETxMqv"
      },
      "source": [
        "为节省时间，请不要每次都重新执行部署环境或全部执行自动跟踪，我按以下顺序工作。\n",
        "\n",
        "1.  通过在**\"【O】视频中的最大人数\"**中输入要从跟踪源视频中跟踪的人数,并执行此单元格\n",
        "\n",
        "2.  执行**\"自动跟踪执行（所有执行）\"**的单元格\n",
        "\n",
        "3. 去谷歌硬盘下载，生成好的动作数据，放入MMD中测试，如果满意，则本次结束，不满意继续执行下面的优化步奏\n",
        "\n",
        "4. 如果多人时，无法识别出该人的痕迹，则查看message.log，并在**【F】输出颜色**中指定顺序。\n",
        "\n",
        "5. 执行**“参数设置” **的单元格\n",
        "\n",
        "6. 执行 **\"A）自动跟踪重新运行（深度估计）\"** 单元格\n",
        "7. 执行 **\"B) 自动跟踪重新运行（2D→3D）\"** 单元格\n",
        "8. 执行 **\"C) 自动跟踪重新运行（3D→VMD）\"** 单元格\n",
        "\n",
        "9. 重复4到8，直到对指定的顺序满意为止\n",
        "\n",
        "10. 如果发生意外旋转，请在查看message.log,并在 **【F】反转数据表** 中指定并设置相应帧的正确反转状态。\n",
        "\n",
        "  如果有人物在跟踪中更改了顺序，则反向指定的人物的 **INDEX** 也将更改，因此最好在指定顺序完成才执行反转校正。\n",
        "11. 执行**\"参数设置\" **的单元格\n",
        "\n",
        "12. 执行 **\" A）自动跟踪重新运行（深度估计）\"单元格**\n",
        "13. 执行 **\"B) 自动跟踪重新运行（2D→3D）\"** 单元格\n",
        "14. 执行 **\"C) 自动跟踪重新运行（3D→VMD）\"** 单元格\n",
        "\n",
        "15. 重复11到14，直到满意为止\n",
        "\n",
        "16. 完成替换或旋转的说明后，开始校正动作细节。\n",
        "\n",
        "17. 调整“参数设置”的[V]值\n",
        "\n",
        "18. 执行**“参数设置” **的单元格\n",
        "\n",
        "19. 执行**“ C）自动跟踪重新运行（3D→VMD）” **的单元格。即使在多人跟踪的情况下，也可以同时进行人数输出。\n",
        "\n",
        "重复17-19，直到满意为止\n",
        "\n",
        "祝好运！"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "AhrZcOhMxO2f"
      },
      "source": [
        "## 易于跟踪的视频"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "assKjr_XxRKL"
      },
      "source": [
        " - 固定相机的视频(非固定相机，就不用要测试的，很惨的，哇哈哈哈)\n",
        " - 关节清晰可见\n",
        "    - 长裙、和服、汉服等关节难以看清的人体不擅长分析\n",
        "    - 当背景是类似于人的颜色，阴影很暗等时，通常会导致识别错误，尽量找背景与人体差异明显的视频。\n",
        "    - 如果看到手腕和脚踝，则精度会更好\n",
        "    - 如果难以用黑色裤子等区分左右，则准确性会下降。\n",
        " - 第一帧正面\n",
        "   - 当第一帧看着背面或侧面时，无法清楚地获得开始的数据（一旦看到正面，通常会自动对其进行修复），但是刚开始未看到正面的那几帧会出现关节变形)\n",
        " - 可以在第一帧中识别出全身的关节\n",
        "   - 如果您藏在某个地方，准确性会下降\n",
        " - 头高脚低\n",
        "   - 如果由于倒立或踢脚而使腿抬高，则脚会被误认为是手。（特别是当脚的关节位于颈部上方时）\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "umaYNzP34XKU"
      },
      "source": [
        "## 待完成的课题"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SpqJj6mY4ahj"
      },
      "source": [
        " - 未跟踪的关节\n",
        "    - 头\n",
        "    - 手腕\n",
        "    - 指\n",
        " - 人体深度（Z轴）(尽管已经有深度估计，但仍然存在很多不足之处)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "EtUG-Qsp1QMp"
      },
      "source": [
        "# License许可"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6OkAoId61RZ7"
      },
      "source": [
        "发布/分发MMD自动跟踪的结果之前，请确保检查许可证。同样适用于Unity。\n",
        "\n",
        "如果您能描述一下许可证，我将不胜感激。\n",
        "\n",
        "[MMD Auto Trace License(Japanese)](https://ch.nicovideo.jp/miu200521358/blomaga/ar1686913)\n",
        "\n",
        "作者：miu\n",
        "\n",
        "Twitter：@miu200521358\n",
        "\n",
        "\n",
        "中文翻译---妖风瑟瑟 \n",
        "\n",
        "Twitter @wolelegedaca\n",
        "\n",
        "如果有任何建议，请B站或者twitter上联系我\n"
      ]
    }
  ]
}