ALMA_7B_Ja_gguf_Free_Colab_sample.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/webbigdata-jp/python_sample/blob/main/ALMA_7B_Ja_gguf_Free_Colab_sample.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9tCLTsfT_qgA"
      },
      "source": [
        "更新)[Version 2](https://github.com/webbigdata-jp/python_sample/blob/main/ALMA_7B_Ja_V2_gguf_Free_Colab_sample.ipynb)が出ています。\n",
        "Update)[Version 2](https://github.com/webbigdata-jp/python_sample/blob/main/ALMA_7B_Ja_V2_gguf_Free_Colab_sample.ipynb) is here.\n",
        "\n",
        "ALMA-7B-Jaは日英・英日に特化した機械翻訳モデルです。  このColabはmmngaさんが作成してくれた[llama.cpp(gguf)版](https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf)の評価コードです。  \n",
        "\n",
        "llama.cpp(gguf)版を使うと、GPUがないMacやColabでも動かす事ができます。\n",
        "\n",
        "ALMA-7B-Ja is a machine translation model specialized for Japanese-English and English-Japanese.  This Colab is an evaluation code for [llama.cpp(gguf) version](https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf) created by mmnga.  \n",
        "\n",
        "With the llama.cpp(gguf) version, you can run it on a Mac or Colab without a GPU."
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "llama.cpp版には、12のモデルがあります。サイズを小さくすれば、少ないメモリで高速に動作させることができますが、モデルの性能も低下します。4ビット(q4_0)くらいがバランスが良いと言われていますが、本サンプルコードでは特定の文章を全モデルで翻訳し、どのモデルが貴方の作業に適切かを確認できるようにしたものです。\n",
        "\n",
        "In llama.cpp version, there are 12 models. It is said that about 4 bits (q4_0) is a good balance, but this sample code translates a particular sentence with all the models so that you can see which model is appropriate for your work. This sample code translates a particular sentence across all models so that you can see which model is appropriate for your work.\n",
        "\n",
        "\n",
        "1. webbigdata-ALMA-7B-Ja-q2_K.gguf 2.83 GB\n",
        "2. webbigdata-ALMA-7B-Ja-q3_K_S.gguf 2.95 GB\n",
        "3. webbigdata-ALMA-7B-Ja-q3_K_M.gguf 3.3 GB\n",
        "4. webbigdata-ALMA-7B-Ja-q3_K_L.gguf 3.6 GB\n",
        "5. webbigdata-ALMA-7B-Ja-q4_0.gguf 3.83 GB\n",
        "6. webbigdata-ALMA-7B-Ja-q4_K_S.gguf 3.86 GB\n",
        "7. webbigdata-ALMA-7B-Ja-q4_K_M.gguf 4.08 GB\n",
        "8. webbigdata-ALMA-7B-Ja-q5_0.gguf 4.65 GB\n",
        "9. webbigdata-ALMA-7B-Ja-q5_K_S.gguf 4.65 GB\n",
        "10. webbigdata-ALMA-7B-Ja-q5_K_M.gguf 4.78 GB\n",
        "11. webbigdata-ALMA-7B-Ja-q6_K.gguf 5.53 GB\n",
        "12. webbigdata-ALMA-7B-Ja-q8_0.gguf 7.16 GB\n",
        "\n",
        "上段メニューより「ランタイム」→「全てのセルを実行」で本NoteBookを実行できます。  \n",
        "You can run this NoteBook by selecting \"Runtime\" > \"Run All Cells\" from the top menu.  \n"
      ],
      "metadata": {
        "id": "EO7RPRM9vaKy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# 評価用日本語文 Japanese text for evaluation\n",
        "TRANSLATE_TEMPLATE_Ja=\"Translate this from Japanese to English:\\nJapanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\\nEnglish:\"\n",
        "\n",
        "# 評価用英語文 English text for evaluation\n",
        "TRANSLATE_TEMPLATE_En=\"Translate this from English to Japanese:\\nEnglish:The book named A Tale of Two Cities was written by Charles. Mary read the book.\\nJapanese:\"\n"
      ],
      "metadata": {
        "id": "c94toQkYzxpk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ygJ-cNOc_qTi",
        "outputId": "c7ea5d07-1ca7-4d1a-b1b1-ce7f2ed6b5e4"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "I llama.cpp build info: \n",
            "I UNAME_S:   Linux\n",
            "I UNAME_P:   x86_64\n",
            "I UNAME_M:   x86_64\n",
            "I CFLAGS:    -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native \n",
            "I CXXFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native \n",
            "I NVCCFLAGS:  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread    -Wno-pedantic -Xcompiler \"-Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native \"\n",
            "I LDFLAGS:    \n",
            "I CC:        cc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0\n",
            "I CXX:       g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0\n",
            "\n",
            "cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native    -c ggml.c -o ggml.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c llama.cpp -o llama.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c common/console.cpp -o console.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c common/grammar-parser.cpp -o grammar-parser.o\n",
            "cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native  -c k_quants.c -o k_quants.o\n",
            "cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native    -c ggml-alloc.c -o ggml-alloc.o\n",
            "cc  -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native    -c ggml-backend.c -o ggml-backend.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c common/train.cpp -o train.o\n",
            "cc -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c11   -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wshadow -Wstrict-prototypes -Wpointer-arith -Wmissing-prototypes -Werror=implicit-int -Werror=implicit-function-declaration -Wdouble-promotion -pthread -march=native -mtune=native  -c tests/test-c.c -o tests/test-c.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c common/common.cpp -o common.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -c common/sampling.cpp -o sampling.o\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  pocs/vdot/vdot.cpp ggml.o k_quants.o ggml-alloc.o ggml-backend.o -o vdot  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  pocs/vdot/q8dot.cpp ggml.o k_quants.o ggml-alloc.o ggml-backend.o -o q8dot  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/benchmark/benchmark-matmult.cpp ggml.o k_quants.o ggml-alloc.o ggml-backend.o -o benchmark-matmult  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/main/main.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-alloc.o ggml-backend.o -o main  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/quantize/quantize.cpp ggml.o llama.o k_quants.o ggml-alloc.o ggml-backend.o -o quantize  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/quantize-stats/quantize-stats.cpp ggml.o llama.o k_quants.o ggml-alloc.o ggml-backend.o -o quantize-stats  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/perplexity/perplexity.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o perplexity  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/embedding/embedding.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o embedding  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/train-text-from-scratch/train-text-from-scratch.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-alloc.o ggml-backend.o -o train-text-from-scratch  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/convert-llama2c-to-ggml/convert-llama2c-to-ggml.cpp ggml.o llama.o k_quants.o ggml-alloc.o ggml-backend.o -o convert-llama2c-to-ggml  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/simple/simple.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o simple  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/batched/batched.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o batched  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/batched-bench/batched-bench.cpp ggml.o llama.o common.o k_quants.o ggml-alloc.o ggml-backend.o -o batched-bench  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/save-load-state/save-load-state.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o save-load-state  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  -Iexamples/server examples/server/server.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-alloc.o ggml-backend.o -o server   \n",
            "g++ --shared -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/embd-input/embd-input-lib.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o libembdinput.so  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/gguf/gguf.cpp ggml.o llama.o k_quants.o ggml-alloc.o ggml-backend.o -o gguf  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/llama-bench/llama-bench.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o llama-bench  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/llava/llava.cpp examples/llava/clip.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o llava   -Wno-cast-qual\n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/baby-llama/baby-llama.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-alloc.o ggml-backend.o -o baby-llama  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/beam-search/beam-search.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o beam-search  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/speculative/speculative.cpp ggml.o llama.o common.o sampling.o grammar-parser.o k_quants.o ggml-alloc.o ggml-backend.o -o speculative  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/infill/infill.cpp ggml.o llama.o common.o sampling.o console.o grammar-parser.o k_quants.o ggml-alloc.o ggml-backend.o -o infill  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/parallel/parallel.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o parallel  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/finetune/finetune.cpp ggml.o llama.o common.o sampling.o train.o k_quants.o ggml-alloc.o ggml-backend.o -o finetune  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/export-lora/export-lora.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o export-lora  \n",
            "g++ -I. -Icommon -D_XOPEN_SOURCE=600 -D_GNU_SOURCE -DNDEBUG -DGGML_USE_K_QUANTS  -std=c++11 -fPIC -O3 -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wmissing-declarations -Wmissing-noreturn -pthread  -Wno-array-bounds -Wno-format-truncation -Wextra-semi -march=native -mtune=native  examples/embd-input/embd-input-test.cpp ggml.o llama.o common.o sampling.o k_quants.o ggml-alloc.o ggml-backend.o -o embd-input-test   -L. -lembdinput\n",
            "\n",
            "====  Run ./main -h for help.  ====\n",
            "\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Cloning into 'llama.cpp'...\n"
          ]
        }
      ],
      "source": [
        "%%bash\n",
        "# setup llama.cpp\n",
        "git clone https://github.com/ggerganov/llama.cpp.git\n",
        "cd llama.cpp\n",
        "make -j"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q2_K.gguf"
      ],
      "metadata": {
        "id": "0stV04t7MQm0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q2_K.gguf 2>>q2_k.log"
      ],
      "metadata": {
        "id": "o6LdO72RMQyS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q2_K.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q2_k.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ayZUdGQXMaZm",
        "outputId": "fc2fac04-70be-42f8-cb5c-c9b6e294c815"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundamon is an active girl living in Tohoku. She says, \"What's your name?\" Zundamon answers, \"My name is Zundamon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q2_K.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q2_k.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wZva70fHMdO7",
        "outputId": "91303057-1aba-4851-80cc-159e111c21ea"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:Ｃ・ダレカの書いた「二人の都市」は、マリーが読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q3_K_S.gguf"
      ],
      "metadata": {
        "id": "yAS9RSKmNCLz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q3_K_S.gguf 2>>q3_k_s.log"
      ],
      "metadata": {
        "id": "JYH3O-yCNCUq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q3_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xhyBRwc7O2Rp",
        "outputId": "72fcac3e-3fcf-492f-c560-8e8c4c90ad8c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundamon is a cheerful and pretty girl living in the Tohoku region. You ask her name. Zundamon says, \"My name is Zundamon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\"  2>>q3_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DFpdiTaFO2Yz",
        "outputId": "96cf2f3b-5ec1-4467-fe52-861904339a2c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:２人の市民と言う題名の本が、チャールズによって書かれました。マリはその本を読んだです。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q3_K_M.gguf"
      ],
      "metadata": {
        "id": "5XSqk8pMPITi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q3_K_M.gguf 2>>q3_k_m.log"
      ],
      "metadata": {
        "id": "_xPB3S-TPHde"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q3_k_m.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ls1oWCQwPJBf",
        "outputId": "94a0bfe2-caa7-44e2-b76e-6b3e29fb808b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda Mom is a vivacious and cute girl who lives in the north. You ask, \"May I know your name?\" She answers, \"My name is Zunda-mom.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\"  2>>q3_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fbs9N-D1PJJg",
        "outputId": "fd326f47-3761-4772-9cbf-d699d8b1bc44"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:Ａ・テールオブツウシティーズと言う本は、シャーレスによって書かれた。メアリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q3_K_L.gguf"
      ],
      "metadata": {
        "id": "G6zprD6TP88-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q3_K_L.gguf 2>>q3_k_l.log"
      ],
      "metadata": {
        "id": "q2zTvOzLP9JN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_L.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q3_k_l.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7RKJPdneP9SF",
        "outputId": "aaa811d5-b4d1-480d-df8f-afeab24cad92"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda-mon is a pretty girl who lives in the Tohoku region. She asks, \"What's your name?\" and Zunda-mon replies, \"My name is Zunda-mon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q3_K_L.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q3_k_l.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "INk_APk6P9aK",
        "outputId": "8ae4b6e0-a375-44de-c577-13e696eb09b0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:その名前を『タイル・オブ・ツーシティーズ』という本は、チャールズによって書かれた。マリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q4_0.gguf"
      ],
      "metadata": {
        "id": "TGS1_SzM-AR9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q4_0.gguf 2>>q4_0.log"
      ],
      "metadata": {
        "id": "GdVpulvJQcxs"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q4_0.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ptsIUNsmQc6r",
        "outputId": "1eca97ae-0494-4497-9be7-77baf440ca35"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundamon is a vivacious and pretty girl who lives in Tohoku. \"What's your name?\" he said. \"My name is Zundamon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q4_0.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AKknOzX9QdCU",
        "outputId": "42c74758-0420-4440-ec72-7761961dbb32"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:「二都物語」という本は、チャールズが書いた。メアリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q4_K_S.gguf"
      ],
      "metadata": {
        "id": "gYHTs6nNQznM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q4_K_S.gguf 2>>q4_k_s.log"
      ],
      "metadata": {
        "id": "vNcTj-jJQz0L"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q4_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ks8-dK27Qz9z",
        "outputId": "85b561e5-f889-4932-bf62-ad1151ff99c1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundamon is an active, pretty girl who lives in the Tohoku region. You ask her, \"What's your name?\" Zundamon replies, \"My name is Zundamon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q4_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RJOhrmP-Q0GV",
        "outputId": "30cba8db-8e4a-4986-8e8f-4b664d22513a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:２つの都市物語という書物はチャールズが書きました。その本を読んだのはメアリーです。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q4_K_M.gguf\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "WfAnueSbRlfU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q4_K_M.gguf 2>>q4_k_m.log"
      ],
      "metadata": {
        "id": "QVmvi8WjQ0O7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q4_k_m.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vq25iGgiRmUY",
        "outputId": "d99ac691-8922-4f8f-bd99-9fde0552ab50"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundamon is a cheerful and active girl who lives in the Tohoku area.You ask her name, \"What is your name?\" Zundamon says, \"My name is Zundamon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q4_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q4_k_m.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2ETTkUTtRmih",
        "outputId": "8544bdb9-a98b-4801-9b70-94786da6663d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:２都市の物語という本はチャールズによって書かれた。マリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q5_0.gguf"
      ],
      "metadata": {
        "id": "pvc2AYCMSaB9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q5_0.gguf 2>>q5_0.log"
      ],
      "metadata": {
        "id": "aK0939KmSaQI"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q5_0.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0aWLZRHZSaZg",
        "outputId": "41172517-da31-454d-e62e-e5279b31e86b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda Man is an active, cheerful girl who lives in Tohoku. You ask, \"What's your name?\" Zunda says \"My name is Zunda.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q5_0.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "-hz77snmSaiK",
        "outputId": "4c8b824a-d66b-4254-f8c4-8d76a09a8ffd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:「二都市物語」という書物は、チャールズが書いた。その本を読んだのがメアリーである。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q5_K_S.gguf"
      ],
      "metadata": {
        "id": "yhcQ3wvL0UEp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q5_K_S.gguf 2>>q5_k_s.log"
      ],
      "metadata": {
        "id": "fJoB-iBuzpin"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q5_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gsA5ZS9ZToK4",
        "outputId": "73b03711-8138-49ce-9b6f-50e2656b5326"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zundammon is an active, attractive girl who lives in the Tohoku region. \"What's your name?\" she asks. \"My name is Zundammon,\" she answers."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_K_S.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q5_k_s.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eaRIRnrBToUB",
        "outputId": "2aa93774-f689-4a83-d065-23698a891237"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:その「二都市物語」という名前の本は、チャールズによって書かれた。メアリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q5_K_M.gguf\n"
      ],
      "metadata": {
        "id": "OBEXufWoTH9m"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q5_K_M.gguf 2>>q5_k_m.log"
      ],
      "metadata": {
        "id": "UZnXgRQvT5tE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q5_k_m.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M7Lw6vE8T539",
        "outputId": "ad47b2c4-2576-4f84-da39-6db134c2f915"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda Mon is a lively and pretty girl who lives in the Tohoku region. You say, \"What's your name?\" She says, \"My name is Zunda Mon.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q5_K_M.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q5_k_m.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2QU470w9T5_n",
        "outputId": "02d2a908-d43b-4734-bb00-2a6f77152ad6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:「二都市の物語」という名の本は、チャールズによって書かれた。メアリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q6_K.gguf"
      ],
      "metadata": {
        "id": "vb2wS-6Z0Zn3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q6_K.gguf 2>>q6_k.log"
      ],
      "metadata": {
        "id": "b7m_Yb3L0X0Q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q6_K.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q6_k.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3ll-dyS9Awu0",
        "outputId": "0fc08bf4-20ad-40d2-e7fc-5e981a607cf5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda-man is an active and pretty girl who lives in the Tohoku region. You \"What's your name?\" Zunda-man \"My name is Zunda-man.\""
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q6_K.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q6_k.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SEZNJP59UwSV",
        "outputId": "0a160395-598d-40a1-967b-d1c2437e88ab"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:Ａ・テイル・オブ・ツー・シティーズという本はチャールズによって書かれた。この本を読んだのがメアリーである。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## webbigdata-ALMA-7B-Ja-q8_0.gguf"
      ],
      "metadata": {
        "id": "XYHKl0661t44"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && wget https://huggingface.co/mmnga/webbigdata-ALMA-7B-Ja-gguf/resolve/main/webbigdata-ALMA-7B-Ja-q8_0.gguf 2>>q8_0.log"
      ],
      "metadata": {
        "id": "St1J9jXv1uRW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q8_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_Ja\" 2>>q8_0.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4UhPst-YA045",
        "outputId": "f70b57af-2d98-4e89-9d8e-88adeb70d7c9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from Japanese to English:\n",
            "Japanese: ずんだもんは東北に住む活発でかわいい女の子です。あなた「きみの名前は？」ずんだもん「ボクの名前はずんだもんなのだ。」\n",
            "English:Zunda Mon is an active, pretty girl who lives in the Tohoku district. \"What's your name?\" she asked. \"My name is Zunda Mon,\" said Zunda."
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "!cd llama.cpp && ./main -m 'webbigdata-ALMA-7B-Ja-q8_0.gguf' -n 128 -p \"$TRANSLATE_TEMPLATE_En\" 2>>q8_k.log"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OanbFIA5I3uj",
        "outputId": "d9c6a417-2e0e-463e-e13d-74bd78b04809"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            " Translate this from English to Japanese:\n",
            "English:The book named A Tale of Two Cities was written by Charles. Mary read the book.\n",
            "Japanese:Ａ・テイル・オブ・ツー・シティーズと言う本はチャールズによって書かれた。マリーはその本を読んだ。"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Original Model ALMA-7B.  (26.95GB)  \n",
        "https://huggingface.co/haoranxu/ALMA-7B  \n",
        "\n",
        "ALMA-7B-Ja is a machine translation model that uses ALMA's learning method to translate Japanese to English and English to Japanese.\n",
        "https://huggingface.co/webbigdata/ALMA-7B-Ja  \n",
        "\n",
        "This work was done by : [webbigdata](https://webbigdata.jp/)"
      ],
      "metadata": {
        "id": "pxPrR_5MI39q"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "mount_file_id": "1RgbYSzxcfWyYSgfiOoPy30SBrRS5k1Rl",
      "authorship_tag": "ABX9TyN3F9sZotwF+nIMuqnJinWU",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}