example.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle, os\n",
    "import numpy as np\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import pretty_midi\n",
    "import json"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.style.use('ggplot')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# track separation\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## the track_separate.py can find melody, bass, chord, accompiment and drum tracks. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The track separation model is a random forest trained each for melody, bass, chord and drum.\n",
    "\n",
    "The accompaniment track is found based on the duration of the track, and it selects the longest duration in the left tracks.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the parameters includes:\n",
    "-f input file\n",
    "-i input folder (will search all the midi file in that folder)\n",
    "The user need to specify one of the above two parameters \n",
    "-o output folder\n",
    "-t required tracks to be found. It will always try to find melody, bass, chord, accompaniment and drum tracks, and -t specifies the mandatory tracks. e.g. \"melody bass chord\" means melody, bass and chord tracks are mandatory for the output. If it cannot find those tracks that file will be omitted. But if the accomaniment and drum are not found and melody, bass, chord are found that file will still be output. The default is \"melody\", which means only the melody is mandatory for the output.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### example of track_separate.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "read in the original file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "original_file = 'example/input/55269aebbc2e7784ca2d1d4472141889.mid'\n",
    "original_pm = pretty_midi.PrettyMIDI(original_file)\n",
    "print(f'the original file has {len(original_pm.instruments)} tracks')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 track_separate.py -h"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 track_separate.py -f example/input/55269aebbc2e7784ca2d1d4472141889.mid -o example/output/ -t \"melody bass chord\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output file is in the example/output/ folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "output_file = 'example/output/55269aebbc2e7784ca2d1d4472141889.mid'\n",
    "output_pm = pretty_midi.PrettyMIDI(output_file)\n",
    "print(f'the output file has {len(output_pm.instruments)} tracks')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the program_result.json has record of the program number of each track"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join('example/output','program_result.json'), 'r') as fp:\n",
    "    programs = json.load(fp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for key in programs.keys():\n",
    "    for name,value in programs[key].items():\n",
    "        print(f'{name} program number is {value}')\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# tension calculation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The tension calculation is based on the spiral array theory (https://dspace.mit.edu/handle/1721.1/9139,\n",
    "https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/11798/Herremans%20Tension%20ribbons%20Quantifying%202016%20Accepted.pdf?sequence=1) \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It maps the pitch to a 3-dim position in spiral array space, and keeps property in that space such as dist(perfect fifth) has shortest space in all the interval pairs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tension_calculation import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "all the interval distance to C note in a scale"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "the vertical step is a parameter to set the note position in the 3d space\n",
    "the original paper uses math.sqrt(2/15) which makes the interval distance of a major third equals a perfect fifth distance\n",
    "Any value from math.sqrt(2/15) to math.sqrt(0.2) can work according to the original paper\n",
    "Here 0.4 is used to make the perfect fifth has the shortest distance\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## some properties of the distance in the spiral array space"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "note_to_note_diff = note_to_note_pos([0,1,2,3,4,5,6,7,8,9,10,11],pitch_index_to_position(note_index_to_pitch_index[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "use ['C','D-','D','E-','E','F','F#','G','A-','A','B-','B'] to map the note to pitch names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pitch_names = ['C','D-','D','E-','E','F','F#','G','A-','A','B-','B']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for num, pitch_name in enumerate(pitch_names):\n",
    "    print(f'the distance from {pitch_name} to C is {note_to_note_diff[num]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "note_to_key_diff = note_to_key_pos([0,1,2,3,4,5,6,7,8,9,10,11],major_key_position(0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "for num, pitch_name in enumerate(pitch_names):\n",
    "    print(f'the distance from {pitch_name} to key pos major C  is {note_to_key_diff[num]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "note_to_key_diff = note_to_key_pos([0,1,2,3,4,5,6,7,8,9,10,11],minor_key_position(3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for num, pitch_name in enumerate(pitch_names):\n",
    "    print(f'the distance from {pitch_name} to key pos minor a  is {note_to_key_diff[num]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chord_to_key_diff = chord_to_key_pos([0,1,2,3,4,5,6,7,8,9,10,11],major_key_position(0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chord_names = ['CM','D-M','DM','E-M','EM','FM','F#M','G-M','A-M','AM','B-M','BM',\n",
    "              'Cm','D-m','Dm','E-m','Em','Fm','F#m','G-m','A-m','Am','B-m','Bm']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for num, pitch_name in enumerate(chord_names):\n",
    "    print(f'the distance from chord pos {chord_names[num]} to  major C key pos is {chord_to_key_diff[num]}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "key_to_key_diff = key_to_key_pos([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], major_key_position(0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for num, pitch_name in enumerate(chord_names):\n",
    "    print(f'the distance from key pos {chord_names[num]} to  major C key pos is {key_to_key_diff[num]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## calculate the tension by tension_calculation.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def draw_tension(time,values):\n",
    "    fig = plt.figure(figsize=(20, 10))\n",
    "    plt.rcParams['xtick.labelsize'] = 14\n",
    "    plt.plot(time,values,marker='o')\n",
    "    plt.tight_layout()\n",
    "    plt.show()\n",
    "   \n",
    "  \n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "three tension measures are calculated including tensile strain, cloud diameter and centroid difference (https://qmro.qmul.ac.uk/xmlui/bitstream/handle/123456789/11798/Herremans%20Tension%20ribbons%20Quantifying%202016%20Accepted.pdf?sequence=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To calculate those measures, the key of the song needs to be detected first. It finds the key by mappping the notes pos in all the keys and find the shortest distance in a key"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It also tries to find one key change of the song which is common in pop music, but the classical music has more key change"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 tension_calculation.py -h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this example, it will try to detect key change, use window size -1 (a downbeat i.e. bar window)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "!python3 tension_calculation.py -f example/output/55269aebbc2e7784ca2d1d4472141889.mid -o example/output -k True -w 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this song should be in a minor, now set the key to a minor by -n \"a minor\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python3 tension_calculation.py -f example/output/55269aebbc2e7784ca2d1d4472141889.mid -o example/output -k True -w 1 -n \"a minor\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cloud diameter result for bar/time x axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "diameter = pickle.load(open('example/output/55269aebbc2e7784ca2d1d4472141889.diameter','rb'))\n",
    "times = pickle.load(open('example/output/55269aebbc2e7784ca2d1d4472141889.time','rb'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'the file has {len(diameter)} bar')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'the max diameter is located at bar {np.argmax(diameter) + 1} ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x label is bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "draw_tension(np.arange(diameter.shape[0]) + 1,diameter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "xlabel is time (s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "draw_tension(times[:len(diameter)],diameter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## tensile strain output for bar/time x axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tensile = pickle.load(open('example/output/55269aebbc2e7784ca2d1d4472141889.tensile','rb'))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'the max tensile strain is located at bar {np.argmax(tensile) + 1} ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x label is bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "draw_tension(np.arange(tensile.shape[0]) + 1,tensile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "xlabel is time (s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "draw_tension(times[:len(tensile)],tensile)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## centroid difference output for bar/time x axis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "centroid_diff = pickle.load(open('example/output/55269aebbc2e7784ca2d1d4472141889.centroid_diff','rb'))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'the max centroid difference is located at bar {np.argmax(centroid_diff) + 1} ')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "x label is bar"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "draw_tension(np.arange(centroid_diff.shape[0])+1,centroid_diff)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "xlabel is time (s)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "draw_tension(times[:len(centroid_diff)],centroid_diff)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(os.path.join('example/output','files_result.json'), 'r') as fp:\n",
    "    keys = json.load(fp)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for key in keys.keys():\n",
    "    print(f'song name is {key}')\n",
    "    print(f'song key is {keys[key][0]}')\n",
    "    print(f'song key change time {keys[key][1]}')\n",
    "    print(f'song key change bar {keys[key][2]}')\n",
    "    print(f'song key change name {keys[key][3]}')\n",
    "    \n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-1 change time(or bar) means key no change"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "midi-miner",
   "language": "python",
   "name": "midi-miner"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}