Deep Learning for Music (DL4M)

The role of this curated list is to gather scientific articles that use deep learning approaches applied to music. The list is currently under construction but feel free to contribute to the missing fields and to add other resources. The resources provided here come from my review of the state-of-the-art for my PhD Thesis for which an article is being written.

Table of contents

• DL4M summary
• DL4M details
• Code without articles
• Statistics and visualisations
• How To Contribute
• FAQ
• Abbreviations used
• Sources
• Contributors
• Other useful related lists

DL4M summary

Article	Code
Monoaural audio source separation using deep convolutional neural networks	GitHub
Transfer learning for music classification and regression tasks	GitHub
Convolutional recurrent neural networks for music classification	GitHub
An evaluation of convolutional neural networks for music classification using spectrograms	No
Basic Filters for Convolutional Neural Networks: Training or Design?	No
Music signal processing using vector product neural networks	No
Transforming musical signals through a genre classifying convolutional neural network	No
Deep convolutional neural networks for predominant instrument recognition in polyphonic music	No
CNN architectures for large-scale audio classification	No
Music emotion recognition via end-to-end multimodal neural networks	No
End-to-end musical key estimation using a convolutional neural network
Sample-level deep convolutional neural networks for music auto-tagging using raw waveforms
Multi-level and multi-scale feature aggregation using sample-level deep convolutional neural networks for music classification	GitHub
A deep multimodal approach for cold-start music recommendation	GitHub
Melody extraction and detection through LSTM-RNN with harmonic sum loss
Timbre analysis of music audio signals with convolutional neural networks	GitHub
Designing efficient architectures for modeling temporal features with convolutional neural networks	GitHub
Music feature maps with convolutional neural networks for music genre classification	No
Automatic drum transcription for polyphonic recordings using soft attention mechanisms and convolutional neural networks	GitHub
Convolutional methods for music analysis
Extending temporal feature integration for semantic audio analysis	No
Audio spectrogram representations for processing with convolutional neural networks	No
Attention and localization based on a deep convolutional recurrent model for weakly supervised audio tagging	GitHub
A study on LSTM networks for polyphonic music sequence modelling	Website
An efficient approach for segmentation, feature extraction and classification of audio signals	No
Towards playlist generation algorithms using rnns trained on within-track transitions	No
Automatic tagging using deep convolutional neural networks	No
Automatic chord estimation on seventhsbass chord vocabulary using deep neural network
Robust downbeat tracking using an ensemble of convolutional networks
Bayesian meter tracking on learned signal representations
Deep learning for music
Learning temporal features using a deep neural network and its application to music genre classification
On the potential of simple framewise approaches to piano transcription
Feature learning for chord recognition: the deep chroma extractor
A deep bidirectional long short-term memory based multi-scale approach for music dynamic emotion prediction
Event localization in music auto-tagging	GitHub
Deep convolutional networks on the pitch spiral for musical instrument recognition
Robust audio event recognition with 1-max pooling convolutional neural networks	No
Singing voice melody transcription using deep neural networks
Singing voice separation using deep neural networks and F0 estimation	Website
Learning to pinpoint singing voice from weakly labeled examples
Note onset detection in musical signals via neural-network-based multi-ODF fusion	No
Convolutional neural network for robust pitch determination
Deep convolutional neural networks and data augmentation for acoustic event detection	No
Unsupervised feature learning based on deep models for environmental audio tagging
Auralisation of deep convolutional neural networks: listening to learned features
Downbeat tracking with multiple features and deep neural networks
Music boundary detection using neural networks on spectrograms and self-similarity lag matrices
Classification of spatial audio location and content using convolutional neural networks
Deep learning, audio adversaries, and music content analysis
Singing voice detection with deep recurrent neural networks
Automatic instrument recognition in polyphonic music using convolutional neural networks
A software framework for musical data augmentation
A deep bag-of-features model for music auto-tagging
Music-noise segmentation in spectrotemporal domain using convolutional neural networks
Musical instrument sound classification with deep convolutional neural network using feature fusion approach
Environmental sound classification with convolutional neural networks
Exploring data augmentation for improved singing voice detection with neural networks	No
An end-to-end neural network for polyphonic music transcription
Deep karaoke: extracting vocals from musical mixtures using a convolutional deep neural network
Deep neural network based instrument extraction from music
A deep neural network for modeling music
"The munich LSTM-RNN approach to the mediaeval 2014 ""emotion in music"" task"
End-to-end learning for music audio	No
Deep learning for music genre classification	No
Recognition of acoustic events using deep neural networks
Deep image features in music information retrieval
From music audio to chord tablature: teaching deep convolutional networks to play guitar
Improved musical onset detection with convolutional neural networks	No
A hybrid recurrent neural network for music transcription
Boundary detection in music structure analysis using convolutional neural networks
Improving content-based and hybrid music recommendation using deep learning
A deep representation for invariance and music classification	No
Multiscale approaches to music audio feature learning
Deep content-based music recommendation
Musical onset detection with convolutional neural networks
Rethinking automatic chord recognition with convolutional neural networks
Moving beyond feature design: deep architectures and automatic feature learning in music informatics
Local-feature-map integration using convolutional neural networks for music genre classification.
Learning sparse feature representations for music annotation and retrieval	No
Unsupervised learning of local features for music classification.
Audio-based music classification with a pretrained convolutional network
Automatic musical pattern feature extraction using convolutional neural network
Audio musical genre classification using convolutional neural networks and pitch and tempo transformations
Unsupervised feature learning for audio classification using convolutional deep belief networks	No
A convolutional-kernel based approach for note onset detection in piano-solo audio signals
A supervised learning approach to musical style recognition

DL4M details

• dl4m.bib - the corresponding bibliography.
• dl4m.tsv - more details about each article:
  • First author name
  • Publication year
  • Article name
  • PDF link
  • Source code link
  • Source code reproducible (Yes/No) If Yes, indicates to what extent
  • Neural network architecture
  • Number of layers
  • Task
  • Dataset
  • Computation time
  • Hardware
  • Data augmentation technique if used
  • Additional notes

Code without articles

• Audio Classifier in Keras using Convolutional Neural Network

Statistics and visualisations

• 87 articles currently referenced.
• Number of articles per year:

How To Contribute

1. Adding/Updating information
   1. Fork the repo.
   2. Add one line per article in dl4m.tsv with every column correctly filled.
   3. Submit your pull request and that's it! (Note: the table in the ReadMe is automatically generated thanks to a python script.)
2. Visualisation
   • Please submit your idea for new visualisation of the data
   • I am looking for a way to display relations between articles automatically like a mindmap. Tell me if you know anything able to handle that.

FAQ

Why a tsv file instead of a regular csv file for storing the detailed information about the articles?

Because:

1. Some articles have a comma in their title and the .bib of each article contains commas.
2. GitHub currently only displays csv and tsv files. The built-in GitHub csv/tsv parser is handy because you can easily search in the file in your browser without downloading anything.

How are the articles sorted?

In dl4m.tsv, the articles are first sorted by decreasing year (to keep up with the latest news) and then alphabetically by author's family name.

Why are preprint from arXiv included in the list?

I want to have exhaustive research and the latest news on DL4M. However, one should take care of the information provided in the articles currently in review. If possible you should wait for the final accepted and peer-reviewed version before citing an arXiv paper. I regularly update the arXiv links to the corresponding published papers when available.

Abbreviations used

Abbreviation	Full name
ADT	Automatic Drum Transcription
BRNN	Bidirectional Recurrent Neural Network
CNN	Convolutional Neural Network
DNN	Deep Neural Network
LSTM	Long Short Term Memory
MER	Music Emotion Recognition
RNN	Recurrent Neural Network
SVD	Singing Voice Detection
SVS	Singing Voice Separation
VAD	Voice Activity Detection
VPNN	Vector Product Neural Network

Sources

Conferences and journals

• DLRS
• ICASSP
• ICML
• INTERSPEECH
• ISMIR
• IWDLM
• LVA/ICA
• MIREX
• RECSYS
• SMC

Aggregators

• arXiv
• PaperScape
• Scholar
• SciRate

Contributors

• Vincent Lostanlen
• Keunwoo Choi

Other useful related lists

• Awesome Python Scientific Audio - Python resources for Audio and Machine Learning
• Cheatsheets AI - Cheat Sheets for Keras, neural networks, scikit-learn,...
• Awesome Deep Learning - General deep learning resources
• ISMIR resources - Community maintained list
• Awesome Python - Audio section of Python resources
• Awesome Web Audio - WebAudio packages and resources
• Awesome Music - Music softwares
• Awesome Music Production - Music creation
• General lists
• The Asimov Institute - 6 deep learning tools for music generation