High-resolution guitar transcription via domain adaptation

High-resolution guitar transcription turns any recording of solo guitar into MIDI, without the need for special equipment or clean recording conditions.

Artist: Walter Rodriguez Jr. Video source
Your browser does not support the audio element. Your browser does not support the audio element. Select Original Audio

Play Video Select Transcribed Audio

Abstract

We present High-resolution guitar transcription, a method for training a guitar transcription model with excellent performance on real-world recordings. We use a domain adaptation approach to train a model on a small dataset of high-quality solo guitar transcriptions, based on the "High-resolution piano transcription" model by Kong et al..

Building on the work of Maman and Bermano, we align existing guitar transcriptions to the model activations of the piano transcription model. We then use these aligned transcriptions to train a new model, which is able to transcribe the entire GuitarSet in a zero shot setting with state-of-the-art accuracy.

Alignments

Using our alignment method, we take a transcribed score and match it to the audio recording with high accuracy.

Here we show a piece from our training data. The original audio is from "Johnny Smith - Autumn Nocturne" and the video shows the aligned transcription. Note how the fine-alignment process recovers the micro-timing variations of chord onsets, despite these notes appearing in the same time instant in the original score.

The source data was obtained from professional transcriber, François Leduc. The GuitarPro files are commercially available from his website and for ease of reproduction are listed as follows:

Training Split

Validation split

Test split

Transcription Performance

Due to diverse training conditions, we are able to transcribe different types of guitar. In all of the following examples the original audio can be heard on the left channel, while the transcribed audio (synthesised as piano) can be heard on the right channel.