A possible approach to data preparation

As more of a note to self, there are some interesting aspects of the final year project, "Musical Instrument Detection" by Gautham J. Mysore and Gregory Sell and SongHui Chon, which I may consider:

1. The ensemble approach to classification:
   "...we attempt the problem of identifying the instrumentation of a musical signal at any given time using several machine learning techniques(logistic regression, K-NN, SVM).We approached the problem as a series of separate binary classifications (as opposed to a multivariate problem) so that we could mix and match the best algorithm for each instrument to create the best overall classifier."
2. The mixture signals were created artificially:
   "Then, to create one of the combinations above, a random signal for each instrument were all combined randomly. In this way, we created 52 total signals for each instrumental combination."

HMM toolboxes

Some toolboxes and links:

Hidden Markov Model Toolbox for Matlab
Mendel HMM Toolbox for Matlab
H2M: A set of Matlab/Octave functions for the estimation of mixtures and hidden markov models

Modelling the temporal dynamics of timbre

"Musical Instrument Timbres Classification with Spectral Features", Giulio Agostini, Maurizio Longari, Emanuele Pollastri (2001):

A considerable number of features is currently available in the literature, each one describing some aspects of audio content [22, 23]. In the digital domain, features are usually calculated from a window of samples, which is normally very short compared to the total duration of a tone. Thus, we must face the problem of summarizing their temporal evolution into a small set of values. Mean, standard deviation, skewness, and autocorrelation have been the preferred strategies for their simplicity, but more advanced methods like hidden Markov models could be employed, as illustrated in [21, 22]. By combining these time-spanning statistics with the known features, an impressive number of variables can be extracted from each sound. The researcher, though, has to carefully select them in order to both keep the time required for the extraction to a minimum and, more importantly, to prevent from incurring into the so-called curse of dimensionality.

Taken from "Computer Models for Musical Instrument Identification", Nicolas D. Chétry (pg.180):

When modelling timbre, our system and the ones encountered in the literature lose time consideration. In other words, the temporal organisation of the various acoustic events is not represented at the model level. This approach is understandable if one considers timbre as a global attribute of sound. However, we showed in our experiments that, for example, onset and steady-state segments of tones have different characteristics, so that they each contribute to a particular aspect of timbre. Instead of averaging them in one single model, one could think of independently and explicitly modelling them. Similar to speaker recognition, the incorporation of
Dynamic Time Warping (DTW) or Hidden Markov Models (HMM) can constitute a possible orientation for future research.

In a discussion with my supervisor, we noted the importance of considering the temporal dynamics of an instrument sound. As the uniqueness of the spectral envelope cannot be absolutely guaranteed across instruments, extra information about the signal temporal behaviour are often required in order to increase the systems performance. The main difficulty in extracting such temporal features resides in the fact that robust automated pre-processing techniques for onsets or transients detection are difficult to design, especially in the case of pitched musical sounds. As noted by Chetry,

"For this reason, a more general approach is preferred. It consists of appending the delta (speed) and delta-deltas (acceleration) coefficients to the feature vector in order to include information about its evolution with time."

Excellent review of statistical pattern recognition

As more of a 'note to self', Jain provides an excellent overview (review) of Statistical Pattern Recognition in the paper: "Jain, A.K., Duin, P.W., Mao, J. 2000 - Statistical Pattern Recognition - A Review.pdf" This can be found online here.

Important note from PRTools 4.1 User Manual

The performance of classification functions can be improved by the following methods:

1. A reject option in which the objects close to the decision boundary are not classified. They are rejected and might be classified by hand or by another classifier.
2. The selection or averaging of classifiers.
3. A multi-stage classifier for combining classification results of several other classifiers.

For all these methods it is profitable or necessary that a classifier yields some distance measure, confidence or posterior probability in addition to the hard, unambiguous assignment of labels.

PCA - tips and tricks

PCA: tips & tricks

• If n < (number of features K), remove directions without data: k <- n-1.
• Usually we set the mean of data to a zero vector: x <- (x - μ).
• PCA is sensitive to scaling, so it’s best to standardize: xi <- (xi - μi) / standdev
• Images and signal classes are usually invariant to scaling; normalize each sample by removing its mean and dividing by its standard deviation.

Which k features are the best to base the decision on?

A good discussion on feature selection based on ideas published in the paper, "T.M. Cover, The best two independent measurements are not the two best", can be found here. The introduction is as follows:

We have all had the experience of waiting for a friend in a crowded place. The friend is late so, we scan the crowd hoping to recognize the friend from far away. Lets say the friend has short, dark hair, they wear glasses, and often wear a favorite green sweater. Then, consciously or unconsciously, we use what we know about our friend, like the short hair and glasses, to try to pick them out of the crowd. We have all also had a case of mistaken identity. Perhaps a stranger is also wearing a green sweater, or has glasses and short, dark hair, and it is only upon glancing at another of the stranger's features that we notice the difference.

Important note on pattern recognition

We know how to construct classifiers. Evaluation is crucial to find the best one.

Online statistics textbook

This is an excellent online resource for learning statistical analysis: The Electronic Statistics Textbook.

Proper citation:

Proper citation: (Electronic Version): StatSoft, Inc. (2007). Electronic Statistics Textbook. Tulsa, OK: StatSoft. WEB: http://www.statsoft.com/textbook/stathome.html.
Proper citation: (Printed Version): Hill, T. & Lewicki, P. (2007). STATISTICS Methods and Applications. StatSoft, Tulsa, OK.

The Cepstrum PDA is flawed!

I received a correspondence from Arturo Camachoa who wrote a paper showing that the Cepstrum PDA does not work on periodic signals. Details of the paper are as follows:

Comment on “Cepstrum pitch determination” [J. Acoust. Soc. Am. 41, 293–309 (1967)] (L)
Arturo Camachoa, Computational NeuroEngineering Laboratory, University of Florida, Gainesville, Florida 32611
[Received 16 May 2008; revised 25 August 2008; accepted 26 August 2008] "In a paper by A. Michael Noll [J. Acoust. Soc. Am. 41, 293–309
1967, the use of the cepstrum was proposed to determine the pitch of a signal. This paper shows that such a method does not work on periodic signals. © 2008 Acoustical Society of America. ][DOI: 10.1121/1.2988293]

Feature extraction online repository

I was thinking to myself, "Researchers should pool their collective thought, not solely through the publication of papers and hence ideas, but also provide a platform on which to build (my interpretation being 'the Matlab source code!!?')"

For surely, the publication of universally accepted de facto code-standards would promote further extensions, and not burden/hamper researchers with the task of having to 'code it themselves'. A chore. A duplicative task - a non-advancement in the collective knowledge - not so?

And hence, by divine nature, an answer popped out of the google blue:

http://www.ifs.tuwien.ac.at/mir/

"the creation of large audio collections" - an online repository. Now that's what I call music.

Magic.

Joey.

Thematic papers

Very relevant to my own work:

• Wieczorkowska,A., Kolczynska, E. 2007 - "Quality of musical instrument sound identification for various levels of accompanying sounds"
• Analysis and recognition of expression in musical gestures.
  

Another interesting Matlab toolbox

PsySound3.

PsySound3 is software for the analysis of sound recordings using physical and psychoacoustical algorithms. It is an easy to use platform that does precise analysis using standard acoustical measurements, as well as implementations of psychoacoustical and musical models (such as loudness, sharpness, roughness, fluctuation strength, pitch, rhythm and running IACC).

I was just looking for pitch detection ideas but there are many 'Audio Analysers' to choose from.

Another toolbox to play with!!

Note segmentation

I spent quite some time writing code for note segmentation based on onset-detection. I came across this interesting approach by Declan Murphy and Kristoffer Jensen. Don't think I have the time to implement the code myself - unfortunately!!

Pitch tracking

Juan Pablo writes about 'Autocorrelation Pitch Tracking' in his ISMIR 2000 paper. Seems like the best option for my work. From my research, there are many other approaches, the most popular of which seem to be:

• Maximum likelihood
• The short-time average magnitude difference function (AMDF)
• And many more...

An interesting explanation of two pitch tracking (or fundamental frequency estimation) approaches exists here.

p.s. And there's more:

• Bozena Kostek provides a good discussion on Pitch Detection Algorithms (PDAs) in her paper "Musical Instrument Classification and DUET Analysis Employing Music Information Retrieval Techniques."
• High accuracy and Octave error immune pitch detection algorithms

And again! A discussion on robust F0 estimation exists in the Auditory forum here.

AMII. New People. New papers.

I uncovered some researchers in AMII. Both work for the Automatic Indexing of Audio project at The University of North Carolina, Charlotte:

• Amanda Cohen Mostafavi
• Wenxin Jiang

The papers of interest from the 'Automatic Indexing of Audio Project' publications page are the following:

• Multiple classifiers for different features in timbre estimation
• Music Instrument Estimation in Polyphonic Sound Based on Short-Term Spectrum Match
• Cascade Classifiers for Hierarchical Decision Systems
• Training of Classifiers for the Recognition of a Musical Instrument Dominating in the Same-Pitch Mix
• Identification of Dominating Instrument in Mixes of Sounds of the Same Pitch
• Quality of Musical Instrument Sound Identification for Various Levels of Accompanying Sounds

Great online pattern recognition course

The "Advanced Statistical Pattern Recognition" is a very useful and informative course. It uses PRTools for Matlab software.

Matlab Pattern Recognition Toolboxes

Just a note on some of the pattern recognition toolboxes I've come across. There are many, but these look like the most interesting and useful:

• Statistical Pattern Recognition Toolbox for Matlab
• PRTools: The Matlab Toolbox for Pattern Recognition
• TOOLDIAG: Pattern recognition toolbox
• MatlabArsenal
• Data Clustering and Pattern Recognition Toolbox
  

Listed on the TOOLDIAG site are a collection of resources. The pick of the bunch is the following, "The Pattern Recognition Group, TU Delft." Wow!

Very interesting final year project on instrument ID!

I came across a very interesting project today by Davyd Madeley of The University of Western Australia. His final year project, "Automatic Computer Classification of Solo Musical Instruments" certainly captures my interest! Davyd interestingly documents the various stages of this project in his 'articles' section:

• Thesis Proposal
• Lightning talk
• Project presentation (Interim)
• Final year project seminar
• Final Year Project dissertation

Of course, all these files are available on this page.