Kernel Selection and Optimization for Support Vector Machines: Application to Speaker Verification
Andrew Oliver Hatch and Nelson Morgan
One of the central problems in the study of support vector machines (SVMs) is kernel selection--that is, the problem of choosing or learning an appropriate kernel function for a particular dataset. Our project considers the problem of learning kernels for binary classification tasks where the training data are partitioned into multiple disjoint classes. The project focuses specifically on the field of speaker verification, which can be framed as a one-versus-all (OVA) decision task with many classes.
The centerpiece of this project is a new framework for learning so-called generalized linear kernels--that is, kernels of the form, k(x1,x2) = x1' R x2, where R is a psd parameter matrix--for binary classification in multiclass settings. We have shown that the optimal R is equal to the inverse of a weighted average of the within-class covariance matrices over all classes. This choice of R is optimal in the sense that it minimizes a particular upper bound on classification error in an SVM. When applied to a state-of-the-art SVM-based speaker verification system, our method of choosing R yields relative reductions in classification error of between 12% and 22% on various speaker verification tasks. We have yet to find any method of training generalized linear kernels that outperforms this approach.