Beyond Words: Affect Recognition and the Math Inside

Izhak Shafran
OGI School of Science & Engineering

Abstract: Measuring distance between variable-length sequences is a key challenge that arises in text processing, for example, machine translation, and in computational biology. To tackle such inherently high dimension variables, kernel methods from statistical learning theory provide an efficient computational tool. Rational kernels provide an attractive solution for text processing. Speech processing introduces additional challenge due to the inherent ambiguity in speech input. In this talk, we will examine how rational kernels can be extended to speech processing and used for combining different input modalities such as visual cues. This general framework can be applied to classify spoken utterances and will be illustrated using a call-center application where a speaker's emotional expression or affect is recognized from his/her voice.