October 11 2007

SVDD and kernel functions

Support Vector Domain Description (SVDD) is the basis of the Support Vector Clustering. The non linear version of the SVDD use the Gaussian kernel and no other kernels has been apparently investigated but the polynomial one which is an example of kernel type that works bad with SVDD.

I wrote an e-mail message to the SVDD author, Dr. David M. J. Tax (from Delft University of Technology, Netherlands). Here the “core” of the message

Even though the Gaussian kernel is the one with the best average performances, some experiments conducted on a specific application domain have given better results with a Laplacian kernel or an Exponential kernel.

What does it theoretically means from an SVDD perspective? Have you never tried kernels other than Gaussian and polynomial ones?

The reply of Dr. Tax was

To be honest the number of kernels I used is relatively limited. For some cases I used a correlation between image patches, and it seemed to work well. Also, some people
have used a modified Haussdorf distance to compare shapes. I don’t have a lot of
experience with it.

The big problem is that we can only say something about generalization for a given representation. By changing the kernel, the representation changes. And what happens then is completely dependent on the data, so it is extremely hard to say something general about what kernel to use (the same like what features to use). For some applications there may be features ‘proven by experience’ (like the RBF kernel for the simple UCI datasets), but theoretically you cannot really proof it, I think.

So, the conclusion is that a deeper investigation about other kernels and SVDD is needed. Currently, I have yet some experimental results in this direction (even if they are from a clustering perspective) and in future we could think to go in depth of the question analyzing the shape of data description and the behavior of various (exponential-based) kernels at different kernel width values.

Post a comment

This blog is multi language by p.osting.it's Babel