Sorry for the late. I was not able to connect for a week because I was not at home.

To be short, this is what I would do:

1. Run K-means several times with different ‘k’ values and choose the best instance. “Best” here means the instance that produce the best value for some validity index (C-index are supposed to be a good choice for K-means).

2. Run SVC several times with different parameters settings (kernel, C, q, etc.) and choose the best instance according to a validity index (the best choice is the index specifically developed for SVC, I guess).

3. Compare the results of the “best k-means instance” and the “best SVC instance”.

That’s all.

And no, I am sure that the validity index developed for the SVC does not fit the K-means because the index relies on specific characteristics of the SVC.

I hope I am of help.

Best,

VR.

I have already have some idea on the evaluation of clustering results now. As you mentioned, from a review of the literature, generally there are three approaches, internal criteria, external criteria, and relative criteria, have been used for the quantitative evaluation of clustering results in many of the research studies.

Internal criteria are the only means to evaluate the clustering quality of a completely new domain.

External criteria imply clustering evaluation by means of external pre-specified structure information of a dataset. Generally, several real-life or synthetic datasets with prior class information such as Iris data, Wisconsin’s breast cancer database, and Spam database, are used as evaluation benchmark for clustering results.

Relative criteria (also known as validity index) evaluate clustering results with different input parameter settings of a same clustering algorithm. A number of validity indices have been developed and proposed in literature.

If i want to compare the performance of K-means and SVC in a completely new domain without prior class information. There are two methods as follows:

1.In the process of running K-means and SVC, validity index is embeded respectively for parameter selection. When the optimal clusering resluts are found respectively, the clstering resluts are evaluated by internal criteria.

2.There is a validity index specific for SVC which is quite useful for parameter selection as you mentioned (I don’t know if it can used for K-means). If different validity indices are used in K-means and SVC, I think they can’t be compared. If the same validity index is used in both clustering algorithms, regardless of it is not appropriate for SVC, can the index value work as a crtieria of the performance of these two clustering algorithms?

Maybe I didn’t well express my question. I hope you can understand. I am a little confused here.

Many thanks and Best regards

Lawrrence

Thank you very very much. It seems clear to me. I’d like to read relevant parts of your thesis and the paper suggested by you first. You really give me a great inspiration on how to evaluate the quality of clustering results when datasets are unlabeled. It’s great appreciated for your help.

Best Regards

Lawrence

welcome back.

Since your datasets are unlabeled, the first thing you need are some criteria to evaluate the quality of clustering results. The most effective way to evaluate clustering results when data are unlabeled, is a relative criterion (aka validity index).

In the section 3.3.3 of my thesis I present some of classical validity indices and in

J. Wang and J. Chiang, “A cluster validity measure with a hybrid parameter search method for the support vector clustering algorithm,” Pattern Recognition, vol. 41, iss. 2, pp. 506-520, 2008.

you can found a validity index specific for SVC. I used it to develop a new stopping criterion for SVC (not available in the software you are using).

So, you have to run the SVC and K-means several times, each with a different parameter settings and then evaluate the results. The better index value, the better the clustering. Since you probably don’t know the number of clusters and K-means need it, a classical way is to run K-means with different number of clusters in input and then choose the instance that yields the best validity index value. As far as the SVC is concerned, you can try different combinations of q/C/kernel and choose the instance that yields the best value of the validity index.

I hope I was clear.

Best,
VR.

I notice that you compared the performance of different clustering methods using real-life benchmarks such as Iris data, Wisconsin’s breast cancer database, and Wine Recognition Database in your thesis. Camastra (2005) also compared the performance of the current clustering methods including K-means, Neural Gas, Self-Organizing Map (SOM), Spectral clustering algorithm and SVC on three kinds of real-life benchmarks. However, there already are class information for these data sets, that is, we have already know the class each observation belongs. It’s quite OK and necessary that these datasets are used to compare the results of clustering methods. If I want to compare the performance of SVC and K-means on a dataset without class information ahead, do you have some suggestions?

Many thanks and Best Regards

Lawrence

the -s switch is supposed to be always used with 0.5 value. It’s just a heuristics that allow to explore the ‘q’ value sequence in a smoother way. After a number of tries, the 0.5 value resulted the most suitable one.

My advice is to perform the experiments both with and without the softening strategy heuristics, even though the softening should be result in more accurate clustering.

The second question: SVC i a hierarchical-like clustering, so it is a so-called non-parametric clustering algorithm. It DOES NOT use the number of clusters as input parameter for determining the clustering. Actually, it should not use it at all. In this version of my software such a parameter is a ‘dirty trick’ and serves for a stopping criterion in case of datasets which we know the exact number of clusters.

However, SVC could find not the same number of clusters you expect. This is why it auto-detects the latent structure. And it is not perfect.

In such cases you need to try different parameters combination: differnet values of ‘C’ manually, different kernels (with -k switch you can use the Gaussian, the Laplace, the Exponential in my software), different metric distance (with -l switch you can use either the L1 or the L2 distance).

Only the ‘q’ value have to be found automatically by SVC software in any of the setups you try.

Best Regards,

VR

Why the number of clusters is usually not equal to the specified c value? If I have already know the number of clusters based on past literature,take 6 as an example, I want to obtain such kind of segmenation. Even the c value is specified to be 6, the results are usually not 6 clusters. What’s the problem? Do you have any suggestions to adjust the parameters q/C or s to obtain the specified number of clusters?

Many thanks and Best Regards

Lawrence

I quite agree with you that research activity should be open and collaborative. Althoughs there are many researchers did svc as well as its applications, You can’t find any code or software from internet except yours. I really appreciate your work and kindness.

One more thing. I checked myself according to your instructions. I run the svc in this way: svc -c 3 -f /path/to/iris-file to obtain the q value which is 0.163274. When I run the svc in another way: svc -c 3 -s 0.5 -f /path/to/iris-file, I will obtain the q/C value suggested by you. The problem is how to control s value. I also test other s values in this way, the output is not as good as s=0.5. Should I also use this s value When I apply svc to my own dataset? or you can obtain the q (0.0891501)value in the way (svc -c 3 -f /path/to/iris-file)?

Best Regards

Lawrence

thank you for the greetings, I’m glad to be of help.

My opinion about research activity is to be open and collaborative. I can’t understand a closed research envinroment, because I think is a model that in constrat with the idea of research itself. While I was working on my master’s thesis, I encountered many people with no intention to share anything at all. Unbelievable, from my perspective. I like to share, I need to share, I want to share.

This is why I wrote the SVC software and put it online. And this is why I will put online the future versions too.

The PhD in London will start in October. In the meanwhile, I am in my city (Naples, Italy) and I collaborate with my University (Federico II, Naples, Italy).

Oh, thanks also for your promises of citation and acknowlegement. I’m glad of this.

Well, keep in touch.
For any question, feel free to write me again.

Best regards,

VR