0
$\begingroup$

I have a set of matrices which should fall into 3 distinct set/groups/clusters. They are unlabelled. I wish to do unsupervised clustering with PCA. I am using matlab as well. At the end I would also like to examine the eigenvectors.

Matlab has a function call "princomp" which I believe can do this task; is this correct?

When I give "princomp" a matrix the output can be interpreted how?

For example:

dataTmp=[1 1; 2 2; 1 2; 2 3; 4 6; -1 1; -2 2; -4 3; -5 8] dataTmp =  1     1  2     2  1     2  2     3  4     6 -1     1 -2     2 -4     3 -5     8  princomp(dataTmp)  ans =  0.9207    0.3902 -0.3902    0.9207 

or should I being using the function "zscore" beforehand to standardise the values first?

princomp(zscore(dataTmp))  ans =  0.7071    0.7071 -0.7071    0.7071 

How do I interpret the answer? The data I made were simple points in either the first or second quandrant.

1 Answers 1

1

I don't see what you want to do: princomp performs a principal component analysis of the data. That is it essentially determines an orthonormal basis of the sample space, such that the orthogonal projection of the data to the line spanned by the first base vector has maximal variance. The second base vector satisfies a similar maximality condition just with the constraint that it be orthogonal to the first. And so on for the third, fourth etc. The output you get is a matrix that has these base vectors as columns.

The principal components are not related in an obvious way to existing clusters in the data set.

Why don't you use kmeans instead? If the matrices you speak of are falling clearly into three classes, as you say, kmeans should work well. The only drawback possibly being the fact that the three clusters are not "convex", in which case kmeans might fail to give an acceptable result.

  • 0
    1)can you please elaborate on what the 'convex' refers too? 2)And I have heard that the method, "eigenfaces", categorizes/recognizes faces so I assume that this is a form of clustering in a way (right?). 3)Having orthogonal vectors is not a restriction? 4)Do I not need to do "zscore" which subtracts the variance to standardise? Thanks2011-09-07
  • 0
    one more question 1) and in the example data I gave the first column was the x dimension and the second the y, so (0.9207, -0.3902)^T means that 0.9207 is the x dimension?2011-09-07
  • 0
    this link http://www.vision.jhu.edu/gpca/CVPR07-Tutorial-GPCA-Algebra.pdf speaks about PCA and clustering.2011-09-07
  • 1
    1. "Convex" is just a rough geometric description of the clusters one expects to find. The data of each cluster should "fill" a convex part of sample space. In fact "spherical" would be even better. 3. Yes orthogonality is a restriction.2011-09-07
  • 0
    so what can I do with the principle components of this very high dimensional data 112^2? How do I extract any useful information? Lets say the first eigenvector is 50% of the variance, what can I do with that vector? thanks2011-09-07
  • 1
    Before discussing any further: what exactly is your aim when analyzing the set of matrices you have at hand. Do you want to create a partition into 3 subsets? Do you have any idea about how to distinguish between the 3 groups? Are there possibly less/more than 3 groups? PCA can be used to reduce the dimension of the sample space through considering only the components of the data with respect to the first few eigenvectors.2011-09-08
  • 0
    I did the partitioning with kmeans like you prescribed. That was great. Now to reduce the dimensionality would be great as the dimensions are many. I take the first 10 eigenvectors which are 70% of the variance, but each vector is 112^2 dimensions. What do I do with these 10 vecotrs to reduce the dimensions? Do I multiply each data matrix by the set of reduced eigenvectors for that to be a 'reduced' dimension size as each product maps to a scalar? Thanks!2011-09-08
  • 1
    The eigen vectors of the covariance matrix form a orthogonal basis of the sample space provided the covariance matrix is regular. So to reduce dimension compute the orthogonal projection of every sample (=matrix) to e.g. the space spanned by the first 10 eigenvectors. This gives you a new set of samples in 10-dimensional space to which you can apply kmeans etc.2011-09-12
  • 0
    that was great, and clear. Thanks again!2011-09-14