数据挖掘讲座7 Cluster Analysis Cluster Validity an.pdf

《数据挖掘讲座7 Cluster Analysis Cluster Validity an.pdf》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、Lecture7:ClusterAnalysis:ClusterValidityandAdvancedTopicsIntelligentDataEngineering,2010ClusterValidityForsupervisedclassificationwehaveavarietyofmeasurestoevaluatehowgoodourmodelis-Accuracy,precision,recallForclusteranalysis,theanalogousquestionishowtoevaluatethe“goodness”oftheresulting

2、clusters?But“clustersareintheeyeofthebeholder”!Thenwhydowewanttoevaluatethem?2ClustersFoundinRandomData110.90.90.80.80.70.7Random0.60.6DBSCANy0.5y0.5Points0.40.40.30.30.20.20.10.10000.20.40.60.8100.20.40.60.81xx110.90.9K-means0.80.8Complete0.70.70.60.6Linky0.5y0.50.40.40.30.30.20.20.10.1

3、0000.20.40.60.8100.20.40.60.81xx3DifferentAspectsofClusterValidationDeterminingtheclusteringtendencyofasetofdata,i.e.,distinguishingwhethernon-randomstructureactually.Comparingtheresultsofaclusteranalysistoexternallyknownresults,e.g.,toexternallygivenclasslabels.Evaluatinghowwelltheresu

4、ltsofaclusteranalysisfitthedatawithoutreferencetoexternalinformation.-UseonlythedataComparingtheresultsoftwodifferentsetsofclusteranalysestodeterminewhichisbetter.-samedata+samealgorithm+differentparameters;-samedata+differentalgorithm-differentdata4MeasuresofClusterValidityNumericalmeas

5、uresthatareappliedtojudgevariousaspectsofclustervalidity,areclassifiedintothefollowingthreetypes.ExternalIndex:Usedtomeasuretheextenttowhichclusterlabelsmatchexternallysuppliedclasslabels.-E.g.,Entropy.InternalIndex:Usedtomeasurethegoodnessofaclusteringstructurewithoutrespecttoexternalin

6、formation.-E.g.,SumofSquaredError(SSE)RelativeIndex:Usedtocomparetwodifferentclusteringsorclusters.5MeasuringClusterValidityViaCorrelationTwomatrices-SimilarityMatrix-IdealSimilarityMatrix:Onerowandonecolumnforeachdatapoint.Anentryis1iftheassociatedpairofpointsbelongtothesamecluster.Anen

7、tryis0iftheassociatedpairofpointsbelongstodifferentclusters.Computethecorrelationbetweenthetwomatrices-Sincethematricesaresymmetric,onlythecorrelationbetweenn(n-1)/2entriesneedstobecalculated.Highcorrelationindicatesthatpointsthatbelongtothesameclusterareclo