模式识别Chapter 6-2 PCA PNN.ppt

《模式识别Chapter 6-2 PCA PNN.ppt》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。

1、1PatternRecognition北京交通大学电子信息工程学院2ContentsIntroductionSinglelayerneuralnetworksApplicationsinglelayerneuralnetworkMulti-layerneuralnetworks3ApplicationsinglelayerneuralnetworkProblemsofdimensionality--PrincipalComponentAnalysis(PCA)--FisherLinearDiscriminant(F

2、LD)PolynomialNeuralNetwork(PNN)4ProblemsofdimensionalityInpracticalmulti-categoryapplications:--involvefiftyoroverhundredsfeaturesGeneexpressionFaceimagesHandwrittendigits5ProblemsofdimensionalityInpracticalmulti-categoryapplications:--involvefiftyoroverhundre

3、dsfeaturesHowclassificationaccuracydependsonthedimensionality?Howabouttheclassificationcomplexity?6ProblemsofdimensionalityTheBayesianoferrorrate--twoclassesproblem--samepriorprobability,normaldensityp(e)decreaseswhilerincreaseifrcanbeincreasedwithoutlimit,the

4、errorcanbemadearbitrarysmall.7ProblemsofdimensionalityTheerrorrate:thedifferencebetweenthemeansislargethedeviationissmallnofeatureisuselesscontributiontoclassificationisdifferent8ProblemsofdimensionalityTheerrorrate:nofeatureisuselessaddingnewfeatureseemstobeb

5、enefitbeyondapoint,addingnewfeatureleadstoworseperformance!9ProblemsofdimensionalityComputationalcomplexity--Additions,multiplications…Over-fittingsExcessivedimensionalityReducethedimensionalityoffeaturespace10ProblemsofdimensionalityDimensionalityreduction--P

6、rincipalComponentAnalysis(PCA)--FisherLinearDiscriminant(FLD)11PrincipalComponentAnalysis(PCA)1947年美国的统计学家斯通(stone),国民经济的研究--,利用美国1929一1938年各年的数据--得到了17个反映国民收入与支出的变量雇主补贴消费资料和生产资料纯公共支出净增库存、股息、利息外贸平衡等等。12PrincipalComponentAnalysis(PCA)PCA是对原数据坐标系的一次正交线性变换--将原始数据

7、投影到所谓的数据的主轴上--将高维的数据投影到低维--主轴又称为主成份、特征：主轴、主成份、特征如何找到主成份？13Representingallofthevectorsinasetofnd-dimensionalsamplesbyasinglevectorPrincipalComponentAnalysis(PCA)assmallaspossibleFindavectorsuchthatthesumofsquareddistancebetweenandthevariousisThesquared-errorcri

8、terionfunction14PrincipalComponentAnalysis(PCA)Thesquared-errorcriterionfunction15PrincipalComponentAnalysis(PCA)Easytoverify:=016PrincipalComponentAnalysis(PCA)Independentofx0Mini