Classification and Discriminant Analysis

《Classification and Discriminant Analysis》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ClassificationandDiscriminantAnalysisKwok-LeungTsuiSystemsEngineering&EngineeringManagementCityUniversityofHongKong2/14/20121LearningMethods•SupervisedLearning•Statisticaldataminingtechniquescreatingafunctionsfromatrainingdataset.•Predictionorclassificationforunknowndatasets.•Un

2、supervisedLearning•Statisticaldataminingtechniquesgroupingorpartitioningdatasets.•Visualizationorvariablereduction.•Associationrulesorcorrelationanalysis.2/14/20122SupervisedLearningMethodsContinuousRegressionXYClassificationCategorical2/14/20123DatasetsTrainingSetDatasetusedfor

3、creatingclassifiersTestingSetDatasetusedforvalidatingclassifierobtainedfromtrainingset.2/14/20124PredictionorClassificationErrorOverfittingTesterrorPredictionErrorTrainingerrorLowModelComplexityHigh2/14/20125ExampleofClassificationProblemEmailspamKnownNormalSpamEmails…………………Newe

4、mailsSpamMethod,Classifier,Learner…UnknownNormalemails2/14/20126BasicApproachestoPrediction/ClassificationLinearmethodsRegression(Prediction):LinearRegressionClassification:•LinearDiscriminantAnalysis(LDA)•LogisticRegressionK-Nearestneighbor(KNN)Regression(Prediction)Class

5、ification2/14/20127BayesDecisionTheoryReviewofBasicBayesTheoryClassprobabilities(Prior)PjConditionaldensityofX:Px

6、jnUnconditionaldensityofXPxPx

7、jPjj1Px

8、PPosteriorprobabilityP

9、xjjfromBayesFormula:jPx2/14/20128BayesClassifierThebasicideaofBayesianclas

10、sification:Findthemaximumposteriorprobability!!Foragiven,ifxPx

11、1p()1Px

12、2p(2),thenisclassifiedtoclass1,otherwisxe,class2.2/14/20129BayesClassifierClassificationMethodsBasedonBayesClassifiers:NaïveBayesClassifiersassumingconditionalindependentdist.oninputsLinearandquad

13、raticdiscriminantanalysisusingGaussianclassconditionaldist.KernelDensityClassificationMixtureDensityClassification2/14/201210NaïveBayesClassification2/14/201211NaïveBayesClassifiersConditionalIndependenceThevariablesinXaresaidtobeconditionallyindependentofY,givenZ,ifthefollow

14、ingconditionholds:P(X

15、Y,Z)P(X

16、Z)dHowitworksP(