AI2013S_Chap05-Linear_Models_for_Classification

《AI2013S_Chap05-Linear_Models_for_Classification》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ArtificialIntelligenceLinearModelsforClassificationDonghuiWangAIInstitute@ZJU2013SpringContents•BasicConcepts•DiscriminantFunctions•ProbabilisticGenerativeModels•ProbabilisticDiscriminativeModels•TheLaplaceApproximation•BayesianLogisticRegressionReferences:1.Bishop.“PatternRec

2、ognitionandMachineLearning”,Chapter4.2006.BasicConceptsLinearlyseparable•Decisionregions:–Inputspaceisdividedintoseveralregions•Decisionboundaries(surfaces):–Underlinearmodels,it’salinearfunctionoftheinputvectorx–(D-1)-dimensionalhyper-planewithintheD-dimensionalinputspace•Dat

3、asetswhoseclassescanbeseparatedexactlybylineardecisionsurfacesaresaidtobelinearseparable.TRepresentationofClassLabels•Twoclasses(K=2):–Targetvariablet∈{0,1},t=1representsclassC,elseclassC12•K-classes(K>2):–1-of-Kcodingscheme:t=(0,1,0,0,0)TW:weightvectorw0:bias/threshold•Predic

4、tdiscreteclasslabels:–Linearmodelprediction(lineardiscriminantfunction):y(x)=wTx+w0–Nonlinearfunctionf(.):R→(0,1)–Generalizedlinearmodels:f(·):activationfunctionf-1(·):linkfunction–Decisionsurface:•y(x)=constant→wTx+w0=constantThreeclassificationapproaches•Discriminantfunction

5、:–Least-squaresapproach:makingthemodelpredictionsascloseaspossibletoasetoftargetvalues–Fisher’slineardiscriminant:maximumclassseparationintheoutputspace–TheperceptronalgorithmofRosenblatt:generalizedlinearmodel•Generativeapproach:–Modeltheclass-conditionaldensitiesandtheclassp

6、riors–ComputeposteriorprobabilitiesthroughBayes’stheorem•Discriminativeapproach:–Directlytrainingposteriorprobabilities.DiscriminantFunctions(nonprobabilisticmethods)Twoclasses•Lineardiscriminantfunction:–ify(x)≥0,assignxtoclassC1,elseclassC2–decisionsurfaceΩ:–thenormaldistanc

7、efromtheorigintothedecisionsurface:xandxlieonthedecisionsurface:ABwisorthogonaltoeveryvectorlyingwithinΩisthenormalvectorofΩΩMultipleclasses•HowtobuildaK-classdiscriminantfunction?–One-versus-the-restclassifier•K-1classifierseachofwhichsolvesatwo-classproblem–One-versus-onecla

8、ssifier•K(K−1)/2binarydiscriminantfunctionsambiguousregionsOn