01_Linear_Models_for_Binary_Classification_21-35

《01_Linear_Models_for_Binary_Classification_21-35》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、MachineLearningFoundations(_hxÒúó)Lecture11:LinearModelsforClassificationHsuan-TienLin(—Ò0)htlin@csie.ntu.edu.twDepartmentofComputerScience&InformationEngineeringNationalTaiwanUniversity(Ëðc'xÇåû)Hsuan-TienLin(NTUCSIE)MachineLearningFoundations0/25LinearModels

2、forClassificationRoadmap1WhenCanMachinesLearn?2WhyCanMachinesLearn?3HowCanMachinesLearn?Lecture10:LogisticRegressiongradientdescentoncross-entropyerrortogetgoodlogistichypothesisLecture11:LinearModelsforClassificationLinearModelsforBinaryClassificationStochastic

3、GradientDescentMulticlassviaLogisticRegressionMulticlassviaBinaryClassification4HowCanMachinesLearnBetter?Hsuan-TienLin(NTUCSIE)MachineLearningFoundations1/25LinearModelsforClassificationLinearModelsforBinaryClassificationLinearModelsRevisitedlinearscoringfuncti

4、on:s=wTxlinearclassificationlinearregressionlogisticregressionh(x)=sign(s)h(x)=sh(x)=(s)x0x0x0x1x1x1sssx2h()xx2h()xx2h()xxdxdxdplausibleerr=0/1friendlyerr=squaredplausibleerr=cross-entropydiscreteEin(w):quadraticconvexEin(w):smoothconvexEin(w):NP-hardtosolvec

5、losed-formsolutiongradientdescentcanlinearregressionorlogisticregressionhelplinearclassification?Hsuan-TienLin(NTUCSIE)MachineLearningFoundations2/25LinearModelsforClassificationLinearModelsforBinaryClassificationErrorFunctionsRevisitedlinearscoringfunction:s=wT

6、xforbinaryclassificationy2f1;+1glinearclassificationlinearregressionlogisticregressionh(x)=sign(s)h(x)=sh(x)=(s)err(h;x;y)=Jh(x)6=yKerr(h;x;y)=(h(x)y)2err(h;x;y)=lnh(yx)err0=1(s;y)errSQR(s;y)errCE(s;y)=Jsign(s)6=yK=(sy)2=ln(1+exp(ys))=Jsign(ys)6=1K=(ys1)

7、2(ys):classificationcorrectnessscoreHsuan-TienLin(NTUCSIE)MachineLearningFoundations3/25LinearModelsforClassificationLinearModelsforBinaryClassificationVisualizingErrorFunctions0=1err0=1(s;y)=Jsign(ys)6=1Ksqrerr(s;y)=(ys1)2SQRceerrCE(s;y)=ln(1+exp(ys))scaledce

8、errSCE(s;y)=log2(1+exp(ys))60/10/1:1iffys0sqr:largeifys14butover-chargeys1err2smallerrSQR!smallerr0=11ce:monotonicofys0smallerrCE$smallerr0=1−3−2−10123ysscaledce:aproperupperbound