资源描述：
《CMU高级机器学习支持向量机.pdf》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、AdvancedMachineLearningSupportVectorMachinesEricXingLecture6,August11,2009Reading:©EricXing@CMU,2006-20091WhatisagoodDecisionBoundary?òConsiderabinaryclassificationtaskwithy=±1labels(not0/1asbefore).Class2òWhenthetrainingexamplesarelinearlyseparable,wecansettheparametersofalin
2、earclassifiersothatallthetrainingexamplesareclassifiedcorrectlyòManydecisionboundaries!Class1òGenerativeclassifiersòLogisticregressions…òArealldecisionboundariesequallygood?©EricXing@CMU,2006-200921NotAllDecisionBoundariesAreEqual!òWhywemayhavesuchboundaries?òIrregulardistribu
3、tionòImbalancedtrainingsizesòoutliners©EricXing@CMU,2006-20093ClassificationandMarginòParameterzingdecisionboundaryòLetwdenoteavectororthogonaltothedecisionboundary,andbdenoteascalar"offset"term,thenwecanwritethedecisionboundaryas:Twx+b=0Class2d-d+Class1©EricXing@CMU,2006-2009
4、42ClassificationandMarginòParameterzingdecisionboundaryòLetwdenoteavectororthogonaltothedecisionboundary,andbdenoteascalar"offset"term,thenwecanwritethedecisionboundaryas:Twx+b=0òMarginwTx+b>+cforallxinclass2iiwTx+b<−cforallxinclass1iiClass2Ormorecompactly:(wTxi+b)yi>cd-d+Them
5、arginbetweentwopointsClass1m=d−+d+=©EricXing@CMU,2006-20095MaximumMarginClassificationòThemarginis:Tw()2cm=x*−x*=ijwwòHereisourMaximumMarginClassificationproblem:2cmaxwwTs.ty(wx+b)≥c,∀iii©EricXing@CMU,2006-200963MaximumMarginClassification,con'd.òTheoptimizationproblem:cmaxw,b
6、ws.tTy(wx+b)≥c,∀iiiòButnotethatthemagnitudeofcmerelyscaleswandb,anddoesnotchangetheclassificationboundaryatall!(why?)òSoweinsteadworkonthiscleanerproblem:1maxw,bws.tTy(wx+b)≥1,∀iiiòThesolutiontothisleadstothefamousSupportVectorMachines---believedbymanytobethebest"off-the-shelf
7、"supervisedlearningalgorithm©EricXing@CMU,2006-20097SupportvectormachineòAconvexquadraticprogrammingproblemwithlinearconstrains:1d+max-w,bdws.tTy(wx+b)≥1,∀iii1òTheattainedmarginisnowgivenbywòOnlyafewoftheclassificationconstraintsarerelevantÎsupportvectorsòConstrainedoptimizati
8、onòWecandirectlysolvethisusingcommercialquadraticprogramming(
