Coherence Functions with Applications in Large-Margin Classification Methods.pdf

《Coherence Functions with Applications in Large-Margin Classification Methods.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、JournalofMachineLearningResearch13(2012)2655-2684Submitted4/11;Revised2/12;Published9/12CoherenceFunctionswithApplicationsinLarge-MarginClassificationMethodsZhihuaZhangZHZHANG@ZJU.EDU.CNDehuaLiuDEHUALIU0427@GMAIL.COMGuangDaiGUANG.GDAI@GMAIL.COMCollegeofComput

2、erScienceandTechnologyZhejiangUniversityHangzhou,Zhejiang310027,ChinaMichaelI.JordanJORDAN@CS.BERKELEY.EDUComputerScienceDivisionandDepartmentofStatisticsUniversityofCaliforniaBerkeley,CA94720-1776,USAEditor:XiaotongShenAbstractSupportvectormachines(SVMs)nat

3、urallyembodysparsenessduetotheiruseofhingelossfunc-tions.However,SVMscannotdirectlyestimateconditionalclassprobabilities.Inthispaperweproposeandstudyafamilyofcoherencefunctions,whichareconvexanddifferentiable,assur-rogatesofthehingefunction.Thecoherencefunct

4、ionisderivedbyusingthemaximum-entropyprincipleandischaracterizedbyatemperatureparameter.Itbridgesthehingefunctionandthelogitfunctioninlogisticregression.Thelimitofthecoherencefunctionatzerotemperaturecorrespondstothehingefunction,andthelimitoftheminimizerofi

5、tsexpectederroristheminimizeroftheexpectederrorofthehingeloss.Werefertotheuseofthecoherencefunctioninlarge-marginclas-sificationasC-learning,andwepresentefficientcoordinatedescentalgorithmsforthetrainingofregularizedC-learningmodels.Keywords:large-marginclassi

6、fiers,hingefunctions,logisticfunctions,coherencefunctions,C-learning1.IntroductionLarge-marginclassificationmethodshavebecomeincreasinglypopularsincetheadventofboost-ing(Freund,1995),supportvectormachines(SVM)(Vapnik,1998)andtheirvariantssuchasψ-learning(Shene

7、tal.,2003).Large-marginclassificationmethodsaretypicallydevisedbasedonamajorization-minimizationprocedure,whichapproximatelysolvesanotherwiseintractableopti-mizationproblemdefinedwiththe0-1loss.Forexample,theconventionalSVMemploysahingeloss,theAdaBoostalgorith

8、memploystheexponentialloss,andψ-learningemploysaso-calledψ-loss,asmajorizationsofthe0-1loss.Large-marginclassificationmethodscanbeunifiedusingthetoolsofregularizationtheory;thatis,theycanbeexpress