Surpassing Human-Level Performance on ImageNet Classification

《Surpassing Human-Level Performance on ImageNet Classification》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、DelvingDeepintoRectifiers:SurpassingHuman-LevelPerformanceonImageNetClassificationKaimingHeXiangyuZhangShaoqingRenJianSunMicrosoftResearchAbstracttechniques[13,30,10,36],aggressivedataaugmentationRectifiedactivationunits(rectifiers)areessentialfor[18,14,29,33],andlarge-scaledata[4,26].state-

2、of-the-artneuralnetworks.Inthiswork,westudyAmongtheseadvances,therectifierneuron[24,9,23,rectifierneuralnetworksforimageclassificationfromtwo38],e.g.,RectifiedLinearUnit(ReLU),isoneofseveralaspects.First,weproposeaParametricRectifiedLinearkeystotherecentsuccessofdeepnetworks[18].Itexpe-Unit(P

3、ReLU)thatgeneralizesthetraditionalrectifiedunit.ditesconvergenceofthetrainingprocedure[18]andleadsPReLUimprovesmodelfittingwithnearlyzeroextracom-tobettersolutions[24,9,23,38]thanconventionalsigmoid-putationalcostandlittleoverfittingrisk.Second,wederivelikeunits.Despitetheprevalenceofrectifi

4、ernetworks,arobustinitializationmethodthatparticularlyconsidersrecentimprovementsofmodels[37,28,12,29,33]andtherectifiernonlinearities.Thismethodenablesustotraintheoreticalguidelinesfortrainingthem[8,27]haverarelyextremelydeeprectifiedmodelsdirectlyfromscratchandtofocusedonthepropertiesoft

5、herectifiers.investigatedeeperorwidernetworkarchitectures.BasedUnliketraditionalsigmoid-likeunits,ReLUisnotasym-onthelearnableactivationandadvancedinitialization,wemetricfunction.Asaconsequence,themeanresponseofachieve4.94%top-5testerrorontheImageNet2012clas-ReLUisalwaysnosmallerthanzero;

6、besides,evenassum-sificationdataset.Thisisa26%relativeimprovementoveringtheinputs/weightsaresubjecttosymmetricdistribu-theILSVRC2014winner(GoogLeNet,6.66%[33]).Toourtions,thedistributionsofresponsescanstillbeasymmetricknowledge,ourresultisthefirst1tosurpassthereportedbecauseofthebehaviorof

7、ReLU.ThesepropertiesofReLUhuman-levelperformance(5.1%,[26])onthisdataset.influencethetheoreticalanalysisofconvergenceandempir-icalperformance,aswewilldemonstrate.1.IntroductionInthispaper,weinvestigateneuralnetworksfromtwoaspectsparticularlydrivenbytherectifierproperties.Fi