Efficient and Accurate Approximations of Nonlinear Convolutional Networks英文学习资料

《Efficient and Accurate Approximations of Nonlinear Convolutional Networks英文学习资料》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、EfficientandAccurateApproximationsofNonlinearConvolutionalNetworksXiangyuZhang1∗JianhuaZou1XiangMing1∗KaimingHe2JianSun21Xi’anJiaotongUniversity2MicrosoftResearchAbstracttheoriginalfilterscanbeapproximatelydecomposedintoaseriesofsmallerfilters,andthecomplexityisreduced.Thispaperaimstoaccelera

2、tethetest-timecomputationThesemethodshaveshownpromisingspeedupratiosonaofdeepconvolutionalneuralnetworks(CNNs).Unlikeex-single[3]orafewlayers[10]withsomedegradationofistingmethodsthataredesignedforapproximatinglinearaccuracy.filtersorlinearresponses,ourmethodtakesthenonlinearThealgorithmsan

3、dapproximationsinthepreviousworkunitsintoaccount.Weminimizethereconstructionerroraredevelopedforreconstructinglinearfilters[3,10]andofthenonlinearresponses,subjecttoalow-rankconstraintlinearresponses[10].However,thenonlinearitylikethewhichhelpstoreducethecomplexityoffilters.WedevelopRectified

4、LinearUnits(ReLU)[14,11]isnotinvolvedinaneffectivesolutiontothisconstrainednonlinearoptimiza-theiroptimization.Ignoringthenonlinearitywillimpacttionproblem.Analgorithmisalsopresentedforreducingthequalityoftheapproximatedlayers.Letusconsideratheaccumulatederrorwhenmultiplelayersareapproxi-c

5、asethatthefiltersareapproximatedbyreconstructingthemated.Awhole-modelspeedupratioof4×isdemonstratedlinearresponses.BecausetheReLUwillfollow,themodelonalargenetworktrainedforImageNet,whilethetop-5er-accuracyismoresensitivetothereconstructionerroroftherorrateisonlyincreasedby0.9%.Ouraccelerat

6、edmodelpositiveresponsesthantothatofthenegativeresponses.hasacomparablyfastspeedasthe“AlexNet”[11],butisMoreover,itisachallengingtaskofacceleratingthe4.7%moreaccurate.wholenetwork(insteadofjustoneoraveryfewlayers).Theerrorswillbeaccumulatedifseverallayersareapprox-imated,especiallywhenthem

7、odelisdeep.Actually,inthe1.Introductionrecentwork[3,10]theapproximationsareappliedonasin-Thispaperaddressesefficienttest-timecomputationofglelayeroflargeCNNmodels,suchasthosetrainedondeepconvolutionalneuralnetworks(CNNs)[12,11].SinceImageNet[2,16].Itisinsufficie