Convolutional Neural Networks at Constrained Time Cost

《Convolutional Neural Networks at Constrained Time Cost》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ConvolutionalNeuralNetworksatConstrainedTimeCostKaimingHeJianSunMicrosoftResearchfkahe,jiansung@microsoft.comAbstractcommercialsearchengineneedstoresponsetoarequestinreal-time;acloudserviceisrequiredtohandlethousandsThoughrecentadvancedconvolutionalneuralnetworksofu

2、ser-submittedimagespersecond;evenforoff-linepro-(CNNs)havebeenimprovingtheimagerecognitionac-cesseslikeweb-scaleimageindexing,thesystemneedstocuracy,themodelsaregettingmorecomplexandtime-handletensofbillionsofimagesinafewdays.Increas-consuming.Forreal-worldapplicati

3、onsinindustrialandingthecomputationalpowerofthehardwarecanpartiallycommercialscenarios,engineersanddevelopersareoftenrelieftheseproblems,butwilltakeveryexpensivecommer-facedwiththerequirementofconstrainedtimebudget.Incialcost.Furthermore,onsmartphonesorportabledevic

4、es,thispaper,weinvestigatetheaccuracyofCNNsundercon-thelowcomputationalpower(CPUsorlow-endGPUs)lim-strainedtimecost.Underthisconstraint,thedesignsoftheitsthespeedofthereal-worldrecognitionapplications.Sonetworkarchitecturesshouldexhibitastrade-offsamonginindustriala

5、ndcommercialscenarios,engineersandde-thefactorslikedepth,numbersoffilters,filtersizes,etc.velopersareoftenfacedwiththerequirementofconstrainedWithaseriesofcontrolledcomparisons,weprogressivelytimebudget.modifyabaselinemodelwhilepreservingitstimecomplex-Besidesthetest-

6、timedemands,theoff-linetrainingpro-ity.Thisisalsohelpfulforunderstandingtheimportanceofcedurecanalsobeconstrainedbyaffordabletimecost.Thethefactorsinnetworkdesigns.Wepresentanarchitecturerecentmodels[1,9,22,23]takeahigh-endGPUormulti-thatachievesverycompetitiveaccur

7、acyintheImageNetpleGPUs/clustersoneweekorseveralweekstotrain,whichdataset(11.8%top-5error,10-viewtest),yetis20%fastercansometimesbetoodemandingfortherapidlychangingthan“AlexNet”[14](16.0%top-5error,10-viewtest).industry.Moreover,evenifthepurposeispurelyforpush-ingth

8、elimitsofaccuracy(likefortheImageNetcompe-tition[20]),themaximumtolerabletrainingtimeisstilla1.Introductionmajorbottleneckforexperimentalr