Compact Deep Convolutional Neural Networks With Coarse Pruning

《Compact Deep Convolutional Neural Networks With Coarse Pruning 》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、UnderreviewasaconferencepaperatICLR2017COMPACTDEEPCONVOLUTIONALNEURALNETWORKSWITHCOARSEPRUNINGSajidAnwar,WonyongSungDepartmentofElectricalEngineeringandComputerScienceSeoulNationalUniversityGwanak-Gu,08826,RepublicofKoreasajid@dsp.snu.ac.kr,wysung@snu.ac.krABS

2、TRACTThelearningcapabilityofaneuralnetworkimproveswithincreasingdepthathighercomputationalcosts.Widerlayerswithdensekernelconnectivitypatternsfurhterincreasethiscostandmayhinderreal-timeinference.Weproposefeaturemapandkernellevelpruningforreducingthecomputatio

3、nalcomplexityofadeepconvolutionalneuralnetwork.Pruningfeaturemapsreducesthewidthofalayerandhencedoesnotneedanysparserepresentation.Further,kernelpruningcon-vertsthedenseconnectivitypatternintoasparseone.Duetocoarsenature,thesepruninggranularitiescanbeexploited

4、byGPUsandVLSIbasedimplementations.Weproposeasimpleandgenericstrategytochoosetheleastadversarialpruningmasksforbothgranularities.Theprunednetworksareretrainedwhichcompen-satesthelossinaccuracy.Weobtainthebestpruningratioswhenwepruneanetworkwithbothgranularities

5、.ExperimentswiththeCIFAR-10datasetshowthatmorethan85%sparsitycanbeinducedintheconvolutionlayerswithlessthan1%increaseinthemissclassificationrateofthebaselinenetwork.1INTRODUCTIONDeepandwiderneuralnetworkshavethecapacitytolearnacomplexunknownfunctionfromthetrain

6、ingdata.ThenetworkreportedinDeanetal.(2012)has1.7BillionparametersandistrainedontensofthousandsofCPUcores.SimilarlySimonyan&Zisserman(2014)hasemployed16-18layersandachievedexcellentclassificationresultsontheImageNetdataset.Thehighcomputationallycomplexityofwide

7、anddeepnetworksisamajorobstacleinportingthebenefitsofdeeplearningtoresourcelimiteddevices.Therefore,manyresearchershaveproposedideastoacceleratedeepnetworksforreal-timeinferenceYuetal.(2012);Hanetal.(2015b;a);Mathieuetal.(2013);Anwaretal.(2015b).Networkpruningi

8、sonepromisingtechiquethatfirstlearnsafunctionwithasuficientlylargesizedarXiv:1610.09639v1[cs.LG]30Oct2016networkfollowedbyremovinglessimportantconnectionsYuetal.(2012);Hanetal.(2015b