cvpr18-Towards Effective Low-Bitwidth Convolutional Neural Networks

《cvpr18-Towards Effective Low-Bitwidth Convolutional Neural Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、TowardsEffectiveLow-bitwidthConvolutionalNeuralNetworksBohanZhuang1,2,ChunhuaShen1,2∗,MingkuiTan3,LingqiaoLiu1,IanReid1,21TheUniversityofAdelaide,Australia,2AustralianCentreforRoboticVision3SouthChinaUniversityofTechnology,China{bohan.zhuang,chunhua.shen,lin

2、gqiao.liu,ian.reid}@adelaide.edu.au,mingkuitan@scut.edu.cnAbstractworkweights[7,8],lowrankapproximationofweights[16,34],andtrainingalow-bit-precisionnetwork[4,3638].Thispapertacklestheproblemoftrainingadeepcon-Inthiswork,wefollowtheideaoftrainingalow-precisi

3、onvolutionalneuralnetworkwithbothlow-precisionweightsnetworkandourfocusistoimprovethetrainingprocessandlow-bitwidthactivations.Optimizingalow-precisionofsuchanetwork.Notethatintheliterature,manyworksnetworkisverychallengingsincethetrainingprocesscanadoptthis

4、ideabutonlyattempttoquantizetheweightsofeasilygettrappedinapoorlocalminima,whichresultsinanetworkwhilekeepingtheactivationsto32-bitfloatingsubstantialaccuracyloss.Tomitigatethisproblem,wepro-point[4,19,36,38].Althoughthistreatmentleadstolowerposethreesimple-y

5、et-effectiveapproachestoimprovetheperformancedecreasecomparingtoitsfull-precisioncoun-networktraining.First,weproposetouseatwo-stageterpart,itstillneedssubstantialamountofcomputationalre-optimizationstrategytoprogressivelyfindgoodlocalmin-sourcerequirementtoh

6、andlethefull-precisionactivations.ima.Specifically,weproposetofirstoptimizeanetwithThus,ourworktargetstheproblemoftrainingnetworkwithquantizedweightsandthenquantizedactivations.Thisisbothlow-bitquantizedweightsandactivations.incontrasttothetraditionalmethodswh

7、ichoptimizethemThesolutionsproposedinthispapercontainthreecom-simultaneously.Second,followingasimilarspiritoftheponents.Theycanbeappliedindependentlyorjointly.Thefirstmethod,weproposeanotherprogressiveoptimizationfirstmethodistoadoptatwo-stagetrainingprocess.A

8、ttheapproachwhichprogressivelydecreasesthebit-widthfromfirststage,onlytheweightsofanetworkisquantized.Af-high-precisiontolow-precisionduringthecourseoftrain-terobtainingasufficientlygoodsolutionof