@[NIPS 2010 Andrew] Tiled Convolutional Neural Networks

《@[NIPS 2010 Andrew] Tiled Convolutional Neural Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、TiledconvolutionalneuralnetworksQuocV.Le,JiquanNgiam,ZhenghaoChen,DanielChia,PangWeiKoh,AndrewY.NgComputerScienceDepartment,StanfordUniversity{quocle,jngiam,zhenghao,danchia,pangwei,ang}@cs.stanford.eduAbstractConvolutionalneuralnetworks(CNNs)havebeensuccessfullyappliedtomanytaskssuchasdig

2、itandobjectrecognition.Usingconvolutional(tied)weightssignificantlyreducesthenumberofparametersthathavetobelearned,andalsoallowstranslationalinvariancetobehard-codedintothearchitecture.Inthispa-per,weconsidertheproblemoflearninginvariances,ratherthanrelyingonhard-coding.Weproposetiledconvol

3、utionneuralnetworks(TiledCNNs),whichusearegular“tiled”patternoftiedweightsthatdoesnotrequirethatadjacenthiddenunitsshareidenticalweights,butinsteadrequiresonlythathiddenunitskstepsawayfromeachothertohavetiedweights.Bypoolingoverneighboringunits,thisarchitectureisabletolearncomplexinvarianc

4、es(suchasscaleandrotationalinvariance)beyondtranslationalinvariance.Further,italsoenjoysmuchofCNNs'advantageofhavingarelativelysmallnumberoflearnedparameters(suchaseaseoflearningandgreaterscalability).WeprovideanefficientlearningalgorithmforTiledCNNsbasedonTopographicICA,andshowthatlearning

5、complexinvariantfeaturesallowsustoachievehighlycompetitiveresultsforboththeNORBandCIFAR-10datasets.1IntroductionConvolutionalneuralnetworks(CNNs)[1]havebeensuccessfullyappliedtomanyrecognitiontasks.Thesetasksincludedigitrecognition(MNISTdataset[2]),objectrecognition(NORBdataset[3]),andnatu

6、rallanguageprocessing[4].CNNstaketranslatedversionsofthesameba-sisfunction,and“pool”overthemtobuildtranslationalinvariantfeatures.Bysharingthesamebasisfunctionacrossdifferentimagelocations(weight-tying),CNNshavesignificantlyfewerlearn-ableparameterswhichmakesitpossibletotrainthemwithfewerex

7、amplesthanifentirelydifferentbasisfunctionswerelearnedatdifferentlocations(untiedweights).Furthermore,CNNsnaturallyenjoytranslationalinvariance,sincethisishard-codedintothenetworkarchitecture.However,onedisadvantageofthishard-codingapproachisthatthepoolingarch