[NIPS 2012 Hinton] ImageNet Classification with Deep Convolutional Neural Networks

《[NIPS 2012 Hinton] ImageNet Classification with Deep Convolutional Neural Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ImageNetClassificationwithDeepConvolutionalNeuralNetworksAlexKrizhevskyIlyaSutskeverGeoffreyHintonUniversityofTorontoCanadaPaperwithsamenametoappearinNIPS2012MainideaArchitectureTechnicaldetailsNeuralnetworks●Aneuron●AneuralnetworkOutputf(x)w1wHidden3w2f(z)f(z)f(z)Data123x=wf(z)

2、+wf(z)+wf(z)112233Aneuralnetworkcomputesadifferentiablexiscalledthetotalinputfunctionofitsinput.Forexample,ourscomputes:totheneuron,andf(x)p(label

3、aninputimage)isitsoutputConvolutionalneuralnetworks●Here'saone-dimensionalconvolutionalneuralnetwork●Eachhiddenneuronappliesthesamel

4、ocalized,linearfiltertotheinputOutputHiddenDataConvolutionin2DInput“image”FilterbankOutputmapLocalpoolingMaxOverviewofourmodel●Deep:7hidden“weight”layers●Learned:allfeatureextractorsinitializedatwhiteGaussiannoiseandlearnedfromthedata●Entirelysupervised●Moredata=goodConvolutiona

5、llayer:convolvesitsinputwithabankof3Dfilters,thenappliespoint-wisenon-linearityImageFully-connectedlayer:applieslinearfilterstoitsinput,thenappliespoint-wisenon-linearityOverviewofourmodel●TrainedwithstochasticgradientdescentontwoNVIDIAGPUsforaboutaweek●650,000neurons●60,000,000

6、parameters●630,000,000connections●Finalfeaturelayer:4096-dimensionalConvolutionallayer:convolvesitsinputwithabankof3Dfilters,thenappliespoint-wisenon-linearityImageFully-connectedlayer:applieslinearfilterstoitsinput,thenappliespoint-wisenon-linearity96learnedlow-levelfiltersMain

7、ideaArchitectureTechnicaldetailsLocalconvolutionalfiltersTrainingFully-connectedfiltersUsingstochasticgradientdescentandthebackpropagationalgorithm(justrepeatedapplicationofthechainrule)BsascakpwdarardwpraosFsImageImageOurmodel●Max-poolinglayersfollowfirst,second,andfifthconvolu

8、tionallayers●Thenumberofneuronsineachlayerisgivenby253440,186624,64896,64896,43264,4096,4096,1000MainideaArchitectureTechnicaldetailsInputrepresentation●Centered(0-mean)RGBvalues.Aninputimage(256x256)MinussignThemeaninputimageNeuronsf(x)=tanh(x)f(x)=max(0,x)f(x)w1w3w2f(z)f(z)f(z

9、)123x=wf(z)+wf(z)+wf(z)112233xiscalledthetotali