Convolutional networks(Honglak Lee)

《Convolutional networks(Honglak Lee)》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ConvolutionalNetworksHonglakLeeCSEdivision,EECSdepartmentUniversityofMichigan,AnnArbor8/6/2015DeepLearningSummerSchool@Montreal1UnsupervisedConvolutionalNetworks2LearningFeatureHierarchy[Leeetal.,NIPS2007;Ranzatoetal.,2007]NaturalImagesLearnedbases:“Edges”5010015020050

2、25010030015035020040025050450300100500501001502002503003504004505003501502004002504503005005010015020035025030035040045050040045050050100150200250300350400450500Testexample~++x~1*b+1*b+1*b426536[0,0,…,0,1,0,…,0,1,0,…,0,1,…]Compact&easily=coefficients(featurerepresentat

3、ion)interpretable3Motivation?Salientfeatures,CompactrepresentationLearningFeatureHierarchyHigherlayer(Combinationsofedges)“Sparsecoding”(edges)Inputimage(pixels)Leeetal.,NIPS2007:DBN(Hintonetal.,2006)withadditionalsparsenessconstraint.[Relatedwork:Bengioetal.,2006;Ranz

4、atoetal.,2007,andothers.]4Note:Noexplicit“pooling.”Describemoreconcretely..Learningobjectrepresentations•Learningobjectsandpartsinimages•Largeimagepatchescontaininterestinghigher-levelstructures.–E.g.,objectpartsandfullobjects•Challenge:high-dimensionalityandspatialcor

5、relations5Illustration:Learningan“eye”detectorAdvantageofshrinking“Eyedetector”1.Filtersizeiskeptsmall2.Invariance“Shrink”(maxover2x2)filter1filter2filter3filter4“Filtering”outputExampleimageRelatedwork:ConvnetsbyLeCunetal.,19896ConvolutionalarchitecturesMax-poolinglay

6、erWeightsharingby“filtering”maximum2x2grid(convolution)[Lecunetal.,1989]Detectionlayermax“Max-pooling”convolutionInvarianceconvMax-poolinglayerComputationalefficiencymaximum2x2gridConvolutionalRestrictedBoltzmannmachine.Detectionlayermax-Unsupervised-Probabilisticma

7、x-poolingconvolution-CanbestackedtoformconvolutionalconvolutionfilterDBNInputconv7ShowexamplesbeforethisfigureConvolutionalRBM(CRBM)[Leeetal.,ICML2009]‘’max-pooling’’node(binary)For“filter”k,Max-poolinglayerPDetectionlayerHHiddennodes(binary)Constraint:AtmostWkonehidde

8、nnodeis1“Filter“weights(shared)(active).InputdataV(visiblelayer)VAtmostonehiddennodesareactive.Keyproperties:-RBM(pr