[ICML 2009 Honglak, Andrew] Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations

《[ICML 2009 Honglak, Andrew] Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ConvolutionalDeepBeliefNetworksforScalableUnsupervisedLearningofHierarchicalRepresentationsHonglakLeehllee@cs.stanford.eduRogerGrossergrosse@cs.stanford.eduRajeshRanganathrajeshr@cs.stanford.eduAndrewY.Ngang@cs.stanford.eduComputerScienceDepartment,StanfordUnive

2、rsity,Stanford,CA94305,USAAbstractlower-levelambiguitiesintheimageorinfertheloca-Therehasbeenmuchinterestinunsuper-tionsofhiddenobjectparts.visedlearningofhierarchicalgenerativemod-Deeparchitecturesconsistoffeaturedetectorunitsar-elssuchasdeepbeliefnetworks.Scal

3、ingrangedinlayers.Lowerlayersdetectsimplefeaturessuchmodelstofull-sized,high-dimensionalandfeedintohigherlayers,whichinturndetectmoreimagesremainsadicultproblem.Toad-complexfeatures.Therehavebeenseveralapproachesdressthisproblem,wepresenttheconvolu-tolearningde

4、epnetworks(LeCunetal.,1989;Bengiotionaldeepbeliefnetwork,ahierarchicalgen-etal.,2006;Ranzatoetal.,2006;Hintonetal.,2006).erativemodelwhichscalestorealisticimageInparticular,thedeepbeliefnetwork(DBN)(Hintonsizes.Thismodelistranslation-invariantandetal.,2006)isamu

5、ltilayergenerativemodelwheresupportsecientbottom-upandtop-downeachlayerencodesstatisticaldependenciesamongtheprobabilisticinference.Keytoourapproachunitsinthelayerbelowit;itistrainedto(approxi-isprobabilisticmax-pooling,anoveltechniquemately)maximizethelikeliho

6、odofitstrainingdata.whichshrinkstherepresentationsofhigherDBNshavebeensuccessfullyusedtolearnhigh-levellayersinaprobabilisticallysoundway.Ourstructureinawidevarietyofdomains,includinghand-experimentsshowthatthealgorithmlearnswrittendigits(Hintonetal.,2006)andhum

7、anmotionusefulhigh-levelvisualfeatures,suchasob-capturedata(Tayloretal.,2007).Webuilduponthejectparts,fromunlabeledimagesofobjectsDBNinthispaperbecauseweareinterestedinlearn-andnaturalscenes.Wedemonstrateexcel-ingagenerativemodelofimageswhichcanbetrainedlentperf

8、ormanceonseveralvisualrecogni-inapurelyunsupervisedmanner.tiontasksandshowthatourmodelcanper-formhierarchical(bottom-upandtop-down)WhileDBNshavebeensuccessfulincontro