欢迎来到天天文库
浏览记录
ID:34211275
大小:68.08 KB
页数:12页
时间:2019-03-04
《(Hinton1995)The wake-sleep algorithm for unsupervised neural networks.pdf》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、Thewake-sleepalgorithmforunsupervisedneuralnetworksGeoffreyEHintonPeterDayanBrendanJFreyRadfordMNealDepartmentofComputerScienceUniversityofToronto6KingsCollegeRoadTorontoM5S1A4,Canada3rdApril1995AbstractAnunsupervisedlearningalgorithmforamultilayernetworkofstochasticneuronsisdescribed.Bottom-upre
2、cognitionconnectionsconverttheinputintorepresentationsinsuccessivehiddenlayersandtop-downgenerativeconnectionsreconstructtherepresentationinonelayerfromtherepresentationinthelayerabove.Inthewakephase,neuronsaredrivenbyrecognitionconnections,andgenerativeconnectionsareadaptedtoincreasetheprobabilit
3、ythattheywouldreconstructthecorrectactivityvectorinthelayerbelow.Inthesleepphase,neuronsaredrivenbygenerativeconnectionsandrecognitionconnectionsareadaptedtoincreasetheprobabilitythattheywouldproducethecorrectactivityvectorinthelayerabove.Supervisedlearningalgorithmsformultilayerneuralnetworksface
4、twoproblems:Theyrequireateachertospecifythedesiredoutputofthenetworkandtheyrequiresomemethodofcommunicatingerrorinformationtoalloftheconnections.Thewake-sleepalgorithmavoidsboththeseproblems.Whenthereisnoexternalteachingsignaltobematched,someothergoalisrequiredtoforcethehiddenunitstoextractunderly
5、ingstructure.Inthewake-sleepalgorithmthegoalistolearnrepresentationsthatareeconomicaltodescribebutallowtheinputtobereconstructedaccurately.Wecanquantifythisgoalbyimaginingacommunicationgameinwhicheachvectorofrawsensoryinputsiscommunicatedtoareceiverbyfirstsendingitshiddenrepresentationandthensendin
6、gthedifferencebetweentheinputvectoranditstop-downreconstructionfromthehiddenrepresentation.Theaimoflearningistominimizethedescriptionlengthwhichisthetotalnumberofbitsthatwouldberequiredtocommunicatetheinputvectorsinthisway[1].Nocommunicationactuallytakesplace,butminimizingthedescriptionlengththatw
7、ouldberequiredforcesthenetworktolearneconomicalrepresentationsthatcapturetheunderlyingregularitiesinthedata[2].1Theneuralnetworkhastwoquitedifferentsetsofconnections.Thebottom-uprecognitionconnectionsareusedtocon
此文档下载收益归作者所有