资源描述：
《Information Geometry of the EM and em Algorithms for Neural Networks (1995)》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、InformationGeometryoftheEMandemAlgorithmsforNeuralNetworksShun-ichiAmariDepartmentofMathematicalEngineeringandInformationPhysicsFacultyofEngineering,UniversityofTokyoBunkyo-ku,Tokyo113,JAPANThepresentworkissupportedinpartbyGrant-in-AidforScienti cRes
2、earchonPriorityAreasontheHigher-OrderBrainFunctionsfromtheMinistryofEducation,ScienceandCultureofJapan.RequestsforreprintsshouldbesenttotheauthorattheDepartmentofMathematicalEngineering,theUniversityofTokyo,Bunkyo-ku,Hongo,Tokyo113,Japan:fax+81-3-568
3、9-5752.runningtitle:GeometryofEMAlgorithmInformationGeometryoftheEMandemAlgorithmsforNeuralNetworksShun-ichiAmariAbstractInordertorealizeaninput-outputrelationgivenbynoise-contaminatedexam-ples,itise ectivetouseastochasticmodelofneuralnetworks.Amodel
4、networkincludeshiddenunitswhoseactivationvaluesarenotspeci ednorobserved.Itisusefultoestimatethehiddenvariablesfromtheobservedorspeci edinput-outputdatabasedonthestochasticmodel.Twoalgorithms,theEM-andem-algorithms,havesofarbeenproposedforthispurpose
5、.TheEM-algorithmisaniterativesta-tisticaltechniqueofusingtheconditionalexpectation,andtheem-algorithmisageometricalonegivenbyinformationgeometry.Theem-algorithmminimizesiter-ativelytheKullback-Leiblerdivergenceinthemanifoldofneuralnetworks.Thesetwoal
6、gorithmsareequivalentinmostcases.Thepresentpapergivesauni edinformationgeometricalframeworkforstudyingstochasticmodelsofneuralnet-works,byforcussingontheEMandemalgorithms,andprovesaconditionwhichguaranteestheirequivalence.Examplesinclude1)Boltzmannma
7、chineswithhid-denunits,2)mixturesofexperts,3)stochasticmultilayerperceptron,4)normalmixturemodel,5)hiddenMarkovmodel,amongothers.keywords:EMalgorithm,informationgeometry,stochasticmodelofneuralnet-works,learning,identi cationofneuralnetwork,e-project
8、ion,m-projection,hiddenvari-able11IntroductionNeuralnetworkshavebeenremarkedasuniversalapproximatorsofnonlinearfunctionsthatcanbetrainedfromexamplesofinput-outputdata.Whenthedataincludesnoise,theinput-outputrelationisdescribedstochasticallyintermsoft
