模式识别与机器学习.pdf

《模式识别与机器学习.pdf》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。

1、TechnicalReportNo.9815,DepartmentofStatistics,UniversityofTorontoMarkovChainSamplingMethodsforDirichletProcessMixtureModelsRadfordM.NealDepartmentofStatisticsandDepartmentofComputerScienceUniversityofToronto,Toronto,Ontario,Canadahttp://www.cs.utoronto.ca/radford/radford@

2、stat.utoronto.ca1September1998Abstract.MarkovchainmethodsforsamplingfromtheposteriordistributionofaDirichletprocessmixturemodelarereviewed,andtwonewclassesofmethodsarepre-sented.OnenewapproachistomakeMetropolis-Hastingsupdatesoftheindicatorsspecifyingwhichmixturecomponenti

3、sassociatedwitheachobservation,perhapssup-plementedwithapartialformofGibbssampling.TheothernewapproachextendsGibbssamplingfortheseindicatorsbyusingasetofauxiliaryparameters.Thesemeth-odsaresimpletoimplementandaremoreecientthanpreviouswaysofhandlinggeneralDirichletprocessm

4、ixturemodelswithnon-conjugatepriors.1IntroductionModelingadistributionasamixtureofsimplerdistributionsisusefulbothasanon-parametricdensityestimationmethodandasawayofidentifyinglatentclassesthatcanexplainthedependenciesobservedbetweenvariables.Mixtureswithacountablyinniten

5、umberofcomponentscanreasonablybehandledinaBayesianframework,byemployingapriordistributionformixingproportions,suchasaDirichletprocess,thatleadstoafewofthesecomponentsdominating.Useofcountablyinnitemixturesby-passestheneedtodeterminethecorrect"numberofcomponentsinanitemi

6、xturemodel,ataskwhichisfraughtwithtechnicaldiculties.Inmanycontexts,acountablyinnitemixtureisalsoamorerealisticmodelthanamixturewithasmallnumberofcomponents.UseofDirichletprocessmixturemodelshasbecomecomputationallyfeasiblewiththedevelopmentofMarkovchainmethodsforsamplin

7、gfromtheposteriordistributionoftheparametersofthecomponentdistributionsand/oroftheassociationsofmixturecomponentswithobservations.MethodsbasedonGibbssamplingcaneasilybeimple-mentedformodelsbasedonconjugatepriordistributions,butwhennon-conjugatepriorsareused,asisappropriate

8、inmanycontexts,straightforwardGibbssamplingrequiresthatanoftendicultnumericalintegration