13[Nov 19]Bayesian nonparametric methods (Dirichlet processes) [Kurt Miller].pdf

《13[Nov 19]Bayesian nonparametric methods (Dirichlet processes) [Kurt Miller].pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、Dr.NonparametricBayesOr:HowILearnedtoStopWorryingandLovetheDirichletProcessKurtMillerCS294:PracticalMachineLearningNovember19,2009TodaywewilldiscussNonparametricBayesianmethods.KurtT.MillerDr.NonparametricBayes2TodaywewilldiscussNonparametricBayesianmethods.“NonparametricBayesianmethods”?Whatdoesth

2、atmean?KurtT.MillerDr.NonparametricBayes2IntroductionNonparametricNonparametric:DoesNOTmeantherearenoparameters.KurtT.MillerDr.NonparametricBayes3IntroductionExample:Classification++++++⇒+⇒+++++⇒Data46⇒BuildmodelNonparametricApproachPredictusingmodelParametricApproachKurtT.MillerDr.NonparametricBaye

3、s4IntroductionExample:Regression⇒⇒Data⇒BuildmodelNonparametricApproachPredictusingmodelParametricApproachKurtT.MillerDr.NonparametricBayes5IntroductionExample:Clustering⇒⇒Data⇒BuildmodelNonparametricApproachParametricApproachKurtT.MillerDr.NonparametricBayes6IntroductionSonowweknowwhatnonparametric

4、means,butwhatdoesBayesianmean?Statistics:BayesianBasics(Slidefromtutoriallecture)TheBayesianapproachtreatsstatisticalproblemsbymaintainingprobabilitydistributionsoverpossibleparametervalues.Thatis,wetreattheparametersthemselvesasrandomvariableshavingdistributions:1Wehavesomebeliefsaboutourparameter

5、valuesθbeforeweseeanydata.ThesebeliefsareencodedinthepriordistributionP(θ).2Treatingtheparametersθasrandomvariables,wecanwritethelikelihoodofthedataXasaconditionalprobability:P(X

6、θ).3WewouldliketoupdateourbeliefsaboutθbasedonthedatabyobtainingP(θ

7、X),theposteriordistribution.Solution:byBayes’theorem

8、,P(X

9、θ)P(θ)P(θ

10、X)=P(X)where!P(X)=P(X

11、θ)P(θ)dθKurtT.MillerDr.NonparametricBayes7IntroductionWhyBeBayesian?Youcantakeacourseonthisquestion.KurtT.MillerDr.NonparametricBayes8IntroductionWhyBeBayesian?Youcantakeacourseonthisquestion.Oneanswer:InfiniteExchangeability:∀np(x1,...,xn)=p(xσ(1),...,xσ(n))Kurt

12、T.MillerDr.NonparametricBayes8IntroductionWhyBeBayesian?Youcantakeacourseonthisquestion.Oneanswer:InfiniteExchangeability:∀np(x1,...,xn)=p(xσ(1),...,xσ(n))DeFinetti’sTheorem(1955):If(x1,x2,...)areinfinitelyex