Approximate Expectation Propagation for Bayesian Inference on Large-scale Problems

《Approximate Expectation Propagation for Bayesian Inference on Large-scale Problems》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ApproximateExpectationPropagationforBayesianInferenceonLarge-scaleProblemsYuanQi,TommiS.Jaakkola,andDavidK.Giord,MITComputerScienceandArticialIntelligenceLaboratoryOctober,20051IntroductionInthisreport,wepresentanovelapproachforapproximateBayesianinferenc

2、eonlarge-scalenetworks.Specically,weconsiderthefollowingmodel.First,wewritedownthelikelihoodfunctionofthedataasYYkp(yjb;s)=p(yijb;s)(1)kiYYXk=N(yijajijjsjbj;i):(2)kij:ajijj>0wherekindexesexperimentalreplicates,iindexestheprobepositions,jindexesthebindi

3、ngPpositions,andN(
jjaPjijjsjbj;i)representstheprobabilitydensityfunctionofaGaussiandistributionwithmeanjajijjsjbjandvariancei.Weassignpriordistributionsonthebindingeventbjandthebindingstrengthsj:p(bj)=bj(1)1bj(3)jjjjp0(sj)=Gamma(sjjc0;d0)(4)whereG

4、amma(
jc0;d0)standsfortheprobabilitydensityfunctionsofGammadistributionswithhyperparametersc0andd0.Weassignahyperpriordistributiononthebindingprobabilityjas:p0(j)=Beta(jj0;0)(5)2ApproximateExpectationPropagationforBayesianinferenceFirst,giventhedatalike

5、lihood(2),thepriordistributions(3)and(4)onthebindingeventbandstrengths,andthehyperpriordistribution(5)onthebindingprobability,theposteriordistributionp(b;s;jy)isproportionaltothejointdistributionp(b;s;;y):YYp(b;s;jy)/p(b;s;;y)=gi(b;s)p0(j)fj(bj;j)p0(

6、sj)ij1whereiindexesprobepositions,Pjindexesbindingpositions,fj(bj;j)=p(bjjj)isthepriorforb,g(b;s)=N(yjasb;)isthelikelihoodfortheobservationattheithjiijjijjjjiprobeposition,p0(j)isthehyperpriordistributionofj,andp(bjjj)andp0(sj)arethepriordistribution

7、sofbjandsj,respectively.Forsimplicityandclarity,herewedropthesuperscriptk,whichindexesreplicates,andonlyconsiderthecaseofonereplicate.Sincetheposteriordistributionp(b;s;jy)cannotbecomputedinaclosedform,weuseEPtoapproximatethiscomplicatedposteriordistributi

8、onbyadistributionintheexponentialfamily.EPexploitsthefactthattheposteriorisaproductofsimpleterms.EPiterativelyrenestheapproximationofeachtermtoimprovetheapproximationoftheposterior.Mathemati-cally,EPa