Maximum_Likelihood_Estimation.pdf

《Maximum_Likelihood_Estimation.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、MaximumLikelihoodEstimationBUFN758NProf.SkoulakisBUFN758N(Prof.Skoulakis)MaximumLikelihoodEstimation1/12Example:CoinTossingConsidertossinga(notnecessarilyfair)coinNtimesTheprobabilityoftailsisp=P[T]whiletheprobabilityofheadsisq=P[H]=1pTheparameterofinte

2、restispSupposethatthecoinistossed10timesandweobserve3tailsSupposetherearetwocandidatesfortheparameterptobeconsidered:1and2.Whichoneismorereasonable?33BUFN758N(Prof.Skoulakis)MaximumLikelihoodEstimation2/12Example:CoinTossing(cont'd)Idea:selectthevalueofp

3、underwhichtheobservedoutcomeismorelikely.LetXbethenumberoftailsinNtrials.TherandomvariableXtakesvalues0;1;:::;NandfollowsthebinomialdistributionwithprobabilityfunctionNkNkP[X=k]=p(1p);k=0;:::;N:kInourexample,ifp=1thentheobservedoutcomehas

3
37
7

4、probability1012=102.3333310Moreover,ifp=2thentheobservedoutcomehasprobability

3
73
31021=102.3333310BUFN758N(Prof.Skoulakis)MaximumLikelihoodEstimation3/12Example:CoinTossing(cont'd)Hence,theobservedoutcomeismorelikelyunderp=1,which3weconcludetobeth

5、emorereasonableselectionforp.But,wedonothavetofocusonjusttwopossibilities.Followingthesamelogic,wecanaskwhatvalueofpmakestheobservedoutcomemostlikely.Inotherwords,whatvalueofpmaximizes
L(p)=10p3(1p)7?3Sincethelogarithmicfunctionisstrictlyincreasing,and

6、ignoring
10theconstantterm,itsucestomaximize337F(p)=logp(1p)=3log(p)+7log(1p)BUFN758N(Prof.Skoulakis)MaximumLikelihoodEstimation4/12Example:CoinTossing(cont'd)ThederivativeofF(p)is011310pF(p)=37=p1pp(1p)SettingthederivativeF0(p)equalto0,weobtai

7、ntheestimatep^=3.10Ingeneral,ifweobservektailsinNtrialsthentheprobabilityof
theobservedoutcomeisL(p)=Npk(1p)Nk.Tomaximizekthisprobability(likelihood),weneedtomaximizeF(p)=klog(p)+(Nk)log(1p)SettingF0(p)=0,kNp=0,weobtaintheestimator^p=k.p(1p)NBUFN7

8、58N(Prof.Skoulakis)MaximumLikelihoodEstimation5/12MaximumLikelihoodEstimator(MLE)Ingeneral,theprobabilityoftheobserveddata,sayX1;:::;XT,dependsontheunderlyingparameter,say.Thisprobabilityisdenotedbyp(X1;:::;XTj).Viewedas