Lecture 7_Statistical estimation英文学习材料

《Lecture 7_Statistical estimation英文学习材料》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、ConvexOptimization—Boyd&Vandenberghe7.Statisticalestimation•maximumlikelihoodestimation•optimaldetectordesign•experimentdesign7–1Parametricdistributionestimation•distributionestimationproblem:estimateprobabilitydensityp(y)ofarandomvariablefromobservedvalues•param

2、etricdistributionestimation:choosefromafamilyofdensitiespx(y),indexedbyaparameterxmaximumlikelihoodestimationmaximize(overx)logpx(y)•yisobservedvalue•l(x)=logpx(y)iscalledlog-likelihoodfunction•canaddconstraintsx∈Cexplicitly,ordefinepx(y)=0forx6∈C•aconvexoptimizat

3、ionproblemiflogpx(y)isconcaveinxforfixedyStatisticalestimation7–2LinearmeasurementswithIIDnoiselinearmeasurementmodelTyi=aix+vi,i=1,...,mn•x∈Risvectorofunknownparameters•viisIIDmeasurementnoise,withdensityp(z)mQmT•yiismeasurement:y∈Rhasdensitypx(y)=i=1p(yi−aix)max

4、imumlikelihoodestimate:anysolutionxofPmmaximizel(x)=logp(y−aTx)i=1ii(yisobservedvalue)Statisticalestimation7–3examples22−1/2−z2/(2σ2)•GaussiannoiseN(0,σ):p(z)=(2πσ)e,m1Xm2T2l(x)=−log(2πσ)−2(aix−yi)22σi=1MLestimateisLSsolution•Laplaciannoise:p(z)=(1/(2a))e−

5、z

6、/a,1

7、XmTl(x)=−mlog(2a)−

8、aix−yi

9、ai=1MLestimateisℓ1-normsolution•uniformnoiseon[−a,a]:−mlog(2a)

10、aTx−y

11、≤a,i=1,...,ml(x)=ii−∞otherwiseMLestimateisanyxwith

12、aTx−y

13、≤aiiStatisticalestimation7–4Logisticregressionrandomvariabley∈{0,1}withdistributionexp(aTu+b)p=prob(y=1)=1+exp

14、(aTu+b)n•a,bareparameters;u∈Rare(observable)explanatoryvariables•estimationproblem:estimatea,bfrommobservations(ui,yi)log-likelihoodfunction(fory1=···=yk=1,yk+1=···=ym=0):Ykexp(aTu+b)Ym1l(a,b)=logi1+exp(aTui+b)1+exp(aTui+b)i=1i=k+1XkXmTT=(aui+b)−log(1+exp(aui

15、+b))i=1i=1concaveina,bStatisticalestimation7–5example(n=1,m=50measurements)10.8=1)0.6y(0.4prob0.200246810u•circlesshow50points(ui,yi)•solidcurveisMLestimateofp=exp(au+b)/(1+exp(au+b))Statisticalestimation7–6(Binary)hypothesistestingdetection(hypothesistesting)pro

16、blemgivenobservationofarandomvariableX∈{1,...,n},choosebetween:•hypothesis1:Xwasgeneratedbydistributionp=(p1,...,pn)•hypothesis2:Xwasgeneratedbydistributionq=(