资源描述：
《r for additive model》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、AdditiveModels36-350,DataMining,Fall20092November2009Readings:PrinciplesofDataMining,pp.393{395;Berk,ch.2.Contents1PartialResidualsandBack ttingforLinearModels12AdditiveModels33TheCurseofDimensionality44Example:CaliforniaHousePricesRevisited71PartialRes
2、idualsandBack ttingforLinearModelsThegeneralformofalinearregressionmodelishiXpEYjX~=~x=0+~~x=jxj(1)j=0whereforj21:p,thexjarethecomponentsof~x,andx0isalwaystheconstant1.(Addinga ctitiousconstantfeature"likethisisastandardwayofhandlingtheinterceptjustli
3、keanyotherregressioncoecient.)Supposewedon'tconditiononallofX~butjustonecomponentofit,sayXk.WhatistheconditionalexpectationofY?E[YjXk=xk]=E[E[YjX1;X2;:::Xk;:::Xp]jXk=xk](2)23Xp=E4jXjjXk=xk5(3)j=023X=kxk+E4jXjjXk=xk5(4)j6=k1wherethe rstlineusesthelawoft
4、otalexpectation1,andthesecondlineusesEq.1.Turnedaround,23Xkxk=E[YjXk=xk] E4jXjjXk=xk5(5)j6=k2013X=E4Y @jXjAjXk=xk5(6)j6=kTheexpressionintheexpectationisthekthpartialresidual
5、the(total)residualisthedi erencebetweenYanditsexpectation,thepartialresidualist
6、hedi erencebetweenYandwhatweexpectittobeignoringthecontributionfromX.Let'sintroduceasymbolforthis,sayY(k).khix=EY(k)jX=x(7)kkkkInwords,iftheover-allmodelislinear,thenthepartialresidualsarelinear.AndnoticethatXkistheonlyinputfeatureappearinghere
7、ifwecoul
8、dsomehowgetholdofthepartialresiduals,thenwecan ndkbydoingasimpleregression,ratherthanamultipleregression.Ofcoursetogetthepartialresidualweneedtoknowalltheotherjs...Thissuggeststhefollowingestimationschemeforlinearmodels,knownastheGauss-Seidelalgorithm,o
9、rmorecommonlyandtransparentlyasback- tting;thepseudo-codeisinExample1.Thisisaniterativeapproximationalgorithm.Initially,welookathowfareachpointisfromtheglobalmean,anddosimpleregressionsofthosedeviationsontheinputfeatures.Thisthengivesusabetterideaofwhat
10、theregressionsurfacereallyis,andweusethedeviationsfromthatsurfaceinournextsetofsimpleregressions.Ateachstep,eachcoecientisadjustedto tinwithwhatwealreadyknowabouttheothercoecients
11、that'swhyit'scalledback tting".Itisnotobvious2
