elements of statistical learning sol1

《elements of statistical learning sol1》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ASolutionManualandNotesfortheText:TheElementsofStatisticalLearningbyJeromeFriedman,TrevorHastie,andRobertTibshiraniJohnL.Weatherwax∗December15,2009∗wax@alum.mit.edu1Chapter2(OverviewofSupervisedLearning)NotesontheTextStatisticalDecisionTheoryOurexpectedpredictederror(EPE)underthesquarederrorlossand

2、assumingalinearmodelforyi.e.y=f(x)≈xTβisgivenbyZT2EPE(β)=(y−xβ)Pr(dx,dy).(1)Consideringthisafunctionofthecomponentsofβi.e.βitominimizethisexpressionwithrespecttoβiwetaketheβiderivative,settheresultingexpressionequaltozeroandsolveforβi.TakingthevectorderivativewithrespecttothevectorβweobtainZZ∂EPETT

3、=2(y−xβ)(−1)xPr(dx,dy)=−2(y−xβ)xPr(dx,dy).(2)∂βNowthisexpressionwillcontaintwoparts.ThefirstwillhavetheintegrandyxandthesecondwillhavetheintegrandxTβx.Thislatterexpressionintermsofitscomponentsisgivenbyx0x1Txxβx=(x0β0+x1β1+x2β2+···+xpβp)2...xpx0x0β0+x0x1β1+x0x2β2+...+x0xpβpx1x0β0+x

4、1x1β1+x1x2β2+...+x1xpβpT=..=xxβ..xpx0β0+xpx1β1+xpx2β2+...+xpxpβpSowiththisrecognition,thatwecanwritexTβxasxxTβ,weseethattheexpression∂EPE=0∂βgivesTE[yx]−E[xxβ]=0.(3)Sinceβisaconstant,itcanbetakenoutoftheexpectationtogiveT−1β=E[xx]E[yx],(4)whichgivesaverysimplederivationofequation2.16inthebook

5、.Notesincey∈Randx∈Rpweseethatxandycommutei.e.xy=yx.ExerciseSolutionsEx.2.1(targetcoding)IfeachofoursamplesfromKclassesiscodedasatargetvectortkwhichhasaoneinthekthspot.Thenonewayofdevelopingaclassifierisbyregressingtheindependentvariablesontothetargetvectorstk.Thenourclassificationprocedurewouldthenbe

6、comethefollowing.GiventhemeasurementvectorX,predictatargetvectorˆyvialinearregressionandtoselecttheclasskcorrespondingtothecomponentofˆywhichhasthelargestvalue.Thatisk=argmaxi(ˆyi).Nowconsidertheexpressionargmink

7、

8、yˆ−tk

9、

10、,whichfindstheindexofthetargetvectorthatisclosesttotheproducedregressionoutputˆ

11、y.Byexpandingthequadraticwefindthat2argmink

12、

13、yˆ−tk

14、

15、=argmink

16、

17、yˆ−tk

18、

19、XK2=argmink(ˆyi−(tk)i)i=1XK
22=argmink(ˆyi)−2ˆyi(tk)i+(tk)ii=1XK
2=argmink−2ˆyi(tk)i+(tk)i,i=1PsincethesumKyˆ2isthesameforallclasseskand