slt12 Reproducing Kernel Hilbert Spaces and Kernel Methods

《slt12 Reproducing Kernel Hilbert Spaces and Kernel Methods》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、STAT598YStatisticalLearningTheoryInstructor:JianZhangLecture12:ReproducingKernelHilbertSpacesandKernelMethodsWefirstdefineHilbertspaceandthenintroducetheconceptofReproducingKernelHilbertSpace(RKHS)whichplaysanimportantroleinmachinelearning.Definition.AHilbertspaceisaninnerproductspa

2、cewhichisalsocompleteandseparable1withrespecttothenorm/distancefunctioninducedbytheinnerproduct.Foranyf,g∈Handα∈R,".,.#isaninnerproductifandonlyifitsatisfiesthefollowingconditions:1."f,g#="g,f#;2."f+g,h#="f,h#+"g,h#and"αf,g#=α"f,g#;3."f,f#≥0and"f,h#=0ifandonlyiff=0.!!Thenorm/dista

3、nceinducedbytheinnerproductisdefinedas%f%="f,f#and%f−g%="f−g,f−g#.".,.#iscalledasemi-innerproductifthethirdconditiononlysays"f,f#≥0.Inthiscase,theinducednormisactuallyasemi-norm.ExamplesofHilbertspaceincludes:1.Rnwith"a,b#=aTb;"∞2."2spaceofsquaresummablesequencewithinnerproduct"x,

4、y#=i=1xiyi;´3.ThespaceofL2squareintegrablefunctionswithinnerproduct"f,g#=f(x)g(x)dx.AclosedlinearsubspaceGofaHilbertspaceHisalsoaHilbertspace.Thedistancebetweenanelementf∈HandGisdefinedasinfg∈G%f−g%.SinceGisclosed,theinfimumcanbeattainedandwehavefG∈Gsuchthat%f−fG%=infg∈G%f−g%.Suchf

5、GiscalledtheprojectionoffontoG.Itcanbeshownthatsuchfisunique,and"f−f,g#=0forallg∈G.ThelinearsubspaceGc={f:"f,g#=0,∀g∈G}iscalledGGctheorthogonalcomplementofG.ItcanbeshownthatGisalsoclosedandf=fG+fGcforanyf∈H,cwherefGandfGcareprojectionsoffontoGandG.Thedecompositionf=fG+fGciscalled

6、atensorsumdecompositionandisdenotedbyH=G⊕Gc,Gc=H)GorG=H)Gc.AsimpleexampleofdecompositionwouldbeH=R2andG={(x,0):x∈R}andGc={(0,y):y∈R}.Anyelement(x,y)inHcanbedecomposedas(x,y)=(x,0)+(0,y)andthisdecompositionisunique.Theorem12-1(Riesz).ForeverycontniuouslinearfunctionalLinaHilbertsp

7、aceH,thereexistsauniquegL∈HsuchthatL(f)="gL,f#for∀f∈H.Proof.DefineNL={f:L(f)=0}tobethenullspaceofL.SinceLiscontinuouswehaveNLaclosedlinearsubspace.AssumeNL⊂Hthenthereexistsanonzeroelementg0∈H)NL.Wehave(L(f))g0−(L(g0))f∈NL,andthus"(L(f))g0,(L(g0))f,g0#=0.Thusweget#$L(g0)L(f)=g0,f."

8、g0,g0#HencewetakegL=(L(g0))g0/"g0,g0#.If