class03_rkhs Reproducing Kernel Hilbert Spaces

《class03_rkhs Reproducing Kernel Hilbert Spaces》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ReproducingKernelHilbertSpacesLorenzoRosasco9.520Class03February12,2007L.RosascoRKHSAboutthisclassGoalTointroduceaparticularlyusefulfamilyofhypothesisspacescalledReproducingKernelHilbertSpaces(RKHS)andtoderivethegeneralsolutionofTikhonovregularizationinRKHS.L.RosascoRKHSFunctionApproxim

2、ationfromSamplesHereisagraphicalexampleforgeneralization:givenacertainnumberofsamples...f(x)xL.RosascoRKHSFunctionApproximationfromSamples(cont.)Supposethisisthe“true”solution...f(x)xL.RosascoRKHSTheProblemisIll-Posed...butsupposeERMgivesthissolution!f(x)xL.RosascoRKHSRegularizationTheb

3、asicideaofregularization(originallyintroducedindependentlyofthelearningproblem)istorestorewell-posednessofERMbyconstrainingthehypothesisspaceH.PenalizedMinimizationApossiblewaytodothisisconsideringpenalizedempiricalriskminimization,thatiswelookforsolutionsminimizingatwotermfunctionalERR

4、(f)+λpen(f)

5、{z}

6、{z}empiricalerrorpenalizationtermtheregularizationparameterλtrade-offsthetwoterms.L.RosascoRKHSTikhonovRegularizationTikhonovregularizationamountstominimizeXn12V(f(xi),yi)+λkfkH,λ>0(1)ni=1V(f(x),y)isthelossfunction,thatisthepricewepaywhenwepredictf(x)inplaceofyk·kHisthen

7、orminthefunctionspaceHSuchapenalizationtermshouldencodesomenotionofsmoothnessoff.L.RosascoRKHSThe"Ingredients"ofTikhonovRegularizationTheschemewejustdescribedisverygeneralandbychoosingdifferentlossfunctionsV(f(x),y)wecanrecoverdifferentalgorithmsThemainpointwewanttodiscussishowtochoosea

8、normencodingsomenotionofsmoothness/complexityofthesolutionReproducingKernelHilbertSpacesallowustodothisinaverypowerfulwayL.RosascoRKHSSomeFunctionalAnalysisAfunctionspaceFisaspacewhoseelementsarefunctionsf,forexamplef:Rd→R.Anormisanonnegativefunctionk·ksuchthat∀f,g∈Fandα∈R1kfk≥0andkfk=0

9、ifff=0;2kf+gk≤kfk+kgk;3kαfk=

10、α

11、kfk.pAnormcanbedefinedviaadotproductkfk=hf,fi.AHilbertspace(besidesothertechnicalconditions)isa(possibly)infinitedimensionallinearspaceendowedwithadotproduct.L.RosascoRKHSSomeFunctionalAnalysisAfunctionspaceFisaspacewhoseelementsarefunctionsf,forexa