Approximation in Learning Theory

《Approximation in Learning Theory》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、Constr.Approx.(2008)27:33–74DOI:10.1007/s00365-006-0655-2CONSTRUCTIVEAPPROXIMATION©2007SpringerScience+BusinessMedia,Inc.ApproximationinLearningTheoryV.N.TemlyakovAbstract.ThispaperaddressessomeproblemsofsupervisedlearninginthesettingformulatedbyCuckerandSma

2、le.Supervisedlearning,orlearning-from-examples,referstoaprocessthatbuildsonthebaseofavailabledataofinputsxiandoutputsyi,i=1,...,m,afunctionthatbestrepresentstherelationbetweentheinputsx∈Xandthecorrespondingoutputsy∈Y.Thegoalistofindanestimatorfzonthebaseofgiv

3、endataz:=((x1,y1),...,(xm,ym))thatapproximateswelltheregressionfunctionfρ(oritsprojection)ofanunknownBorelprobabilitymeasureρdefinedonZ=X×Y.Weassumethat(xi,yi),i=1,...,m,areindependentanddistributedaccordingtoρ.Wediscussthefollowingtwoproblems:I.theprojection

4、learningproblem(improperfunctionlearningproblem);II.universal(adaptive)estimatorsintheproperfunctionlearningproblem.InthefirstproblemwedonotimposeanyrestrictionsonaBorelmeasureρexceptourstandardassumptionthat

5、y

6、≤Ma.e.withrespecttoρ.Inthiscaseweusethedataztoes

7、timate(approximate)theL2(ρX)projection(fρ)WoffρontoafunctionclassWofourchoice.Here,ρXisthemarginalprobabilitymeasure.In[KT1,2]thisproblemhasbeenstudiedforWsatisfyingthedecayconditionεn(W,B)≤Dn−roftheentropynumbersεn(W,B)ofWinaBanachspaceBinthecaseB=C(X)orB=L

8、2(ρX).Inthispaperweobtaintheupperestimatesinthecaseεn(W,L1(ρX))≤Dn−rwithanextraassumptionthatWisconvex.Inthesecondproblemweassumethatanunknownmeasureρsatisfiessomecon-ditions.Followingthestandardwayfromnonparametricstatisticsweformulatetheseconditionsofthefor

9、mfρ∈.Next,weassumethattheonlyaprioriinformationavail-ableisthatfρbelongstoaclass(unknown)fromaknowncollection{}ofclasses.Wewanttobuildanestimatorthatprovidesapproximationoffρclosetotheoptimalfortheclass.Alongwithstandardpenalizedleastsquaresestimatorswec

10、onsideranewmethodofconstructionofuniversalestimators.Thismethodisbasedonacombinationoftwopowerfulideasinbuildinguniversalestimators.Thefirstoneistheuseofpenal-izedleastsquaresestimators.Thisideaw