rademacher and gaussian complexities risk bounds and structural results

《rademacher and gaussian complexities risk bounds and structural results》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、Radema herandGaussianComplexities:RiskBoundsandStru turalResults12eter.BartlettandShaharendelson1B
wulfTe hnologies2030AddisonStreet,Suite102Berkeley,CA94704,USAbartlettbarnhillte hnologies. om2Resear hS hoolof
nformationS ien esandEngineeringAustralianationalUnivers

2、ityCanberra0200,Australiashahar sl.anu.edu.auAbstra t.Weinvestigatetheuseof ertaindata-dependentestimatesofthe omplexityofafun tion lass, alledRadema herandgaussian omplexities.
nade isiontheoreti setting,weprovegeneralriskboundsintermsofthese omplexities.We onsiderfun ti

3、on lassesthat anbeexpressedas ombinationsoffun tionsfrombasis lassesandshowhowtheRadema herandgaussian omplexitiesofsu hafun tion lass anbeboundedintermsofthe omplexityofthebasis lasses.Wegiveexam-plesoftheappli ationofthesete hniquesinndingdata-dependentriskboundsforde i

4、siontrees,neuralnetworksandsupportve torma hines.1
ntrodu tion
nlearningproblemslikepattern lassi ationandregression,a onsiderableamountofeorthasbeenspentonobtaininggooderrorbounds.Theseareuseful,forexample,fortheproblemofmodelsele tion

5、 hoosingamodelofsuitable omplexity

6、.Typi ally,su hboundstaketheformofasumoftwoterms:somesample-basedestimateofperforman eandapenaltytermthatislargeformore omplexmodels.Forexample,inpattern lassi ation,thefollowingtheoremisanimprovementofa lassi alresultofVapnikandChervonenkis[20℄.Theorem1.etFbea lassoff1

7、g-valuedfun tionsdenedonasetX.etbeaprobabilitydistributiononXf1g,andsupposethat(X;Y);:::;(X;Y)11nnand(X;Y)are hosenindependentlya ordingto.Then,thereisanabsoluteonstantsu hthatforanyintegern,withprobabilityatleast1Æoversamplesoflengthn,everyfinFsatisesrVCdim(F)^(Y

8、6=f(X))(Y6=f(X))+;nn^whereVCdim(F)denotestheVapnik-ChervonenkisdimensionofF,(S)=nn(1=n)1(X;Y),and1istheindi atorfun tionofS,SiiSi=12eterBartlettandShaharendelson
nthis ase,thesample-basedestimateofperforman eistheproportionofexamplesinthetrainingsamplethataremis lass

9、iedbythefun tionf,andthe omplexitypenaltyterminvolvestheVC-dimensionofthe lassoffun tion