资源描述:
《rademacher and gaussian complexities risk bounds and structural results》由会员上传分享,免费在线阅读,更多相关内容在教育资源-天天文库。
1、Radema
herandGaussianComplexities:RiskBoundsandStru
turalResults12eter.BartlettandShaharendelson1BwulfTe
hnologies2030AddisonStreet,Suite102Berkeley,CA94704,USAbartlettbarnhillte
hnologies.
om2Resear
hS
hoolofnformationS
ien
esandEngineeringAustralianationalUnivers
2、ityCanberra0200,Australiashahar
sl.anu.edu.auAbstra
t.Weinvestigatetheuseof
ertaindata-dependentestimatesofthe
omplexityofafun
tion
lass,
alledRadema
herandgaussian
omplexities.nade
isiontheoreti
setting,weprovegeneralriskboundsintermsofthese
omplexities.We
onsiderfun
ti
3、on
lassesthat
anbeexpressedas
ombinationsoffun
tionsfrombasis
lassesandshowhowtheRadema
herandgaussian
omplexitiesofsu
hafun
tion
lass
anbeboundedintermsofthe
omplexityofthebasis
lasses.Wegiveexam-plesoftheappli
ationofthesete
hniquesinndingdata-dependentriskboundsforde
i
4、siontrees,neuralnetworksandsupportve
torma
hines.1ntrodu
tionnlearningproblemslikepattern
lassi
ationandregression,a
onsiderableamountofeorthasbeenspentonobtaininggooderrorbounds.Theseareuseful,forexample,fortheproblemofmodelsele
tion
5、
hoosingamodelofsuitable
omplexity
6、.Typi
ally,su
hboundstaketheformofasumoftwoterms:somesample-basedestimateofperforman
eandapenaltytermthatislargeformore
omplexmodels.Forexample,inpattern
lassi
ation,thefollowingtheoremisanimprovementofa
lassi
alresultofVapnikandChervonenkis[20℄.Theorem1.etFbea
lassoff1
7、g-valuedfun
tionsdenedonasetX.etbeaprobabilitydistributiononXf1g,andsupposethat(X;Y);:::;(X;Y)11nnand(X;Y)are
hosenindependentlya
ordingto.Then,thereisanabsoluteonstantsu
hthatforanyintegern,withprobabilityatleast1 Æoversamplesoflengthn,everyfinFsatisesrVCdim(F)^(Y
8、6=f(X))(Y6=f(X))+;nn^whereVCdim(F)denotestheVapnik-ChervonenkisdimensionofF,(S)=nn(1=n)1(X;Y),and1istheindi
atorfun
tionofS,SiiSi=12eterBartlettandShaharendelsonnthis
ase,thesample-basedestimateofperforman
eistheproportionofexamplesinthetrainingsamplethataremis
lass
9、iedbythefun
tionf,andthe
omplexitypenaltyterminvolvestheVC-dimensionofthe
lassoffun
tion