欢迎来到天天文库
浏览记录
ID:39973780
大小:373.45 KB
页数:22页
时间:2019-07-16
《Ridge Regression》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、RidgeregressionSelectionofRidgeregressioninR/SASRidgeRegressionPatrickBrehenySeptember1PatrickBrehenyBST764:AppliedStatisticalModeling1/22RidgeregressionDenitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgeregression:DenitionAsmentionedinthepreviouslectu
2、re,ridgeregressionpenalizesthesizeoftheregressioncoecientsSpecically,theridgeregressionestimatebisdenedasthevalueofthatminimizesXXp(y xT)2+2iijij=1PatrickBrehenyBST764:AppliedStatisticalModeling2/22RidgeregressionDenitionandsolutionSelectionofPropertiesRidgeregressi
3、oninR/SASRidgeregression:SolutionTheorem:Thesolutiontotheridgeregressionproblemisgivenbyb=(XTX+I) 1XTyNotethesimilaritytotheordinaryleastsquaressolution,butwiththeadditionofaridge"downthediagonalridgeOLSCorollary:As!0,b!bridgeCorollary:As!1,b!0PatrickBrehenyBST764:App
4、liedStatisticalModeling3/22RidgeregressionDenitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgeregression:Solution(Cont'd)Corollary:Inthespecialcaseofanorthonormaldesignmatrix,^OLS^ridge=JJ1+Thisillustratestheessentialfeatureofridgeregression:shrinkageApp
5、lyingtheridgeregressionpenaltyhastheeectofshrinkingtheestimatestowardzero{introducingbiasbutreducingthevarianceoftheestimatePatrickBrehenyBST764:AppliedStatisticalModeling4/22RidgeregressionDenitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgevs.OLSinthep
6、resenceofcollinearityThebenetsofridgeregressionaremoststrikinginthepresenceofmulticollinearity,asillustratedinthefollowingexample:>x1<-rnorm(20)>x2<-rnorm(20,mean=x1,sd=.01)>y<-rnorm(20,mean=3+x1+x2)>lm(y~x1+x2)$coef(Intercept)x1x22.58206439.971344-38.040040>lm.ridge(y~x
7、1+x2,lambda=1)x1x22.62149980.99067730.8973912PatrickBrehenyBST764:AppliedStatisticalModeling5/22RidgeregressionDenitionandsolutionSelectionofPropertiesRidgeregressioninR/SASInvertibilityRecallfromBST760thattheordinaryleastsquaresestimatesdonotalwaysexist;ifXisnotfullran
8、k,XTXisnotOLSinvertibleandthereisnouniquesolutionforbThisproblemdoesnotoccurwithridgeregression,
此文档下载收益归作者所有