Ridge Regression

Ridge Regression

ID:39973780

大小:373.45 KB

页数:22页

时间:2019-07-16

Ridge Regression_第1页
Ridge Regression_第2页
Ridge Regression_第3页
Ridge Regression_第4页
Ridge Regression_第5页
资源描述:

《Ridge Regression》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库

1、RidgeregressionSelectionofRidgeregressioninR/SASRidgeRegressionPatrickBrehenySeptember1PatrickBrehenyBST764:AppliedStatisticalModeling1/22RidgeregressionDe nitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgeregression:De nitionAsmentionedinthepreviouslectu

2、re,ridgeregressionpenalizesthesizeoftheregressioncoecientsSpeci cally,theridgeregressionestimatebisde nedasthevalueofthatminimizesXXp(yxT)2+2iijij=1PatrickBrehenyBST764:AppliedStatisticalModeling2/22RidgeregressionDe nitionandsolutionSelectionofPropertiesRidgeregressi

3、oninR/SASRidgeregression:SolutionTheorem:Thesolutiontotheridgeregressionproblemisgivenbyb=(XTX+I)1XTyNotethesimilaritytotheordinaryleastsquaressolution,butwiththeadditionofaridge"downthediagonalridgeOLSCorollary:As!0,b!bridgeCorollary:As!1,b!0PatrickBrehenyBST764:App

4、liedStatisticalModeling3/22RidgeregressionDe nitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgeregression:Solution(Cont'd)Corollary:Inthespecialcaseofanorthonormaldesignmatrix,^OLS^ridge=JJ1+Thisillustratestheessentialfeatureofridgeregression:shrinkageApp

5、lyingtheridgeregressionpenaltyhasthee ectofshrinkingtheestimatestowardzero{introducingbiasbutreducingthevarianceoftheestimatePatrickBrehenyBST764:AppliedStatisticalModeling4/22RidgeregressionDe nitionandsolutionSelectionofPropertiesRidgeregressioninR/SASRidgevs.OLSinthep

6、resenceofcollinearityThebene tsofridgeregressionaremoststrikinginthepresenceofmulticollinearity,asillustratedinthefollowingexample:>x1<-rnorm(20)>x2<-rnorm(20,mean=x1,sd=.01)>y<-rnorm(20,mean=3+x1+x2)>lm(y~x1+x2)$coef(Intercept)x1x22.58206439.971344-38.040040>lm.ridge(y~x

7、1+x2,lambda=1)x1x22.62149980.99067730.8973912PatrickBrehenyBST764:AppliedStatisticalModeling5/22RidgeregressionDe nitionandsolutionSelectionofPropertiesRidgeregressioninR/SASInvertibilityRecallfromBST760thattheordinaryleastsquaresestimatesdonotalwaysexist;ifXisnotfullran

8、k,XTXisnotOLSinvertibleandthereisnouniquesolutionforbThisproblemdoesnotoccurwithridgeregression,

当前文档最多预览五页,下载文档查看全文

此文档下载收益归作者所有

当前文档最多预览五页,下载文档查看全文
温馨提示:
1. 部分包含数学公式或PPT动画的文件,查看预览时可能会显示错乱或异常,文件下载后无此问题,请放心下载。
2. 本文档由用户上传,版权归属用户,天天文库负责整理代发布。如果您对本文档版权有争议请及时联系客服。
3. 下载前请仔细阅读文档内容,确认文档内容符合您的需求后进行下载,若出现内容与标题不符可向本站投诉处理。
4. 下载文档时可能由于网络波动等原因无法下载或下载错误,付费完成后未能成功下载的用户请联系客服处理。