2[Sep 3]Regression [Fabian Wauthier]

《2[Sep 3]Regression [Fabian Wauthier]》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、RegressionPracticalMachineLearningFabianWauthier09/10/2009AdaptedfromslidesbyKurtMillerandRomainThibaux1Outline•OrdinaryLeastSquaresRegression-Onlineversion-Normalequations-Probabilisticinterpretation•OverfittingandRegularization•Overviewofadditionaltopics-L1Regression-QuantileRe

2、gression-Generalizedlinearmodels-KernelRegressionandLocallyWeightedRegression2Outline•OrdinaryLeastSquaresRegression-Onlineversion-Normalequations-Probabilisticinterpretation•OverfittingandRegularization•Overviewofadditionaltopics-L1Regression-QuantileRegression-Generalizedlinear

3、models-KernelRegressionandLocallyWeightedRegression3Regressionvs.Classification:ClassificationX⇒YAnything:•Discrete:•continuous(ℜ,ℜd,…)–{0,1}binary•discrete({0,1},{1,…k},…)–{1,…k}!multi-class•structured(tree,string,…)–tree,etc.structured•…4Regressionvs.Classification:Classificati

4、onX⇒YPerceptronAnything:LogisticRegression•continuous(ℜ,ℜd,…)SupportVectorMachine•discrete({0,1},{1,…k},…)DecisionTreeRandomForest•structured(tree,string,…)•…Kerneltrick5Regressionvs.Classification:RegressionX⇒YAnything:•continuous:d•continuous(ℜ,ℜd,…)–ℜ,ℜ•discrete({0,1},{1,…k},…

5、)•structured(tree,string,…)•…6Examples•Voltage⇒Temperature•Processes,memory⇒Powerconsumption•Proteinstructure⇒Energy•Robotarmcontrols⇒Torqueateffector•Location,industry,pastlosses⇒Premium7LinearregressionGivenexamplesPredictgivenanewpointy40y262420222030402030010200102010x0x08Lin

6、earregressionWewishtoestimatebyalinearfunctionofourdata:yˆxyˆn+1=w0+w1xn+1,1+w2xn+1,2!=wxn+1wherewisaparametertobeestimatedandwehaveusedthestandardconventionoflettingthefirstcomponentofxbe1.y40y262420222030402030010200102010x0x09ChoosingtheregressorOfthemanyregressionfitsthatappr

7、oximatethedata,whichshouldwechoose?Observation!"1Xi=xi00201010LMSAlgorithm(LeastMeanSquares)Inordertoclarifywhatwemeanbyagoodchoiceof,wewillwdefineacostfunctionforhowwellwearedoingonthetrainingdata:Erroror“residual”ObservationPrediction!"1Xi=xi00201!n!2Cost=(wxi−yi)2i=111LMSAlgor

8、ithm(LeastMeanSquares)Thebestchoiceofist