机器学习与概率图模型_王立威

《机器学习与概率图模型_王立威》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、MachineLearningandGraphicalModels(LectureI)王立威北京大学信息科学技术学院http://www.cis.pku.edu.cn/faculty/vision/wangliwei/wanglw@cis.pku.edu.cn1OutlineAbriefoverviewofMachineLearningGraphicalModels•Representation•Inference•Learning2DefinitionofMachineLearning:•Learningfromexperiences.“Acompute

2、rprogramissaidtolearnfromexperienceEwithrespecttosomeclassoftasksTandperformancemeasureP,ifitsperformanceattasksinT,asmeasuredbyP,improveswithexperienceE.”-TomMitchell3“Classical”MachineLearningTasks:•Classification:nf:R{1,1}spamfilter,facerecognition,…n•Regressionf:RRHook’slaw

3、,Kepler’slaw,…n•Rankingf:RRSearchengine•Probability(Distribution)Estimation4“Classical”MachineLearningAlgorithms•ClassificationSVMBoostingRandomForestBagging(Deep)NeuralNetworks•RegressionLassoBoosting5SupportVectorMachines(SVMs)SVM:thelargel/lmarginclassifier22SVM:hingel

4、ossminimization+regularizationBoostingBoosting:(implicit)largemarginl/lclassifier1Boosting:explossminimization(+regularization)“Classical”MachineLearningTheories•VCtheoryCapacityofthehypothesisspace•PAC-theory•MargintheoryConfidence•EmpiricalProcessesCapacity•PAC-BayestheoryPACin

5、Bayesframework•RegularizationCapacity,smoothness8MLtheories:QuantificationofOccam’sRazorLengthHook’slawForceComparisonof“Classical”MachineLearningTheories•Regularization:BayesianoptimalityOnlyasymptotic(convergence,rate,non-uniform)•VC/PAC,Margin,PAC-Bayes,…Relativeoptimality(opt

6、imalinahypothesisspace)Non-asymptotic(finitesamplebounds)10Limitationsofthe“Classical”ML•RepresentationEuclideanrepresentationforinput.Simplerepresentationforoutput.HowtorepresentSTRUCTURESindata?11OutlineAbriefoverviewofMachineLearningGraphicalModels•Representation•Inference•

7、Learning12ChapterI:Representation13ProbabilisticGraphicalModels:WhatandWhy•PGMs:Amodelforjointprobabilitydistributionoverrandomvariables.Representdependenciesandindependenciesbetweentherandomvariables.•Whyisprobabilitydistributionimportant?Genesanddiseases,andeverything•WhyPGMwas

8、inventedbyco