资源描述：
《lecun-20151022-baylearn.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、YLeCunObstaclesOnthepathtoAI(or“howIlearnedtostopworryingandloveunsupervisedlearning”)YannLeCunFacebookAIResearch&CenterforDataScience,NYUyann@cs.nyu.eduhttp://yann.lecun.comOr:YLeCunHowdoweincorporateReasoning/PlanningwithRepresentationlearning?YannLeCunFacebookAIResearch&CenterforDa
2、taScience,NYUyann@cs.nyu.eduhttp://yann.lecun.comOr:YLeCunHowdoweincorporateEpisodicMemorywithRepresentationlearning?YannLeCunFacebookAIResearch&CenterforDataScience,NYUyann@cs.nyu.eduhttp://yann.lecun.comAndYLeCunHowtheHelldowedoUnsupervisedLearning?YannLeCunFacebookAIResearch&Center
3、forDataScience,NYUyann@cs.nyu.eduhttp://yann.lecun.comWait,whataboutreinforcementlearning?YLeCunMeh,it'sthecherryonthecake.Thissoundsliketrolling.Ok,justalittlebit.ButthereisnowayinhellthatyoucanlearnbillionsofparameterswithRL.–Atleastnotwithinareasonableamountoftime.–Onescalarrewardp
4、ertrialisn'tgoingtocutit.YLeCunGeometryoftheLossFunctionDeepNetswithReLUs:ObjectiveFunctionisPiecewisePolynomialYLeCunIfweuseahingeloss,deltanowdependsonlabelYk:L(W)=∑C(X,Y,W)(∏W)31pijP(ij)∈PPiecewisepolynomialinWwithrandomW31,22coefficientsAlotisknownaboutthedistributionofcritical22p
5、ointsofpolynomialsonthespherewithrandom(Gaussian)coefficients[BenArousetal.]W22,14High-ordersphericalspinglasses14RandommatrixtheoryW14,33Z3YLeCunReasoning&RepresentationLearningReasoningasEnergyMinimization(structuredprediction++)YLeCunDeepLearningsystemscanbeassembledintoF(X,Y)=Marg
6、_zE(X,Y,Z)energymodelsAKAfactorgraphsEnergyfunctionisasumoffactorsFactorscanembedwholedeeplearningE(X,Y,Z)systemsX:observedvariables(inputs)EnergyModelZ:neverobserved(latentvariables)EnergyModel(factorgraph)(factorgraph)Y:observedontrainingset(outputvariables)Inferenceisenergyminimiza
7、tion(MAP)orfreeZenergyminimization(marginalization)overZ(unobserved)andYgivenanXF(X,Y)=MIN_zE(X,Y,Z)XYF(X,Y)=-logSUM_zexp[-E(X,Y,Z)](observed)(observedontrainingset)Energy-BasedLearning[LeCunetal.2006]YLeCunPushdownontheenergyofdesiredoutputsPushuponeverythingelse[LeCunetal2006]“Atuto
8、rialo
