资源描述：
《Memory Networks》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、PublishedasaconferencepaperatICLR2015MEMORYNETWORKSJasonWeston,SumitChopra&AntoineBordesFacebookAIResearch770BroadwayNewYork,USA{jase,spchopra,abordes}@fb.comABSTRACTWedescribeanewclassoflearningmodelscalledmemorynetworks.Memorynetworksreasonwithinferencecomponentscombinedwithalong-
2、termmemorycomponent;theylearnhowtousethesejointly.Thelong-termmemorycanbereadandwrittento,withthegoalofusingitforprediction.Weinvestigatethesemodelsinthecontextofquestionanswering(QA)wherethelong-termmem-oryeffectivelyactsasa(dynamic)knowledgebase,andtheoutputisatextualresponse.Weev
3、aluatethemonalarge-scaleQAtask,andasmaller,butmorecomplex,toytaskgeneratedfromasimulatedworld.Inthelatter,weshowthereasoningpowerofsuchmodelsbychainingmultiplesupportingsentencestoan-swerquestionsthatrequireunderstandingtheintensionofverbs.1INTRODUCTIONMostmachinelearningmodelslacka
4、neasywaytoreadandwritetopartofa(potentiallyverylarge)long-termmemorycomponent,andtocombinethisseamlesslywithinference.Hence,theydonottakeadvantageofoneofthegreatassetsofamoderndaycomputer.Forexample,considerthetaskofbeingtoldasetoffactsorastory,andthenhavingtoanswerquestionsonthatsu
5、bject.Inprinciplethiscouldbeachievedbyalanguagemodelersuchasarecurrentneuralnetwork(RNN)(Mikolovetal.,2010;Hochreiter&Schmidhuber,1997)asthesemodelsaretrainedtopredictthenext(setof)word(s)tooutputafterhavingreadastreamofwords.However,theirmemory(en-codedbyhiddenstatesandweights)isty
6、picallytoosmall,andisnotcompartmentalizedenoughtoaccuratelyrememberfactsfromthepast(knowledgeiscompressedintodensevectors).RNNsareknowntohavedifficultyinperformingmemorization,forexamplethesimplecopyingtaskofoutputtingthesameinputsequencetheyhavejustread(Zaremba&Sutskever,2014).Thesi
7、tuationissimilarforothertasks,e.g.,inthevisionandaudiodomainsalongtermmemoryisrequiredtowatchamovieandanswerquestionsaboutit.arXiv:1410.3916v10[cs.AI]19May2015Inthiswork,weintroduceaclassofmodelscalledmemorynetworksthatattempttorectifythisproblem.Thecentralideaistocombinethesuccessf
8、ullearningstrategie
