欢迎来到天天文库
浏览记录
ID:40405117
大小:5.98 MB
页数:13页
时间:2019-08-01
《Teaching Machines to Read and Comprehend》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、TeachingMachinestoReadandComprehendKarlMoritzHermannyToma´sKoˇciskˇy´zEdwardGrefenstetteyLasseEspeholtyWillKayyMustafaSuleymanyPhilBlunsomyzyGoogleDeepMindzUniversityofOxfordfkmh,etg,lespeholt,wkay,mustafasul,pblunsomg@google.comtomas@kocisky.euAbstractTea
2、chingmachinestoreadnaturallanguagedocumentsremainsanelusivechal-lenge.Machinereadingsystemscanbetestedontheirabilitytoanswerquestionsposedonthecontentsofdocumentsthattheyhaveseen,butuntilnowlargescaletrainingandtestdatasetshavebeenmissingforthistypeofevalu
3、ation.Inthisworkwedefineanewmethodologythatresolvesthisbottleneckandprovideslargescalesupervisedreadingcomprehensiondata.Thisallowsustodevelopaclassofattentionbaseddeepneuralnetworksthatlearntoreadrealdocumentsandanswercomplexquestionswithminimalpriorknowle
4、dgeoflanguagestructure.1IntroductionProgressonthepathfromshallowbag-of-wordsinformationretrievalalgorithmstomachinesca-pableofreadingandunderstandingdocumentshasbeenslow.Traditionalapproachestomachinereadingandcomprehensionhavebeenbasedoneitherhandengineer
5、edgrammars[1],orinformationextractionmethodsofdetectingpredicateargumenttriplesthatcanlaterbequeriedasarelationaldatabase[2].Supervisedmachinelearningapproacheshavelargelybeenabsentfromthisspaceduetoboththelackoflargescaletrainingdatasets,andthedifficultyin
6、structuringstatisticalmodelsflexibleenoughtolearntoexploitdocumentstructure.Whileobtainingsupervisednaturallanguagereadingcomprehensiondatahasproveddifficult,someresearchershaveexploredgeneratingsyntheticnarrativesandqueries[3,4].Suchapproachesallowthegenera
7、tionofalmostunlimitedamountsofsuperviseddataandenableresearcherstoisolatetheperformanceoftheiralgorithmsonindividualsimulatedphenomena.WorkonsuchdatahasshownarXiv:1506.03340v1[cs.CL]10Jun2015thatneuralnetworkbasedmodelsholdpromiseformodellingreadingcompreh
8、ension,somethingthatwewillbuilduponhere.Historically,however,manysimilarapproachesinComputationalLinguisticshavefailedtomanagethetransitionfromsyntheticdatatorealenvironments,assuchclosedworldsinevitablyfailt
此文档下载收益归作者所有