Teaching Machines to Read and Comprehend

《Teaching Machines to Read and Comprehend》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、TeachingMachinestoReadandComprehendKarlMoritzHermannyToma´sKoˇciskˇy´zEdwardGrefenstetteyLasseEspeholtyWillKayyMustafaSuleymanyPhilBlunsomyzyGoogleDeepMindzUniversityofOxfordfkmh,etg,lespeholt,wkay,mustafasul,pblunsomg@google.comtomas@kocisky.euAbstractTea

2、chingmachinestoreadnaturallanguagedocumentsremainsanelusivechal-lenge.Machinereadingsystemscanbetestedontheirabilitytoanswerquestionsposedonthecontentsofdocumentsthattheyhaveseen,butuntilnowlargescaletrainingandtestdatasetshavebeenmissingforthistypeofevalu

3、ation.Inthisworkwedefineanewmethodologythatresolvesthisbottleneckandprovideslargescalesupervisedreadingcomprehensiondata.Thisallowsustodevelopaclassofattentionbaseddeepneuralnetworksthatlearntoreadrealdocumentsandanswercomplexquestionswithminimalpriorknowle

4、dgeoflanguagestructure.1IntroductionProgressonthepathfromshallowbag-of-wordsinformationretrievalalgorithmstomachinesca-pableofreadingandunderstandingdocumentshasbeenslow.Traditionalapproachestomachinereadingandcomprehensionhavebeenbasedoneitherhandengineer

5、edgrammars[1],orinformationextractionmethodsofdetectingpredicateargumenttriplesthatcanlaterbequeriedasarelationaldatabase[2].Supervisedmachinelearningapproacheshavelargelybeenabsentfromthisspaceduetoboththelackoflargescaletrainingdatasets,andthedifficultyin

6、structuringstatisticalmodelsflexibleenoughtolearntoexploitdocumentstructure.Whileobtainingsupervisednaturallanguagereadingcomprehensiondatahasproveddifficult,someresearchershaveexploredgeneratingsyntheticnarrativesandqueries[3,4].Suchapproachesallowthegenera

7、tionofalmostunlimitedamountsofsuperviseddataandenableresearcherstoisolatetheperformanceoftheiralgorithmsonindividualsimulatedphenomena.WorkonsuchdatahasshownarXiv:1506.03340v1[cs.CL]10Jun2015thatneuralnetworkbasedmodelsholdpromiseformodellingreadingcompreh

8、ension,somethingthatwewillbuilduponhere.Historically,however,manysimilarapproachesinComputationalLinguisticshavefailedtomanagethetransitionfromsyntheticdatatorealenvironments,assuchclosedworldsinevitablyfailt