Reasoning about Entailment with Neural Attention

《Reasoning about Entailment with Neural Attention》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、ReasoningaboutEntailmentwithNeuralAttentionTimRocktaschel¨EdwardGrefenstette&KarlMoritzHermannUniversityCollegeLondonGoogleDeepMindt.rocktaschel@cs.ucl.ac.ukfetg,kmhg@google.comToma´sKoˇciskˇy´PhilBlunsomUniversityofOxfordGoogleDeepMind&UniversityofOxfordtomas@kocisky.e

2、upblunsom@google.comAbstractAutomaticallyrecognizingentailmentrelationsbetweenpairsofnaturallanguagesentenceshassofarbeenthedominionofclassifiersemployinghandengineeredfeaturesderivedfromnaturallanguageprocessingpipelines.End-to-enddiffer-entiableneuralarchitectureshavef

3、ailedtoapproachstate-of-the-artperformanceuntilveryrecently.Inthispaper,weproposeaneuralmodelthatreadstwosen-tencestodetermineentailmentusinglongshort-termmemoryunits.Weextendthismodelwithaword-by-wordneuralattentionmechanismthatencouragesrea-soningoverentailmentsofpair

4、sofwordsandphrases.Furthermore,wepresentaqualitativeanalysisofattentionweightsproducedbythismodel,demonstratingsuchreasoningcapabilities.Onalargeentailmentdatasetthismodeloutperformsthepreviousbestneuralmodelandaclassifierwithengineeredfeaturesbyasub-stantialmargin.Itist

5、hefirstgenericend-to-enddifferentiablesystemthatachievesstate-of-the-artaccuracyonatextualentailmentdataset.1IntroductionTheabilitytodeterminethesemanticrelationshipbetweentwosentencesisanintegralpartofma-chinesthatunderstandandreasonwithnaturallanguage.Recognizingtextua

6、lentailment(RTE)isthetaskofdeterminingwhethertwonaturallanguagesentencesare(i)contradictingeachother,(ii)notrelated,orwhether(iii)thefirstsentence(calledpremise)entailsthesecondsentence(calledarXiv:1509.06664v1[cs.CL]22Sep2015hypothesis).Thistaskisimportantsincemanynatur

7、allanguageprocessing(NLP)problems,suchasinformationextraction,relationextraction,textsummarizationormachinetranslation,relyonitexplicitlyorimplicitlyandcouldbenefitfrommoreaccurateRTEsystems[Daganetal.,2006].State-of-the-artsystemsforRTEsofarreliedheavilyonengineeredNLPp

8、ipelines,extensiveman-ualcreationoffeatures,aswellasvariousexternalresourcesandspecializedsubcomponentssuchasn