Blunsom - Natural Language Processing Language Modelling and Machine Translation - DLSS 2017

《Blunsom - Natural Language Processing Language Modelling and Machine Translation - DLSS 2017》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、NaturalLanguageProcessing,LanguageModellingandMachineTranslationPhilBlunsomincollaborationwiththeDeepMindNaturalLanguageGrouppblunsom@google.comNaturalLanguageProcessingLinguisticsWhyarehumanlanguagesthewaythattheyare?Howdoesthebrainmapfromrawlinguisticinputtomeaningandbackagain?

2、Andhowdochildrenlearnlanguagesoquickly?ComputationalLinguisticsComputationalmodelsoflanguageandcomputationaltoolsforstudyinglanguage.NaturalLanguageProcessingBuildingtoolsforprocessinglanguageandapplicationsthatuselanguage:Intrinsic:Parsing,LanguageModelling,etc.Extrinsic:ASR,M

3、T,QA/Dialogue,etc.LanguagemodelsAlanguagemodelassignsaprobabilitytoasequenceofwords,Psuchthatw2p(w)=1:Giventheobservedtrainingtext,howprobableisthisnewutterance?Thuswecancomparedierentorderingsofwords(e.g.Translation):p(helikesapples)>p(appleslikeshe)orchoiceofwords(e.g.Speech

4、Recognition):p(helikesapples)>p(helicksapples)History:cryptographyLanguagemodelsMuchofNaturalLanguageProcessingcanbestructuredas(conditional)languagemodelling:Translationplm(LeschiensaimentlesosjjjDogslovebones)QuestionAnsweringplm(Whatdodogslove?jjjbones.j)Dialogueplm(Howareyou?

5、jjjFinethanks.Andyou?j)LanguagemodelsMostlanguagemodelsemploythechainruletodecomposethejointprobabilityintoasequenceofconditionalprobabilities:p(w1;w2;w3;:::;wN)=p(w1)p(w2jw1)p(w3jw1;w2):::p(wNjw1;w2;:::wN1)Notethatthisdecompositionisexactandallowsustomodelcomplexjointdistribu

6、tionsbylearningconditionaldistributionsoverthenextword(wn)giventhehistoryofwordsobserved(w1;:::;wn1).LanguagemodelsThesimpleobjectiveofmodellingthenextwordgiventheobservedhistorycontainsmuchofthecomplexityofnaturallanguageunderstanding.Considerpredictingtheextensionoftheutteranc

7、e:p(
jThereshebuilta)Withmorecontextweareabletouseourknowledgeofbothlanguageandtheworldtoheavilyconstrainthedistributionoverthenextword:p(
jAlicewenttothebeach.Thereshebuilta)Thereisevidencethathumanlanguageacquisitionpartlyreliesonfutureprediction.EvaluatingaLanguageModelAgoodmo

8、delassignsrealutteranceswNfromalanguagea