欢迎来到天天文库
浏览记录
ID:40352345
大小:6.66 MB
页数:91页
时间:2019-07-31
《Blunsom - Natural Language Processing Language Modelling and Machine Translation - DLSS 2017》由会员上传分享,免费在线阅读,更多相关内容在学术论文-天天文库。
1、NaturalLanguageProcessing,LanguageModellingandMachineTranslationPhilBlunsomincollaborationwiththeDeepMindNaturalLanguageGrouppblunsom@google.comNaturalLanguageProcessingLinguisticsWhyarehumanlanguagesthewaythattheyare?Howdoesthebrainmapfromrawlinguisticinputtomeaningandbackagain?
2、Andhowdochildrenlearnlanguagesoquickly?ComputationalLinguisticsComputationalmodelsoflanguageandcomputationaltoolsforstudyinglanguage.NaturalLanguageProcessingBuildingtoolsforprocessinglanguageandapplicationsthatuselanguage:Intrinsic:Parsing,LanguageModelling,etc.Extrinsic:ASR,M
3、T,QA/Dialogue,etc.LanguagemodelsAlanguagemodelassignsaprobabilitytoasequenceofwords,Psuchthatw2p(w)=1:Giventheobservedtrainingtext,howprobableisthisnewutterance?Thuswecancomparedierentorderingsofwords(e.g.Translation):p(helikesapples)>p(appleslikeshe)orchoiceofwords(e.g.Speech
4、Recognition):p(helikesapples)>p(helicksapples)History:cryptographyLanguagemodelsMuchofNaturalLanguageProcessingcanbestructuredas(conditional)languagemodelling:Translationplm(LeschiensaimentlesosjjjDogslovebones)QuestionAnsweringplm(Whatdodogslove?jjjbones.j)Dialogueplm(Howareyou?
5、jjjFinethanks.Andyou?j)LanguagemodelsMostlanguagemodelsemploythechainruletodecomposethejointprobabilityintoasequenceofconditionalprobabilities:p(w1;w2;w3;:::;wN)=p(w1)p(w2jw1)p(w3jw1;w2):::p(wNjw1;w2;:::wN 1)Notethatthisdecompositionisexactandallowsustomodelcomplexjointdistribu
6、tionsbylearningconditionaldistributionsoverthenextword(wn)giventhehistoryofwordsobserved(w1;:::;wn 1).LanguagemodelsThesimpleobjectiveofmodellingthenextwordgiventheobservedhistorycontainsmuchofthecomplexityofnaturallanguageunderstanding.Considerpredictingtheextensionoftheutteranc
7、e:p(jThereshebuilta)Withmorecontextweareabletouseourknowledgeofbothlanguageandtheworldtoheavilyconstrainthedistributionoverthenextword:p(jAlicewenttothebeach.Thereshebuilta)Thereisevidencethathumanlanguageacquisitionpartlyreliesonfutureprediction.EvaluatingaLanguageModelAgoodmo
8、delassignsrealutteranceswNfromalanguagea
此文档下载收益归作者所有