欢迎来到天天文库
浏览记录
ID:55796460
大小:373.50 KB
页数:22页
时间:2020-06-07
《NLP自然语言处理—N-gramlanguagemodel.ppt》由会员上传分享,免费在线阅读,更多相关内容在教育资源-天天文库。
1、1CS388:NaturalLanguageProcessing:N-GramLanguageModelsRaymondJ.MooneyUniversityofTexasatAustinLanguageModelsFormalgrammars(e.g.regular,contextfree)giveahard“binary”modelofthelegalsentencesinalanguage.ForNLP,aprobabilisticmodelofalanguagethatgivesaprobabilitythatastringis
2、amemberofalanguageismoreuseful.Tospecifyacorrectprobabilitydistribution,theprobabilityofallsentencesinalanguagemustsumto1.UsesofLanguageModelsSpeechrecognition“Iateacherry”isamorelikelysentencethan“EyeeightuhJerry”OCR&HandwritingrecognitionMoreprobablesentencesaremorelik
3、elycorrectreadings.MachinetranslationMorelikelysentencesareprobablybettertranslations.GenerationMorelikelysentencesareprobablybetterNLgenerations.Contextsensitivespellingcorrection“Theirareproblemswitthissentence.”CompletionPredictionAlanguagemodelalsosupportspredictingt
4、hecompletionofasentence.Pleaseturnoffyourcell_____Yourprogramdoesnot______Predictivetextinputsystemscanguesswhatyouaretypingandgivechoicesonhowtocompleteit.N-GramModelsEstimateprobabilityofeachwordgivenpriorcontext.P(phone
5、Pleaseturnoffyourcell)Numberofparametersrequired
6、growsexponentiallywiththenumberofwordsofpriorcontext.AnN-grammodelusesonlyN1wordsofpriorcontext.Unigram:P(phone)Bigram:P(phone
7、cell)Trigram:P(phone
8、yourcell)TheMarkovassumptionisthepresumptionthatthefuturebehaviorofadynamicalsystemonlydependsonitsrecenthistory.Inparticu
9、lar,inakth-orderMarkovmodel,thenextstateonlydependsonthekmostrecentstates,thereforeanN-grammodelisa(N1)-orderMarkovmodel.N-GramModelFormulasWordsequencesChainruleofprobabilityBigramapproximationN-gramapproximationEstimatingProbabilitiesN-gramconditionalprobabilitiescanb
10、eestimatedfromrawtextbasedontherelativefrequencyofwordsequences.Tohaveaconsistentprobabilisticmodel,appendauniquestart()andend()symboltoeverysentenceandtreattheseasadditionalwords.Bigram:N-gram:GenerativeModel&MLEAnN-grammodelcanbeseenasaprobabilisticautomataforge
11、neratingsentences.Relativefrequencyestimatescanbeproventobemaximumlikelihoodestimates(MLE)sincetheymaxi
此文档下载收益归作者所有