NLP自然语言处理—N-gramlanguagemodel.ppt

《NLP自然语言处理—N-gramlanguagemodel.ppt》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、1CS388:NaturalLanguageProcessing:N-GramLanguageModelsRaymondJ.MooneyUniversityofTexasatAustinLanguageModelsFormalgrammars(e.g.regular,contextfree)giveahard“binary”modelofthelegalsentencesinalanguage.ForNLP,aprobabilisticmodelofalanguagethatgivesaprobabilitythatastringis

2、amemberofalanguageismoreuseful.Tospecifyacorrectprobabilitydistribution,theprobabilityofallsentencesinalanguagemustsumto1.UsesofLanguageModelsSpeechrecognition“Iateacherry”isamorelikelysentencethan“EyeeightuhJerry”OCR&HandwritingrecognitionMoreprobablesentencesaremorelik

3、elycorrectreadings.MachinetranslationMorelikelysentencesareprobablybettertranslations.GenerationMorelikelysentencesareprobablybetterNLgenerations.Contextsensitivespellingcorrection“Theirareproblemswitthissentence.”CompletionPredictionAlanguagemodelalsosupportspredictingt

4、hecompletionofasentence.Pleaseturnoffyourcell_____Yourprogramdoesnot______Predictivetextinputsystemscanguesswhatyouaretypingandgivechoicesonhowtocompleteit.N-GramModelsEstimateprobabilityofeachwordgivenpriorcontext.P(phone

5、Pleaseturnoffyourcell)Numberofparametersrequired

6、growsexponentiallywiththenumberofwordsofpriorcontext.AnN-grammodelusesonlyN1wordsofpriorcontext.Unigram:P(phone)Bigram:P(phone

7、cell)Trigram:P(phone

8、yourcell)TheMarkovassumptionisthepresumptionthatthefuturebehaviorofadynamicalsystemonlydependsonitsrecenthistory.Inparticu

9、lar,inakth-orderMarkovmodel,thenextstateonlydependsonthekmostrecentstates,thereforeanN-grammodelisa(N1)-orderMarkovmodel.N-GramModelFormulasWordsequencesChainruleofprobabilityBigramapproximationN-gramapproximationEstimatingProbabilitiesN-gramconditionalprobabilitiescanb

10、eestimatedfromrawtextbasedontherelativefrequencyofwordsequences.Tohaveaconsistentprobabilisticmodel,appendauniquestart()andend()symboltoeverysentenceandtreattheseasadditionalwords.Bigram:N-gram:GenerativeModel&MLEAnN-grammodelcanbeseenasaprobabilisticautomataforge

11、neratingsentences.Relativefrequencyestimatescanbeproventobemaximumlikelihoodestimates(MLE)sincetheymaxi