资源描述：
《《自然语言处理》PPT课件.ppt》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、StatisticalNLP:Lecture8StatisticalInference:n-gramModelsoverSparseData1OverviewStatisticalInferenceconsistsoftakingsomedata(generatedinaccordancewithsomeunknownprobabilitydistribution)andthenmakingsomeinferencesaboutthisdistribution.Therearethreeissuestoconsi
2、der:DividingthetrainingdataintoequivalenceclassesFindingagoodstatisticalestimatorforeachequivalenceclassCombiningmultipleestimators2FormingEquivalenceClassesIClassificationProblem:trytopredictthetargetfeaturebasedonvariousclassificatoryfeatures.==>Reliability
3、versusdiscriminationMarkovAssumption:Onlythepriorlocalcontextaffectsthenextentry:(n-1)thMarkovModelorn-gramSizeofthen-grammodelsversusnumberofparameters:wewouldlikentobelarge,butthenumberofparametersincreasesexponentiallywithn.Thereexistotherwaystoformequival
4、enceclassesofthehistory,buttheyrequiremorecomplicated.methods==>willusen-gramshere.3StatisticalEstimatorsI:OverviewGoal:ToderiveagoodprobabilityestimateforthetargetfeaturebasedonobserveddataRunningExample:Fromn-gramdataP(w1,..,wn)’spredictP(wn
5、w1,..,wn-1)Solu
6、tionswewilllookat:MaximumLikelihoodEstimationLaplace’s,Lidstone’sandJeffreys-Perks’LawsHeldOutEstimationCross-ValidationGood-TuringEstimation4StatisticalEstimatorsII:MaximumLikelihoodEstimationPMLE(w1,..,wn)=C(w1,..,wn)/N,whereC(w1,..,wn)isthefrequencyofn-gra
7、mw1,..,wnPMLE(wn
8、w1,..,wn-1)=C(w1,..,wn)/C(w1,..,wn-1)ThisestimateiscalledMaximumLikelihoodEstimate(MLE)becauseitisthechoiceofparametersthatgivesthehighestprobabilitytothetrainingcorpus.MLEisusuallyunsuitableforNLPbecauseofthesparsenessofthedata==>UseaDiscoun
9、tingor.Smoothingtechnique.5StatisticalEstimatorsIII:SmoothingTechniques:LaplacePLAP(w1,..,wn)=(C(w1,..,wn)+1)/(N+B),whereC(w1,..,wn)isthefrequencyofn-gramw1,..,wnandBisthenumberofbinstraininginstancesaredividedinto.==>AddingOneProcessTheideaistogivealittlebit
10、oftheprobabilityspacetounseenevents.However,inNLPapplicationsthatareverysparse,Laplace’sLawactuallygivesfartoomuchoftheprobabilityspacetounseenevents.6StatisticalEstimatorsIV:SmoothingTec
