2004-ACL-Discriminative language modeling with conditional random fields and the perceptron algorithm英文学习材料

《2004-ACL-Discriminative language modeling with conditional random fields and the perceptron algorithm英文学习材料》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、DiscriminativeLanguageModelingwithConditionalRandomFieldsandthePerceptronAlgorithmBrianRoarkMuratSaraclarMichaelCollinsMarkJohnsonAT&TLabs-ResearchMITCSAILBrownUniversityfroark,muratg@research.att.commcollins@csail.mit.eduMarkJohnson@Brown.eduAbstractticesthataretheoutputfromabaselinerecogniz

2、er.Wealsogiveanumberofexperimentscomparingthetwoap-Thispaperdescribesdiscriminativelanguagemodelingproaches.Theperceptronmethodgavea1.3%absoluteforalargevocabularyspeechrecognitiontask.Wecon-improvementinrecognitionerrorontheSwitchboarddo-trasttwoparameterestimationmethods:theperceptronmain;t

3、heCRFmethodswedescribegiveafurthergain,algorithm,andamethodbasedonconditionalrandomthefinalabsoluteimprovementbeing1.8%.fields(CRFs).Themodelsareencodedasdetermin-Acentralissuewefocusonconcernsfeatureselection.isticweightedfinitestateautomata,andareappliedbyThenumberofdistinctn-gramsinourtrainin

4、gdataisintersectingtheautomatawithword-latticesthatarethecloseto45million,andweshowthatCRFtrainingcon-outputfromabaselinerecognizer.Theperceptronalgo-vergesveryslowlyevenwhentrainedwithasubset(ofrithmhasthebenefitofautomaticallyselectingarela-size12million)ofthesefeatures.Becauseofthis,weex-ti

5、velysmallfeaturesetinjustacoupleofpassesovertheploremethodsforpickingasmallsubsetoftheavailabletrainingdata.However,usingthefeaturesetoutputfrom1features.Theperceptronalgorithmcanbeusedasonetheperceptronalgorithm(initializedwiththeirweights),methodforfeatureselection,selectingaround1.5million

6、CRFtrainingprovidesanadditional0.5%reductioninfeaturesintotal.TheCRFtrainedwiththisfeatureset,worderrorrate,foratotal1.8%absolutereductionfromandinitializedwithparametersfromperceptrontraining,thebaselineof39.2%.convergesmuchmorequicklythanotherapproaches,andalsogivestheoptimalperformanceonth

7、eheld-outset.1IntroductionWeexploreotherapproachestofeatureselection,butfindAcrucialcomponentofanyspeechrecognizeristhelan-thattheperceptron-basedapproachgivesthebestresultsguagemodel(LM),whichassignsscoresorprobabilitiesinourexperiments.tocan