面向大规模语料的语言模型研究新进展(1)

《面向大规模语料的语言模型研究新进展(1)》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、万方数据计算机研究与发展ISSN1000—12391CN11-17771TPJournalofComputerResearchandDevelopment46(10)：1704—1712，2009面向大规模语料的语言模型研究新进展骆卫华1’2刘群2白硕31(中国科学院研究生院北京100049)2(中国科学院计算技术研究所智能信息处理重点实验室北京100190)3(上海证券交易所上海200120)(1uoweihua@ict．ac．cn)AReviewoftheState-of-the-ArtofResearchonLarge-ScaleCorporaOrient

2、edLanguageModelingLuoWeihual”，LiuQun2，andBaiShu031(GraduateUniversityofChineseAcademyofSciences，Beijing100049)2(KeyLaboratoryofIntelligentInformationProcessing，InstituteofComputingTechnology，ChineseAcademyofSciences，Beijing100190)3(ShanghaiSecuritiesExchange，Shanghai200120)AbstractN—

3、gramlanguagemodel(LM)isakeycomponentinmanyresearchareasofnaturallanguageprocessing，suchasstatisticalmachinetranslation，informationretrieval，Speechrecognition，etc．Usinghigher-ordermodelsandmoretrainingdatacansignificantlyimprovetheperformanceofapplications．However，forlimitedresourceso

4、fthesystems(e．g．，memory，usageofCPU，etc)．thecostoftrainingandaccessinglarge-scaleI。Mbecomesprohibitivewithmoreandmoremonolingualcorporaavailable．Therefore，theresearchonlarge-scalelanguagemodelingdrawsmoreattention．Theauthorsintroducethestate—of-the-artoftheideasandprogressoftheissue，w

5、hichfocusesonsomerepresentativeapproaches，includinganadhoemethod，arandomizedrepresentationmodelandadistributedparallelframework．TheadhoemethodiSaunifiedoneintegratingdivisionandconqueringofdata，compactdatastructrue，datacompressionbasedonquantizationandmemorymapping．Therandomizedrepre

6、sentationofLMisalossycompressionmodelbasedonBloomfilter．ThedistributedparallelframeworkcarriesoutthetrainingofLMbasedonMapReduceandperformstherequestsofN—gramsinabatchmodeofremotecall．Theperformanceofsystemsofstatisticalmachinetranslationutilizingtheapproachesisdescribedrespectivelyw

7、ithexperiments，andfinallyprosandconsarecompared．Keywordslanguagemodel；datacompression；randomizedaccessmodel；Bloomfilter；distributedparallelarchitecture摘要N元语言模型是统计机器翻译、信息检索、语音识别等很多自然语言处理研究领域的重要工具．由于扩大训练语料规模和增加元数对于提高系统性能很有帮助，随着可用语料迅速增加，面向大规模训练语料的高元语言模型(如N≥5)的训练和使用成为新的研究热点．介绍了当前这个问题的最新研

8、究进展，包括了集成数据分