资源描述：
《A New Unsupervised Approach to Word Segmentation》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、ANewUnsupervisedApproachtoWordSegmentationHanshiWang∗BeijingInstituteofTechnologyJianZhu∗∗BeijingInstituteofTechnologyShipingTang†BeijingInstituteofTechnologyXiaozhongFan‡BeijingInstituteofTechnologyThisarticleproposesESA,anewunsupervisedapproachtowordsegmentation.ESAisaniterativeprocessconsistin
2、gofthreephases:Evaluation,Selection,andAdjustment.InEval-uation,boththecertaintyanduncertaintyofcharactersequenceco-occurrenceincorporaareconsideredasstatisticalevidencesupportinggoodnessmeasurement.Additionally,thestatisticaldataofcharactersequenceswithvariouslengthsbecomecomparablewitheachother
3、byusingasimpleprocesscalledBalancing.InSelection,alocalmaximumstrategyisadoptedwithoutthresholds,andthestrategycanbeimplementedwithdynamicprogramming.InAdjustment,apartofthestatisticaldataisupdatedtoimprovesuccessiveresults.Inourexperiment,ESAwasevaluatedontheSIGHANBakeoff-2dataset.Theresultssugg
4、estthatESAiseffectiveonChinesecorpora.ItisnoteworthythattheF-measuresoftheresultsarebasicallymonotoneincreasingandcanrapidlyconvergetorelativelyhighvalues.Furthermore,empiricalformulaebasedontheresultscanbeusedtopredicttheparameterinESAtoavoidparameterestimationthatisusuallytime-consuming.1.Intro
5、ductionWordsegmentationisanimportanttaskinnaturallanguageprocessing(NLP)forlanguageswithoutworddelimiters(e.g.,Chinese).Todate,mostexistingapproachestoChinesewordsegmentation(CWS)aresupervised.Althoughsupervisedapproaches∗SchoolofComputerScienceandTechnology,BeijingInstituteofTechnology,Beijing,1
6、00081China.E-mail:necrostone@gmail.com.∗∗SchoolofComputerScienceandTechnology,BeijingInstituteofTechnology,Beijing,100081China.†SchoolofComputerScienceandTechnology,BeijingInstituteofTechnology,Beijing,100081China.‡SchoolofComputerScienceandTechnology,BeijingInstituteofTechnology,Beijing,100081Ch
7、ina.E-mail:fxz@bit.edu.cn.Submissionreceived:8December2009;revisedsubmissionreceived:14October2010;acceptedforpublication:18November2010.©2011AssociationforComputationalLinguisticsComputationalLinguisticsVolume37,Numbe
