文本分类系统的设计与实现

《文本分类系统的设计与实现》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、北京交通大学硕士学位论文文本分类系统的设计与实现姓名：高亚波申请学位级别：硕士专业：软件工程指导教师：范辉20080601ABSTRACT：ABSTRACTWiththepopularityofcomputersandthecontinualdevelopmentofInternet，moreandmoreelectronicdocumentshavebecomeaccumulationofhugevolumesofdata．Howtomanagethesemassivedatainordertoprovidef

2、acilitationforUSerStosearchquicklyhasbecomeacoreofknowledgediscoveryindatabases(KDD)．Addressingthisissue，textcategorizationtechnologyhasproposedaseriesofsolutions．Textcategorizationisanimportantintelligenceinformationprocessingtechnology．Thistechnologyhashigh

3、valueininformationfiltering,informationretrieval，textdatabases，digitallibraries．andotheraspects．Thearticleintroducesthetextcategorizationtechnologyanditsrelatedalgorithms．Atextcategorizationsystemisdesignedandimplemented．ThisSystemisdividedintosixmodules：(1)T

4、extpreprocessor,slicingthewordsinthedocument，filteringthestop-word；(2)frequencystatisticsmodule，countingthefrequencyofcharacteristicWordsaccordingtovariousclassificationalgorithmsandcharacteristicsoffeatureselectionalgorithm；(3)Featureselectionmodule，achievin

5、gtheinformationgain(IG)，mutualinformation(MI)，cross—entropy(CE)，andX一2statisticsthesefourfeatureselectionalgorithm；(4)weightcalculation，achievingaTF'TF—IDFalgorithm；(5)theclassificationalgorithm，achievingtheBayesianandKneighborstextclassificationalgorithm；(6)

6、classifierevaluationmodule，achievingtheclassificationperformance’Sevaluationmechanismfromtherecallrate，check-rateandvalueofF1．Threeexperimentshavebeendonebyusingthissystem：kvalue’SaffectsontheclassificationintheKNNalgorithm，differentfeatureselectionalgorithms

7、’effectsontheclassificationunderKNNalgorithm，thecomparisonofNBalgorithmandKNNclassificationalgorithm．Someconclusionshavebeendrawfromtheexperiments．KEYWORDS：textcategorization；vectorspacemodel；featureselection；weightcalculation；naiveBayesian；Knearestneighboral

8、gorithmCI．ASSNO：TP311．5学位论文版权使用授权书本学位论文作者完全了解北京交通大学有关保留、使用学位论文的规定。特授权北京交通大学可以将学位论文的全部或部分内容编入有关数据库进行检索，并采用影印、缩印或扫描等复制手段保存、汇编以供查阅和借阅。同意学校向国家有关部门或机构送交论文的复印件和磁盘。(保密的学位论文在解密后适用本授权说明)学位论文作者签名：南