文本过滤关键技术研究

《文本过滤关键技术研究》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、复旦大学博士学位论文文本过滤关键技术研究姓名：夏迎炬申请学位级别：博士专业：计算机软件与理论指导教师：吴立德20030418塑茎一——AbstractMoreandmoreinformations01．1rezsarenowavailableinmachine—readableformduetotherapiddevelopmentofcommunicationnetworksandinexpensivemassivestorage．Foraspecialuser,theinformafionneededisrelative

2、lysmall．Sincemostofthedataaremassiveinscaleanddiverseinsubjectareas，theymaketheinformationacquisitionbottleneckmoresevere，thusgreatlychallengetheprocessingsystemsinspeed，precisionandrobustness．Inordertofindusefulinformationembeddedinelectronicform．emcientandeffecti

3、vetechniquesforlargescalerealtextprocessinghavebecomeoneofthemosturgentdemands．Tllisdissertationfocusesonthekeytechniquesofadaptivetextfiltering．Wehavedesignedanddevelopedanexperimentplatform．BasedonthatwetookpartintheFilteringtrackofTextREtrievalConference(TRECl0a

4、ndTRECll)andobtainedverygoodresults．InTREell，WeareselectedforthefirstofonlythreespeakingslotsonFilteringtrack．Wealsohavedevelopedseveralsystems，including‘‘Chinesetextfilteringsystem'’and“Web—BasedTrendSearchSystem”．VectorSpaceModel(VSM)isusedtorepresenttext，Therear

5、etwoprincipleproblemsaboutVSM：termselectionandtermweighting．Words，concepts，andterminologiesareselectedasterms．whiletermweightsarecalculated、衍thstatisticalinformationandheuristicrules．WehaveappliedtheWordNettofilteringsystem，usedthesemanticinformationandtrieddisambi

6、guation．WehavedevelopedtheinterfaceofHowNetandappliedittoChinesefilteringsystembyusingitsconceptioninformation．Thismethodhasenhancedthesystem’sperformancewhilelowerthedimensionofvectorgreatly．MachineLearninginadaptivetextfilteringincludesprofilelearningandthreshold

7、learning．Wemakeresearchonthresholdlearning．InTRECIO，wepresentedanovelthreshold—adjustingalgorithm．珊salgorithmadjuststhethresholdfastandefficientlyusingasmallnumberofsamples．InTRECII，wepresentedanovelmethodthatusesawinnowclassifierbuildingfromthesewordstoassistthete

8、xtfilteringsystem．Thismethodcanenhancethesystem’sperformancegreatly．Keywords：AdaptiveTextFiltering，TextFeatureExtraction，VectorSpaceModelMachineL