基于lucene相似句子检索系统论文英文翻译原文

《基于lucene相似句子检索系统论文英文翻译原文》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、AnalysisAnalysis,inLucene，istheprocessofconvertingfieldtextintoitsmostfundamentalindexedrepresentation，terms.Thesetermsareusedtodeterminewhatdocumentsmatchaqueryduringsearches.Forexample，ifthissentencewereindexedintoafield(let’sassumetypeField.Text)，thet

2、ermsmightstartwith/^randexample,andsoon，asseparatetermsinsequence.Ananalyzerisanencapsulationoftheanalysisprocess.Ananalyzertokenizestextbyperforminganynumberofoperationsonit，whichcouldincludeextractingwords,discardingpunctuation,removingaccentsfromchara

3、cters,lowercasing(alsocallednormalizing),removingcommonwords,reducingwordstoarootform(stemming),orchangingwordsintothebasicform(lemmatization).Thisprocessisalsocalledtokenization,andthechunksoftextpulledfromastreamoftextarecalledtokens.Tokens,combinedwit

4、htheirassociatedfieldname,areterms.Lucene’sprimarygoalistofacilitateinformationretrieval.Theemphasisonretrievalisimportant.YouwanttothrowgobsoftextatLuceneandhavethemberichlysearchablebytheindividualwordswithinthattext.InorderforLucenetoknowwhat“words”ar

5、e，itanalyzesthetextduringindexing，extractingitintoterms.Thesetermsaretheprimitivebuildingblocksforsearching.ChoosingtherightanalyzerisacrucialdevelopmentdecisionwithLucene.Onesizedoesn’tfitallwhenitcomestochoosingananalyzer.Languageisonefactorinchoosinga

6、nanalyzer,becauseeachhasitsownuniquefeatures.Anotherfactortoconsiderinchoosingananalyzeristhedomainofthetextbeinganalyzed;differentindustrieshavedifferentterminology，acronyms，andabbreviationsthatmaydeserveattention.Althoughwepresentmanyoftheconsideration

7、sforchoosinganalyzers,nosingleanalyzerwillsufficeforallsituations.It’spossiblethatnoneofthebuilt-inanalysisoptionsareadequateforyourneeds,andyou’llneedtoinvestincreatingacustomanalysissolution;pleasantly，Lucene’sbuildingblocksmakethisquiteeasy.Oneofthebe

8、stquestionsyoucanaskasyoucontemplatetheanalysisprocessis，“WhatwouldGoogledo?”Google’sactualalgorithmsareproprietaryandkeptrelativelysecret，buttheresultsfromsearchesgivesomeinsight.Searchingforthephrase“tobeornottobe”withan