剑桥Information retrieval(信息检索)课件04const.pdf

《剑桥Information retrieval(信息检索)课件04const.pdf》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、Onlineedition(c)2009CambridgeUPDRAFT!©April1,2009CambridgeUniversityPress.Feedbackwelcome.674IndexconstructionInthischapter,welookathowtoconstructaninvertedindex.WecallthisINDEXINGprocessindexconstructionorindexing;theprocessormachinethatperformsitINDEXERtheindexer.Thed

2、esignofindexingalgorithmsisgovernedbyhardwarecon-straints.Wethereforebeginthischapterwithareviewofthebasicsofcom-puterhardwarethatarerelevantforindexing.Wethenintroduceblockedsort-basedindexing(Section4.2),anefficientsingle-machinealgorithmde-signedforstaticcollectionsth

3、atcanbeviewedasamorescalableversionofthebasicsort-basedindexingalgorithmweintroducedinChapter1.Sec-tion4.3describessingle-passin-memoryindexing,analgorithmthathasevenbetterscalingpropertiesbecauseitdoesnotholdthevocabularyinmemory.Forverylargecollectionsliketheweb,index

4、inghastobedis-tributedovercomputerclusterswithhundredsorthousandsofmachines.WediscussthisinSection4.4.Collectionswithfrequentchangesrequiredy-namicindexingintroducedinSection4.5sothatchangesinthecollectionareimmediatelyreflectedintheindex.Finally,wecoversomecomplicatingi

5、s-suesthatcanariseinindexingsuchassecurityandindexesforrankedretrievalinSection4.6.Indexconstructioninteractswithseveraltopicscoveredinotherchapters.Theindexerneedsrawtext,butdocumentsareencodedinmanyways(seeChapter2).Indexerscompressanddecompressintermediatefilesandthefi

6、nalindex(seeChapter5).Inwebsearch,documentsarenotonalocalfilesystem,buthavetobespideredorcrawled(seeChapter20).Inenter-prisesearch,mostdocumentsareencapsulatedinvariedcontentmanage-mentsystems,emailapplications,anddatabases.WegivesomeexamplesinSection4.7.Althoughmostofth

7、eseapplicationscanbeaccessedviahttp,nativeApplicationProgrammingInterfaces(APIs)areusuallymoreefficient.Thereadershouldbeawarethatbuildingthesubsystemthatfeedsrawtexttotheindexingprocesscaninitselfbeachallengingproblem.Onlineedition(c)2009CambridgeUP684Indexconstruction◮

8、Table4.1Typicalsystemparametersin2007.Theseektimeisthetimeneededtopositionthediskheadinanewposition.Thetransfe