资源描述：
《google_deep_crawl》由会员上传分享，免费在线阅读，更多相关内容在行业资料-天天文库。
1、Google’sDeep-WebCrawl∗JayantMadhavanDavidKoŁucjaKotGoogleInc.GoogleInc.CornellUniversityjayant@google.comdko@google.comlucja@cs.cornell.edu∗VigneshGanapathyAlexRasmussenAlonHalevyGoogleInc.UniversityofCalifornia,SanDiegoGoogleInc.vignesh@google.comarasmuss@cs.ucsd.eduhalevy@google.co
2、mABSTRACTenginesrelyonhyperlinkstodiscovernewwebpagesandtypicallylacktheabilitytoperformsuchformsubmissions.TheDeepWeb,i.e.,contenthiddenbehindHTMLforms,VariousaccountshavehypothesizedthattheDeepWebhashaslongbeenacknowledgedasasignificantgapinsearchanorderofmagnitudemoredatathanthecur
3、rentlysearch-enginecoverage.SinceitrepresentsalargeportionoftheableWorldWideWeb[2,9,12].Furthermore,theDeepstructureddataontheWeb,accessingDeep-WebcontentWebhasbeenalong-standingchallengeforthedatabasehasbeenalong-standingchallengeforthedatabasecommu-community[1,9,10,13,14,18,19]beca
4、useitrepresentsanity.ThispaperdescribesasystemforsurfacingDeep-WeblargefractionofthestructureddataontheWeb.content,i.e.,pre-computingsubmissionsforeachHTMLTherearetwocommonapproachestoofferingaccesstoformandaddingtheresultingHTMLpagesintoasearchDeep-Webcontent.Thefirstapproach(essentia
5、llyadataengineindex.Theresultsofoursurfacinghavebeenincor-integrationsolution)istocreateverticalsearchenginesforporatedintotheGooglesearchengineandtodaydrivemorespecificdomains(e.g.cars,books,orrealestate).InthisthanathousandqueriespersecondtoDeep-Webcontent.approachwecouldcreateamedi
6、atorformforeachdomainSurfacingtheDeepWebposesseveralchallenges.First,andsemanticmappingsbetweenindividualdatasourcesandourgoalistoindexthecontentbehindmanymillionsofthemediatorform.Forasearchengine,thisapproachsuf-HTMLformsthatspanmanylanguagesandhundredsoffersfromseveraldrawbacks.Fi
7、rst,thecostofbuildinganddomains.Thisnecessitatesanapproachthatiscompletelymaintainingthemediatorformsandthemappingsishigh.automatic,highlyscalable,andveryefficient.Second,aSecond,identifyingwhichqueriesarerelevanttoeachdo-largenumberofformshavetextinputsandrequirevalidmainisextremelych
8、allenging.Fi
