资源描述：
《a new data storage and service model of china web》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、ANewDataStorageandServiceModelofChinaWebInfoMall1HongfeiYan,LianenHuang,ChongChenandZhengmaoXieComputerNetworksandDistributedSystemsLaboratory,DepartmentofComputerScienceandTechnology,PekingUniversity,P.R.China{yhf,hle,cc,xzm}@net.pku.edu.cnAbstractTheWebconsists
2、ofenormouspageswhichiseasiervanishingthantraditionalmediasuchasnewspaper,journals.Topreservethewebresources,webegantheChinaWebarchivingproject,namedWebInfoMall,from2001.ThepaperdescribesthedatastorageandservicemodelofWebInfoMall2.0tomeetthegoalsofcollectingthestu
3、ffbroadly,storingthemperennially,andlocatingrequestsefficiently.CurrentlytheWebInfoMallholds0.7billionpages(10.6terabyte)togetherwith5terabyteofdigitalwebresourcesotherthanwebpages,havingtheabilityofcollectingmorethan1millionpagesperday,astoragecapacitytoholdmore
4、than10billionpages(about150terabyte),andaschemetomanagelargenumbersofpages.Keywords:webresources,WebInfoMall,distributedcrawling,Tianwangstorageformat,servicemodelIntroductionTheInternetwasintroducedtoChinain1994,andthefirstwebsitesemergedfrom1995.Fromthenon,theC
5、hinaWebgotanunprecedenteddevelopment.Attheendof2003,theregisteredsitesamountedto595,550with".cn"domainsuffix[CNNIC,2004].Duringthenineyears,theChinaWebenjoysexponentialgrowth,withthesameratetothewholeWeb.Tianwang[Tianwang,2004]crawledmorethan60millionwebpagesupto
6、theJulyof2002,and105millionpagesinJanuaryof2003.Accordingtoaquantitativeanalysis,Chinesepagesisaboutdoubleeveryyear[Li,2003].Tostoreasintegralityaspossible,itisnecessarytohaveaneffectiverepositorythatiscapableofholding1billionpages,aswellassupportingefficientacce
7、ss.Fromthebeginningof2001,areactiontothisstateofwantwasinitiatedundertheleadershipofXiaomingLiatPekingUniversity.SuchaneffortleadstothebornoftheWebInfoMall(http://www.infomall.cn).ThegoaloftheWebInfoMallis1ThisworkissupportedbytheChina973Grant(G1999032706)andtheM
8、OEproject(20030001076).th4InternationalWebArchivingWorkshop(2004)2HongfeiYan,LianenHuang,ChongChenandZhengmaoXietoconstructaplatformthatsupportsWebinformations
