The Minimum Description Length Principle

《The Minimum Description Length Principle》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、TheMinimumDescriptionLengthPrinciplePeterD.GrünwaldTheMITPressCambridge,MassachusettsLondon,England©2007MassachusettsInstituteofTechnologyAllrightsreserved.Nopartofthisbookmaybereproducedinanyformbyanyelectronicormechanicalmeans(includingphotocopying,record

2、ing,orinformationstorageandretrieval)withoutpermissioninwritingfromthepublisher.TypesetinPalatinobytheauthorusingLATEX2εwithC.Manning’sfbook.clsandstatnlpbook.stymacros.PrintedandboundintheUnitedStatesofAmerica.LibraryofCongressCataloging-in-PublicationInfo

3、rmationGrünwald,PeterD.Theminimumdescriptionlengthprinciple/PeterD.Grünwald.p.cm.—(Adaptivecomputationandmachinelearning)Includesbibliographicalreferencesandindex.ISBN-13:978-0-262-07281-6(alk.paper)1.Minimumdescriptionlength(Informationtheory)I.TitleQA276.

4、9G782007003’.54—dc22200604664610987654321BriefContentsIIntroductoryMaterial11Learning,Regularity,andCompression32ProbabilisticandStatisticalPreliminaries413Information-TheoreticPreliminaries794Information-TheoreticPropertiesofStatisticalModels1095CrudeTwo-P

5、artCodeMDL131IIUniversalCoding1656UniversalCodingwithCountableModels1717ParametricModels:NormalizedMaximumLikelihood2078ParametricModels:Bayes2319ParametricModels:PrequentialPlug-in25710ParametricModels:Two-Part27111NMLWithInfiniteComplexity29512LinearRegres

6、sion33513BeyondParametrics369IIIRefinedMDL40314MDLModelSelection40915MDLPredictionandEstimation45916MDLConsistencyandConvergence50117MDLinContext523viiiBriefContentsIVAdditionalBackground59718TheExponentialor“MaximumEntropy”Families59919Information-Theoretic

7、PropertiesofExponentialFamilies623ContentsListofFiguresxixSeriesForewordxxiForewordxxiiiPrefacexxvIIntroductoryMaterial11Learning,Regularity,andCompression31.1RegularityandLearning41.2RegularityandCompression41.3Solomonoff’sBreakthrough–KolmogorovComplexity

8、81.4MakingtheIdeaApplicable101.5CrudeMDL,RefinedMDLandUniversalCoding121.5.1FromCrudetoRefinedMDL141.5.2UniversalCodingandRefinedMDL171.5.3RefinedMDLforModelSelection181.5.4RefinedMDLforPredictionandHypothe