Principle of maximum entropy

《Principle of maximum entropy》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、Principleofmaximumentropy1PrincipleofmaximumentropyThisarticleisabouttheprobabilitytheoreticprinciple.Fortheclassifierinmachinelearning,seemaximumentropyclassifier.Forotheruses,seemaximumentropy(disambiguation).BayesianstatisticsTheory•Admissibledecisionrule•Bayesianefficiency

2、•Bayesianprobability•Probabilityinterpretations•Bayes'theorem•Bayes'rule•Bayesfactor•Bayesianinference•Bayesiannetwork•Prior•Posterior•Likelihood•Conjugateprior•Posteriorpredictive•Hyperparameter•Hyperprior•Principleofindifference•Principleofmaximumentropy•EmpiricalBayesmethod

3、•Cromwell'srule•Bernstein–vonMisestheorem•Bayesianinformationcriterion•Credibleinterval•MaximumaposterioriestimationTechniques•Bayesianlinearregression•Bayesianestimator•ApproximateBayesiancomputation•Statisticsportal•v•t•e[1]Theprincipleofmaximumentropystatesthat,subjecttopre

4、ciselystatedpriordata(suchasapropositionthatexpressestestableinformation),theprobabilitydistributionwhichbestrepresentsthecurrentstateofknowledgeistheonewithlargestentropy.Anotherwayofstatingthis:Takepreciselystatedpriordataortestableinformationaboutaprobabilitydistributionfun

5、ction.Considerthesetofalltrialprobabilitydistributionsthatwouldencodethepriordata.Ofthose,onewithmaximalinformationentropyistheproperdistribution,accordingtothisprinciple.Principleofmaximumentropy2HistoryTheprinciplewasfirstexpoundedbyE.T.Jaynesintwopapersin1957whereheemphasiz

6、edanaturalcorrespondencebetweenstatisticalmechanicsandinformationtheory.Inparticular,JaynesofferedanewandverygeneralrationalewhytheGibbsianmethodofstatisticalmechanicsworks.Hearguedthattheentropyofstatisticalmechanicsandtheinformationentropyofinformationtheoryareprincipallythe

7、samething.Consequently,statisticalmechanicsshouldbeseenjustasaparticularapplicationofageneraltooloflogicalinferenceandinformationtheory.OverviewInmostpracticalcases,thestatedpriordataortestableinformationisgivenbyasetofconservedquantities(averagevaluesofsomemomentfunctions),as

8、sociatedwiththeprobabilitydistributioninquestion.Thisistheway