l06 entropy rates of a stochastic process

《l06 entropy rates of a stochastic process》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。

1、教学内容应用信息论基础IntroductionandPreviewEntropyRelativeEntropyandMutualInformationEntropy,RelativeEntropy,andMutualInformation金明录教授AsymptoticEquipartitionPropertyEntropyRatesofaStochasticProcessEntropyRatesofaStochasticProcessDataCompressionChannelCapa

2、cityDifferentialEntropyGaussianChannelRateDistortionTheoryNetworkInformationTheory12-13学年第二学期DUT应用信息论基础金明录教授DUT应用信息论基础金明录教授ContentsReviewEtEntropyratte.TdfTwodefiiinititiftonsofentropyrattfeforastthtochastiicprocessReviewareSourceCodingSourceCo

3、dingTreeofCodeKftiKraftinequalitlityWhatWeCannotDo:FundamentalLimitationsofSCSourceCoddiingForastationarystochasticprocess,SummaryofSourceCodingTheoremEfficiencyofCodesEntropyrateofastationaryMarkovchainWhatWeCanDo:AnalysisofSomeGoodCodesWhatW

4、eCanDo:AnalysisofSomeGoodCodesSummaryDUT应用信息论基础金明录教授DUT应用信息论基础金明录教授ReviewFunctionsofaMarkovchain.IfX1,X2,...,XnformastationaryMarkovchainandYi=φ((),Xi),thenSCSourceCoddiing1、Amotivatinggpexample2、Codesforrandomvariables3、ExamplesofCodesand4、Prefix-F

5、reeorInstantaneousCodesFreeorInstantaneousCodesDUT应用信息论基础金明录教授DUT应用信息论基础金明录教授AmotivatingexampleCodesforrandomvariablesYouwouldliketosetupyourowntelephonesystemthatconnectsyouNotation:theconcatenationoftwostringsxandyisdenotedbyxy.toyourthreebestfrie

6、nds.ThesetofallstringpgsoverafinitealphabetDisdenotedbyDכ.W.l.o.g.assumeD=0,1,...,D−1whereD=

7、D

8、.Thequestionishowtodesignefficientbinaryphonenumbers.Definition:asourcecodeforarandomvariableXisamapasourcecodeforarandomvariableXisamapInTable4.1youfind

9、sixdifferentwaysofhowyoucouldchoosethem.C:X→Dכx→C(x)(codeword)whereC(x)isthecodewordassociatedwithisthecodewordassociatedwithx,lx,l(x)isthelengthofisthelengthofC(x)ThelengthofacodeCisL(C)=EX[l(x)]DUT应用信息论基础金明录教授DUT应用信息论基础金明录教授CodesforrandomvariablesE

10、xampleofsomesourcecodesExamplesofCodesSourcepCodeⅠCodeⅡCodeⅢCodeⅣCodeⅤCisnonsingularifeveryelementofXmapsontoadifferentelementiofDכU1/20000001TheextensionofacodeC:Xכ→DכisthecodeU1/4010110012Cכ:Xכ→DכU1/8101001100113xn→Cכ(xn)=C(x1)C(x2)...C(xn