资源描述：
《南邮数字通信课件chapter 6》由会员上传分享，免费在线阅读，更多相关内容在教育资源-天天文库。
1、Chapter6Chapter6IntroductiontoInformationTheoryThischapterwillintroduceinformationsourcescodingandchannelcodingfromtheinformationtheoryangleofviewThefirstpartofthischapter:SCSourceCoddiingThTheorem6.1Mathmodelsforinformationsourcesa.Discretesources(DMS&non-DMSbutstationary):Lpossibleletters{xx121,2
2、,....,xL},pkk=PX()()1==x,∑pk1k=1b.Analogsources(Band-limitedsources):∞nsi[2in[2πWtWt(−n/2/2)]WW)]X()tX=−∑(22WW)π(t−n/2W)(6.11)−∞6.2ALogarithmicmeasureofinformationSelf-information1I()lxx==ogl−logP(),bitsornatt(bs(base2ore)iiPx()iInformationofeventX=xiperoutperoutputxofofthethesourceiMutualinformati
3、onPxy(
4、)ijIxy(;)log==Iyx(;)ijjiPx()iInformationofpossibleoutcomesofinputandoutputconditionalself-information1Ixy(
5、)l=og=−logPxy(
6、)ijijPxy(
7、)ij∴I(x;y))=I(x))I(-I(x
8、
9、y))ijiijRTLossxxAveragemutualinformationandentropyPxy()Foralli(totaln)andj(totalm),ij=logPx()inmIXY(;)=∑∑PxyIxy(,)(;)ijijij=11=nmPxy(,)
10、ij=∑∑Pxy(,)lijogij==11PxPy()()ijTheaverageself-information(sourceentropy)nnHX()==∑∑PxIx()()ii−Px()liogPx()iii==11IXY(;)=HX()−HXY(
11、)((
12、)(6.210)6.2-10)LossRTxxexampleInformationmeasuresforcontinuousrandomvariables∞∫insteadof∑differentialentropy:HX()=−∫px()logPxdx()−∞∞∞averagpgeconditionalentropyy:H(X
13、
14、Y)=−∫∫p(x)logp(x
15、y)dxdy−∞∞−averagemutualinformationI(X;Y)=H(X)−H(X
16、Y)=H(Y)−H(Y
17、X)*ForYcontinuousandXdiscrete,see(3-2-20,21)inthefourthedition63Losslesscodingofinformation6.3LosslesscodingofinformationsourcesSourcecodingcanbeclassifiedinto(1)Losslesscoding(Discretesources)Thegoalisthatminimizethenu
18、mberofbitsThegoalisthatminimizethenumberofbitsbutthesourcecanbeperfectlyreconstructed.(2)Lossycoding(Analogsources)ThThegoallithtthdtisthatthedataarecompresseddsubjecttoamaximumtolerabledistortion.6.3-1TheLosslessSourceCodingTheoremTheLimitonlosslesssourcecoding,(Shannon’sfirsttheorem,1948)Lossless
19、sourcecodingtheorem:Losslesssourcecodingtheorem:LetXdenoteDMSwithentropyH(X).ThereexistsalosslesssourcecodeforthissourceatanyrateRifsourceatanyrateRifR>H(X).ThereexistsThereexistsnolosslesssourcecodeforthis
