欢迎来到天天文库
浏览记录
ID:35996859
大小:126.63 KB
页数:10页
时间:2019-04-29
《信息论试题new》由会员上传分享,免费在线阅读,更多相关内容在教育资源-天天文库。
1、EEE315InformationTheoryandCodingAssignment1DatePerformed:2011.11.4DateSubmitted:2011.11.59IntroductionInformationtheoryanswerstwofundamentalquestionsincommunicationtheory:whatistheultimatedatacompressionandwhatistheultimatetransmissionrateofcommu-nic
2、ation.ThesetwoaspectscanbealsoregardastheentropyHandthechannelcapacityC.Intheearly1940s,Shannonraisedthatrandomprocesseshaveanirreduci-blecomplexitybelowwhichthesignalcannotbecompressandthishenamedentropy.Healsoarguedthatiftheentropyofthesourceisless
3、thanthecapacityofthechannel,thenasymptoticallyerrorfreecommunicationcanbeachieved.Shannon'sinformationcontentShannon’sinformationcontentshortforSICalsonamedasself-information.Ininfor-mationtheory,itisameasureoftheinformationcontentcontainsinasingleev
4、ent.Bydefinition,theamountofSICcontainedinaprobabilisticeventdependsonlyontheprobabilityofthatevent,andSIChasaninverserelationshipwithprobability.ThenaturalmeasureoftheuncertaintyofaneventXistheprobabilityofXdenotebypx.Bydefinition,theinformationcont
5、entinaneventasInfo{X}=-logpxThemeasureofinformationhassomeintuitivepropertiessuchas:1.Informationcontainedintheeventsoughttobedefinedintermsofsomemeasureofuncertaintyoftheevent.2.Lesscertaineventsoughttocontainmoreinformationthanmorecertainevents.3.T
6、heinformationofunrelatedeventstakenasasingleeventshouldequalthesumoftheinformationoftheunrelatedevents.TheunitofSICis“bits”ifbase2isusedforthelogarithm,and“nats”ifthenaturallogarithmisused.EntropyTheentropyquantifiestheexpectedvalueoftheinformationco
7、ntainedinamessage.Theentropycanbeviewedas:1.Ameasureoftheminimumcostneededtosendsomeformofinformation.2.“Theamountofsurprisefactor”oftheinformationmeasuredinbits.91.Orhowmuchenergyitisworthspendingtocarrytheinformationwhichtranslatestotheminimumnumbe
8、rofbitsneededtocodetheinformation.TheentropyisdefinedasHX=-x∈XpxlogpxItcanbeviewedfromanumberofperspectives:1.TheaverageSICofX2.Theamountofinformationgainedifitsvaluesareknown.3.Theaveragenumberofbinaryquestionneededtofindoutitsvalueisin[H(X),H(X)+1]
此文档下载收益归作者所有