Chapter_3._Information_Theory_Notes

《Chapter_3._Information_Theory_Notes》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、InformationTheory建模Informationtheorydealswithmathematicalmodellingandanalysisofacommunicationssystemratherthanwithphysicalsourcesandphysicalchannels.物理Specifically,foraninformationsourceandanoisychannel,informationtheoryprovideslimitson:1.Theminimumnumberofbitspersymbolrequir

2、edtofullyrepresentthesource.(i.e.theefficiencywithwhichinformationfromagivensourcecanberepresented.)2.Themaximumrateatwhichreliable(error-free)communicationscantakeplaceoveranoisychannel.Sincethewholepurposeofacommunicationssystemistotransportinformationfromasourcetoadestinat

3、ion,thequestionarisesastohowmuchinformationcanbetransmittedinagiventime.(Normallythegoalwouldbetotransmitasmuchinformationaspossibleinassmallatimeaspossiblesuchthatthisinformationcanbecorrectlyinterpretedatthedestination.)解释Thisofcourseleadstothefollowingquestions,whichare;量化

4、Howcaninformationbequantified?最佳率Ifwearetosendinformationatanoptimalrate,howdowemeasuretherateatwhichinformationisemittedfromasource?排放•Supposethatweobservetheoutputemittedbyadiscretesource(everyunitintervalorsignallinginterval.)离散随机事件•Thesourceoutputcanbeconsideredasaset,S,o

5、fdiscreterandomevents(oroutcomes).Theseeventsaresymbolsfromafixedfinitealphabet.符号有限字母(Forexample,theset,oralphabet,couldbethenumbers1to6onadie.Thealphabetisfinitebecausethereisalimitednumberofpossibleoutcomes.Eachrollofthedieoutputsasymbol(i.e.thenumberonthedieupperfacewhent

6、hediecomestorest).(Anotherexampleisadigitalbinarysource,wherethefinitealphabetconsistsofthedigits"0"and"1“.Thesourceoutputseithera1or0symbolatrandom)•If,ingeneral,weconsideradiscreterandomsourcewhichoutputssymbolsfromafixedfinitealphabetconsistingofksymbols,thenthesetScontain

7、sallksymbolsandwecanwrite;i(k1)thesymbolprobabilitiessumto1i.e.allpossiblesymbolsareelementsofSandasymbolSs0,s1,s3,......sk1andp(si)1willbeemittedbythesourceateveryunitinterval.i0•Inaddition,weassumethesymbolsemittedbythesourceduringsuccessivesignalling统计独立区间interval

8、stobestatisticallyindependent(i.e.theprobabilityofanysymbolbeingemit