markov chains and random walks(Takis Konstantopoulos) by Takis Konstantopoulos

《markov chains and random walks(Takis Konstantopoulos) by Takis Konstantopoulos》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、IntroductorylecturenotesonMARKOVCHAINSANDRANDOMWALKSTakisKonstantopoulos∗Autumn2009∗cTakisKonstantopoulos2006-2009Contents1Introduction12ExamplesofMarkovchains22.1Amouseinacage.................................22.2Bankaccount....................................32.3Simplerandomwalk(drunk

2、ard’swalk)......................42.4Simplerandomwalk(drunkard’swalk)inacity.................42.5Actuarialchains..................................53TheMarkovproperty63.1DefinitionoftheMarkovproperty........................63.2Transitionprobabilityandinitialdistribution..................6

3、3.3Time-homogeneouschains.............................74Stationarity94.1Findingastationarydistribution.........................135Topologicalstructure155.1Thegraphofachain................................155.2Therelation“leadsto”...............................155.3Therelation“communicat

4、eswith”........................165.4Period........................................186Hittingtimesandfirst-stepanalysis207Gambler’sruin248StoppingtimesandthestrongMarkovproperty279RegenerativestructureofaMarkovchain2810Recurrenceandtransience2910.1Firsthittingtimedecomposition...........

5、..............3110.2Communicatingclassesandrecurrence/transience................3211Positiverecurrence33i12Law(=Theorem)ofLargeNumbersinProbabilityTheory3413LawofLargeNumbersforMarkovchains3413.1Functionsofexcursions..............................3513.2Averagereward.....................

6、..............3514Constructionofastationarydistribution3915Positiverecurrenceisaclassproperty4116Uniquenessofthestationarydistribution4317Structureofastationarydistribution4418Couplingandstability4518.1Definitionofstability................................4518.2Thefundamentalstabilitythe

7、oremforMarkovchains.............4718.3Coupling.......................................4718.4Proofofthefundamentalstabilitytheorem...................4918.5Terminology.....................................5118.6Periodicchains...................................5119LimitingtheoryforMarkovc