duke_math_applied_stochastic_process_class_note 英文文献资料

《duke_math_applied_stochastic_process_class_note 英文文献资料》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、AnidiosyncraticintroductiontostochasticprocessesClassnotesforMath216Notes-Fall2010JonathanC.MattinglySeptember22,20101FiniteStateMarkovChainsAdiscretetimestochasticprocess(Xn)n0isacollectionofrandomvariablesindexedbythenon-negativeintegersZ+=fn2Z:n0g.ThesetinwhichtheXntakevaluesiscalledthest

2、atespaceofthestochasticprocess.Denition.Astochasticprocess(Xn)n0isaMarkovchainifP(Xn+1=jjXn=in;


;X0=i0)=P(Xn+1=jjXn=in)forallj;in;


;i02I.Denition.AMarkovchainistimehomogeneousifforallk2Z+andi;j2IP(Xk+1=ijXk=j)=P(X1=ijX=j)UnlesswesayotherwisewewillalwaysassumethatallMarkovchainsaretimeho

3、mogeneous.Insuchcaseswewillwritepn(i;j)=P(Xn=jjX0=i)BytheMarkovpropertyonehasP(Xn=xn;Xn1xn1;


X1=x1jX0=x0)=p1(xn;xn1)p1(xn1;xn2)


p1(x1;x0)WewillbeginbyconcentratingonstochasticprocessesonanitestatespaceI.Withoutlossofgenerality,wecantakethestatespacetobeI=f0;1::::;Ng.1.1Markovchainsa

4、ndmatricesThereisaveryfruitfulcorrespondencebetweennitestateMarkovchainsandMatrices.WebeginbyconsideringrandomvariablesonastatespaceI=f0;:::;N1g.SucharandomvariableXcanbespeciedcompletelybyNnon-negativenumbersfi:i2IgsuchthatP(X=i)=i:PClearlywehavethati2Ii=1.Itisconvenienttoorganizetheii

5、narow-vector=(0;:::;N1)2RN.ThevectoriscalledthedistributionoftherandomvariableX.Withthisinmindwemakethefollowingdenition.Denition.Arowvector=(;:::;)2RNcalledadistributionif0.Ifinaddition0N1iPN1i=0i=1,itiscalledaprobabilitydistribution.1LetP2RN;Nbeamatrixwithnon-negativeentries.W

6、ewillwritePfortheijthentryofP,thati;jistosay01p0;0


p0;N1B...CP=@......ApN1;0


pN1;N1Denition.AsquarematrixPwithnon-negativeentriesiscalledastochasticmatrixifallrowssumPtoone.Thatistosay,forallj,iPji=1.Stochasticmatricesareinone-to-onecorrespondencewithtimehomogeneousMarkovprocessesona

7、nitestatespace.ThecorrespondenceisgivenbyPij=P(X1=jjX0=i)ItthenfollowsthatP(X=jjX=i)=(Pn)n0ijOrinotherwords,thedistributionoftherandomvariableXnwhenconditionedtohaveX0=iisgivenbytherowvector(Pn)bywhichwemeantheithrowofthematrixPn.Inoth