the nonlinear state space model

《the nonlinear state space model》由会员上传分享，免费在线阅读，更多相关内容在工程资料-天天文库。

Chapter7ThenonlinearstatespacemodelInapplyingtheresultsandconceptsofPartIinthedomainsoftimesseriesorsystemstheory,wehavesofaranalyzedonlylinearmodelsinanydetail,albeitrathergeneralandmultidimensionalones.ThischapterisintendedasarelativelycompletedescriptionofthewayinwhichnonlinearmodelsmaybeanalyzedwithintheMarkoviancontextdevelopedthusfar.Wewillconsiderboththegeneralnonlinearstatespacemodel,andsomespecificapplicationswhichtakeonthisparticularform.Thepatternofthisanalysisistoconsiderfirstsomeparticularstructuralorsta-bilityaspectoftheassociateddeterministiccontrol,orCM(F),modelandthenunderappropriatechoiceofconditionsonthedisturbanceornoiseprocess(typicallyaden-sityconditionasinthelinearmodelsofSection6.3.2)toverifyarelatedstructuralorstabilityaspectofthestochasticnonlinearstatespaceNSS(F)model.Highlightsofthisdualityare(i)iftheassociatedCM(F)modelisforwardaccessible(aformofcontrollability),andthenoisehasanappropriatedensity,thentheNSS(F)modelisaT-chain(Section7.1);(ii)aformofirreducibility(theexistenceofagloballyattractingstatefortheCM(F)model)isthenequivalenttotheassociatedNSS(F)modelbeingaψ-irreducibleT-chain(Section7.2);(iii)theexistenceofperiodicclassesfortheforwardaccessibleCM(F)modelisfur-therequivalenttotheassociatedNSS(F)modelbeingaperiodicMarkovchain,withtheperiodicclassescoincidingforthedeterministicandthestochasticmodel(Section7.3).ThuswecanreinterpretsomeoftheconceptswhichwehaveintroducedforMarkovchainsinthisdeterministicsetting;andconversely,bystudyingthedeterministicmodelweobtaincriteriaforourbasicassumptionstobevalidinthestochasticcase.InSection7.4.3theadaptivecontrolmodelisconsideredtoillustratehowtheseresultsmaybeappliedinspecificapplications:forthismodelweexploitthefactthat146 7.1.Forwardaccessibilityandcontinuouscomponents147ΦisgeneratedbyaNSS(F)modeltogiveasimpleproofthatΦisaψ-irreducibleandaperiodicT-chain.Wewillendthechapterbyconsideringthenonlinearstatespacemodelwithoutforwardaccessibility,andshowinghowe-chainpropertiesmaythenbeestablishedinlieuoftheT-chainproperties.7.1Forwardaccessibilityandcontinuouscompo-nentsThenonlinearstatespacemodelNSS(F)maybeinterpretedasacontrolsystemdrivenbyanoisesequenceexactlyasthelinearmodelisinterpreted.Wewilltakesuchaview-pointinthissectionaswegeneralizetheconceptsusedintheproofofProposition6.3.3,whereweconstructedacontinuouscomponentforthelinearstatespacemodel.7.1.1ScalarmodelsandforwardaccessibilityWefirstconsiderthescalarmodelSNSS(F)definedbyXn=F(Xn−1,Wn),forsomesmooth(C∞)functionF:R×R→Randsatisfying(SNSS1)–(SNSS2).Recallthatin(2.5)wedefinedthemapFkinductively,forx0andwiarbitraryrealnumbers,byFk+1(x0,w1,...,wk+1)=F(Fk(x0,w1,...,wk),wk+1),sothatforanyinitialconditionX0=x0andanyk∈Z+,Xk=Fk(x0,W1,...,Wk).Nowlet{uk}betheassociatedscalar“controlsequence”forCM(F)asin(CM1),andusethistodefinetheresultingstatetrajectoryforCM(F)byxk=Fk(x0,u1,...,uk),k∈Z+.(7.1)Justasinthelinearcase,iffromeachinitialconditionx0∈Xasufficientlylargesetofstatesmaybereachedfromx0,thenwewillfindthatacontinuouscomponentmaybeconstructedfortheMarkovchainX.Itisnotimportantthateverystatemaybereachedfromagiveninitialcondition;themainideaintheproofofProposition6.3.3,whichcarriesovertothenonlinearcase,isthatthesetofpossiblestatesreachablefromagiveninitialconditionisnotconcentratedinsomelowerdimensionalsubsetofthestatespace.Recallalsothatwehaveassumedin(CM1)thatfortheassociateddeterministiccontrolmodelCM(F)withtrajectory(7.1),thecontrolsequence{uk}isconstrainedsothatuk∈Ow,k∈Z+,wherethecontrolsetOwisanopensetinR.Forx∈X,k∈Z,wedefineAk(x)tobethesetofallstatesreachablefromxat++timekbyCM(F):thatis,A0(x)={x},and+$%Ak(x):=F(x,u,...,u):u∈O,1≤i≤k,k≥1.(7.2)+k1kiw 148ThenonlinearstatespacemodelWedefineA+(x)tobethesetofallstateswhicharereachablefromxatsometimeinthefuture,givenby*∞A(x):=Ak(x).(7.3)++k=0Theanalogueofcontrollabilitythatweuseforthenonlinearmodeliscalledforwardaccessibility.ForwardaccessibilityTheassociatedcontrolmodelCM(F)iscalledforwardaccessibleifforeachx0∈X,thesetA+(x0)⊂Xhasnon-emptyinterior.Forgeneralnonlinearmodels,forwardaccessibilitydependscriticallyonthepartic-ularcontrolsetOwchosen.Thisisincontrasttothelinearstatespacemodel,whereconditionsonthedrivingmatrixpair(F,G)sufficedforcontrollability.Nonetheless,forthescalarnonlinearstatespacemodelwemayshowthatforwardaccessibilityisequivalenttothefollowing“rankcondition”,similarto(LCM3):RankconditionforthescalarCM(F)model(CM2)Foreachinitialconditionx0∈Rthereexistsk∈Zanda0+sequence(u0,...,u0)∈Oksuchthatthederivative1kw∂000∂000Fk(x0,u1,...,uk)|···|Fk(x0,u1,...,uk)(7.4)∂u1∂ukisnon-zero.Inthescalarlinearcasethecontrolsystem(7.1)hastheformxk=Fxk−1+Guk,withFandGscalars.Inthisspecialcasethederivativein(CM2)becomesexactly[Fk−1G|···|FG|G],whichshowsthattherankcondition(CM2)isageneralizationofthecontrollabilitycondition(LCM3)forthelinearstatespacemodel.Thisconnectionwillbestrengthenedwhenweconsidermultidimensionalnonlinearmodelsbelow.Theorem7.1.1.ThecontrolmodelCM(F)isforwardaccessibleifandonlyiftherankcondition(CM2)issatisfied.Aproofofthisresultwouldtakeustoofarfromthepurposeofthisbook.ItissimilartothatofProposition7.1.2,anddetailsmaybefoundin[271,272]. 7.1.Forwardaccessibilityandcontinuouscomponents1497.1.2ContinuouscomponentsforthescalarnonlinearmodelUsingthecharacterizationofforwardaccessibilitygiveninTheorem7.1.1wenowshowhowthisconditiononCM(F)leadstotheexistenceofacontinuouscomponentfortheassociatedSNSS(F)model.Todothisweneedtoincreasethestrengthofourassumptionsonthenoiseprocess,aswedidforthelinearmodelortherandomwalk.DensityfortheSNSS(F)model(SNSS3)ThedistributionΓofWisabsolutelycontinuous,withadensityγwonRwhichislowersemicontinuous.ThecontrolsetfortheSNSS(F)modelistheopensetOw:={x∈R:γw(x)>0}.Weknowfromthedefinitionsthat,withprobabilityone,Wk∈Owforallk∈Z+.Commonlyassumednoisedistributionssatisfyingthisassumptionincludethosewhichpossessacontinuousdensity,suchastheGaussianmodel,oruniformdistributionsonboundedopenintervalsinR.Wecannowdevelopanexplicitcontinuouscomponentforsuchscalarnonlinearstatespacemodels.Proposition7.1.2.SupposethatfortheSNSS(F)model,thenoisedistributionsatis-fies(SNSS3),andthattheassociatedcontrolsystemCM(F)isforwardaccessible.ThentheSNSS(F)modelisaT-chain.ProofSinceCM(F)isforwardaccessiblewehavefromTheorem7.1.1thattherankcondition(CM2)holds.Forsimplicityofnotation,assumethatthederivativewithrespecttothekthdisturbancevariableisnon-zero:∂Fk000(x0,w1,...,wk)=0(7.5)∂wkwith(w0,...,w0)∈Ok.DefinethefunctionFk:R×Ok→R×Ok−1×Ras1kwwwFk(x,w,...,w)=x,w,...,w,x,01k01k−1kwherex=F(x,w,...,w).ThetotalderivativeofFkcanbecomputedaskk01k10···0....0..DFk=,...10∂Fk∂Fk···∂Fk∂x0∂w1∂wk 150Thenonlinearstatespacemodelwhichisevidentlyfullrankat(x0,w0,...,w0).ItfollowsfromtheInverseFunction01kTheoremthatthereexistsanopensetB=Bx0×Bw0×···×Bw0,01kcontaining(x0,w0,...,w0),andasmoothfunctionGk:{Fk{B}}→Rk+1suchthat01kGk(Fk(x,w,...,w))=(x,w,...,w),01k01kforall(x0,w1,...,wk)∈B.TakingGtobethefinalcomponentofGk,weseethatforall(x,w,...,w)∈B,k01kGk(x0,w1,...,wk−1,xk)=Gk(x0,w1,...,wk−1,Fk(x0,w1,...,wk))=wk.Wenowmakeachangeofvariables,similartothelinearcase.Foranyx0∈Bx0,and0anypositivefunctionf:R→R+,

Pkf(x)=···f(F(x,w,...,w))γ(w)···γ(w)dw···dw(7.6)0k01kwkw11k

≥···f(Fk(x0,w1,...,wk))γw(wk)···γw(w1)dw1···dwk.Bw0Bw01kWewillfirstintegrateoverwk,keepingtheremainingvariablesfixed.Bymakingthechangeofvariablesxk=Fk(x0,w1,...,wk),wk=Gk(x0,w1,...,wk−1,xk),sothat∂Gkdwk=|(x0,w1,...,wk−1,xk)|dxk,∂xkweobtainfor(x0,w1,...,wk−1)∈Bx0×···×Bw0,0k−1

f(Fk(x0,w1,...,wk))γw(wk)dwk=f(xk)qk(x0,w1,...,wk−1,xk)dxk(7.7)B0Rwkwherewedefine,withξ:=(x0,w1,...,wk−1,xk),k∂Gkqk(ξ):=I{G(ξ)∈B}γw(Gk(ξ))|(ξ)|.∂xkSinceqispositiveandlowersemicontinuousontheopensetFk{B},andzeroonkFk{B}c,itfollowsthatqislowersemicontinuousonRk+1.kDefinethekernelT0foranarbitraryboundedfunctionfas

T0f(x0):=···f(xk)qk(ξ)γw(w1)···γw(wk−1)dw1···dwk−1dxk.(7.8)ThekernelTisnon-trivialatx0since00000∂Gk0000qk(ξ)γw(w1)···γw(wk−1)=|(ξ)|γw(wk)γw(w1)···γw(wk−1)>0,∂xk 7.1.Forwardaccessibilityandcontinuouscomponents151whereξ0=(x0,w0,...,w0,x0).WewillshowthatTfislowersemicontinuouson01k−1k0Rwheneverfispositiveandbounded.Sinceqk(x0,w1,...,wk−1,xk)γw(w1)···γw(wk−1)isalowersemicontinuousfunc-tionofitsargumentsinRk+1,thereexistsasequenceofpositive,continuousfunctionsr:Rk+1→R,i∈Z,suchthatforeachi,thefunctionrhasboundedsupportand,i++iasi↑∞,ri(x0,w1,...,wk−1,xk)↑qk(x0,w1,...,wk−1,xk)γw(w1)···γw(wk−1)foreach(x,w,...,w,x)∈Rk+1.DefinethekernelTusingras01k−1kii
Tif(x0):=f(xk)ri(x0,w1,...,wk−1,xk)dw1···dwk−1dxk.RkItfollowsfromthedominatedconvergencetheoremthatTifiscontinuousforanyboundedfunctionf.Iffisalsopositive,thenasi↑∞,Tif(x0)↑T0f(x0),x0∈R,whichimpliesthatT0fislowersemicontinuouswhenfispositive.Using(7.6)and(7.7)weseethatTisacontinuouscomponentofPkwhichisnon-0zeroatx0.FromTheorem6.2.4,themodelisaT-chainasclaimed.07.1.3SimplebilinearmodelTheforwardaccessibilityoftheSNSS(F)modelisusuallyimmediatesincetherankcondition(CM2)iseasilychecked.ToillustratetheuseofProposition7.1.2,andinparticularthecomputationofthe“controllabilityvector”(7.4)in(CM2),weconsiderthescalarexamplewhereΦisthebilinearstatespacemodelonX=Rdefinedin(SBL1)byXk+1=θXk+bWk+1Xk+Wk+1whereWisadisturbanceprocess.ToplacethisbilinearmodelintotheframeworkofthischapterweassumeDensityforthesimplebilinearmodel(SBL2)ThesequenceWisadisturbanceprocessonR,whosemarginaldistributionΓpossessesafinitesecondmoment,andadensityγwwhichislowersemicontinuous.Under(SBL1)and(SBL2),thebilinearmodelXisanSNSS(F)modelwithFdefinedin(2.7).Firstobservethattheone-steptransitionkernelPforthismodelcannotpossessaneverywherenon-trivialcontinuouscomponent.Thismaybeseenfromthefactthat 152ThenonlinearstatespacemodelP(−1/b,{−θ/b})=1,yetP(x,{−θ/b})=0forallx=−1/b.ItfollowsthattheonlypositivelowersemicontinuousfunctionwhichismajorizedbyP(·,{−θ/b})iszero,andthusanycontinuouscomponentTofPmustbetrivialat−1/b:thatis,T(−1/b,R)=0.Thiscouldbeanticipatedbylookingatthecontrollabilityvector(7.4).Thefirstordercontrollabilityvectoris∂F(x0,u1)=bx0+1,∂uwhichiszeroatx0=−1/b,andthusthefirstordertestforforwardaccessibilityfails.Hencewemusttakek≥2in(7.4)ifwehopetoconstructacontinuouscomponent.Whenk=2thevector(7.4)canbecomputedusingthechainruletogive∂F∂F∂F(x1,u2)(x0,u1)|(x1,u2)∂x∂u∂u=[(θ+bu2)(bx0+1)|bx1+1]=[(θ+bu)(bx+1)|θbx+b2ux+bu+1]200101whichisnon-zeroforalmosteveryu1∈R2.Hencetheassociatedcontrolmodelisu2forwardaccessible,andthistogetherwithProposition7.1.2givesProposition7.1.3.If(SBL1)and(SBL2)hold,thenthebilinearmodelisaT-chain.7.1.4MultidimensionalmodelsMostnonlinearprocessesthatareencounteredinapplicationscannotbemodeledbyascalarMarkovianmodelsuchastheSNSS(F)model.ThemoregeneralNSS(F)modelisdefinedby(NSS1),andwenowanalyzethisinasimilarwaytothescalarmodel.WeagaincalltheassociatedcontrolsystemCM(F)withtrajectoriesxk=Fk(x0,u1,...,uk),k∈Z+,(7.9)forwardaccessibleifthesetofattainablestatesA+(x),definedas*∞$%A+(x):=Fk(x,u1,...,uk):ui∈Ow,1≤i≤k,k≥1,(7.10)k=0hasnon-emptyinteriorforeveryinitialconditionx∈X.Toverifyforwardaccessibilitywedefineafurthergeneralizationofthecontrollabilitymatrixintroducedin(LCM3).Forx0∈Xandasequence{uk:uk∈Ow,k∈Z+}let{Ξk,Λk:k∈Z+}denotethematrices∂FΞk+1=Ξk+1(x0,u1,...,uk+1):=,∂x(xk,uk+1)∂FΛk+1=Λk+1(x0,u1,...,uk+1):=,∂u(xk,uk+1) 7.1.Forwardaccessibilityandcontinuouscomponents153wherex=F(x,u,...,u).LetCk=Ck(u,...,u)denotethegeneralizedcon-kk01kx0x01ktrollabilitymatrix(alongthesequenceu1,...,uk)Ck:=[Ξ···ΞΛ|Ξ···ΞΛ|···|ΞΛ|Λ].(7.11)x0k21k32kk−1kIfFtakesthelinearformF(x,u)=Fx+Gu,(7.12)thenthegeneralizedcontrollabilitymatrixagainbecomesCk=[Fk−1G|···|G],x0whichisthecontrollabilitymatrixintroducedin(LCM3).RankconditionforthemultidimensionalCM(F)model(CM3)Foreachinitialconditionx∈Rn,thereexistsk∈Zanda0+sequenceu0=(u0,...,u0)∈Oksuchthat1kwrankCk(u0)=n.(7.13)x0ThecontrollabilitymatrixCkisthederivativeofthestatex=F(y,u,...,u)yk1kattimekwithrespecttotheinputsequence(u,...,u).Thefollowingresultisk1aconsequenceofthisfacttogetherwiththeImplicitFunctionTheoremandSard’sTheorem(see[173,272]andtheproofofProposition7.1.2fordetails).Proposition7.1.4.ThenonlinearcontrolmodelCM(F)satisfying(7.9)isforwardaccessibleifandonlytherankcondition(CM3)holds.Toconnectforwardaccessibilitytothestochasticmodel(NSS1)weagainassumethatthedistributionofWpossessesadensity.DensityfortheNSS(F)model(NSS3)ThedistributionΓofWpossessesadensityγonRpwhichiswlowersemicontinuous,andthecontrolsetfortheNSS(F)modelistheopensetOw:={x∈R:γw(x)>0}.Usinganargumentwhichissimilarto,butmorecomplicatedthantheproofofProposition7.1.2,wemayobtainthefollowingconsequenceofforwardaccessibility. 154ThenonlinearstatespacemodelProposition7.1.5.IftheNSS(F)modelsatisfiesthedensityassumption(NSS3),andtheassociatedcontrolmodelisforwardaccessible,thenthestatespaceXmaybewrittenastheunionofopensmallsets,andhencetheNSS(F)modelisaT-chain.NotethatthisonlyguaranteestheT-chainproperty:wenowmoveontoconsidertheequallyneededirreducibilitypropertiesoftheNNS(F)models.7.2MinimalsetsandirreducibilityWenowdevelopamoredetaileddescriptionofreachablestatesandtopologicalirre-ducibilityforthenonlinearstatespaceNSS(F)model,andexhibitmoreoftheinterplaybetweenthestochasticandtopologicalcommunicationstructuresforNSS(F)models.SinceoneofthemajorgoalshereistoexhibitfurtherthelinksbetweenthebehavioroftheassociateddeterministiccontrolmodelandtheNSS(F)model,itisfirsthelpfultostudythestructureoftheaccessiblesetsforthecontrolsystemCM(F)withtrajectories(7.9).AlargepartofthisanalysisdealswithaclassofsetscalledminimalsetsforthecontrolsystemCM(F).Inthissectionwewilldevelopcriteriafortheirexistenceandpropertiesoftheirtopologicalstructure.ThiswillallowustodecomposethestatespaceofthecorrespondingNSS(F)modelintodisjoint,closed,absorbingsetswhicharebothψ-irreducibleandtopologicallyirreducible.7.2.1MinimalityforthedeterministiccontrolmodelWedefineA+(E)tobethesetofallstatesattainablebyCM(F)fromthesetEatsometimek≥0,andweletE0denotethosestateswhichcannotreachthesetE:*A(E):=A(x),E0:={x∈X:A(x)∩E=∅}.+++x∈EBecausethefunctionsFk(·,u1,...,uk)havethesemi-grouppropertyFk+j(x0,u1,...,uk+j)=Fj(Fk(x0,u1,...,uk),uk+1,...,uk+j),forx∈X,u∈O,k,j∈Z,thesetmaps{Ak:k∈Z}alsohavethisproperty:0iw+++thatis,k+jkjA+(E)=A+(A+(E)),E⊂X,k,j∈Z+.IfE⊂XhasthepropertythatA+(E)⊂E,thenEiscalledinvariant.Forexample,forallC⊂X,thesetsA(C)andC0are+invariant,andsincetheclosure,union,andintersectionofinvariantsetsisinvariant,theset∞$*∞%Ω+(C):=Ak+(C)(7.14)N=1k=Nisalsoinvariant.Thefollowingresultsummarizestheseobservations: 7.2.Minimalsetsandirreducibility155Proposition7.2.1.Forthecontrolsystem(7.9)wehaveforanyC⊂X,(i)A+(C)andA+(C)areinvariant;(ii)Ω+(C)isinvariant;(iii)C0isinvariant,andC0isalsoclosedifthesetCisopen.AsaconsequenceoftheassumptionthatthemapFissmooth,andhencecontinuous,wethenhaveimmediatelyProposition7.2.2.IftheassociatedCM(F)modelisforwardaccessible,thenfortheNSS(F)model:(i)aclosedsubsetA⊂XisabsorbingforNSS(F)ifandonlyifitisinvariantforCM(F);(ii)ifU⊂Xisopen,thenforeachk≥1andx∈X,Ak(x)∩U=∅⇐⇒Pk(x,U)>0;+(iii)ifU⊂Xisopen,thenforeachx∈X,A+(x)∩U=∅⇐⇒Kaε(x,U)>0.WenowintroduceminimalsetsforthegeneralCM(F)model.MinimalsetsWecallasetminimalforthedeterministiccontrolmodelCM(F)ifitis(topologically)closed,invariant,anddoesnotcontainanyclosedinvariantsetasapropersubset.Forexample,considertheLCM(F,G)modelintroducedin(1.4).Theassumption(LCM2)simplystatesthatthecontrolsetOisequaltoRp.wInthiscasethesystempossessesauniqueminimalsetMwhichisequaltoX0,therangespaceofthecontrollabilitymatrix,asdescribedafterProposition4.4.3.Iftheeigenvaluecondition(LSS5)holdsthenthisistheonlyminimalsetfortheLCM(F,G)model.Thefollowingcharacterizationsofminimalityfollowdirectlyfromthedefinitions,andthefactthatbothA+(x)andΩ+(x)areclosedandinvariant.Proposition7.2.3.ThefollowingareequivalentforanonemptysetM⊂X:(i)MisminimalforCM(F);(ii)A+(x)=Mforallx∈M;(iii)Ω+(x)=Mforallx∈M. 156Thenonlinearstatespacemodel7.2.2M-Irreducibilityandψ-irreducibilityProposition7.2.3assertsthatanystateinaminimalsetcanbe“almostreached”fromanyotherstate.ThispropertyissimilarinflavortotopologicalirreducibilityforaMarkovchain.ThelinkbetweentheseconceptsisgiveninthefollowingcentralresultfortheNSS(F)model.Theorem7.2.4.LetM⊂XbeaminimalsetforCM(F).IfCM(F)isforwardacces-sibleandthedisturbanceprocessoftheassociatedNSS(F)modelsatisfiesthedensitycondition(NSS3),then(i)thesetMisabsorbingforNSS(F);(ii)theNSS(F)modelrestrictedtoMisanopensetirreducible(andsoψ-irreducible)T-chain.ProofThatMisabsorbingfollowsdirectlyfromProposition7.2.3,provingM=A+(x)forsomex;Proposition7.2.1,provingA+(x)isinvariant;andProposition7.2.2,provinganyclosedinvariantsetisabsorbingfortheNSS(F)model.ToseethattheprocessrestrictedtoMistopologicallyirreducible,letx0∈M,andletU⊆XbeanopensetforwhichU∩M=∅.ByProposition7.2.3wehaveA+(x0)∩U=∅.HencebyProposition7.2.2Kaε(x0,U)>0,whichestablishesopensetirreducibility.Theprocessisthenψ-irreduciblefromProposition6.2.2sinceweknowitisaT-chainfromProposition7.1.5.Clearly,undertheconditionsofTheorem7.2.4,ifXitselfisminimalthentheNSS(F)modelisbothψ-irreducibleandopensetirreducible.TheconditionthatXbemini-malisastrongrequirementwhichwenowweakenbyintroducingadifferentformof“controllability”forthecontrolsystemCM(F).WesaythatthedeterministiccontrolsystemCM(F)isindecomposableifitsstatespaceXdoesnotcontaintwodisjointclosedinvariantsets.ThisconditionisclearlynecessaryforCM(F)topossessauniqueminimalset.Indecomposabilityisnotsufficienttoensuretheexistenceofaminimalset:takeX=R,Ow=(0,1),andxk+1=F(xk,uk+1)=xk+uk+1,sothatallproperclosedinvariantsetsareoftheform[t,∞)forsomet∈R.Thissystemisindecomposable,yetnominimalsetsexist.IrreduciblecontrolmodelsIfCM(F)isindecomposableandalsopossessesaminimalsetM,thenCM(F)willbecalledM-irreducible.IfCM(F)isM-irreducibleitfollowsthatM0=∅:otherwiseMandM0wouldbedisjointnon-emptyclosedinvariantsets,contradictingindecomposability.Toestablish 7.3.Periodicityfornonlinearstatespacemodels157necessaryandsufficientconditionsforM-irreducibilityweintroduceaconceptfromdynamicalsystemstheory.Astatex∈Xiscalledgloballyattractingifforally∈X,x∈Ω(y).+Thefollowingresulteasilyfollowsfromthedefinitions.Proposition7.2.5.(i)Thenonlinearcontrolsystem(7.9)isM-irreducibleifandonlyifagloballyattractingstateexists.(ii)IfagloballyattractingstatexexiststhentheuniqueminimalsetisequaltoA(x)=Ω(x).++Wecannowprovidethedesiredconnectionbetweenirreducibilityofthenonlinearcontrolsystemandψ-irreducibilityforthecorrespondingMarkovchain.Theorem7.2.6.SupposethatCM(F)isforwardaccessibleandthedisturbanceprocessoftheassociatedNSS(F)modelsatisfiesthedensitycondition(NSS3).ThentheNSS(F)modelisψ-irreducibleifandonlyifCM(F)isM-irreducible.ProofIftheNSS(F)modelisψ-irreducible,letxbeanystateinsuppψ,andletUbeanyopensetcontainingx.Bydefinitionwehaveψ(U)>0,whichimpliesthatK(x,U)>0forallx∈X.ByProposition7.2.2itfollowsthatxisgloballyaεattracting,andhenceCM(F)isM-irreduciblebyProposition7.2.5.Conversely,supposethatCM(F)possessesagloballyattractingstate,andletUbeanopenpetitesetcontainingx.ThenA(x)∩U=∅forallx∈X,whichby+Proposition7.2.2andProposition5.5.4impliesthattheNSS(F)modelisψ-irreducibleforsomeψ.7.3PeriodicityfornonlinearstatespacemodelsWenowlookattheperiodicstructureofthenonlinearNSS(F)modeltoseehowthecyclesofSection5.4.3canbefurtherdescribed,andinparticulartheirtopologicalstructureelucidated.WefirstdemonstratethatminimalsetsforthedeterministiccontrolmodelCM(F)exhibitperiodicbehavior.Thisperiodicityextendstothestochasticframeworkinanaturalway,andundermildconditionsonthedeterministiccontrolsystem,wewillseethattheperiodisinfacttrivial,sothatthechainisaperiodic.7.3.1PeriodicityforcontrolmodelsTodevelopaperiodicstructureforCM(F)wemimictheconstructionofacycleforanirreducibleMarkovchain.Todothiswefirstrequireadeterministicanalogueofsmallsets:wesaythatthesetCisk-accessiblefromthesetB,foranyk∈Z+,ifforeachy∈B,C⊂Ak(y).+ 158ThenonlinearstatespacemodelkThiswillbedenotedB−→C.FromtheImplicitFunctionTheorem,inamannersimilartotheproofofProposition7.1.2,wecanimmediatelyconnectk-accessibilitywithforwardaccessibility.Proposition7.3.1.SupposethattheCM(F)modelisforwardaccessible.Thenforeachx∈X,thereexistopensetsBx,Cx⊂X,withx∈Bxandanintegerkx∈Z+kxsuchthatBx−→Cx.InordertoconstructacycleforanirreducibleMarkovchain,wefirstconstructedaνn-smallsetAwithνn(A)>0.AsimilarconstructionisnecessaryforCM(F).Lemma7.3.2.SupposethattheCM(F)modelisforwardaccessible.IfMisminimalforCM(F)thenthereexistsanopensetE⊂M,andanintegern∈Z+,suchthatnE−→E.ProofUsingProposition7.3.1wefindthatthereexistopensetsBandC,andankintegerkwithB−→C,suchthatB∩M=∅.SinceMisinvariant,itfollowsthatC⊂A+(B∩M)⊂M,(7.15)andbyProposition7.2.1,minimality,andthehypothesisthatthesetBisopen,A+(x)∩B=∅(7.16)foreveryx∈M.Combining(7.15)and(7.16)itfollowsthatAm(c)∩B=∅forsomem∈Z,and++c∈C.BycontinuityofthefunctionFweconcludethatthereexistsanopensetE⊂CsuchthatAm(x)∩B=∅forallx∈E.+ThesetEsatisfiestheconditionsofthelemmawithn=m+ksincebythesemi-groupproperty,An(x)=Ak(Am(x))⊃Ak(Am(x)∩B)⊃C⊃E+++++forallx∈E.CallafiniteorderedcollectionofdisjointclosedsetsG:={Gi:1≤i≤d}aperiodicorbitifforeachi,A1(G)⊂G,i=1,...,d(modd).+ii+1TheintegerdiscalledtheperiodofG.ThecyclicresultforCM(F)isgiveninTheorem7.3.3.SupposethatthefunctionF:X×Ow→Xissmooth,andthatthesystemCM(F)isforwardaccessible.IfMisaminimalset,thenthereexistsanintegerd≥1,anddisjointclosedsets!dG={Gi:1≤i≤d}suchthatM=i=1Gi,andGisaperiodicorbit.ItisuniqueinthesensethatifHisanotherperiodicorbitwhoseunionisequaltoMwithperiodd,thenddividesd,andforeachithesetHmaybewrittenasaunionofsetsfromiG. 7.3.Periodicityfornonlinearstatespacemodels159ProofUsingLemma7.3.2wecanfixanopensetEwithE⊂M,andanintegerkksuchthatE−→E.DefineI⊂Z+bynI:={n≥1:E−→E}.(7.17)Thesemi-grouppropertyimpliesthatthesetIisclosedunderaddition:forifi,j∈I,thenforallx∈E,i+jijjA+(x)=A+(A+(x))⊃A+(E)⊃E.Letddenoteg.c.d.(I).TheintegerdwillbecalledtheperiodofM,andMwillbecalledaperiodicwhend=1.For1≤i≤dwedefine*∞kd−iGi:={x∈M:A+(x)∩E=∅}.(7.18)k=1!dByProposition7.2.1itfollowsthatM=i=1Gi.SinceEisanopensubsetofM,itfollowsthatforeachi∈Z+,thesetGiisopenintherelativetopologyonM.Oncewehaveshownthatthesets{Gi}aredisjoint,itwillfollowthattheyareclosedintherelativetopologyonM.SinceMitselfisclosed,thiswillimplythatforeachi,thesetGiisclosed.Wenowshowthatthesets{Gi}aredisjoint.Supposethatonthecontraryx∈Gi∩Gjforsomei=j.Thenthereexistski,kj∈Z+suchthatAkid−i(y)∩E=∅andAkjd−j(y)∩E=∅(7.19)++wheny=x.SinceEisopen,wemayfindanopensetO⊂Xcontainingxsuchthat(7.19)holdsforally∈O.ByProposition7.2.1,thereexistsv∈Eandn∈Z+suchthatAn(v)∩O=∅.(7.20)+k0By(7.20),(7.19),andsinceE−→Ewehaveforδ=i,j,andallz∈E,Ak0+kδd−δ+n+k0(z)⊃Ak0+kδd−δ+n(E)++⊃Ak0+kδd−δ(An(v)∩O)++⊃Ak0(Akδd−δ(An(v)∩O)∩E)⊃E.+++Thisshowsthat2k0+kδd−δ+n∈Iforδ=i,j,andthiscontradictsthedefinitionofd.Weconcludethatthesets{Gi}aredisjoint.WenowshowthatGisaperiodicorbit.Letx∈Gi,andu∈Ow.Sincethesets{Gi}formadisjointcoverofMandsinceMisinvariant,thereexistsaunique1≤j≤dsuchthatF(x,u)∈Gj.Itfollowsfromthesemi-grouppropertythatx∈Gj−1,andhencei=j−1.Theuniquenessofthisconstructionfollowsfromthedefinitiongiveninequation(7.18).ThefollowingconsequenceofTheorem7.3.3furtherillustratesthetopologicalstruc-tureofminimalsets. 160ThenonlinearstatespacemodelProposition7.3.4.UndertheconditionsofTheorem7.3.3,ifthecontrolsetOwisconnected,thentheperiodicorbitGconstructedinTheorem7.3.3ispreciselyequaltotheconnectedcomponentsoftheminimalsetM.Inparticular,inthiscaseMisaperiodicifandonlyifitisconnected.nProofFirstsupposethatMisaperiodic.LetE−→E,andconsiderafixedstatev∈E.ByaperiodicityandLemmaD.7.4thereexistsanintegerN0withthepropertythate∈Ak(v)(7.21)+forallk≥N.SinceAk(v)isthecontinuousimageoftheconnectedsetv×Ok,the0+wset*∞A(AN0(v))=Ak(v)(7.22)+++k=N0isconnected.Itsclosureisthereforealsoconnected,andbyProposition7.2.1theclosureoftheset(7.22)isequaltoM.Theperiodiccaseistreatedsimilarly.FirstweshowthatforsomeN0∈Z+wehave*∞Gd=Akd+(v),k=N0wheredistheperiodofM,andeachofthesetsAkd(v),k≥N,containsv.+0ThisshowsthatGdisconnected.Next,observethatG1=A1+(Gd),andsincethecontrolsetOwandGdarebothconnected,itfollowsthatG1isalsoconnected.Byinduction,eachofthesets{Gi:1≤i≤d}isconnected.7.3.2PeriodicityAlloftheresultsdescribedabovedealingwithperiodicityofminimalsetswereposedinapurelydeterministicframework.Wenowreturntothestochasticmodeldescribedby(NSS1)–(NSS3)toseehowthedeterministicformulationofperiodicityrelatestothestochasticdefinitionwhichwasintroducedforMarkovchainsinSection5.4.Asonemighthope,theconnectionsareverystrong.Theorem7.3.5.IftheNSS(F)modelsatisfiesconditions(NSS1)–(NSS3)andtheassociatedcontrolmodelCM(F)isforwardaccessiblethen:(i)ifMisaminimalset,thentherestrictionoftheNSS(F)modeltoMisaψ-irreducibleT-chain,andtheperiodicorbit{Gi:1≤i≤d}⊂MwhoseexistenceisguaranteedbyTheorem7.3.3isψ-a.e.equaltothed-cycleconstructedinThe-orem5.4.4;(ii)ifCM(F)isM-irreducible,andifitsuniqueminimalsetMisaperiodic,thentheNSS(F)modelisaψ-irreducibleaperiodicT-chain. 7.4.Forwardaccessibleexamples161ProofTheproofof(i)followsdirectlyfromthedefinitions,andtheobservationthatbyreducingEifnecessary,wemayassumethatthesetEwhichisusedintheproofofTheorem7.3.3issmall.HencethesetEplaysthesameroleasthesmallsetusedintheproofofTheorem5.2.1.Theproofof(ii)followsfrom(i)andTheorem7.2.4.7.4ForwardaccessibleexamplesWenowseehowspecificmodelsmaybeviewedinthisgeneralcontext.Itwillbecomeapparentthatwithoutmakinganyunnaturalassumptions,bothsimplemodelssuchasthedependentparameterbilinearmodel,andrelativelymorecomplexnonlinearmodelssuchasthegumleafattractorwithnoiseandadaptivecontrolmodelscanbehandledwithinthisframework.7.4.1ThedependentparameterbilinearmodelThedependentparameterbilinearmodelisasimpleNSS(F)modelwherethefunctionFisgivenin(2.15)byYZαθ+ZF,=.(7.23)θWθY+WUsingProposition7.1.4itiseasytoseethattheassociatedcontrolmodelisforwardaccessible,andthenthemodeliseasilyanalyzed.WehaveProposition7.4.1.ThedependentparameterbilinearmodelΦsatisfyingassump-tions(DBL1)–(DBL2)isaT-chain.Iffurtherthereexistssomeonez∗∈Osuchzthatz∗||<1,(7.24)1−αthenΦisψ-irreducibleandaperiodic.ZProofWiththenoiseconsidereda“control”,thefirstordercontrollabilityWmatrixmaybecomputedtogiveθ11∂Y110Cθ,y=Z=.∂101W1θThecontrolmodelisthusforwardaccessible,andhenceΦ=isaT-chain.YSupposenowthatthebound(7.24)holdsforz∗andletw∗denoteanyelementofO⊆R.IfZandWaresetequaltoz∗andw∗respectivelyin(7.23)thenask→∞wkkθz∗(1−α)−1k∗→x:=.Ykw∗(1−α)(1−α−z∗)−1Thestatex∗isgloballyattracting,anditimmediatelyfollowsfromProposition7.2.5andTheorem7.2.6thatthechainisψ-irreducible.Aperiodicitythenfollowsfromthefactthatanycyclemustcontainthestatex∗. 162Thenonlinearstatespacemodel7.4.2ThegumleafattractorConsidertheNSS(F)modelwhosesamplepathsevolvetocreatetheversionofthe“gumleafattractor”illustratedinFigure2.3.Thismodelisgivenin(2.12)byXa−1/Xa+1/XbWX=n=n−1n−1+nnXbXa0nn−1whichisoftheform(NSS1),withtheassociatedCM(F)modeldefinedasabxa−1/x+1/xuFxb,u=a+.(7.25)x0Fromtheformulae∂F(1/xa)2−(1/xb)2∂F1==∂x10∂u0weseethatthesecondordercontrollabilitymatrixisgivenby(1/xa)21C2(u,u)=1x01210awherex=x0andxa=−1/xa+1/xb+u.Hence,sinceC2isfullrankfor0xb1001x00allx0,u1andu2,itfollowsthatthecontrolsystemisforwardaccessible.ApplyingProposition7.2.6givesProposition7.4.2.TheNSS(F)model(2.12)isaT-chainifthedisturbancesequenceWsatisfiescondition(NSS3).7.4.3TheadaptivecontrolmodelTheadaptivecontrolmodeldescribedby(2.22)–(2.24)isofthegeneralformoftheNSS(F)modelandtheresultsoftheprevioussectionarewellsuitedtotheanalysisofthisspecificexampleAnapparentdifficultywiththismodelisthatthestatespaceXisnotanopensubsetofEuclideanspace,sothatthegeneralresultsobtainedfortheNSS(F)modelmaynotseemtoapplydirectly.However,givenourassumptionsonthemodel,theinteriorofthestatespace,(σ,σz)×R2,isabsorbing,andisreachedinonestepwithz1−α2probabilityonefromeachinitialcondition.Hencetoobtainacontinuouscomponent,andtoaddressperiodicityfortheadaptivemodel,wecanapplythegeneralresultsobtainedforthenonlinearstatespacemodelsbyfirstrestrictingΦtotheinteriorofX.Proposition7.4.3.If(SAC1)and(SAC2)holdfortheadaptivecontrolmodeldefinedby(2.22)–(2.24),andifσ2<1,thenΦisaψ-irreducibleandaperiodicT-chain.z 7.5.Equicontinuityandthenonlinearstatespacemodel163ProofToprovetheresultweshowthattheassociateddeterministiccontrolmodelforthenonlinearstatespacemodeldefinedby(2.22)–(2.24)isforwardaccessibleand,fortheassociateddeterministiccontrolsystem,agloballyattractingpointexists.Thesecond-ordercontrollabilitymatrixhastheform2220−2ασwΣ1Y100∂(Σ,θ˜,Y)(Σ1Y2+σ2)222221wCΦ0(Z2,W2,Z1,W1):=∂(Z,W,Z,W)=••1•2211••01where“•”denotesavariablewhichdoesnotaffecttherankofthecontrollabilitymatrix.ItisevidentthatC2isfullrankwheneverY=θ˜Y+Wisnon-zero.Φ01001ThisshowsthatforeachinitialconditionΦ∈X,thematrixC2isfullrankfora.e.0Φ0{(Z,W),(Z,W)}∈R4,andsotheassociatedcontrolmodelisforwardaccessible,1122andhencethestochasticmodelisaT-chainbyProposition7.1.5.ZItiseasilycheckedthatifissetequaltozeroin(2.22)–(2.23)then,sinceα<1Wandσ2<1,zσ2Φ→(z,0,0)ask→∞.k1−α2ThisshowsthatthecontrolmodelassociatedwiththeMarkovchainΦisM-irreducible,andhencebyProposition7.2.6thechainitselfisψ-irreducible.Thelimitabovealsoshowsthateveryelementofacycle{Gi}fortheuniqueminimalsetmustcontainthe2point(σz,0,0).FromProposition7.3.4itfollowsthatthechainisaperiodic.1−α27.5Equicontinuityandthenonlinearstatespacemodel7.5.1e-ChainpropertiesofnonlinearstatespacemodelsWehaveseeninthischapterthattheNSS(F)modelisaT-chainifthenoisevariable,viewedasacontrol,can“steerthestateprocessΦ”toasufficientlylargesetofstates.Iftheforwardaccessibilitypropertydoesnotholdthenthechainmustbeanalyzedusingdifferentmethods.TheprocessisalwaysaFellerMarkovchain,becauseofthecontinuityofF,asshowninProposition6.1.2.InthissectionwesearchforconditionsunderwhichtheprocessΦisalsoane-chain.TodothisweconsiderthesensitivityprocessassociatedwiththeNSS(F)model,definedby∇Φ=Iand0∇Φ=[DF(Φ,w)]∇Φ,k∈Z(7.26)k+1kk+1k+Φwhere∇takesvaluesinthesetofn×nmatrices,andDFdenotesthederivativeofFwithrespecttoitsfirstvariable.Since∇Φ=Iitfollowsfromthechainruleandinductionthatthesensitivityprocess0isinfactthederivativeofthepresentstatewithrespecttotheinitialstate:thatis,d∇Φ=Φforallk∈Z.kk+dΦ0 164ThenonlinearstatespacemodelThemainresultinthissectionconnectsstabilityofthederivativeprocesswithequicontinuityofthetransitionfunctionforΦ.Sincethesystem(7.26)iscloselyrelatedtothesystem(NSS1),linearizedaboutthesamplepath(Φ0,Φ1,...),itisreasonableΦtoexpectthatthestabilityofΦwillbecloselyrelatedtothestabilityof∇.Theorem7.5.1.Supposethat(NSS1)–(NSS3)holdfortheNSS(F)model.Thenlet-ting∇ΦdenotethederivativeofΦwithrespecttoΦ,k∈Z,wehavekk0+(i)ifforsomeopenconvexsetN⊂X,E[sup∇Φ]<∞(7.27)kΦ0∈Nthenforallx∈N,dE[Φ]=E[∇Φ];xkxkdx(ii)supposethat(7.27)holdsforallsufficientlysmallneighborhoodsNofeachy0∈X,andfurtherthatforanycompactsetC⊂X,supsupE[∇Φ]<∞.yky∈Ck≥0ThenΦisane-chain.ProofThefirstresultisaconsequenceoftheDominatedConvergenceTheorem.Toprovethesecondresult,letf∈C(X)∩C∞(X).ThencdkdΦPf(x)=Ex[f(Φk)]≤f∞Ex[∇k]dxdxwhichbytheassumptionsof(ii),impliesthatthesequenceoffunctions{Pkf:k∈Z}+isequicontinuousoncompactsubsetsofX.SinceC∞∩CisdenseinC,thiscompletescctheproof.Itmayseemthatthetechnicalassumption(7.27)willbedifficulttoverifyinpractice.However,wecanimmediatelyidentifyonelargeclassofexamplesbyconsideringthecasewherethei.i.d.processWisuniformlybounded.ItfollowsfromthesmoothnessconditiononFthatsup∇ΦisalmostsurelyfiniteforanycompactsubsetN⊂X,Φ0∈Nkwhichshowsthatinthiscase(7.27)istriviallysatisfied.Thefollowingresultprovidesanotherlargeclassofmodelsforwhich(7.27)issatis-fied.ObservethattheconditionsimposedonWinProposition7.5.2aresatisfiedforanyi.i.d.Gaussianprocess.Theproofisstraightforward.Proposition7.5.2.FortheMarkovchaindefinedby(NSS1)–(NSS3),supposethatFisarationalfunctionofitsarguments,andthatforsomeε0>0,E[exp(ε0|W1|)]<∞.Thenletting∇ΦdenotethederivativeofΦwithrespecttoΦ,wehaveforanycompactkk0setC⊂X,andanyk≥0,E[sup∇Φ]<∞.kΦ0∈C 7.6.Commentary*165Henceundertheseconditions,dE[Φ]=E[∇Φ].xkxkdx7.5.2LinearstatespacemodelsWecaneasilyspecializeTheorem7.5.1togiveconditionsunderwhichalinearmodelisane-chain.Proposition7.5.3.SupposetheLSS(F,G)modelXsatisfies(LSS1)and(LSS2),andthattheeigenvaluecondition(LSS5)alsoholds.ThenΦisane-chain.mm−1iProofUsingtheidentityXm=FX0+i=0FGWm−iweseethat∇Φ=Fm,kwhichtendstozeroexponentiallyfast,byLemma6.3.4.TheconditionsofTheo-rem7.5.1arethussatisfied,whichcompletestheproof.ObservethatProposition7.5.3usestheeigenvaluecondition(LSS5),thesameas-sumptionwhichwasusedinProposition4.4.3toobtainψ-irreducibilityfortheGaussianmodel,andthesameconditionthatwillbeusedtoobtainstabilityinlaterchapters.TheanalogousProposition6.3.3usescontrollabilitytogiveconditionsunderwhichthelinearstatespacemodelisaT-chain.Notethatcontrollabilityisnotrequiredhere.Otherspecificnonlinearmodels,suchasbilinearmodels,canbeanalyzedsimilarlyusingthisapproach.7.6Commentary*WehavealreadynotedthatinthedegeneratecasewherethecontrolsetOwconsistsofasinglepoint,theNSS(F)modeldefinesasemi-dynamicalsystemwithstatespaceX,andinfactmanyoftheconceptsintroducedinthischapteraregeneralizationsofstandardconceptsfromdynamicalsystemstheory.Threestandardapproachestothequalitativetheoryofdynamicalsystemsaretopo-logicaldynamicswhoseprincipaltoolispointsettopology;ergodictheory,whereoneassumes(orproves,frequentlyusingacompactnessargument)theexistenceofaner-godicinvariantmeasure;andfinally,thedirectmethodofLyapunov,whichconcernscriteriaforstability.ThelattertwoapproacheswillbedevelopedinastochasticsettinginPartsIIandIII.Thischapteressentiallyfocusedongeneralizationsofthefirstapproach,whichisalsobasedupon,toalargeextent,thestructureandexistenceofminimalsets.Twoexcellentexpositionsinapurelydeterministicandcontrol-freesettingarethebooksbyBhatiaandSzeg¨o[34]andBrown[55].Saperstone[346]considersinfinitedimensionalspacessothat,inparticular,themethodsmaybeapplieddirectlytothedynamicalsystemonthespaceofprobabilitymeasureswhichisgeneratedbyaMarkovprocesses. 166ThenonlinearstatespacemodelTheconnectionsbetweencontroltheoryandirreducibilitydescribedherearetakenfromMeyn[259]andMeynandCaines[272,271].ThedissertationsofChan[61]andMokkadem[286],andalsoDieboltandGu´egan[92],treatdiscretetimenonlinearstatespacemodelsandtheirassociatedcontrolmodels.Dieboltin[91]considersnonlinearmodelswithadditivenoiseoftheformΦk+1=F(Φk)+Wk+1usinganapproachwhichisverydifferenttothatdescribedhere.JakubsczykandSontagin[173]presentasurveyoftheresultsobtainableforforwardaccessiblediscretetimecontrolsystemsinapurelydeterministicsetting.Theygiveadifferentcharacterizationofforwardaccessibility,basedupontherankofanassociatedLiealgebra,ratherthanacontrollabilitymatrix.TheoriginoftheapproachtakeninthischapterliesintheoftencitedpaperbyStroockandVaradhan[378].Thereitisshownthatthesupportofthedistributionofadiffusionprocessmaybecharacterizedbyconsideringanassociatedcontrolmodel.IchiharaandKunitain[167]andKliemannin[211]usethisapproachtodevelopanergodictheoryfordiffusions.Theinvariantcontrolsetsof[211]maybecomparedtominimalsetsasdefinedhere.Atthisstage,introductionofthee-chainclassofmodelsisnotwellmotivated.ThereaderwhowishestoexplorethemimmediatelyshouldmovetoChapter12.InDuflo[102],aconditioncloselyrelatedtothestabilityconditionwhichweimposeΦon∇isusedtoobtaintheCentralLimitTheoremforanonlinearstatespacemodel.DufloassumesthatthefunctionFsatisfies|F(x,w)−F(y,w)|≤α(w)|x−y|whereαisafunctiononOwsatisfying,forsomesufficientlylargem,E[α(W)m]<1.ItiseasytoseethatanyprocessΦgeneratedbyanonlinearstatespacemodelsatisfyingthisboundisane-chain.FormodelsmorecomplexthanthelinearmodelofSection7.5.2itwillnotbeaseasyΦtoprovethat∇convergestozero,soalengthierstabilityanalysisofthissensitivityΦprocessmaybenecessary.Since∇isessentiallygeneratedbyarandomlinearsystemitisthereforelikelytoeitherconvergetozeroorevanesce.ItseemsprobablethatthestochasticLyapunovfunctionapproachofKushner[232]orKhas’minskii[206],oramoredirectanalysisbaseduponlimittheoremsforproductsofrandommatricesasdevelopedin,forinstance,FurstenbergandKesten[134]willbeΦwellsuitedforassessingthestabilityof∇.Commentaryforthesecondedition:Theconjecturevoicedinthefirsteditionwasconfirmedtenyearsafteritwasfirstputintoprint.AstochasticLyapunovapproachisintroducedin[165]forverificationofstabilityofthesensitivityprocess1foraclassofMarkovmodels.AsignificantomissioninthefirsteditionisanydiscussionoftherelationshipbetweenΦstabilityofthesensitivityprocess∇andLyapunovexponents(see[212,255]).Fora1Thesensitivityprocesswascalledthederivativeprocessinthefirstedition. 7.6.Commentary*167giveninitialconditionx,thetopLyapunovexponentisdefinedastherandomvariable1Λ:=limsuplog∇Φ.xnn→∞nThechoiceofnormisarbitrary.Thereisalsoaversiondefinedinexpectation:foranyp>0denote1Λ(p):=limsuplogE[∇Φp].xxnn→∞nOneapproachtoestablishingthee-chainpropertyistoshowthatΛx(p)isindependentofx,andnegativeforallpsufficientlysmall[165].MethodsforestimatingtheLyapunovexponentandconditionsforverifyingequicon-tinuityareestablishedforversionsoftheNSS(F)model,incontinuousordiscretetime,inseveralrecentpapersunderavarietyofassumptions[370,371,22,165,20,323].AhiddenMarkovmodel(HMM)isaMarkovchainΦ,alongwithanobservationprocessYevolvingonastatespaceY.Itisassumedthatthereisani.i.d.sequenceDevolvingonitsownstatespaceD,alongwithafunctionG:X×D→YsuchthattheobservationprocesscanbeexpressedasanoisyfunctionofthechainYn=G(Φn,Dn),n≥0.TheconditionaldistributionofXngivenY0,...,Ynisdenotedˆπn.ItisknownthatΥn:=(Yn,πˆn)isitselfaMarkovchain[106,107],butonethatisrarelyψ-irreducible.Consequentlyweareforcedtoconsideralternativeapproachestoaddressstabilityofthefilteringprocess{πˆn}.LyapunovexponentsaswellasequicontinuityhaveprovedvaluableintheanalysisofΥ.LyapunovexponentsforΥareexaminedinaseriesofpapersbyZeitouniandcoauthors[85,11].UndercertainconditionsonthemodeltheLyapunovexponentΛxisnegativeandindependentofx,whichimpliesthatthefilterisinsensitivetoitsinitialcondition.Thee-chainpropertyisestablisheddirectlyin[87,213],underconditionsmoregeneralthan[11].TherecentsurveyofChiganskyetal.[68]containsanextensivebibliography. PartIISTABILITYSTRUCTURES