资源描述：
《Chapter-51-Structural-estimation-of-markov-decision-processes_1994_Handbook-of-Econometrics》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、Chapter51STRUCTURALESTIMATIONOFMARKOVDECISIONPROCESSES*JOHNRUSTUniversityofWisconsinContents1.Introduction30822.SolvingMDP'sviadynamicprogramming:Abriefreview30882.1.Finite-horizondynamicprogrammingandtheoptimalityofMarkoviandecisionrules30892.2.Infinite-horizondynamicprogrammingandBe
2、llman'sequation30912.3.Bellman'sequation,contractionmappingsandoptimality30912.4.AgeometricseriesrepresentationforMDP's30942.5.Overviewofsolutionmethods30953.Econometricmethodsfordiscretedecisionprocesses30993.1.Alternativemodelsofthe"errorterm"31003.2.MaximumlikelihoodestimationofDDP
3、's31013.3.Alternativeestimationmethods:Finite-horizonDDPproblems31183.4.Alternativeestimationmethods:Infinite-horizonDDP's31233.5.Theidentificationproblem31254.Empiricalapplications31304.1.Optimalreplacementofbusengines31304.2.Optimalretirementfromafinn3134References3139*Thisisanabrid
4、gedversionofamonograph,StochasticDecisionProcesses:Theory,Computation,andEstimationwrittenfortheLeifJohansenlecturesattheUniversityofOslointhefallof1991.IamgratefulforgenerousfinancialsupportfromtheCentralBankofNorwayandtheUniversityofOsloandcommentsfromJohnDagsvik,PeterFrengerandStei
5、narStr¢m.HandbookofEconometrics,Volume1V,EditedbyR.F.EngleandD.L.McFadden©1994ElsevierScienceB.V.Allrightsreserved3082JohnRust1.IntroductionMarkovdecisionprocesses(MDP)provideabroadframeworkformodellingsequentialdecisionmakingunderuncertainty.MDP'shavetwosortsofvariables:statevariable
6、sstandcontrolvariablesdr,bothofwhichareindexedbytimet=0,1,2,3....,T,wherethehorizonTmaybeinfinity.Adecision-makeroragentcanberepresentedbyasetofprimitives(u,p,~)whereu(st,dr)isautilityfunctionrepresentingtheagent'spreferencesattimet,p(st+1Is,d,)isaMarkovtransitionprobabilityrepresenti
7、ngtheagent'ssubjectivebeliefsaboutuncertainfuturestates,andfit(0,1)istherateatwhichtheagentdiscountsutilityinfutureperiods.Agentsareassumedtoberational:theybehaveaccordingtoanoptimaldecisionruledt=(~(St)thatsolvesvr(s)-max~Eo{Erofltu(s,,d,)lSo=s}whereEadenotesexpec-tationwithrespectto
8、theco
