Analysis of Time Series Data Using R.pdf

《Analysis of Time Series Data Using R.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

AnalysisofTimeSeriesDataUsingR1ZONGWUCAIa,b,cE-mailaddress:zcai@uncc.eduaDepartmentofMathematics&StatisticsandDepartmentofEconomics,UniversityofNorthCarolina,Charlotte,NC28223,U.S.A.bWangYananInstituteforStudiesinEconomics,XiamenUniversity,ChinacCollegeofEconomicsandManagement,ShanghaiJiaotongUniversity,ChinaJuly30,2006c2006,ALLRIGHTSRESERVEDbyZONGWUCAI1Thismanuscriptmaybeprintedandreproducedforindividualorinstructionaluse,butmaynotbeprintedforcommercialpurposes. PrefaceThepurposeofthislecturenotesisdesignedtoprovideanoverviewofmethodsthatareusefulforanalyzingunivariateandmultivariatephenomenameasuredovertime.Sincethisisacourseemphasizingapplicationswithboththeoryandapplications,thereaderisguidedthroughexamplesinvolvingrealtimeseriesinthelectures.Acollectionofsimpletheoreticalandappliedexercisesassumingabackgroundthatincludesabeginninglevelcourseinmathematicalstatisticsandsomecomputingskillsfollowseachchapter.Moreimportantly,thecomputercodeinRanddatasetsareprovidedformostofexamplesanalyzedinthislecturenotes.SomematerialsarebasedonthelecturenotesgivenbyProfessorRobertH.Shumway,DepartmentofStatistics,UniversityofCaliforniaatDavisandmycolleague,ProfessorStanislavRadchenko,DepartmentofEconomics,UniversityofNorthCarolinaatCharlotte.SomedatasetsareprovidedbyProfessorRobertH.Shumway,DepartmentofStatistics,Uni-versityofCaliforniaatDavisandProfessorPhillipsHansFransesatUniversityofRotterdam,Netherland.Iamverygratefultothemforprovidingtheirlecturenotesanddatasets. Contents1PackageRandSimpleApplications11.1ComputationalToolkits.............................11.2HowtoInstallR?................................31.3DataAnalysisandGraphicsUsingR–AnIntroduction(109pages).....41.4CRANTaskView:EmpiricalFinance......................41.5CRANTaskView:ComputationalEconometrics................82CharacteristicsofTimeSeries152.1Introduction....................................152.2StationaryTimeSeries..............................172.2.1Detrending................................212.2.2Differencing................................242.2.3Transformations..............................262.2.4LinearFilters...............................282.3OtherKeyFeaturesofTimeSeries.......................302.3.1Seasonality................................302.3.2AberrantObservations..........................332.3.3ConditionalHeteroskedasticity......................342.3.4Nonlinearity................................362.4TimeSeriesRelationships............................382.4.1AutocorrelationFunction.........................392.4.2CrossCorrelationFunction........................402.4.3PartialAutocorrelationFunction....................452.5Problems......................................492.6ComputerCode..................................532.7References.....................................663UnivariateTimeSeriesModels693.1Introduction....................................693.2LeastSquaresRegression.............................743.3ModelSelectionMethods.............................793.3.1SubsetApproaches............................793.3.2SequentialMethods............................833.3.3LikelihoodBased-Criteria........................853.3.4Cross-ValidationandGeneralizedCross-Validation..........87ii CONTENTSiii3.3.5PenalizedMethods............................883.4IntegratedModels-I(1).............................903.5AutoregressiveModels-AR(p).........................933.5.1Model...................................933.5.2Forecasting................................993.6MovingAverageModels–MA(q)........................1023.7AutoregressiveIntegratedMovingAverageModel-ARIMA(p,d,q).....1063.8SeasonalARIMAModels.............................1083.9RegressionModelsWithCorrelatedErrors...................1203.10EstimationofCovarianceMatrix........................1303.11LongMemoryModels...............................1333.12PeriodicityandBusinessCycles.........................1363.13ImpulseResponseFunction...........................1413.13.1FirstOrderDifferenceEquations....................1423.13.2HigherOrderDifferenceEquations...................1463.14Problems......................................1503.15ComputerCode..................................1573.16References.....................................1824Non-stationaryProcessesandStructuralBreaks1854.1Introduction....................................1854.2RandomWalks..................................1884.2.1InappropriateDetrending........................1894.2.2Spurious(nonsense)Regressions.....................1904.3UnitRootandStationaryProcesses.......................1904.3.1ComparisonofForecastsofTSandDSProcesses...........1914.3.2RandomWalkComponentsandStochasticTrends...........1934.4TrendEstimationandForecasting........................1944.4.1ForecastingaDeterministicTrend....................1944.4.2ForecastingaStochasticTrend......................1954.4.3ForecastingARMAmodelswithDeterministicTrends.........1954.4.4ForecastingofARIMAModels......................1964.5UnitRootTests..................................1974.5.1TheDickey-FullerandAugmentedDickey-FullerTests........1974.5.2Cautions..................................2004.6StructuralBreaks.................................2004.6.1TestingforBreaks............................2014.6.2ZivotandAndrews’sTestingProcedure.................2034.6.3Cautions..................................2054.7Problems......................................2054.8ComputerCode..................................2094.9References.....................................213 CONTENTSiv5VectorAutoregressiveModels2155.1Introduction....................................2155.1.1PropertiesofVARModels........................2185.1.2StatisticalInferences...........................2205.2Impulse-ResponseFunction...........................2225.3VarianceDecompositions.............................2255.4GrangerCausality.................................2265.5Forecasting....................................2295.6Problems......................................2295.7References.....................................2336Cointegration2346.1Introduction....................................2346.2CointegratingRegression.............................2356.3TestingforCointegration.............................2366.4CointegratedVARModels............................2396.5Problems......................................2426.6References.....................................2437NonparametricDensity,Distribution&QuantileEstimation2447.1MixingConditions................................2447.2DensityEstimate.................................2457.2.1AsymptoticProperties..........................2467.2.2Optimality.................................2497.2.3BoundaryCorrection...........................2517.3DistributionEstimation.............................2547.3.1SmoothedDistributionEstimation...................2547.3.2RelativeEfficiencyandDeficiency....................2577.4QuantileEstimation...............................2587.4.1ValueatRisk...............................2587.4.2NonparametricQuantileEstimation...................2607.5ComputerCode..................................2617.6References.....................................2648NonparametricRegressionEstimation2678.1BandwidthSelection...............................2678.1.1SimpleBandwidthSelectors.......................2678.1.2Cross-ValidationMethod.........................2688.2MultivariateDensityEstimation.........................2708.3RegressionFunction...............................2718.4KernelEstimation.................................2738.4.1AsymptoticProperties..........................2748.4.2BoundaryBehavior............................2778.5LocalPolynomialEstimate............................2798.5.1Formulation................................279 CONTENTSv8.5.2ImplementationinR...........................2808.5.3ComplexityofLocalPolynomialEstimator...............2818.5.4PropertiesofLocalPolynomialEstimator...............2848.5.5BandwidthSelection...........................2898.6FunctionalCoefficientModel...........................2928.6.1Model...................................2928.6.2LocalLinearEstimation.........................2938.6.3BandwidthSelection...........................2948.6.4SmoothingVariableSelection......................2968.6.5Goodness-of-FitTest...........................2968.6.6AsymptoticResults............................2998.6.7ConditionsandProofs..........................3018.6.8MonteCarloSimulationsandApplications...............3118.7AdditiveModel..................................3118.7.1Model...................................3118.7.2BackfittingAlgorithm..........................3158.7.3ProjectionMethod............................3178.7.4Two-StageProcedure...........................3198.7.5MonteCarloSimulationsandApplications...............3228.8ComputerCode..................................3228.9References.....................................326 ListofTables3.1AICCvaluesfortenmodelsfortherecruitsseries...............984.1Large-samplecriticalvaluesfortheADFstatistic...............1984.2SummaryofDFtestforunitrootsintheabsenceofserialcorrelation....1994.3CriticalValuesoftheQLRstatisticwith15%Trimming...........2035.1SimsvariancedecompositioninthreevariableVARmodel..........2285.2Simsvariancedecompositionincludinginterestrates..............2286.1CriticalvaluesfortheEngle-GrangerADFstatistic..............2388.1Samplesizesrequiredforp-dimensionalnonparametricregressiontohavecom-parableperformancewiththatof1-dimensionalnonparametricregressionus-ingsize100....................................277vi ListofFigures2.1MonthlySOI(left)andsimulatedrecruitment(right)fromamodel(n=453months,1950-1987).................................182.2SimulatedMA(1)withθ1=0.9..........................212.3LogofannualindicesofrealnationaloutputinChina,1952-1988.......222.4Monthlyaveragetemperatureindegreescentigrade,January,1856-February2005,n=1790months.Thestraightline(wideandgreen)isthelineartrendy=−9.037+0.0046tandthecurve(wideandred)isthenonparametricestimatedtrend...................................232.5Detrendedmonthlyglobaltemperatures:leftpanel(linear)andrightpanel(nonlinear).....................................242.6Differencedmonthlyglobaltemperatures.....................252.7AnnualstockofmotorcyclesintheNetherlands,1946-1993..........262.8QuarterlyearningsforJohnson&Johnson(4thquarter,1970to1stquarter,1980,leftpanel)withlogtransformedearnings(rightpanel)..........272.9TheSOIseries(blacksolidline)comparedwitha12pointmovingaverage(redthickersolidline).Thetoppanel:originaldataandthebottompanel:filteredseries....................................292.10USRetailSalesDatafrom1967-2000.......................312.11Four-weeklyadvertisingexpendituresonradioandtelevisioninTheNether-lands,1978.01−1994.13..............................322.12Firstdifferenceinlogpricesversustheinflationrate:thecaseofArgentina,1970.1−1989.4...................................332.13Japanese-U.S.dollarexchangeratereturnseries{yt},fromJanuary1,1974toDecember31,2003...............................362.14QuarterlyunemploymentrateinGermany,1962.1−1991.4(seasonallyad-justedandnotseasonallyadjusted)intheleftpanel.Thescatterplotofun-employmentrate(seasonallyadjusted)versusunemploymentrate(seasonallyadjusted)oneperiodlaggedintherightpanel..................372.15MultiplelaggedscatterplotsshowingtherelationshipbetweenSOIandthepresent(xt)versusthelaggedvalues(xt+h)atlags1≤h≤16.........392.16AutocorrelationfunctionsofSOIandrecruitmentandcrosscorrelationfunc-tionbetweenSOIandrecruitment........................412.17MultiplelaggedscatterplotsshowingtherelationshipbetweentheSOIattimet+h,sayxt+h(x-axis)versusrecruitsattimet,sayyt(y-axis),0≤h≤15.42vii LISTOFFIGURESviii2.18MultiplelaggedscatterplotsshowingtherelationshipbetweentheSOIattimet,sayxt(x-axis)versusrecruitsattimet+h,sayyt+h(y-axis),0≤h≤15.422.19PartialautocorrelationfunctionsfortheSOI(leftpanel)andtherecruits(rightpanel)series.................................462.20VarvedataforProblem5.............................502.21GasandoilseriesforProblem6.........................512.22Handgunsales(per10,000,000)inCaliforniaandmonthlygundeathrate(per100,00)inCalifornia(February2,1980-December31,1998..........533.1Autocorrelationfunctions(ACF)forsimple(left)andlog(right)returnsforIBM(toppanels)andforthevalue-weightedindexofUSmarket(bottompanels),January1926toDecember1997.....................723.2Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthedetrended(toppanel)anddifferenced(bottompanel)globaltemper-atureseries.....................................773.3Atypicalrealizationoftherandomwalkseries(leftpanel)andthefirstdif-ferenceoftheseries(rightpanel).........................913.4Autocorrelationfunctions(ACF)(left)andpartialautocorrelationfunctions(PACF)(right)fortherandomwalk(toppanel)andthefirstdifference(bot-tompanel)series..................................923.5Autocorrelation(ACF)ofresidualsofAR(1)forSOI(leftpanel)andtheplotofAICandAICCvalues(rightpanel)......................993.6Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthelogvarveseries(toppanel)andthefirstdifference(bottompanel),showingapeakintheACFatlagh=1.....................1043.7Numberoflivebirths1948(1)−1979(1)andresidualsfrommodelswithafirstdifference,afirstdifferenceandaseasonaldifferenceoforder12andafittedARIMA(0,1,1)×(0,1,1)12model........................1113.8Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthebirthseries(toptwopanels),thefirstdifference(secondtwopanels)anARIMA(0,1,0)×(0,1,1)12model(thirdtwopanels)andanARIMA(0,1,1)×(0,1,1)12model(lasttwopanels).........................1123.9Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthelogJ&Jearningsseries(toptwopanels),thefirstdifference(sec-ondtwopanels),ARIMA(0,1,0)×(1,0,0)4model(thirdtwopanels),andARIMA(0,1,1)×(1,0,0)4model(lasttwopanels)...............1153.10Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forARIMA(0,1,1)×(0,1,1)4model(toptwopanels)andtheresidualplotsofARIMA(0,1,1)×(1,0,0)4(leftbottompanel)andARIMA(0,1,1)×(0,1,1)4model(rightbottompanel)............................1163.11MonthlysimplereturnofCRSPDecile1indexfromJanuary1960toDecember2003:Timeseriesplotofthesimplereturn(lefttoppanel),timeseriesplotofthesimplereturnafteradjustingforJanuaryeffect(righttoppanel),theACFofthesimplereturn(leftbottompanel),andtheACFoftheadjustedsimplereturn....................................119 LISTOFFIGURESix3.12Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthedetrendedlogJ&Jearningsseries(toptwopanels)andthefittedARIMA(0,0,0)×(1,0,0)4residuals.......................1233.13TimeplotsofU.S.weeklyinterestrates(inpercentages)fromJanuary5,1962toSeptember10,1999.Thesolidline(black)istheTreasury1-yearconstantmaturityrateandthedashedlinetheTreasury3-yearconstantmaturityrate(red).........................................1253.14ScatterplotsofU.S.weeklyinterestratesfromJanuary5,1962toSeptember10,1999:theleftpanelis3-yearrateversus1-yearrate,andtherightpanelischangesin3-yearrateversuschangesin1-yearrate.............1253.15ResidualseriesoflinearregressionModelIfortwoU.S.weeklyinterestrates:theleftpanelistimeplotandtherightpanelisACF..............1263.16TimeplotsofthechangeseriesofU.S.weeklyinterestratesfromJanuary12,1962toSeptember10,1999:changesintheTreasury1-yearconstantmaturityrateareindenotedbyblacksolidline,andchangesintheTreasury3-yearconstantmaturityrateareindicatedbyreddashedline.............1273.17Residualseriesofthelinearregressionmodels:ModelII(top)andModelIII(bottom)fortwochangeseriesofU.S.weeklyinterestrates:timeplot(left)andACF(right)..................................1273.18SampleautocorrelationfunctionoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(lefttoppanel)andequal-weighted(righttoppanel)indexes.ThelogspectraldensityoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(leftbottompanel)andequal-weighted(rightbottompanel)indexes...........................1353.19TheautocorrelationfunctionofanAR(2)model:(a)φ1=1.2andφ2=−0.35,(b)φ1=1.0andφ2=−0.7,(c)φ1=0.2andφ2=0.35,(d)φ1=−0.2andφ2=0.35......................................1393.20ThegrowthrateofUSquarterlyrealGNPfrom1947.IIto1991.I(seasonallyadjustedandinpercentage):theleftpanelisthetimeseriesplotandtherightpanelistheACF..................................1393.21Thetime-seriesytisgeneratedwithwt∼N(0,1),y0=5.Atperiodt=50,thereisanadditionalimpulsetotheerrorterm,i.e.we50=w50+1.Theimpulseresponsefunctioniscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse...............1433.22Thetime-seriesytisgeneratedwithwt∼N(0,1),y0=3.Atperiodt=50,thereisanadditionalimpulsetotheerrorterm,i.e.we50=w50+1.Theimpulseresponsefunctioniscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse...............1443.23Exampleofimpulseresponsefunctionsforfirstorderdifferenceequations...1463.24Thetimeseriesytisgeneratedwithwt∼N(0,1),y0=3.Forthetransitoryimpulse,thereisanadditionalimpulsetotheerrortermatperiodt=50,i.e.we50=w50+1.Forthepermanentimpulse,thereisanadditionalimpulseforperiodt=50,···,100,i.e.wet=wt+1,t=50,51,···,100.Theimpulseresponsefunction(IRF)iscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse...............147 LISTOFFIGURESx3.25Exampleofimpulseresponsefunctionsforsecondorderdifferenceequation..149 Chapter1PackageRandSimpleApplications1.1ComputationalToolkitsWhenyouworkwithlargedatasets,messydatahandling,models,etc,youneedtochoosethecomputationaltoolsthatareusefulfordealingwiththesekindsofproblems.Thereare“menudrivensystems”whereyouclicksomebuttonsandgetsomeworkdone-buttheseareuselessforanythingnontrivial.Todoseriouseco-nomicsandfinanceinthemoderndays,youhavetowritecom-puterprograms.Andthisistrueofanyfield,forexample,empiricalmacroeconomics-andnotjustof“computationalfinance”whichisahotbuzzwordrecently.Thequestionishowtochoosethecomputationaltools.Accord-ingtoAjayShah(December2005),youshouldpayattentiontofourelements:price,freedom,elegantandpowerfulcomputerscience,andnetworkeffects.Lowpriceisbetterthanhighprice.Price=0isobviouslybestofall.Freedomhereisinmanyaspects.Agoodsoftwaresystemisonethatdoesn’ttieyoudownintermsofhardware/OS,sothatyouareabletokeepmoving.Anotheraspectoffreedomisinworkingwithcolleagues,collaboratorsandstudents.Withcommercialsoftware,thisbecomesaproblem,becauseyourcolleaguesmaynothavethesamesoftwarethatyouareusing.Here1 CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS2freesoftwarereallywinsspectacularly.Goodpracticeinresearchin-volvesagreataccentonreproducibility.Reproducibilityisimportantbothsoastoavoidmistakes,andbecausethenextpersonworkinginyourfieldshouldbestandingonyourshoulders.Thisrequiresanabilitytoreleasecode.Thisisonlypossiblewithfreesoftware.SystemslikeSASandGaussusearchaiccomputerscience.Thecodeisinelegant.Thelanguageisnotpowerful.Inthisdayandage,writingCorfortranbyhandis“toolowlevel”.Hell,withGauss,evenaminimalthinglikeonlinehelpistawdry.Oneprefersasystemtobebuiltbypeoplewhoknowtheircomputerscience-itshouldbeanelegant,powerfullanguage.AllstandardCSknowledgeshouldbenicelyinplaytogiveyouagorgeoussystem.Goodcomputersciencegivesyoumoreproductivehumans.LotsofeconomistsuseGauss,andgiveoutGausssourcecode,sothereisanetworkeffectinfavorofGauss.AsimilarthingisrightnowhappeningwithstatisticiansandR.HereIcitecomparisonsamongmostcommonlyusedpackages(seeAjayShah(December2005));seethewebsiteathttp://www.mayin.org/ajayshah/COMPUTING/mytools.html.RisaveryconvenientprogramminglanguagefordoingstatisticalanalysisandMonteCarolsimulationsaswellasvariousapplicationsinquantitativeeconomicsandfinance.Indeed,weprefertothinkofitofanenvironmentwithinwhichstatisticaltechniquesareimple-mented.Iwillteachitattheintroductorylevel,butNOTICEthatyouwillhavetolearnRonyourown.Notethatabout97%ofcom-mandsinS-PLUSandRaresame.Inparticular,foranalyzingtimeseriesdata,Rhasalotofbundlesandpackages,whichcanbedown-loadedforfree,forexample,athttp://www.r-project.org/. CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS3R,likeS,isdesignedaroundatruecomputerlanguage,anditallowsuserstoaddadditionalfunctionalitybydefiningnewfunctions.MuchofthesystemisitselfwrittenintheRdialectofS,whichmakesiteasyforuserstofollowthealgorithmicchoicesmade.Forcomputationally-intensivetasks,C,C++andFortrancodecanbelinkedandcalledatruntime.AdvanceduserscanwriteCcodetomanipulateRobjectsdirectly.1.2HowtoInstallR?(1)gotothewebsitehttp://www.r-project.org/;(2)clickCRAN;(3)chooseasitefordownloading,sayhttp://cran.cnr.Berkeley.edu;(4)clickWindows(95andlater);(5)clickbase;(6)clickR-2.3.1-win32.exe(Versionof06-01-2006)tosavethisfilefirstandthenrunittoinstall.ThebasicRisinstalledintoyourcomputer.Ifyouneedtoinstallotherpackages,youneedtodothefollowings:(7)Afteritisinstalled,thereisanicononthescreen.ClicktheicontogetintoR;(8)Gotothetopandfindpackagesandthenclickit;(9)GodowntoInstallpackage(s)...andclickit;(10)Thereisanewwindow.Choosealocationtodownloadthepackages,sayUSA(CA1),movemousetothereandclickOK;(11)Thereisanewwindowlistingallpackages.YoucanselectanyoneofpackagesandclickOK,oryoucanselectallofthemandthenclickOK. CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS41.3DataAnalysisandGraphicsUsingR–AnIntro-duction(109pages)Seethefiler-notes.pdf(109pages)whichcanbedownloadedfromhttp://www.math.uncc.edu/˜zcai/r-notes.pdf.Iencourageyoutodownloadthisfileandlearnitbyyourself.1.4CRANTaskView:EmpiricalFinanceThisCRANTaskViewcontainsalistofpackagesusefulforempiricalworkinFinance,groupedbytopic.Besidesthesepackages,averywidevarietyoffunctionssuitableforempiricalworkinFinanceisprovidedbyboththebasicRsystem(anditssetofrecommendedcorepackages),andanumberofotherpackagesontheComprehen-siveRArchiveNetwork(CRAN).Consequently,severaloftheotherCRANTaskViewsmaycontainsuitablepackages,inparticulartheEconometricsTaskView.Thewebsiteishttp://cran.r-project.org/src/contrib/Views/Finance.html1.Standardregressionmodels:Linearmodelssuchasordi-naryleastsquares(OLS)canbeestimatedbylm()(frombythestatspackagecontainedinthebasicRdistribution).Max-imumLikelihood(ML)estimationcanbeundertakenwiththeoptim()function.Non-linearleastsquarescanbeestimatedwiththenls()function,aswellaswithnlme()fromthenlmepackage.Forthelinearmodel,avarietyofregressiondiagnostictestsareprovidedbythecar,lmtest,strucchange,urca,uroot,andsandwichpackages.TheRcmdrandZeligpack-agesprovideuserinterfacesthatmaybeofinterestaswell. CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS52.Timeseries:Classicaltimeseriesfunctionalityisprovidedbythearima()andKalmanLike()commandsinthebasicRdistribution.Thedsepackagesprovidesavarietyofmoreadvancedestimationmethods;fracdiffcanestimatefraction-allyintegratedseries;longmemocoversrelatedmaterial.Forvolatilymodeling,thestandardGARCH(1,1)modelcanbeestimatedwiththegarch()functioninthetseriespackage.Unitrootandcointegrationtestsareprovidedbytseries,urcaanduroot.TheRmetricspackagesfSeriesandfMultivarcontainanumberofestimationfunctionsforARMA,GARCH,longmemorymodels,unitrootsandmore.TheArDecim-plementsautoregressivetimeseriesdecompositioninaBayesianframework.Thedynanddynlmaresuitablefordynamic(lin-ear)regressionmodels.Severalpackagesprovidewaveletanal-ysisfunctionality:rwt,wavelets,waveslim,wavethresh.SomemethodsfromchaostheoryareprovidedbythepackagetseriesChaos.3.Finance:TheRmetricsbundlecomprisedofthefBasics,fCalendar,fSeries,fMultivar,fPortfolio,fOptionsandfExtremespackagescontainsaverylargenumberofrelevantfunctionsfordifferentaspectofempiricalandcomputationalfinance.TheRQuantLibpackageprovidesseveraloption-pricingfunctionsaswellassomefixed-incomefunctionalityfromtheQuantLibprojecttoR.Theportfoliopackagecontainsclassesforequityportfoliomanagement.4.RiskManagement:TheVaRpackageestimatesValue-at-Risk,andseveralpackagesprovidefunctionalityforExtremeValueTheorymodels:evd,evdbayes,evir,extRremes,ismec,POT.Themvtnormpackageprovidescodeformul- CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS6tivariateNormalandt-distributions.TheRmetricspackagesfPortfolioandfExtremesalsocontainanumberofrelevantfunctions.Thecopulaandfgacpackagescovermultivariatedependencystructuresusingcopulamethods.5.DataandDateManagement:Theits,zooandfCalendar(partofRmetrics)packagesprovidesupportforirregularly-spacedtimeseries.fCalendaralsoaddressescalendarissuessuchasrecurringholidaysforalargenumberoffinancialcen-ters,andprovidescodeforhigh-frequencydatasets.CRANpackages:*ArDec*car*copula*dse*dyn*dynlm*evd*evdbayes*evir*extRemes*fBasics(core)*fCalendar(core)*fExtremes(core)*fgac CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS7*fMultivar(core)*fOptions(core)*fPortfolio(core)*fracdiff*fSeries(core)*ismev*its(core)*lmtest*longmemo*mvtnorm*portfolio*POT*Rcmdr*RQuantLib(core)*rwt*sandwich*strucchange*tseries(core)*tseriesChaos*urca(core)*uroot*VaR*wavelets*waveslim*wavethresh CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS8*Zelig*zoo(core)Relatedlinks:*CRANTaskView:Econometrics.Thewebsiteishttp://cran.cnr.berkeley.edu/src/contrib/Views/Econometrics.htmlorseethenextsection.*RmetricsbyDiethelmWuertzcontainsawealthofRcodeforFinance.Thewebsiteishttp://www.itp.phys.ethz.ch/econophysics/R/*QuantlibisaC++libraryforquantitativefinance.Thewebsiteishttp://quantlib.org/*Mailinglist:RSpecialInterestGroupFinance1.5CRANTaskView:ComputationalEconometricsBaseRshipswithalotoffunctionalityusefulforcomputationaleconometrics,inparticularinthestatspackage.ThisfunctionalityiscomplementedbymanypackagesonCRAN,abriefoverviewisgivenbelow.ThereisalsoaconsiderableoverlapbetweenthetoolsforeconometricsinthisviewandfinanceintheFinanceview.Fur-thermore,thefinanceSIGisasuitablemailinglistforobtaininghelpanddiscussingquestionsaboutbothcomputationalfinanceandeconometrics.Thepackagesinthisviewcanberoughlystructuredintothefollowingtopics.Thewebsiteis CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS9http://cran.r-project.org/src/contrib/Views/Econometrics.html1.Linearregressionmodels:Linearmodelscanbefitted(viaOLS)withlm()(fromstats)andstandardtestsformodelcom-parisonsareavailableinvariousmethodssuchassummary()andanova().Analogousfunctionsthatalsosupportasymp-totictests(zinsteadofttests,andChi-squaredinsteadofFtests)andplug-inofothercovariancematricesarecoeftest()andwaldtest()inlmtest.Testsofmoregenerallinearhy-pothesesareimplementedinlinear.hypothesis()incar.HCandHACcovariancematricesthatcanbepluggedintothesefunctionsareavailableinsandwich.Thepackagescarandlmtestalsoprovidealargecollectionoffurthermethodsfordiagnosticcheckinginlinearregressionmodels.2.Microeconometrics:Manystandardmicro-econometricmod-elsbelongtothefamilyofgeneralizedlinearmodels(GLM)andcanbefittedbyglm()frompackagestats.Thisincludesinparticularlogitandprobitmodelsformodellingchoicedataandpoissonmodelsforcountdata.NegativebinomialGLMsareavailableviaglm.nb()inpackageMASSfromtheVRbundle.Zero-inflatedcountmodelsareprovidedinzicounts.Furtherover-dispersedandinflatedmodels,includinghurdlemodels,areavailableinpackagepscl.Bivariatepoissonregressionmodelsareimplementedinbivpois.Basiccensoredregressionmodels(e.g.,tobitmodels)canbefittedbysurvreg()insurvival.FurthermorerefinedtoolsformicroecnometricsareprovidedinmicEcon.ThepackagebayesmimplementsaBayesianap-proachtomicroeconometricsandmarketing.Inferenceforrela-tivedistributionsiscontainedinpackagereldist. CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS103.Furtherregressionmodels:VariousextensionsofthelinearregressionmodelandothermodelfittingtechniquesareavailableinbaseRandseveralCRANpackages.Nonlinearleastsquaresmodellingisavailableinnls()inpackagestats.Relevantpack-agesincludequantreg(quantileregression),sem(linearstruc-turalequationmodels,includingtwo-stageleastsquares),systemfit(simultaneousequationestimation),betareg(betaregression),nlme(nonlinearmixed-effectmodels),VR(multinomiallogitmodelsinpackagennet)andMNP(Bayesianmultinomialpro-bitmodels).ThepackagesDesignandHmiscprovidesev-eraltoolsforextendedhandlingof(generalized)linearregressionmodels.4.Basictimeseriesinfrastructure:Theclasstsinpack-agestatsisR’sstandardclassforregularlyspacedtimeserieswhichcanbecoercedbackandforthwithoutlossofinforma-tiontozooregfrompackagezoo.zooprovidesinfrastructureforbothregularlyandirregularlyspacedtimeseries(thelatterviatheclass“zoo”)wherethetimeinformationcanbeofar-bitraryclass.Severalotherimplementationsofirregulartimeseriesbuildingonthe“POSIXt”time-dateclassesareavailableinits,tseriesandfCalendarwhichareallaimedparticularlyatfinanceapplications(seetheFinanceview).5.Timeseriesmodelling:Classicaltimeseriesmodellingtoolsarecontainedinthestatspackageandincludearima()forARIMAmodellingandBox-Jenkins-typeanalysis.FurthermorestatsprovidesStructTS()forfittingstructuraltimeseriesanddecompose()andHoltWinters()fortimeseriesfilteringanddecomposition.ForestimatingVARmodels,severalmethodsareavailable:simplemodelscanbefittedbyar()instats,more CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS11elaboratemodelsareprovidedbyestVARXls()indseandaBayesianapproachisavailableinMSBVAR.Aconvenientinter-faceforfittingdynamicregressionmodelsviaOLSisavailableindynlm;adifferentapproachthatalsoworkswithotherregres-sionfunctionsisimplementedindyn.Moreadvanceddynamicsystemequationscanbefittedusingdse.Unitrootandcoin-tegrationtechniquesareavailableinurca,urootandtseries.Timeseriesfactoranalysisisavailableintsfa.6.Matrixmanipulations:Asavector-andmatrix-basedlan-guage,baseRshipswithmanypowerfultoolsfordoingma-trixmanipulations,whicharecomplementedbythepackagesMatrixandSparseM.7.Inequality:Formeasuringinequality,concentrationandpovertythepackageineqprovidessomebasictoolssuchasLorenzcurves,Pen’sparade,theGinicoefficientandmanymore.8.Structuralchange:Risparticularlystrongwhendealingwithstructuralchangesandchangepointsinparametricmod-els,seestrucchangeandsegmented.9.Datasets:Manyofthepackagesinthisviewcontaincollec-tionsofdatasetsfromtheeconometricliteratureandthepackageEcdatcontainsacompletecollectionofdatasetsfromvariousstandardeconometrictextbooks.micEcdatprovidesseveraldatasetsfromtheJournalofAppliedEconometricsandtheJournalofBusiness&EconomicStatisticsdataarchives.Pack-ageCDNmoneyprovidesCanadianmonetaryaggregatesandpwtprovidesthePennworldtable.CRANpackages: CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS12*bayesm*betareg*bivpois*car(core)*CDNmoney*Design*dse*dyn*dynlm*Ecdat*fCalendar*Hmisc*ineq*its*lmtest(core)*Matrix*micEcdat*micEcon*MNP*MSBVAR*nlme*pscl*pwt*quantreg CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS13*reldist*sandwich(core)*segmented*sem*SparseM*strucchange*systemfit*tseries(core)*tsfa*urca(core)*uroot*VR*zicounts*zoo(core)Relatedlinks:*CRANTaskView:Finance.Thewebsiteishttp://cran.cnr.berkeley.edu/src/contrib/Views/Finance.htmlorseetheabovesection.*Mailinglist:RSpecialInterestGroupFinance*ABriefGuidetoRforBeginnersinEconometrics.Thewebsiteishttp://people.su.se/˜ma/R−intro/. CHAPTER1.PACKAGERANDSIMPLEAPPLICATIONS14*RforEconomists.Thewebsiteishttp://www.mayin.org/ajayshah/KB/R/R−for−economists.html. Chapter2CharacteristicsofTimeSeries2.1IntroductionTheverynatureofdatacollectedindifferentfieldsasasdiverseaseconomics,finance,biology,medicine,andengineeringleadsonenat-urallytoaconsiderationoftimeseriesmodels.Samplestakenfromallofthesedisciplinesaretypicallyobservedoverasequenceoftimeperiods.Often,forexample,oneobserveshourlyordailyormonthlyoryearlydata,eventick-by-ticktradedata,anditisclearfromexam-iningthehistoriesofsuchseriesoveranumberoftimeperiodsthattheadjacentobservationsarebynomeansindependent.Hence,theusualtechniquesfromclassicalstatistics,developedprimarilyforin-dependentidenticallydistributed(iid)observations,arenotapplicable.Clearly,wecannothopetogiveacompleteaccountingofthethe-oryandapplicationsoftimeseriesinthelimitedtimetobedevotedtothiscourse.Therefore,whatwewilltrytoaccomplish,inthispresentationisaconsiderablymoremodestsetofobjectives,withmoredetailedreferencesquotedfordiscussionsindepth.First,wewillattempttoillustratethekindsoftimeseriesanalysesthatcanariseinscientificcontexts,particularly,ineconomicsandfinance,andgiveexamplesofapplicationsusingrealdata.Thisnecessarily15 CHAPTER2.CHARACTERISTICSOFTIMESERIES16willincludeexploratorydataanalysisusinggraphicaldisplaysandnumericalsummariessuchastheautocorrelationandcrosscorrela-tionfunctions.Theuseofscatterdiagramsandvariouslinearandnonlineartransformationsalsowillbeillustrated.Wewilldefineclassicaltimeseriesstatisticsformeasuringthepatternsdescribedbytimeseriesdata.Forexample,thecharacterizationofconsistenttrendprofilesbydynamiclinearorquadraticregressionmodelsaswellastherepresentationofperiodicpatternsusingspectralanalysiswillbeillustrated.Wewillshowhowonemightgoaboutexaminingplausiblepatternsofcauseandeffect,bothwithinandamongtimeseries.Finally,sometimeseriesmodelsthatareparticularlyusefulsuchasregressionwithcorrelatederrorsaswellasmultivariateau-toregressiveandstate-spacemodelswillbedeveloped,togetherwithunitroot,co-integration,andnonlineartimeseriesmodels,andsomeothermodels.Formsofthesemodelsthatappeartoofferhopeforapplicationswillbeemphasized.Itisrecognizedthatadiscussionofthemodelsandtechniquesinvolvedisnotenoughifonedoesnothaveavailabletherequisiteresourcesforcarryingouttimeseriescompu-tations;thesecanbeformidable.Hence,weincludeacomputingpackage,calledR.Inthischapter,wewilltrytominimizetheuseofmathematicalnotationthroughoutthediscussionsandwillnotspendtimedevel-opingthetheoreticalpropertiesofanyofthemodelsorprocedures.Whatisimportantforthispresentationisthatyou,thereader,cangainamodestunderstandingaswellashavingaccesstosomeoftheprincipaltechniquesoftimeseriesanalysis.Ofcourse,wewillrefertoHamilton(1994)foradditionalreferencesormorecompletedis-cussionsrelatingtoanapplicationorprincipleandwilldiscussthemindetail. CHAPTER2.CHARACTERISTICSOFTIMESERIES172.2StationaryTimeSeriesWebeginbyintroducingseveralenvironmentalandeconomicaswellasfinancialtimeseriestoserveasillustrativedatafortimeseriesmethodology.Figure2.1showsmonthlyvaluesofanenvironmentalseriescalledtheSouthernOscillationIndex(SOI)andassociatedrecruitment(numberofnewfish)computedfromamodelbyPierreKleiber,SouthwestFisheriesCenter,LaJolla,California.Bothseriesareforaperiodof453monthsrangingovertheyears1950−1987.TheSOImeasureschangesinairpressurethatarerelatedtoseasurfacetemperaturesinthecentralPacific.ThecentralPacificOceanwarmsupeverythreetosevenyearsduetotheElNi˜noeffectwhichhasbeenblamed,inparticular,forfoodsinthemidwesternportionsoftheU.S.BothseriesinFigure2.1tendtoexhibitrepetitivebehavior,withregularlyrepeating(stochastic)cyclesthatareeasilyvisible.Thisperiodicbehaviorisofinterestbecauseunderlyingprocessesofinterestmayberegularandtherateorfrequencyofoscillationchar-acterizingthebehavioroftheunderlyingserieswouldhelptoidentifythem.OnecanalsoremarkthatthecyclesoftheSOIarerepeatingatafasterratethanthoseoftherecruitmentseries.Therecruitseriesalsoshowsseveralkindsofoscillations,afasterfrequencythatseemstorepeataboutevery12monthsandaslowerfrequencythatseemstorepeataboutevery50months.Thestudyofthekindsofcyclesandtheirstrengthswillbediscussedlater.Thetwoseriesalsotendtobesomewhatrelated;itiseasytoimaginethatsomehowthefishpopulationisdependentontheSOI.Perhapsthereisevenalaggedrelation,withtheSOIsignallingchangesinthefishpopulation.Thestudyofthevariationinthedifferentkindsofcyclicalbehav- CHAPTER2.CHARACTERISTICSOFTIMESERIES18iorinatimeseriescanbeaidedbycomputingthepowerspectrumwhichshowsthevarianceasafunctionofthefrequencyofoscilla-tion.Comparingthepowerspectraofthetwoserieswouldthengivevaluableinformationrelatingtotherelativecyclesdrivingeachone.Onemightalsowanttoknowwhetherornotthecyclicalvariationsofaparticularfrequencyinoneoftheseries,saytheSOI,areasso-ciatedwiththefrequenciesintherecruitmentseries.Thiswouldbemeasuredbycomputingthecorrelationasafunctionoffrequency,calledthecoherence.ThestudyofsystematicperiodicvariationsSouthernOscillationIndexRecruit−1.0−0.50.00.51.002040608010001002003004000100200300400Figure2.1:MonthlySOI(left)andsimulatedrecruitment(right)fromamodel(n=453months,1950-1987).intimeseriesiscalledspectralanalysis.SeeShumway(1988)andShumwayandStoffer(2001)fordetails.Wewillneedacharacterizationforthekindofstabilitythatisexhibitedbytheenvironmentalandfishseries.Onecannotethatthetwoseriesseemtooscillatefairlyregularlyaroundcentralvalues(0forSOIand64forrecruitment).Also,thelengthsofthecyclesandtheirorientationsrelativetoeachotherdonotseemtobechangingdrasticallyoverthetimehistories.Inordertodescribethisinasimplemathematicalway,itiscon- CHAPTER2.CHARACTERISTICSOFTIMESERIES19venienttointroducetheconceptofastationarytimeseries.Supposethatweletthevalueofthetimeseriesatsometimepointtbedenotedby{xt}.Then,theobservedvaluescanberepresentedasx1,theinitialtimepoint,x2,thesecondtimepointandsoforthouttoxn,thelastobservedpoint.Astationarytimeseriesisoneforwhichthestatisticalbehaviorofxt1,xt2,...,xtkisidenticaltothatoftheshiftedsetxt1+h,xt2+h,...,xtk+hforanycollectionoftimepointst1,t2,...,tkandforanyshifth.Thismeansthatallofthemultivariateprobabilitydensityfunctionsforsubsetsofvariablesmustagreewiththeircounterpartsintheshiftedsetforallvaluesoftheshiftparameterh.Thisiscalledstrictlystrongstation-ary,whichcanberegardedasamathematicalassumption.Theaboveversionofstationarityistoostrongformostapplicationsandisdifficultorimpossibletobeverifiedstatisticallyinapplica-tions.Therefore,torelaxthismathematicalassumption,wewilluseaweakerversion,calledweakstationarityorcovariancesta-tionarity,whichrequiresonlythatfirstandsecondmomentssatisfytheconstraints.ThisimpliesthatE(xt)=µandE[(xt+h−µ)(xt−µ)]=γx(h),(2.1)whereEdenotesexpectationoraveragingoverthepopulationdensi-tiesandhistheshiftorlag.Thisimplies,first,thatthemeanvaluefunctiondoesnotchangeovertimeandthatγx(h),thepopulationcovariancefunction,isthesameaslongasthepointsareseparatedbyaconstantshifth.Estimatorsforthepopulationcovarianceareimportantdiagnostictoolsfortimecorrelationasweshallseelater.Whenweusethetermstationarytimeseriesinthesequel,wemeanweaklystationaryasdefinedby(2.1).Theautocorrelationfunction(ACF)isdefinedasascaledversionof(2.1)andiswrittenasρx(h)=γx(h)/γx(0),(2.2) CHAPTER2.CHARACTERISTICSOFTIMESERIES20whichisalwaysbetween−1and1.Thedenominatorof(2.2)isthemeansquareerrororvarianceoftheseriessinceγ(0)=E[(x−µ)2].xtExercise:Forgiventimeseries{x}n,howdoyoucheckwhethertt=1thetimeseries{xt}isweaklyorstrongstationary?Thankaboutthisproblem.Example1.1:Weintroduceinthisexampleasimpleexampleofatimedomainmodeltobeconsideredindetaillater.Asimplemovingaveragemodelassumesthattheseriesxtisgeneratedfromlinearcombinationsofindependentoruncorrelated“shocks”wt,sometimescalledwhitenoise1(WN),tothesystem.Forexample,thesimplefirstordermovingaverageseriesxt=wt−0.9wt−1isstationarywhentheinputs{wt}areassumedindependentwithE(w)=0andE(w2)=1.ItcanbeeasilyverifiedthatE(x)=0tttandγ(h)=1+0.92ifh=0,−0.9ifh=±1,0ifh>1(pleasexverifythis).Wecanseewhatsuchaseriesmightlooklikebydrawingrandomnumberswtfromastandardnormaldistributionandthencomputingthevaluesofxt.OnesuchsimulatedseriesisshowninFigure2.2forn=200values;theseriesresemblesvaguelytherealdatainthebottompanelofFigure2.1.Manyofourtechniquesarebasedontheideathatasuitablymodifiedtimeseriescanberegardedasstationary(weakly).Thisrequiresfirstthatthemeanvaluefunctionbeconstantasin(2.1).Severalsimplecommonlyoccurringnonstationarytimeseriescanbeillustratedbylettingthisassumptionbeviolated.Forexample,the1whitenoiseisdefinedasasequenceofuncorrelatedransomvariableswithmeanzeroandsamevariance. CHAPTER2.CHARACTERISTICSOFTIMESERIES21SimulatedMA(1)−3−2−10123050100150200Figure2.2:SimulatedMA(1)withθ1=0.9.seriesyt=t+xt,wherextisthemovingaverageseriesofExample1.1,willbenonstationarybecauseE(xt)=tandtheconstantmeanassumptionof(2.1)isclearlyviolated.Fourtechniquesformodifyingthegivenseriestoimprovetheap-proximationtostationarityaredetrending,differencing,trans-formations,andlinearfilteringasdiscussedbelow.Asimpleexampleofanonstationaryseriesisalsogivenlater.2.2.1DetrendingOneofthedominantfeaturesonmanyeconomicandbusinesstimeseriesisthetrend.Suchatrendcanbeupwardordownward,itcanbesteepornot,anditcanbeexponentialorapproximatelylinear.Sinceatrendshouldbedefinitelysomehowbeincorporatedinatimeseriesmodel,simplybecauseitcanbeexploitedforout-of-sampleforecasting,ananalysisoftrendbehaviortypicallyrequiresquitesomeresearchinput.Thediscussionlaterwillshowthatthetypeoftrendhasanimportantimpactonforecasting.Thegeneralversionofthenonstationarytimeseriesgivenaboveis CHAPTER2.CHARACTERISTICSOFTIMESERIES22toassumeageneraltrendoftheformyt=Tt+xt,particularly,thelineartrendTt=β1+β2t.Ifonelooksforamethodofmodifyingtheaboveseriestoachievestationarity,itisnaturaltoconsidertheresidualcccxbt=yt−Tt=yt−β1−β2tccasaplausiblestationaryserieswhereβ1andβ2aretheestimatedinterceptandslopeoftheleastsquareslineforytasafunctionoft.Theuseoftheresidualordetrendedseriesiscommonandtheprocessofconstructingresidualisknownasdetrending.Example1.2:Toillustratethepresenceoftrendsineconomicdata,considerthefivegraphsinFigure2.3,whicharetheannualindicesofagriculturecommerceconsumptionindustrytransport56781955196019651970197519801985Figure2.3:LogofannualindicesofrealnationaloutputinChina,1952-1988.realnationaloutput(inlogs)inChinainfivedifferentsectorsforthesampleperiod1952−1988.Thesesectorsareagriculture,industry,construction,transportation,andcommerce.Fromthisfigure,itcanbeobservedthatthefivesectorshavegrownovertheyearatdifferentrates,andalsothatthefivesectorsseemtohavebeenaffectedbythe,likely,exogenousshockstotheChineseeconomyaround1958and1968.Theseshocksroughlycorrespond CHAPTER2.CHARACTERISTICSOFTIMESERIES23tothetwomajorpoliticalmovementsinChina:theGreat-Leap-Forwardaround1958until1962andtheCulturalRevolutionfrom1966to1976.Italsoappearsfromthegraphsthatthesepoliticalmovementsmaynothaveaffectedeachofthefivesectorsinasimilarfashion.Forexample,thedeclineoftheoutputintheconstructionsectorin1961seemsmuchlargerthanthatintheindustrysectorinthesameyear.ItalsoseemsthattheGreat-Leap-Forwardshockalreadyhadanimpactontheoutputintheagriculturesectorasearlyas1959.ToquantifythetrendsinthefiveChineseoutputseries,onemightconsiderasimpleregressionmodelwithalineartrendasmentionedearlierorsomemorecomplexmodels.Example1.3:Asanothermoreinterestingexample,considertheglobaltemperatureseriesgiveninFigure2.4.ThereappearstobeanincreasingtrendinglobaltemperaturewhichmaysignalglobalOriginalDatawithLinearandNonlinearTrendooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo−1.0−0.50.00.5o190019502000Figure2.4:Monthlyaveragetemperatureindegreescentigrade,January,1856-February2005,n=1790months.Thestraightline(wideandgreen)isthelineartrendy=−9.037+0.0046tandthecurve(wideandred)isthenonparametricestimatedtrend.warmingoritmaybejustanormalfluctuation.FittingastraightlinerelatingtimettotemperatureindegreesCentigradebysimpleleastccsquaresleadstoβ1=−9.037,β2=0.0046andadetrendedseriesshownintheleftpanelofFigure2.5.Notethatthedetrendedseries CHAPTER2.CHARACTERISTICSOFTIMESERIES24Detrended:LinearDetrended:Nonlinear−0.50.00.5−0.6−0.4−0.20.00.20.40.6190019502000190019502000Figure2.5:Detrendedmonthlyglobaltemperatures:leftpanel(linear)andrightpanel(nonlinear).stillcontainsatrendlikebulgethatishighestataboutt=60years.Inthiscasetheslopeofthelineisoftenusedtoarguethatthereisaglobalwarmingtrendandthattheaverageincreaseisapproximately0.83degreesFper100years.ItisclearthattheresidualsinFigure2.5stillcontainsubstantialcorrelationandtheordinaryleastsquaresmodelmaynotbeappropriate.Theremayalsobeotherfunctionalformsthatdoabetterjobofdetrending;forexample,quadraticorlogarithmicrepresentationsarecommonornonparametricapproachcanbeused(Wewilldiscussthisapproachindetaillater);seethedetrendedseriesshownintherightpanelofFigure2.5.Detrendingisparticularlyessentialwhenoneisestimatingthecovariancefunctionandpowerspectrum.2.2.2DifferencingAcommonmethodforachievingstationarityinnonstationarycasesiswiththefirstdifference∆yt=yt−yt−1, CHAPTER2.CHARACTERISTICSOFTIMESERIES25where∆iscalledthedifferencingoperator.Theuseofdifferenc-ingasamethodfortransformingtostationarityiscommonalsoforserieswithtrend.Forexample,inthetrendinExample1.3,thedifferencedserieswouldbe∆yt=b+xt−xt−1,whichisstationarybecausethedifferencext−xt−1canbeshowntobestationary.Example1.4:Thefirstdifferenceoftheglobaltemperaturese-riesisshowninFigure2.6andweseethattheupwardlineartrendhasdisappearedashasthetrendlikebulgethatremainedintheDifferencedTimeSeries−0.50.00.51.0190019502000Figure2.6:Differencedmonthlyglobaltemperatures.detrendedseries.Higherorderdifferencesaredefinedassuccessiveapplicationsoftheoperator∆.Forexample,theseconddifferenceis∆2y=∆∆ysothat∆2y=y−2y+y.Ifthemodelalsottttt−1t−2containsaquadratictrendtermct2,itiseasytoshowthattakingtheseconddifferencereducesthemodeltoastationaryform.ThetrendsinFigures2.3and2.4areallofthefamiliartype,thatis,manyeconomictimeseriesdisplayanupwardmovingtrend.Itishowevernotnecessaryforatrendtomoveupwardstobecalledatrend.Itisalsothatatrendislesssmoothandmaydisplayslowlychangingtendencieswhichonceinawhilechangedirections. CHAPTER2.CHARACTERISTICSOFTIMESERIES26Example1.5:AnexampleofsuchatrendingpatternisgiveninthetopleftpanelofFigure2.7andthefirstdifferenceinthetoprightDataFirstdifference02040601001502002505060708090010203040Secondorderdifference−200−10010102030203040Figure2.7:AnnualstockofmotorcyclesintheNetherlands,1946-1993.panelofFigure2.7andthesecondorderdifferenceinthebottomleftpanelofFigure2.7,wheretheannualstockofmotorcyclesinTheNetherlandisdisplayed,for1946−1993,withthefirstorderandsecondorderdifferencingtimeseries.Fromthefiguresintherighttopandbottompanels,wecanseethatthedifferencingmightnotworkwellforthisexample.Onewaytodescribethischangingtrendsistoallowtheparameterstochangeovertime,drivenbysomeexogenousshocks(macroeconomicvariables)2,forexample,oilshockin1974.2.2.3TransformationsAtransformationthatcutsdownthevaluesoflargerpeaksofatimeseriesandemphasizesthelowervaluesmaybeeffectiveinreducing2seethepaperbyCai(2006) CHAPTER2.CHARACTERISTICSOFTIMESERIES27nonstationarybehaviorduetochangingvariance.Anexampleisthelogarithmictransformationyt=log(xt),wherelogdenotestheexponential-baselogarithm.Example1.6:Forexample,thedatashowninFigure2.8representquarterlyearningspersharefortheAmericanCompanyJohnson&J&JEarningstransformedlog(earnings)012051015020406080020406080Figure2.8:QuarterlyearningsforJohnson&Johnson(4thquarter,1970to1stquarter,1980,leftpanel)withlogtransformedearnings(rightpanel).Johnsonfromthefromthefourthquarterof1970tothefirstquar-terof1980.Itiseasytonotesomeverynonstationarybehaviorinthisseriesthatcannotbeeliminatedcompletelybydifferencingordetrendingbecauseofthelargerfluctuationsthatoccurneartheendoftherecordwhentheearningsarehigher.TherightpanelofFig-ure2.8showsthelog-transformedseriesandwenotethatthelatterpeakshavebeenattenuatedsothatthevarianceofthetransformedseriesseemsmorestable.Onewouldhavetoeliminatethetrendstillremainingintheaboveseriestoobtainstationarity.Formoredetailsonthecurrentanalysesofthisseries,seethelateranalysesandthepapersbyBurmanandShumway(1998)andCaiandChen(2006).Ageneraltransformationisthewell-knownBox-Coxtransfor-mation;seeHamilton(1994,p.126),Shumway(1988),andShumway CHAPTER2.CHARACTERISTICSOFTIMESERIES28andStoffer(2000),definedintermsofarbitrarypowerxαforsometαinacertainrange,whichcanbechosenbasedonsomeoptimalcriterionsuchasthesmallestmeansquarederror.2.2.4LinearFiltersThefirstdifferenceisalinearcombinationofthevaluesoftheseriesattwolags,say0and1andhastheeffectofretainingthefasteroscillationsandattenuatingorreducingthesloweroscillations.Wemaydefinemoregenerallinearfilterstodootherkindsofsmoothingorrougheningofatimeseriestoenhancesignalsandattenuatenoise.ConsiderthegenerallinearcombinationofpastandfuturevaluesofatimeseriesgivenasX∞yt=ajxt−jj=−∞whereaj,j=0,±1,±2,...,defineasetoffixedfiltercoefficientstobeappliedtotheseriesofinterest.Anexampleisthefirstdifferencewherea0=1,a1=−1,aj=0otherwise.Notethattheabove{yt}isalsocalledalinearprocessinprobabilityliterature.Example1.7:Togiveasimpleillustration,considerthetwelvemonthmovingaverageaj=1/12,j=0,±1,±2,±3,±4,±5,±6andzerootherwise.TheresultofapplyingthisfiltertotheSOIindexisshowninFigure2.9.Itisclearthatthisfilterremovessomehigheroscillationsandproducesasmootherseries.Infact,theyearlyoscillationshavebeenfilteredout(seethebottompanelinFigure2.9)andalowerfrequencyoscillationappearswithacyclingrateofabout42months.Thisistheso-calledElNi˜noeffectthataccountsforallkindsofphenomena.Thisfilteringeffectwillbeexaminedfurtherlateronspectralanalysissinceitisextremelyimportanttoknowexactlyhowoneisinfluencingtheperiodicoscillationsbyfiltering. CHAPTER2.CHARACTERISTICSOFTIMESERIES29−0.50.00.5−1.0−0.50.00.51.001002003004000100200300400Figure2.9:TheSOIseries(blacksolidline)comparedwitha12pointmovingaverage(redthickersolidline).Thetoppanel:originaldataandthebottompanel:filteredseries.Tosummarize,thegraphicalexaminationoftimehistoriescanpointthewaytofurtheranalysesbynotingperiodicitiesandtrendsthatmaybepresent.Furthermore,lookingattimehis-toriesoftransformedorfilteredseriesoftengivesanintuitiveideaastowhetheroneseriescouldalsobeassociatedwithanother.Figure2.1indicatesthattheSOIseriestendstoprecedeorleadtherecruitseries.Naturally,onecanaskforamoredetailedspecificationoftheleadinglaggingrelation.Inthefollowingsections,wewilltrytoshowhowclassicaltimeseriesmethodscanbeusedtoprovidepar-tialanswerstothesekindsofquestions.Beforedoingso,wespendsomespacetointroducesomeotherfeatures,suchasseasonality,outliers,nonlinearity,andconditionalheteroscedasticitycommonseenintheeconomicandfinancialaswellasenvironmentaldata. CHAPTER2.CHARACTERISTICSOFTIMESERIES302.3OtherKeyFeaturesofTimeSeries2.3.1SeasonalityWhentimeseries(particularly,economicandfinancialtimeseries)areobservedeachdayormonthorquarter,itisoftenthecasethatsuchasaseriesdisplaysaseasonalpattern(deterministiccyclicalbe-havior).Similartothefeatureoftrend,thereisnoprecisedefinitionofseasonality.Usuallywerefertoseasonalitywhenobservationsincertainseasonsdisplaystrikinglydifferentfeaturestoothersea-sons.Forexample,whentheretailsalesarealwayslargeinthefourthquarter(becauseoftheChristmasspending)andsmallinthefirstquarterascanbeobservedfromFigure2.10.Itmayalsobepossiblethatseasonalityisreflectedinthevarianceofatimeseries.Forex-ample,fordailyobservedstockmarketreturnsthevolatilityseemsoftenhighestonMondays,basicallybecauseinvestorshavetodigestthreedaysofnewsinsteadofonlyday.Formodedetails,seethebookbyTaylor(2005,§4.5)andTsay(2005).Example1.8:InthisexampleweconsiderthemonthlyUSretailsalesseries(notseasonallyadjusted)fromJanuaryof1967toDecem-berof2000(inbillionsofUSdollars).Thedatacanbedownloadedfromthewebsiteathttp://marketvector.com.TheU.S.retailsalesindexisoneofthemostimportantindicatorsoftheUSeconomy.Therearevaststudiesoftheseasonalseries(likethisseries)intheliterature;see,e.g.,Franses(1996,1998)andGhyselsandOsborn(2001)andCaiandChen(2006).FromFigure2.10,wecanobservethatthepeaksoccurinDecemberandwecansaythatretailsalesdis-playseasonality.Also,itcanbeobservedthatthetrendisbasicallyincreasingbutnonlinearly.ThesamephenomenoncanbeobservedfromFigure2.8forthequarterlyearningsforJohnson&Johnson. CHAPTER2.CHARACTERISTICSOFTIMESERIES31500001000002000003000000100200300400Figure2.10:USRetailSalesDatafrom1967-2000.Ifsimplegraphsarenotinformativeenoughtohighlightpossibleseasonalvariation,aformalregressionmodelcanbeused,forexam-ple,onemighttrytoconsiderthefollowingregressionmodelwithseasonaldummyvariablesXs∆yt=yt−yt−1=βjDj,t+εt,j=1whereDj,tisaseasonaldummyvariableandsisthenumberofsea-sons.Ofcourse,onecanuseaseasonalARIMAmodel,denotedbyARIMA(p,d,q)×(Q,D,Q)s,whichwillbediscussedlater.Example1.9:Inthisexample,weconsideratimeserieswithpronouncedseasonalitydisplayedinFigure2.11,wherelogsoffour-weeklyadvertisingexpendituresonratioandtelevisioninTheNether-landsfor1978.01−1994.13.Forthesetwomarketingtimeseriesonecanobserveclearlythatthetelevisionadvertisingdisplaysquitesomeseasonalfluctuationthroughouttheentiresampleandtheradioad-vertisinghasseasonalityonlyforthelastfiveyears.Also,thereseemstobeastructuralbreakintheradioseriesaroundobservation53.ThisbreakisrelatedtoanincreaseinradiobroadcastingminutesinJanuary1982.Furthermore,thereisavisualevidencethatthetrend CHAPTER2.CHARACTERISTICSOFTIMESERIES32television891011radio050100150200Figure2.11:Four-weeklyadvertisingexpendituresonradioandtelevisioninTheNether-lands,1978.01−1994.13.changesovertime.Generally,itappearsthatmanytimeseriesseasonallyobservedfrombusinessandeconomicsaswellasotherappliedfieldsdisplayseasonalityinthesensethattheobservationsincertainseasonshavepropertiesthatdifferfromthosedatapointsinotherseasons.Asecondfeatureofmanyseasonaltimeseriesisthattheseasonalitychangesovertime,likewhatstudiedbyCaiandChen(2006).Some-times,thesechangesappearabrupt,asisthecaseforadvertisingontheradioinFigure2.11,andsometimessuchchangesoccuronlyslowly.Tocapturethesephenomena,CaiandChen(2006)proposedamoregeneralflexibleseasonaleffectmodelhavingthefollowingform:yij=α(ti)+βj(ti)+eij,i=1,...,n,j=1,...,s,whereyij=y(i−1)s+j,ti=i/n,α(·)isa(smooth)commontrendfunctionin[0,1],{βj(·)}are(smooth)seasonaleffectfunctionsin[0,1],eitherfixedorrandom,subjecttoasetofconstraints,andtheerrortermeijisassumedtobestationary.Formoredetails,seeCaiandChen(2006). CHAPTER2.CHARACTERISTICSOFTIMESERIES332.3.2AberrantObservationsPossiblydistortingobservationsdonotnecessarilycomeinase-quenceasintheradioadvertisingexample(whichmighthaveso-calledregimeshifts).Itmayalsobethatonlyfewobservationshaveamajorimpactontimeseriesmodelingandforecasting.Suchdatapointsarecalledaberrantobservations(outliersinstatistics).Example1.10:Asanillustrationexample,weconsiderthediffer-encedyt,thatis∆yt=yt−yt−1,whereyt=log(wt),withwtthepricelevel,andtheinflationrateft=(wt−wt−1)/wt−1inArgentina,forthesample1970.1−1989.4inFigure2.12.Fromthefigure,itdifferenceinflation0123457075808590Figure2.12:Firstdifferenceinlogpricesversustheinflationrate:thecaseofArgentina,1970.1−1989.4.isobviousthatinthecasewherethequarterlyinflationrateishigh(asisthecasein1989.3whereitisabout500percent),∆ytseriesisnotagoodapproximationtotheinflationrate(sincethe1989.3observationwouldnotnowcorrespondtoabout200percent).Also,wecanobservethatthedatain1989seemtobequitedifferentfromthoseobservationstheyearbefore.Infact,ifthereisanycorrelationbetween∆ytand∆yt−1,suchacorrelationmaybeaffectedbythese CHAPTER2.CHARACTERISTICSOFTIMESERIES34observations.Inotherwords,ifasimpleregressionisusedtomodelthecorrelationbetween∆ytand∆yt−1,wewouldexpectthatanes-timateofthecoefficientρisinfluencedbythedatapointsinthelastyear.Theordinaryleastsquareestimatesofρareρb=0.561(0.094)fortheentiresampleandρb=0.704(0.082)forthesamplewithoutthedatapointsinthelastyear,wheretheestimatedstandarderrorisinparentheses.Itnowturnstothequestiononhowtohandlewiththeseaberrantobservations.SeeChapter6ofFranses(1998)fordetailsondiscus-sionofmethodstodeleteseveraltypesofaberrantobservations,andalsomethodstotakeaccountofsuchdataforforecasting.2.3.3ConditionalHeteroskedasticityAnimportantfeatureofeconomictimeseries,andinparticular,offinancialtimeseriesisthataberrantobservationstendtoemergeinclusters(persistence).Theintuitiveinterpretationisthatifacertaindaynewsarrivesonastockmarket,thereactiontothisnewsistobuyorsellmanystocks,whilethedayafterthenewshasdigestedandvaluedproperly,thestockmarketreturnsbacktothelevelofbeforethearrivalofthenews.Thispatternwouldbereflectedby(apossiblylarge)increasingordecreasing(usually,thenegativeimpactislargerthanthepositiveimpact,calledasymmetric)inthereturnsononedayfollowedbyanoppositechangeonthenextday.Asaresult,wecanregardthemasaberrantobservationsinarowandtwosuddenchangesinreturnsarecorrelatedsincethesecondsharpchangeiscausedbythefirst.Thisiscalledconditionalheteroskedasticity.Tocharacterizethisphenomenon,onemightusethesocalledtheautoregressiveconditionalheteroscedasticity(ARCH)modelofEngle(1982)andthegeneralizedautoregressive CHAPTER2.CHARACTERISTICSOFTIMESERIES35conditionalheteroscedasticity(GARCH)modelofBollerslev(1986)orotherGARCHtypemodels;seeTaylor(2005)andTsay(2005).Example1.11:ThisexampleconcernstheclosingbidpricesoftheJapaneseYen(JPY)intermsofU.S.dollar.Thereisavastamountofliteraturedevotedtothestudyoftheexchangeratetimeseries;seeSercuandUppal(2000)andthereferencesthereinfordetails.Hereweexplorethepossiblenonlinearityfeature(seethenextsection),heteroscedasticity,andpredictabilityoftheexchangerateseries(Wewilldiscussthislater).ThedataisaweeklyseriesfromJanuary1,1974toDecember31,2003.ThedailynoonbuyingratesinNewYorkCitycertifiedbytheFederalReserveBankofNewYorkforcustomsandcabletransferspurposeswereobtainedfromtheChicagoFederalReserveBoard(www.frbchi.org).TheweeklyseriesisgeneratedbyselectingtheWednesdaysseries(ifaWednesdayisaholidaythenthefollowingThursdayisused),whichhas1566observations.Theuseofweeklydataavoidstheso-calledweekendeffectaswellasotherbiasesassociatedwithnontrading,bid-askspread,asynchronousratesandsoon,whichareoftenpresentinhigherfrequencydata.Weconsiderthelogreturnseriesyt=100log(ξt/ξt−1),plottedinFigure2.13,whereξtisanexchangeratelevelonthet-thweek.Aroundthe44thweekof1998(theperiodoftheAsianfiancecrisis),thereturnsontheJapaneseandU.S.dollarexchangeratedecreasedby9.7%.Immediatelyafterthatobservation,wecanfindseveraldatapointsthatarelargeintheabsolutevalue.Additionally,inotherpartsofsample,wecanobserve“bubbles”,i.e.,clustersofobservationswithlargevariances.Thisphenomenoniscalledvolatilityclustering(persistence)or,conditionalheteroskedasticity.Inotherwords,thevariancechangesovertime.Toallowforthepossibilitythathighvolatilityisfollowedbyhigh CHAPTER2.CHARACTERISTICSOFTIMESERIES36−10−505050010001500Figure2.13:Japanese-U.S.dollarexchangeratereturnseries{yt},fromJanuary1,1974toDecember31,2003.volatility,andthatlowvolatilitywillbefollowedbylowvolatility,wherevolatilityisdefinedintermsofreturnsthemselves,onecanconsiderthepresenceofso-calledconditionalheteroskedasticityoritsvariantssuchasARCHorGARCHtypemodelsbyusing(∆y)2tasthevarianceofthereturns.Also,oncecanreplay(∆y)2by|∆y|,tttheabsolutevalueofreturn.Themainpurposeofexploitingvolatilityclusteringistoforecastfuturevolatility.Sincethisvariableisameasureofrisk,suchforecastscanbeusefultoevaluateinvestmentstrategiesorportfolioselectionorriskmanagement.Furthermore,itcanbeusefulfordecisionsonbuyingorseelingoptionsorderivatives.SeeHamilton(1994,Chapter21),Taylor(2005),andTsay(2005)fordetails.2.3.4NonlinearityThenonlinearfeatureoftimeseriescanbeseenoftenineconomicandfinancialdataaswellasotherappliedfields;seethepopularbooksbyTong(1990),GrangerandTer¨asvirta(1993),FransesandvanDijk(2000),andFanandYao(2003).Beyondlineardomain,thereare CHAPTER2.CHARACTERISTICSOFTIMESERIES37infinitemanynonlinearformstobeexplored.Earlydevelopmentofnonlineartimeseriesanalysisfocusedonvariousnonlinear(some-timesnon-Gaussian)parametricforms.Thesuccessfulexamplesin-clude,amongothers,theARCH-modelingoffluctuatingstructureforfinancialtimeseries,andthethresholdmodelingforbiologicalandeconomicdata,aswellasregimeswitchesorstructuralchangemodelingforeconomicandfinancialtimeseries.Example1.12:ConsidertheexampleoftheunemploymentrateinGermanyfor1962.1to1991.4inFigure2.14.Fromthegraphintheunadjustedseasonallyadjusted246802468106570758085902468Figure2.14:QuarterlyunemploymentrateinGermany,1962.1−1991.4(seasonallyadjustedandnotseasonallyadjusted)intheleftpanel.Thescatterplotofunemploymentrate(sea-sonallyadjusted)versusunemploymentrate(seasonallyadjusted)oneperiodlaggedintherightpanel.leftpanel,itisclearthatunemploymentratesometimesrisesquiterapidly,usuallyintherecessionyears1967,1974−1975,and1980−1982,whileitdecreasesveryslowly,usuallyintimesofexpansions.Thisasymmetrycanbeformalizedbyestimatingtheparametersinthefollowingsimpleregression∆yt=yt−yt−1=β1It(E)+β2It(R)+εt,whereIt(·)istheindicatorvariable,whichallowstheabsolutevalueoftherateofchangetovaryacrossthetwostate,say,“decreasing CHAPTER2.CHARACTERISTICSOFTIMESERIES38yt”and“increasingyt”fromβ1toβ2,whereβ1maybedifferentfrom−β2.FortheGermanseasonallyadjustedunemploymentrate,ccwefindthatβ1=−0.040andβ2=0.388,indicatingthatwhentheunemploymentrateincreases(inrecessions),itrisesfasterthanwhenitgoesdown(inexpansions).Furthermore,fromthegraphintherightpanel,thescatterplotofytversusyt−1,whereytistheseasonallyadjustedunemploymentrate,wecanobservethatthisseriesdisplayscyclicalbehavioraroundpointsthatshiftovertime.Whentheseshiftsareendogenous,i.e.,causedbypastobservationsonytthemselves,thiscanbeviewedasatypicalfeatureofnonlineartimeseries.Forthedetailedanalysisofthisdatasetusinganonlinearmethods,thereaderisreferredtothebookbyFranses(1998)fornonlinearparametricmodelandthepaperbyCai(2002)fornonparametricmodel.2.4TimeSeriesRelationshipsOnecanidentifytwobasickindsofassociationorcorrelationthatareimportantintimeseriesconsiderations.Thefirstistheno-tionofselfcorrelationorautocorrelationintroducedin(2.2).Thesecondisthatseriesaresomehowrelatedtoeachothersothatonemighthypothesizesomecausalrelationshipexistingbetweenthephe-nomenageneratingtheseries.Forexample,onemighthypothesizethatthesimultaneousoscillationsoftheSOIandrecruitmentseriessuggestthattheyarerelated.Weintroducebelowthreestatisticsforidentifyingthesourcesoftimecorrelation.Assumethatwehavetwoseriesxtandytthatareobservedoversomesetoftimepoints,sayt=1,...,n. CHAPTER2.CHARACTERISTICSOFTIMESERIES392.4.1AutocorrelationFunctionCorrelationatadjacentpointsofthesameseriesismeasuredbytheautocorrelationfunctiondefinedin(2.2).Forexample,theSOIseriesinFigure2.1containsregularfluctuationsatintervalsofap-proximately12months.Anindicationofpossiblelinearaswellasnonlinearrelationscanbeinferredbyexaminingthelaggedscatter-plots,definedasplotsthatputxtonthehorizontalFigure2.15.Mul-tiplelaggedscatterplotsshowingtherelationsbetweenSOIandthepresent(xt)andlaggedvalues(xt+h)ofSOIatlagsh=0,1,...,12axisandxt+hontheverticalaxisforvariousvaluesofthelagh=1,2,3,...,12.Example1.13:InFigure2.15,wehavemadealaggedscatterplotoftheSOIseriesattimet+hagainsttheSOIseriesattimetandobtainedahighcorrelation,0.412,betweentheseriesxt+12andtheoooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo1ooooooo2oooooo3ooooooo4oooooooooooo−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo5ooooo6oooooo7ooooooo8oooooooooooo−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo9ooooooo10ooooooo11ooooooo12oooooooooooo−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo13ooooo14ooooooo15ooooooo16oooooooooooo−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0−1.00.01.00.00.51.0Figure2.15:MultiplelaggedscatterplotsshowingtherelationshipbetweenSOIandthepresent(xt)versusthelaggedvalues(xt+h)atlags1≤h≤16.seriesxtshiftedby12years.Lowerorderlagsatt−1,t−2also CHAPTER2.CHARACTERISTICSOFTIMESERIES40showcorrelation.Thescatterplotshowsthedirectionoftherelationwhichtendstobepositiveforlags1,2,11,12,13,andtendstobenegativeforlags6,7,8.Thescatterplotcanalsoshownosignificantnonlinearitiestobepresent.Inordertodevelopameasureforthisselfcorrelationorautocorrelation,weutilizeasampleversionofthescaledautocovariancefunction(2.2),sayρbx(h)=γbx(h)/γbx(0),where1nX−hγbx(h)=(xt+h−x)(xt−x),nt=1Pnwhichisthesamplecounterpartof(2.2)withx=t=1xt/n.Undertheassumptionthattheunderlyingprocessxtiswhitenoise,theapproximatestandarderrorofthesampleACFis1σρ=√.(2.3)nThatis,ρbx(h)isapproximatelynormalwithmean0andvariance1/n.Example1.14:Asanillustration,considertheautocorrelationfunctionscomputedfortheenvironmentalandrecruitmentseriesshowninthetoptwopanelsofFigure2.16.Bothoftheautocor-relationfunctionsshowsomeevidenceofperiodicrepetition.TheACFofSOIseemstorepeatatperiodsof12whiletherecruitmenthasadominantperiodthatrepeatsatabout12to16timepoints.Again,themaximumvaluesarewellabovetwostandarderrorsshownasdottedlinesaboveandbelowthehorizontalaxis.2.4.2CrossCorrelationFunctionThefactthatcorrelationsmayoccuratsometimedelaywhentryingtorelatetwoseriestooneanotheratsomelaghforpurposesof CHAPTER2.CHARACTERISTICSOFTIMESERIES41ACFofSOIIndexACFofRecruits−0.50.00.51.0−0.50.00.51.00102030405001020304050CCFofSOIandRecruits−0.50.00.51.0−40−2002040Figure2.16:AutocorrelationfunctionsofSOIandrecruitmentandcrosscorrelationfunctionbetweenSOIandrecruitment.predictionsuggeststhatitwouldalsobeusefultoplotxt+hagainstyt.Example1.15:Inordertoexaminethispossibility,considerthelaggedscatterplotmatrixshowninFigures2.17and2.18.Figure2.17plotstheSOIattimet+h,xt+h,versustherecruitmentseriesytatlag0≤h≤15inFigure2.17.Therearenoparticularlystronglinearrelationsapparentinthisplots,i.e.futurevaluesofSOIarenotrelatedtocurrentrecruitment.Thismeansthatthetemperaturesarenotrespondingtopastrecruitment.InFigure2.18,thecurrentSOIvalues,xtareplottedagainstthefuturerecruitmentvalues,yt+hfor0≤h≤15.ItisclearfromFigure2.18thattheseriesarecorrelatednegativelyforlagsh=5,...,9.Thecorrelationatlag6,forexample,is−0.60implyingthatincreasesintheSOIleaddecreasesinnumberofrecruitsbyabout6months.Ontheotherhand,theseriesarehardlycorrelated(0.025)atallintheconventionalsense, CHAPTER2.CHARACTERISTICSOFTIMESERIES42ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020160o100ooooooooooooooooooooooo020260100ooooooooooooooooooooooooo020360100oooooooooooooooooooooooooo020460100ooooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020560100ooooooooooooooooooooooo020660o100ooooooooooooooooooooo020o760o100ooooooooooooooooooooooo020oo860ooo100oooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020oo960ooo100oooooooooooooooooooo020o10oo60oooo100ooooooooooooooooooo020o11oo60oooo100ooooooooooooooooooo020o12o60ooo100ooooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020o13o60ooo100ooooooooooooooooooooo020o1460o100ooooooooooooooooooo0201560o100ooooooooooooooooooooo0201660o100ooooooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0Figure2.17:MultiplelaggedscatterplotsshowingtherelationshipbetweentheSOIattimet+h,sayxt+h(x-axis)versusrecruitsattimet,sayyt(y-axis),0≤h≤15.oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020160o100ooooooooooooooooooooooo020260oo100ooooooooooooooooooooooooo0203o60o100oooooooooooooooooooooooo0204o60100oooooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020560100ooooooooooooooooooooooo020660100ooooooooooooooooooooo020760100oooooooooooooooooooo020860100ooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo020960100oooooooooooooooooooooooo0201060100ooooooooooooooooooooooooo0201160100oooooooooooooooooooooooo0201260100oooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo0201360100oooooooooooooooooooooo0201460100ooooooooooooooooooo0201560oo100oooooooooooooooooooooo0201660oo100ooooooooooooooooooooooooo−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0−1.00.00.51.0Figure2.18:MultiplelaggedscatterplotsshowingtherelationshipbetweentheSOIattimet,sayxt(x-axis)versusrecruitsattimet+h,sayyt+h(y-axis),0≤h≤15.measuredatlagh=0.ThegeneralpatternsuggeststhatpredictingrecruitsmightbepossibleusingtheElNi˜noatlagsof5,6,7,... CHAPTER2.CHARACTERISTICSOFTIMESERIES43months.Ameasureofthecorrelationbetweenseveralseries,xtandytisthecrosscovariancefunction(CCF),definedintermsofthecounterpartcovarianceto(2.2),E{(xt+h−µx)(yt−µy)}=γxy(h).Thecrosscorrelationisthescaled(toliebetween−1and1)versionofabove,sayrρxy(h)=γxy(h)/γx(0)γy(0).Theabovequantitiesareexpressedintermsofpopulationaveragesandmustgenerallybeestimatedfromsampledata.Theestimatedsamplecrosscorrelationfunctionscanbeusedtoinvestigatethepos-sibilityoftheseriesbeingrelatedatdifferentlags.Inordertoinves-tigatecrosscorrelationsbetweentwoseriesxt,yt,t=1,...,n,wenotethatareasonablesampleversionofρxy(h)mightbecomputedas1nX−hγbxy(h)=(xt+h−x)(yt−y),nt=1whereyisthesamplemeanof{y}n.Theestimatedsamplecrosstt=1rcorrelationfunctionthenbecomesρbxy(h)=γbxy(h)/γbx(0)γby(0),wherehdenotestheamountthatoneseriesislaggedrelativetotheother.Generally,onecomputesthefunctionρbxy(h)foranumberofpositiveandnegativevalues,h=0,±1,±2,uptoabout0.3n,sayanddisplaystheresultsasafunctionoflagh.Themaximumvalue,attainedforh=0,isρxy(0)=1andthefunctiontakesvaluesbetween−1and1.Thisfactmakesiteasytocomparevaluesofthecrosscorrelationwitheachother.Furthermore,underthehypothe-sisthatthereisnotrelationattimehandthatatleastoneofthetwoseriesareindependentandidenticallydistributed,thedistribu-tionofρbxy(h)isapproximatelynormalwithmean0andstandard CHAPTER2.CHARACTERISTICSOFTIMESERIES44deviationgivenagainby(2.3).Hence,onecancomparevaluesofthesamplecrosscorrelationwithsomeappropriatenumberofsam-plestandarddeviationsbasedonnormaltheory.Generally,valueswithin±1.96σρmightbereasonableifoneiswillingtolivewitheachtestatasigni.cancelevelof0.05.Otherwise,broaderlimitswouldbeappropriate.Ingeneral,ifmtestsaremade,eachatlevelatheoveralllevelofsignificanceisboundedbymα.Example1.16:Togiveanexample,considerthecrosscorrelationsbetweentheenvironmentalseriesandrecruitmentshowninthebot-tompanelofFigure2.16.ThecrosscorrelationbetweentheSOI,xt+handrecruitmentytshowspeaksath=−6,−7whichimpliesthatlaggedproductsinvolvingxt−6andytaswellasthoseinvolv-ingxt−7andytmatchupclosely.Thevalueshownonthegraphisρbxy(−6)=−0.6.Thismeans,inpracticethatthevaluesoftheSOIseriestendtoleadtherecruitmentseriesby6or7units.Also,onemaynotethat,sincethevalueisnegative,lowervaluesofSOIareassociatedwithhigherrecruitment.Thestandarderrorinthiscaseisapproximatelyσρ=0.047andthe−0.6easilyexceedstwostandarddeviations,shownaslinesaboveandbelowtheaxisinFigure2.16.Hence,wecanrejectthehypothesisthatthecorrelationiszeroatthatlag.Itisclearalsothattherearesomeperiodicfluctuationsapparentinthecrosscorrelations.Forexample,intheSOIRecruitmentexample,thereseemstobesystematicfluctuationwithaperiod(1fullcycle)ofabout12months.Thisproducesanumberofsecondarypeaksinthecrosscorrelationfunction.TheanalysisofthisperiodicbehaviorandtheaccountingforperiodicitiesintheseriesisconsideredinlatersectionsorthebooksbyFranses(1996)andGhyselsandOsborn(2001). CHAPTER2.CHARACTERISTICSOFTIMESERIES452.4.3PartialAutocorrelationFunctionAthirdkindoftimeseriesrelationshipexpresseswhatisessentiallytheselfpredictabilityofaseriesthroughthepartialautocorrelationfunction(PACF).Thereareseveralwaysofthinkingaboutthismea-sure.OnemayregardthePACFasthesimplecorrelationbetweentwopointsseparatedbyalagh,sayxtandxt−h,withtheeffectoftheinterveningpointsxt−1,xt−2,...,xt−h+1conditionedout,i.e.thepurecorrelationbetweenthetwopoints.Thisinterpretationisoftengiveninmorepracticalstatisticalsettings.Forexample,onemaygetsillycausalinferencesbyquotingcorrelationsbetweentwovariables,e.g.teachers’incomeandwineconsumption,thatmayoccursimplybecausebotharecorrelatedwithsomecommondrivingfactor,inthiscase,thegrossdomesticproduct,GDP,orsomeotherforceinfluencingdisposableincome.Intimeseriesanalysiswearereallymoreinterestedinthepre-dictionorforecastingproblem.Inthiscasewemightconsidertheproblemofpredictingxtbasedonobservationobservedhunitsbackinthepast,sayxt−1,xt−2,...,xt−h.Supposewewanttopredictxtfromxt−1,...,xt−husingsomelinearfunctionofthesepastvalues.Considerminimizingthemeansquarepredictionerror2MSE=E[(xt−xbt)]usingthepredictorxbt=a1xt−1+a2xt−2+···+ahxt−hoverthepossiblevaluesoftheweightingcoefficientsa1,...,ahwhereweassume,forconveniencethatxthasbeenadjustedtohavezeromean.Considertheresultofminimizingtheabovemeansquarepredic- CHAPTER2.CHARACTERISTICSOFTIMESERIES46tionerrorforaparticularlagh.Then,thepartialautocorrelationfunctionisdefinedasthevalueofthelastcoefficientabh,i.e.Φhh=ah.Asapracticalmatter,weminimizethesampleerrorsumofsquares2XnXhSSE=(xt−x)−ak(xt−k−x)t=h+1k=1cwiththeestimatedpartialcorrelationdefinedasΦhh=abh.Thecoefficients,asdefinedabove,arealsobetween−1and1andhavetheusualpropertiesofcorrelationcoefficients.Inparticular,itsstandarderrorunderthehypothesisofnopartialautocorrelationisstill(2.3).Theintuitionoftheaboveargumentisthatthelastcoefficientwillgetverysmalloncetheforecasthorizonorlaghislargeenoughtogivegoodprediction.Inparticular,theorderoftheautoregressivemodelinthenextchapterwillbeexactlythelaghbeyondwhichΦhh=0.Example1.17:Asanexample,weshowintherightpanelspanelofFigure2.19thepartialautocorrelationfunctionsoftheSOIseriesPACFofSOIPACFofRecruits−0.50.00.51.0−0.50.00.51.0051015202530051015202530Figure2.19:PartialautocorrelationfunctionsfortheSOI(leftpanel)andtherecruits(rightpanel)series.(leftpanel)andtherecruitsseries(rightpanel).NotethatthePACF CHAPTER2.CHARACTERISTICSOFTIMESERIES47oftheSOIhasasinglepeakatlagh=1andthenrelativelysmallvalues.Thismeans,ineffect,thatfairlygoodpredictioncanbeachievedbyusingtheimmediatelyprecedingpointandthataddingfurthervaluesdoesnotreallyimprovethesituation.Hencewemighttryanautoregressivemodelwithp=1.Therecruitsserieshastwopeaksandthensmallvalues,implyingthatthepurecorrelationbetweenpointsissummarizedbythefirsttwolags.AmajorapplicationofthePACFistodiagnosingtheappropriateorderofanautoregressivemodelfortheseriesunderconsideration.Autoregressivemodelswillbestudiedextensivelylaterbutwenoteherethattheyarelinearmodelsexpressingthepresentvalueofaseriesasalinearcombinationofanumberofpreviousvalues,withanadditiveerror.Hence,anautoregressivemodelusingtwopreviousvaluesforpredictionmightbewrittenintheformxt=φ1xt−1+φ2xt−2+wtwhereφ1andφ2arefixedunknowncoefficientsandwtarevaluesfromanindependentserieswithzeromeanandcommonvarianceσ2.Forexample,ifthePACFatlaghisroughlyzerowbeyondafixedvalue,sayh=2asobservedfortherecruitsseriesintheleftpanelofFigure2.19,thenonemightassumeamodeloftheformaboveforthatrecruitsseries.Tofinishtheintroductorydiscussion,wenotethattheextensionofwhathasbeensaidabovetomultivariatetimeseriesisfairlystraight-forwardifonerestrictsthediscussiontorelationsbetweentwoseriesatatime.Thetwoseriesxtandytcanbelaidoutinthevector(xt,yt)whichbecomesamultivariateseriesofdimensiontwo(bi-variate).Togeneralize,considerthepseries(xt1,xt2,...,xtp)=xtastherowvectordefiningthemultivariateseriesxt.Theautocovari-ancematrixofthevectorxtcanthenbedefinedasthep×pmatrix CHAPTER2.CHARACTERISTICSOFTIMESERIES48containingaselementsγij(h)=E{(xt+h,i−µi)(xtj−µj)}andtheanalysisofthecrosscovariancestructurecanproceedontwoelementsatatime.Thediscussionofpossiblemultiplerelationsisdeferreduntillater.Torecapitulate,theprimaryobjectiveofthischapterhasbeentodefineandillustratewithrealexamplesthreestatisticsofinterestindescribingrelationshipswithinandamongtimeseries.Theau-tocorrelationfunctionmeasuresthecorrelationovertimeinasingleseries;thiscorrelationcanbeexploitedforpredictionpurposesorforsuggestingperiodicities.Thecrosscorrelationfunctionmeasuresthecorrelationbetweenseriesovertime;itmaybe,forexample,thataseriesisbetterrelatedtothepastofanotherseriesthanitistoitsownpast.Thepartialautocorrelationfunctiongivesadirectmea-sureofthelaglengthnecessarytopredictaseriesfromitself,i.e.toforecastfuturevalues.Itisalsocriticalindeterminingtheorderofanautoregressivemodelsatisfiedbysomerealdataset.Itshouldbenotedthatallthreeofthemeasuresgiveninthissectioncanbedistortediftherearesignificanttrendsinthedata.Itisobviousthatlaggedproductsoftheform(xt+h−x)(xt−x)willbeartificiallylargeifthereisatrendpresent.Sincethecorrelationsofinterestareusuallyassociatedwiththestationarypartoftheseries,i.e.,thepartthatcanbethoughtofasbeingsuperimposedontrend,itisusualtoevaluatethecorrelationsofthedetrendedseries.Thisbmeans,ineffect,thatwereplacexintheequationsbyab+btifthetrendcanbeconsideredtobelinear.Ifthetrendisquadraticorlogarithmic,theappropriatealternatenonlinearpredictedvalueissubtractedbeforecomputingthelaggedproducts.Notealsothat CHAPTER2.CHARACTERISTICSOFTIMESERIES49differencingtheseries,asdiscussedinSection2.2.2,canaccomplishthesameresult.2.5Problems1.ConsiderageneralizationofthemodelgiveninExample1.1,namely,xt=µ+wt−θwt−1,where{wt}areindependentzero-meanrandomvariableswithvarianceσ2.ProvethatE(x)=µ,wtγ(0)=(1+θ2)σ2,γ(1)=−θσ2,γ(h)=0if|h|>1,andxwxwxfinallyshowthatxtisweaklystationary.2Considerthetimeseriesgeneratedbyx1=µ+w1andxt=µ+xt−1+wtfort≥2.Showthatxtisnotstationarynomatterµ=0ornotandfindγx(h).3.Supposethatxtisstationarywithmeanµxandcovariancefunc-tiongivenbyγx(h).Findthemeanandcovariancefunctionof(a)yt=a+bxt,whereaandbareconstants,(b)zt=xt−xt−1.4.ConsiderthelinearprocessX∞yt=ajwt−j,j=−∞wherewisawhitenoiseprocesswithvarianceσ2anda,j=0,twj±1,±2,ldots,areconstants.Theprocessytwillexist(asaPlimitinmeansquare)ifj|aj|<∞;youdonotneedtoprovethis.Showthattheseriesytisstationary,withautocovariancefunctionX∞2γy(h)=σwaj+haj.j=−∞ CHAPTER2.CHARACTERISTICSOFTIMESERIES50Applytheresulttocalculatingtheautocovariancefunctionofthe3-pointmovingaverage(xt−1+xt+xt+1)/3.Forthefollowingproblems,youneedtouseacomputerpackage.5.Meltingglaciersdeposityearlylayersofsandandsiltduringthespringmeltingseasonswhichcanbereconstructedyearlyoveraperiodrangingfromthetimede-glaciationbeganinNewEng-land(about12,600yearsago)tothetimeitended(about6,000yearsago).Suchsedimentarydeposits,calledvarves,canbeusedasaproxyforpaleoclimaticparameterssuchastempera-ture.Thefilemass2.datcontainsyearlyrecordsfor634yearsbeginning11,834yearsago,collectedfromonelocationinMas-VarvethicknessfromMassachusetts(n=634)varvethickness0501001500100200300400500600yearFigure2.20:VarvedataforProblem5.sachusetts.Forfurtherinformation,seeShumwayandVerosub(1992).(a)Plotthevarverecordsandexaminetheautocorrelationandpartialautocorrelationfunctionsforevidenceofnonstationarity.(b)Arguethatthetransformationyt=logxtmightbeusefulforstabilizingthevariance.Computeγx(0)andγy(0)overtwotime CHAPTER2.CHARACTERISTICSOFTIMESERIES51intervalsforeachseriestodeterminewhetherthisisreasonable.Plotthehistogramsoftherawandtransformedseries.(c)Plottheautocorrelationoftheseriesytandarguethatafirstdifferenceproducesareasonablystationaryseries.Canyouthinkofapracticalinterpretationforut=yt−yt−1=logxt−logxt−1?(d)ComputetheautocorrelationfunctionofthedifferencedtransformedseriesandarguethatageneralizationofthemodelgivenbyExample1.1mightbereasonable.Assumethatut=wt−θwt−1isstationarywhentheinputswtareassumedindependentwithE(w)=0and(w2)=σ2.UsingthesamplettwACFandtheprintedautocovarianceγbu(0),deriveestimatorsforθandσ2.w6.TwotimeseriesrepresentingaveragewholesaleU.S.gasolineandoilpricesover180months,beginninginJuly,1973andendinginDecember,1987aregiveninthefileoil-gas.dat.AnalyzethedatausingsomeofthetechniquesinthischapterwiththeideathatoneshouldbelookingathowchangesinoilpricesinfluenceGasandoilprices(n=180months)GASOILprice100200300400500600700050100150monthFigure2.21:GasandoilseriesforProblem6.changesingasprices.Forfurtherreading,seeLiu(1991).In CHAPTER2.CHARACTERISTICSOFTIMESERIES52particular,considerthefollowingoptions:(a)Plottherawdataandlookattheautocorrelationfunctionstoarguethattheuntransformeddataseriesarenonstationary.(b)Itisoftenarguedineconomicsthatpricechangesareim-portant,inparticular,thepercentagechangeinpricesfromonemonthtothenext.Onthisbasis,arguethatatransformationoftheformyt=logxt−logxt−1mightbeappliedtothedatawherextistheoilorgaspriceseries.(c)Uselaggedmultiplescatterplotsandtheautoandcrosscor-relationfunctionsofthetransformedoilandgaspriceseriestoinvestigatethepropertiesoftheseseries.Isitpossibletoguesswhethergaspricesareraisedmorequicklyinresponsetoin-creasingoilpricesthantheyaredecreasedwhenoilpricesaredecreased?Doyouthinkthatitmightbepossibletopredictlogpercentagechangesingaspricesfromlogpercentagechangesinoilprices?Plotthetwoseriesonthesamescale.7.MonthlyHandgunSalesandFirearmsRelatedDeathsinCalifor-niaLegalhandgunpurchaseinformationfor227monthsspan-ningthetimeperiodFebruary1,1980throughDecember31,1998wasobtainedfromtheDepartmentofJusticesautomatedFirearmssystemsdatabase.CaliforniaresidentfirearmsdeathdatawasobtainedformtheCaliforniaDepartmentofHealthServices.Thedataareplottedinthefigure,withbothratesgiveninnumbersper100,000residents.Supposethatthemainquestionofinterestforthisdatapertainstothepossiblerelationsbetweenhandgunsalesanddeathratesoverthistimeperiod.Includethepossibilityoflaggingrelationsinyouranalysis.Inparticular,answerthequestionsbelow: CHAPTER2.CHARACTERISTICSOFTIMESERIES53GunsalesandgundeathrateHangunsales/100per100,0001.01.52.0Gundeathrateper100,000050100150200monthsFigure2.22:Handgunsales(per10,000,000)inCaliforniaandmonthlygundeathrate(per100,00)inCalifornia(February2,1980-December31,1998.(a)Usescatterplotstoarguethatthereisapotentialnonlinearrelationbetweendeathratesandhandgunsalesandindicatewhetheryouthinkthattheremightbealag.(b)Bolsteryourargumentforalaggingrelationshipbyexamin-ingthecrosscorrelationfunction.Whatdotheautocorrelationfunctionsindicateforthisdata?(c)Examinethefirstdifferenceforthetwoprocessesandindi-catewhattheACF’sandCCF’sshowforthedifferenceddata.(d)Smooththetwoserieswitha12pointmovingaverageandplotthetwoseriesonthesamegraph.Subtractthemovingav-eragefromtheoriginalunsmoothedseries.WhatdotheresidualseriesshowintheACFandCCFforthiscase?2.6ComputerCodeThefollowingRcommandsareusedformakingthegraphsinthischapter. CHAPTER2.CHARACTERISTICSOFTIMESERIES54#3-28-2006graphics.off()#cleanthepreviousgraphonthescreen#################################################################ThisisSouthernOscillationIndexdataandRecruitsdata##############################################################y<-read.table("c:\teaching\timeseries\data\soi.dat",header=T)#readdatafilex<-read.table("c:\teaching\timeseries\data\recruit.dat",header=T)y=y[,1]x=x[,1]postscript(file="c:\teaching\timeseries\figs\fig-1.1.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)#savethegraphasapostscriptfilets.plot(y,type="l",lty=1,ylab="",xlab="")#makeatimeseriesplottitle(main="SouthernOscillationIndex",cex=0.5)#setupthetitleoftheplotabline(0,0)#makeastraightlinets.plot(x,type="l",lty=1,ylab="",xlab="")abline(mean(x),0)title(main="Recruit",cex=0.5)dev.off()z=arima.sim(n=200,list(ma=c(0.9)))#simulateaMA(1)mo CHAPTER2.CHARACTERISTICSOFTIMESERIES55postscript(file="c:\teaching\timeseries\figs\fig-1.2.eps",horizontal=F,width=6,height=6)ts.plot(z,type="l",lty=1,ylab="",xlab="")title(main="SimulatedMA(1)",cex=0.5)abline(0,0)dev.off()n=length(y)n2=n-12yma=rep(0,n2)for(iin1:n2){yma[i]=mean(y[i:(i+12)])}#computetheyy=y[7:(n2+6)]yy0=yy-ymapostscript(file="c:\teaching\timeseries\figs\fig-1.9.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)ts.plot(yy,type="l",lty=1,ylab="",xlab="")points(1:n2,yma,type="l",lty=1,lwd=3,col=2)ts.plot(yy0,type="l",lty=1,ylab="",xlab="")points(1:n2,yma,lty=1,lwd=3,col=2)#makeapointplotabline(0,0)dev.off()m=17n1=n-my.soi=rep(0,n1*m)dim(y.soi)=c(n1,m)y.rec=y.soifor(iin1:m){ CHAPTER2.CHARACTERISTICSOFTIMESERIES56y.soi[,i]=y[i:(n1+i-1)]y.rec[,i]=x[i:(n1+i-1)]}text_soi=c("1","2","3","4","5","6","7","8","9","10","11","12","13""14","15","16")postscript(file="c:\teaching\timeseries\figs\fig-1.15.eps",horizontal=F,width=6,height=6)par(mfrow=c(4,4),mex=0.4)for(iin2:17){plot(y.soi[,1],y.soi[,i],type="p",pch="o",ylab="",xlab="",ylim=c(-1,1),xlim=c(-1,1))text(0.8,-0.8,text_soi[i-1],cex=2)}dev.off()text1=c("ACFofSOIIndex")text2=c("ACFofRecruits")text3=c("CCFofSOIandRecruits")SOI=yRecruits=xpostscript(file="c:\teaching\timeseries\figs\fig-1.16.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(y,ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")#makeanACFplotlegend(10,0.8,text1)#setupthelegendacf(x,ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")legend(10,0.8,text2)ccf(y,x,ylab="",xlab="",ylim=c(-0.5,1),lag.max=50,main="")legend(-40,0.8,text3)dev.off() CHAPTER2.CHARACTERISTICSOFTIMESERIES57postscript(file="c:\teaching\timeseries\figs\fig-1.17.eps",horizontal=F,width=6,height=6)par(mfrow=c(4,4),mex=0.4)for(iin1:16){plot(y.soi[,i],y.rec[,1],type="p",pch="o",ylab="",xlab="",ylim=c(0,100),xlim=c(-1,1))text(-0.8,10,text_soi[i],cex=2)}dev.off()postscript(file="c:\teaching\timeseries\figs\fig-1.18.eps",horizontal=F,width=6,height=6)par(mfrow=c(4,4),mex=0.4)for(iin1:16){plot(y.soi[,1],y.rec[,i],type="p",pch="o",ylab="",xlab="",ylim=c(0,100),xlim=c(-1,1))text(-0.8,10,text_soi[i],cex=2)}dev.off()postscript(file="c:\teaching\timeseries\figs\fig-1.19.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)pacf(y,ylab="",xlab="",lag=30,ylim=c(-0.5,1),main="")text(10,0.9,"PACFofSOI")pacf(x,ylab="",xlab="",lag=30,ylim=c(-0.5,1),main="")text(10,0.9,"PACFofRecruits")dev.off()################################################################################################################################### CHAPTER2.CHARACTERISTICSOFTIMESERIES58#Thisisglobaltemperaturedata#################################y1<-matrix(scan("c:\teaching\timeseries\data gtemp.dat"),byrow=T,ncol=1)a<-1:12a=a/12y=y1[,1]n=length(y)x<-rep(0,n)for(iin1:149){x[((i-1)*12+1):(12*i)]<-1856+i-1+a}x[n-1]<-2005+1/12x[n]=2005+2/13##########################NonparametricFitting####################################################################################DefinetheEpanechnikovkernelfunctionlocalestimatorkernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}################################################################Definethefunctionforcomputingthelocallinearestimatlocal<-function(y,x,z,h){#parameters:y=response,x=designmatrix;h=bandwidth;z= CHAPTER2.CHARACTERISTICSOFTIMESERIES59nz<-length(z)ny<-length(y)beta<-rep(0,nz*2)dim(beta)<-c(nz,2)for(kin1:nz){x0=x-z[k]w0<-sqrt(kernel(x0/h))beta[k,]<-glm(y~x0,weight=w0)$coeff}return(beta)}###################################################################z=xh=12#takeabadnwidthfit=local(y,x,z,h)#fitmodely=m(x)+emhat=fit[,1]#obtainthenonparametricestimateresid1=y-(-9.037+0.0046*x)resid2=y-mhatpostscript(file="c:\teaching\timeseries\figs\fig-1.4.eps",horizontal=F,width=6,height=6)matplot(x,y,type="p",pch="o",ylab="",xlab="",cex=0.5)#makemultipleplotspoints(z,mhat,type="l",lty=1,lwd=3,col=2)abline(-9.037,0.0046,lty=1,lwd=5,col=3)#makeastrightlinewithaninterceptandslopetitle(main="OriginalDatawithLinearandNonlinearTrend",cex=0.5)dev.off() CHAPTER2.CHARACTERISTICSOFTIMESERIES60postscript(file="c:\teaching\timeseries\figs\fig-1.5.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)matplot(x,resid1,type="l",lty=1,ylab="",xlab="",cex=0.5)abline(0,0)title(main="Detrended:Linear",cex=0.5)matplot(x,resid2,type="l",lty=1,ylab="",xlab="",cex=0.5)abline(0,0)title(main="Detrended:Nonlinear",cex=0.5)dev.off()y_diff=diff(y)postscript(file="c:\teaching\timeseries\figs\fig-1.6.eps",horizontal=F,width=6,height=6)plot(x[-1],y_diff,type="l",lty=1,ylab="",xlab="",cex=0.5)abline(0,0)title(main="DifferencedTimeSeries",cex=0.5)dev.off()####################################################################ThisisChinadata###################################data<-read.table("c:/teaching/stat3150/data/data1.txt",header=T)#readdatafromafilecontaining6columnsofdatay<-data[,1:5]#putthefirst5columnsofdataintoyx<-data[,6]text1<-c("agriculture","commerce","consumption","industry","transp#setthetextforlegendinagraph CHAPTER2.CHARACTERISTICSOFTIMESERIES61postscript(file="c:\teaching\timeseries\figs\fig-1.3.eps",horizontal=F,width=6,height=6)matplot(x,log(y),type="l",lty=1:5,ylab="",xlab="")legend(1960,8,text1,lty=1:5,col=1:5)dev.off()####################################################################Thisismotorcyclesdata###################################data<-read.table("c:/teaching/stat3150/data/data7.txt",header=T)#readdatafromafilecontaining6columnsofdatay<-data[,1]x<-data[,2]-1900y_diff1=diff(y)y_diff2=diff(y_diff1)postscript(file="c:\teaching\timeseries\figs\fig-1.7.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)matplot(x,y,type="l",lty=1,ylab="",xlab="")text(60,250,"Data")ts.plot(y_diff1,type="l",lty=1,ylab="",xlab="")text(20,40,"Firstdifference")abline(0,0)ts.plot(y_diff2,type="l",lty=1,ylab="",xlab="")text(20,25,"Secondorderdifference")abline(0,0)dev.off() CHAPTER2.CHARACTERISTICSOFTIMESERIES62####################################################################ThisisJohnsonandJohnsondata###################################y<-matrix(scan("c:\teaching\timeseries\data\jj.dat"),byrow=T,ncol=1)n=length(y)y_log=log(y)#logofdatapostscript(file="c:\teaching\timeseries\figs\fig-1.8.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2))ts.plot(y,type="l",lty=1,ylab="",xlab="")title(main="J&JEarnings",cex=0.5)ts.plot(y_log,type="l",lty=1,ylab="",xlab="")title(main="transformedlog(earnings)",cex=0.5)dev.off()####################################################################Thisisretailsalesdata###################################y=matrix(scan("c:\res\0published\cai-chen\retail\retail-sales.dat"),byrow=T,ncol=1)postscript(file="c:\teaching\timeseries\figs\fig-1.10.eps",horizontal=F,width=6,height=6)ts.plot(y,type="l",lty=1,ylab="",xlab="")dev.off()####################################################################Thisismarketingdata###################################text_tv=c("television") CHAPTER2.CHARACTERISTICSOFTIMESERIES63text_radio=c("radio")data<-read.table("c:/teaching/stat3150/data/data4.txt",header=T)TV=log(data[,1])RADIO=log(data[,2])postscript(file="c:\teaching\timeseries\figs\fig-1.11.eps",horizontal=F,width=6,height=6)ts.plot(cbind(TV,RADIO),type="l",lty=c(1,2),col=c(1,2),ylab=text(20,10.5,text_tv)text(165,8,text_radio)dev.off()####################################################################ThisisArgentinadata###################################text_ar=c("difference","inflation")y<-read.table("c:/teaching/stat3150/data/data8.txt",header=T)y=y[,1]n=length(y)y_t=diff(log(y))f_t=diff(y)/y[1:(n-1)]x=seq(70.25,by=0.25,89.75)postscript(file="c:\teaching\timeseries\figs\fig-1.12.eps",horizontal=F,width=6,height=6)matplot(x,cbind(y_t,f_t),type="l",lty=c(1,2),col=c(1,2),ylablegend(72,5,text_ar,lty=c(1,2),col=c(1,2))dev.off()####################################################################Thisisexchangeratedata################################### CHAPTER2.CHARACTERISTICSOFTIMESERIES64x<-matrix(scan(file="c:\res\cai-xu\jpy\jpy.dat"),byrow=T,ncol=1)n<-length(x)nweek<-(n-7)/5week1<-rep(0,n)#Datesforweekweek1[1:4]<-2:5for(jin1:nweek){i1<-4+(j-1)*5+1i2<-4+j*5week1[i1:i2]<-c(1,2,3,4,5)}i2<-(nweek+1)*5week1[i2:n]<-1:3y<-x[week1==3]#Wednsdayx1<-x[week1==4]#Thursdayx1<-append(x1,0)x1<-(1-(y>0))*x1#TakevaluefromThursdayifNDonWednsdax1<-y+x1#Wednsday+Thursdayn<-length(x1)x<-100*(log(x1[2:n])-log(x1[1:(n-1)]))#logreturnpostscript(file="c:\teaching\timeseries\figs\fig-1.13.eps",horizontal=F,width=6,height=6)ts.plot(x,type="l",ylab="",xlab="")abline(0,0)dev.off()####################################################################Thisisunemploymentdata###################################text_unemploy=c("unadjusted","seasonallyadjusted") CHAPTER2.CHARACTERISTICSOFTIMESERIES65data<-read.table("c:/teaching/stat3150/data/data10.txt",header=T)y1=data[,1]y2=data[,2]n=length(y1)x=seq(62.25,by=0.25,92)postscript(file="c:\teaching\timeseries\figs\fig-1.14.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)matplot(x,cbind(y1,y2),type="l",lty=c(1,2),col=c(1,2),ylab="legend(66,10,text_unemploy,lty=c(1,2),col=c(1,2))plot(y2[1:(n-1)],y2[2:n],type="l",lty=c(1),col=c(1),ylab="",dev.off()####################################################################Thisisvarvedata#####################x<-matrix(scan("c:\teaching\timeseries\data\mass2.dat"),byrow=T,ncol=1)postscript(file="c:\teaching\timeseries\figs\fig-1.20.eps",horizontal=F,width=6,height=6)ts.plot(x,type="l",lty=1,ylab="varvethickness",xlab="year")title(main="VarvethicknessfromMassachusetts(n=634)",cex=0.5)dev.off()####################################################################Thisisoil-gasdata#######################data<-matrix(scan("c:\teaching\timeseries\data\gas-oil.dat"),byrow=T,ncol=2)text4=c("GAS","OIL")postscript(file="c:\teaching\timeseries\figs\fig-1.21.eps", CHAPTER2.CHARACTERISTICSOFTIMESERIES66horizontal=F,width=7,height=7)ts.plot(data,type="l",lty=c(1,2),col=c(1,2),ylab="price",xlab=title(main="Gasandoilprices(n=180months)",cex=0.5)legend(20,700,text4,lty=c(1,2),col=c(1,2))dev.off()####################################################################Thisishandgundata#####################y<-matrix(scan("c:\teaching\timeseries\data\guns.dat"),byrow=T,ncol=2)sales=y[,1]y=cbind(y[,1]/100,y[,2])text5=c("Hangunsales/100per100,000")text6=c("Gundeathrateper100,000")postscript(file="c:\teaching\timeseries\figs\fig-1.22.eps",horizontal=F,width=7,height=7)par(mex=0.4)ts.plot(y,type="l",lty=c(1,2),col=c(1,2),ylab="",xlab="months"title(main="Gunsalesandgundeathrate",cex=0.5)legend(20,2,lty=1,col=1,text5)legend(20,0.8,lty=2,col=2,text6)dev.off()###################################################################2.7ReferencesBollerslev,T.(1986).Generalizedautoregressiveconditionalheteroskedasticity.JournalofEconometrics,31,307-327.Burman,P.andR.H.Shumway(1998).Semiparametricmodelingofseasonaltimeseries.JournalofTimeSeriesAnalysis,19,127-145. CHAPTER2.CHARACTERISTICSOFTIMESERIES67Cai,Z.(2002).Atwo-stageapproachtoadditivetimeseriesmodels.StatisticaNeerlandica,56,415-433.Cai,Z.(2006).Trendingtimevaryingcoefficienttimeseriesmodelswithseriallycorrelatederrors.ForthcominginJournalofEconometrics.Cai,Z.andR.Chen(2006).Flexibleseasonaltimeseriesmodels.AdvancesinEconomet-rics,20B,63-87.Engle,R.F.(1982).AutoregressiveconditionalheteroscedasticitywithestimatesofthevarianceofUnitedKingdominflations.Econometrica,50,987-1007.Fan,J.andQ.Yao(2003).NonlinearTimeSeries:NonparametricandParametricMeth-ods.Springer-Verlag,NewYork.Franses,P.H.(1996).PeriodicityandStochasticTrendsinEconomicTimeSeries.NewYork:CambridgeUniversityPress.Franses,P.H.(1998).TimeSeriesModelsforBusinessandEconomicForecasting.NewYork:CambridgeUniversityPress.Franses,P.H.andD.vanDijk(2000).NonlinearTimeSeriesModelsforEmpiricalFinance.NewYork:CambridgeUniversityPress.Ghysels,E.andD.R.Osborn(2001).TheEconometricAnalysisofSeasonalTimeSeries.NewYork:CambridgeUniversityPress.Granger,C.W.J.andT.Ter¨asvirta(1993).ModelingNonlinearEconomicRelationships.Oxford,U.K.:OxfordUniversityPress.Hamilton,J.D.(1994).TimeSeriesAnalysis.PrincetonUniversityPress,NJ.Liu,L.M.(1991).DynamicrelationshipanalysisofU.S.gasolineandcrudeoilprices.JournalofForecasting,10,521-547.Sercu,P.andR.Uppal(2000).ExchangeRateVolatility,Trade,andCapitalFlowsunderAlternativeRateRegimes.Cambridge:CambridgeUniversityPress.Shumway,R.H.(1988).AppliedStatisticalTimeSeriesAnalysis.EnglewoodCliffs,NJ:Prentice-Hall.Shumway,R.H.andD.S.Stoffer(2000).TimeSeriesAnalysis&ItsApplications.NewYork:Springer-Verlag.Shumway,R.H.andK.L.Verosub(1992).StatespacemodelingofPaleoclimatictimese-ries.Proceedingof5thInternationalMeetingonStatisticalClimatology,Toronto,22-26June,1992.Taylor,S.(2005).AssetPriceDynamics,Volatility,andPrediction.PrincetonUniversityPress,Princeton,NJ. CHAPTER2.CHARACTERISTICSOFTIMESERIES68Tong,H.(1990).NonlinearTimeSeries:ADynamicalSystemApproach.OxfordUniver-sityPress,Oxford.Tsay,R.S.(2005).AnalysisofFinancialTimeSeries,2thEdition.JohnWiley&Sons,NewYork. Chapter3UnivariateTimeSeriesModels3.1IntroductionTheorganizationofthischapterispatternedafterthelandmarkap-proachtodevelopingmodelsfortimeseriesdatapioneeredbyBoxandJenkins(seeBox,etal,1994).Thisassumesthattherewillbearepresentationoftimeseriesdataintermsofadifferenceequa-tionthatrelatesthecurrentvaluetoitspast.Suchmodelsshouldbeflexibleenoughtoincludenon-stationaryrealizationsliketherandomwalkgivenaboveandseasonalbehavior,wherethecurrentvalueisrelatedtopastvaluesatmultiplesofanunderlyingseason;acom-mononemightbemultiplesof12months(1year)formonthlydata.Themodelsareconstructedfromdifferenceequationsdrivenbyran-dominputshocksandarelabelledinthemostgeneralformulationasARMA(AutoregressiveMovingAverage)modeloramoregen-eralmodelARIMA,i.e.,AutoregressiveIntegratedMovingAverageprocesses.Theanalogieswithdifferentialequations,whichmodelmanyphysicalprocesses,areobvious.Forclarity,wedeveloptheseparatecomponentsofthemodelse-quentially,consideringtheintegrated,autoregressive,andmovingaverageinorder,followedbytheseasonalmodification.TheBox-Jenkinsapproachsuggeststhreestepsinaprocedurethattheysum-69 CHAPTER3.UNIVARIATETIMESERIESMODELS70marizeasidentification,estimation,andforecasting.Iden-tificationusesmodelselectiontechniques,combiningtheACFandPACFasdiagnosticswiththeversionsoftheAkaikeInformationcriterion(AIC)typemodelselectioncriteriagivenbelowtofindaparsimonious(simple)modelforthedata.Estimationofparametersinthemodelwillbethenextstep.Statisticaltechniquesbasedonmaximumlikelihoodandleastsquaresareparamountforthisstageandwillonlybesketchedinthiscourse.Hopefully,wecandiscusstheminagreatlengthiftimepermits.Finally,forecastingoftimeseriesbasedontheestimatedparameters,withsensibleestimatesofuncertainty,isthebottomline,foranyassumedmodel.CorrelationandAutocorrelationThecorrelationcoefficientbetweentworandomvariablesxtandytisdefinedasρxy(0),whichisthespecialcaseofthecrosscorrelationcoefficientρxy(h)definedinChapter2.Thecorrelationcoefficientbetweenxtandxt+hiscalledthelaghautocorrelationofxtandiscommonlydenotedbyρx(h),whichisundertheweakstationarityassumption.Thedefinitionofρx(h)isgiveninChapter2.Thesampleversionofρx(h)isgivenbyρbx(h)=γbx(h)/γbx(0),where,forgivendata{x}n,tt=11nX−h1Xnγbx(h)=(xt+h−x)(xt−x)withx=xt.n−ht=1nt=1Undersomegeneralconditions,ρbx(1)isaconsistentestimateofρx(1).Forexample,if{xt}isanindependentandidenticallydistributed(iid)sequenceandE(x2)<∞,thenρb(1)isasymptoticallynormaltxwithmeanzeroandvariance1/n;seeBrockwellandDavis(1991,Theorem7.2.2).ThisresultcanbeusedinpracticetotestthenullhypothesisH0:ρx(1)=0versusthealternativehypothesis CHAPTER3.UNIVARIATETIMESERIESMODELS71Ha:ρx(1)6=0.Thetestvstatisticistheusualt-ratio,whichis√nρbx(1)andfollowsasymptoticallythestandardnormaldistribu-tion.Ingeneral,forthelaghsampleautocorrelationofxt,if{xt}isaniidsequencesatisfyingE(x2)<∞,then,ρb(h)isasymptoticallytxnormalwithmeanzeroandvariance1/nforanyfixedpositivein-tegerh.Formoreinformationabouttheasymptoticdistributionofsampleautocorrelations,seeBrockwellandDavis(1991,Chapter7).Infinitesamples,ρbx(h)isabiasedestimatorofρx(h).Thebiasisintheorderof1/n,whichcanbesubstantialwhenthesamplesizenissmall.Inmosteconomicandfinancialapplications,nisrelativelylargesothatthebiasisnotserious.PortmanteauTestEconomicandfinancialapplicationsoftenrequiretotestjointlythatseveralautocorrelationsofxtarezero.BoxandPierce(1970)pro-posedthePortmanteaustatisticXm∗2Q(m)=nρbx(h)h=1asateststatisticforthenullhypothesisH0:ρx(1)=···=ρx(m)=0againstthealternativehypothesisHa:ρx(i)6=0forsomei∈[1,...,m].Undertheassumptionthat{xt}isaniidsequencewithcertainmomentconditions,Q∗(m)isasymptoticallyachi-squaredrandomvariablewithmdegreesoffreedom.LjungandBox(1978)modifiedtheQ∗(m)statisticasbelowtoincreasethepowerofthetestinfinitesamples,Xm2Q(m)=n(n+2)ρbx(h)/(n−h).h=1Inpractice,theselectionofmmayaffecttheperformanceoftheQ(m)statistic.Severalvaluesofmareoftenused.Simulationstud- CHAPTER3.UNIVARIATETIMESERIESMODELS72iessuggestthatthechoiceofm≈log(n)providesbetterpowerperformance.Thefunctionρbx(h)iscalledthesampleautocorrelationfunction(ACF)ofxt.Itplaysanimportantroleinlineartimeseriesanalysis.Asamatteroffact,alineartimeseriesmodelcanbecharacterizedSimpleReturnsLogReturnsIBMIBM−0.2−0.10.00.10.2−0.2−0.10.00.10.2020406080100020406080100SimpleReturnsLogReturnsvalue−weightedindexvalue−weightedindex−0.2−0.10.00.10.2−0.2−0.10.00.10.2020406080100020406080100Figure3.1:Autocorrelationfunctions(ACF)forsimple(left)andlog(right)returnsforIBM(toppanels)andforthevalue-weightedindexofUSmarket(bottompanels),January1926toDecember1997.byitsACF,andlineartimeseriesmodelingmakesuseofthesampleACFtocapturethelineardynamicofthedata.ThetoppanelsofFigure3.1showthesampleautocorrelationfunctionsofmonthlysimple(lefttoppanel)andlog(righttoppanel)returnsofIBMstockfromJanuary1926toDecember1997.ThetwosampleACFsareveryclosetoeachother,andtheysuggestthattheserialcorrelationsofmonthlyIBMstockreturnsareverysmall,ifany.ThesampleACFsareallwithintheirtwostandard-errorlimits,indicatingthattheyarenotsignificantatthe5%level.Inaddition,forthesimplereturns, CHAPTER3.UNIVARIATETIMESERIESMODELS73theLjungBoxstatisticsgiveQ(5)=5.4andQ(10)=14.1,whichcorrespondtop-valueof0.37and0.17,respectively,basedonchi-squareddistributionswith5and10degreesoffreedom.Forthelogreturns,wehaveQ(5)=5.8andQ(10)=13.7withp-value0.33and0.19,respectively.ThejointtestsconfirmthatmonthlyIBMstockreturnshavenosignificantserialcorrelations.ThebottompanelsofFigure3.1showthesameforthemonthlyreturns(simpleintheleftpanelandlogintherightpanel)ofthevalue-weightedindexfromtheCenterforResearchinSecurityPrices(CRSP),UniversityofChicago.Therearesomesignificantserialcorrelationsatthe5%levelforbothreturnseries.TheLjungBoxstatisticsgiveQ(5)=27.8andQ(10)=36.0forthesimplereturnsandQ(5)=26.9andQ(10)=32.7forthelogreturns.Thep-valuesofthesefourteststatisticsarealllessthan0.0003,suggestingthatmonthlyreturnsofthevalue-weightedindexareseriallycorrelated.Thus,themonthlymarketindexreturnseemstohavestrongerserialdependencethanindividualstockreturns.Inthefinanceliterature,aversionoftheCapitalAssetPricingModel(CAPM)theoryisthatthereturn{xt}ofanassetisnotpredictableandshouldhavenoauto-correlations.Testingforzeroautocorrelationshasbeenusedasatooltochecktheefficientmarketassumption.However,thewaybywhichstockpricesaredeterminedandindexreturnsarecalculatedmightintroduceautocorrelationsintheobservedreturnseries.Thisisparticularlysoinanalysisofhigh-frequencyfinancialdata.Beforewediscussunivariateandmultivariatetimeseriesmeth-ods,wefirstreviewmultipleregressionmodelsandmodelselectionmethodsforbothiidandtimeseriesdata. CHAPTER3.UNIVARIATETIMESERIESMODELS743.2LeastSquaresRegressionWebeginourdiscussionofunivariateandmultivariatetimeseriesmethodsbyconsideringtheideaofasimpleregressionmodel,whichwehavemetbeforeinothercontextssuchasstatisticsoreconomet-ricscourse.Allofthemultivariatemethodsfollow,insomesense,fromtheideasinvolvedinsimpleunivariatelinearregression.Inthiscase,weassumethatthereissomecollectionoffixedknownfunc-tionsoftime,sayzt1,zt2,...,ztqthatareinfluencingouroutputytwhichweknowtoberandom.Weexpressthisrelationbetweentheinputsandoutputsasyt=β1zt1+β2zt2+···+βqztq+et(3.1)atthetimepointst=1,2,...,n,whereβ1,...,βqareunknownfixedregressioncoefficientsandetisarandomerrorornoise,assumedtobewhitenoise;thismeansthattheobservationshavezeromeans,equalvariancesσ2andareindependent.Wetraditionallyassumealsothatthewhitenoiseseries,et,isGaussianornormallydistributed.Example2.1:Wehaveassumedimplicitlythatthemodelyt=β1+β2t+etisreasonableinourdiscussionofdetrendinginExample1.2ofChapter2.Figure2.4showsthemonthlyaverageglobaltemperatureseriesanditisplausiblethatastraightlineisareasonablemodel.Thisisintheformoftheregressionmodel3.1whenonemakestheidentificationzt1=1andzt2=t.Theproblemindetrendingistoestimatethecoefficientsβ1andβ2intheaboveequationanddetrendbyconstructingtheestimatedresidualserieset,whichisshowninthetoppanelofFigure2.4.Asindicatedintheexample,estimatesforβ1ccandβ2canbetakenasβ1=−9.037andβ2=0.0046,respectively. CHAPTER3.UNIVARIATETIMESERIESMODELS75ThelinearregressionmodeldescribedbyEquation3.1canbecon-venientlywritteninslightlymoregeneralmatrixnotationbydefiningthecolumnvectorsz=(z,...,z)′andβ=(β,...,β)′sott1tq1qthatwewrite(2.1)inthealternateform′yt=βzt+et.(3.2)Tofindestimatorsforβandσ2,itisnaturaltodeterminetheco-P2efficientvectorβminimizingetwithrespecttoβ.Thisyieldscleastsquaresormaximumlikelihoodestimatorβandthemaximumlikelihoodestimatorforσc2whichisproportionaltotheunbiased1nX−1′22cσc=yt−βzt.(3.3)n−qt=0Analternatewayofwritingthemodel3.2isasy=Zβ+e,(3.4)′whereZ=(z1,z2,...,zn)isaq×nmatrixcomposedofthevaluesoftheinputvariablesattheobservedtimepointsandy=(y1,y2,...,yn)isthevectorofobservedoutputswiththeerrorsstackedinthevectore=(e,e,...,e)′.Theordinaryleast12ncsquaresestimatorsβarethesolutionstothenormalequations′ZZβ=Zy.Youneednotbeconcernedastohowtheaboveequationissolvedinpracticeasallcomputerpackageshaveefficientsoftwareforinverting′theq×qmatrixZZtoobtainc′−1β=(ZZ)Zy.(3.5)Animportantquantitythatallsoftwareproducesisameasureofuncertaintyfortheestimatedregressioncoefficients,sayc2′−122Covβ=σ(ZZ)≡σC≡σ(cij).(3.6) CHAPTER3.UNIVARIATETIMESERIESMODELS76Then,Cov(β,β)=σ2canda100(1−α)%confidenceintervalijijcforβiis√cβi±tn−q(α/2)σccii,(3.7)wheretdf(α/2)denotestheupper100(1−α)%pointonatdistri-butionwithdfdegreesoffreedom.Example2.1:ConsiderestimatingthepossibleglobalwarmingtrendalludedtoinSection2.2.1.Theglobaltemperatureseries,shownpreviouslyinFigure2.4suggeststhepossibilityofagraduallyincreasingaveragetemperatureoverthe149yearperiodcoveredbythelandbasedseries.IfwefitthemodelinExample2.1,replacingtbyt/100toconverttoa100yearbasesothattheincreasewillbeccindegreesper100years,weobtainβ1=−9.037andβ2=0.4607using(3.5).theerrorvariance,from(3.3),is0.0337,withq=2andn=1790.Then,(3.6)yieldscc0.0379−0.0020Cov(β1,β2)=−0.00200.0001√leadingtoanestimatedstandarderrorof0.001=0.01008.Thevalueoftwithn−q=1790−2=1788degreesoffreedomforα=0.025isabout1.96,leadingtoanarrowconfidenceintervalof0.4607±0.0198fortheslopeleadingtoaconfidenceintervalontheonehundredyearincreaseofabout0.4409to0.4805degrees.WewouldconcludefromthisanalysisthatthereisasubstantialincreaseinglobaltemperatureamountingtoanincreaseofroughlyonedegreeFper100years.ccIfthemodelisreasonable,theresidualset=yt−β1−β2tshouldbeessentiallyindependentandidenticallydistributedwithnocorre-lationevident.TheplotthatwehavemadeinFigure2.5(thetoppanel)ofthedetrendedglobaltemperatureseriesshowsthatthisis CHAPTER3.UNIVARIATETIMESERIESMODELS77DetrendedTemperatureACFPACF−0.50.00.51.0−0.50.00.51.0051015205101520LagLagDifferencedTemperatureACFPACF−0.50.00.51.0−0.50.00.51.0051015205101520LagLagFigure3.2:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthedetrended(toppanel)anddifferenced(bottompanel)globaltemperatureseries.probablynotthecasebecauseofthelonglowfrequencyintheob-servedresiduals.However,thedifferencedseries,alsoshowninFigure2.6,appearstobemoreindependentsuggestingthatperhapstheap-parentglobalwarmingismoreconsistentwithalongtermswinginanunderlyingrandomwalkthanitisofafixed100yeartrend.Ifwechecktheautocorrelationfunctionoftheregressionresiduals,shownhereinFigure3.2,itisclearthatthesignificantvaluesathigherlagsimplythatthereissignificantcorrelationintheresiduals.Suchcorrelationcanbeimportantsincetheestimatedstandarderrorsofthecoefficientsundertheassumptionthattheleastsquaresresidu-alsareuncorrelatedisoftentoosmall.Wecanpartiallyrepairthedamagecausedbythecorrelatedresidualsbylookingatamodelwithcorrelatederrors.Theprocedureandtechniquesfordealingwithcor-relatederrorsarebasedontheautoregressivemovingaveragemodelstobeconsideredinthenextsections.Anothermethodofreducingcorrelationistoapplyafirstdifference∆xt=xt−xt−1totheglobal CHAPTER3.UNIVARIATETIMESERIESMODELS78trenddata.TheACFofthedifferencedseries,alsoshowninFigure3.2,seemstohavelowercorrelationsatthehigherlags.Figure2.6showsqualitativelythatthistransformationalsoeliminatesthetrendintheoriginalseries.Sincewehaveagainmadesomeratherarbitrarylookingspec-ificationsfortheconfigurationofdependentvariablesintheaboveregressionexamples,thereadermaywonderhowtoselectamongvar-iousplausiblemodels.WementionthattwocriteriawhichrewardreducingthesquarederrorandpenalizeforadditionalparametersaretheAkaikeInformationCriterionandtheSchwarzInformationCriterion(SIC)(Schwarz,1978)withacommonform2log(σc)+C(K)/n,(3.8)whereKisthenumberofparametersfitted(exclusiveofvarianceparameters),σc2isthemaximumlikelihoodestimatorforthevariance,andC(K)=2KforAICandKlog(n)forSIC.SICissometimestermedtheBayesianInformationCriterion,BICandwilloftenyieldmodelswithfewerparametersthantheotherselectionmethods.AmodificationtoAICthatisparticularlywellsuitedforsmallsampleswassuggestedbyHurvichandTsai(1989).ThisisthecorrectedAIC,calledAICCgivenbylog(σc2)+(n+K)/(n−K−2).TheruleforallthreemeasuresaboveistochoosethevalueofKleadingtothesmallestvalueofAICorSICorAICC.Wewillgiveanexamplelatercomparingtheabovesimpleleastsquaresmodelwithamodelwheretheerrorshaveatimeseriescorrelationstructure.Asummaryofmodelselectionmethodsisgiveninthenextsection.Notethatallmethodsaregeneralpurposes. CHAPTER3.UNIVARIATETIMESERIESMODELS793.3ModelSelectionMethodsGivenapossiblylargesetofpotentialpredictors,whichonesdoweincludeinourmodel?Suppose[X1,X2,···]isapoolofpotentialpredictors.Themodelwithallpredictors,Y=β0+β1X1+β2X2+···+ε,isthemostgeneralmodel.Itholdsevenifsomeoftheindividualβj’sarezero.Butifsomeβj’szeroorclosetozero,itisbettertoomitthoseXj’sfromthemodel.Reasonswhyyoushouldomitvariableswhosecoefficientsareclosetozero:(a)Parsimonyprinciple:Giventwomodelsthatperformequallywellintermsofprediction,oneshouldchoosethemodelthatismoreparsimonious(simple).(b)Predictionprinciple:Themodelshouldgivepredictionsthatareasaccurateaspossible,notjustforcurrentobservation,butforfutureobservationsaswell.Includingunnecessarypre-dictorscanapparentlyimprovepredictionforthecurrentdata,butcanharmpredictionforfuturedata.Notethatthesumofsquarederrors(SSE)neverincreasesasweaddmorepredictors.Therefore,whenwebuildastatisticalmodel,weshouldfollowtheseprinciples.3.3.1SubsetApproachesTheall-possible-regressionsprocedurecallsforconsideringallpos-siblesubsetsofthepoolofpotentialpredictorsandidentifyingfordetailedexaminationafewgoodsub-setsaccordingtosomecrite-rion.Thepurposeofall-possible-regressionsapproachisidentifying CHAPTER3.UNIVARIATETIMESERIESMODELS80asmallgroupofregressionmodelsthataregoodaccordingtoaspec-ifiedcriterion(summarystatistic)sothatadetailedexaminationcanbemadeofthesemodelsleadingtotheselectionofthefinalregres-sionmodeltobeemployed.Themainproblemofthisapproachiscomputationallyexpensive.Forexample,withk=10predictors,weneedtoinvestigate210=1024potentialregressionmodels.Withtheaidofmoderncomputingpower,thiscomputationispossible.Butstillthenumberof1024possiblemodelstoexaminecarefullywouldbeanoverwhelmingtaskforadataanalyst.Differentcriteriaforcomparingtheregressionmodelsmaybeusedwiththeall-possible-regressionsselectionprocedure.Wediscusssev-eralsummarystatistics:(i)R2(orSSE),(ii)R2(orMSE),(iii)C,(iv)ppadj;pppPRESSp,(v)SequentialMethods,and(vi)AICtypecriteriaWeshalldenotethenumberofallpotentialpredictorsinthepoolbyP−1.Henceincludinganinterceptparameterβ0,wehavePpotentialparameters.Thenumberofpredictorsinasubsetwillbedenotedbyp−1,asalways,sothattherearepparametersintheregressionfunctionforthissubsetofpredictors.Thuswehave1≤p≤P:1.R2(orSSE):R2indicatesthattherearepparameters(or,p−1ppppredictors)intheregressionmodel.ThecoefficientofmultipledeterminationR2isdefinedasp2Rp=1−SSEp/SSTO,whereSSEpisthesumofsquarederrorsofthemodelincludingallp−1predictorsandSSTOisthesumofsquaredtotalvariations. CHAPTER3.UNIVARIATETIMESERIESMODELS81ItiswellknownthatR2measurestheproportionofvarianceofpYexplainedbyp−1predictors,italwaysgoesupasweaddapredictor,anditvariesinverselywithSSEpbecauseSSTOisconstantforallpossibleregressionmodels.Thatis,choosingthemodelwiththelargestR2isequivalenttochoosingthemodelpwithsmallestSSEp.2.R2(orMSE):OneoftenconsidersmodelswithalargeR2adj;pppvalue.However,R2alwaysincreaseswiththenumberofpredic-ptors.Henceitcannotbeusedtocomparemodelswithdifferentsizes.TheadjustedcoefficientofmultipledeterminationR2adj;phasbeensuggestedasanalternativecriterion:2SSEp/(n−p)n−1SSEpMSEpRadj;p=1−=1−=1−.SSTO/(n−1)n−pSSTOSSTO/(n−1)ItislikeR2butwithapenaltyforaddingunnecessaryvariables.pR2cangodownwhenauselesspredictorisaddedanditadj,pcanbeevennegative.R2variesinverselywithMSEbecauseadj;ppSSTO/(n−1)isconstantforallpossibleregressionmodels.Thatis,choosingthemodelwiththelargestR2isequivalentadj;ptochoosingthemodelwithsmallestMSE.NotethatR2isppusefulwhencomparingmodelsofthesamesize,whileR2(oradj;pCp)isusedtocomparemodelswithdifferentsizes.3.MallowsCp:TheMallowsCpisconcernedwiththetotalmeansquarederrorofthenfittedvaluesforeachsubsetregressionmodel.Themeansquarederrorconceptinvolvesthetotalerrorineachfittedvalue:ccccYi−µi=Yi−E(Yi)+E(Yi)−µi,|{z}|{z}randomerrorbias CHAPTER3.UNIVARIATETIMESERIESMODELS82whereµiisthetruemeanresponseatithobservation.ThecmeanssquarederrorforYiisdefinedastheexpectedvalueofthesquareofthetotalerrorintheabove.Itcanbeshownthat2cc2ccmse(Yi)=E(Yi−µi)=Var(Yi)+Bias(Yi),ccwhereBias(Yi)=E(Yi)−µi.ThetotalmeansquareerrorforcallnfittedvaluesYiisthesumovertheobservationi:XnXnXn2cccmse(Yi)=Var(Yi)+Bias(Yi).i=1i=1i=1ItcanbeshownthatXnXn2c2c22Var(Yi)=pσandBias(Yi)=(n−p)[E(Sp)−σ],i=1i=1whereS2istheMSEfromthecurrentmodel.Usingthis,wephaveXnc222mse(Yi)=pσ+(n−p)[E(Sp)−σ],(3.9)i=1Dividing(3.9)byσ2,wemakeitscale-free:nc22Xmse(Yi)E(Sp)−σ=p+(n−p),i=1σ2σ2Ifthemodeldoesnotfitwell,thenS2isabiasedestimateofσ2.pWecanestimateE(S2)byMSEandestimateσ2bytheMSEppfromthemaximalmodel(thelargestmodelwecanconsider),i.e.,σc2=MSE=MSE(X,...,X).UsingtheestimatorsP−11P−1forE(S2)andσ2givespMSEp−MSE(X1,...,XP−1)SSEpCp=p+(n−p)=−MSE(X1,...,XP−1)MSE(X1,...,XP−1)SmallCpisagoodthing.AsmallvalueofCpindicatesthatthemodelisrelativelyprecise(hassmallvariance)inestimating CHAPTER3.UNIVARIATETIMESERIESMODELS83thetrueregressioncoefficientsandpredictingfutureresponses.Thisprecisionwillnotimprovemuchbyaddingmorepredictors.LookformodelswithsmallCp.Ifwehaveenoughpredictorsintheregressionmodelsothatallthesignificantpredictorsareincluded,thenMSEp≈MSE(X1,...,XP−1)anditfollowsthatCp≈p.ThusCpclosetopisevidencethatthepredictorsinthepoolofpotentialpredictors(X1,...,XP−1)butnotinthecurrentmodel,arenotimportant.ModelswithconsiderablelackoffithavevaluesofCplargerthanp.TheCpcanbeusedtocomparemodelswithdifferentsizes.Ifweuseallthepotentialpredictors,thenCp=P.4.PRESSp:ThePRESS(predictionsumofsquares)isdefinedasXn2PRESS=εb,(i)i=1whereεb(i)iscalledPRESS(predictionsumofsquares)residualforthetheithobservation.ThePRESSresidualisdefinedasccεb(i)=Yi−Y(i),whereY(i)isthefittedvalueobtainedbyleavingtheithobservation.ModelswithsmallPRESSpfitwellinthesenseofhavingsmallpredictionerrors.PRESSpcanbecalcu-latedwithoutfittingthemodelntimes,eachtimedeletingoneofthencases.Onecanshowthatεb(i)=εbi/(1−hii),wherehistheithdiagonalelementofH=X(X′X)−1X′.ii3.3.2SequentialMethods1.Forwardselection(a)Startwiththenullmodel. CHAPTER3.UNIVARIATETIMESERIESMODELS84(b)Addthesignificantvariableifp-valueislessthanpenter,(equiv-alently,FislargerthanFenter).(c)Continueuntilnomorevariablesenterthemodel.2.Backwardelimination(a)Startwiththefullmodel.(b)Eliminatetheleastsignificantvariablewhosep-valueislargerthanpremove,(equivalently,FissmallerthanFremove).(c)Continueuntilnomorevariablescanbediscardedfromthemodel.3.Stepwiseselection(a)Startwithanymodel.(b)Checkeachpredictorthatiscurrentlyinthemodel.SupposethecurrentmodelcontainsX1,...,Xk.ThenFstatisticforXiisSSE(X1,...,Xi−1,Xi+1,...,Xk)−SSE(X1,...,Xk)F=∼F(1;n−kMSE(X1,...,Xk)Eliminatetheleastsignificantvariablewhosep-valueislargerthanpremove,(equivalently,FissmallerthanFremove).(c)Continueuntilnomorevariablescanbediscardedfromthemodel.(d)Addthesignificantvariableifp-valueislessthanpenter,(equiv-alently,FislargerthanFenter).(e)Gotostep(ii)(f)Repeatuntilnomorepredictorscanbeenteredandnomorecanbediscarded. CHAPTER3.UNIVARIATETIMESERIESMODELS853.3.3LikelihoodBased-CriteriaThebasicideaofAkaike’sandalikeapproachescanbefoundinAkaike(1973)andsubsequentpapers;seetherecentbookbyBurn-hamandAnderson(2003).Supposethatf(y):truemodel(unknown)givingrisetodata(isavectorofdata)andg(y,θ):candidatemodel(parametervector).Wewanttofindamodelg(y,θ)“closeto”f(y).TheKullback-Leiblerdiscrepancy:f(Y)K(f,g)=Eflog.g(Y,θ)Thisisameasureofhow“far”modelgisfrommodelf(withrefer-encetomodelf).Properties:K(f,g)≥,0K(f,g)=0⇐⇒f=g.Ofcourse,wecanneverknowhowfarourmodelgisfromf.ButAkaike(1973)showedthatwemightbeabletoestimatesomethingalmostasgood.Supposewehavetwomodelsunderconsideration:g(y,θ)andh(y,φ).Akaike(1973)showedthatwecanestimateK(f,g)−K(f,h).Itturnsoutthatthedifferenceofmaximizedlog-likelihoods,cor-rectedforabias,estimatesthedifferenceofK-Ldistances.Thecbccbcmaximizedlikelihoodsare,Lg=g(y,θ)andLh(y,φ),whereθandφcaretheMLestimatesoftheparameters.Akaike’sresult:[log(Lg)−cq]−[log(Lh)−r]isanasymptoticallyunbiasedestimate(i.e.biasapproacheszeroassamplesizeincreases)ofK(f,g)−K(f,h).Hereqisthenumberofparametersestimatedinθ(modelg)andris CHAPTER3.UNIVARIATETIMESERIESMODELS86thenumberofparametersestimatedinφ(modelh).Thepriceofparameters:thelikelihoodsintheaboveexpressionarepenalizedbythenumberofparameters.TheAICformodelgisgivenbycAIC=−2log(Lg)+2q.TheAICmightnotperformwellforthesmallsamplesizecase.Toovercomethisshortcoming,abiasedcorrectionversionofAICwasproposedbyHurvichandTsai(1989),definedbycAICC=−2log(Lg)+2(q+1)/(n−q−2)=AIC+2(q+1)(q+2)/(n−q−2).TheAICCisinthebetweentheAIC(lesspenalty)andtheBIC(heavypenalty).AnotherapproachisgivenbythemucholdernotionofBayesianstatistics.IntheBayesianapproach,weassumethataprioriuncer-taintyaboutthevalueofmodelparametersisrepresentedbyapriordistribution.Uponobservingthedata,thispriorisupdated,yieldingaposteriordistribution.Inordertomakeinferencesaboutthemodel(ratherthanitsparameters),weintegrateacrosstheposteriordis-tribution.Undertheassumptionthatallmodelsareaprioriequallylikely(becausetheBayesianapproachrequiresmodelpriorsaswellasparameterpriors),Bayesianmodelselectionchoosesthemodelwithhighestmarginallikelihood.TheratiooftwomarginallikelihoodsiscalledaBayesfactor(BF),whichisawidelyusedmethodofmodelselectioninBayesianinference.ThetwointegralsintheBayesfac-torarenontrivialtocomputeunlesstheyformaconjugatedfamily.MonteCarlomethodsareusuallyrequiredtocomputeBF,especiallyforhighlyparameterizedmodels.AlargesampleapproximationofBFyieldstheeasilycomputableBICcBIC=−2log(Lg)+qlogn. CHAPTER3.UNIVARIATETIMESERIESMODELS87Inasum,bothAICandBICaswellastheirgeneralizationshaveasimilarformascLC=−2log(Lg)+λq,whereλisfixedconstant.Therecentdevelopmentssuggesttheuseofadataadaptivepenaltytoreplacethefixedpenalties.See,Bai,RaoandWu(1999)andShenandYe(2002).Thatistoestimateλbydatainacomplexityformbasedonaconceptofgeneralizeddegreeoffreedom.3.3.4Cross-ValidationandGeneralizedCross-ValidationThecrossvalidation(CV)isthemostcommonlyusedmethodformodelassessmentandselection.Themainideaisadirectestimateofextra-sampleerror.ThegeneralversionofCVistosplitdataintoKroughlyequal-sizedpartsandtofitthemodeltotheotherK−1partsandcalculatepredictionerrorontheremainingpart.Xnc2CV=(Yi−Y−i)i=1cwhereY−iisthefittedvaluecomputedwithk-thpartofdatare-moved.AconvenientapproximationtoCVforlinearfittingwithsquarederrorlossisgeneralizedcrossvalidation(GCV).Alinearfittingmethodcchasthefollowingproperty:Y=SY,whereYiisthefittedvaluewiththewholedata.Formanylinearfittingmethodswithleave-one-out(k=1),itcanbeshowedeasilythat2ncXnXYi−Yic2CV=(Yi−Y−i)=.i=1i=11−Sii CHAPTER3.UNIVARIATETIMESERIESMODELS88Duetotheintensivecomputation,theCVcanbeapproximatedbytheGCV,definedbyc2Pnc2XnYi−Yii=1(Yi−Yi)GCV==.i=11−trace(S)/n(1−trace(S)/n)2IthasbeenshownthatboththeCVandGCVmethodareveryappallingtononparametricmodeling.Recently,theleave-one-outcross-validationmethodwaschallengedbyShao(1993).Shao(1993)claimedthatthepopularleave-one-outcross-validationmethod,whichisasymptoticallyequivalenttomanyothermodelselectionmethodssuchastheAIC,theCp,andthebootstrap,isasymptoticallyinconsistentinthesensethattheproba-bilityofselectingthemodelwiththebestpredictiveabilitydoesnotconvergeto1asthetotalnumberofobservationsn→∞andheshowedthattheinconsistencyoftheleave-one-outcross-validationcanberectifiedbyusingaleave-nν-outcross-validationwithnν,thenumberofobservationsreservedforvalidation,satisfyingnν/n→1asn→∞.3.3.5PenalizedMethods1.BridgeandRidge:FrankandFriedman(1993)proposedtheLq(q>0)penalizedleastsquaresasXnXX2q(Yi−βjXij)+λ|βj|,i=1jjwhichresultsintheestimatorwhichiscalledthebridgeestima-tor.Ifq=2,theresultingestimatoriscalledtheridgeestimatorcT−1Tgivenbyβ=(XX+λI)XY. CHAPTER3.UNIVARIATETIMESERIESMODELS892.LASSO:Tibshirani(1996)proposedtheso-calledLASSOwhichistheminimizerofthefollowingconstrainedleastsquaresXnXX2(Yi−βjXij)+λ|βj|,i=1jjcc0c0+whichresultsinthesoftthreshingruleβj=sign(βj)(|βj|−λ).3.Non-concavePenalizedLS:FanandLi(2001)proposedthenon-concavepenalizedleastsquaresXnXX2(Yi−βjXij)+pλ(|βj|),i=1jjwherethehardthreshingpenaltyfunctionp(|θ|)=λ2−(|θ|−λ2cλ)|(|θ|<λ),whichresultsinthehardthreshingruleβj=c0c0βjI(|βj|>λ).Finally,FanandLi(2001)proposedtheso-calledthesmoothlyclippedabsolutedeviation(SCAD)modelselectioncriterionwiththepenalizedfunctiondefinedas(aλ−θ)+′pλ(θ)=λI(θ≤λ)−I(θ>λ)forsomea>2and(a−1)λwhichresultsintheestimatorsign(βc0)(|βc0|−λ)+when|βc0|≤2λ,jjjcc0c0c0βj=(a−1)βj−sign(βj)aλ/(a−2)when2λ≤|βj|≤aλ,c0c0βjwhen|βj|>aλ.Also,FanandLi(2001)showedthattheSCADestimatorsat-isfiesthreeproperties:(1)unbiasedness,(2)sparsity,and(3)continuityandFanandPeng(2004)consideredthecasethatthenumberofregressorscandependonthesamplesizeandgoestoinfinityinacertainrate.Remark:Notethatthetheoryforthepenalizedmethodsisstillopenfortimeseriesdataanditwouldbeaveryinterestingresearchtopic. CHAPTER3.UNIVARIATETIMESERIESMODELS903.4IntegratedModels-I(1)Webeginourstudyoftimecorrelationbymentioningasimplemodelthatwillintroducestrongcorrelationsovertime.Thisistheran-domwalkorunitrootmodelwhichdefinesthecurrentvalueofthetimeseriesasjusttheimmediatelyprecedingvaluewithadditivenoise.Themodelformsthebasis,forexample,oftherandomwalktheoryofstockpricebehavior.Inthismodelwedefinext=xt−1+wt,(3.10)wherewisawhitenoiseserieswithmeanzeroandvarianceσ2.ThetleftpanelofFigure3.3showsatypicalrealizationofsuchaseries(wt∼N(0,1))andweobservethatitbearsapassingresemblancetotheglobaltemperatureseries.Appealingto(3.10),thebestpre-dictionofthecurrentvaluewouldbeexpectedtobegivenbyitsimmediatelyprecedingvalue.Themodelis,inasense,unsatisfac-tory,becauseonewouldthinkthatbetterresultswouldbepossiblebyamoreefficientuseofthepast.TheACFoftheoriginalseries,showninFigure3.4,exhibitsaslowdecayaslagsincrease.Inordertomodelsuchaserieswithoutknowingthatitisnecessarilygener-atedby(3.10),onemighttrylookingatafirstdifference,shownintherightpanelofFigure3.3,andcomparingtheresulttoawhitenoiseorcompletelyindependentprocess.Itisclearfrom(3.10)thatthefirstdifferencewouldbe∆xt=xt−xt−1=wtwhichisjustwhitenoise.TheACFofthedifferencedprocess,inthiscase,wouldbeexpectedtobezeroatalllagsh6=0andthesampleACFshouldreflectthisbehavior.ThefirstdifferenceoftherandomwalkintherightpanelofFigure3.3isalsoshowninthebottompanelsofFigure3.4andwenotethatitappearstobemuchmorerandom.TheACF,shownintheleftbottompanelofFigure3.4,reflectsthispredicted CHAPTER3.UNIVARIATETIMESERIESMODELS91FirstDifferenceRandomWalk0510−3−2−1012050100150200050100150200Figure3.3:Atypicalrealizationoftherandomwalkseries(leftpanel)andthefirstdifferenceoftheseries(rightpanel).behavior,withnosignificantvaluesforlagsotherthanzero.Itisclearthat(3.10)isareasonablemodelforthisdata.Theoriginalse-riesisnonstationary,withanautocorrelationfunctionthatdependsontimeoftheformr1/(t+h),ifh≥0,ρ(xt+h,xt)=r(t+h)/t,ifh<0.Theaboveexample,usingadifferencetransformationtomakearandomwalkstationary,showsaveryparticularcaseofthemodelidentificationprocedureadvocatedbyBox,etal(1994).Namely,weseekalinearlyfilteredtransformationoftheoriginalseries,basedstrictlyonthepastvalues,thatwillreduceittocompletelyrandomwhitenoise.Thisgivesamodelthatenablespredictiontobedonewitharesidualnoisethatsatisfiestheusualstatisticalassumptionsaboutmodelerror.Wewillintroduce,inthefollowingdiscussion,moregeneralver-sionsofthissimplemodelthatareusefulformodelingandforecast-ingserieswithobservationsthatarecorrelatedintime.Thenotation CHAPTER3.UNIVARIATETIMESERIESMODELS92RandomWalkACFPACF−0.50.00.51.0−0.50.00.51.0051015205101520lagFirstDifferenceACFPACF−0.50.00.51.0−0.50.00.51.0051015205101520lagFigure3.4:Autocorrelationfunctions(ACF)(left)andpartialautocorrelationfunctions(PACF)(right)fortherandomwalk(toppanel)andthefirstdifference(bottompanel)series.andterminologywereintroducedinthelandmarkworkbyBoxandJenkins(1970).ArequirementfortheARMAmodelofBoxandJenkinsisthattheunderlyingprocessbestationary.ClearlythefirstdifferenceoftherandomwalkisstationarybuttheACFofthefirstdifferenceshowsrelativelylittledependenceonthepast,meaningthatthedifferencedprocessisnotpredictableintermsofitspastbehavior.Tointroduceanotationthathasadvantagesfortreatingmoregeneralmodels,definetheback-shiftoperatorLastheresultofshiftingtheseriesbackbyonetimeunit,i.e.Lxt=xt−1(3.11)andapplyingsuccessivelyhigherpowers,Lkx=x.Theoperatortt−khasmanyoftheusualalgebraicpropertiesandallows,forexample,writingtherandomwalkmodel(3.10)as(1−L)xt=wt.Notethat CHAPTER3.UNIVARIATETIMESERIESMODELS93thedifferenceoperatordiscussedpreviouslyin1.2.2isjust∆=1−L.IdentifyingnonstationarityisanimportantfirststepintheBox-Jenkinsprocedure.Fromtheabovediscussion,wenotethattheACFofanonstationaryprocesswilltendtodecayratherslowlyasafunctionoflagh.Forexample,astraightlylinewouldbeperfectlycorrelated,regardlessoflag.Basedonthisobservation,wementionthefollowingpropertiesthataidinidentifyingnon-stationarity.Property2.1:TheACFofanon-stationarytimeseriesdecaysveryslowlyasafunctionoflagh.ThePACFofanon-stationarytimeseriestendstohaveapeakverynearunityatlag1,withothervalueslessthanthesignificancelevel.NotethatsinceI(1)modelisveryimportantinmodelingeco-nomicandfinancialdata,wewilldiscussmoreonthemodelandthestatisticalinferenceinthelaterchapter.3.5AutoregressiveModels-AR(p)3.5.1ModelNow,extendingthenotionsabovetomoregenerallinearcombina-tionsofpastvaluesmightsuggestwritingxt=φ1xt−1+φ2xt−2+···+φpxt−p+wt(3.12)asafunctionofppastvaluesandanadditivenoisecomponentwt.Themodelgivenby(3.12)iscalledanautoregressivemodeloforderp,sinceitisassumedthatoneneedsppastvaluestopredictxt.Thecoefficientsφ1,φ2,···,φpareautoregressivecoefficients,chosentoproduceagoodfitbetweentheobservedxtanditspredictionbased CHAPTER3.UNIVARIATETIMESERIESMODELS94onxt−1,xt−2,···,xt−p.Itisconvenienttorewrite(3.12),usingtheback-shiftoperator,as2pφ(L)xt=wt,whereφ(L)=1−φ1L−φ2L−···−φpL,(3.13)isapolynomialwithroots(solutionsofφ(L)=0)outsidetheunitcircle(|L|>1)1.Therestrictionsarenecessaryforexpressingthejsolutionxtof(3.13)intermsofpresentandpastvaluesofwt,whichiscalledinvertibilityofanARMAseries.ThatsolutionhastheformX∞kxt=ψ(L)wt,whereψ(L)=ψkL,(3.14)k=0isaninfinitepolynomial(ψ0=1),withcoefficientsdeterminedbyequatingcoefficientsofBinψ(L)φ(L)=1.(3.15)Equation(3.14)canbeobtainedformallybynotingthatchoosingψ(L)satisfying(3.15),andmultiplyingbothsidesof(3.14)byψ(L)givestherepresentation(3.14).Itisclearthattherandomwalkhasφ1=1andφk=0forallk≥2,whichdoesnotsatisfytherestrictionPandtheprocessisnonstationary.xtisstationaryifk|ψk|<∞;seeProposition3.1.2inBrockwellandDavis(1991,p.84),whichcanbeP2weakenedbykψk<∞;seeHamilton(1994,p.52).Example2.2:Supposethatwehaveanautoregressivemodel(3.12)withp=1,i.e.,xt−φ1xt−1=(1−φ1L)xt=wt.Then(3.15)becomes(1+ψL+ψL2+···)(1−φL)=1.Equatingcoefficients121ofLimpliesthatψ−φ=0orψ=φ.ForL2,wewouldget1111ψ−ψφ=0,orψ=φ2.Continuing,weobtainψ=φkandthe21121k11ThisrestrictionisasufficientandnecessaryconditionforanARMAtimeseriestobeinvertible;seeSection3.7inHamilton(1994)orTheorem3.1.2inBrockwellandDavis(1991,p.86)andtherelateddiscussions. CHAPTER3.UNIVARIATETIMESERIESMODELS95representationisX∞kkψ(L)=1+φ1Lk=1P∞kandwehavext=k=0φ1wt−k.Therepresentation(3.14)isfun-damentalfordevelopingapproximateforecastsandalsoexhibitstheseriesasalinearprocessoftheformconsideredinChapter2.Fordatainvolvingsuchautoregressive(AR)modelsasdefinedabove,themainselectionproblemsaredecidingthattheautoregres-sivestructureisappropriateandthenindeterminingthevalueofpforthemodel.TheACFoftheprocessisapotentialaidfordeter-miningtheorderoftheprocessasarethemodelselectionmeasuresdescribedinSection3.3.TodeterminetheACFofthepthorderARin(3.12),writetheequationaspXxt−φkxt−k=wtk=1andmultiplybothsidesbyxt−hforanyh≥1.AssumingthatthemeanE(xt)=0,andusingthedefinitionoftheautocovariancefunctionleadstotheequationpXE(xtxt−h−φkxt−kxt−h=E[wtxt−h].k=1PpTheleft-handsideimmediatelybecomesγx(h)−k=1φkγx(h−k).Therepresentation(3.14)impliesthatσ2,ifh=0,E[wtxt−h]=E[wt(wt−h+φ1wt−h−1+φ2wt−h−2+···)]=w0otherwise.Hence,wemaywritetheequationsfordeterminingγx(h)aspX2γx(0)−φkγx(−k)=σw(3.16)k=1 CHAPTER3.UNIVARIATETIMESERIESMODELS96andpXγx(h)−φkγx(h−k)=0forh≥1.(3.17)k=1Notethatonewillneedthepropertyγx(h)=γx(−h)insolvingtheseequations.Equations(3.16)and(3.17)arecalledtheYule-WalkerEquations(seeYule,1927,Walker,1931).Example2.3:ConsiderfindingtheACFofthefirst-orderautore-gressivemodel.First,(3.17)impliesthatγ(0)−φγ(1)=σ2.x1xwForh≥1,weobtainγx(h)−φ1γx(h−1)=0.Solvingthesesuccessivelygivesγ(h)=γ(0)φh.Combiningwith(3.16)yieldsxx1γ(0)=σ2/(1−φ2).Itfollowsthattheautocovariancefunctionisxw1γ(h)=σ2φh/(1−φ2).Takingintoaccountthatγ(h)=γ(−h),xw11xx|h|weobtainρx(h)=φ1forallh.Theexponentialdecayistypicalofautoregressivebehaviorandtheremayalsobesomeperiodicstructure.However,themosteffec-tivediagnosticofARstructureisinthePACFandissummarizedbythefollowingidentificationproperty:Property2.2:Thepartialautocorrelationfunctionasafunc-tionoflaghiszeroforh>p,theorderoftheautoregressiveprocess.ThisenablesonetomakeapreliminaryidentificationoftheorderpoftheprocessusingthepartialautocorrelationfunctionPACF.SimplychoosetheorderbeyondwhichmostofthesamplevaluesofthePACFareapproximatelyzero.Toverifytheabove,notethatthePACF(seeSection2.4.3)isbasicallythelastcoefficientobtainedwhenminimizingthesquarederror2XhMSE=Ext+h−akxt+h−k.k=1 CHAPTER3.UNIVARIATETIMESERIESMODELS97Settingthederivativeswithrespecttoajequaltozeroleadstotheequations2XhExt+h−akxt+h−kxt+h−j=0k=1ThiscanbewrittenasXhγx(j)−akγx(j−k)=0k=1for1≤j≤h.Now,fromEquationand(3.17),itisclearthat,foranAR(p),wemaytakeak=φkfork≤pandak=0fork>ptogetasolutionfortheaboveequation.ThisimpliesProperty2.2above.Havingdecidedontheorderpofthemodel,itisclearthat,fortheestimationstep,onemaywritethemodel(3.12)intheregressionform′xt=φzt+wt,(3.18)whereφ=(φ,φ,···,φ)′correspondstoβandz=(x,x,···,x12ptt−1t−2t−pisthevectorofdependentvariablesin(3.2).Takingintoaccountthefactthatxtisnotobservedfort≤0,wemayruntheregressionapproachinSection3.2fort=p+1,···,ntogetestimatorsforφandforσ2,thevarianceofthewhitenoiseprocess.Theseso-calledconditionalmaximumlikelihoodestimatorsarecommonlyusedbecausetheexactmaximumlikelihoodestimatorsinvolvesolv-ingnonlinearequations;seeChapter5inHamilton(1994)fordetailsandwewilldiscussthisissuelater.Example2.4:WeconsiderthesimpleproblemofmodelingtherecruitseriesshownintherightpanelofFigure2.1usinganautore-gressivemodel.ThetoprightpanelofFigure2.16andthetoprightpanelofFigure2.19showstheautocorrelationandpartialautocorre-lationfunctionsoftherecruitseries.ThePACFhaslargevaluesfor CHAPTER3.UNIVARIATETIMESERIESMODELS98Table3.1:AICCvaluesfortenmodelsfortherecruitsseriesp12345678910AICC5.755.525.535.545.545.555.555.565.575.58h=1and2andthenisessentiallyzeroforhigherorderlags.ThisimpliesbyProperty2.2abovethatasecondorder(p=2)ARmodelmightprovideagoodfit.RunningtheregressionprogramforanAR(2)modelwithinterceptxt=φ0+φ1xt−1+φ2xt−2+wtccleadstotheestimatorsφ0=61.8439(4.0121),φ1=1.3512(0.0417),c2φ2=−0.4612(0.0416)andσc=89.53,wheretheestimatedstan-darddeviationsareinparentheses.Todeterminewhethertheaboveorderisthebestchoice,wefittedmodelsfor1≤p≤10,obtain-ingcorrectedAICCvaluessummarizedinTable3.1using(3.8)withK=2.ThisshowsthattheminimumAICCobtainsforp=2andwechoosethesecondordermodel.Example2.5:Thepreviousexampleusedvariousautoregressivemodelsfortherecruitsseries,fittingasecond-orderregressionmodel.WemayalsousethisregressionideatofitthemodeltootherseriessuchasadetrendedversionoftheSOIgiveninpreviousdiscussions.WehavenotedinourdiscussionsofFigure2.19fromthepartialautocorrelationfunctionthataplausiblemodelforthisseriesmightbeafirstorderautoregressionoftheformgivenabovewithp=1.Again,puttingthemodelaboveintotheregressionframework(3.2)cforasinglecoefficientleadstotheestimatorsφ1=0.59withstan-darderror0.04,σc2=0.09218andAICC(1)=−1.375.TheACFoftheseresidualsshownintheleftpanelofFigure3.5,however,willstillshowcyclicalvariationanditisclearthattheystillhaveanumber CHAPTER3.UNIVARIATETIMESERIESMODELS99√ofvaluesexceedingthe1.96/nthreshold.AsuggestedprocedureooooooooACFofresidulsofAR(1)forSOIoAICAICCooooooooooooooooooooooooooooooooooooooooooooooooo−0.50.00.51.0−1.70−1.65−1.60−1.55−1.50−1.45−1.40−1.35oo05101520051015202530LagFigure3.5:Autocorrelation(ACF)ofresidualsofAR(1)forSOI(leftpanel)andtheplotofAICandAICCvalues(rightpanel).istotryhigherorderautoregressivemodelsandsuccessivemodelsfor1≤p≤30werefittedandtheAICC(K)valuesareplottedintherightpanelofFigure3.5.Thereisaclearminimumforap=16ordermodel.Thecoefficientvectorisφwithcomponentsandtheirstandarderrorsintheparentheses0.4050(0.0469),0.0740(0.0505),0.1527(0.0499),0.0915(0.0505),−0.0377(0.0500),−0.0803(0.0493),−0.0743(0.0493),−0.0679(0.0492),0.0096(0.0492),0.1108(0.0491),0.1707(0.0492),0.1606(0.0499),0.0281(0.0504),−0.1902(0.0501),−0.1283(0.0510),−0.0413(0.0476),andσc2=0.07166.3.5.2ForecastingTimeseriesanalysishasprovedtobefairlygoodwayofproducingforecasts.Itsdrawbackisthatitistypicallynotconducivetostruc-turaloreconomicanalysisoftheforecast.Themodelhasforecastingpoweronlyifthefuturevariablebeingforecastedisrelatedtocurrentvaluesofthevariablesthatweincludeinthemodel. CHAPTER3.UNIVARIATETIMESERIESMODELS100ThegoalistoforecastthevariableysbasedonasetofvariablesX(Xmayconsistofthelagsofvariabley).Letysdenoteafore-ttttcastofysbasedonXt.AquadraticlossfunctionisthesameasinOLSregression,i.e.chooseyttominimizeE(yt−y)2andthemeanssshisquarederror(MSE)isdefinedasMSE(yt)=E(yt−y)2|X.ItssstcanbeshownthattheforecastwiththesmallestMSEistheexpec-tationofyconditionalonX,thatisyt=E(y|X).Then,thestsstMSEoftheoptimalforecastistheconditionalvarianceofysgivenXt,thatisVar(ys|Xt).Wenowconsidertheclassofforecaststhatarelinearprojection.Theseforecastsareusedveryofteninempiricalanalysisoftimeseriesdata.Therearetwoconditionsfortheforecastyttobealinearsprojection:(1)TheforecastytneedstobealinearfunctionofX,stthatisyt=E(y|X)=β′X,and(2)thecoefficientsβshouldbesstt′′′choseninsuchawaythatE[(ys−βXt)Xt]=0.TheforecastβXtsatisfying(1)and(2)iscalledthelinearprojectionofysonXt.OneofthereasonslinearprojectsarepopularisthatthelinearprojectionproducesthesmallestMSEamongtheclassoflinearforecastingrules.Finally,wegiveageneralapproachtoforecastingforanyprocessthatcanbewrittenintheform(3.14),alinearprocess.ThisincludestheAR,MAandARMAprocesses.Webeginbydefininganh-stepforecastoftheprocessxtastxt+h=E[xt+h|xt,xt−1,···]Notethatthisisnotexactlyrightbecauseweonlyhavex1,x2,···,xtavailable,sothatconditioningontheinfinitepastisonlyanapproximation.Fromthisdefinition,itisreasonabletointuitthat CHAPTER3.UNIVARIATETIMESERIESMODELS101xt=xfors≤tandsttE[ws|xt,xt−1,···]=E[ws|wt,wt−1,···]=ws=ws(3.19)fors≤t.Fors>t,usextandstE[ws|xt,xt−1,···]=E[ws|wt,wt−1,···]=ws=E(ws)=0(3.20)sincewswillbeindependentofpastvaluesofwt.Wedefinetheh-stepforecastvarianceastt2Pt+h=E[(xt+h−xt+h)|xt,xt−1,···](3.21)Todevelopanexpressionforthismeansquareerror,notethat,withψ0=1,wecanwriteX∞xt+h=ψkwt+h−k.k=0Then,sincewt=0fort+h−k>t,i.e.k2.Forecastsouttolagh=4andbeyond,ifneces-sary,canbefoundbysolving(3.15)forψ1,ψ2andψ3,andsub-stitutinginto(3.21).ByequatingcoefficientsofB,L2andL3in(1−φL−φL2)(1+ψL+ψL2+ψL3+···)=1,weobtain12123ψ1=ψ1,ψ2−φ2+φ1ψ1=0andψ3−φ1ψ2−φ2ψ1=0.Thisgivesthecoefficientsψ=φ,ψ=φ−φ2,ψ=21122132cc,φ2φ1−φ1.FromExample2.4,wehaveφ1=1.35,φ2=2c−0.46,σcw=90.31andβ0=6.74.Theforecastsareoftheformxt=6.74+1.35xt−0.46xt.Fortheforecastvari-t+ht+h−1t+h−2cccance,weevaluateψ1=1.35,ψ2=2.282,ψ3=−3.065,leadingto90.31,90.31(2.288),90.31(7.495)and90.31(16.890)forforecastsath=1,2,3,4.Thestandarddeviationsoftheforecastsare9.50,14.37,26.02and39.06forthestandarderrorsoftheforecasts.Therecruitsseriesvaluesrangefrom20to100sotheforecastuncertaintywillberatherlarge.3.6MovingAverageModels–MA(q)Wemayalsoconsiderprocessesthatcontainlinearcombinationsofunderlyingunobservedshocks,say,representedbywhitenoiseserieswt.ThesemovingaveragecomponentsgenerateaseriesoftheformqXxt=wt−θkwt−k,(3.23)k=1whereqdenotestheorderofthemovingaveragecomponentandθk(1≤k≤q)areparameterstobeestimated.Usingtheback-shift CHAPTER3.UNIVARIATETIMESERIESMODELS103notation,theaboveequationcanbewrittenintheformqXkxt=θ(L)wtwithθ(L)=1−θkL,(3.24)k=1whereθ(L)isanotherpolynomialintheshiftoperatorB.ItshouldbenotedthattheMAprocessoforderqisalinearprocessoftheformconsideredearlierinProblem4inChapter2withψ0=1,ψ1=−θ1,···,ψq=−θq.ThisimpliesthattheACFwillbezeroforlagslargerthanqbecausetermsintheformofthecovariancefunctiongiveninProblem4ofChapter2willallbezero.Specifically,theexactformsareqq−hXX222γx(0)=σw1+θkandγx(h)=σw−θh+θk+hθkk=1k=1(3.25)for1≤h≤q−1,withγ(q)=−σ2θ,andγ(h)=0forh>q.xwqxHence,wewillhavethepropertyofACFforforMASeries.Property2.3:Foramovingaverageseriesoforderq,notethattheautocorrelationfunction(ACF)iszeroforlagsh>q,i.e.ρx(h)=0forh>q.Sucharesultenablesustodiagnosetheorderofamovingaveragecomponentbyexaminingρx(h)andchoosingqasthevaluebeyondwhichthecoefficientsareessentiallyzero.Example2.7:ConsiderthevarvethicknessesinFigure2.19,whichisdescribedinProblem7ofChapter2.Figure3.6showstheACFandPACFoftheoriginallog-transformedvarveseries{xt}andthefirstdifferences.TheACFoftheoriginalseries{xt}indicatesapos-siblenon-stationarybehavior,andsuggeststakingafirstdifference∆xt,interpretedhearasthepercentageyearlychangeindeposition.TheACFofthefirstdifference∆xtshowsaclearpeakath=1and CHAPTER3.UNIVARIATETIMESERIESMODELS104ACFPACFlogvarves−0.50.00.51.0−0.50.00.51.0051015202530051015202530Firstdifference−0.50.00.51.0−0.50.00.51.0051015202530051015202530Figure3.6:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthelogvarveseries(toppanel)andthefirstdifference(bottompanel),showingapeakintheACFatlagh=1.noothersignificantpeaks,suggestingafirstordermovingaverage.Fittingthefirstordermovingaveragemodel∆xt=wt−θ1wt−1tothisdatausingtheGauss-Newtonproceduredescribednextleadstob2θ1=0.77andσcw=0.2358.Fittingthepuremovingaveragetermturnsintoanonlinearprob-lemaswecanseebynotingthateithermaximumlikelihoodorre-gressioninvolvessolving(3.23)or(3.24)forwt,andminimizingthesumofthesquarederrors.Supposethattherootsofπ(L)=0arealloutsidetheunitcircle,thenthisispossiblebysolvingπ(L)θ(L)=1,sothat,forthevectorparameterθ=(θ,···,θ)′,wemaywrite1qwt(θ)=π(L)xt(3.26)Pn2andminimizeSSE(θ)=t=q+1wt(θ)asafunctionofthevectorparameterθ.Wedonotreallyneedtofindtheoperatorπ(L)butcansimplysolve(3.26)recursivelyforwt,withw1,w2,···,wq=0, CHAPTER3.UNIVARIATETIMESERIESMODELS105Pqandwt(θ)=xt+k=1θkwt−kforq+1≤t≤n.ItiseasytoverifythatSSE(θ)willbeanonlinearfunctionofθ1,θ2,···,θq.However,notethatbytheTaylorexpansion∂wt(θ)wt(θ)≈wt(θ0)+(θ−θ0),∂θ0wherethederivativeisevaluatedatthepreviousguessθ0.Rearrang-ingtheaboveequationleadsto∂wt(θ)wt(θ0)≈−(θ−θ0)+wt(θ),∂θ0whichisjusttheregressionmodel(3.2).Hence,wecanbeginwithaninitialguessθ=(0.1,0.1,···,0.1)′,sayandsuccessivelyminimize0SSE(θ)untilconvergence.SeeChapter5inHamilton(1994)fordetailsandwewilldiscussthisissuelater.Forecasting:Inordertoforecastamovingaverageseries,notePqthatxt+h=wt+h−k=1θkwt+h−k.Theresultsbelow(3.19)implythatxt=0ifh>qandifh≤q,t+hqtXxt+h=−θkwt+h−k,k=hwherethewtvaluesneededfortheabovearecomputedrecursivelyasbefore.Becauseof(3.14),itisclearthatψ0=1andψk=−θkfor1≤k≤qandthesevaluescanbesubstituteddirectlyintothevarianceformula(3.22).Thatis,hX−1Pt=σ21+θ2.t+hwkk=1 CHAPTER3.UNIVARIATETIMESERIESMODELS1063.7AutoregressiveIntegratedMovingAverageModel-ARIMA(p,d,q)NowcombiningtheautoregressiveandmovingaveragecomponentsleadstotheautoregressivemovingaverageARMA(p,q)model,writ-tenasφ(L)xt=θ(L)wtwherethepolynomialsinBareasdefinedearlierin(3.13)and(3.24),withpautoregressivecoefficientsandqmovingaveragecoefficients.Inthedifferenceequationform,thisbecomespqXXxt−φkxt−k=wt−θkwt−k.k=1k=1ThemixedprocessesdonotsatisfytheProperties2.1-2.3anymorebuttheytendtobehaveinapproximatelythesameway,evenforthemixedcases.EstimationandforecastingforsuchproblemsaretreatedinessentiallythesamemannerasfortheARandMAprocesses.Wenotethatwecanformallydividebothsidesof(3.20)byφ(L)andnotethattheusualrepresentation(3.14)holdswhenψ(L)φ(L)=θ(L).(3.27)Forforecasting,wedeterminethe{ψk}byequatingcoefficientsof{Lk}in(3.27),asbefore,assumingthealltherootsofφ(L)=0aregreaterthanoneinabsolutevalue.Similarly,wecanalwayssolvefortheresiduals,saypqXXwt=xt−φkxt−k+θkwt−kk=1k=1togetthetermsneededforforecastingandestimation.Example2.8:Considertheabovemixedprocesswithp=q=1,i.e.ARMA(1,1).By(3.21),wemaywritext=φ1xt−1+wt−θ1wt−1. CHAPTER3.UNIVARIATETIMESERIESMODELS107Now,x=φx+w−θwsothatxt=φx+0−θw=t+11tt+11tt+11t1tφx−θwandxt=φxtforh>1,leadingtoverysimple1t1tt+h1t+h−1forecastsinthiscase.EquatingcoefficientsofLkin(1−φL)(1+1ψL+ψL2+···)=(1−θL)leadstoψ=(φ−θ)φk−1for121k111k≥1.Using(3.22)leadstotheexpression"hX−1Pt=σ21+(φ−θ)2φ2(k−1)=σ21+(φ−θ)2(1−φ2(h−1))/(1−t+hw111w111k=1fortheforecastvariance.Inthefirstexampleofthischapter,itwasnotedthatnonstationaryprocessesarecharacterizedbyaslowdecayintheACFasinFigure3.4.Inmanyofthecaseswhereslowdecayispresent,theuseofafirstorderdifference∆xt=xt−xt−1=(1−L)xtwillreducethenonstationaryprocessxttoastationaryseries∆xt.OncanchecktoseewhethertheslowdecayhasbeeneliminatedintheACFofthetransformedseries.Higherorderdifferences,∆dx=∆∆d−1xarettpossibleandwecalltheprocessobtainedwhenthedthdifferenceisanARMAseriesanARIMA(p,d,q)serieswherepistheorderoftheautoregressivecomponent,distheorderofdifferencingneededandqistheorderofthemovingaveragecomponent.Symbolically,theformisdφ(L)∆xt=θ(L)wt.TheprinciplesofmodelselectionforARIMA(p,d,q)seriesareob-tainedusingthelikelihoodbasedmethodssuchasAIC,BICorAICCwhichreplaceKbyK=p+qthetotalnumberofARMAparame-tersorothermethodssuchaspenalizedmethodsdescribedinSection3.3. CHAPTER3.UNIVARIATETIMESERIESMODELS1083.8SeasonalARIMAModelsSomeeconomicandfinancialaswellasenvironmentaltimeseriessuchasquarterlyearningpershareofacompanyexhibitscertaincyclicalorperiodicbehavior;seethelaterchaptersonmoredis-cussionsoncyclesandperiodicity.Suchatimeseriesiscalledaseasonal(deterministiccycle)timeseries.Figure2.8showsthetimeplotofquarterlyearningpershareofJohnsonandJohnsonfromthefirstquarterof1960tothelastquarterof1980.Thedatapossesssomespecialcharacteristics.Inparticular,theearninggrewexponentiallyduringthesampleperiodandhadastrongseasonal-ity.Furthermore,thevariabilityofearningincreasedovertime.Thecyclicalpatternrepeatsitselfeveryyearsothattheperiodicityoftheseriesis4.Ifmonthlydataareconsidered(e.g.,monthlysalesofWal-MartStores),thentheperiodicityis12.Seasonaltimeseriesmodelsarealsousefulinpricingweather-relatedderivativesanden-ergyfutures.SeeExample1.8andExample1.9inChapter2formoreexampleswithseasonality.Analysisofseasonaltimeserieshasalonghistory.Insomeappli-cations,seasonalityisofsecondaryimportanceandisremovedfromthedata,resultinginaseasonallyadjustedtimeseriesthatisthenusedtomakeinference.Theproceduretoremoveseasonalityfromatimeseriesisreferredtoasseasonaladjustment.MosteconomicdatapublishedbytheU.S.governmentareseasonallyadjusted(e.g.,thegrowthrateofdomesticgrossproductandtheunemploymentrate).Inotherapplicationssuchasforecasting,seasonalityisasim-portantasothercharacteristicsofthedataandmustbehandledaccordingly.Becauseforecastingisamajorobjectiveofeconomicandfinancialtimeseriesanalysis,wefocusonthelatterapproach CHAPTER3.UNIVARIATETIMESERIESMODELS109anddiscusssomeeconometricmodelsthatareusefulinmodelingseasonaltimeseries.Whentheautoregressive,differencing,orseasonalmovingaveragebehaviorseemstooccuratmultiplesofsomeunderlyingperiods,aseasonalARIMAseriesmayresult.Theseasonalnonstationarityischaracterizedbyslowdecayatmultiplesofsandcanoftenbeeliminatedbyaseasonaldifferencingoperatoroftheform∆Dx=(1−Ls)Dx.Forexample,whenwehavemonthlydata,itsttisreasonablethatayearlyphenomenonwillinduces=12andtheACFwillbecharacterizedbyslowlydecayingspikesat12,24,36,48,···,andwecanobtainastationaryseriesbytransformingwiththeoperator(1−L12)x=x−xwhichisthedifferencebetweenttt−12thecurrentmonthandthevalueoneyearor12monthsago.Iftheautoregressiveormovingaveragebehaviorisseasonalatperiods,wedefineformallytheoperatorsss2sPsΦ(L)=1−Φ1L−Φ2L−···−ΦPL(3.28)andss2sQsΘ(L)=1−Θ1L−Θ2L−···−ΘQL.(3.29)ThefinalformoftheseasonalARIMA(p,d,q)×(P,D,Q)smodelissDdsΦ(L)φ(L)∆s∆xt=Θ(L)θ(L)wt.(3.30)Notethatonespecialmodelof(3.30)isARIMA(0,1,1)×(0,1,1)s,thatisss(1−L)(1−L)xt=(1−θ1L)(1−Θ1L)wt.Thismodelisreferredtoastheairlinemodelormultiplicativeseasonalmodelintheliterature;seeBox,Jenkins,andReinsel CHAPTER3.UNIVARIATETIMESERIESMODELS110(1994,Chapter9).Ithasbeenfoundtobewidelyapplicableinmodelingseasonaltimeseries.TheARpartofthemodelsimplyconsistsoftheregularandseasonaldifferences,whereastheMApartinvolvestwoparameters.WemayalsonotethepropertiesbelowcorrespondingtoProp-erties2.1-2.3.Property2.1’:TheACFofaseasonallynon-stationarytimeseriesdecaysveryslowlyatlagmultipless,2s,3s,···,withzerosinbetween,wheresdenotesaseasonalperiod,usually4forquarterlydataor12formonthlydata.ThePACFofanon-stationarytimeseriestendstohaveapeakverynearunityatlags.Property2.2’:ForaseasonalautoregressiveseriesoforderP,thepartialautocorrelationfunctionΦhhasafunctionoflaghhasnonzerovaluesats,2s,3s,···,Ps,withzerosinbetween,andiszeroforh>Ps,theorderoftheseasonalautoregressiveprocess.Thereshouldbesomeexponentialdecay.Property2.3’:ForaseasonalmovingaverageseriesoforderQ,notethattheautocorrelationfunction(ACF)hasnonzerovaluesats,2s,3s,···,Qsandiszeroforh>Qs.Remark:Notethatthereisabuild-incommandinRcalledarima()whichisapowerfultoolforestimatingandmakinginferenceforanARIMAmodel.Thecommandisarima(x,order=c(0,0,0),seasonal=list(order=c(0,0,0),period=NAxreg=NULL,include.mean=TRUE,transform.pars=TRUE,fixed=NULL,init=NULL,method=c("CSS-ML","ML","CSS"),n.cond,optim.control=list(),kappa CHAPTER3.UNIVARIATETIMESERIESMODELS111SeethemanualsofRfordetailsaboutthiscommend.Example2.9:Weillustratebyfittingthemonthlybirthseriesfrom1948-1979showninFigure3.7.TheperiodencompassestheFirstdifferenceBirths250300350400−40−200204001002003000100200300ARIMA(0,1,0)X(0,1,0)_{12}ARIMA(0,1,1)X(0,1,1)_{12}−40−2002040−40−20020400501002003000100200300Figure3.7:Numberoflivebirths1948(1)−1979(1)andresidualsfrommodelswithafirstdifference,afirstdifferenceandaseasonaldifferenceoforder12andafittedARIMA(0,1,1)×(0,1,1)12model.boomthatfollowedtheSecondWorldWarandthereistheexpectedrisewhichpersistsforabout13yearsfollowedbyadeclinetoaround1974.Theseriesappearstohavelong-termswings,withseasonaleffectssuperimposed.Thelong-termswingsindicatepossiblenon-stationarityandweverifythatthisisthecasebycheckingtheACFandPACFshowninthetoppanelofFigure3.8.NotethatbyProp-erty2.1,slowdecayoftheACFindicatesnon-stationarityandwerespondbytakingafirstdifference.TheresultsshowninthesecondpanelofFigure2.5indicatethatthefirstdifferencehaseliminatedthestronglowfrequencyswing.TheACF,showninthesecond CHAPTER3.UNIVARIATETIMESERIESMODELS112ACFPACFdata−0.500.5102030405060−0.500.5102030405060ARIMA(0,1,0)−0.500.5102030405060−0.500.5102030405060ARIMA(0,1,0)X(0,1,0)_{12}−0.500.5102030405060−0.500.5102030405060ARIMA(0,1,0)X(0,1,1)_{12}−0.500.5102030405060−0.500.5102030405060ARIMA(0,1,1)X(0,1,1)_{12}−0.500.5102030405060−0.500.5102030405060Figure3.8:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthebirthseries(toptwopanels),thefirstdifference(secondtwopanels)anARIMA(0,1,0)×(0,1,1)12model(thirdtwopanels)andanARIMA(0,1,1)×(0,1,1)12model(lasttwopanels). CHAPTER3.UNIVARIATETIMESERIESMODELS113panelfromthetopinFigure3.8showspeaksat12,24,36,48,···,withnowdecay.Thisbehaviorimpliesseasonalnon-stationarity,byProperty2.1’above,withs=12.AseasonaldifferenceofthefirstdifferencegeneratesanACFandPACFinFigure3.8thatweexpectforstationaryseries.TakingtheseasonaldifferenceofthefirstdifferencegivesaseriesthatlooksstationaryandhasanACFwithpeaksat1and12andaPACFwithasubstantialpeakat12andlesserpeaksat12,24,···.Thissuggeststryingeitherafirstordermovingaverageterm,byProperty2.3,orafirstorderseasonalmovingaveragetermwiths=12,byProperty2.3’above.Wechoosetoeliminatethelargestpeakfirstbyapplyingafirst-orderseasonalmovingaveragemodelwiths=12.TheACFandPACFoftheresidualseriesfromthismodel,i.e.fromARIMA(0,1,0)×(0,1,1)12,writtenas(1−L)(1−L12)x=(1−ΘL12)w,isshowninthefourthpanelt1tfromthetopinFigure3.8.Wenotethatthepeakatlagoneisstillthere,withattendingexponentialdecayinthePACF.Thiscanbeeliminatedbyfittingafirst-ordermovingaveragetermandweconsiderthemodelARIMA(0,1,1)×(0,1,1)12,writtenas1212(1−L)(1−L)xt=(1−θ1L)(1−Θ1L)wt.TheACFoftheresidualsfromthismodelarerelativelywellbehavedwithanumberofpeakseithernearorexceedingthe95%testofnocorrelation.FittingthisfinalARIMA(0,1,1)×(0,1,1)12modelleadstothemodel1212(1−L)(1−L)xt=(1−0.4896L)(1−0.6844L)wtwithAICC=4.95,R2=0.98042=0.961,andthep-valuesare(0.000,0.000).TheARIMAsearchleadstothemodel12212(1−L)(1−L)xt=(1−0.4088L−0.1645L)(1−0.6990L)wt, CHAPTER3.UNIVARIATETIMESERIESMODELS114yieldingAICC=4.92andR2=0.9812=0.962,slightlybetterthantheARIMA(0,1,1)×(0,1,1)12model.Evaluatingtheselattermod-elsleadstotheconclusionthattheextraparametersdonotaddapracticallysubstantialamounttothepredictability.Themodelisexpandedasxt=xt−1+xt−12−xt−13+wt−θ1wt−1−Θ1wt−12+θ1Θ1wt−13.Theforecastistxt+1=xt+xt−11−xt−12−θ1wt−Θ1wt−11+θ1Θ1wt−12ttxt+2=xt+1+xt−10−xt−11−Θ1wt−10+θ1Θ1wt−11.Continuinginthesamemanner,weobtainttxt+12=xt+11+xt−xt−1−Θ1wt+θ1Θ1wt−1forthe12monthforecast.Example2.10:Figure3.9showstheautocorrelationfunctionofthelog-transformedJ&JearningsseriesthatisplottedinFigure2.8andwenotetheslowdecayindicatingthenonstationaritywhichhasalreadybeenobviousintheChapter2discussion.WemayalsocomparetheACFwiththatofarandomwalk,showninFigure3.2,andnotetheclosesimilarity.Thepartialautocorrelationfunctionisveryhighatlagonewhich,underordinarycircumstances,wouldindicateafirstorderautoregressiveAR(1)model,exceptthat,inthiscase,thevalueisclosetounity,indicatingarootcloseto1ontheunitcircle.Theonlyquestionwouldbewhetherdifferencingordetrendingisthebettertransformationtostationarity.FollowingintheBox-Jenkinstradition,differencingleadstotheACFandPACFshowninthesecondpanelandnosimplestructureisapparent.Toforceanextstep,weinterpretthepeaksat4,8,12,16,···,as CHAPTER3.UNIVARIATETIMESERIESMODELS115ACFPACFlog(J&J)−0.50.000.51.051015202530−0.500.00.51.051015202530FirstDifference−0.50.000.51.051015202530−0.500.00.51.051015202530ARIMA(0,1,0)X(1,0,0,)_4−0.50.000.51.051015202530−0.500.00.51.051015202530ARIMA(0,1,1)X(1,0,0,)_4−0.50.000.51.051015202530−0.500.00.51.051015202530Figure3.9:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthelogJ&Jearningsseries(toptwopanels),thefirstdifference(secondtwopanels),ARIMA(0,1,0)×(1,0,0)4model(thirdtwopanels),andARIMA(0,1,1)×(1,0,0)4model(lasttwopanels).contributingtoapossibleseasonalautoregressiveterm,leadingtoapossibleARIMA(0,1,0)×(1,0,0)4andwesimplyfitthismodelandlookattheACFandPACFoftheresiduals,showninthethirdtwopanels.Thefitimprovessomewhat,withsignificantpeaksstillremainingatlag1inboththeACFandPACF.ThepeakintheACFseemsmoreisolatedandthereremainssomeexponentiallydecaying CHAPTER3.UNIVARIATETIMESERIESMODELS116behaviorinthePACF,sowetryamodelwithafirst-ordermovingaverage.ThebottomtwopanelsshowtheACFandPACFoftheresultingARIMA(0,1,1)×(1,0,0)4andwenoteonlyrelativelyminorexcursionsaboveandbelowthe95%intervalsundertheassumptionthatthetheoreticalACFiswhitenoise.Thefinalmodelsuggestedis(yt=logxt)4(1−Φ1L)(1−L)yt=(1−θ1L)wt,(3.31)cb2whereΦ1=0.820(0.058),θ1=0.508(0.098),andσcw=0.0086.Themodelcanbewritteninforecastformasyt=yt−1+Φ1(yt−4−yt−5)+wt−θ1wt−1.TheresidualplotoftheaboveisplottedintheleftbottompanelofFigure3.10.Toforecasttheoriginalseriesfor,say4quarters,weACFPACFARIMA(0,1,1)X(0,1,1,)_4−0.50.00.51.0−0.50.00.51.0051015202530051015202530ResidualPlotResidualPlotARIMA(0,1,1)X(1,0,0,)_4ARIMA(0,1,1)X(0,1,1,)_4−0.10.00.10.2−0.2−0.10.00.10.2020406080020406080Figure3.10:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forARIMA(0,1,1)×(0,1,1)4model(toptwopanels)andtheresidualplotsofARIMA(0,1,1)×(1,0,0)4(leftbottompanel)andARIMA(0,1,1)×(0,1,1)4model(rightbottompanel). CHAPTER3.UNIVARIATETIMESERIESMODELS117computetheforecastlimitsforyt=logxtandthenexponentiate,i.e.xt=exp(yt).t+ht+hBasedonthetheexactlikelihoodmethod,Tsay(2005)consideredthefollowingseasonalARIMA(0,1,1)×(0,1,1)4model44(1−L)(1−L)yt=(1−0.678L)(1−0.314L)wt,(3.32)withσc2=0.089,wherestandarderrorsofthetwoMAparametersware0.080and0.101,respectively.TheLjung-BoxstatisticsoftheresidualsshowQ(12)=10.0withp-value0.44.Themodelappearstobeadequate.TheACFandPACFoftheARIMA(0,1,1)×(0,1,1)4modelaregiveninthetoptwopanelsofFigure3.10andtheresidualplotisdisplayedintherightbottompanelofFigure3.10.BasedonthecomparisonofACFandPACFoftwomodel(3.31)and(3.32)[thelasttwopanelsofFigure3.9andthetoptwopanelsinFigure3.10],itseemsthatARIMA(0,1,1)×(0,1,1)4modelin(3.32)mightperformbetterthanARIMA(0,1,1)×(1,0,0)4modelin(3.31).Toillustratetheforecastingperformanceoftheseasonalmodelin(3.32),were-estimatethemodelusingthefirst76observationsandreservethelasteightdatapointsforforecastingevaluation.Wecom-pute1-stepto8-stepaheadforecastsandtheirstandarderrorsofthefittedmodelattheforecastorigint=76.Ananti-logtransformationistakentoobtainforecastsofearningpershareusingtherelationshipbetweennormalandlog-normaldistributions.Figure2.15inTsay(2005,p.77)showstheforecastperformanceofthemodel,wheretheobserveddataareinsolidline,pointforecastsareshownbydots,andthedashedlinesshow95%intervalforecasts.Theforecastsshowastrongseasonalpatternandareclosetotheobserveddata.Formorecomparisonsforforecastsusingdifferentmodelsincludingsemipara-metricandnonparametricmodels,thereaderisreferredtothebook CHAPTER3.UNIVARIATETIMESERIESMODELS118byShumway(1988),andShumwayandStoffer(2000)andthepapersbyBurmanandShummay(1998)andCaiandChen(2006).Whentheseasonalpatternofatimeseriesisstableovertime(e.g.,closetoadeterministicfunction),dummyvariablesmaybeusedtohandletheseasonality.Thisapproachistakenbysomeanalysts.However,deterministicseasonalityisaspecialcaseofthemultiplica-tiveseasonalmodeldiscussedbefore.Specifically,ifΘ1=1,thenmodelcontainsadeterministicseasonalcomponent.Consequently,thesameforecastsareobtainedbyusingeitherdummyvariablesoramultiplicativeseasonalmodelwhentheseasonalpatternisdeter-ministic.Yetuseofdummyvariablescanleadtoinferiorforecastsiftheseasonalpatternisnotdeterministic.Inpractice,werecommendthattheexactlikelihoodmethodshouldbeusedtoestimateamul-tiplicativeseasonalmodel,especiallywhenthesamplesizeissmallorwhenthereisthepossibilityofhavingadeterministicseasonalcomponent.Example2.11:Todeterminedeterministicbehavior,considerthemonthlysimplereturnoftheCRSPDecile1indexfromJanuary1960toDecember2003for528observations.TheseriesisshowninthelefttoppanelofFigure3.11andthetimeseriesdoesnotshowanyclearpatternofseasonality.However,thesampleACfofthereturnseriesshownintheleftbottompanelofFigure3.11containssignificantlagsat12,24,and36aswellaslag1.IfseasonalAIMAmodelsareentertained,amodelinform1212(1−φ1L)(1−Φ1L)xt=α+(1−Θ1L)wtisidentified,wherextisthemonthlysimplereturn.Usingthecon-ditionallikelihood,thefittedmodelis1212(1−0.25L)(1−0.99L)xt=0.0004+(1−0.92L)wt CHAPTER3.UNIVARIATETIMESERIESMODELS119SimpleReturnsJanuary−adjustedreturns−0.20.00.20.40100200300400500−0.30−0.10.10.20.30.4100200300400500ACFACF−0.50.00.51.0−0.50.00.51.0010203040010203040Figure3.11:MonthlysimplereturnofCRSPDecile1indexfromJanuary1960toDecember2003:Timeseriesplotofthesimplereturn(lefttoppanel),timeseriesplotofthesimplereturnafteradjustingforJanuaryeffect(righttoppanel),theACFofthesimplereturn(leftbottompanel),andtheACFoftheadjustedsimplereturn.withσw=0.071.TheMAcoefficientisclosetounity,indicatingthatthefittedmodelisclosetobeingnon-invertible.Iftheexactlikelihoodmethodisused,wehave1212(1−0.264L)(1−0.996L)xt=0.0002+(1−0.999L)wtwithσw=0.067.CancellationbetweenseasonalARandMAfactorsisclearly.Thishighlightstheusefulnessofusingtheexactlikelihoodmethod,andtheestimationresultsuggeststhattheseasonalbehaviormightbedeterministic.Tofurtherconfirmthisassertion,wedefinethedummyvariableforJanuary,thatis1iftisJanuary,Jt=0otherwise,andemploythesimplelinearregressionxt=β0+β1Jt+et. CHAPTER3.UNIVARIATETIMESERIESMODELS120TherightpanelsofFigure3.11showthetimeseriesplotofandtheACFoftheresidualseriesofthepriorsimplelinearregression.FromtheACF,therearenosignificantserialcorrelationatanymultiplesof12,suggestingthattheseasonalpatternhasbeensuccessfullyre-movedbytheJanuarydummyvariable.Consequently,theseasonalbehaviorinthemonthlysimplereturnofDecile1isduetotheJan-uaryeffect.3.9RegressionModelsWithCorrelatedErrorsInmanyapplications,therelationshipbetweentwotimeseriesisofmajorinterest.Themarketmodelinfinanceisanexamplethatrelatesthereturnofanindividualstocktothereturnofamarketindex.Thetermstructureofinterestratesisanotherexampleinwhichthetimeevolutionoftherelationshipbetweeninterestrateswithdifferentmaturitiesisinvestigated.Theseexamplesleadtotheconsiderationofalinearregressionintheformyt=β1+β2xt+et,whereytandxtaretwotimeseriesandetdenotestheerrorterm.Theleastsquares(LS)methodisoftenusedtoestimatetheabovemodel.If{et}isawhitenoiseseries,thentheLSmethodproducesconsistentestimates.Inpractice,however,itiscommontoseethattheerrortermetisseriallycorrelated.Inthiscase,wehavearegressionmodelwithtimeserieserrors,andtheLSestimatesofβ1andβ2maynotbeconsistentandefficient.Regressionmodelwithtimeserieserrorsiswidelyapplicableineconomicsandfinance,butitisoneofthemostcommonlymisusedeconometricmodelsbecausetheserialdependenceinetisoftenover-looked.Itpaystostudythemodelcarefully.Thestandardmethod CHAPTER3.UNIVARIATETIMESERIESMODELS121fordealingwithcorrelatederrorsetintheregressionmodel′yt=βzt+etistotrytotransformtheerrorsetintouncorrelatedonesandthenapplythestandardleastsquaresapproachtothetransformedob-servations.Forexample,letPbeann×nmatrixthattransformsthevectore=(e,···,e)′intoasetofindependentidentically1ndistributedvariableswithvarianceσ2.Then,transformthematrixversion(3.4)toPy=PZβ+Peandproceedasbefore.Ofcourse,themajorproblemisdecidingonwhattochooseforPbutinthetimeseriescase,happily,thereisareasonablesolution,basedagainontimeseriesARMAmodels.Supposethatwecanfind,forexample,areasonableARMAmodelfortheresiduals,say,forexample,theARMA(p,0,0)modelpXet=φket+wt,k=1whichdefinesalineartransformationofthecorrelatedettoase-quenceofuncorrelatedwt.Wecanignoretheproblemsnearthebeginningoftheseriesbystartingatt=p.IntheARMAnotation,usingtheback-shiftoperatorB,wemaywriteφ(L)et=wt,(3.33)wherepXkφ(L)=1−φkL(3.34)k=1andapplyingtheoperatortobothsidesof(3.2)leadstothemodelφ(L)yt=φ(L)zt+wt,(3.35) CHAPTER3.UNIVARIATETIMESERIESMODELS122wherethe{wt}’snowsatisfytheindependenceassumption.Doingordinaryleastsquaresonthetransformedmodelisthesameasdo-ingweightedleastsquaresontheuntransformedmodel.Theonlyproblemisthatwedonotknowthevaluesofthecoefficientsφk(1≤k≤p)inthetransformation(3.34).However,ifweknewtheresidualset,itwouldbeeasytoestimatethecoefficients,since(3.34)canbewrittenintheform′et=φet−1+wt,(3.36)whichisexactlytheusualregressionmodel(3.2)withφ=(φ,···,φ)′1preplacingβande=(e,e,···,e)′replacingz.Thet−1t−1t−2t−ptabovecommentssuggestageneralapproachknownastheCochran-Orcuttprocedure(CochraneandOrcutt,1949)fordealingwiththeproblemofcorrelatederrorsinthetimeseriescontext.1.Beginbyfittingtheoriginalregressionmodel(3.2)byleastsquares,c′obtainingβandtheresidualset=yt−βzt.2.FitanARMAtotheestimatedresiduals,sayφ(L)et=θ(L)wt.3.ApplytheARMAtransformationfoundtobothsidesoftheregressionequation(3.2)toobtainφ(L)′φ(L)yt=βzt+wt.θ(L)θ(L)4.Runanordinaryleastsquaresonthetransformedvaluestoob-tainthenewβ.5.Returnto2.ifdesired.Often,oneiterationisenoughtodeveloptheestimatorsunderarea-sonablecorrelationstructure.Ingeneral,theCochran-Orcuttproce-dureconvergestothemaximumlikelihoodorweightedleastsquaresestimators. CHAPTER3.UNIVARIATETIMESERIESMODELS123Example2.12:WemightconsideranalternativeapproachtotreatingtheJohnsonandJohnsonearningsseries,assumingthatyt=log(xt)=β1+β2t+et.Inordertoanalyzethedatawiththisap-cproach,firstwefitthemodelabove,obtainingβ1=−0.6678(0.0349)cccandβ2=0.0417(0.0071).Thecomputedresidualset=yt−β1−β2tcanbecomputedeasily,theACFandPACFareshowninthetoptwopanelsofFigure3.12.NotethattheACFandPACFsuggestACFPACFdetrended−0.50.00.51.0−0.50.00.51.0051015202530051015202530ARIMA(1,0,0,)_4−0.50.00.51.0−0.50.00.51.0051015202530051015202530Figure3.12:Autocorrelationfunctions(ACF)andpartialautocorrelationfunctions(PACF)forthedetrendedlogJ&Jearningsseries(toptwopanels)andthefittedARIMA(0,0,0)×(1,0,0)4residuals.thataseasonalARserieswillfitwellandweshowtheACFandPACFoftheseresidualsinthebottompanelsofFigure3.12.TheseasonalARmodelisoftheformet=Φ1et−4+wtandweobtainc2Φ1=0.7614(0.0639),withσcw=0.00779.Usingthesevalues,wetransformyttocccyt−Φ1yt−4=β1(1−Φ1)+β2[t−Φ1(t−4)]+wtcusingtheestimatedvalueΦ1=0.7614.Withthistransformedre- CHAPTER3.UNIVARIATETIMESERIESMODELS124cgression,weobtainthenewestimatorsβ1=−0.7488(0.1105)andcβ2=0.0424(0.0018).Thenewestimatorhastheadvantageofbeingunbiasedandhavingasmallergeneralizedvariance.Toforecast,weconsidertheoriginalmodel,withthenewlyesti-cctmatedβ1andβ2.Weobtaintheapproximateforecastforyt+h=cctβ1+β2(t+h)+et+hforthelogtransformedseries,alongwithupperandlowerlimitsdependingontheestimatedvariancethatonlyin-corporatesthepredictionvarianceofet,consideringthetrendandt+hseasonalautoregressiveparametersasfixed.Thenarrowerupperandlowerlimits(Thefigureisnotpresentedhere)aremainlyarefectionofaslightlybetterfittotheresidualsandtheabilityofthetrendmodeltotakecareofthenonstationarity.Example2:13:WeconsidertherelationshipbetweentwoU.S.weeklyinterestrateseries:xt:the1-yearTreasuryconstantmaturityrateandyt:the3-yearTreasuryconstantmaturityrate.Bothserieshave1967observationsfromJanuary5,1962toSeptember10,1999andaremeasuredinpercentages.TheseriesareobtainedfromtheFederalReserveBankofStLouis.Figure3.13showsthetimeplotsofthetwointerestrateswithsolidlinedenotingthe1-yearrateanddashedlineforthe3-yearrate.TheleftpanelofFigure3.14plotsytversusxt,indicatingthat,asexpected,thetwointerestratesarehighlycorrelated.Anaivewaytodescribetherelationshipbetweenthetwointerestratesistousethesimplemodel,ModelI:yt=β1+β2xt+et.Thisresultsinafittedmodely=0.911+0.924x+e,withσc2=0.538andttteR2=95.8%,wherethestandarderrorsofthetwocoefficientsare0.032and0.004,respectively.Thissimplemodel(ModelI)confirmsthehighcorrelationbetweenthetwointerestrates.However,the CHAPTER3.UNIVARIATETIMESERIESMODELS125468101214161970198019902000Figure3.13:TimeplotsofU.S.weeklyinterestrates(inpercentages)fromJanuary5,1962toSeptember10,1999.Thesolidline(black)istheTreasury1-yearconstantmaturityrateandthedashedlinetheTreasury3-yearconstantmaturityrate(red).ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo4oo6ooooooooooooooooooooooooooooooooooooooo810121416−1.0−0.5oo0.00.51.01.5oooooooooooooooooooooooooooooooooooooooooooooooooooooo46810121416−1.5−0.50.51.01.5Figure3.14:ScatterplotsofU.S.weeklyinterestratesfromJanuary5,1962toSeptember10,1999:theleftpanelis3-yearrateversus1-yearrate,andtherightpanelischangesin3-yearrateversuschangesin1-yearrate.modelisseriouslyinadequateasshownbyFigure3.15,whichgivesthetimeplotandACFofitsresiduals.Inparticular,thesampleACFoftheresidualsishighlysignificantanddecaysslowly,showingthepatternofaunitrootnonstationarytimeseries2.Thebehavioroftheresidualssuggeststhatmarkeddifferencesexistbetweenthetwointerestrates.Usingthemoderneconometricterminology,ifone2Wewilldiscussindetailonhowtodounitroottestlater CHAPTER3.UNIVARIATETIMESERIESMODELS126−1.5−1.0−0.50.00.51.0−0.50.00.51.01970198019902000051015202530Figure3.15:ResidualseriesoflinearregressionModelIfortwoU.S.weeklyinterestrates:theleftpanelistimeplotandtherightpanelisACF.assumesthatthetwointerestrateseriesareunitrootnonstationary,thenthebehavioroftheresidualsindicatesthatthetwointerestratesarenotco-integrated;seelaterchaptersfordiscussionofunitrootandco-integration.Inotherwords,thedatafailtosupportthehypothesisthatthereexistsalong-termequilibriumbetweenthetwointerestrates.Insomesense,thisisnotsurprisingbecausethepatternof“invertedyieldcurve”didoccurduringthedataspan.Bytheinvertedyieldcurve,wemeanthesituationunderwhichinterestratesareinverselyrelatedtotheirtimetomaturities.Theunitrootbehaviorofbothinterestratesandtheresidualsleadstotheconsiderationofthechangeseriesofinterestrates.Let∆xt=yt−yt−1=(1−L)xtbechangesinthe1-yearinterestrateand∆yt=yt−yt−1=(1−L)ytdenotechangesinthe3-yearinterestrate.Considerthelinearregression,ModelII:∆yt=β1+β2∆xt+et.Figure3.16showstimeplotsofthetwochangeseries,whereastherightpanelofFigure3.14providesascatterplotbetweenthem.Thechangeseriesremainhighlycorrelatedwithafittedlinearregressionmodelgivenby∆yt=0.0002+0.7811∆xt+etwithσc2=0.0682andR2=84.8%.Thestandarderrorsofthetwoe CHAPTER3.UNIVARIATETIMESERIESMODELS127−1.5−1.0−0.50.00.51.01.51970198019902000Figure3.16:TimeplotsofthechangeseriesofU.S.weeklyinterestratesfromJanuary12,1962toSeptember10,1999:changesintheTreasury1-yearconstantmaturityrateareindenotedbyblacksolidline,andchangesintheTreasury3-yearconstantmaturityrateareindicatedbyreddashedline.coefficientsare0.0015and0.0075,respectively.Thismodelfurtherconfirmsthestronglineardependencebetweeninterestrates.ThetwotoppanelsofFigure3.17showthetimeplot(left)andsampleACF(right)oftheresiduals(ModelII).Onceagain,theACFshows−0.50.00.51.0−0.40−0.20.00.25000.4100015002000051015202530−0.50.00.51.0−0.40−0.20.00.25000.4100015002000051015202530Figure3.17:Residualseriesofthelinearregressionmodels:ModelII(top)andModelIII(bottom)fortwochangeseriesofU.S.weeklyinterestrates:timeplot(left)andACF(right). CHAPTER3.UNIVARIATETIMESERIESMODELS128somesignificantserialcorrelationintheresiduals,butthemagnitudeofthecorrelationismuchsmaller.Thisweakserialdependenceintheresidualscanbemodeledbyusingthesimpletimeseriesmodelsdiscussedintheprevioussections,andwehavealinearregressionwithtimeserieserrors.Themainobjectiveofthissectionistodiscussasimpleapproachforbuildingalinearregressionmodelwithtimeserieserrors.Theapproachisstraightforward.Weemployasimpletimeseriesmodeldiscussedinthischapterfortheresidualseriesandestimatethewholemodeljointly.Forillustration,considerthesimplelinearregressioninModelII.Becauseresidualsofthemodelareseriallycorrelated,weidentifyasimpleARMAmodelfortheresiduals.FromthesampleACFoftheresidualsshownintherighttoppanelofFigure3.17,wespecifyanMA(1)modelfortheresidualsandmodifythelinearregressionmodelto(ModelIII):∆yt=β1+β2∆xt+etandet=wt−θ1wt−1,where{wt}isassumedtobeawhitenoiseseries.Inotherwords,wesimplyuseanMA(1)model,withouttheconstantterm,tocapturetheserialdependenceintheerrortermofModelII.ThetwobottompanelsofFigure3.17showthetimeplot(left)andsampleACF(right)oftheresiduals(ModelIII).Theresultingmodelisasimpleexampleoflinearregressionwithtimeserieserrors.Inpractice,moreelaboratedtimeseriesmodelscanbeaddedtoalinearregressionequationtoformageneralregressionmodelwithtimeserieserrors.Estimatingaregressionmodelwithtimeserieserrorswasnoteasybeforetheadventofmoderncomputers.SpecialmethodssuchastheCochrane-Orcuttestimatorhavebeenproposedtohandletheserialdependenceintheresiduals.Bynow,theestimationisaseasyasthatofothertimeseriesmodels.Ifthetimeseriesmodelusedis CHAPTER3.UNIVARIATETIMESERIESMODELS129stationaryandinvertible,thenonecanestimatethemodeljointlyviathemaximumlikelihoodmethodorconditionalmaximumlikelihoodmethod.ThisistheapproachwetakebyusingthepackageRwiththecommandarima().FortheU.S.weeklyinterestratedata,thefittedversionofModelIIis∆yt=0.0002+0.7824∆xt+etande=w+0.2115wwithσc2=0.0668andR2=85.4%.Thettt−1wstandarderrorsoftheparametersare0.0018,0.0077,and0.0221,respectively.Themodelnolongerhasasignificantlag-1residualACF,eventhoughsomeminorresidualserialcorrelationsremainatlags4and6.TheincrementalimprovementofaddingadditionalMAparametersatlags4and6totheresidualequationissmallandtheresultisnotreportedhere.Comparingtheabovethreemodels,wemakethefollowingob-servations.First,thehighR2andcoefficient0.924ofModleIaremisleadingbecausetheresidualsofthemodelshowstrongserialcor-relations.Second,forthechangeseries,R2andthecoefficientof∆xtofModelIIandModelIIIareclose.Inthisparticularin-stance,addingtheMA(1)modeltothechangeseriesonlyprovidesamarginalimprovement.Thisisnotsurprisingbecausetheesti-matedMAcoefficientissmallnumerically,eventhoughitisstatis-ticallyhighlysignificant.Third,theanalysisdemonstratesthatitisimportanttocheckresidualserialdependenceinlinearregressionanalysis.BecausetheconstanttermofModelIIIisinsignificant,themodelshowsthatthetwoweeklyinterestrateseriesarerelatedasyt=yt−1+0.782(xt−xt−1)+wt+0.212wt−1.Theinterestratesareconcurrentlyandseriallycorrelated.Finally,weoutlineageneralprocedureforanalyzinglinearre-gressionmodelswithtimeserieserrors:First,fitthelinearregres-sionmodelandcheckserialcorrelationsoftheresiduals.Second, CHAPTER3.UNIVARIATETIMESERIESMODELS130iftheresidualseriesisunit-rootnonstationary,takethefirstdiffer-enceofboththedependentandexplanatoryvariables.Gotostep1.Iftheresidualseriesappearstobestationary,identifyanARMAmodelfortheresidualsandmodifythelinearregressionmodelac-cordingly.Third,performajointestimationviathemaximumlike-lihoodmethodandcheckthefittedmodelforfurtherimprovement.Tochecktheserialcorrelationsofresiduals,werecommendthattheLjung-BoxstatisticsbeusedinsteadoftheDurbin-Watson(DW)statisticbecausethelatteronlyconsidersthelag-1serialcorrelation.Therearecasesinwhichresidualserialdependenceappearsathigherorderlags.Thisisparticularlysowhenthetimeseriesinvolvedexhibitssomeseasonalbehavior.Remark:ForaresidualseriesetwithTobservations,theDurbin-WatsonstatisticisXTXT22DW=(et−et−1)/et.t=2t=1StraightforwardcalculationshowsthatDW≈2(1−ρbe(1)),whereρe(1)isthelag-1ACFof{et}.3.10EstimationofCovarianceMatrixConsideragaintheregressionmodelin(3.2).Theremayexistsit-uationswhichtheerrorethasserialcorrelationsand/orconditionalheteroscedasticity,butthemainobjectiveoftheanalysisistomakeinferenceconcerningtheregressioncoefficientsβ.Whenethasserialcorrelations,wediscussedmethodsinExample2.12andExam-ple2.13abovetoovercomethisdifficulty.However,weassume CHAPTER3.UNIVARIATETIMESERIESMODELS131thatetfollowsanARIMAtypemodelandthisassumptionmightnotbealwayssatisfiedinsomeapplications.Here,weconsiderageneralsituationwithoutmakingthisassumption.Insituationsun-derwhichtheordinaryleastsquaresestimatesofthecoefficientsre-mainconsistent,methodsareavailabletoprovideconsistentestimateofthecovariancematrixofthecoefficients.Twosuchmethodsarewidelyusedineconomicsandfinance.Thefirstmethodiscalledheteroscedasticityconsistent(HC)estimator;seeEicker(1967)andWhite(1980).Thesecondmethodiscalledheteroscedasticityandautocorrelationconsistent(HAC)estimator;seeNeweyandWest(1987).Toeaseindiscussion,weshallre-writetheregressionmodelas′yt=βxt+et,whereyisthedependentvariable,x=(x,···,x)′isap-tt1tptdimensionalvectorofexplanatoryvariablesincludingconstantandlaggedvariables,andβ=(β,···,β)′istheparametervector.1pTheLSestimateofβisgivenbyn−1nXXβc=xx′xy,ttttt=1t=1andtheassociatedcovariancematrixhastheso-called“sandwich”formasXn−1Xn−1Xn−1Σ=Cov(βc)=xx′Cxx′ifet=isiidσ2xx′,βttttettt=1t=1t=1whereCiscalledthe“meat”givenbyXnC=Varetxt,t=1σ2isthevarianceofeandisestimatedbythevarianceofresidualsetoftheregression.Inthepresenceofserialcorrelationsorconditional CHAPTER3.UNIVARIATETIMESERIESMODELS132heteroscedasticity,thepriorcovariancematrixestimatorisinconsis-ctent,oftenresultingininflatingthet-ratiosofβ.TheestimatorofWhite(1980)isbasedonfollowing:Xn−1Xn−1Σc=xx′Ccxx′,β,hctthcttt=1t=1c′wherewithebt=yt−βxtbeingtheresidualattimet,nXnc2′Chc=ebtxtxt.n−pt=1TheestimatorofNeweyandWest(1987)isXn−1Xn−1Σc=xx′Ccxx′,β,hactthacttt=1t=1cwhereChacisgivenbyXnXlXnc2′′′Chac=ebtxtxt+wjxtebtebt−jxt−j+xt−jebt−jebtxtt=1j=1t=j+1withlisatruncationparameterandwjisweightfunctionsuchastheBarlettweightfunctiondefinedbywj=1−j/(l+1).Otherweightfunctioncanalsoused.NeweyandWest(1987)suggestedchoosingltobetheintegerpartof4(n/100)2/9.ThisestimatoressentiallyusesanonparametricmethodtoestimatethecovariancePnmatrixoft=1etxtandaclassofkernel-basedheteroskedasticityandautocorrelationconsistent(HAC)covariancematrixestimatorswasintroducedbyAndrews(1991).Example2.14:(ContinuationofExample2.13)Forillustration,weconsiderthefirstdifferencedinterestrateseriesinModelIIinExample2.13.Thet-ratioofthecoefficientof∆xtis104.63ifbothserialcorrelationandconditionalheteroscedasticityinresiduals CHAPTER3.UNIVARIATETIMESERIESMODELS133areignored;itbecomes46.73whentheHCestimatorisused,anditreducesto40.08whentheHACestimatorisemployed.TouseHCorHACestimator,wecanusethepackagesandwichinRandthecommandsarevcovHC()orvcovHAC().3.11LongMemoryModelsWehavediscussedthatforastationarytimeseriestheACFdecaysexponentiallytozeroaslagincreases.Yetforaunitrootnonsta-tionarytimeseries,itcanbeshownthatthesampleACFconvergesto1forallfixedlagsasthesamplesizeincreases;seeChanandWei(1988)andTiaoandTsay(1983).ThereexistsometimeserieswhoseACFdecaysslowlytozeroatapolynomialrateasthelagincreases.Theseprocessesarereferredtoaslongmemoryorlongrangedependenttimeseries.Onesuchanexampleisthefractionallydifferencedprocessdefinedbyd(1−L)xt=wt,|d|<0.5,(3.37)where{wt}isawhitenoiseseriesanddiscalledthelongmem-oryparameter.Propertiesofmodel(3.37)havebeenwidelystudiedintheliterature(e.g.,Beran,1994).Wesummarizesomeofthesepropertiesbelow.1.Ifd<0.5,thenxtisaweaklystationaryprocessandhastheinfiniteMArepresentationX∞k+d−1xt=wt+ψkwt−kwithψk=d(d+1)···(d+k−1)/k!=k=1k2.Ifd>−0.5,thenxtisinvertibleandhastheinfiniteARrepre-sentation.X∞k−dxt=wt+ψkwt−kwithψk=(0−d)(1−d)···(k−1−d)/k!=k=1k CHAPTER3.UNIVARIATETIMESERIESMODELS1343.For|d|<0.5,theACFofxtisd(1+d)···(h−1+d)ρx(h)=,h≥1.(1−d)(2−d)···(h−d)Inparticular,ρx(1)=d/(1−d)andash→∞,(−d)!2d−1ρx(h)≈h.(d−1)!4.For|d|<0.5,thePACFofxtisφh,h=d/(h−d)forh≥1.5.For|d|<0.5,thespectraldensityfunctionfx(·)ofxt,whichistheFouriertransformoftheACFγx(h)ofxt,thatis1X∞fx(ν)=γx(h)exp(−ihν)2πh=−∞√forν∈[−π,π],wherei=−1,satisfies−2dfx(ν)∼νasν→0,(3.38)whereν∈[0,π]denotesthefrequency.SeeChapter6ofHamil-ton(1994)fordetailsaboutthespectralanalysis.OfparticularinteresthereisthebehaviorofACFofxtwhend<0.5.Thepropertysaysthatρ(h)∼h2d−1,whichdecaysatapolyno-xmial,insteadofexponentialrate.Forthisreason,suchanxtprocessiscalledalong-memorytimeseries.Aspecialcharacteristicofthespectraldensityfunctionin(3.38)isthatthespectrumdivergestoinfinityasν→0.However,thespectraldensityfunctionofasta-tionaryARMAprocessisboundedforallν∈[−π,π].Earlierweusedthebinomialtheoremfornon-integerpowersX∞ddkk(1−L)=(−1)L.k=0k CHAPTER3.UNIVARIATETIMESERIESMODELS135Ifthefractionallydifferencedseries(1−L)dxfollowsanARMA(p,q)tmodel,thenxtiscalledanfractionallydifferencedautoregressivemovingaverage(ARFIMA(p,d,q))process,whichisageneralizedARIMAmodelbyallowingfornon-integerd.Inpractice,ifthesam-pleACFofatimeseriesisnotlargeinmagnitude,butdecaysslowly,thentheseriesmayhavelongmemory.Formorediscussions,werefertothebookbyBeran(1994).Forthepurefractionallydiffer-encedmodelin(3.37),onecanestimatedusingeitheramaximumlikelihoodmethodinthetimedomainortheWhittlelikelihoodoraregressionmethodwithloggedperiodogramatthelowerfrequen-ciesinthefrequencydomain.Finally,long-memorymodelshaveattractedsomeattentioninthefinanceliteratureinpartbecauseoftheworkonfractionalBrownianmotioninthecontinuoustimemodels.ACFforvalue−weightedindexACFforequal−weightedindex−0.10.00.10.20.30.4−0.10.00.10.20.30.401002003004000100200300400LogSpectralDensityofEWLogSpectralDensityofVW−12−11−10−9−8−7−6−13−11−9−8−7−60.000.020.040.060.000.020.040.06Figure3.18:SampleautocorrelationfunctionoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(lefttoppanel)andequal-weighted(righttoppanel)indexes.ThelogspectraldensityoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(leftbottompanel)andequal-weighted(rightbottompanel)indexes. CHAPTER3.UNIVARIATETIMESERIESMODELS136Example2.15:Asanillustration,Figure3.18showthesampleACFsoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(lefttoppanel)andequal-weighted(righttoppanel)indexesfromJuly3,1962toDecember31,1997.TheACFsarerel-ativelysmallinmagnitude,butdecayveryslowly;theyappeartobesignificantatthe5%levelevenafter300lags.FormoreinformationaboutthebehaviorofsampleACFofabsolutereturnseries,seeDing,Granger,andEngle(1993).Toestimatethelongmemoryparameterestimated,wecanusethepackagefracdiffinRandresultsarebd=0.1867fortheabsolutereturnsofthevalue-weightedindexandbd=0.2732fortheabsolutereturnsoftheequal-weightedindex.Tosupportourconclusionabove,weplotthelogspectraldensityoftheabsoluteseriesofdailysimplereturnsfortheCRSPvalue-weighted(leftbottompanel)andequal-weighted(rightbottompanel).Theyshowclearlythatbothlogspectraldensitiesdecaylikealogfunctionandtheysupportthespectraldensitiesbehaviorlike(3.38).3.12PeriodicityandBusinessCyclesLetusfirstrecallwhatwehaveobservedfromfromFigure2.1inChapter2.FromFigure2.1,wecanconcludethatbothseriesintendtoexhibitrepetitivebehavior,withregularlyrepeating(stochastic)cyclesorperiodicitythatareeasilyvisible.Thisperiodicbe-haviorisofinterestbecauseunderlyingprocessesofinterestmayberegularandtherateorfrequencyoftimeseriescharacterizingthebe-havioroftheunderlyingserieswouldhelptoidentifythem.OnecanalsoremarkthatthecyclesoftheSOIarerepeatingatafasterratethanthoseoftherecruitmentseries.Therecruitsseriesalsoshowsseveralkindsofoscillations,afasterfrequencythatseemstorepeataboutevery12monthsandaslowerfrequencythatseemstorepeat CHAPTER3.UNIVARIATETIMESERIESMODELS137aboutevery50months.Thestudyofthekindsofcyclesandtheirstrengthsarealsoveryimportant,particularlyinmacroeconomicstodeterminethebusinesscycles.Formorediscussions,werefertothebooksbyFranses(1996,1998)andGhyselsandOsborn(2001).AswementioninChapter2,onewaytoidentifythecyclesistocom-putethepowerspectrumwhichshowsthevarianceasafunctionofthefrequencyofoscillation.Forothermodelingmethodssuchasperiodicautoregressivemodel(PAR),PAR(p)modelingtechniques,werefertothebooksbyFranses(1996,1998)andGhyselsandOs-born(2001)fordetails.Next,weintroduceonemethodtodescribethecyclicalbehaviorusingtheACF.Asindicatedabove,thereexitscyclicalbehaviorfortherecruits.FromExample2.4,anAR(2)fitsthisseriesquitewell.There-fore,weconsidertheACFρx(h)ofastationaryAR(2)series,whichsatisfiesthesecondorderdifferenceequation2(1−φ1L−φ2L)ρx(h)=φ(L)ρx(h)=0forh≥2,ρx(0)=1,andρx(1)=φ1/(1−φ2).ThisdifferenceequationdeterminesthepropertiesoftheACFofastationaryAR(2)timeseries.Italsodeterminesthebehavioroftheforecastsofxt.Correspondingtothepriordifferenceequation,thereisasecondorderpolynomialequation21−φ1x−φ2x=0.Solutionsofthisequationareqφ±φ2+4φ112x=.−2φ2Inthetimeseriesliterature,theinverseoftwosolutionsarereferredasthecharacteristicrootsoftheAR(2)model.Denotethe CHAPTER3.UNIVARIATETIMESERIESMODELS138twosolutionsbyω1andω2.Ifbothωiarerealvalued,thenthesecondorderdifferenceequationofthemodelcanbefactoredas(1−ω1L)(1−ω2L)andtheAR(2)modelcanberegardedasanAR(1)modeloperatesontopofanotherAR(1)model.TheACFofxisthenamixtureoftwoexponentialdecays.Yetifφ2+4φ<0,t12thenω1andω2arecomplexnumbers(calledacomplexconjugatepair),andtheplotofACFofxtwouldshowapictureofdampingsineandcosinewaves;seethetoptwopanelsofFigure2.14fortheACFoftheSOIandrecruits.Inbusinessandeconomicappli-cations,complexcharacteristicrootsareimportant.Theygiverisetothebehaviorofbusinesscycles.Itisthencommonforeconomictimeseriesmodelstohavecomplexvaluedcharacteristicroots.ForanAR(2)modelwithapairofcomplexcharacteristicroots,theav-eragelengthofthestochasticcyclesis2πT0=√,cos−1[φ1/(2−φ2)]wherethecosineinverseisstatedinradians.Ifonewishesthecom-√plexsolutionsasa±bi,thenwehaveφ1=2a,φ2=−a2+b2,and2πT0=√,cos−1(a/a2+b2)√wherea2+b2istheabsolutevalueofa±bi.Toillustratetheaboveidea,Figure3.19showstheACFoffourstationaryAR(2)models.TherighttoppanelistheACFoftheAR(2)model(1−0.6L+0.4L2)x=w.Becauseφ2+4φ=1.0+tt124×(−0.7)=−1.8<0,thisparticularAR(2)modelcontainstwocomplexcharacteristicroots,andhenceitsACFexhibitsdampingsineandcosinewaves.TheotherthreeAR(2)modelshavereal-valuedcharacteristicroots.TheirACFsdecayexponentially. CHAPTER3.UNIVARIATETIMESERIESMODELS139(a)(b)−1.0−0.50.00.51.0−1.0−0.50.00.51.051015205101520(c)(d)−1.0−0.50.00.51.0−1.0−0.50.00.51.051015205101520Figure3.19:TheautocorrelationfunctionofanAR(2)model:(a)φ1=1.2andφ2=−0.35,(b)φ1=1.0andφ2=−0.7,(c)φ1=0.2andφ2=0.35,(d)φ1=−0.2andφ2=0.35.Example2.16:Asanillustration,considerthequarterlygrowthrateofU.S.realgrossnationalproduct(GNP),seasonallyadjusted,fromthesecondquarterof1947tothefirstquarterof1991,whichisshownintheleftpanelofFigure3.20.TherightpanelofFigure0.00.20.40.60.81.0−0.02−0.010.000.010.020.030.0419501960197019801990051015202530Figure3.20:ThegrowthrateofUSquarterlyrealGNPfrom1947.IIto1991.I(seasonallyadjustedandinpercentage):theleftpanelisthetimeseriesplotandtherightpanelistheACF. CHAPTER3.UNIVARIATETIMESERIESMODELS1403.20displaystheACFofthisseriesanditshowsthatapictureofdampingsineandcosinewaves.Wecanconcludethatcyclesexist.Thisseriescanbeusedasanexampleofnonlineareconomictimeseries;seeTsay(2005,Chapter4)forthedetailedanalysesusingtheMarkovswitchingmodel.HerewesimplyemployanAR(3)modelforthedata.Denotingthegrowthratebyxt,wecanusethemodelbuildingproceduretoestimatethemodel.Thefittedmodelis2xt=0.0047+0.35xt−1+0.18xt−2−0.14xt−3+wt,withσcw=0.0098.Rewritingthemodelasxt−0.35xt−1−0.18xt−2+0.14xt−3=0.0047+wt,weobtainacorrespondingthird-orderdifferenceequa-tion(1−0.35L−0.18L2+0.14L3)=0,whichcanbefactoredas(1+0.52L)(1−0.87L+0.27L2)=0.Thefirstfactor(1+0.52L)showsanexponentiallydecayingfeatureoftheGNPgrowthrate.Focusingonthesecondorderfactor1−0.87L−(−0.27)L2=0,wehaveφ2+4φ=0.872+4(−0.27)=−0.3231<0.Therefore,the12secondfactoroftheAR(3)modelconfirmstheexistenceofstochas-ticbusinesscyclesinthequarterlygrowthrateofU.S.realGNP.ThisisreasonableastheU.S.economywentthroughexpansionandcontractionperiods.Theaveragelengthofthestochasticcyclesisapproximately2πT0=√=10.83quarters,cos−1[φ1/(2−φ2)]whichisabout3years.IfoneusesanonlinearmodeltoseparateU.S.economyinto“expansion”and“contraction”periods,thedatashowthattheaveragedurationofcontractionperiodsisaboutthreequartersandthatofexpansionperiodsisabout3years.Theaveragedurationof10.83quartersisacompromisebetweenthetwoseparatedurations.Theperiodicfeatureobtainedhereiscommonamong CHAPTER3.UNIVARIATETIMESERIESMODELS141growthratesofnationaleconomies.Forexample,similarfeaturescanbefoundforothercountries.ForastationaryAR(p)series,theACFsatisfiesthedifferenceequation(1−φL−φL2−···−φLp)ρ(h)=0,forh>0.The12pxplotofACFofastationaryAR(p)modelwouldthenshowamixtureofdampingsineandcosinepatternsandexponentialdecaysdependingonthenatureofitscharacteristicroots.Finally,wecontinueouranalysisoftherecruitsseriesasenter-tainedinExample2.4,fromwhichanAR(2)modelisfittedforthisseriesasxt−1.3512xt−1+0.4612xt−2=61.8439+wt.Clearly,φ2+4φ=1.35122−4×0.4612=−0.0191<0,whichim-12pliestheexistenceofstochasticbusinesscyclesintherecruitsseries.TheaveragelengthofthestochasticcyclesbasedontheabovefittedAR(2)modelisapproximately2πT0=√=61.71months,cos−1[φ1/(2−φ2)]whichisabout5years.Notethatthisaveragelengthofcyclesisnotclosetowhatwehaveobserved(about50months).Pleasefigureoutthereasonwhythereisabigdifference.3.13ImpulseResponseFunctionThetaskfacingthemoderntime-serieseconometricianistodevelopreasonablysimpleandintuitivemodelscapableofforecasting,inter-pretingandhypothesistestingregardingeconomicandfinancialdata.Inthisrespect,thetimeserieseconometricianisoftenconcernedwith CHAPTER3.UNIVARIATETIMESERIESMODELS142theestimationofdifferenceequationscontainingstochasticcompo-nents.3.13.1FirstOrderDifferenceEquationsSupposewearegiventhedynamicequationyt=φ1yt−1+wt,whereytisthevalueofvariableatperiodt,wtisthevalueofvariableatperiodt,1≤t≤n.Indeed,thisisanAR(1)model.Theequationrelatesavariableyttoitsprevious(lagged)valueswithonlythefirstlagappearsontherighthandside(RHS)oftheequation.Fornow,theinputvariable,{w1,w2,w3,...},willsimplyberegardedasasequenceofdeterministicnumberswhichcanbegeneratedfromadistribution.Lateron,wewillassumethattheyarestochastic.Bysolvingadifferenceequationbyrecursivesubstitution,assumingthatweknowthestartingvalueofy−1,calledtheinitialcondition,then,wehavey0=φ1y−1+w0,2y1=φ1y0+w1=φ(φ1y−1+w0)+w1=φ1y−1+φ1w0+w1,232y2=φ1y1+w2=φ1(φ1y−1+φ1w0+w1)+w2=φ1y−1+φ1w0+φ1w1+...t+1tt−1yt=φ1yt−1+wt=φ1y−1+φ1w0+φw1+···+φ1wt−1+wtXtt+1k=φ1y−1+φ1wt−k.(3.39)k=0Theproceduretoexpressytintermofthepastvaluesofwtandthestartingvaluey−1isknownasrecursivesubstitution.ThelasttermonRHSof(3.39)iscalledthelinearprocess(withfiniteterms)generatedby{wt}if{wt}israndom.Nextweconsiderthecomputationofdynamicmultiplier.Todoso,weconsideronesimpleexperimentbyassumingthaty−1and CHAPTER3.UNIVARIATETIMESERIESMODELS143{w0,w1,···,wn}aregiven,fixedatthismoment,andthevalueofφ1isknown.Thenwecancomputethetimeseriesytbyyt=φ1yt−1+wtfor1≤t≤n.Whathappenstotheseriesytifattheperiodt=50,wechangethevalueofdeterministiccomponentw50andsetthenewwtobewf50=w50+1?Ofcourse,thechangeinw50leadstochangesinyt:ye50=φ1y49+wf50=φ1y49+w50+1=y50+1,ye51=φ1y50+w51=φ1(y50+1)+w51=y51+φ1,2ye52=φ1ye51+w52=φ1(y51+φ1)+w52=y52+φ1,andsoon.Ifwelookatthedifferencebetweenyetandytthenweobtainthat:ye−y=1,ye−y=φ,ye−y=φ2,andsoon.50505151152521ThisexperimentisillustratedinFigures3.21and3.22.Therefore,NoImpulseNoImpulseWithImpulseWithImpulsephi=0.8phi=−0.8−4−2024−4−202020406080100020406080100phi=0.8phi=−0.8ImpulseResponseFunctionImpulseResponseFunction−1.0−0.50.00.51.0−1.0−0.50.00.51.0020406080100020406080100Figure3.21:Thetime-seriesytisgeneratedwithwt∼N(0,1),y0=5.Atperiodt=50,thereisanadditionalimpulsetotheerrorterm,i.e.we50=w50+1.Theimpulseresponsefunctioniscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse.onecansaythatoneunitincreaseinw50leadstoy50increasedby1, CHAPTER3.UNIVARIATETIMESERIESMODELS144NoImpulseWithImpulseNoImpulseWithImpulsephi=−1.0151015202530phi=1.01020406080100−200−100201020406080100phi=1.01phi=−1.01ImpulseResponseFunctionImpulseResponseFunction−2−1012−2−1012020406080100020406080100Figure3.22:Thetime-seriesytisgeneratedwithwt∼N(0,1),y0=3.Atperiodt=50,thereisanadditionalimpulsetotheerrorterm,i.e.we50=w50+1.Theimpulseresponsefunctioniscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse.yincreasedbyφ,yincreasedbyφ2,andsoso.Thequestionis511521howyt+jchangesifchangewtbyone?Thisisexactlythequestionthatdynamicmultipliersanswer.Assumethatwestartwithyt−1insteadofy−1,i.e.weobservethevalueofyt−1atperiodt−1.Canwesaysomethingaboutyt+j?Well,letusanswerthisquestion.yt=φ1yt−1+wt,2yt+1=φ1yt+wt+1=φ1(φ1yt−1+wt)+wt+1=φ1yt−1+φ1wt+wt+1,...j+1jj−1yt+j=φ1yt−1+φ1wt+φwt+1+···+φ1wt+j−1+wt+jThedynamicmultiplierisdefinedby∂yt+jj=φ1foranAR(1)model.∂wt CHAPTER3.UNIVARIATETIMESERIESMODELS145Notethatthemultiplierdoesnotdependont.Thedynamicmulti-plier∂yt+j/∂wtiscalledsometimestheimpactmultiplier.ImpulseResponseFunctionWecanplotdynamicmultipliersasafunctionoflagj,i.eplot∂yt+jJ{∂wt}j=1.Becausedynamicmultiplierscalculatetheresponseofyt+jtoasingleimpulseinwt,itisalsoreferredtoastheimpulseresponsefunction(IRF).Thisfunctionhasmanyimportantapplicationsintimeseriesanalysisbecauseitshowshowtheentirepathofavariableiseffectedbyastochasticshock.Obviously,thedynamicofimpulseresponsefunctiondependsonthevalueofφ1foranAR(1)model.LetuslookatwhattherelationshipisbetweentheIRFandsystem.(a)0<φ1<1impliesthattheimpulseresponseconvergestozeroandthesystemisstable.(b)−1<φ1<0givesthattheimpulseresponseoscillatesbutconvergestozeroandthesystemisstable.(c)φ1>1concludesthattheimpulseresponseisexplosiveandthesystemisunstable.(d)φ1<−1impliesthattheimpulseresponseisexplosive(os-cillationally)andthesystemisunstable.ImpulseresponsefunctionsforallpossiblecasesarepresentedinFigure3.23.PermanentChangeinwtIncalculatingdynamicmultipliersinFigure3.23,wewereaskingwhatwouldhappenifwtweretoincreasebyoneunitwithwt+1,wt+2,···,,wt+ CHAPTER3.UNIVARIATETIMESERIESMODELS146oooooooooooophi1=−0.99ooooooophi1=0.99ooooooooooooooooooophi1=−0.5oophi1=−0.9ooooooooooooooooooooooooooooooooooophi1=0.5oophi1=0.9ooooooooooooooooooooooo0.00.20.40.60.8o1.0ooooooooooooooooooo−1.0−0.50.00.51.051015205101520oooophi1=−1.2ooophi1=1.2ooooooooooooooooooooooooooooo0oo10o2030−30−10010203040o51015205101520Figure3.23:Exampleofimpulseresponsefunctionsforfirstorderdifferenceequations.unaffected.Wewerefindingtheeffectofapurelytransitorychangeinwt.Permanentchangeinwtmeansthatwt,wt+1,···,,wt+jwouldallincreasebyoneunit.Theeffectonyt+jofapermanentchangeinwtbeginninginperiodtisthengivenbyj+1∂yt+j∂yt+j∂yt+jjj−11−φ1++···+=φ1+φ1+···+φ1=fortheAR(1)mo∂wt∂wt+1∂wt+j1−φ1Thedifferentbetweentransitoryandpermanentchangeinwtisil-lustratedinFigure3.24.3.13.2HigherOrderDifferenceEquationsConsiderthemodel:Alinearpthorderdifferenceequationhasthefollowingform:yt=φ1yt−1+φ2yt−2+···+φpyt−p+wt,whichisanAR(p)modelifwtisthewhitenoise.Itiseasytodrivethepropertiesofthepthorderdifferenceequation.Toillustratethepropertiesofpthorderdifferenceequation,weconsiderthesecond CHAPTER3.UNIVARIATETIMESERIESMODELS147NoImpulseNoImpulseWithImpulseWithImpulse−3−2−1012phi=0.8phi=0.8020406080100−40−202024406080100phi=0.8ImpulseResponseFunctionImpulseResponseFunctionphi=0.80.00.20.40.60.81.0012345020406080100020406080100Figure3.24:Thetimeseriesytisgeneratedwithwt∼N(0,1),y0=3.Forthetransitoryimpulse,thereisanadditionalimpulsetotheerrortermatperiodt=50,i.e.we50=w50+1.Forthepermanentimpulse,thereisanadditionalimpulseforperiodt=50,···,100,i.e.wet=wt+1,t=50,51,···,100.Theimpulseresponsefunction(IRF)iscomputedasthedifferencebetweentheseriesytwithoutimpulseandtheseriesyetwiththeimpulse.orderdifferenceequation:yt=φ1yt−1+φ2yt−2+wtsothatp=2.Wewriteequationasafirstordervectordifferenceequation.ξt=Fξt−1+vt,whereytφ1φ2wtξt=,F=,andvt=.yt−1100Ifthestartingvalueξ−1isknownweusetherecursivesubstitutiontoobtain:t+1tt−1ξt=Fξ−1+Fv0+Fv1+···+Fvt−1+vt.Orifthestartingvalueξt−1isknownweusetherecursivesubstitu-tiontoobtain:j+1jj−1ξt+j=Fξt−1+Fvt+Fvt+1+···+Fvt+j−1+vt+j CHAPTER3.UNIVARIATETIMESERIESMODELS148foranyj≥0.Tocomputethedynamicmultipliers,werecalltherulesofofvectorandmatrixdifferentiation.Ifx(β)isanm×1vectorthatdependsonthen×1vectorβ,then,∂x1(β)∂x1(β)∂x∂β1···∂βn=..........′∂βm×n∂xm(β)∂xm(β)···∂β1∂βnItiseasytoverifythatthedynamicmultipliersaregivenbyj∂ξt+j∂Fvtj==F.∂v′v′ttSincethefirstelementinξt+jisyt+jandthefirstelementinvtisjwtthentheelement(1,1)inthematrixFis∂yt+j/∂wt,i.e.thedynamicmultiplier.Forlargervaluesofj,aneasywaytoobtainanumericalvaluesfordynamicmultiplier∂yt+j/∂wtistosimulatethesystem.Thisisdoneasfollows.Sety−1=y−2=···=y−p=0andw0=1,andsetthevaluesofwforallotherdatesto0.ThenuseAR(p)tocalculatethevalueforytfor≤t≤n.Toillustratedynamicmultipliersforthepthorderdifferenceequa-tion,weconsiderthesecondorderdifferenceequationasfollows:φ1φ2yt=φ1yt−1+φ2yt−2+wt,sothatF=.Theimpulse10responsefunctionsforthisexamplewithfourdifferentsettingsarepresentedinFigure3.25:(a)φ1=0.6andφ2=0.2,(b)φ1=0.8andφ2=0.4,(c)φ1=−0.9andφ2=−0.5,and CHAPTER3.UNIVARIATETIMESERIESMODELS149(a)(b)oooImpulseResponseFunctionImpulseResponseFunctionooooophi1=0.8andphi2=0.4ooooooophi1=0.6andphi2=0.2ooooooooooooooooooooooooooo0246810120.00.20.40.60.81.00510152005101520(c)(d)ooImpulseResponseFunctionImpulseResponseFunctionoooooooooooooooooooooooooooooooooooophi1=−0.9andphi2=−0.5phi1=−0.5andphi2=−1.5ooo−40−200102030o−1.0−0.50.00.51.00510152005101520Figure3.25:Exampleofimpulseresponsefunctionsforsecondorderdifferenceequation.(d)φ1=−0.5andφ2=−1.5.Similartothefirstorderdifferenceequation,theimpulseresponsefunctionforthepthorderdifferenceequationcanbeexplosiveorcon-vergetozero.Whatdeterminesthedynamicsofimpulseresponsefunction?TheeigenvaluesofmatrixFdeterminewhethertheim-pulseresponseoscillates,convergesorisexplosive.Theeigenvaluesofap×pmatrixAarethosenumbersλforwhich|A−λIp|=0.TheeigenvaluesofthegeneralmatrixFdefinedabovearethevaluesofλthatsatisfy:pp−1p−2−pλ−φ1λ−φ2λ−···−φp−1λ−φp=φ(λ)=0.NotethattheeigenvaluesarealsocalledthecharacteristicrootsofAR(p)model.Iftheeigenvaluesarerealbutatleastoneeigen-valueisgreaterthanunityinabsolutevalue,thesystemisexplosive. CHAPTER3.UNIVARIATETIMESERIESMODELS150Whydoeigenvaluesdeterminethedynamicsofdynamicmultipliers?Recallthatiftheeigenvaluesofp×pmatrixAaredistinct,there−1existsanonsingularp×pmatrixTsuchthatA=TΛT,whereΛisap×pmatrixwiththeeigenvaluesofAontheprincipaldi-jj−1agonalandzeroselsewhere.Then,F=TΛTandthevalueofjeigenvaluesinΛdetermineswhethertheelementsofFexplodeornot.Recallthatthedynamicmultiplierisequalto∂ξ/∂v′=Fj.t+jtTherefore,thesizeofeigenvaluesdetermineswhetherthesystemisstableornot.Nowwecomputethetheeigenvaluesforeachcaseintheaboveexample:(a)λ1=0.838andλ2=−0.238sothat|λk|<1andthesystemisstable.(b)λ1=1.148andλ2=−0.348sothat|λ1|>1andthesystemisunstable.r(c)λ=−0.45±0.545isothat|λ|=(−0.45)2+0.5452=0.706<1andthesystemisstable.Sinceeigenvaluesarecomplex,theimpulseresponsefunctionoscillates.r(d)λ=−0.25±1.198isothat|λ|=(−0.25)2+1.1982=1.223>1andthesystemisunstable.Sinceeigenvaluesarecomplex,theimpulseresponsefunctionoscillates.3.14Problems1.Considertheregressionmodelyt=β1yt−1+wt,wherewtiswhitenoisewithzero-meanandvarianceσ2.Assumethatwewobserve{y}n.Showthattheleastsquaresestimatorofβistt=21cPnP2β1=t=2ytyt−1/t=2yt−1.Ifwepretendthatyt−1wouldbe CHAPTER3.UNIVARIATETIMESERIESMODELS151fixed,showthatVar(βc)=σ2/Pny2.Relateyouranswerto1et=2t−1amethodforfittingafirst-orderARmodeltothedatayt.2.ConsidertheautoregressivemodelAR(1),i.e.xt−φ1xt−1=wt.(a)Findthenecessaryconditionfor{xt}tobeinvertible.(b)Showthatxtcanbeexpressedasalinearprocess.(c)ShowthatE[wx]=σ2andE[wx]=0,sothatfuturettwtt−1errorsareuncorrelatedwithpastdata.3.Theauto-covarianceandautocorrelationfunctionsforARpro-cessesareoftenderivedfromtheYule-Walkerequations,ob-tainedbymultiplyingbothsidesofthedefiningequation,succes-sivelybyxt,xt−1,···.UsetheYule-Walkerequationstodriveρx(h)forthefirst-orderAR.4.ForanARMAserieswedefinetheoptimalforecastbasedonxt,x,···astheconditionalexpectationxt=E[x|x,x,···]t−1t+ht+htt−1forh≥1.(a)Show,forthegeneralARMAmodelthatE[wt+h|xt,xt−1,···]=0ifh>0,wt+hifh≤0.(b)FortheAR(1)andAR(2)models,derivetheoptimalforecastxtandthepredictionerrorvarianceoftheone-stepforecast.t+h5.Supposewehavethesimplelineartrendmodelyt=β1t+xt,1≤t≤n,wherext=φ1xt−1+wt.Givetheexactformoftheequationsthatyouwoulduseforestimatingβ,φandσ2using11wtheCochran-Orcuttprocedure.6.SupposethatthesimplereturnofamonthlybondindexfollowstheMA(1)modelxt=wt+0.2wt−1,σw=0.025. CHAPTER3.UNIVARIATETIMESERIESMODELS152Assumethatw=0.01.Computethe1-step(xt)and2-100t+1step(xt)aheadforecastsofthereturnattheforecastorigint+2t=100.Whatarethestandarddeviationsoftheassociatedforecasterrors?Alsocomputethelag-1(ρ(1))andlag-2(ρ(2))autocorrelationsofthereturnseries.7.Supposethatthedailylogreturnofasecurityfollowsthemodelxt=0.01+0.2xt−2+wt,where{wt}isaGaussianwhitenoiseserieswithmeanzeroandvariance0.02.Whatarethemeanandvarianceofthereturnseriesxt?Computethelag-1(ρ(1))andlag-2(ρ(2))autocor-relationsofxt.Assumethatx100=−0.01,andx99=0.02.Computethe1-step(xt)and2-step(xt)aheadforecastsoft+1t+2thereturnseriesattheforecastorigint=100.Whataretheassociatedstandarddeviationsoftheforecasterrors?8.Considerthefile“la-regr.dat”,inthesyllabus,whichcontainscardiovascularmortality,temperaturevaluesandparticulatelev-elsover6-dayperiodsfromLosAngelesCounty(1970-1979).Thefilealsocontainstwodummyvariablesforregressionpur-poses,acolumnofonesfortheconstanttermandatimeindex.Theorderisasfollows:Column1:508cardiovascularmortalityvalues(6-dayaverages),Column2:508ones,Column3:theintegers1,2,···,508,Column3:TemperatureindegreesFandColumn4:Particulatelevels.AreferenceisShumwayet.al.(1988).Thepointhereistoexaminepossiblerelationsbetweenthetemperatureandmortalityinthepresenceofatimetrendincardiovascularmortality.(a)Usescatterdiagramstoarguethatparticulatelevelmaybelinearlyrelatedtomortalityandthattemperaturehaseithera CHAPTER3.UNIVARIATETIMESERIESMODELS153linearorquadraticrelation.Checkforlaggedrelationsusingthecrosscorrelationfunction.(b)Adjusttemperatureforitsmeanvalueandfitthemodel2Mt=β0+β1(Tt−T)+β2(Tt−T)+β3Pt+et,whereMt,TtandPtdenotethemortality,temperatureandpar-ticulatepollutionseries.YoucanuseasinputsColumns2and3forthetrendtermsandruntheregressionanalysiswithouttheconstantoption.(c)Plottheresidualsandcomputetheautocorrelation(ACF)andpartialautocorrelation(PACF)functions.Dotheresidualsappeartobewhite?SuggestanARIMAmodelfortheresidualsandfittheresiduals.ThesimpleARIMA(2,0,0)modelisagoodcompromise.(d)ApplytheARIMAmodelobtainedinpart(c)toalloftheinputvariablesandtocardiovascularmortalityusingatransfor-mation.Retaintheforecastvaluesforthetransformedmortality,saymt=Mt−φ1Mt−1−φ2Mt−2.9.Generate10realizationsofa(n=200pointseach)seriesfromanARIMA(1,0,1)Modelwithφ=0.90,θ=0.20andσ2=0.25.11wFittheARIMAmodeltoeachoftheseriesandcomparetheestimatorstothetruevaluesbycomputingtheaverageoftheestimatorsandtheirstandarddeviations.10.ConsiderthebivariatetimeseriesrecordcontainingmonthlyU.S.ProductionasmeasuredmonthlybytheFederalReserveBoardProductionIndexandunemploymentasgiveninthefile“frb.asd”.Thefilecontainsn=372monthlyvaluesforeachse-ries.Beforeyoubegin,besuretoplottheseries.Fitaseasonal CHAPTER3.UNIVARIATETIMESERIESMODELS154ARIMAmodelofyourchoicetotheFederalReserveProductionIndex.Developa12monthforecastusingthemodel.11.Thefilelabelled“clim-hyd.asd”has454monthsofmeasuredvaluesfortheclimaticvariablesAirTemperature,DewPoint,CloudCover,WindSpeed,Preciptation,andInflowatShastaLake.WewouldliketolookatpossiblerelationsbetweentheweatherfactorsandbetweentheweatherfactorsandtheinflowtoShastaLake.(a)FittheARIMA(0,0,0)×(0,1,1)12modeltotransformed√precipitationPt=ptandtransformedflowit=log(it).Savetheresidualsfortransformedprecipitationforuseinpart(b).(b)ApplytheARIMAmodelfittedinpart(a)fortransformedprecipitationtotheflowseries.ComputethecrosscorrelationbetweentheflowresidualsusingtheprecipitationARIMAmodelandtheprecipitationresidualsusingtheprecipitationmodelandinterpret.12.ConsiderthedailysimplereturnofCRSPequal-weightedindex,includingdistributions,fromJanuary1980toDecember1999inthefile“d-ew8099.txt”(day,ew).TheindicatorvariablesforMondays,Tuesdays,Wednesdays,andThursdaysareinthefirstfourcolumnsofthefile“wkdays8099.dat”.Usearegressionmodel,possiblywithtimeserieserrors,tostudytheeffectsoftradingdaysontheindexreturn.Whatisthefittedmodel?Aretheweekdayeffectssignificantinthereturnsatthe5%level?UsetheHACestimatorofthecovariancematrixtoobtainthet-ratiosofregressionestimates.Doesitchangetheconclusionofweekdayeffect?Arethereserialcorrelationsintheresiduals?UsetheLjung-Boxtesttoperformthetest.Drawyourconclusion.If CHAPTER3.UNIVARIATETIMESERIESMODELS155yes,buildaregressionmodelwithtimeserieserrortostudyweekdayeffects.13.Thisproblemisconcernedwiththedynamicrelationshipbe-tweenthespotandfuturespricesoftheS&P500index.Thedatafile“sp5may.dat”hasthreecolumns:log(futuresprice),log(spotprice),andcost-of-carry(×100).Thedatawereob-tainedfromtheChicagoMercantileExchangefortheS&P500stockindexinMay1993anditsJunefuturescontract.Thetimeintervalis1minute(intraday).Severalauthorsusedthedatatostudyindexfuturesarbitrage.Herewefocusonthefirsttwocolumns.Letftandstbethelogpricesoffuturesandspot,respectively.Buildaregressionmodelwithtimeserieserrorsbe-tween{ft}and{st},withftbeingthedependentvariable.Youneedtoprovidealldetails(reasonsandanalysisresults)ateachstep.14.Thequarterlygrossdomesticproductimplicitpricedeflatorisoftenusedasameasureofinflation.Thefile“q-gdpdef.dat”containsthedataforU.S.fromthefirstquarterof1947tothefirstquarterof2004.Thedataareseasonallyadjustedandequalto100foryear2000.ThedataareobtainedfromtheFederalReserveBankofStLouis.Builda(seasonal)ARIMAmodelfortheseriesandcheckthevalidityofthefittedmodel.Usethemodeltoforecastthegrossdomesticproductimplicitpricedeflatorfortherestof2004.15.ConsiderthemonthlysimplereturnsoftheDecile1,Decile5,andDecile10ofNYSE/AMEX/NASDAQbasedonmarketcap-italization.ThedataspanisfromJanuary1960toDecember2003,andthedataareobtainedfromCRSPwiththefilename: CHAPTER3.UNIVARIATETIMESERIESMODELS156“m-decile1510.txt”.Foreachseries,testthenullhypothesisthatthefirst12lagsofautocorrelationsarezeroatthe5%level.Drawyourconclusion.BuildanARandMAmodelfortheseriesDecile5.UsetheARandMAmodelbuilttoproduce1-stepand3-stepaheadforecastsoftheseries.ComparethefittedARandMAmodels.16.Thedatafile“q-unemrate.txt”containstheU.S.quarterlyun-employmentrate,seasonallyadjusted,from1948tothesecondquarterof1991.Considerthechangeseries∆xt=xt−xt−1,wherextisthequarterlyunemploymentrate.BuildanARmodelforthe∆xtseries.Doesthefittedmodelsuggesttheexistenceofbusinesscycles?17.Inthisexercise,pleaseconstructtheimpulseresponsefunctionforaathirdorderdifferenceequation:yt=φ1yt−1+φ2yt−2+φy+wfor1≤t≤n,whereitisassumedthat{w}nisa3t−3ttt=1sequenceofdeterministicnumbers,saygeneratedfromN(0,1).(a)Setφ1=1.1,φ2=−0.8,φ3=0.1,y0=y−1=y−2=0,andn=150.Generateytusingathirdorderdifferenceequationfor1≤t≤n.(b)CheckeigenvaluesofthismodeltodeterminewhetherIRFareconvergingorexplosive.(c)Constructtheimpulseresponsefunctionforthegeneratedyt.SetthenumberofperiodsintheimpulseresponsefunctiontoJ=25.Commentyourresults.(d)Setφ1=1.71andrepeatsteps(a)-(c).Commentyourresults. CHAPTER3.UNIVARIATETIMESERIESMODELS1573.15ComputerCodeThefollowingRcommandsareusedformakingthegraphsinthischapter.#3-28-2006graphics.off()###################################################################ibm<-matrix(scan("c:\teaching\timeseries\data\m-ibm2697.txt"),byrow=T,ncol=1)vw<-matrix(scan("c:\teaching\timeseries\data\m-vw2697.txt"),byrow=T,ncol=1)n=length(ibm)ibm1=ibmibm2=log(ibm1+1)vw1=vwvw2=log(1+vw1)postscript(file="c:\teaching\timeseries\figs\fig-2.1.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(ibm1,ylab="",xlab="",ylim=c(-0.2,0.2),lag=100,main="SimpleReturns",cex=0.5)text(50,0.2,"IBM")acf(ibm2,ylab="",xlab="",ylim=c(-0.2,0.2),lag=100,main="LogReturns",cex=0.5)text(50,0.2,"IBM")acf(vw1,ylab="",xlab="",ylim=c(-0.2,0.2),lag=100,main="SimpleReturns",cex=0.5) CHAPTER3.UNIVARIATETIMESERIESMODELS158text(50,0.2,"value-weightedindex")acf(vw2,ylab="",xlab="",ylim=c(-0.2,0.2),lag=100,main="LogReturns",cex=0.5)text(50,0.2,"value-weightedindex")dev.off()######################################################################################################################################y1<-matrix(scan("c:\teaching\timeseries\data gtemp.dat"),byrow=T,ncol=1)y=y1[,1]n=length(y)a<-1:12a=a/12y=y1[,1]n=length(y)x<-rep(0,n)for(iin1:149){x[((i-1)*12+1):(12*i)]<-1856+i-1+a}x[n-1]<-2005+1/12x[n]=2005+2/13x=x/100x1=cbind(rep(1,n),x)z=t(x1)%*%x1fit1=lm(y~x)#fitaregressionmodelresid1=fit1$resid#obatinresidulssigma2=mean(resid1^2)y.diff=diff(y)#computedifference CHAPTER3.UNIVARIATETIMESERIESMODELS159var_beta=sigma2*solve(z)postscript(file="c:\teaching\timeseries\figs\fig-2.2.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(resid1,lag.max=20,ylab="",ylim=c(-0.5,1),main="DetrendedTemperature",cex=0.5)text(5,0.8,"ACF")pacf(resid1,lag.max=20,ylab="",ylim=c(-0.5,1),main="")text(5,0.8,"PACF")acf(y.diff,lag.max=20,ylab="",ylim=c(-0.5,1),main="DifferencedTemperature",cex=0.5)text(5,0.8,"ACF")pacf(y.diff,lag.max=20,ylab="",ylim=c(-0.5,1),main="")text(5,0.8,"PACF")dev.off()####################################################################simulateanI(1)seriesn=200#y=arima.sim(list(order=c(0,1,0)),n=200)#simulatetheintegradtedx2=rnorm(n)#awhitenoiseseriesy=diffinv(x)#simulateI(1)withpostscript(file="c:\teaching\timeseries\figs\fig-2.3.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)ts.plot(y,type="l",lty=1,ylab="",xlab="") CHAPTER3.UNIVARIATETIMESERIESMODELS160text(100,0.8*max(y),"RandomWalk")ts.plot(x2,type="l",ylab="",xlab="")text(100,0.8*max(x2),"FirstDifference")abline(0,0)dev.off()postscript(file="c:\teaching\timeseries\figs\fig-2.4.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(y,ylab="",xlab="",main="RandomWalk",cex=0.5,ylim=c(-0.5,1.0))text(15,0.8,"ACF")pacf(y,ylab="",xlab="lag",main="",ylim=c(-0.5,1.0))text(15,0.8,"PACF")acf(x2,ylab="",xlab="",main="FirstDifference",cex=0.5,ylim=c(-0.5,1.0))text(15,0.8,"ACF")pacf(x2,ylab="",xlab="lag",main="",ylim=c(-0.5,1.0))text(15,0.8,"PACF")dev.off()#################################################################ThisisExample2.5inChapter2###################################x<-read.table("c:\teaching\timeseries\data\soi.dat",header=T)x.soi=x[,1]n=length(x.soi)aicc=0if(aicc==1){aic.value=rep(0,30)#max.lag=10 CHAPTER3.UNIVARIATETIMESERIESMODELS161aicc.value=aic.valuesigma.value=rep(0,30)for(iin1:30){fit3=arima(x.soi,order=c(i,0,0))#fitanAR(i)aic.value[i]=fit3$aic/n-2#computeAICsigma.value[i]=fit3$sigma2#obtaintheestimatedsigma^2aicc.value[i]=log(sigma.value[i])+(n+i)/(n-i-2)#computeAICCprint(c(i,aic.value[i],aicc.value[i]))}data=cbind(aic.value,aicc.value)write(t(data),"c:\teaching\timeseries\soi_aic.dat",ncol=2)}else{data<-matrix(scan("c:\teaching\timeseries\soi_aic.dat"),byrow=T,ncol=2)}text4=c("AIC","AICC")postscript(file="c:\teaching\timeseries\figs\fig-2.5.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)acf(resid1,ylab="",xlab="",lag.max=20,ylim=c(-0.5,1),main="")text(10,0.8,"ACFofresidulsofAR(1)forSOI")matplot(1:30,data,type="b",pch="o",col=c(1,2),ylab="",xlab="Lag"legend(16,-1.40,text4,lty=1,col=c(1,2))dev.off()#fit2=arima(x.soi,order=c(16,0,0))#print(fit2)####################################################################ThisisExample2.7inChapter2 CHAPTER3.UNIVARIATETIMESERIESMODELS162####################################varve<-read.table("c:\teaching\timeseries\data\mass2.dat",header=T)varve=varve[,1]n_varve=length(varve)varve_log=log(varve)varve_log_diff=diff(varve_log)postscript(file="c:\teaching\timeseries\figs\fig-2.6.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(varve_log,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),main="ACtext(10,0.7,"logvarves",cex=0.7)pacf(varve_log,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),main="Pacf(varve_log_diff,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),maitext(10,0.7,"Firstdifference",cex=0.7)pacf(varve_log_diff,ylab="",xlab="",lag.max=30,ylim=c(-0.5,1),madev.off()####################################################################ThisisExample2.9inChapter2####################################x<-matrix(scan("c:\teaching\timeseries\data\birth.dat"),byrow=T,ncol=1)n=length(x)x_diff=diff(x)x_diff_12=diff(x_diff,lag=12)fit1=arima(x,order=c(0,0,0),seasonal=list(order=c(0,0,0)),inresid_1=fit1$residfit2=arima(x,order=c(0,1,0),seasonal=list(order=c(0,0,0)),inresid_2=fit2$resid CHAPTER3.UNIVARIATETIMESERIESMODELS163fit3=arima(x,order=c(0,1,0),seasonal=list(order=c(0,1,0),periinclude.mean=F)resid_3=fit3$residpostscript(file="c:\teaching\timeseries\figs\fig-2.8.eps",horizontal=F,width=6,height=6)par(mfrow=c(5,2),mex=0.4)acf(resid_1,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="ACF",cex=0.7)pacf(resid_1,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="PACF",ctext(20,0.7,"data",cex=1.2)acf(resid_2,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")#differenceddatapacf(resid_2,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")text(30,0.7,"ARIMA(0,1,0)")acf(resid_3,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")#seasonaldifferenceofdifferenceddatapacf(resid_3,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")text(30,0.7,"ARIMA(0,1,0)X(0,1,0)_{12}",cex=0.8)fit4=arima(x,order=c(0,1,0),seasonal=list(order=c(0,1,1),period=12),include.mean=F)resid_4=fit4$residfit5=arima(x,order=c(0,1,1),seasonal=list(order=c(0,1,1),period=12),include.mean=F)resid_5=fit5$residacf(resid_4,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")#ARIMA(0,1,0)*(0,1,1)_12pacf(resid_4,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")text(30,0.7,"ARIMA(0,1,0)X(0,1,1)_{12}",cex=0.8) CHAPTER3.UNIVARIATETIMESERIESMODELS164acf(resid_5,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")#ARIMA(0,1,1)*(0,1,1)_12pacf(resid_5,ylab="",xlab="",ylim=c(-0.5,1),lag=60,main="")text(30,0.7,"ARIMA(0,1,1)X(0,1,1)_{12}",cex=0.8)dev.off()postscript(file="c:\teaching\timeseries\figs\fig-2.7.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)ts.plot(x,type="l",lty=1,ylab="",xlab="")text(250,375,"Births")ts.plot(x_diff,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))text(255,45,"Firstdifference")abline(0,0)ts.plot(x_diff_12,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))#timeseriesplotoftheseasonaldifference(s=12)ofdiffetext(225,40,"ARIMA(0,1,0)X(0,1,0)_{12}")abline(0,0)ts.plot(resid_5,type="l",lty=1,ylab="",xlab="",ylim=c(-50,50))text(225,40,"ARIMA(0,1,1)X(0,1,1)_{12}")abline(0,0)dev.off()###################################################################################################################################ThisisExample2.10inChapter2#####################################y<-matrix(scan("c:\teaching\timeseries\data\jj.dat"),byrow=T,ncol=1) CHAPTER3.UNIVARIATETIMESERIESMODELS165n=length(y)y_log=log(y)#logofdatay_diff=diff(y_log)#first-orderdifferencey_diff_4=diff(y_diff,lag=4)#first-orderseasonaldifferencefit1=ar(y_log,order=1)#fitAR(1)model#print(fit1)library(tseries)#calllibrary(tseries)library(zoo)fit1_test=adf.test(y_log)#doAugmentedDicky-Fullertestfortestingunitroot#print(fit1_test)fit1=arima(y_log,order=c(0,0,0),seasonal=list(order=c(0,0,0))include.mean=F)resid_21=fit1$residfit2=arima(y_log,order=c(0,1,0),seasonal=list(order=c(0,0,0))include.mean=F)resid_22=fit2$resid#residualforARIMA(0,1,0)*(0,0,0)fit3=arima(y_log,order=c(0,1,0),seasonal=list(order=c(1,0,0),include.mean=F,method=c("CSS"))resid_23=fit3$resid#residualforARIMA(0,1,0)*(1,0,0)#notethatthismodelisnon-stationarysothat"CSS"isusedpostscript(file="c:\teaching\timeseries\figs\fig-2.9.eps",horizontal=F,width=6,height=6)par(mfrow=c(4,2),mex=0.4)acf(resid_21,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="ACF",cex=0.7)text(16,0.8,"log(J&J)")pacf(resid_21,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF", CHAPTER3.UNIVARIATETIMESERIESMODELS166acf(resid_22,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")text(16,0.8,"FirstDifference")pacf(resid_22,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")acf(resid_23,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")text(16,0.8,"ARIMA(0,1,0)X(1,0,0,)_4",cex=0.8)pacf(resid_23,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")fit4=arima(y_log,order=c(0,1,1),seasonal=list(order=c(1,0,0),period=4),include.mean=F,method=c("CSS"))resid_24=fit4$resid#residualforARIMA(0,1,1)*(1,0,0)#notethatthismodelisnon-stationary#print(fit4)fit4_test=Box.test(resid_4,lag=12,type=c("Ljung-Box"))#print(fit4_test)acf(resid_24,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")text(16,0.8,"ARIMA(0,1,1)X(1,0,0,)_4",cex=0.8)#ARIMA(0,1,1)*(1,0,0)_4pacf(resid_24,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")dev.off()fit5=arima(y_log,order=c(0,1,1),seasonal=list(order=c(0,1,1),include.mean=F,method=c("ML"))resid_25=fit5$resid#residualforARIMA(0,1,1)*(0,1,1)#print(fit5)fit5_test=Box.test(resid_25,lag=12,type=c("Ljung-Box"))#print(fit5_test)postscript(file="c:\teaching\timeseries\figs\fig-2.10.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(resid_25,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="ACF") CHAPTER3.UNIVARIATETIMESERIESMODELS167text(16,0.8,"ARIMA(0,1,1)X(0,1,1,)_4",cex=0.8)#ARIMA(0,1,1)*(0,1,1)_4pacf(resid_25,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF")ts.plot(resid_24,type="l",lty=1,ylab="",xlab="")title(main="ResidualPlot",cex=0.5)text(40,0.2,"ARIMA(0,1,1)X(1,0,0,)_4",cex=0.8)abline(0,0)ts.plot(resid_25,type="l",lty=1,ylab="",xlab="")title(main="ResidualPlot",cex=0.5)text(40,0.18,"ARIMA(0,1,1)X(0,1,1,)_4",cex=0.8)abline(0,0)dev.off()#######################################################################################################################################ThisisExample2.11inChapter2z<-matrix(scan("c:\teaching\timeseries\data\m-decile1510.txt"),byrow=T,ncol=4)decile1=z[,2]#Model1:anARIMA(1,0,0)*(1,0,1)_12fit1=arima(decile1,order=c(1,0,0),seasonal=list(order=c(1,0,1)period=12),include.mean=T)#print(fit1)e1=fit1$residn=length(decile1)m=n/12jan=rep(c(1,0,0,0,0,0,0,0,0,0,0,0),m)feb=rep(c(0,1,0,0,0,0,0,0,0,0,0,0),m) CHAPTER3.UNIVARIATETIMESERIESMODELS168mar=rep(c(0,0,1,0,0,0,0,0,0,0,0,0),m)apr=rep(c(0,0,0,1,0,0,0,0,0,0,0,0),m)may=rep(c(0,0,0,0,1,0,0,0,0,0,0,0),m)jun=rep(c(0,0,0,0,0,1,0,0,0,0,0,0),m)jul=rep(c(0,0,0,0,0,0,1,0,0,0,0,0),m)aug=rep(c(0,0,0,0,0,0,0,1,0,0,0,0),m)sep=rep(c(0,0,0,0,0,0,0,0,1,0,0,0),m)oct=rep(c(0,0,0,0,0,0,0,0,0,1,0,0),m)nov=rep(c(0,0,0,0,0,0,0,0,0,0,1,0),m)dec=rep(c(0,0,0,0,0,0,0,0,0,0,0,1),m)de=cbind(decile1[jan==1],decile1[feb==1],decile1[mar==1],decile1[adecile1[may==1],decile1[jun==1],decile1[jul==1],decile1[aug==1],decile1[sep==1],decile1[oct==1],decile1[nov==1],decile1[dec==1])#Model2:asimpleregressionmodelwithoutcorrelatederro#toseetheeffectfromJanuaryfit2=lm(decile1~jan)e2=fit2$resid#print(summary(fit2))#Model3:aregressionmodelwithcorrelatederrorsfit3=arima(decile1,xreg=jan,order=c(0,0,1),include.mean=T)e3=fit3$resid#print(fit3)postscript(file="c:\teaching\timeseries\figs\fig-2.11.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)ts.plot(decile1,type="l",lty=1,col=1,ylab="",xlab="")title(main="SimpleReturns",cex=0.5)abline(0,0)ts.plot(e3,type="l",lty=1,col=1,ylab="",xlab="") CHAPTER3.UNIVARIATETIMESERIESMODELS169title(main="January-adjustedreturns",cex=0.5)abline(0,0)acf(decile1,ylab="",xlab="",ylim=c(-0.5,1),lag=40,main="ACF")acf(e3,ylab="",xlab="",ylim=c(-0.5,1),lag=40,main="ACF")dev.off()####################################################################ThisisExample2.12inChapter2#####################################z<-matrix(scan("c:\teaching\timeseries\data\jj.dat"),byrow=T,ncol=1)n=length(z)z_log=log(z)#logof#MODEL1:y_t=beta_0+beta_1t+e_tz1=1:nfit1=lm(z_log~z1)#fitlog(z)versustimee1=fit1$resid#Now,weneedtore-fitthemodelusingthetransformeddatax1=5:ny_1=z_log[5:n]y_2=z_log[1:(n-4)]y_fit=y_1-0.7614*y_2x2=x1-0.7614*(x1-4)x1=(1-0.7614)*rep(1,n-4)fit2=lm(y_fit~-1+x1+x2)e2=fit2$residpostscript(file="c:\teaching\timeseries\figs\fig-2.12.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4)acf(e1,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="ACF")text(10,0.8,"detrended") CHAPTER3.UNIVARIATETIMESERIESMODELS170pacf(e1,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="PACF")acf(e2,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")text(15,0.8,"ARIMA(1,0,0,)_4")pacf(e2,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")dev.off()################################################################################################################################ThisisExample2.13inChapter2#####################################z<-matrix(scan("c:\teaching\timeseries\data\w-gs1n36299.txt"),byrow=T,ncol=3)#firstcolumn=oneyearTreasuryconstantmaturityrate;#secondcolumn=threeyearTreasuryconstantmaturityrate;#thirdcolumn=datex=z[,1]y=z[,2]n=length(x)u=seq(1962+1/52,by=1/52,length=n)x_diff=diff(x)y_diff=diff(y)#Fitasimpleregressionmodelandexaminetheresidualsfit1=lm(y~x)#Model1e1=fit1$residpostscript(file="c:\teaching\timeseries\figs\fig-2.13.eps",horizontal=F,width=6,height=6)matplot(u,cbind(x,y),type="l",lty=c(1,2),col=c(1,2),ylab="", CHAPTER3.UNIVARIATETIMESERIESMODELS171dev.off()postscript(file="c:\teaching\timeseries\figs\fig-2.14.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)plot(x,y,type="p",pch="o",ylab="",xlab="",cex=0.5)plot(x_diff,y_diff,type="p",pch="o",ylab="",xlab="",cex=0.5)dev.off()postscript(file="c:\teaching\timeseries\figs\fig-2.15.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4)plot(u,e1,type="l",lty=1,ylab="",xlab="")abline(0,0)acf(e1,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")dev.off()#Takedifferentandfitasimpleregressionagainfit2=lm(y_diff~x_diff)#Model2e2=fit2$residpostscript(file="c:\teaching\timeseries\figs\fig-2.16.eps",horizontal=F,width=6,height=6)matplot(u[-1],cbind(x_diff,y_diff),type="l",lty=c(1,2),col=c(ylab="",xlab="")abline(0,0)dev.off()postscript(file="c:\teaching\timeseries\figs\fig-2.17.eps",horizontal=F,width=6,height=6) CHAPTER3.UNIVARIATETIMESERIESMODELS172par(mfrow=c(2,2),mex=0.4)ts.plot(e2,type="l",lty=1,ylab="",xlab="")abline(0,0)acf(e2,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")#fitamodeltothedifferenceddatawithanMA(1)errorfit3=arima(y_diff,xreg=x_diff,order=c(0,0,1))#Modele3=fit3$residts.plot(e3,type="l",lty=1,ylab="",xlab="")abline(0,0)acf(e3,ylab="",xlab="",ylim=c(-0.5,1),lag=30,main="")dev.off()#################################################################################################################################ThisisExample14ofusingHCandHAC##########################################library(sandwich)#HCandHACareinthepackage"sandwich"library(zoo)z<-matrix(scan("c:\teaching\timeseries\data\w-gs1n36299.txt"),byrow=T,ncol=3)x=z[,1]y=z[,2]x_diff=diff(x)y_diff=diff(y)#Fitasimpleregressionmodelandexaminetheresidualsfit1=lm(y_diff~x_diff)print(summary(fit1))e1=fit1$resid#Heteroskedasticity-ConsistentCovarianceMatrixEstimation CHAPTER3.UNIVARIATETIMESERIESMODELS173#hc0=vcovHC(fit1,type="const")#print(sqrt(diag(hc0)))#type=c("const","HC","HC0","HC1","HC2","HC3","HC4")#HC0istheWhiteestimatorhc1=vcovHC(fit1,type="HC0")print(sqrt(diag(hc1)))#Heteroskedasticityandautocorrelationconsistent(HAC)estimation#ofthecovariancematrixofthecoefficientestimatesina#(generalized)linearregressionmodel.hac1=vcovHAC(fit1,sandwich=T)print(sqrt(diag(hac1)))####################################################################ThisistheExample2.15inChapter2#######################################z1<-matrix(scan("c:\teaching\timeseries\data\d-ibmvwewsp6203.txt"),byrow=T,ncol=5)vw=abs(z1[,3])n_vw=length(vw)ew=abs(z1[,4])postscript(file="c:\teaching\timeseries\figs\fig-2.18.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4,bg="lightgreen")acf(vw,ylab="",xlab="",ylim=c(-0.1,0.4),lag=400,main="")text(200,0.38,"ACFforvalue-weightedindex")acf(ew,ylab="",xlab="",ylim=c(-0.1,0.4),lag=400,main="")text(200,0.38,"ACFforequal-weightedindex")library(fracdiff)d1=fracdiff(vw,ar=0,ma=0) CHAPTER3.UNIVARIATETIMESERIESMODELS174d2=fracdiff(ew,ar=0,ma=0)print(c(d1$d,d2$d))m1=round(log(n_vw)/log(2)+0.5)pad1=1-n_vw/2^m1vw_spec=spec.pgram(vw,spans=c(3,3,3),demean=T,detrend=T,pad=pad1ew_spec=spec.pgram(ew,spans=c(3,3,3),demean=T,detrend=T,pad=pad1vw_x=vw_spec$freq[1:1000]vw_y=vw_spec$spec[1:1000]ew_x=ew_spec$freq[1:1000]ew_y=ew_spec$spec[1:1000]scatter.smooth(vw_x,log(vw_y),span=1/15,ylab="",xlab="",col=6,cetext(0.04,-7,"LogSpectralDensityofVW",cex=0.8)scatter.smooth(ew_x,log(ew_y),span=1/15,ylab="",xlab="",col=7,cetext(0.04,-7,"LogSpectralDensityofEW",cex=0.8)dev.off()####################################################################ThisistheExample2.16inChapter2#######################################phi=c(1.2,-0.35,1.0,-0.70,0.2,0.35,-0.2,0.35)dim(phi)=c(2,4)phi=t(phi)postscript(file="c:\teaching\timeseries\figs\fig-2.19.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4,bg="darkgrey")for(jin1:4){rho=rep(0,20)rho[1]=1rho[2]=phi[j,1]/(1-phi[j,2]) CHAPTER3.UNIVARIATETIMESERIESMODELS175for(iin3:20){rho[i]=phi[j,1]*rho[i-1]+phi[j,2]*rho[i-2]}plot(1:20,rho,type="h",ylab="",ylim=c(-1,1),xlab="")if(j==1){title(main="(a)",cex=0.8)}if(j==2){title(main="(b)",cex=0.8)}if(j==3){title(main="(c)",cex=0.8)}if(j==4){title(main="(d)",cex=0.8)}abline(0,0)}dev.off()z1<-matrix(scan("c:\teaching\timeseries\data\q-gnp4791.txt"),byrow=T,ncol=1)n=length(z1)x=1:nx=x/4+1946.25postscript(file="c:\teaching\timeseries\figs\fig-2.20.eps",horizontal=F,width=6,height=6)par(mfrow=c(1,2),mex=0.4,bg="lightpink")plot(x,z1,type="o",ylab="",xlab="")abline(0,0)acf(z1,main="",ylab="",xlab="",lag=30)dev.off()#################################################################ThisisformakinggraphsforIRF################################################n=100w_t1=rnorm(n,0,1)w_t2=w_t1w_t2[50]=w_t1[50]+1 CHAPTER3.UNIVARIATETIMESERIESMODELS176y1=rep(0,2*(n+1))dim(y1)=c(n+1,2)y1[1,]=c(5,5)y2=y1phi1=c(0.8,-0.8)for(iin2:(n+1)){y1[i,1]=phi1[1]*y1[(i-1),1]+w_t1[i-1]y1[i,2]=phi1[1]*y1[(i-1),2]+w_t2[i-1]y2[i,1]=phi1[2]*y2[(i-1),1]+w_t1[i-1]y2[i,2]=phi1[2]*y2[(i-1),2]+w_t2[i-1]}y1=y1[2:101,]y2=y2[2:101,]irf1=y1[,2]-y1[,1]irf2=y2[,2]-y2[,1]text1=c("NoImpulse","WithImpulse")postscript(file="c:\teaching\timeseries\figs\fig-2.21.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4,bg="darkgrey")ts.plot(y1,type="l",lty=1,col=c(1,2),ylab="",xlab="")abline(0,0)legend(40,0.9*max(y1[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.text(40,0.8*min(y1[,1]),"phi=0.8")ts.plot(y2,type="l",lty=1,col=c(1,2),ylab="",xlab="")abline(0,0)legend(40,0.9*max(y2[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.text(60,0.8*min(y2[,1]),"phi=-0.8")plot(1:n,irf1,type="l",ylab="",ylim=c(-1,1),xlab="")abline(0,0)text(40,-0.6,"ImpulseResponseFunction",cex=0.8) CHAPTER3.UNIVARIATETIMESERIESMODELS177text(20,0.8,"phi=0.8")plot(1:n,irf2,type="l",ylab="",ylim=c(-1,1),xlab="")abline(0,0)text(40,-0.6,"ImpulseResponseFunction",cex=0.8)text(20,0.8,"phi=-0.8")dev.off()n=100w_t1=rnorm(n,0,1)w_t2=w_t1w_t2[50]=w_t1[50]+1y1=rep(0,2*(n+1))dim(y1)=c(n+1,2)y1[1,]=c(3,3)y2=y1phi1=c(1.01,-1.01)for(iin2:(n+1)){y1[i,1]=phi1[1]*y1[(i-1),1]+w_t1[i-1]y1[i,2]=phi1[1]*y1[(i-1),2]+w_t2[i-1]y2[i,1]=phi1[2]*y2[(i-1),1]+w_t1[i-1]y2[i,2]=phi1[2]*y2[(i-1),2]+w_t2[i-1]}y1=y1[2:101,]y2=y2[2:101,]irf1=y1[,2]-y1[,1]irf2=y2[,2]-y2[,1]text1=c("NoImpulse","WithImpulse")postscript(file="c:\teaching\timeseries\figs\fig-2.22.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4,bg="lightpink") CHAPTER3.UNIVARIATETIMESERIESMODELS178ts.plot(y1,type="l",lty=1,col=c(1,2),ylab="",xlab="")abline(0,0)legend(40,0.9*max(y1[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.text(40,0.8*min(y1[,1]),"phi=1.01")ts.plot(y2,type="l",lty=1,col=c(1,2),ylab="",xlab="")abline(0,0)legend(40,0.9*max(y2[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.text(60,0.8*min(y2[,1]),"phi=-1.01")plot(1:n,irf1,type="l",ylab="",ylim=c(-2,2),xlab="")abline(0,0)text(40,-0.6,"ImpulseResponseFunction",cex=0.8)text(20,0.8,"phi=1.01")plot(1:n,irf2,type="l",ylab="",ylim=c(-2,2),xlab="")abline(0,0)text(40,-0.6,"ImpulseResponseFunction",cex=0.8)text(20,0.8,"phi=-1.01")dev.off()x=1:20phi1=cbind(0.5^x,0.9^x,0.99^x)phi2=cbind((-0.5)^x,(-0.9)^x,(-0.99)^x)postscript(file="c:\teaching\timeseries\figs\fig-2.23.eps",horizontal=F,width=6,height=6)#win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightgreen")matplot(x,phi1,type="o",lty=1,pch="o",ylab="",xlab="")text(4,0.3,"phi1=0.5",cex=0.8)text(15,0.3,"phi1=0.9",cex=0.8)text(15,0.9,"phi1=0.99",cex=0.8)matplot(x,phi2,type="o",lty=1,pch="o",ylab="",xlab="") CHAPTER3.UNIVARIATETIMESERIESMODELS179abline(0,0)text(4,0.3,"phi1=-0.5",cex=0.8)text(15,0.3,"phi1=-0.9",cex=0.8)text(15,0.9,"phi1=-0.99",cex=0.8)matplot(x,1.2^x,type="o",lty=1,pch="o",ylab="",xlab="")text(14,22,"phi1=1.2",cex=0.8)matplot(x,(-1.2)^x,type="o",lty=1,pch="o",ylab="",xlab="")abline(0,0)text(13,18,"phi1=-1.2",cex=0.8)dev.off()n=100w_t1=rnorm(n,0,1)w_t2=w_t1w_t3=w_t1w_t2[50]=w_t1[50]+1w_t3[50:n]=w_t1[50:n]+1y=rep(0,3*(n+1))dim(y)=c(n+1,3)y[1,]=c(3,3,3)phi1=0.8for(iin2:(n+1)){y[i,1]=phi1*y[(i-1),1]+w_t1[i-1]y[i,2]=phi1*y[(i-1),2]+w_t2[i-1]y[i,3]=phi1*y[(i-1),3]+w_t3[i-1]}y=y[2:101,1:3]irf1=y[,2]-y[,1]irf2=y[,3]-y[,1]text1=c("NoImpulse","WithImpulse") CHAPTER3.UNIVARIATETIMESERIESMODELS180postscript(file="c:\teaching\timeseries\figs\fig-2.24.eps",horizontal=F,width=6,height=6)#win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightblue")ts.plot(y[,1:2],type="l",lty=1,col=c(1,2),ylab="",xlab="")abline(0,0)legend(40,0.9*max(y[,1]),text1,lty=c(1,2),col=c(1,2),cex=0.8text(40,0.8*min(y[,1]),"phi=0.8")ts.plot(cbind(y[,1],y[,3]),type="l",lty=1,col=c(1,2),ylab="",xabline(0,0)legend(40,0.9*max(y[,3]),text1,lty=c(1,2),col=c(1,2),cex=0.8text(10,0.8*min(y[,3]),"phi=0.8")plot(1:n,irf1,type="l",ylab="",xlab="")abline(0,0)text(40,0.6,"ImpulseResponseFunction",cex=0.8)text(20,0.8,"phi=0.8")plot(1:n,irf2,type="l",ylab="",xlab="")abline(0,0)text(40,3,"ImpulseResponseFunction",cex=0.8)text(20,0.8,"phi=0.8")dev.off()ff=c(0.6,0.2,1,0,0.8,0.4,1,0,-0.9,-0.5,1,0,-0.5,-1.5,1,0)dim(ff)=c(4,4)mj=20x=0:mjirf=rep(0,(mj+1)*4)dim(irf)=c(mj+1,4)irf[1,]=1 CHAPTER3.UNIVARIATETIMESERIESMODELS181for(jin1:4){aa=c(1,0,0,1)dim(aa)=c(2,2)ff1=ff[,j]dim(ff1)=c(2,2)ff1=t(ff1)for(iin1:mj){aa=aa%*%ff1irf[i+1,j]=aa[1,1]}}postscript(file="c:\teaching\timeseries\figs\fig-2.25.eps",horizontal=F,width=6,height=6)#win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightyellow")plot(x,irf[,1],type="o",pch="o",ylab="",ylim=c(0,1),xlab="",main="(a)",cex=0.8)text(10,0.8,"ImpulseResponseFunction",cex=0.8)text(12,0.3,"phi1=0.6andphi2=0.2",cex=0.8)plot(x,irf[,2],type="o",pch="o",ylab="",ylim=c(0,12),xlab="",main="(b)",cex=0.8)text(10,10,"ImpulseResponseFunction",cex=0.8)text(9,6,"phi1=0.8andphi2=0.4",cex=0.8)plot(x,irf[,3],type="o",pch="o",ylab="",ylim=c(-1,1),xlab="",main="(c)",cex=0.8)abline(0,0)text(10,0.8,"ImpulseResponseFunction",cex=0.8)text(11,-0.3,"phi1=-0.9andphi2=-0.5",cex=0.8)plot(x,irf[,4],type="o",pch="o",ylab="",ylim=c(-40,30),xlab="",main="(d)",cex=0.8)abline(0,0) CHAPTER3.UNIVARIATETIMESERIESMODELS182text(10,25,"ImpulseResponseFunction",cex=0.8)text(9,-16,"phi1=-0.5andphi2=-1.5",cex=0.8)dev.off()###################################################################3.16ReferencesAkaike,H.(1973).Informationtheoryandanextensionofthemaximumlikelihoodprinci-ple.InProceedingof2ndInternationalSymposiumonInformationTheory(V.PetrovandF.Cs´aki,eds.)267281.Akad´emiaiKiad´o,Budapest.Andrews,D.W.K.(1991).Heteroskedasticityandautocorrelationconsistentcovariancematrixestimation.Econometrica,59,817-858.Bai,Z.,C.R.RaoandY.Wu(1999).Modelselectionwithdata-orientedpenalty.JournalofStatisticalPlanningandInferences,77,103-117.Beran,J.(1994).StatisticsforLong-MemoryProcesses.ChapmanandHall,London.Box,G.E.P.andJenkins,G.M.(1970).TimeSeriesAnalysis,Forecasting,andControl.HoldenDay,SanFrancisco.Box,G.E.P.,G.M.JenkinsandG.C.Reinsel(1994).TimeSeriesAnalysis,ForecastingandControl.3thEdn.EnglewoodCliffs,NJ:Prentice-Hall.Brockwell,P.J.andDavis,R.A.(1991).TimeSeriesTheoryandMethods.NewYork:Springer.Burman,P.andR.H.Shumway(1998).Semiparametricmodelingofseasonaltimeseries.JournalofTimeSeriesAnalysis,19,127-145.Burnham,K.P.andD.Anderson(2003).ModelSelectionAndMulti-ModelInference:APracticalInformationTheoreticApproach,2ndedition.NewYork:Springer-Verlag.Cai,Z.andR.Chen(2006).Flexibleseasonaltimeseriesmodels.AdvancesinEconomet-rics,20B,63-87.Chan,N.H.andC.Z.Wei(1988).Limitingdistributionsofleastsquaresestimatesofunstableautoregressiveprocesses.AnnalsofStatistics,16,367-401.Cochrane,D.andG.H.Orcutt(1949).Applicationsofleastsquaresregressiontorelation-shipscontainingautocorrelatederrors.JournaloftheAmericanStatisticalAssociation,44,32-61. CHAPTER3.UNIVARIATETIMESERIESMODELS183Ding,Z.,C.W.J.GrangerandR.F.Engle(1993).Alongmemorypropertyofstockreturnsandanewmodel.JournalofEmpiricalFinance,1,83-106.Eicker,F.(1967).Limittheoremsforregressionwithunequalanddependenterrors.InProceedingsofthe5thBerkeleySymposiumonMathematicalStatisticsandProbability(L.LeCamandJ.Neyman,eds.),UniversityofCaliforniaPress,Berkeley.FanandLi(2001).Variableselectionvianonconcavepenalizedlikelihoodanditsoracleproperties.JournaloftheAmericanStatisticalAssociation,96,1348-1360.FanandPeng(2004).Nonconcavepenalizedlikelihoodwithadivergingnumberofparam-eters.AnnalsofStatistics,32,928-961.Frank,I.E.andJ.H.Friedman(1993).Astatisticalviewofsomechemometricregressiontools(withdiscussion).Technometrics,35,109-148.Franses,P.H.(1996).PeriodicityandStochasticTrendsinEconomicTimeSeries.NewYork:CambridgeUniversityPress.Franses,P.H.(1998).TimeSeriesModelsforBusinessandEconomicForecasting.NewYork:CambridgeUniversityPress.Franses,P.H.andD.vanDijk(2000).NonlinearTimeSeriesModelsforEmpiricalFinance.NewYork:CambridgeUniversityPress.Ghysels,E.andD.R.Osborn(2001).TheEconometricAnalysisofSeasonalTimeSeries.NewYork:CambridgeUniversityPress.Hurvich,C.M.andC.-L.Tsai(1989).Regressionandtimeseriesmodelselectioninsmallsamples.Biometrika,76,297-307.Newey,W.K.andK.D.West(1987).Asimple,positive-definite,heteroskedasticityandautocorrelationconsistentcovariancematrix.Econometrica,55,703-708.Schwarz,G.(1978).Estimatingthedimensionofamodel.TheAnnalsofStatistics,6,461-464.Shao(1993).Linearmodelselectionbycross-validation.JournaloftheAmericanStatisticalAssociation,88,486-494.Shen,X.T.andJ.M.Ye(2002).Adaptivemodelselection.JournaloftheAmericanSta-tisticalAssociation,97,210-221.Shumway,R.H.(1988).AppliedStatisticalTimeSeriesAnalysis.EnglewoodCliffs,NJ:Prentice-Hall.Shumway,R.H.,A.S.AzariandY.Pawitan(1988).ModelingmortalityfluctuationsinLosAngelesasfunctionsofpollutionandweathereffects.EnvironmentalResearch,45,224-241. CHAPTER3.UNIVARIATETIMESERIESMODELS184Shumway,R.H.andD.S.Stoffer(2000).TimeSeriesAnalysis&ItsApplications.NewYork:Springer-Verlag.Tiao,G.C.andR.S.Tsay(1983).ConsistencypropertiesofleastsquaresestimatesofautoregressiveparametersinARMAmodels.AnnalsofStatistics,11,856-871.Tibshirani,R.J.(1996).RegressionshrinkageandselectionviatheLasso.JournaloftheRoyalStatisticalSociety,SeriesB,58,267-288.Tsay,R.S.(2005).AnalysisofFinancialTimeSeries,2thEdition.JohnWiley&Sons,NewYork.Walker,G.(1931).Ontheperiodicityinseriesofrelatedterms.ProceedingsoftheRoyalSocietyofLondon,SeriesA,131,518-532.Yule,G.U.(1927).OnamethodofinvestigatingperiodicitiesindisturbedserieswithspecialreferencetoWolfer’sSunspotnumbers.PhilosophicalTransactionsoftheRoyalSocietyofLondon,SeriesA,226,267-298. Chapter4Non-stationaryProcessesandStructuralBreaks4.1IntroductionInouranalysissofar,wehaveassumedthatthevariablesinthemod-elsthatwehaveanalyzed,univariateARMAmodels,arestationaryyt=µ+ψ(B)et,whereytmaybeavectorofnvariablesattimeperiodt.Aglanceatgraphsofmosteconomictimeseriessufficestorevealinvalidityofthatassumption,becauseeconomiesevolve,grow,andchangeovertimeinbothrealandnominaltermsandeconomicforecastsoftenareverywrong,althoughthattheyshouldoccurrelativelyinfrequentlyinastationaryprocess.Thepracticalproblemthataneconometricianfacesistofindanyrelationshipsthatsurviveforrelativelylongperiodoftimesothattheycanbeusedforforecastingandpolicyanalysis.HendryandJuselius(2000)pointedoutthatfourissuesimmediatelyariseintheissueofnonstationarity:1.Howimportantistheassumptionofstationarityformod-elingandinference?Itisveryimportant.Whendata185 CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS186meansandvariancesarenon-constant,observationscomefromdifferentdistributionsovertime,creatingdifficultprob-lemsforempiricalmodeling.2.Whatistheeffectofincorrectlyassumingit?Itispoten-tiallyhazardous.Assumingconstantmeansandvari-anceswhenthatisfalsecaninduceseriousstatisticalmis-takes.Ifthevariablesinytarenotstationary,thencon-ventionalhypothesistests,confidenceintervalsandforecastscanbeunreliable.Standardasymptoticdistributiontheoryoftendoesnotapplytoregressionsinvolvingvariableswithunitroots,andinferencemaybemisleadingifthisisignored.3.Whatarethesourcesofnon-stationarity?Therearemanyandtheyarevaried.Non-stationaritymaybeduetoevolutionoftheeconomy,legislativechanges,technologicalchanges,politicalevents,etc.4.Canempiricalanalysisbetransformedsostationaritybe-comesavalidassumption?Itissometimespossible,dependingonthesourceofnon-stationarity.Someofnon-stationaritycanbeeliminatedbytransformationssuchasde-trendinganddifferencing,orsomeothertypetransformations;seeParkandPhillips(1999)andChang,ParkandPhillips(2001)formorediscussions.TrendingTimeSeriesAnon-stationaryprocessisonewhichviolatesthestationaryre-quirement,soitsmeansandvariancesarenon-constantovertime.Atrendisapersistentlong-termmovementofavariableovertime CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS187andatimeseriesfluctuatesarounditstrend.Thecommonfeaturesofatrendingtimeseriescanbesummarizedasthefollowingsituations:1.Stochastictrends.Astochastictrendisrandomandvariesovertime:(1−L)yt=δ+ψ(L)et.Thatis,aftertakingthedifference,thetimeseriesismod-elledasalinearprocess.2.Deterministictrends.Adeterministictrendisanon-randomfunctionoftime.Forexample,adeterministictrendmightbelinearorquadraticorhigherorderintime:yt=a+δt+ψ(L)et.Thedifferencebetweenalinearstochastictrendandadeterministictrendisthatthechangesofastochastictrendarerandom,whereasthoseofadeterministictrendareconstantovertime.3.Permanenceofshocks.Macroeconomistsusedtode-trenddataandregardedbusinesscyclesasthestationaryde-viationsfromthattrend.EconomistsinvestigatedwhetherGNPisbetterdescribedasrandomwalkortruestationaryprocess.4.Statisticalissues.Wecouldhavemistakenatimeserieswithunitrootsfortrendstationarytimeseriessinceatimeserieswithunitrootsmightdisplayatrendingphenomena.Thetrendingtimeseriesmodelshavegainedalotofattentionduringthelasttwodecadesduetomanyapplicationsineconomics CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS188andfinance.SeeCai(2006)formorereferences.Followingaresomeexamples.Themarketmodelinfinanceisanexamplethatrelatesthereturnofanindividualstocktothereturnofamarketindexoranotherindividualstockandthecoefficientusuallyiscalledabeta-coefficientinthecapitalassetpricingmodel(CAPM);seethebooksbyCochrane(2001)andTsay(2005)formoredetails.However,somerecentstudiesshowthatthebeta-coefficientsmightvaryovertime.Thetermstructureofinterestratesisanotherexampleinwhichthetimeevolutionoftherelationshipbetweeninterestrateswithdifferentmaturitiesisinvestigated;seeTsay(2005).Thelastexampleistherelationshipbetweentheprototypeelectricitydemandandothervariablessuchastheincomeorproduction,therealpriceofelectricity,andthetemperature,andChangandMartinez-Chombo(2003)foundthatthisrelationshipmaychangeovertime.Althoughtheliteratureisalreadyvastandcontinuestogrowswiftly,aspointedoutbyPhillips(2001),theresearchinthisareaisjustbeginning.4.2RandomWalksThebasicrandomwalkisyt=yt−1+etwithEt−1(et)=0,whereEt−1(·)istheconditionalexpectationgiventhepastinformationuptotimet−1,whichimpliesthatEt(yt+1)=yt.Randomwalkshaveanumberofinterestingproperties:1.Theimpulse-responsefunction(IRF)ofrandomwalkisoneatallhorizons.TheIRFofastationaryprocessesdiesouteventually. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS1892.Theforecastvarianceoftherandomwalkgrowslinearlywiththeforecasthorizon:Var(y|y)=Var(y−y)=kσ2t+ktt+kte3.TheautocovariancesofrandomwalkaredefinedinSection3.4.StatisticalIssuesSupposeaseriesisgeneratedbyarandomwalk:yt=yt−1+et.Youmighttestforarandomwalkbyrunningyt=µ+φyt−1+etbytheordinaryleastsquare(OLS)andtestingwhetherφ=1.How-ever,thisisnotcorrectsincetheassumptionsunderlyingtheusualasymptotictheoryforOLSestimatesandteststatisticsareviolated.Exercise:WhyistheasymptotictheoryforOLSestimatesnottrue?4.2.1InappropriateDetrendingThingsgetevenmorecomplicatedwithatrendinthemodel.Sup-posethetruemodelisyt=µ+yt−1+et.SupposeyoudetrendthemodelandthenfitanAR(1)model,i.e.thefittingmodelis:(1−φL)(yt−bt)=et. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS190Buttheabovemodelcanbewrittenasfollows:yt=α+γt+φyt−1+et,soyoucouldalsorundirectlyyonatimetrendandlaggedy.Incthiscase,φisbiaseddownward(under-estimate)andthestandardOLSerrorsaremisleading.4.2.2Spurious(nonsense)RegressionsSupposetwoseriesaregeneratedbyrandomwalks:yt=yt−1+et,xt=xt−1+vt,E(etvs)=0,forallt,s.Now,supposeyourunytonxtbyOLS:yt=α+βxt+ut.Theassumptionsforclassicalregressionareviolatedandwetendtosee“significant”βmoreoftenthanOLSformulassayweshould.Exercise:PleaseconductaMonteCarolsimulationtoverifytheaboveconclusion.4.3UnitRootandStationaryProcessesAmoregeneralprocessthanpurerandomwalkmayhavethefollow-ingform:(1−L)yt=µ+ψ1(L)et.Thesearecalledunitrootordifferencestationary(DS)processes.Inthesimplestcaseψ1(L)=1,theDSprocessbecomesarandomwalkwithdrift:yt=µ+yt−1+et. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS191Alternatively,wemayconsideraprocesstobestationaryaroundalineartrend:yt=µt+ψ2(L)et.Thisprocessiscalledtrendingstationary(TS)process.TheTSprocesscanbeconsideredasaspecialcaseofDSmodel.Indeed,wecanwritetheTSmodelas:(1−L)yt=(1−L)µt+(1−L)ψ2(L)et=µ+ψ1(L)et.Therefore,ifTSmodeliscorrect,theDSmodelisstillvalidandstationary.FormorestudiesontheTSmodels,wereferthereadertothepapersbyPhillips(2001)andCai(2006).Onecanthinkaboutunitrootsasthestudyoftheimplicationsforlevelsofaprocessthatisstationaryindifferences.Therefore,itisveryimportanttokeeptrackofwhetheryouarethinkingabouttheleveloftheprocessytoritsfirstdifference.Letusexaminetheimpulseresponsefunction(IRF)fortheabovetwomodels.FortheTSmodel,theIRFisdeterminedbyMApoly-nomialψ2(L),i.e.bjisthej-thperiodaheadresponse.FortheDSmodel,ajgivestheresponseofthedifference(1−L)yt+jtoashockattimet.Theresponseoftheleveloftheseriesyt+jisthesumoftheresponseofthedifferences.Responseofyt+jtoshockatperiodtis:IRFj=(yt−yt−1)+(yt+1−yt)+···+(yt+j−yt+j−1)=a0+a1+···+aj.4.3.1ComparisonofForecastsofTSandDSProcessesTheforecastofatrend-stationaryprocessisasfollows:tybt+s=µ(t+s)+bset+bs+1et−1+bs+2et−2+···. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS192Theforecastofadifferencestationaryprocesscanbewrittenasfol-lows:ttttybt+s=∆ybt+s+∆ybt+s−1+···+∆ybt+1+yt=(µ+bset+bs+1et−1+bs+2et−2+···)+(µ+bs−1et+bset−1+bs+1et−2+···)+···+(µ+b1et+b2et−1+b3et−2+···)+ytortybt+s=µs+yt+(bs+bs−1···+b1)et+(bs+1+bs+···+b2)et−1+···ToseethedifferencebetweenforecastsforTSandDSprocesses,weconsideracaseinwhichb1=b2=···=0.Then,tTS:ybt+s=µ(t+s)tDS:ybt+s=µs+ytNext,comparetheforecasterrorfortheTSandDSprocesses.FortheTSprocess:tyt+s−ybt+s=(µ(t+s)+et+s+b1et+s−1+b2et+s−2+···+bs−1et+1+bset+−(µ(t+s)+bset+bs+1et−1+···)=et+s+b1et+s−1+···+bTheMSEofthisforecastis2t2222Eyt+s−ybt+s=(1+b1+b2+···+bs−1)σ.FortheDSprocess:tttyt+s−ybt+s=(∆yt+s+···+∆yt+1+yt)−(∆ybt+s+···+∆ybt+1+yt)=et+s+(1+b1)et+s−1+(1+b1+b2)et+s−2+···+(1+b1+b2TheMSEofthisforecastis2t22Eyt+s−ybt+s=(1+b1)+(1+b1+b2)+···+(1+b1+b2+···+bs−1 CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS193NotethatfortheMSEforaTSprocessastheforecastinghorizonincreases,thoughassbecomeslarge,theaddeduncertaintyfromforecastingintofuturebecomesnegligible:2t222limEyt+s−ybt+s=(1+b1+b2+···)σs→∞andthelimitingMSEisjusttheunconditionalvarianceofthesta-tionarycomponentψ2(L).ThisisnottruefortheDSprocess.TheMSEforaDSprocessdoesnotconvergetoanyfixedvalueassgoestoinfinity.Tosummarize,foraTSprocess,theMSEreachesafiniteboundastheforecasthorizonbecomeslarge,whereasforaunitrootprocess,theMSEeventuallygrowslinearlywiththeforecasthorizon.4.3.2RandomWalkComponentsandStochasticTrendsItiswellknownthatanyDSprocesscanbewrittenasasumofarandomwalkandastationarycomponent.AdecompositionwithanicepropertyisduetotheBeveridge-Nelson(1981,BN).If(1−L)yt=µ+ψ1(L)et,thenwecanwriteyt=ct+zt,wherezt=∗∗Pµ+zt−1+ψ1(1)etandct=ψ1(L)etwithaj=k>jak.Toseewhythisdecompositionistrue,weneedtonoticethatanylagpolynomialψ(L)canbewrittenasψ(L)=ψ(1)+(1−L)ψ∗(L),where1111∗Paj=k>jak.Toseethis,justwriteitout:ψ1(1):a0+a1+a2+a3···(1−L)ψ∗(L):−a−a−a···1123+a1L+a2L+a3L···−a2L−a3L·········andthetermsψ1(L)remainwhenyoucancelalltheterms.Therearemanywaystodecomposeaunitrootintostationaryandrandomwalkcomponents.TheBNdecompositionisapopularchoicebecause CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS194ithasaspecialproperty:therandomwalkcomponentisasensibledefinitionofthe“trend”inyt.Thecomponentztisthelimitingforecastoffuturey,i.e.today’syplusallfutureexpectedchangesiny.IntheBNdecompositiontheinnovationstothestationaryandrandomwalkcomponentsareperfectlycorrelated.Consideranarbitrarycombinationofstationaryandrandomwalkcomponents:yt=zt+ct,wherezt=µ+zt−1+vtandct=ψ2(L)et.Itcanbeshownthatineverydecompositionofytintostationaryandrandomwalkcomponents,thevarianceofchangestotherandomwalkcom-ponentisthesame,ψ(1)2σ2.Sincetheunitrootprocessiscomposed1eofastationaryplusrandomwalkcomponent,theunitrootprocesshasthesamevarianceofforecastsbehaviorastherandomwalkwhenthehorizonislongenough.4.4TrendEstimationandForecasting4.4.1ForecastingaDeterministicTrendConsiderthelinerdeterministicmodel:yt=α+βt+et,t=1,2,...,T.tcTheh-step-aheadforecastisgivenbyyt+h=αc+β(t+h),wherecαcandβaretheOLSestimatesoftheparametersαandβ.Theforecastvariancemaybecomputedusingthefollowingformula"#12t+h−(t+1)/2t22Eyt+h−yt+h=σ1++Pt2≈σ,tm=1(m−(t+1)/2)wherethelastapproximationisvalidift(theperiodatwhichforecastisconstructed)islargerelativetotheforecasthorizonh. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS1954.4.2ForecastingaStochasticTrendConsidertherandomwalkwithdriftyt=α+yt−1+et,t=2,3,...,T.Letαcbeanestimateofaobtainedfromthefollowingregressionmodel:∆yt=α+et.Theh-step-aheadforecastisgivenbyyt=y+αhc.Theforecastt+htvariancemaybecomputedusingthefollowingformula"#h22t22Eyt+h−yt+h=σh+≈hσ,t−1wherethelastapproximationisvalidiftislargerelativetoh.4.4.3ForecastingARMAmodelswithDeterministicTrendsThebasicmodelsfordeterministicandstochastictrendignorepossi-bleshort-runfluctuationsintheseries.ConsiderthefollowingARMAmodelwithdeterministictrendψ(L)yt=α+βt+θ(L)et,wherethepolynomialφ(L)satisfiesthestationarityconditionandθ(L)satisfiestheinvertibilitycondition.Theforecastisconstructedasfollows:1.Lineardetrending.Estimatethefollowingregressionmodel:yt=δ1+δ2t+zet,bbandcomputezt=yt−δ1−δ2t. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS1962.EstimateanappropriateARMA(p,q)modelforcovariancestationaryvariablezt:ψ(L)zt=θ(L)et.TheestimatedARMA(p,q)modelmaybeusedtoconstructforecastsh-period-aheadforecastsofz,zt.tt+h3.Constructtheh-period-aheadforecastofyasfollows:yt=tt+htbbzt+h+δ1+δ2(t+h).TheMSEofytmaybeapproximatedbytheMSEofzt.t+ht+h4.4.4ForecastingofARIMAModelsConsidertheforecastingoftimeseriesthatareintegratedoforder1thataredescribedbytheARIMA(p,1,q)model:φ(L)(1−L)yt=α+θ(L)et,wherethepolynomialφ(L)satisfiesthestationarityconditionandθ(L)satisfiestheinvertibilitycondition.Theforecastisconstructedasfollows:1.Computethefirstdifferenceofyt,i.e.zt=∆yt.2.EstimateanappropriateARMA(p,q)modelforcovariancestationaryvariablezt:φ(L)zt=θ(L)et.TheestimatedARMA(p,q)modelmaybeusedtoconstructforecastsh-period-aheadforecastsofz,zt.tt+h3.Constructtheh-period-aheadforecastofytasfollows:ttyt+h=yt+zt+h CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS1974.5UnitRootTestsAlthoughitmightbeinterestingtoknowwhetheratimeserieshasaunitroot,severalpapershavearguedthatthequestioncannotbeansweredonthebasisofafinitesampleofobservations.Nevertheless,youwillhavetoconducttestofunitrootindoingempiricalprojects.Itcanbedoneusinginformalorinformalmethods.Theinformalmethodsinvolveinspectingatimeseriesplotofthedataandcom-putingtheautocorrelationcoefficients,aswhatwedidinChapters1and2.Ifaserieshasastochastictrend,thefirstautocorrelationcoefficientwillbenearone.Asmallfirstautocorrelationcoefficientcombinedwithatimeseriesplotthathasnoapparenttrendsuggeststhattheseriesdoesnothaveatrend.Dickey-Fuller’s(1979,DF)testisamostpopularformalstatisticalprocedureforunitroottesting.4.5.1TheDickey-FullerandAugmentedDickey-FullerTestsThestartingpointfortheDFtestistheautoregressivemodeloforderone,AR(1):yt=α+ρyt−1+et.(4.1)Ifρ=1,ytisnonstationaryandcontainsastochastictrend.There-fore,withintheAR(1)model,thehypothesisthatythasatrendcanbetestedbytesting:H0:ρ=1vs.H1:ρ<1.Thistestismosteasilyimplementedbyestimatingamodifiedversionof(4.1).Subtractyt−1frombothsidesandletδ=ρ−1.Then,model(4.1)becomes:∆yt=α+δyt−1+et(4.2) CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS198Table4.1:Large-samplecriticalvaluesfortheADFstatisticDeterministicregressors10%5%1%Interceptonly-2.57-2.86-3.43Interceptandtimetrend-3.12-3.41-3.96andthetestinghypothesisis:H0:δ=0vs.H1:δ<0.TheOLSt-statisticin(4.2)testingδ=0isknownastheDickey-Fullerstatistic.TheextensionoftheDFtesttotheAR(p)modelisatestofthenullhypothesisH0:δ=0againsttheone-sidedalternativeH1:δ<0inthefollowingregression:∆yt=α+δyt−1+γ1∆yt−1+···+γp∆yt−p+et.(4.3)Underthenullhypothesis,ythasastochastictrendandunderthealternativehypothesis,ytisstationary.Ifinsteadthealternativehypothesisisthatytisstationaryaroundadeterministiclineartimetrend,thenthistrendmustbeaddedasanadditionalregressorinmodel(4.3)andtheDFregressionbecomes∆yt=α+βt+δyt−1+γ1∆yt−1+···+γp∆yt−p+et.(4.4)ThisiscalledtheaugmentedDickey-Fuller(ADF)testandtheteststatisticistheOLSt-statistictestingthatδ=0inequation(4.4).TheADFstatisticdoesnothaveanormaldistribution,eveninlargesamples.Criticalvaluesfortheone-sidedADFtestdependonwhetherthetestisbasedonequation(4.3)or(4.4)andaregiveninTable4.1.Table17.1ofHamilton(1994,p.502)presentsasummaryofDFtestsforunitrootsintheabsenceofserialcorrelationfortesting CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS199Table4.2:SummaryofDFtestforunitrootsintheabsenceofserialcorrelationCase1:Trueprocess:yt=yt−1+ut,ut∼N(0,σ2)iid.Estimatedregression:yt=ρyt−1+ut.T(ρb−1)hasthedistributiondescribedundertheheadingCase1inTableB.5.(ρb−1)/σb2hasthedistributiondescribedunderCase1inTableB.6.ρbCase2:Trueprocess:yt=yt−1+ut,ut∼N(0,σ2)iid.Estimatedregression:yt=α+ρyt−1+ut.T(ρb−1)hasthedistributiondescribedunderCase2inTableB.5.(ρb−1)/σ2hasthedistributiondescribedunderCase2inTableB.6.ρbOLSF-testofjoinhypothesisthatα=0andρ=1hasthedistributiondescribedunderCase2inTableB.7.Case3:Trueprocess:yt=α+yt−1+ut,α6=0,ut∼N(0,σ2)iid.Estimatedregression:yt=α+ρyt−1+ut.(ρb−1)/σ2→N(0,1).ρbCase4:Trueprocess:yt=α+yt−1+ut,α6=0,ut∼N(0,σ2)iid.Estimatedregression:yt=α+ρyt−1+δt+ut.T(ρb−1)hasthedistributiondescribedunderCase4inTableB.5.(ρb−1)/σ2hasthedistributiondescribedunderCase4inTableB.6.ρbOLSF-testofjoinhypothesisthatρ=1andδ=0hasthedistributiondescribedunderCase4inTableB.7.thenullhypothesisofunitrootagainstsomedifferentalternativehypothesis.Itisveryimportantforyoutounderstandwhatyouralternativehypothesisisinconductingunitroottests.Ireproducethistablehere,butyouneedtocheckHamilton’s(1994)bookforthecriticalvaluesofDFstatisticfordifferentcases.ThecriticalvaluesarepresentedintheAppendixofthebook.Intheabovemodels(4cases),thebasicassumptionisthatutisiid.Butthisassumptionisviolatedifutisseriallycorrelatedandpotentiallyheteroskedastic.Totakeaccountofserialcorrelationandpotentialheteroskedasticity,onewayistousethePPtestpro-posedbyPhillipsandPerron(1988).Forothertestsforunitroots,pleasereadthebookbyHamilton(1994,p.532).Somerecenttesting CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS200methodshavebeenproposed.Forexample,Juhl(2005)usedthefunctionalcoefficienttypemodelofCai,FanandYao(2000)totestunitrootandPhillipsandPark(2005)employedthenonparametricregression.Finally,noticethatinR,thereareatleastthreepackagestoprovideunitroottestssuchastseries,urcaanduroot.4.5.2CautionsThemostreliablewaytohandleatrendinaseriesistotransformtheseriessothatitdoesnothavethetrend.Iftheserieshasastochastictrend,unitroot,thenthefirstdifferenceoftheseriesdoesnothaveatrend.Inpractice,youcanrarelybesurewhetheraserieshasastochastictrendornot.Recallthatafailuretorejectthenullhypothesisdoenotnecessarilymeanthatthenullhypothesisistrue;itsimplymeansthattherearenotenoughevidencetoconcludethatitisfalse.Therefore,failuretorejectthenullhypothesisofaunitrootusingtheADFtestdoesnotmeanthattheseriesactuallyhasaunitroot.Havingsaidthat,eventhoughfailuretorejectthenullhypothesisofaunitrootdoesnotmeantheserieshasaunitroot,itstillcanbereasonabletoapproximatethetrueautoregressiverootasequalingoneandusethefirstdifferenceoftheseriesratherthanitslevels.4.6StructuralBreaksAnothertypeofnonstationarityariseswhenthepopulationregres-sionfunctionchangesoverthesampleperiod.Thismayoccurbe-causeofchangesineconomicpolicy,changesinthestructureoftheeconomyorindustry,eventsthatchangethedynamicsofspecificindustriesorfirmrelatedquantities(inventories,sales,production), CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS201etc.Ifsuchchanges,calledbreaks,occurthenregressionmodelsthatneglectthosechangesleadtoamisleadinginferenceorforecast-ing.Breaksmayresultfromadiscretechange(orchanges)inthepop-ulationregressioncoefficientsatdistinctdatesorfromagradualevo-lutionofthecoefficientsoveralongerperiodoftime.Discretebreaksmaybearesultofsomemajorchangesineconomicpolicyorintheeconomy(oilshocks)while“gradual”breaks,populationparametersevolveslowlyovertime,maybearesultofslowevolutionofeconomicpolicy.Ifabreakoccursinthepopulationparametersduringthesample,thentheOLSregressionestimatesoverthefullsamplewillestimatearelationshipthatholdson“average”.4.6.1TestingforBreaksTestsforbreaksintheregressionparametersdependonwhetherthebreakdateisknowornot.Ifthedateofthehypothesizedbreakinthecoefficientsisknown,thenthenullhypothesisofnobreakcanbetestingusingadummyvariable.Considerthefollowingmodel:yt=β0+β1yt−1+δ1xt−1+γ0Dt(τ)+γ1Dt(τ)yt−1+γ2Dt(τ)xt−1+utβ0+β1yt−1+δ1xt−1+ut,ift≤τ,=(β+γ)+(β+γ)y+(δ+γ)x+u,ift>τ,0011t−112t−1twhereτdenotesthehypothesizedbreakdate,Dt(τ)isabinaryvariablethatequalszerobeforethebreakdateandoneafter,i.e.Dt(τ)=0ift≤τandDt(τ)=1ift>τ.Underthenullhypoth-esisofnobreak,γ0=γ1=γ2=0,andthehypothesisofabreak CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS202canbetestedusingtheF-statistic.ThisiscalledaChowtestforabreakataknownbreakdate.Indeed,theabovestructuralbreakmodelcanberegardedasaspecialcaseofthefollowingtrendingtimeseriesmodelyt=β0(t)+β1(t)yt−1+δ1(t)xt−1+ut.Formorediscussions,seeCai(2006).Iftherearemorevariablesormorelags,thistestcanbeextendedbyconstructingbinaryvariableinteractionvariablesforallthedepen-dentvariables.Thisapproachcanbemodifiedtocheckforabreakinasubsetofthecoefficients.Thebreakdateisunknowninmostoftheapplicationsbutyoumaysuspectthatabreakoccurredsome-timebetweentwodates,τ0andτ1.TheChowtestcanbemodifiedtohandlethisbytestingforbreakatallpossibledatestinbetweenτ0andτ1,thenusingthelargestoftheresultingF-statisticstotestforabreakatanunknowndate.ThismodifiedtestisoftencalledQuandtlikelihoodratio(QLR)statisticorthesup-Waldstatistic:QLR=max{F(τ0),F(τ0+1),···,F(τ1)}.SincetheQLRstatisticisthelargestofmanyF-statistics,itsdistri-butionisnotthesameasanindividualF-statistic.ThecriticalvaluesforQLRstatisticmustbeobtainedfromaspecialdistribution.Thisdistributiondependsonthenumberofrestrictionbeingtested,m,τ0,τ1,andthesubsampleoverwhichtheF-statisticsarecomputedexpressedasafractionofthetotalsamplesize.Forthelarge-sampleapproximationtothedistributionoftheQLRstatistictobeagoodone,thesubsampleendpoints,τ0andτ1,cannotbetooclosetotheendofthesample.ThatiswhytheQLRstatisticiscomputedovera“trimmed”subsetofthesample.A CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS203Table4.3:CriticalValuesoftheQLRstatisticwith15%TrimmingNumberofrestrictions(m)10%5%1%17.128.6812.1625.005.867.7834.094.716.0243.594.095.1253.263.664.5363.023.374.1272.843.153.8282.692.983.5792.582.843.38102.482.713.23popularchoiceistouse15%trimming,thatis,tosetforτ0=0.15Tandτ1=0.85T.With15%trimming,theF-statisticiscomputedforbreakdatesinthecentral70%ofthesample.Table4.3presentsthecriticalvaluesforQLRstatisticcomputedwith15%trimming.ThistableisfromStockandWatson(2003)andyoushouldcheckthebookforacompletetable.TheQLRtestcandetectasinglebreak,multiplediscretebreaks,andaslowevolutionoftheregressionparameters.Ifthereisadistinctbreakintheregressionfunction,thedateatwhichthelargestChowstatisticoccursisanestimatorofthebreakdate.InR,thepackagesstrucchangeandsegmentedprovideseveraltestingmethodsfortestingbreaksoryouusethefunctionStructTS.4.6.2ZivotandAndrews’sTestingProcedureSometimes,youwouldsuspectthataseriesmayeitherhaveaunitrootorbeatrendstationaryprocessthathasastructuralbreakatsomeunknownperiodoftimeandyouwouldwanttotestthenull CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS204hypothesisofunitrootagainstthealternativeofatrendstationaryprocesswithastructuralbreak.ThisisexactlythehypothesistestedbyZivotandAndrews’s(1992)test.Inthistestingprocedure,thenullhypothesisisaunitrootprocesswithoutanystructuralbreaksandthealternativehypothesisisatrendstationaryprocesswithpos-siblestructuralchangeoccurringatanunknownpointintime.ZivotandAndrews(1992)suggestedestimatingthefollowingregression:Xkxt=µ+θDUt(τ)+βt+γDTt(τ)+αxt−1+ci∆xt−i+et,(4.5)i=1whereτ=TB/Tisthebreakfraction;DU(τ)=1ift>τTandt0otherwise;DTt(τ)=t−Tτift>τTand0otherwise;andxtisthetimeseriesofinterest.ThisregressionallowsboththeslopeandintercepttochangeatdateTB.Notethatfort≤τT(t≤TB)model(4.5)becomes:Xkxt=µ+βt+αxt−1+ci∆xt−i+et,i=1whilefort>τT(t>TB)model(4.5)becomes:XkBxt=[µ+θ]+[βt+γ(t−T)]+αxt−1+ci∆xt−i+et.i=1Model(4.5)isestimatedbyOLSwiththebreakpointsrangingoverthesampleandthet-statisticfortestingα=1iscomputed.Theminimumt-statisticisreported.Criticalvaluesfor1%,5%and10%criticalvaluesare−5.34,−4.8and−4.58,respectively.Theappro-priatenumberoflagsindifferencesisestimatedforeachvalueofτ.PleasereadthepaperbySadorsky(1999)formoredetailsaboutthismethodandempiricalapplications. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS2054.6.3CautionsTheappropriatewaytoadjustforabreakinthepopulationparam-etersdependsonthesourcesofthebreak.Ifadistinctbreakoccursataspecificdate,thisbreakwillbedetectedwithhighprobabilitybytheQLRstatistic,andthebreakdatecanbeestimated.Theregressionmodelcanbeestimatedusingadummyvariable.Ifthereisadistinctbreak,theninferenceontheregressioncoefficientscanproceedasusualusingt-statisticsforhypothesistesting.Forecastscanbeproducedusingtheestimatedregressionmodelthatappliestotheendofthesample.Theproblemismoredifficultifthebreakisnotdistinctandtheparametersslowlyevolveovertime.Inthiscaseastate-spacemodellingisrequired.4.7Problems1.Youwillbuildatime-seriesmodelforrealoilpricelistedinthetenthcolumninfile“MacroData.xls”.(a)Constructgraphsoftimeseriesdata:timeplotsandscatterplotsofthelevelsofrealoilprices(OP)andthelog-differenceofoilprices.Thelog-differenceofoilpriceisdefinedasfollows∆log(OPt)=log(OPt)−log(OPt−1).Commentyourresults.(b)Trytoidentifylagstructureforlevelsofrealoilpricesandthelog-differenceofoilpricesbyusingACFandPACF.Commentyourresults.(c)WewillnotestimateARMAmodelsyet.So,estimateAR(p)model,1≤p≤8.ComputeAkaikeInformationCriteria(AIC)orAICCandSchwarzInformationCriteria(SIC).Presentyourresults.ChoosetheARlaglengthbasedontheAICorAICC. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS206WhichlaglengthwouldyouchoosebasedontheAICorAICCorSIC?Comment.(d)EstimatetheAR(p)modelwiththeoptimallaglength.Presentyourresultsnicely.(e)ConductthediagnosticcheckingoftheestimatedARmodel.Youneeddothefollowingsandcommentyourresults:(i)Con-structgraphsofresiduals(timeseriesplots,scatterplots,squaredresiduals).(ii)CheckforserialcorrelationusingsampleACF.(iii)CheckforserialcorrelationusingLjung-Boxteststatistic.(iv)ConductJarque-BeratestofnormalityandmakeaQ-Qplot.(v)EstimatethefollowingAR(1)modelfortheestimatedsquaredresidualsandtestthenullhypothesisthattheslopeisinsignificant.Howwouldinterpretyourresults?Whatdoesitsayabouttheconstancyofvariance?(vi)Basedondiagnosticcheckingintheabove,canyouusethemodeloryoushouldgobacktoidentificationoflagstructurein(b)?(f)Isthereanystructurechangeforoilprice?Note:TheJarque-Bera(1980,1987)testevaluatesthehypothesisthatXhasanormaldistributionwithunspecifiedmeanandvariance,againstthealternativethatXdoesnothaveanormaldistribu-tion.ThetestisbasedonthesampleskewnessandkurtosisofX.Foratruenormaldistribution,thesampleskewnessshouldbenear0andthesamplekurtosisshouldbenear3.Atesthasthefollowinggeneralform:T(K−3)22JB=Sk+→χ2,64whereSkandKarethemeasuresofskewnessandkurtosisre-spectively.Tousethebuild-infunctionfortheJarque-Beratest CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS207inthepackagetseriesinR,thecommandfortheJarque-Beratestislibrary(tseries)#callthepackage"tseries"jb=jarque.bera.test(x)#xistheseriesforthetestprint(jb)#printthetestingresult2.Basedontheeconomictheory,thedescriptionofinflationsug-geststwofundamentalcauses,excessmonetarygrowth(fasterthanrealoutput)andthedissipationofexternalshocks.Theprecisemechanismsatwork,appropriatelagstructuresarenotperfectlydefined.Inthisexercise,youwillestimatethefollowingsimplemodel:∆log(Pt)=β1+β2[∆log(M1t−1)−∆log(Qt−1)]+β3∆log(Pt−1)+ut,wherePtisthequarterlypricelevel(CPI)listedintheeighthcolumnin“MacroData.xls”,Qtisthequarterlyrealoutput(listedinthethirdcolumn),andM1tisthequarterlymoneystock(listedinthethirteenthcolumn).(a)NicelypresenttheresultsofOLSestimation.Commentyourresults.(b)Explainwhatmaybeanadvantageoftheabovemodelcom-paredtoasimpleautoregressivemodelofprices.Toanswerthisquestion,youmightneedtodosomestatisticalanalysis.(c)IsthereanystructurechangeforCPI?(d)Anysuggestionstobuildabettermodel?3.Inthisexercise,youwillbuildatimeseriesmodelfortherealGDPprocesslistedinthesecondcolumnin“MacroData.xls”. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS208(a)BuildinformativelyandformallyanAR(p)modelwiththeop-timallaglengthbasedonsomecriterionandconductthediag-nosticcheckingoftheestimatedARmodel.(b)Basedonthedata1959.1-1999.3constructforecastsforthequar-ters1999.4-2002.3.Plottheconstructedforecastsandthereal-izedvalues.Commentyourresults.4.YouneedtoreplicatesomeofthestepsintheanalysisofoilpriceshocksandstockmarketactivityconductedbySadorsky(1999).Youshouldwriteyourreportinsuchawaythatanoutsidereadermayunderstandwhatthereportisaboutandwhatyouaredoing.WritearefereereportforthepaperofSadorsky(1999).Onepossiblestructureoftherefereereportis:(a)Summaryofthepaper(Thisassuresthatyoureallyreadthepapercarefully):(i)Istheeconomic/financialquestionofrelevance?(ii)Whathaveyoulearnedfromreadingthispaper?(iii)Whatcontributiondoesthispapermaketotheliterature?(b)Canyouthinkofinterestingextensionsforthepaper?(c)Expositoryqualityofthepaper:(i)Isthepaperwellstruc-tured?Ifnot,suggestandalternativestructure.(ii)Isthepapereasytoread?5.Analyzethestochasticpropertiesofthefollowinginterestrates:(i)federalfundsrate,(ii)90-dayT-billrate,(iii)1-yearT-bondinterestrate,(iv)5-yearT-bondinterestrate;(v)10-yearT-bondinterestrate.TheinterestratesmaybefoundintheExcelfile“IntRates.xls”.(a)UsetheADForPPapproachtotestthenullhypothesisthatfiveinterestratearedifferencestationaryagainstthealternative CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS209thattheyarestationary.Explaincarefullyhowyouconductthetest.(b)UsetheADForPPapproachtotestthenullhypothesisthatfiveinterestratearedifferencestationaryagainstthealternativethattheyarestationaryaroundadeterministictrend.Explaincarefullyhowyouconductthetest.(c)UsetheQLRtestingproceduretotestwhethertherewasatleastonestructuralbreakininterestratesseries.4.8ComputerCodeThefollowingRcommandsareusedformakingthegraphsinthischapter.#5-20-2006graphics.off()###################################################################y=read.csv("c:\teaching\timeseries\data\MacroData.csv",header=T,skip=1)cpi=y[,8]qt=y[,3]m0=y[,12]m1=y[,13]m2=y[,14]m3=y[,15]op=y[,10]v0=cpi*qt/m0v1=cpi*qt/m1v2=cpi*qt/m2 CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS210v3=cpi*qt/m3vt=cbind(v0,v1,v2,v3)win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightblue")ts.plot(cpi,type="l",lty=1,ylab="",xlab="")title(main="CPI",col.main="red")ts.plot(qt,type="l",lty=1,ylab="",xlab="")title(main="IndustryOutput",col.main="red")ts.plot(qt,type="l",lty=1,ylab="",xlab="")title(main="OilPrice",col.main="red")win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightgrey")ts.plot(m0,type="l",lty=1,ylab="",xlab="")title(main="MoneyAggregate",col.main="red")ts.plot(m1,type="l",lty=1,ylab="",xlab="")title(main="MoneyAggregate",col.main="red")ts.plot(m2,type="l",lty=1,ylab="",xlab="")title(main="MoneyAggregate",col.main="red")ts.plot(m3,type="l",lty=1,ylab="",xlab="")title(main="MoneyAggregate",col.main="red")win.graph()par(mfrow=c(2,2),mex=0.4,bg="yellow")ts.plot(v0,type="l",lty=1,ylab="",xlab="")title(main="Velocity",col.main="red")ts.plot(v1,type="l",lty=1,ylab="",xlab="")title(main="Velocity",col.main="red")ts.plot(v2,type="l",lty=1,ylab="",xlab="") CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS211title(main="Velocity",col.main="red")ts.plot(v3,type="l",lty=1,ylab="",xlab="")title(main="Velocity",col.main="red")library(tseries)#calllibrary(tseries)library(urca)#calllibrary(urca)library(quadprog)library(zoo)adf_test=adf.test(cpi)#AugmentedDickey-Fullertestprint(adf_test)adf_test=pp.test(cpi)#doPhillips-Perrontestprint(adf_test)#adf_test2=ur.df(y=cpi,lag=5,type=c("drift"))#print(adf_test2)adf_test=adf.test(op)#AugmentedDickey-Fullertestprint(adf_test)adf_test=pp.test(op)#doPhillips-Perrontestprint(adf_test)for(iin1:4){adf_test=pp.test(vt[,i])print(adf_test)adf_test=adf.test(vt[,i])print(adf_test)}################################################################### CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS212y=read.csv("c:\teaching\timeseries\data\MacroData.csv",header=T,skip=1)op=y[,10]library(strucchange)win.graph()par(mfrow=c(2,2),mex=0.4,bg="green")op=ts(op)fs.op<-Fstats(op~1)#nolagsandcovariateplot(op,type="l")plot(fs.op)sctest(fs.op)##visualizethebreakpointimpliedbytheargmaxoftheFstaplot(op,type="l")lines(breakpoints(fs.op))######################################ThefollowingistheexamplefromR######################################win.graph()par(mfrow=c(2,2),mex=0.4,bg="red")if(!"package:stats"%in%search())library(ts)##Niledatawithonebreakpoint:theannualflowsdrop##becausethefirstAshwandamwasbuiltdata(Nile)plot(Nile)##testwhethertheannualflowremainsconstantoverthefs.nile<-Fstats(Nile~1)plot(fs.nile)sctest(fs.nile) CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS213plot(Nile)lines(breakpoints(fs.nile))###################################################################4.9ReferencesBeveridge,S.andC.R.Nelson(1981).Anewapproachtodecompositionofeconomictimeseriesintopermanentandtransitorycomponentswithparticularattentiontomeasurementofthebusinesscycle.JournalofMonetaryEconomics,7,151-174.Cai,Z.(2006).Trendingtimevaryingcoefficienttimeseriesmodelswithseriallycorrelatederrors.ForthcominginJournalofEconometrics.Cai,Z.,J.Fan,andQ.Yao(2000).Functional-coefficientregressionmodelsfornonlineartimeseries.JournaloftheAmericanStatisticalAssociation,95,941-956.Chang,Y.,J.Y.ParkandP.C.B.Phillips(2001).Nonlineareconometricmodelswithco-integratedanddeterministicallytreadingregressors.EconometricsJournal,4,1-36.Chang,Y.andE.Martinez-Chombo(2003).Electricitydemandanalysisusingcointegrationanderror-correctionmodelswithtimevaryingparameters:TheMexicancase,Workingpaper,DepartmentofEconomics,RiceUniversity.Cochrane,J.H.(1997).Timeseriesformacroeconomicsandfinance.LectureNotes.http://gsb.uchicago.edu/fac/john.cochrane/research/Papers/timeser1.pdfCochrane,J.H.(2001).AssetPricing.NewJersey:PrincetonUniversityPress.Dickey,D.A.andW.A.Fuller(1979).Distributionoftheestimatorsforautoregressivetimeserieswithaunitroot.JournaloftheAmericanStatisticalAssociation,74,427-431.Hamilton,J.D.(1994).TimeSeriesAnalysis.PrincetonUniversityPress.Heij,C.,P.deBoer,P.H.FransesandH.K.vanDijk(2004).EconometricMethodswithApplicationsinBusinessandEconomics.OxfordUniversityPress.Hendry,F.D.andK.Juselius(2000).Explainingcointegrationanalysis:PartI.JournalofEnergy,21,1-42.Jarque,C.M.andA.K.Bera(1980).Efficienttestsfornormality,homoscedasticityandserialindependenceofregressionresiduals.EconomicsLetters,6,255-259.Jarque,C.M.andA.K.Bera(1987).Atestfornormalityofobservationsandregressionresiduals.InternationalStatisticalReview,55,163-172. CHAPTER4.NON-STATIONARYPROCESSESANDSTRUCTURALBREAKS214Juhl,T.(2005).Functionalcoefficientmodelsunderunitrootbehavior.EconometricsJournal,8,197-213.Park,J.Y.andP.C.B.Phillips(1999).Asymptoticsfornonlineartransformationsofinte-gratedtimeseries.EconometricTheory,15,269-298.Park,J.Y.andP.C.B.Phillips(2002).Nonlinearregressionswithintegratedtimeseries.Econometrica,69,117-161.Phillips,P.C.B.(2001).Trendingtimeseriesandmacroeconomicactivity:Somepresentandfuturechallenges.JournalofEconometrics,100,21-27.Phillips,P.C.B.andJ.Park(2005).Non-stationarydensityandkernelautoregression.UnderrevisionforEconometricsTheory.Phillips,P.C.B.andP.Perron(1988).Testingforaunitrootintimeseriesregression.Biometrika,75,335-346.Sadorsky,P.(1999).Oilpriceshocksandstockmarketactivity.EnergyEconomics,21,449-469.Stock,J.H.andM.W.Watson(2003).IntroductiontoEconometrics.Addison-Wesley.Tsay,R.S.(2005).AnalysisofFinancialTimeSeries,2thEdition.JohnWiley&Sons,NewYork.Zivot,E.andD.W.K.Andrews(1992).Furtherevidenceonthegreatcrash,theoilpriceshockandtheunitroothypothesis.JournalofBusinessandEconomicsStatistics,10,251-270. Chapter5VectorAutoregressiveModels5.1IntroductionAunivariateautoregressionisasingle-equation,single-variablelinearmodelinwhichthecurrentvalueofavariableisexplainedbyitsownlaggedvalues.Multivariatemodelslookliketheunivariatemodelswiththelettersre-interpretedasvectorsandmatrices.Consideramultivariatetimeseries:ytxt=.ztRecallthatbymultivariatewhitenoiseet∼N(0,Σ),wemeanthat2e=vt,E(e)=0,E(ee′)=Σ=σvσvu,E(ee′)=0.tutttσσ2tt−jtuvuTheAR(1)modelfortherandomvectorxtisxt=φxt−1+etwhichinmultivariateframeworkmeansthatytφyyφyzyt−1vt=+.ztφzyφzzzt−1utNoticethatbothlaggedyandzappearineachequationwhichmeansthatthemultivariateAR(1)process(VAR)capturescross-variabledynamics(co-movements).215 CHAPTER5.VECTORAUTOREGRESSIVEMODELS216AgeneralVARisann-equation,n-variablelinearmodelinwhicheachvariableisexplainedbyitsownlaggedvalues,pluscurrentandpastvaluesoftheremainingn−1variables:yt=Φ1yt−1+···+Φpyt−1+et,(5.1)where(i)(i)(i)y1te1tφ11φ12···φ1n(i)(i)(i)y=y2t,e=e2t,Φ=φ21φ22···φ2n,t...t...i............(i)(i)(i)yntentφn1φn2···φnnandtheerrortermsethaveavariance-covariancematrixΩ.Thename“vectorautoregression”isusuallyusedintheplaceof“vectorARMA”becauseitisveryuncommontoestimatemovingaverageterms.AutoregressionsareeasytoestimatebecausetheOLSas-sumptionsstillapplyandeachequationmaybeestimatedbyordi-naryleastsquaresregression.TheMAtermshavetobeestimatedbymaximumlikelihood.However,sinceeveryMAprocesshasAR(∞)representation,pureautoregressioncanapproximateMAprocessifenoughlagsareincludedinARrepresentation.TheusefulnessofVARmodelsisthatmacroeconomistscandofourthings:(1)describeandsummarizemacroeconomicdata;(2)makemacroeconomicforecasts;(3)quantifywhatwedoordonotknowaboutthetruestructureofthemacroeconomy;(4)advisemacroeconomicpolicymakers.Indatadescriptionandforecasting,VARshaveprovedtobepowerfulandreliabletoolsthatarenowineverydayuse.PolicyanalysisismoredifficultinVARframeworkbe-causeitrequiresdifferentiatingbetweencorrelationandcausation,socalled“identificationproblem”.Economictheoryisrequiredtosolvetheidentificationproblem.StandardpracticeinVARanalysisis CHAPTER5.VECTORAUTOREGRESSIVEMODELS217toreportresultsfromGranger-causality,impulseresponsesandforecasterrorvariancedecompositionwhichwillbedis-cussedrespectivelyinthenextsections.Formoreaboutthehistoryandrecentdevelopmentsaswellasapplication,seethepaperbyStockandWatson(2001).VARmodelscomeinthreevarieties:reducedform,recursiveandstructuralforms.Here,weonlyfocusonthefirstoneandthedetailsforthelasttwocanbefoundinHamilton(1994,Chapter11).AreducedformVARprocessoforderphasthesameformasequation(5.1):yt=Φ1yt−1+···+Φpyt−1+et,(5.2)whereytisann×1vectorofthevariables,Φiisann×nmatrixoftheparameters,cisann×1vectorofconstants,andet∼N(0,Ω).Theerrortermsintheseregressionsarethe“surprise”movementsinthevariablesaftertakingtheirpastvaluesintoaccount.Themodel(5.2)canbepresentedinmanydifferentways:(1)usingthelagoperatornotationasΦ(L)yt=c+et,whereΦ(L)=I−ΦL−···−ΦLp;(2)inthematrixnotationn1pasY=c+XΠ+EwhereEisaT×nmatrixofthedisturbancesandYisaT×nmatrixoftheobservations;(3)intermsofdeviationsfromthemean(centerized).ForestimatingVARmodels,severalmethodsareavailable:simplemodelscanbefittedbythefunctionar()inthepackagestats(built-in),moreelaboratemodelsareprovidedbyestVARXls()inthe CHAPTER5.VECTORAUTOREGRESSIVEMODELS218packagedse1andaBayesianapproachisavailableinthepackageMSBVAR.5.1.1PropertiesofVARModelsAvectorprocessissaidtobeacovariance-stationary(weaklystation-ary)ifitsfirstandsecondmomentsdonotdependont.TheVAR(p)modelinequation(5.1)canbewrittenintheformofVAR(1),calledcompanionform,processas:ξt=Fξt−1+vt,(5.3)whereΦ1Φ2Φ3···Φp−1ΦpytetIn00···00yt−10ξt=..,vt=..,F=0In0···00.....................yt−p+10000···In0Tounderstandconditionsforstationarityofavectorprocess,notefromtheaboveequationthat2s−1sξt+s=vt+s+Fvt+s−1+Fvt+s−2+···+Fvt+1+Fξt.Proposition4.1TheVARprocessiscovariance-stationaryiftheeigenvaluesofthematrixFarelessthanunityinabsolutevalue.Forcovariance-stationaryn-dimensionalvectorprocess,thej-thauto-covarianceisdefinedtobethefollowingn×nmatrix′Γj=E(yt−µ)(yt−j−µ).NotethatΓ6=ΓbutΓ=Γ′.j−jj−j CHAPTER5.VECTORAUTOREGRESSIVEMODELS219AvectormovingaverageprocessoforderqtakesthefollowingformsMA(q):yt=et+Θ1et−1+···+Θqet−q,whereetisavectorofwhitenoisesandet∼N(0,Ω).TheVAR(p)modelmaybepresentedasanMA(∞)modelXpX∞yt=Φjyt−j+et=Ψket−k,j=1k=0wherethesequence{Ψk}isassumedtobeabsolutelysummable.TocomputethevarianceofaVARprocess,letusrewriteaVAR(p)processintheformofVAR(1)processasin(5.3).Assumethatvectorsξandyarecovariance-stationary,letΣdenotethevarianceofξ.Then,Σisdefinedasfollows:Γ0Γ1···Γp−1′ΓΓ···ΓΣ=10p−2=FΣF′+Q,............Γ′Γ′···Γp−1p−20whereQ=Var(vt).WecanapplyVecoperator(IfAd×dissymmet-ric,Vec(A)denotesthed(d+1)/2columnvectorrepresentingthestackedupcolumnsofAwhichareonandbelowthediagonalofA)tobothsidesoftheaboveequation,′Vec(Σ)=Vec(FΣF)+Vec(Q)=(F⊗F)Vec(Σ)+Vec(Q),andwithA=F⊗F,−1Vec(Σ)=(Ir2−A)Vec(Q)providedthatthematrixIr2−Aisnonsingular,r=np.Iftheprocessξtiscovariance-stationary,thenIr2−Aisnonsingular. CHAPTER5.VECTORAUTOREGRESSIVEMODELS2205.1.2StatisticalInferencesSupposewehaveasampleofsizeT,{y}T,drawnfromann-tt=1dimensionalcovariance-stationaryprocesswithE(yt)=µandE[(yt−µ)(y−µ)′]=Γ.Asusually,thesamplemeanisdefined:t−jjXTyT=yt.t=1Itiseasytoshowthatforavariance-covariancestationaryprocess:1XTE(yT)=µ,andVar(yT)=TΓ0+(T−j){Γj+Γ−j}.T2j=1Proposition4.2Letytbeacovariance-stationaryprocesswiththemeanµandtheauto-covarianceΓjandwithabsolutelysummableautocovariances.ThenthesamplemeanyTsatisfies:yTconvergesP∞toµinprobabilityandTVar(yT)convergestoj=−∞Γj≡S.AconsistentestimateofScanbeconstructedbasedontheNeweyandWest’s(1987)HACestimatorasfollows(seeSection3.10):XqjSc=Γb+1−Γb+Γb,0−jjj=1q+1where1XTb′Γj=(yt−yT)(yt−j−yT)T−jv=j+1andonecansetthevalueofqasfollows:q=0.75T1/3.ToestimatetheparametersinaVAR(p)model,weconsiderthefollowingVAR(p)modelgivenin(5.2)asyt=c+Φ1yt−1+···+Φpyt−1+et, CHAPTER5.VECTORAUTOREGRESSIVEMODELS221whereytisann×1vectorcontainingthevaluesthatnvariablesassumeatdatet;et∼N(0,Ω).Weassumethatwehaveobservedeachofthesenvariablesfor(T+p)timeperiods,i.eweobservethefollowingsample{y−p+1,...,y0,y1,y2,...,yT}.Thesimplestapproachtomodelestimationistoconditiononthefirstpobserva-tions{y−p+1,y−p+2,...,y0}andtodoestimationusingthelastTobservations{y1,y2,...,yT}.Weusethefollowingnotation:1c′′yΦ11xt=..,Π=..,..yΦ′t−ppwherextisa(np+1)×1vectorandΠisann×(np+1)matrix.Thenequation(5.2)canbewrittenas:′yt=Πxt+et,whereytisann×1vectorofthevariablesatperiodt,etisann×1vectorofthedisturbancesandet∼N(0,Ω).Thelikelihoodfunctionforthemodel(5.2)canbecalculatedinthesamewayasforaunivariateautoregression.Theloglikelihoodfortheentiresampleisasfollows:T1XT′′−1′L(Π,Ω)=C−ln|Ω|−(yt−Πxt)Ω(yt−Πxt),22t=1whereCisaconstant.TofindtheMLEestimatesoftheparametersΠandΩ,wefindthederivativesofthelog-likelihoodintheabovewithrespecttoΠandΩandsetthemequaltozero.ItcanbeshownthattheMLEestimatesofΠandΩareasfollows:c′−1′ccc′ccΠ=(XX)XY,andΩ=EE/TwithE=Y−XΠ.TheOLSestimatorofΠisthesameastheunrestrictedMLEesti-mator. CHAPTER5.VECTORAUTOREGRESSIVEMODELS222TotestthehypothesisH0:θ=θ0,wecanusethepopularap-proach:likelihoodratiotest.Todoso,weneedtocalculatethebbmaximumvaluesθandθ0underH0andH1,respectively.TheLike-lihoodRatioteststatisticisfoundas:bbλT=−2[L(θ)−L(θ0)].Underthenullhypothesis,λasymptoticallyhasaχ2distributionTwithdegreesoffreedomequaltothenumberofrestrictionsimposedunderH0.ToderivetheasymptoticpropertiesoftheMLE,letusdefinecccπc=Vec(Π),whereΠistheMLEofΠ.NotethatsinceΠisannp×nmatrix,πcisann2p×1vector.ItcanbeshowedinProposition11.1ofPT′Hamilton(1994)that:(1)(1/T)t=1xtxt→Qinprobability,where′cQ=E(xtxt)isannp×npmatrix;(2)πc→πinprobability;(3)Ω→√Ωinprobability;(4)T(πc−π)→N0,Ω⊗Q−1.Therefore,πccanbetreatedasapproximately:πc≈Nπ,Ω⊗(X′X)−1.TotestthehypothesisoftheformH0:Rπ=rwecanusethefollowingformoftheWaldtest:−12′c′−1′χ(m)=(Rπc−r)RΩ⊗(XX)R(Rπc−r).5.2Impulse-ResponseFunctionImpulseresponsestraceoutresponsesofcurrentandfuturevaluesofeachofthevariablestoaone-unitincreaseinthecurrentvalueofoneoftheVARerrors,assumingthatthiserrorreturnstozeroinsubsequentperiodandthatallothererrorsareequaltozero.Thisfunctionisofinterestforseveralreasons:Itisanothercharacteri-zationofthebehaviorofmodelsanditallowsonetothinkabout“causes”and“effects”. CHAPTER5.VECTORAUTOREGRESSIVEMODELS223RecallthatforanAR(1)process,themodelisxt=φxt−1+etorP∞jxt=j=0φet−j.BasedontheMA(∞)representation,weseefromSection3.13thattheimpulse-responsefunctionis∂xt+jj=φ.∂etVectorprocessworksthesameway.Thecovariance-stationaryVARmodelcanbewritteninMA(∞)formasX∞yt=µ+Ψjet−j.j=0Then,∂yt+j=Ψj.∂e′tTheelementψijofΨsidentifiestheconsequencesofaone-unitin-creaseinthej-thvariable’sinnovationatdatet(ejt)forthevalueofthei-thvariableattimet+s(yi,t+s),holdingallotherinnovationsatalldatesconstant.Onemayalsofindtheresponseofaspecificvariabletoshocksinallothervariablesortheresponseofallvariablestoaspecificshock∂yi,t+s(s)∂yt+s(s)′=Ψ.i,and=Ψj..∂et∂ejtIfoneisinterestedinhowthevariablesofthevectoryt+sareaffectedifthefirstelementofetchangedbyδ1atthesametimethatthesecondelementchangedbyδ2,andthen-thelementbyδn,thenthecombinedeffectofthesechangesonthevalueofyt+sisgivenbyXn∂yt+s∆yt+s=′δj=Ψsδ.j=1∂ejtAplotoftherowi,columnjelementofΨs,S∂yi,t+s∂ejts=0 CHAPTER5.VECTORAUTOREGRESSIVEMODELS224asafunctionofsiscalledtheorthogonalimpulse-responsefunc-tion.Itdescribestheresponseofyi,t+stoaone-timeimpulseinyjtwithallothervariablesdatedtorearlierheldconstant.Supposethatthedatetvalueofthefirstvariableintheautore-gression,y1t,washigherthanexpected.Howdoesthiscauseustorevisetheforecastofyi,t+s?Toanswerthisquestion,wedefinex′=(y′,y′,...,y′),whereyisann×1vectorandxt−1t−1t−2t−pt−it−1isannp×1vector.Thequestionbecomes,whatis∂E(yi,t+s|y1t,xt−1)?∂y1tNotethat∂E(yi,t+s|y1t,xt−1)∂E(yi,t+s|y1t,xt−1)∂E(et|y1t,xt−1)(s)=′×=ψ.1.∂y1t∂E(et|y1t,xt−1)∂y1tLetusexaminetheforecastrevisionresultingfromnewinformationaboutthesecondvariable,y2t,beyondthatcontainedinthefirstvariables,y1t,∂E(yi,t+s|y1t,y2t,xt−1)∂E(yi,t+s|y1t,y2t,xt−1)∂E(et|y1t,y2t,xt−1)(=′×=ψ.2∂y2t∂E(et|y1t,y2t,xt−1)∂y2tSimilarlywemightfindtheforecastrevisionforthethirdvariableandsoon.Forvariableynt,∂E(yi,t+s|y1t,···,ynt,xt−1)∂E(yi,t+s|y1t,···,ynt,xt−1)∂E(et|y1t,···,y=×∂ynt∂E(e′t|y1t,···,yn2t,xt−1)∂yntThefollowingsarethreeimportantpropertiesofimpulse-responses:First,theMA(∞)representationisthesamethingastheimpulse-responsefunction;second,theeasiestwaytocalculateMA(∞)rep-resentationistosimulatetheimpulseresponsefunction;finally,theimpulseresponsefunctionisthesameasEt(yt+j)−Et−1(yt+j). CHAPTER5.VECTORAUTOREGRESSIVEMODELS2255.3VarianceDecompositionsIntheorganizedsystem,wecancomputeanaccountingofforecaster-rorvariance:whatpercentofthekstepaheadforecasterrorvarianceisduetowhichvariable.Todothis,westartwithMArepresentation:yt=Ψ(L)etwherey=(x,z)′,e=(e,e)′,E(ee′)=I,andΨ(L)=ttttxtztttP∞jj=0ΨjL.Theonestepforecasterroriset+1=yt1+−Et(yt+1)=Ψ0etanditsvarianceis22Vart(xt+1)=ψxx,0+ψxz,0.ψ2givestheamountofone-stepaheadforecasterrorvarianceofxx,0xduetotheeshockandψ2givestheamountduetoeshock.xxz,0zInpractice,oneusuallyreportsfractionsψ2/(ψ2+ψ2).Morexx,0xx,0xz,0formally,wecanwrite′Vart(yt+1)=Ψ0Ψ0.Define1000I1=andI2=.0001Then,thepartdueoftheonestepaheadforecasterrorvariancedue′tothefirstshockxisΨ0I1Ψ0andthepartduetothesecondshock′zisΨ0I2Ψ0.Generalizingtokstepscanbedoneasfollows:kX−1′Vart(yt+k)=ΨjΨj.j=0Then,kX−1′wk,τ=ΨjIτΨjj=0 CHAPTER5.VECTORAUTOREGRESSIVEMODELS226isthevarianceofkstepaheadforecasterrorsduetotheτ-thshockandthevarianceissumofthesecomponents,e.g.Vart(yt+k)=Pτwk,τ.5.4GrangerCausalityThefirstthingthatyoulearnineconometricsisacautionthatputtingxontherighthandsideofy=x′β+edoesnotmeanthatx“causes”y.Thenyoulearnthatcausalityisnotsomethingyoucantestforstatistically,butmustbeknownapriori.Itturnsoutthatthereisalimitedsenseinwhichwecantestwhetheronevariable“causes”anotherandviceversa.Granger-causalitystatisticsexaminewhetherlaggedvaluesofonevariablehelptopredictanothervariable.ThevariableyfailstoGranger-causethevariablexifforalls>0theMSEofaforecastofxt+sbasedon(xt,xt−1,...)isthesameastheMSEofaforecastofxt+sthatusesboth(xt,xt−1,...)and(yt,yt−1,...).Ifoneconsidersonlylinearfunctions,yfailstoGranger-causexifccMSE[E(xt+s|xt,xt−1,...)]=MSE[E(xt+s|xt,xt−1,...,yt,yt−1,...)].Equivalently,wesaythatxisexogenousinthetimeseriessensewithrespecttoyiftheaboveholds.Or,yisnotlinearinformativeaboutfuturex.InabivariateVARmodeldescribingxandy,xdoesnotGranger-causeyifthecoefficientsmatricesΦjarelowertriangularforallj,i.e.(1)(p)xt=c1+φ110xt−1+···+φ110xt−p+e1t(1)(1)(p)(p)ytc2φ21φ22yt−1φ21φ22yt−pe2t CHAPTER5.VECTORAUTOREGRESSIVEMODELS227TheGranger-causalitycanbetestedbyconductingF-testofthenullhypothesis(1)(2)(p)H0:φ21=φ21=···=φ21=0.ThefirstandmostfamousapplicationofGrangercausalitywasthequestionofwhether“moneygrowthcauseschangesinGNP”.Fried-manandSchwartz(1963)documentedacorrelationbetweenmoneygrowthandGNP.ButTobin(1970)arguedthataphaseleadandacorrelationmaynotindicatecausality.Sims(1980)answeredthiscriticismandhadshownthatmoneyGrangercausesGNPandnotviceversa(hefounddifferentresultslater).Sims(1980)analyzedthefollowingregressiontostudyeffectofmoneyonGNP:X∞yt=bjmt−j+ut.j=0Thisregressionisknownasa“St.LouisFed”equation.Thecoeffi-cientswereinterpretedastheresponseofytochangesinm,i.e.iftheFedsetsm,{bj}givestheresponseofy.Sincethecoefficientswererelativelybig,itimpliedthatconstantmoneygrowthrulesweredesirable.Theobviousobjectiontothisstatementisthatcoefficientsmayreflectreversecausality:theFedsetsmoneyinanticipationofsub-sequenteconomicgrowth,ortheFedsetsmoneyinresponsetopasty.ThismeansthattheerrortermuiscorrelatedwithcurrentandlaggedmsoOLSestimatesoftheparametersbareinconsistentduetotheendogeneityproblem.Whyis“Grangercausality”not“causal-ity”?Grangercausalityisnotcausalitybecauseofthepossibleeffectofothervariables.Ifxleadstoywithonelagbuttozwithtwolags,thenywillGrangercausezinbivariatesystem.Thereasonisthatywillhelpforecastzbecauseitrevealsinformationaboutthe“true CHAPTER5.VECTORAUTOREGRESSIVEMODELS228Table5.1:SimsvariancedecompositioninthreevariableVARmodelExplainedbyshockstoVar.ofM1IPWPIM19721IP374418WPI14780Table5.2:SimsvariancedecompositionincludinginterestratesExplainedbyshockstoVar.ofRM1IPWPIR5019428M1564211IP232606WPI3041452cause”x.Butitdoesnotfollowthatifyouchangeythenachangeinzwillfollow.Thiswouldnotbeaproblemiftheestimatedpatternofcausalityinmacroeconomictimeserieswasstableovertheinclusionofseveralvariables.AnexamplebySims(1980)illustratedthatisnotthecaseveryoften.Sims(1980)estimatedthreevariableVARwithmoney,industrialproductionandwholesalepriceindexandfourvariableVARwithinterestrate,money,industrialproductionandwholesalepriceindex.TheresultsareinTable5.1.ThefirstrowinTable5.1verifiesthatM1isexogenousbecauseitdoesnotrespondtoothervariables’shocks.ThesecondrowshowsthatM1“causes”changesinIP,since37%ofthe48monthaheadvarianceofIPisduetoM1shocks.ThethirdrowispuzzlingbecauseitshowsthatWPIisexogenous.Table5.2showswhathappenswhenonmorevariable,interestrate,isaddedtothemodel.ThesecondrowshowssubstantialresponseofM1tointerestrateshocks.InthismodelM1 CHAPTER5.VECTORAUTOREGRESSIVEMODELS229isnotexogenous.InthethirdrowonecanseethatM1doesinfluenceIP;thefourthrowshowsthatM1doesnotinfluenceWPI,interestratedoes.5.5ForecastingTodoforecasting,weneedtodothefollowingthings.First,choosethelaglengthofVARusingeitheroneoftheinformationcriteria(AIC,SIC,AICC)oroneoftheforecastingcriteria;second,estimatetheVARmodelusingtheOLSandobtaintheparameterestimatescΦj;andfinally,theh-period-aheadforecastsareconstructedrecur-sively:tcccybt+1=Φ1yt+Φ2yt−1+···+Φpyt−p+1tctccybt+2=Φ1ybt+1+Φ2yt+···+Φpyt−p+2tctctcybt+3=Φ1ybt+2+Φ2ybt+1+···+Φpyt−p+3andsoon.HowwelldoVARsperformthetasks?BecauseVARinvolvescurrentandlaggedvaluesofmultipletimeseries,theycaptureco-movementsthatcannotbedetectedinunivariatemodels.VARsummarystatisticslikeGranger-causalitytests,impulseresponsefunctionsandvariancedecompositionsarewell-acceptedmethodsforportrayingco-movements.SmallVARshavebecomeabenchmarkagainstwhichnewforecastingsystemsarejudged.TheproblemisthatsmallVARsareoftenunstableandthuspoorpredictorsofthefuture.5.6Problems1.WritearefereereportforthepaperbySims(1992). CHAPTER5.VECTORAUTOREGRESSIVEMODELS2302.UsequarterlydataforrealGDP,GDPdeflator,CPI,theFederalFundsrate,Moneybasemeasure,M1,andtheindexofcommod-itypricesfortheperiod1959.I-2002.III(file“TEps8data.xls”).Also,usethemonthlydatafortheTotalReserves(file“TotRe-sAdjRR.xls”)andNon-BorrowedReserves(file“BOGNONBR.xls”)fortheperiod1959.1-2002.9.Transformmonthlydataintoquar-terlydata.WeexaminetheVARmodel,investigatedbyChris-tiano,EichenbaumandEvans(2000).Themodelhassevenvariables:thelogofrealGDP(Y),logofconsumerpricein-dex(CPI),changeintheindexofsensitivecommodityprices(Pcom),Federalfundsrate(FF),logoftotalreserves(TR),logofnon-borrowedreserves(NBR),andlogofM1monetaryaggregate(M1).Amonetarypolicyshockinthemodelisrep-resentedbyashocktotheFederalfundsrate:theinformationsetconsistsofcurrentandlaggedvaluesofY,CPIandPcom,andonlylaggedvaluesofFF,TR,NBRandM0.Itimpliesthefollowingorderingofthevariablesinthemodel:′xt=(Y,CPI,Pcom,FF,TR,NBR,M1).ThereferenceforthispaperisChristiano,EichenbaumandEvans(2000).(a)Constructgraphsofalltimeseriesdata.Commentyourresults.(b)EstimateaVARmodelforxt.(i)Nicelypresenttheimpulsere-sponsefunctionsrepresentingtheresponseofallvariablesinthemodeltoamonetarypolicyshock(FFrate).Carefullyexplainyourresults.(ii)Nicelypresentthevariance-decompositionre-sultsforallvariablesfortheforecasthorizonsk=2,4,12,36.Carefullyexplainyourresults.(iii)ConductGrangerCausal-itytestsforallvariablesinthemodel.Carefullyexplainyour CHAPTER5.VECTORAUTOREGRESSIVEMODELS231results.(c)Mostmacroeconomistsagreethattherewasashiftofmonetarypolicytowardinflationduringthelate1970sfromaccommodat-ingtoaggressive.Estimatemodelfortwoperiods1959.I-1979.IIand1979.III-2002.III.Nicelypresenttheimpulseresponsefunc-tionsrepresentingtheresponseofallvariablesinthemodeltoamonetarypolicyshock(FFrate)forbothperiods.Carefullyexplainyourresults.Howdoimpulseresponsefunctionrevealthechangeinthemonetarypolicy?(d)UsetheVARmodeltoconstruct8-period-aheadforecastsforallthevariablesinthemodel.3.SomemacroeconomistslookedattheNBRandNBR/TRspecificationsofmonetarypolicyshocks.InthecaseofaNBRmonetarypolicyshock,theinformationsetisidenticaltoaFFshock,whileinthecaseofaNBR/TRshocktheinformationsetincludesalsothecurrentvalueoftotalreserves.UsearecursiveschemeforidentificationandexaminetheeffectofamonetarypolicyshockforaNBRspecification(thinkhowyoushouldreorderthevariablesinmodel).(a)Nicelypresenttheimpulseresponsefunctionsrepresentingtheresponseofallvariablesinthemodeltoamonetarypolicyshock(thelevelofNBR).Explainyourresults.(b)Nicelypresentthevariance-decompositionresultsforallvariablesshowingthecontributionofNBRshockonlyfortheforecasthorizonsk=2,4,12,36.Explainyourresults.4.WriteaprogramforimplementingFAVARapproachofBernankeetal.(2005). CHAPTER5.VECTORAUTOREGRESSIVEMODELS232(a)Runtheprogramandexplainthemainstepsintheestimation.(b)Usingtheestimationresults,carefullyexplaintheeffectofmon-etarypolicyon90-dayT-billrate,1-yearT-bondinterestrate,5-yearT-bondinterestrateand10-yearT-bondinterestrate.(c)Carefullyexplainyourfindingontheeffectofmonetarypolicyonemployment.Youneedtousetheimpulseresponsefunctionsforemploymentseries,unemploymentseries,averagehoursworked,andnewclaimsforunemployment.(d)ExplaintheeffectofashocktotheFederalFundsrateondiffer-entaggregatemeasuresofmoneysupply.(e)Explaintheeffectofamonetarypolicyshockonexchangerateandrealstockprices.(f)Explaintheeffectofamonetarypolicyshockondifferentmea-suresofGDP(realGDP,differentmeasuresofIndustrialPro-duction,etc.).(g)WhathappenstotheresultsifthenumberofDiffusionIndexes(seeStockandWatson(2002))isincreasedfromthreetofive?5.Assumethatyouarefacedwithataskofforecastingdifferentinterestrates.ExplainhowyoumayapplytheDiffusionIndexesapproachofStockandWatson(2002)touseasmanyvariablesaspossibleintheforecasting.6.WritearefereereportforthepaperbyBachmeier,LeelahanonandLi(2005).Pleasethinkaboutanypossiblefutureprojects. CHAPTER5.VECTORAUTOREGRESSIVEMODELS2335.7ReferencesBachmeier,L.,S.LeelahanonandQ.Li(2005).MoneygrowthandinflationintheUnitedStates.WorkingPaper,DepartmentofEconomics,TexasA&MUniversity.Bernanke,B.S.,J.BoivinandE.Piotr(2005).Measuringtheeffectsofmonetarypolicy:afactor-augmentedvectorautoregressive(FAVAR)approach.TheQuarterlyJournalofEconomics,120,387-422.Christiano,L.J.,M.EichenbaumandC.L.Evans(2000).Monetarypolicyshocks:whathavewelearnedandtowhatend?HandbookofMacroeconomics,Vol.1A.Cochrane,J.H.(1994).Shocks.NBERworkingpaper#46984.Cochrane,J.H.(1997).Timeseriesformacroeconomicsandfinance.LectureNotes.http://www-gsb.uchicago.edu/fac/john.cochrane/research/Papers/timeser1.pdfFriedman,M.andA.J.Schwartz(1963).AMonetaryHistoryoftheUnitedStates,1867-1960.PrincetonUniversityPress.Hamilton,J.D.(1994).TimeSeriesAnalysis.PrincetonUniversityPress.Hendry,F.D.andK.Juselius(2000).Explainingcointegrationanalysis:PartI.JournalofEnergy,21,1-42.Newey,W.K.andK.D.West(1987).Asimple,positive-definite,heteroskedasticityandautocorrelationconsistentcovariancematrix.Econometrica,55,703-708.Sims,C.(1980).Macroeconomicsandreality.Econometrica,48,1-48.Sims,C.A.(1992).Interpretingthemacroeconomictimeseriesfacts:theeffectsofmonetarypolicy.EuropeanEconomicReview,36,975-1000.Stock,J.H.andM.W.Watson(2001).Vectorautoregressions.JournalofEconomicPer-spectives,15,101-115.Stock,J.H.andM.W.Watson(2002).Macroeconomicforecastingusingdiffusionindexes.JournalofBusinessandEconomicStatistics,20,147-162.Stock,J.H.andM.W.Watson(2003).IntroductiontoEconometrics.Addison-Wesley.Tobin,J.(1970).Moneyandincome.QuarterlyJournalofEconomics. Chapter6Cointegration6.1IntroductionCointegrationisageneralizationofunitroottovectormodelsasasingleseriescannotbecointegrated.Cointegrationanalysisisdesignedtofindlinearcombinationsofvariablestoremoveunitroots.SupposethattwoseriesareeachintegratedwiththefollowingMArepresentation:(1−L)yt=a(L)ut,and(1−L)xt=b(L)vt.Ingeneral,linearcombinationsofytandxtwillalsohaveaunitroots.Butifthereissomelinearcombination,yt−θxt,thatisstationary,yandxaresaidtobecointegratedandα=(1,−θ)′istttheircointegratingvector.Cointegrationvectorsareofconsiderableinterestwhentheyexist,sincetheydetermineI(0)relationsthatholdbetweenvariablesthatareindividuallynon-stationary.Asanexample,wemaylookattherealGNPandconsumption.Eachoftheseseriesprobablyhasaunitrootbuttheratioofconsump-tiontorealGNPisstableoverthelongperiodsoftime.Therefore,logconsumptionminuslogGNPisstationary,andlogGNPandconsumptionarecointegrated.Otherpossibleexamplesincludethedividend/priceratioormoneyandprices.However,cointegration234 CHAPTER6.COINTEGRATION235doesnotsayanythingaboutthedirectionofcausality.6.2CointegratingRegressionCointegratingvectorsare“super-consistent”whichmeansthatyoucanestimatethembyOLSevenwhentherighthandsidevariablesarecorrelatedwiththeerrorterm,andtheestimatesconvergeatafasterratethanusualOLSestimates.Supposeytandxtarecointe-gratedsothatyt−θxtisstationary.EstimatethefollowingmodelusingOLSregression:yt=βxt+et.(6.1)OLSestimatesofβconvergetoθ,eveniftheerrorsarecorrelatedwithxt.NotethatifytandxtareeachindividualI(1)butarenotcointegratedthentheregressioninequation(6.1)resultsinspuri-ousregression.Therefore,youhavetocheckwhethertheestimatedresidualsebtareI(1)orI(0).Wewilldiscussitlaterinthenotes.RepresentationofCointegratingSystemLetytbeafirstdifferencestationaryvectortimeseries.Theelementsofytarecointegratedifthereisatleastonevectorα,cointegratingvector,suchthatα′yisstationaryinlevels.Sincethedifferenceoftytisstationary,ithasamovingaveragerepresentation(1−L)yt=A(L)et.Sincethestationarityofα′yisanextrarestriction,itmustimplyatrestrictiononA(L).SimilartotheunivariateBeveridge-Nelson(1981)decomposition,themultivariateBeveridge-Nelsondecompositioncanbedoneinthe CHAPTER6.COINTEGRATION236samewayas:yt=zt+ct∗∗P∞where(1−L)zt=A(1)etandct=A(L)etwithAj=k=j+1Ak.TherestrictiononA(1)impliedbycointegration:theelementsofytarecointegratedwithcointegratingvectorsαifandonlyif(iff)α′A(1)=0.ThisimpliesthattherankofA(1)isthenumberofelementsofytminutesnumberofcointegratingvectorsα.TherearethreecasesforA(1):First,A(1)=0iffytisstationaryinlevelsandalllinearcombinationsofytarestationaryinlevels.Second,A(1)isnotfullrankiff(1−L)ytisstationaryandsomelinearcombinationsα′yarestationary.Finally,A(1)hasfullrankiff(1−tL)ytisstationaryandnolinearcombinationsofytarestationary.ImpulseresponsefunctionA(1)isthelimitingimpulse-responseofthelevelsofthevectoryt=(x,z)′.ToseehowcointegrationaffectsA(1),considerasimplettcase,α=(1,−1)′.ThereducedrankofA(1)means:′A(1)xxA(1)zxαA(1)=0,or(1−1)=0.A(1)xzA(1)zzTherefore,A(1)xx=A(1)zxandA(1)xz=A(1)zzsothateachvari-able’slong-runresponsetoashockmustbethesame.6.3TestingforCointegrationThereareseveralwaystodecidewhethervariablescanbemodeledascointegrated:First,useexpertknowledgeandeconomictheory.Sec-ond,graphtheseriesandseewhethertheyappeartohaveacommonstochastictrend.Finally,performstatisticaltestsforcointegration. CHAPTER6.COINTEGRATION237Allthreemethodsshouldbeusedinpractice.Wewillconsiderresid-ualbasedstatisticaltestforcointegration.TestingforcointegrationwhencointegratingvectorisknownSometimes,aresearchermayknowthecointegratingvectorbasedontheeconomictheory.Forexample,thehypothesisofpurchasingpowerparityimpliesthat:∗Pt=St×Pt,wherePtisanindexofthepricelevelintheU.S.,Stistheexchangerate($/ChineseYuan),P∗isapriceindexforChina.Takinglogs,tthisequationcanbewrittenas:∗pt=st+ptAweakerversionofthehypothesisisthatthevariablevtdefinedby:pt∗vt=pt−st−pt=(1−1−1)stp∗tisstationary,eventhoughtheindividualelements(p,s,p∗)aretttallI(1).Inthiscasethecointegratingvectorαisknowntobe(1,−1,−1).Testingforcointegrationinthiscaseconsistsofseveralsteps:1.Verifythatp,s,p∗areeachindividuallyI(1).Thiswilltttbetrueif(a)Youtestforunitrootinlevelsoftheseseriesandcannorejectthenullhypothesisofunitroot(ADForothertestsforunitroot).(b)Youtestforunitrootinfirstdifferencesoftheseseriesandrejectthenullhypothesisofunitroot(ADForothertestsforunitroot).2.Testthataseriesvtisstationaryornot. CHAPTER6.COINTEGRATION238Table6.1:CriticalvaluesfortheEngle-GrangerADFstatisticNumberofX′sinequation(6.1)10%5%1%1-3.12-3.41-3.962-3.52-3.80-4.363-3.84-4.16-4.734-4.20-4.49-5.07TestingforcointegrationwhencointegratingvectorisunknownConsideranexampleinwhichtwoseriesytandxtarecointegratedwithcointegratedvectorα=(1,−θ)′sothatv=y−θxistttstationary.However,thecointegratingcoefficientθisnotknown.Toestimateθ,wecanusetheEngle-GrangerAugmentedDickey-Fuller(EG-ADF)testforcointegrationwhichconsistsoftwosteps:1.VerifythatytandxtareeachindividuallyI(1).2.EstimatethecointegratingcoefficientθusingOLSestima-tionoftheregressionyt=µ+θxt+vt.3.ADickey-Fullert-test(withinterceptµnotimetrend)isusedtotestforaunitrootintheresidualfromthisregres-sion,vbt.Sinceweestimateresidualsinthefirststep,weneedtousedifferentcriticalvaluesfortheunitroottest.CriticalvaluesforEG-ADFstatisticaregiveninTable6.1.ThistableistakenfromStockandWatson(2002).Ifxtandytarecointegrated,thentheOLSestimatoroftheco-efficientintheregressionissuper-consistent.Therefore,theOLSestimatorhasanon-normaldistribution,andinferencesbasedonitst-statisticcanbemisleading.Toavoidthisproblem,StockandWat-son(1993)developeddynamicOLS(DOLS)estimatorofθfromthe CHAPTER6.COINTEGRATION239followingregression:pXyt=µ+θxt+δj∆xt−j+ut.j=−pIfxandyarecointegrated,statisticalinferencesaboutθandδ′sttbasedonHACstandarderrorsarevalid.Ifxtwerestrictlyexogenous,thenthecoefficientonxt,θ,wouldbethelong-runcumulativemultiplier,thatis,thelong-runeffectonyofachangeinx.Seethelong-runmultiplierbetweenoilandgasolinepricesinthepaperbyBorensteinetal.(1997).6.4CointegratedVARModelsWestartwiththeautoregressiverepresentationoflevelsofyt,B(L)yt=et:yt=−B1yt−1−B2yt−2−···+etApplyingBNdecompositionB(L)=B(1)+(1−L)B∗(L),weobtain:∗∗[B(1)+(1−L)B(L)]yt=B(1)yt+B(L)∆yt=etsothatX∞∗yt=−[B1+B2+···]yt−1−Bj∆yt−j+et.j=1Subtractingyt−1frombothsides,wegetX∞∆yt=−B(1)yt−1−Bj∆yt−j+et.j=1ThematrixB(1)controlsthecointegrationproperties:1.IfB(1)isfullrank,anylinearcombinationofytisstationaryandytisstationary.Inthiscase,werunanormalVAR. CHAPTER6.COINTEGRATION2402.IfB(1)hasrankbetween0andfullrank.Therearesomelinearcombinationsofytthatarestationary,soytissta-tionary.TheVARinlevelsisconsistentbutinefficientifyouknowthecointegratingvectorandtheVARindiffer-encesismisspecifiedinthiscase.3.IfB(1)hasrankzero,sonolinearcombinationofytissta-tionaryand∆ytisstationarywithnocointegration.InthiscasewerunanormalVARindifferences.ErrorCorrectionRepresentationIfB(1)haslessthanfullrank,wecanexpressitas:′B(1)=γα.IfthereareKcointegratingvectors,thentherankofB(1)isKandγandαeachhaveKcolumns.ThenthesystemcanberewrittenasX∞′∗∆yt=−γαyt−1−Bj∆yt−j+et,j=1whereα′isaK×Nmatrixofcointegratingvectors.Theaboveexpressionisthewellknownerror-correctionmodel(ECM)repre-sentationoftheintegratedsystem.Itisnoteasytoestimatethismodelwhenallcointegratedvectorsinαareunknown.ConsideramultivariatemodelconsistingoftwovariablesxtandwtwhichareindividuallyI(1).Onemaymodelthesetwovariablesusingoneofthefollowingmodels:1.AVARmodelinlevels2.AVARinthefirstdifferences CHAPTER6.COINTEGRATION2413.AnECMrepresentation.Withcointegration,apureVARindifferencesismisspecified.∆xt=a(L)∆xt−1+b(L)∆zt−1+et∆zt=c(L)∆xt−1+d(L)∆zt−1+vtLookingattheerror-correctionform,thereisamissingregressor,αxxt−1+αzzt−1.Thisisaproblem.ApureVARinlevelsisalittlebitunconventionalsincethevariablesinthemodelarenonstationary.TheVARinlevelsisnotmisspecifiedandtheestimatesareconsistentbutthecoefficientsmayhavenon-standarddistributionsandtheyarenotefficient.Ifthereiscointegration,itimposesrestrictionsonB(1)thatarenotimposedinapureVARinlevels.Cochrane(1994)suggestedthatonewaytoimposecointegrationistorunanerror-correctionVAR:∆xt=γx(αxxt−1+αzzt−1)+a(L)∆xt−1+b(L)∆zt−1+et∆zt=γw(αxxt−1+αzzt−1)+c(L)∆xt−1+d(L)∆zt−1+vtThisspecificationimposesthatxandzarecointegratedwithcoin-tegratingvectorα.Thisisveryusefulifyouknowthatthevariablesarecointegratedandyouknowthecointegratingvector.Otherwise,youhavetopre-testforcointegrationandestimatethecointegratingvectorinaseparatestep.Anotherdifficultywiththeerror-correctionformisthatitdoesnotfitnicelyintostandardVARpackages.Awaytousestandardpackagesistoestimatecompanionform:∆xt=a(L)∆xt−1+b(L)(αxxt−1+αzzt−1)+etαxxt+αzzt=c(L)∆xt−1+d(L)(αxxt−1+αzzt−1)+vtWeneedtoknowcointegratingvectortousethisprocedure.Thereismuchdebateastowhichapproachisbest.Whenyoudonot CHAPTER6.COINTEGRATION242reallyknowwhetherthereiscointegrationorwhatthecointegratingvectoris,theVARinlevelsseemstobebetter.Whenyouknowthatthereiscointegrationandwhatthecointegratingvectoris,theerror-correctionformmodelorVARincompanionformisbetter.Someofunitrootandcointegrationtestsareprovidedbythepack-agestseries,urcaandurootinR.Forexample,intseries,thefunctionpo.testisforthePhillip-Ouliaris’s(1990)testfortestingthatxisnotcointegrated.Therearemanyothertestmethodsavail-ableinthepackagesurcaanduroot;fordetails,seetheirmanuals.6.5Problems1.TheinterestratesmaybefoundintheExcelfile“IntRates.xls”.Checkwhether90-dayT-billrateand10-yearT-bondinterestratearecointegrated.Carefullyexplainhowyouconductthetestandhowyouinterpretyourfindings.2.Downloaddata(byyourself)fromthewebsitetohavedataonconsumerpriceindex(CPI),producerpriceindex(PPI),three-monthT-billrate,theindexofindustrialproduction,S&P500commonstockpriceindex.Thedataforindustrialproductionisseasonallyadjustedwhileallothervariablesarenotseasonallyadjusted.Conductatestofcointegrationbetweenthevariables.Explainyourresults.3.CollectthefollowingmacroeconomicannualtimeseriesofChinafromChinaStatisticalYearbook:GNP,GDP,GDP1(GDPofprimaryindustry),GDP2(GDPofsecondaryindustry),GDP3(GDPoftertiaryindustry),andpercapitaGDP.Fromboththenominalandrealtermsofthedefinitionsofnationalproducts,derivethecorrespondingpricedeflators. CHAPTER6.COINTEGRATION243(a)Defineandtesttheunitrootsforeachofthe18economicvari-ables(nominal,real,anddeflatorofGDPs)assumingnostruc-turalbreakintheseries.(b)Defineandtesttheunitrootsforeachofthe18economicvari-ables(nominal,real,anddeflatorofGDPs)assumingone-timestructuralbreakintheseries.(c)Conductcointegrationtestsforsomeofthe18economicvari-ables.Explainyourresults.6.6ReferencesBernanke,B.S.,J.BoivinandE.Piotr(2005).Measuringtheeffectsofmonetarypolicy:afactor-augmentedvectorautoregressive(FAVAR)approach.TheQuarterlyJournalofEconomics,120,387-422.Borenstein,S.,A.C.CameronandR.Gilbert(1997).Dogasolinepricesrespondasymmet-ricallytocrudeoilpricechanges?TheQuarterlyJournalofEconomics,112,305-339.Christiano,L.J.,M.EichenbaumandC.L.Evans(2000).Monetarypolicyshocks:whathavewelearnedandtowhatend?HandbookofMacroeconomics,Vol.1A.Cochrane,J.H.(1994).Shocks.NBERworkingpaper#46984.Cochrane,J.H.(1997).Timeseriesformacroeconomicsandfinance.LectureNotes.Engle,R.F.andC.W.J.Granger(1987).Cointegrationanderrorcorrection:Representa-tion,estimationandtesting.Econometrica,55,251-276.Hamilton,J.D.(1994).TimeSeriesAnalysis.PrincetonUniversityPress.Hendry,F.D.andK.Juselius(2000).Explainingcointegrationanalysis:PartI.JournalofEnergy,21,1-42.Phillip,P.C.B.andS.Ouliaris(1990).Asymptoticpropertiesofresidualbasedontestsforcointegration.Econometrica,58,165-193.Stock,J.H.andM.W.Watson(1993).Asimpleestimatorofcointegratingvectorsinhigherorderintegratedsystems.Econometrica,61,1097-1107.Stock,J.H.andM.W.Watson(2003).IntroductiontoEconometrics.Addison-Wesley. Chapter7NonparametricDensity,Distribution&QuantileEstimation7.1MixingConditionsItiswellknownthatα-mixingincludesmanytimeseriesmodelsasaspecialcase.Infact,underverymildassumptionslinearautore-gressiveandmoregenerallybilineartimeseriesmodelsareα-mixingwithmixingcoefficientsdecayingexponentially.Manynonlineartimeseriesmodels,suchasfunctionalcoefficientautoregressiveprocesseswith/withoutexogenousvariables,ARCHandGARCHtypepro-cesses,stochasticvolatilitymodels,andnonlinearadditiveautore-gressivemodelswith/withoutexogenousvariables,arestrongmixingundersomemildconditions.SeeCai(2002)andChenandTang(2005)formoredetails.Tosimplifythenotation,weonlyintroducemixingconditionsforstrictlystationaryprocesses(inspiteofthefactthatamixingprocessisnotnecessarilystationary).Theideaistodefinemixingcoefficientstomeasurethestrength(indifferentways)ofdependenceforthetwosegmentsofatimeserieswhichareapartfromeachotherintime.244 CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION245Let{Xt}beastrictlystationarytimeseries.Forn≥1,defineα(n)=sup|P(A)P(B)−P(AB)|,A∈F0∞−∞;B∈FnjwhereFidenotestheσ-algebrageneratedby{Xt;i≤t≤j}.NotethatF∞↓.Ifα(n)→0asn→∞,{X}iscalledα-mixingorntstrongmixing.Thereareseveralothermixingconditionssuchasρ-mixing,β-mixing,φ-mixing,andψ-mixing;seethebooksbyHallandHeyde(1980)andFanandYao(2003,page68).Itiswellknownthattherelationshipsamongthemixingconditionsareψ-mixing=⇒φ-mixing=⇒ρ-mixingandβ-mixing=⇒α-mixing.Lemma1:(Davydov’sinequality)(i)IfE|X|p+E|X|q<∞forijsomep≥1andq≥1and1/p+1/q<1,itholdsthat1/rp1/pq1/q|Cov(Xi,Xj)|≤8α(|j−i|){E|Xi|}{E|Xj|},wherer=(1−1/p−1/q)−1.(ii)IfP(|Xi|≤C1)=1andP(|Xj|≤C2)=1forsomeconstantsC1andC2,itholdsthat|Cov(Xi,Xj)|≤4α(|j−i|)C1C2.NotethatifweallowXiandXjtobecomplex-valuedrandomvari-ables,(ii)stillholdswiththecoefficient“4”ontheRHSofthein-equalityreplacedby“16”.7.2DensityEstimateLet{Xi}bearandomsamplewitha(unknown)marginaldistri-butionF(·)(CDF)anditsprobabilitydensityfunction(PDF)f(·).Thequestionishowtoestimatef(·)andF(·).SinceZxF(x)=P(Xi≤x)=E[I(Xi≤x)]=f(u)du,−∞ CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION246andF(x+h)−F(x−h)F(x+h)−F(x−h)f(x)=lim≈h↓02h2hifhisverysmall,bythemethodofmomentestimation(MME),F(x)canbeestimatedby1XnFn(x)=I(Xi≤x),ni=1whichiscalledtheempiricalcumulativedistributionfunction(ecdf),sothatf(x)canbeestimatedbyFn(x+h)−Fn(x−h)1Xnfn(x)==Kh(Xi−x),2hni=1whereK(u)=I(|u|≤1)/2andKh(u)=K(u/h)/h.Indeed,thekernelfunctionK(u)canbetakentobeanysymmetricdensityfunction.Exercise:PleaseshowthatFn(x)isunbiasedestimateofF(x)butfn(x)isbiasedestimateoff(x).Thinkaboutintuitively(1)whyfn(x)isbiased(2)wherethebiascomesfrom(3)whyK(·)shouldbesymmetric.7.2.1AsymptoticPropertiesLetuslookatthevarianceofestimators.If{Xi}isstationary,thenXni−1nVar(Fn(x))=Var(I(Xi≤x))+21−Cov(I(X1≤x),I(Xi≤i=2nXn=F(x)[1−F(x)]+2Cov(I(X1≤x),I(Xi≤x))i=2|{z}→σ2(x)byassumingthatσ2(x)<∞ CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION247Xni−1−2Cov(I(X1≤x),I(Xi≤x))i=2n|{z}→0byKroneckerLemmaX∞2→σF(x)≡F(x)[1−F(x)]+2Cov(I(X1≤x),I(Xi≤x))i=2|{zThistermiscalledAdTherefore,2nVar(Fn(x))→σF(x).(7.1)ItisclearthatAd=0if{Xi}areindependent.IfAd6=0,thequestionishowtoestimateit.WecanusetheHCestimatorbyWhite(1980)ortheHACestimatorbyNeweyandWest(1987);seeSection3.10.Next,wederivetheasymptoticvarianceforfn(x).First,defineZi=Kh(Xi−x).Then,ZZE[Z1Zi]=Kh(u−x)Kh(v−x)f1,i(u,v)dudvZZ=K(u)K(v)f1,i(x+uh,x+vh)dudv→f1,i(x,x),wheref1,i(u,v)isthejointdensityof(X1,Xi),sothat2Cov(Z1,Zi)→f1,i(x,x)−f(x).ItiseasytoshowthathVar(Z1)→ν0(K)f(x),Rj2whereνj(K)=uK(u)du.Therefore,Xni−1nhVar(fn(x))=Var(Z1)+2h1−Cov(Z1,Zi)i=2n|{z}≡Af→0undersomeassumptions→ν0(K)f(x). CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION248ToshowthatAf→0,letdn→∞anddnh→0.Then,XdnXn|Af|≤h|Cov(Z1,Zi)|+h|Cov(Z1,Zi)|.i=2i=dn+1Forthefirstterm,iff1,i(u,v)≤M1,then,itisboundedbyhdn=o(1).Forthesecondterm,weapplytheDavydov’sinequalitytoobtainXnXn−β+1−1h|Cov(Z1,Zi)|≤M2α(i)/h=O(dnh)i=dn+1i=dn+1ifα(n)=O(n−β)forsomeβ>2.Ifd=O(h−2/β),then,thensecondtermisdominatedbyO(h1−2/β)whichgoesto0asn→∞.Hence,nhVar(fn(x))→ν0(K)f(x).(7.2)Wecanestablishthefollowingasymptoticnormalityforfn(x)buttheproofwillbediscussedlater.Theorem1:Underregularityconditions,wehave√h2′′2nhfn(x)−f(x)−µ2(K)f(x)+op(h)→N(0,ν0(K)f(x)).2Exercise:Bycomparing(7.1)and(7.2),whatcanyouobserve?Example1:Letusexaminehowimportancethechoiceofband-widthis.Thedata{X}naregeneratedfromN(0,1)(iid)andii=1n=300.Thegridpointsaretakentobe[−4,4]withanincrement∆=0.1.Bandwidthistakentobe0.25,0.5and1.0,respectivelyandthekernelcanbetheEpanechnikovkernelorGaussiankernel.Example2:Next,weapplythekerneldensityestimationtothedensityoftheweekly3-monthTreasurybillfromJanuary2,1970toDecember26,1997. CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION249NotethatthecomputercodeinRfortheabovetwoexamplescanbefoundinSection7.5.Rhasabuild-infunctiondensity()forcomputingthenonparametricdensityestimation.Also,youcanusethecommandplot(density())toplottheestimateddensity.Further,Rhasabuild-infunctionecdf()forcomputingtheempir-icalcumulativedistributionfunctionestimationandplot(ecdf())forplottingthestepfunction.7.2.2OptimalityAswealreadyhaveshownthath2′′2E(fn(x))=f(x)+µ2(K)f(x)+o(h),2andν0(K)f(x)−1Var(fn(x))=+o((nh)),nhsothattheasymptoticmeanintegratedsquareserror(AMISE)ish4Zν(K)AMISE=µ2(K)[f′′(x)]2+0.24nhMinimizingtheAMISEgivestheh=C(K)||f′′||−2/5n−2/5,(7.3)opt12where1/52C1(K)=ν0(K)/µ2(K).Withthisasymptoticallyoptimalbandwidth,theoptimalAMISEisgivenby5′′2/5−4/5AMISEopt=C2(K)||f||2n,4where2/52C2(K)=ν0(K)µ2(K). CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION250Tochoosethebestkernel,itsufficestochooseonetominimizeC2(K).Proposition1:ThenonnegativeprobabilitydensityfunctionKminimizingC2(K)isare-scalingoftheEpanechnikovkernel:322Kopt(u)=(1−u/a)+4aforanya>0.Proof:Firstofall,wenotethatC2(Kh)=C2(K)foranyh>0.LetK0betheEpanechnikovkernel.ForanyothernonnegativenegativeK,byre-scalingifnecessary,weassumethatµ2(K)=µ2(K0).Thus,weneedonlytoshowthatν0(K)≤ν0(K).LetG=K−K0.Then,ZZ2G(u)du=0anduG(u)du=0,whichimpliesthatZ2(1−u)G(u)du=0.UsingthisandthefactthatK0hasthesupport[−1,1],wehaveZ3Z2G(u)K0(u)du=G(u)(1−u)du4|u|≤13Z3Z22=−G(u)(1−u)du=K(u)(u−1)du.4|u|>14|u|>1SinceKisnonnegative,soisthelastterm.Therefore,ZZZZZ2222K(u)du=K0(u)du+2K0(u)G(u)du+G(u)du≥K0(u)du,whichprovesthatK0istheoptimalkernel.Remark:ThispropositionimpliesthattheEpanechnikovkernelshouldbeusedinpractice. CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION2517.2.3BoundaryCorrectionInmanyapplications,thedensityf(·)hasaboundedsupport.Forexample,theinterestratecannotbelessthanzeroandtheincomeisalwaysnonnegative.Itisreasonabletoassumethattheinterestratehassupport[0,1).However,becauseakerneldensityestima-torspreadssmoothlypointmassesaroundtheobserveddatapoints,someofthoseneartheboundaryofthesupportaredistributedout-sidethesupportofthedensity.Therefore,thekerneldensityes-timatorunderestimatesthedensityintheboundaryregions.Theproblemismoresevereforlargebandwidthandfortheleftboundarywherethedensityishigh.Therefore,someadjustmentsareneeded.Togainsomefurtherinsights,letusassumewithoutlossofgeneralitythatthedensityfunctionf(·)hasaboundedsupport[0,1]andwedealwiththedensityestimateattheleftboundary.Forsimplicity,supposethatK(·)hasasupport[−1,1].Fortheleftboundarypointx=ch(0≤c<1),itcaneasilybeseenthatash→0,Z1/h−cE(fn(ch))=f(ch+hu)K(u)du−c′=f(0+)µ0,c(K)+hf(0+)[cµ0,c(K)+µ1,c(K)]+o(7.4)(h),wheref(0+)=limx↓0f(x),Z∞Z∞jj2µj,c=uK(u)du,andνj,c(K)=uK(u)du.−c−cAlso,wecanshowthatVar(fn(ch))=O(1/nh).Therefore,′fn(ch)=f(0+)µ0,c(K)+hf(0+)[cµ0,c(K)+µ1,c(K)]+op(h).Particularly,ifc=0andK(·)issymmetric,thenE(fn(0))=f(0)/2+o(1).Thereareseveralmethodstodealwiththedensityestimationatboundarypoints.Possibleapproachesincludetheboundaryker-nel(seeGasserandM¨uller(1979)andM¨uller(1993)),reflection CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION252(seeSchuster(1985)andHallandWehrly(1991)),transformation(seeWand,MarronandRuppert(1991)andMarronandRuppert(1994))andlocalpolynomialfitting(seeHjortandJones(1996)andLoader(1996)),andothers.BoundaryKernelOnewayofchoosingaboundarykernelis123c2−2c+1K(c)(u)=(1+c)4(1+u)(1−2c)u+2I[−1,c].NoteK(1)(t)=K(t),theEpanechnikovkernelasdefinedabove.Moreover,ZhangandKarunamuni(1998)haveshownthatthiskernelisoptimalinthesenseofminimizingtheMSEintheclassofallkernelsoforder(0,2)withexactlyonechangeofsignintheirsupport.Thedownsidetotheboundarykernelisthatitisnotnecessarilynon-negative,aswillbeseenondensitieswheref(0)=0.ReflectionThereflectionmethodistoconstructthekerneldensityestimatebasedonthesyntheticdata{±Xt;1≤t≤n}where“reflected”dataare{−Xt;1≤t≤n}andtheoriginaldataare{Xt;1≤t≤n}.Thisresultsintheestimate1XnXnfn(x)=Kh(Xt−x)+Kh(−Xt−x),forx≥0.nt=1t=1Notethatwhenxisawayfromtheboundary,thesecondtermintheaboveispracticallynegligible.Hence,itonlycorrectstheestimateintheboundaryregion.Thisestimatoristwicethekerneldensityestimatebasedonthesyntheticdata{±Xt;1≤t≤n}. CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION253TransformationThetransformationmethodistofirsttransformthedatabyYi=g(Xi),whereg(·)isagivenmonotoneincreasingfunction,rangingfrom−∞to∞.Nowapplythekerneldensityestimatortothistransformeddatasettoobtaintheestimatefn(y)forYandapplytheinversetransformtoobtainthedensityofX.Therefore,1Xn′fn(x)=g(x)Kh(g(Xt)−g(x)).nt=1Thedensityatx=0correspondstothetaildensityofthetrans-formeddatasincelog(0)=−∞,whichcannotusuallybeestimatedwellduetolackofthedataattails.Exceptatthispoint,thetrans-formationmethoddoesafairlygoodjob.Ifg(·)isunknowninmanysituations,KarunamuniandAlberts(2003)suggestedaparametricformandthenestimatedtheparameter.Also,KarunamuniandAl-berts(2003)consideredothertypesoftransformations.LocalLikelihoodFittingThemainideaistoconsidertheapproximationlog(f(Xt))≈P(Xt−Ppjx),whereP(u−x)=j=0aj(u−x)withthelocalizedversionoflog-likelihoodXnZlog(f(Xt))Kh(Xt−x)−nKh(u−x)f(u)du.t=1Withthisapproximation,thelocallikelihoodbecomesXnZL(a0,···,dp)=P(Xt−x)Kh(Xt−x)−nKh(u−x)exp(P(u−x))du.t=1Let{abj}bethemaximizeroftheabovelocallikelihoodL(a0,···,dp).Then,thelocallikelihooddensityestimateisfn(x)=exp(ab0). CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION254Themaximizerdoesnotexist,thenfn(x)=0.SeeLoader(1996)andHjortandJones(1996)formoredetails.IfRisusedforthelocalfitfordensityestimation,pleaseusethefunctiondensity.lf()inthepackagelocalfit.Exercise:PleaseconductaMonteCarolsimulationtoseewhattheboundaryeffectsareandhowthecorrectionmethodswork.Forexample,youcanconsidersomedistributiondensitieswithafinitesupportsuchasbeta-distribution.7.3DistributionEstimation7.3.1SmoothedDistributionEstimationThequestionishowtoobtainasmoothedestimateofCDFF(x).Well,onewayofdoingsoistointegratetheestimatedPDFfn(x),givenbyZx1Xnx−XiFc(x)=f(u)du=K,nn−∞ni=1hRxwhereK(x)=−∞K(u)du;thedistributionofK(·).WhydoweneedthissmoothedestimateofCDF?Toanswerthisquestion,weneedtoconsiderthemeansquareserror(MSE).First,wederivetheasymptoticbias.Bytheintegrationbyparts,wehavex−XiZEFc(x)=EK=F(x−hu)K(u)dunhh2′2=F(x)+µ2(K)f(x)+o(h).2Next,wederivetheasymptoticvariance.x−XiZEK2=F(x−hu)b(u)du=F(x)−hf(x)θ+o(h),h CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION255Rwhereb(u)=2K(u)K(u)andθ=ub(u)du.Then,x−XiVarK=F(x)[1−F(x)]−hf(x)θ+o(h).hDefineI(x)=Cov(I(X≤x),I(X≤t))=F(x,x)−F2(x)j1j+1jandx−X1x−Xj+1Inj(x)=CovK,K.hhBymeansofLemma2inLehmann(1966),thecovarianceInj(x)maybewrittenasfollowsZx−X1x−Xj+1Inj(t)=PK>u,K>vhhx−X1x−Xj+1−PK>uPK>vdudv.hhInvertingtheCDFK(·)andmakingtwochangesofvariables,theaboverelationbecomesZInj(x)=[Fj(x−hu,x−hv)−F(x−hu)F(x−hv)]K(u)K(v)dudv.Expandingtheright-handsideoftheaboveequationaccordingtoTaylor’sformula,weobtain2|Inj(x)−Ij(x)|≤Ch.BytheDavydov’sinequality(seeLemma1),wehave|Inj(x)−Ij(x)|≤Cα(j),sothatforany1/2<τ<1,2τ1−τ|Inj(x)−Ij(x)|≤Chα(j).Therefore,1nX−1nX−1X∞2τ1−τ(n−j)|Inj(x)−Ij(x)|≤|Inj(x)−Ij(x)|≤Chα(j)=Onj=1j=1j=1 CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION256providedthatP∞α1−τ(j)<∞forsome1/2<τ<1.Indeed,j=1thisassumptionissatisfiedifα(n)=O(n−β)forsomeβ>2.Bythestationarity,itisclearthatx−X12nX−1nVarFc(x)=VarK+(n−j)I(x).nnjhnj=1Therefore,X∞c2τnVarFn(x)=F(x)[1−F(x)]−hf(x)θ+o(h)+2Ij(x)+O(h)j=12=σF(x)−hf(x)θ+o(h).cWecanestablishthefollowingasymptoticnormalityforFn(x)buttheproofwillbediscussedlater.Theorem2:Underregularityconditions,wehave√h2c′22nFn(x)−F(x)−µ2(K)f(x)+op(h)→N0,σF(x).2Similarly,wehavenh4c2′22nAMSE(Fn(x))=µ2(K)[f(x)]+σF(x)−hf(x)θ.4Ifθ>0,minimizingtheAMSEgivesthe1/3θf(x)−1/3hopt=n,µ2(K)[f′(x)]22andwiththisasymptoticallyoptimalbandwidth,theoptimalAMSEisgivenby222/33θf(x)c2−1/3nAMSEopt(Fn(x))=σF(x)−n.4µ2(K)f′(x) CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION257Remark:Fromtheaforementionedequation,wecanseethatifcθ>0,theAMSEofFn(x)canbesmallerthanthatforFn(x)inthesecondorder.Also,itiseasytothatifK(·)istheEpanechnikovkernel,θ>0.7.3.2RelativeEfficiencyandDeficiencycTomeasuretherelativeefficiencyanddeficiencyofFn(x)overFn(x),wedefineci(n)=mink∈{1,2,...};MSE(Fk(x))≤MSEFn(x).WehavethefollowingresultswithoutthedetailedproofwhichcanbefoundinCaiandRoussas(1998).Proposition2:(i)Underregularityconditions,i(n)4→1,ifandonlyifnhn→0.n(ii)Underregularityconditions,i(n)−n3→θ(x),ifandonlyifnhn→0,nhwhereθ(x)=f(x)θ/σ2(x).FRemark:Itisclearthatthequantityθ(x)maybelookeduponascawayofmeasuringtheperformanceoftheestimateFn(x).SupposethatthekernelK(·)ischosen,sothatθ>0,whichisequivalenttoθ(x)>0.Then,forsufficientlylargen,i(n)>n+nhn(θ(x)−ε).Thus,i(n)issubstantiallylargerthann,and,indeed,i(n)−ntendsto∞.Actually,Reiss(1981)andFalk(1983)posedthequestionofdeterminingtheexactvalueofthesuperiorityofθoveracertain CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION258classofkernels.Morespecifically,letKmbetheclassofkernelsK:[−1,1]→ℜwhichareabsolutelycontinuousandsatisfytheR1µrequirements:K(−1)=0,K(1)=1,and−1uK(u)du=0,µ=1,···,m,forsomem=0,1,···(wherethemomentconditionisvacuousform=0).SetΨm=sup{θ;K∈Km}.Then,Mammitzsch(1984)answeredthequestionposedbyshowinginanelegantmanner.SeeCaiandRoussas(1998)formoredetailsandsimulationresults.7.4QuantileEstimationLetX≤X≤···≤Xdenotetheorderstatisticsof{X}n.(1)(2)(n)tt=1DefinetheinverseofF(x)asF−1(p)=inf{x∈ℜ;F(x)≥p},whereℜistherealline.ThetraditionalestimateofF(x)hasbeentheempiricaldistributionfunctionFn(x)basedonX1,...,Xn,whiletheestimateofthep-thquantileξ=F−1(p),00,minimizingtheAMSEgivesthe1/3θf(ξp)−1/3hopt=n,µ2(K)[f′(ξ)]22pandwiththisasymptoticallyoptimalbandwidth,theoptimalAMSEisgivenby22/33θb22−1/3nAMSEopt(ξp)=σF(ξp)/f(ξp)−n,4µ2(K)f′(ξp)f(ξp)whichindicatesareductiontotheAMSEofthesecondorder.ChenandTang(2005)conductedanintensivestudyonsimulationstobdemonstratetheadvantagesofnonparametricestimationξpoverthesamplequantileξpnundertheVaRsetting.WerefertothepaperbyChenandTang(2005)forsimulationresultsandempiricalexamples.Exercise:Pleaseusetheaboveprocedurestoestimatenonparamet-ricallytheESanddiscussitspropertiesaswellasconductsimulationstudiesandempiricalapplications.7.5ComputerCode#July20,2006graphics.off()#cleanthepreviousgarphsonthesecreen CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION262##########################################################DefinetheEpanechnikovkernelfunctionkernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}################################################################Definethekerneldensityestimatorkernden=function(x,z,h,ker){#parameters:x=variable;h=bandwidth;z=gridpoint;ker=kernelnz<-length(z)nx<-length(x)x0=rep(1,nx*nz)dim(x0)=c(nx,nz)x1=t(x0)x0=x*x0x1=z*x1x0=x0-t(x1)if(ker==1){x1=kernel(x0/h)}#Epanechnikovkernelif(ker==0){x1=dnorm(x0/h)}#normalkernelf1=apply(x1,2,mean)/hreturn(f1)}#######################################################################################################################################Simulationfordifferentbandiwidthsanddifferentkernelsn=300#n=300ker=1#ker=1=>Epan;ker=0=>Gaussianh0=c(0.25,0.5,1)#setinitialbandwidthsz=seq(-4,4,by=0.1)#gridpointsnz=length(z)#numberofgridpointsx=rnorm(n)#simulatex~N(0,1)if(ker==1){h_o=2.34*n^{-0.2}}#optimalbandwidthforEpanechnikov CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION263if(ker==0){h_o=1.06*n^{-0.2}}#optimalbandwidthfornormalf1=kernden(x,z,h0[1],ker)f2=kernden(x,z,h0[2],ker)f3=kernden(x,z,h0[3],ker)f4=kernden(x,z,h_o,ker)text1=c("True","h=0.25","h=0.5","h=1","h=h_o")data=cbind(dnorm(z),f1,f2,f3,f4)#combinethemwin.graph()matplot(z,data,type="l",lty=1:5,col=1:5,xlab="",ylab="")legend(-1,0.2,text1,lty=1:5,col=1:5)#######################################################################################################################################ARealExample##################z1=matrix(scan(file="c:\teaching\timeseries\data\w-3mtbs7097.txt"),byrow=T,ncol=4)#dada:weekly3-monthTreasurybillfrom1970to1997x=z1[,4]/100#decimaln=length(x)y=diff(x)#Deltax_t=x_t-x_{t-1}=changex=x[1:(n-1)]n=n-1x_star=(x-mean(x))/sqrt(var(x))#standardizedden_3mtb=density(x_star,bw=0.30,kernel=c("epanechnikov"),from=-3den_est=den_3mtb$y#estimateddensityvaluesz_star=seq(-3,3,by=0.1)text1=c("EstimatedDensity","StandardNorm")win.graph() CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION264par(bg="lightgreen")plot(den_3mtb,main="Densityof3mtb(Buind-in)",ylab="",xlab="",col.main="red")points(z_star,dnorm(z_star),type="l",lty=2,col=2,ylab="",xlab=""legend(0,0.45,text1,lty=c(1,2),col=c(1,2),cex=0.7)h_den=0.5f_hat=kernden(x_star,z_star,h_den,1)ff=cbind(f_hat,dnorm(z_star))win.graph()par(bg="lightblue")matplot(z_star,ff,type="l",lty=c(1,2),col=c(1,2),ylab="",xlab=title(main="Densityof3mtb",col.main="red")legend(0,0.55,text1,lty=c(1,2),col=c(1,2),cex=0.7)###################################################################7.6ReferencesArtzner,P.,F.Delbaen,J.M.Eber,andD.Heath(1999).Coherentmeasuresofrisk.MathematicalFinance,9,203-228.Cai,Z.(2002).Regressionquantilefortimeseries.EconometricTheory,18,169-192.Cai,Z.andG.G.Roussas(1997).Smoothestimateofquantilesunderassociation.StatisticsandProbabilityLetters,36,275-287.Cai,Z.andG.G.Roussas(1998).Efficientestimationofadistributionfunctionunderquadrantdependence.ScandinavianJournalofStatistics,25,211-224.Chen,S.X.andC.Y.Tang(2005).Nonparametricinferenceofvalueatriskfordependentfinancialreturns.JournalofFinancialEconometrics,3,227-255.Duffie,D.andJ.Pan(1997).Anoverviewofvalueatrisk.JournalofDerivatives,4,7-49.Fan,J.andQ.Yao(2003).NonlinearTimeSeries:NonparametricandParametricMeth-ods.Springer-Verlag,NewYork. CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION265Gasser,T.andH.-G.M¨uller(1979).Kernelestimationofregressionfunctions.InSmoothingTechniquesforCurveEstimation,LectureNotesinMathematics,757,23-68.Springer-Verlag,NewYork.Falk,M.(1983).Relativeefficiencyanddeficiencyofkerneltypeestimatorsofsmoothdistributionfunctions.StatisticaNeerlandica,37,73-83.Hall,P.andC.C.Heyde(1980).MartingaleLimitTheoryanditsApplications.AcademicPress,NewYork.Hall,P.andT.E.Wehrly(1991).Ageometricalmethodforremovingedgeeffectsfromkernel-typenonparametricregressionestimators.JournalofAmericanStatisticalAs-sociation,86,665-672.Hjort,N.L.andM.C.Jones(1996).Locallyparametricnonparametricdensityestimation.TheAnnalsofStatistics,24,1619-1647.Jorion,P.(2001).ValueatRisk,2ndEdition.NewYork:McGraw-Hill.Karunamuni,R.J.andT.Alberts(2003).Onboundarycorrectioninkerneldensityestima-tion.Workingpaper,DepartmentofMathematicalandStatisticalSciences,UniversityofAlberta,Canada.Lehmann,E.(1966).Someconceptsofdependence.AnnalsofMathematicalStatistics,37,1137-1153.Loader,C.R.(1996).Locallikelihooddensityestimation.TheAnnalsofStatistics,24,1602-1618.Mammitzsch,V.(1984).Ontheasymptoticallyoptimalsolutionwithinacertainclassofkerneltypeestimators.StatisticsDecisions,2,247-255.Marron,J.S.andD.Ruppert(1994).Transformationstoreduceboundarybiasinkerneldensityestimation.JournaloftheRoyalStatisticalSocietySeriesB,56,653-671.M¨uller,H.-G.(1993).Ontheboundarykernelmethodfornonparametriccurveestimationnearendpoints.ScandinavianJournalofStatistics,20,313-328.Reiss,R.D.(1981).Nonparametricestimationofsmoothdistributionfunctions.Scandi-naviaJournalofStatistics,8,116-119.Schuster,E.F.(1985).Incorporatingsupportconstraintsintononparametricestimatesofdensities.CommunicationsinStatisticsTheoryandMethods,14,1123-1126.Wand,M.P.,J.S.MarronandD.Ruppert(1991).Transformationsindensityestimation(withdiscussion).JournaloftheAmericanStatisticalAssociation,86,343-361.Yoshihara,K.(1995).TheBahadurrepresentationofsamplequantilesforsequencesofstronglymixingrandomvariables.StatisticsandProbabilityLetters,24,299-304. CHAPTER7.NONPARAMETRICDENSITY,DISTRIBUTION&QUANTILEESTIMATION266Zhang,S.andR.J.Karunamuni(1998).OnKerneldensityestimationnearendpoints.JournalofStatisticalPlanningandInference,70,301-316. Chapter8NonparametricRegressionEstimation8.1BandwidthSelection8.1.1SimpleBandwidthSelectorsTheoptimalbandwidth(7.3)isnotdirectlyusablesinceitdependsontheunknownparameter||f′′||.Whenf(x)isaGaussiandensity2withstandarddeviationσ,itiseasytoseefrom(7.3)that√1/5−1/5hopt=(8π/3)C1(K)σn,whichiscalledthenormalreferencebandwidthselectorinliterature,obtainedbyreplacingtheunknownparameterσintheaboveequationbythesamplestandarddeviations.Inparticular,aftercalculatingtheconstantC1(K)numerically,wehavethefol-lowingnormalreferencebandwidthselector−1/5b1.06snfortheGaussiankernelhopt=−1/52.34snfortheEpanechnikovkernelHjortandJones(1996)proposedanimprovedruleobtainedbyusinganEdgeworthexpansionforf(x)aroundtheGaussiandensity.Sucharuleisgivenby3535385−1/5hb∗=h1+γb+γb2+γ2,optopt43448321024267 CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION268whereγb3andγb4arerespectivelythesampleskewnessandkurtosis.Notethatthenormalreferencebandwidthselectorisonlyasim-pleruleofthumb.ItisagoodselectorwhenthedataarenearlyGaussiandistributed,andisoftenreasonableinmanyapplications.However,itcanleadtoover-smoothwhentheunderlyingdistribu-tionisasymmetricormulti-modal.Inthatcase,onecaneithersubjectivelytunethebandwidth,orselectthebandwidthbymoresophisticatedbandwidthselectors.Onecanalsotransformdatafirsttomaketheirdistributionclosertonormal,thenestimatethedensityusingthenormalreferencebandwidthselectorandapplytheinversetransformtoobtainanestimateddensityfortheoriginaldata.Suchamethodiscalledthetransformationmethod.Therearequiteafewimportanttechniquesforselectingthebandwidthsuchascross-validation(CV)andplug-inbandwidthselectors.Aconceptuallysimpletechnique,withtheoreticaljustificationandgoodempiricalperformance,istheplug-intechnique.Thistechniquereliesonfind-inganestimateofthefunctional||f′′||,whichcanbeobtainedby2usingapilotbandwidth.Animplementationofthisapproachispro-posedbySheatherandJones(1991)andanoverviewontheprogressofbandwidthselectioncanbefoundinJones,MarronandSheather(1996).Functiondpik()inthepackageKernSmoothinRselectsabandwidthforestimatingthekerneldensityestimationusingtheplug-inmethod.8.1.2Cross-ValidationMethodTheintegratedsquarederror(ISE)offn(x)isdefinedbyZ2ISE(h)=[fn(x)−f(x)]dx. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION269Acommonlyusedmeasureofdiscrepancybetweenfn(x)andf(x)isthemeanintegratedsquarederror(MISE)MISE(h)=E[ISE(h)].Itcanbeshowneasily(orseeChiu,1991)thatMISE(h)≈AMISE(h).TheoptimalbandwidthminimizingtheAMISEisgivenin(7.3).Theleastsquarescross-validation(LSCV)methodproposedbyRudemo(1982)andBowman(1984)isapopularmethodtoestimatetheoptimalbandwidthhopt.Cross-validationisveryusefulforassessingtheperformanceofanestimatorviaestimatingitspredictionerror.Thebasicideaistosetoneofthedatapointasideforvalidationofamodelandusetheremainingdatatobuildthemodel.ThemainideaistochoosehtominimizeISE(h).SinceZZZ22ISE(h)=fn(x)dx−2f(x)fn(x)dx+f(x)dx,thequestionishowtoestimatethesecondtermontherighthandside.Well,letusconsiderthesimplestcasewhen{Xt}areiid.Re-expressfn(x)asn−11(−s)fn(x)=fn(x)+Kh(Xs−x)nnforany1≤s≤n,where1Xn(−s)fn(x)=Kh(Xt−x),n−1t6=swhichisthekerneldensityestimatewithoutthesthobservation,commonlycalledthejackknifeestimateorleave-one-outestimate.Itiseasytoseethatforany1≤s≤n,(−s)fn(x)≈fn(x).LetDs={X1,···,Xs−1,Xs+1,···,Xn}.Then,ZZ(−s)(−s)Efn(Xs)|Ds=fn(x)f(x)dx≈fn(x)f(x)dx, CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION270which,byusingthemethodofmoment,canbeestimatedby1Pnf(−s)(X).ns=1nsTherefore,thecross-validationisZ2Xn2(−s)CV(h)=fn(x)dx−fn(Xs)ns=11X2Xn∗=2Kh(Xs−Xt)−Kh(Xs−Xt),ns,tn(n−1)t6=swhereK∗(·)istheconvolutionofK(·)andK(·)ashhhZ∗Kh(u)=Kh(v)Kh(u−v)dv.bLethcvbetheminimizerofCV(h).Then,itiscalledtheoptimalbandwidthbasedonthecross-validation.Stone(1984)showedthatbhcvisaconsistentestimateoftheoptimalbandwidthhopt.Functionlscv()inthepackagelocfitinRselectsabandwidthforestimatingthekerneldensityestimationusingtheleastsquarescross-validationmethod.8.2MultivariateDensityEstimationAswediscussedinChapter7,thekerneldensityordistributionesti-mationisbasicallyone-dimensional.Formultivariatecase,thekerneldensityestimateisgivenby1Xnfn(x)=KH(Xt−x),(8.1)nt=1whereK(u)=K(H−1u)/det(H),K(u)isamultivariatekernelHfunction,andHisthebandwidthmatrixsuchasforall1≤i,j≤p,nhij→∞andhij→0wherehijisthe(i,j)thelementofH.Thebandwidthmatrixisintroducedtocapturethedependentstructureintheindependentvariables.Particularly,ifHisadiagonalmatrix CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION271QpandK(u)=j=1Kj(uj)whereKj(·)isaunivariatekernelfunction,then,fn(x)becomes1XnYpfn(x)=Khj(Xjt−xj),nt=1j=1whichiscalledtheproductkerneldensityestimation.Thiscaseiscommonlyusedinpractice.Similartotheunivariatecase,itiseasytoderivethetheoreticalresultsforthemultivariatecase,whichisleftasanexercise.SeeWandandJones(1995)fordetails.Exercise:Pleasederivetheasymptoticresultsgivenin(8.1)forthegeneralmultivariatecase.InR,thebuild-infunctiondensity()isonlyforunivariatecase.Formultivariatesituations,therearetwopackagesksandKernS-mooth.Functionkde()inkscancomputethemultivariatedensityestimatefor2-to6-dimensionaldataandFunctionbkde2D()inKernSmoothcomputesthe2Dkerneldensityestimate.Also,ksprovidessomefunctionsforsomebandwidthmatrixselectionsuchasHbcv()andHscvfor2DcaseandHlscv()andHpi().8.3RegressionFunctionSupposethatwehavetheinformationsetItattimetandwewanttoforecastthefuturevalue,sayYt+1(onestep-aheadforecast,orYt+s,s-stepahead).Thereareseveralforecastingcriteria.Thegeneralformism(It)=minaE[ρ(Yt+1−a)|It],whereρ(·)isanobjective(loss)function.Therearethreemajorcriteria:(1)Ifρ(·)isthequadraticfunction,then,m(It)=E(Yt+1|It), CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION272calledthemeanregressionfunction.(2)Ifρτ(y)=y(τ−I{y<0})calledthe“check”function,whereτ∈(0,1)andIAistheindicatorfunctionofanysetA,then,m(It)satisfiesZm(It)f(y|It)du=F(m(It)|It)=τ,−∞wheref(y|It)andF(m(It)|It)aretheconditionalPDFandCDFofYt+1,respectively,givenIt.Thism(It)becomestheconditionalquantileorquantileregression,dentedbyqτ(It).Particularly,ifτ=1/2,then,m(It)isthewellknownabsolutedeviation(LAD)regressionwhichisrobust.(3)Ifρ(x)=1x2I+M(|x|−M/2)I,thesocalledHuber2|x|≤M|x|>Mfunctioninliterature,thenitistheHuberrobustregression.Wewillnotdiscussthistopic.Ifyouhaveaninterest,pleasereadthebookbyRousseeuwandLeroy(1987).InR,thelibraryMASShasthefunctionrlmforrobustlinearmodel.Also,thelibrarylqscontainsfunctionsforbounded-influenceregression.SincetheinformationsetItcontainstoomanyvariables(highdi-mension),itisoftentoapproximateItbysomefinitenumbersofvariables,sayX=(X,...,X)T(p≥1),includingthelaggedtt1tpvariablesandexogenousvariables.First,ourfocusisonthemeanregressionm(Xt).Ofcourse,bythesametoken,wecanconsiderthenonparametricestimationoftheconditionalvarianceσ2(x)=Var(Yt|Xt=x).Whydoweneedtoconsidernonlinear(nonpara-metric)modelsineconomicpractice?YoucanfindtheanswerinthebookbyGranger,C.W.J.,andT.Ter¨asvirta(1993). CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION2738.4KernelEstimationLetuslookattheNadaraya-Watsonestimateofthemeanregressionm(Xt).Themainideaisasfollows:RZyf(x,y)dym(x)=yf(y|x)dy=R,f(x,y)dywheref(x,y)isthejointPDFofXtandYt.Toestimatem(x),wecanapplytheplug-inmethod.Thatis,plugthenonparametrickerneldensityestimatefn(x,y)(doublekernelmethod)intotherighthandsideoftheaboveequationtoobtainRyfn(x,y)dymdnw(x)=Rfn(x,y)dy..=.1Xn=YtKh(Xt−x)/fn(x)nt=1Xn=WtYt,t=1wherefn(x)isthekerneldensityestimationoff(x),definedinChap-ter7,andXnWt=Kh(Xt−x)/Kh(Xt−x).t=1mdnw(x)isthewellknownNadaraya-Watson(NW)estimator.Notethattheweights{Wt}donotdependon{Yt}.Therefore,mdnw(x)iscalledalinearestimator,similartotheleastsquaresestimate(LSE).LetuslookattheNWestimatorfromadifferentangle.mdnw(x)canbere-expressedastheminimizeroftheweightedlocallyleastsquares;thatis,Xn2mdnw(x)=min(Yt−a)Kh(Xt−x).at=1 CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION274ThismeansthatwhenXtisinaneighborhoodofx,m(Xt)isapprox-imatedbyaconstanta(localapproximation).Indeed,weconsiderthefollowingworkingmodelYt=m(Xt)+εt≈a+εtwiththeweights{Kh(Xt−x)},whereεt=Yt−E(Yt|Xt).Intheimplementation,foreachx,wecanfitthefollowingtrans-formedlinearmodel∗∗Yt=β1Xt+εt,rrwhereY∗=K(X−x)YandX∗=K(X−x).Therefore,thttthttheNadaraya-Watsonestimatorisalsocalledthelocalconstantestimator.InR,wecanusefunctionslm()orglm()withweights{Kh(Xt−x)}tofitaweightedleastsquaresorgeneralizedlinearmodel.Or,youcanusetheweightedleastsquarestheory(matrixmultiplication).8.4.1AsymptoticPropertiesWederivetheasymptoticpropertiesofthenonparametricestimatorforthetimeseriessituations.Also,weconsiderthesimplecasethatp=1.1XnXnmdnw(x)=m(Xt)Kh(Xt−x)/fn(x)+Wtεt.nt=1t=1|{z}|{z}I1I2WewillshowthatI1contributesonlybiasandI2givestheasymptoticnormality.First,wederivetheasymptoticbiasfortheinteriorbound-arypoints.BytheTaylor’sexpansion,whenXtisin(x−h,x+h),wehave1′′′2m(Xt)=m(x)+m(x)(Xt−x)+m(xt)(Xt−x),2 CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION275wherext=x+θ(Xt−x)with−1<θ<1.Then,1XnI11=m(Xt)Kh(Xt−x)nt=11Xn′=m(x)fn(x)+m(x)(Xt−x)Kh(Xt−x)nt=1|{z}J1(x)11Xn′′2+m(xt)(Xt−x)Kh(Xt−x).2nt=1|{z}J2(x)Then,E[J1(x)]=E[(Xt−x)Kh(Xt−x)]Z=(u−x)Kh(u−x)f(u)duZ=huK(u)f(x+hu)du2′2=hf(x)µ2(K)+o(h).Similartothederivationofthevarianceoffn(x)in(7.2),wecanshowthatnhVar(J1(x))=O(1).Therefore,J(x)=h2f′(x)µ(K)+o(h2).Bythesametoken,we12phave′′2E[J2(x)]=Em(xt)(Xt−x)Kh(Xt−x)Z2′′2=hm(x+θhu)uK(u)f(x+hu)du2′′2=hm(x)µ2(K)f(x)+o(h)andVar(J(x))=O(1/nh).Therefore,J(x)=h2m′′(x)µ(K)f(x)+222o(h2).Hence,p1′I1=m(x)+m(x)J1(x)/fn(x)+J2(x)/fn(x)2h2′′′′2=m(x)+µ2(K)[m(x)+2m(x)f(x)/f(x)]+op(h)2 CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION276bythefactthatfn(x)=f(x)+op(1).Thetermh2′′′′Bnw(x)=µ2(K)[m(x)+2m(x)f(x)/f(x)]2isregardedastheasymptoticbias.Ifp>1(multivariatecase),Bnw(x)becomesh2′′′′TBnw(x)=trµ2(K)m(x)+2f(x)m(x)/f(x),(8.2)2RTwhereµ2(K)=uuK(u)du.Thebiasterminvolvesnotonlycurvaturesofm(x)butalsotheunknowndensityfunctionf(x)anditsderivativef′(x)sothatthedesigncannotbeadaptive.Undersomeregularityconditions,similarto(7.2),wecanshowthatforxbeinganinteriorgridpoint,p22nhVar(I2)→ν0(K)σε(x)/f(x)=σm(x),whereσ2(x)=Var(ε|X=x).Further,wecanestablishtheεttasymptoticnormality(theproofisprovidedlater)√nhpmd(x)−m(x)−B(x)+o(h2)→N0,σ2(x),nwnwpmwhereBnw(x)isgivenin(8.2).Whenpislarge,thereexiststhesocalled“curseofdimension-ality”.Tounderstandthisproblemquantitatively,wecanlookattherateofconvergence.ThebiasisoforderO(h2)andthevarianceisoforderO(/nhp).ThisleadstotheoptimalrateofconvergenceforMSEO(n−2/(4+p))bytradingofftheratesbetweenthebiasandvariance.Tohaveacomparableperformancewithone-dimensionalnonparametricregressionwithn1datapoints,forp-dimensionalnon-parametricregression,weneedn−2/(4+p)=O(n−2/5),1 CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION277Table8.1:Samplesizesrequiredforp-dimensionalnonparametricregressiontohavecompa-rableperformancewiththatof1-dimensionalnonparametricregressionusingsize100dimension2345678910samplesize2526311,5853,98210,00025,11963,096158,490398,108(p+4)/5orn=O(n1).Table8.1showstheresultwithn1=100.Theincreaseofrequiredsamplesizesisexponentiallyfast.8.4.2BoundaryBehaviorAsfortheboundarybehavioroftheNWestimator,wecanfollowFanandGijbels(1996).Withoutlossofgenerality,weconsidertheleftboundarypointx=ch,01involvesnofundamentallynewideas.Notethatmodelswithlargekareoftennotpracticallyusefulduetothe“curseofdimensionality”.Ifkislarge,toovercometheproblem,onewaytodosoistoconsideranindexfunctionalcoefficientmodelproposedbyFan,YaoandCai(2003)pXTm(u,x)=aj(βu)xj,(8.11)j=1whereβ1=1.Fan,YaoandCai(2003)studiedtheestimationprocedures,bandwidthselectionandapplications.HongandLee(2003)consideredtheapplicationsofmodel(8.11)totheexchangerates.8.6.2LocalLinearEstimationAsrecommendedbyFanandGijbels(1996),weestimatetheco-efficientfunctions{aj(·)}usingthelocallinearregressionmethod CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION294nTfromobservations{Ui,Xi,Yi}i=1,whereXi=(Xi1,...,Xip).Weassumethroughoutthataj(·)hasacontinuoussecondderivative.Notethatwemayapproximateaj(·)locallyatu0byalinearfunc-tionaj(u)≈aj+bj(u−u0).Thelocallinearestimatorisdefinedasbabj(u0)=abj,where{(abj,bj)}minimizethesumofweightedsquares2XnXpYi−{aj+bj(Ui−u0)}XijKh(Ui−u0),(8.12)i=1j=1whereK(·)=h−1K(·/h),K(·)isakernelfunctiononℜ1andh>0hisabandwidth.ItfollowsfromtheleastsquarestheorythatXnabj(u0)=Kn,j(Uk−u0,Xk)Yk,(8.13)k=1where!T−1xK(u,x)=eTXgWXgK(u)(8.14)n,jj,2puxhgej,2pisthe2p×1unitvectorwith1atthejthposition,XdenotesTTann×2pmatrixwith(Xi,Xi(Ui−u0))asitsithrow,andW=diag{Kh(U1−u0),...,Kh(Un−u0)}.8.6.3BandwidthSelectionVariousexistingbandwidthselectiontechniquesfornonparametricregressioncanbeadaptedfortheforegoingestimation;see,e.g.,Fan,Yao,andCai(2003)andthenonparametricAICasdiscussedinSection8.5.5.Also,FanandGijbels(1996)andRuppert,Sheather,andWand(1995)developeddata-drivenbandwidthselectionschemesbasedonasymptoticformulasfortheoptimalbandwidths,whicharelessvariableandmoreeffectivethantheconventionaldata-drivenbandwidthselectorssuchasthecross-validationbandwidthrule. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION295Similaralgorithmscanbedevelopedfortheestimationoffunctional-coefficientmodelsbasedon(8.24);however,thiswillbeafutureresearchtopic.Cai,FanandYao(2000)proposedasimpleandquickmethodforselectingbandwidthh.Itcanberegardedasamodifiedmulti-foldcross-validationcriterionthatisattentivetothestructureofstation-arytimeseriesdata.LetmandQbetwogivenpositiveintegersandn>mQ.ThebasicideaisfirsttouseQsubseriesoflengthsn−qm(q=1,,···,Q)toestimatetheunknowncoefficientfunc-tionsandthencomputetheone-stepforecastingerrorsofthenextsectionofthetimeseriesoflengthmbasedontheestimatedmod-els.Moreprecisely,wechoosehthatminimizestheaveragemeansquared(AMS)errorQXAMS(h)=AMSq(h),(8.15)q=1whereforq=1,···,Q,21n−qm+mpXXAMSq(h)=Yi−abj,q(Ui)Xi,j,mi=n−qm+1j=1and{abj,q(·)}arecomputedfromthesample{(Ui,Xi,Yi),1≤i≤n−qm}withbandwidthequalh[n/(n−qm)]1/5.Notethatwere-scalebandwidthhfordifferentsamplesizesaccordingtoitsoptimalrate,i.e.h∝n−1/5.Inpracticalimplementations,wemayusem=[0.1n]andQ=4.TheselectedbandwidthdoesnotdependcriticallyonthechoiceofmandQ,aslongasmQisreasonablylargesothattheevaluationofpredictionerrorsisstable.AweightedversionofAMS(h)canbeused,ifonewishestodown-weightthepredictionerrorsatanearliertime.Webelievethatthisbandwidthshouldbegoodformodelingandforecastingfortimeseries. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION2968.6.4SmoothingVariableSelectionOfimportanceistochooseanappropriatesmoothingvariableUinapplyingfunctional-coefficientregressionmodelsifUisalaggedvariable.Knowledgeonphysicalbackgroundofthedatamaybeveryhelpful,asCai,FanandYao(2000)discussedinmodelingthelynxdata.Withoutanypriorinformation,itispertinenttochooseUintermsofsomedata-drivenmethodssuchastheAkaikeinformationcriterion(AIC)anditsvariants,cross-validation,andothercriteria.Ideally,wewouldchooseUasalinearfunctionofgivenexplanatoryvariablesaccordingtosomeoptimalcriterion,whichcanbefullyexploredintheworkbyFan,YaoandCai(2003).Nevertheless,weproposehereasimpleandpracticalapproach:letUbeoneofthegivenexplanatoryvariablessuchthatAMSdefinedin(8.15)obtainsitsminimumvalue.Obviously,thisideacanbealsoextendedtoselectp(numberoflags)aswell.8.6.5Goodness-of-FitTestTotestwhethermodel(8.10)holdswithaspecifiedparametricformwhichispopularineconomicandfinancialapplications,suchasthethresholdautoregressive(TAR)modelsaj1,ifu≤ηaj(u)=aj2,ifu>η,orgeneralizedexponentialautoregressive(EXPAR)models2aj(u)=αj+(βj+γju)exp(−θju),orsmoothtransitionautoregressive(STAR)models−1aj(u)=[1−exp(−θju)](logistic),or CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION2972aj(u)=1−exp(−θju)(exponential),or−1aj(u)=[1−exp(−θj|u|)](absolute),[formorediscussionsonthosemodels,pleaseseethesurveypapersbyvanDijk,Ter¨asvirtaandFranses(2002)],weproposeagoodness-of-fittestbasedonthecomparisonoftheresidualsumofsquares(RSS)frombothparametricandnonparametricfittings.ThismethodiscloselyrelatedtothesievelikelihoodmethodproposedbyFan,ZhangandZhang(2001).Thoseauthorsdemonstratedtheoptimalityofthiskindofproceduresforindependentsamples.ConsiderthenullhypothesisH0:aj(u)=αj(u,θ),1≤j≤p,(8.16)whereαj(·,θ)isagivenfamilyoffunctionsindexedbyunknowncparametervectorθ.Letθbeanestimatorofθ.TheRSSunderthenullhypothesisisXn2−1ccRSS0=nYi−α1(Ui,θ)Xi1−···−αp(Ui,θ)Xip.i=1Analogously,theRSScorrespondingtomodel(8.10)isXn−12RSS1=n{Yi−ab1(Ui)Xi1−···−abp(Ui)Xip}.i=1TheteststatisticisdefinedasTn=(RSS0−RSS1)/RSS1=RSS0/RSS1−1,andwerejectthenullhypothesis(8.16)forlargevalueofTn.Weusethefollowingnonparametricbootstrapapproachtoevaluatethepvalueofthetest: CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION2981.Generatethebootstrapresiduals{ε∗}nfromtheempiricaldis-ii=1tributionofthecenteredresiduals{εb−¯εb}n,whereii=11Xnεbi=Yi−ab1(Ui)Xi1−···−abp(Ui)Xip,¯εb=εbi,ni=1anddefine∗cc∗Yi=α1(Ui,θ)Xi1+···+αp(Ui,θ)Xip+εi.2.CalculatethebootstrapteststatisticT∗basedonthesamplen∗n{Ui,Xi,Yi}i=1.3.RejectthenullhypothesisH0whenTnisgreaterthantheupper-∗nαpointoftheconditionaldistributionofTngiven{Ui,Xi,Yi}i=1.Thep-valueofthetestissimplytherelativefrequencyoftheevent{T∗≥T}inthereplicationsofthebootstrapsampling.Forthennsakeofsimplicity,weusethesamebandwidthincalculatingT∗asnthatinTn.Notethatwebootstrapthecentralizedresidualsfromthenonparametricfitinsteadoftheparametricfit,becausethenonpara-metricestimateofresidualsisalwaysconsistent,nomatterwhetherthenullorthealternativehypothesisiscorrect.Themethodshouldprovideaconsistentestimatorofthenulldistributionevenwhenthenullhypothesisdoesnothold.Kreiss,Neumann,andYao(1998)considerednonparametricbootstraptestsinageneralnonparametricregressionsetting.Theyprovedthat,asymptotically,theconditionaldistributionofthebootstrapteststatisticisindeedthedistributionoftheteststatisticunderthenullhypothesis.Itmaybeproventhatcthesimilarresultholdshereaslongasθconvergestoθattheraten−1/2.Itisagreatchallengetoderivetheasymptoticpropertyofthetest-ingstatisticsTnundertimeseriescontextandgeneralassumptions. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION299Thatistoshowthat2bn[Tn−λn]→N(0,σ)forsomebnandλn,whichisagreatprojectforfutureresearch.NotethatFan,ZhangandZhang(2001)derivedtheaboveresultfortheiidsample.8.6.6AsymptoticResultsWefirstpresentaresultonmeansquaredconvergencethatservesasabuildingblockforourmainresultandisalsoofindependentinterest.Wenowintroducesomenotation.LetSn,0Sn,1Sn=Sn(u0)=Sn,1Sn,2andTn,0(u0)Tn=Tn(u0)=Tn,1(u0)withj1XnTUi−u0Sn,j=Sn,j(u0)=XiXiKh(Ui−u0)ni=1handj1XnUi−u0Tn,j(u0)=XiKh(Ui−u0)Yi.(8.17)ni=1hThen,thesolutionto(8.12)canbeexpressedasc−1−1β=HSnTn,(8.18)whereH=diag(1,...,1,h,...,h)withp-diagonalelements1’sandpdiagonalelementsh’s.Tofacilitatethenotation,wedenoteTΩ=(ωl,m)p×p=EXX|U=u0.(8.19) CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION300Also,letf(u,x)denotethejointdensityof(U,X)andfu(u)bethemarginaldensityofU.Weusethefollowingconvention:ifU=Xj0forsome1≤j0≤p,thenf(u,x)becomesf(x)thejointdensityofX.Theorem1.LetconditionA.1inhold,andletf(u,x)becon-tinuousatthepointu0.Lethn→0andnhn→∞,asn→∞.ThenitholdsthatE(Sn,j(u0))→fu(u0)Ω(u0)µj,andnhnVar(Sn,j(u0)l,m)→fu(u0)ν2jωl,mforeach0≤j≤3and1≤l,m≤p.AsaconsequenceofTheorem1,wehavePPSn−→fu(u0)S,andSn,3−→µ3fu(u0)Ωinthesensethateachelementconvergesinprobability,whereΩµ1ΩS=.µ1Ωµ2ΩPut2σ(u,x)=Var(Y|U=u,X=x)(8.20)and∗T2Ω(u0)=EXXσ(U,X)|U=u0.(8.21)Letc=µ/µ−µ2andc=−µ/µ−µ2.02211121Theorem2.Letσ2(u,x)andf(u,x)becontinuousatthepointu0.ThenunderconditionsA.1andA.2,√h2µ2−µµ213′′D2nhnab(u0)−a(u0)−2a(u0)−→N0,Θ(u0),2µ2−µ1(8.22) CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION301providedthatfu(u0)6=0,wherec2ν+2ccν+c2νΘ2(u)=0001112Ω−1(u)Ω∗(u)Ω−1(u).0000fu(u0)(8.23)Theorem2indicatesthattheasymptoticbiasofabj(u0)ish2µ2−µµ213′′2aj(u0)2µ2−µ1andtheasymptoticvarianceis(nh)−1θ2(u),wherenj0c2ν+2ccν+c2νθ2(u)=0001112eTΩ−1(u)Ω∗(u)Ω−1(u)e.j0j,p000j,pfu(u0)Whenµ1=0,thebiasandvarianceexpressionscanbesimplifiedash2µa′′(u)/2and2j02ν0T−1∗−1θj(u0)=ej,pΩ(u0)Ω(u0)Ω(u0)ej,p.fu(u0)Theoptimalbandwidthforestimatingaj(·)canbedefinedtobetheonethatminimizesthesquaredbiasplusvariance.Theoptimalbandwidthisgivenby1/522T−1∗−1µ2ν0−2µ1µ2ν1+µ1ν2ej,pΩ(u0)Ω(u0)Ω(u0)ej,p−1/hj,opt=22n2′′fu(u0)(µ2−µ1µ3)aj(u0)(8.24)8.6.7ConditionsandProofsWefirstimposesomeconditionsontheregressionmodelbuttheymightnotbetheweakestpossible. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION302ConditionA.1a.ThekernelfunctionK(·)isaboundeddensitywithaboundedsupport[−1,1].b.|f(u,v|x0,x1;l)|≤M<∞,foralll≥1,wheref(u,v,|x0,x1;l)istheconditionaldensityof(U0,Ul))given(X0,Xl),andf(u|x)≤M<∞,wheref(u|x)istheconditionaldensityofUgivenX=x.c.Theprocess{U,X,Y}isα-mixingwithPkc[α(k)]1−2/δ<∞iiiforsomeδ>2andc>1−2/δ.d.E|X|2δ<∞,whereδisgiveninconditionA.1c.ConditionA.2a.Assumethat22EY0+Yl|U0=u,X0=x0;Ul=v,Xl=x1≤M<∞,(8.25)foralll≥1,x,x∈ℜp,u,andvinaneighborhoodofu.010b.Assumethathn→andnhn→∞.Further,assumethatthereexistsasequenceofpositiveintegerssnsuchthatsn→∞,s=o(nh)1/2,and(n/h)1/2α(s)→0,asn→∞.nnnnc.Thereexistsδ∗>δ,whereδisgiveninConditionA.1c,suchthatδ∗E|Y||U=u,X=x≤M4<∞(8.26)forallx∈ℜpanduinaneighborhoodofu,and0−θ∗α(n)=On,(8.27)whereθ∗≥δδ∗/{2(δ∗−δ)}. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION303d.E|X|2δ∗<∞,andn1/2−δ/4hδ/δ∗−1/2−δ/4=O(1).RemarkA.1.Weprovideasufficientconditionforthemixingcoefficientα(n)tosatisfyconditionsA.1candA.2b.Supposethath=An−ρ(0<ρ<1,A>0),s=(nh/logn)1/2andnnnα(n)=On−dforsomed>0.ThenconditionA.1cissatisfiedford>2(1−1/δ)/(1−2/δ)andconditionA.2bissatisfiedifd>(1+ρ)/(1−ρ).Hencebothconditionsaresatisfiedif1+ρ2(1−1/δ)−dα(n)=On,d>max,.1−ρ1−2/δNotethatthisisatrade-offbetweentheorderδofthemomentofYandtherateofdecayofthemixingcoefficient;thelargertheorderδ,theweakerthedecayrateofα(n).Tostudythejointasymptoticnormalityofab(u0),weneedtocen-terthevectorTn(u0)byreplacingYiwithYi−m(Ui,Xi)intheexpression(8.17)ofTn,j(u0).Letj∗1XnUi−u0Tn,j(u0)=XiKh(Ui−u0)[Yi−m(Ui,Xi)],ni=1hand∗T∗=Tn,0.n∗Tn,1Becausethecoefficientfunctionsaj(u)areconductedintheneigh-borhoodof|Ui−u0|0.(8.44)impliesthatQn,2andQn,3areasymptoticallynegligibleinprobability,(8.45)showsthatthesummandsηjinQn,1areasymptoticallyindependentand(8.46)and(8.47)arethestan-dardLindeberg-FellerconditionsforasymptoticnormalityofQn,1fortheindependentsetup. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION309Wefirstestablish(8.44).Forthispurpose,wechoosethelargeblocksize.ConditionA.2bimpliesthatthereisasequenceofpositiveconstantsγn→∞suchthat√γnsn=onhnand1/2γn(n/hn)α(sn)→0.(8.48)Definethelargeblocksizerbyr=⌊(nh)1/2/γ⌋andthesmallnnnnblocksizesn.Thenitcaneasilybeshownfrom(8.48)thatasn→∞,−1/2sn/rn→0,rn/n→0,rn(nhn)→0,(8.49)and(n/rn)α(sn)→0.(8.50)Observethatq−12XXE[Qn,2]=Var(ξj)+2Cov(ξi,ξj)≡I1+I2.(8.51)j=00≤ii,wethushavejnnjinsnsnXXX|I2|≤2|Cov(Zn,r∗+rn+j1,Zn,r∗+rn+j2)|ij0≤i0andh12=h12(n)>0beband-widthsinthestepofestimatingtheregressionsurface.Here,tohandlevariousdegreesofsmoothness,CaiandFan(2000)proposeusingh11andh12differentlyalthoughtheimplementationmaynotbeeasyinpractice.ThereaderisreferredtothepaperbyCaiandncFan(2000)fordetails.Givenobservations{Xt,Yt,Zt}t=1,letβjbetheminimizerofthefollowinglocallyweightedleastsquaresXn2TTZt−β0−β1(Xt−x)−β2(Yt−y)Kh11(Xt−x)Lh12(Yt−y),t=1whereK(·)=K(·/h)/hpandL(·)=L(·/h)/hq.Then,thelocalhhclinearestimatoroftheregressionsurfacem(x,y)ismd(x,y)=β0. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION318Bycomputingthesampleaverageofmd(·,·)basedon(8.62),theprojectionestimatorsofg1(·)andg2(·)aredefinedas,respectively,1Xngb1(x)=md(x,Yt)−µ,cnt=1and1Xngb2(y)=md(Xt,y)−µ,cnt=1whereµc=n−1PnZ.Undersomeregularityconditions,byusingt=1tthesameargumentsasthoseemployedintheproofofTheorem3inCaiandMasry(2000),itcanbeshown(althoughnoteasyandtedious)thattheasymptoticbiasandasymptoticvarianceofgb1(x)are,respectively,h2tr{µ(K)g′′(x)}/2andv(x)=ν(K)A(x),112110whereZ22−1A(x)=p2(y)σ(x,y)p(x,y)dyand2σ(x,y)=Var(Zt|Xt=x,Yt=y).Here,p(x,y)standsforthejointdensityofXtandYt,p1(x)denotesthemarginaldensityofXt,p2(y)isthemarginaldensityofYt,R2RTν0(K)=K(u)du,andµ2(K)=uuK(u)du.Theforegoingmethodhassomeadvantages,suchasitiseasytounderstand,itcanmakecomputationfast,anditallowsanasymp-toticanalysis.However,itcanbequiteinefficientinanasymptoticsense.Todemonstratethisidea,letusconsidertheidealsituationthatg2(·)andµareknown.Insuchacase,onecanestimateg1(·)byfdirectlyregressingthepartialerrorZt=Zt−µ−g2(Yt)onXtandsuchanidealestimatorisoptimalinanasymptoticminimaxsense(see,e.g.,FanandGijbels,1996).Theasymptoticbiasfortheideal CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION319estimatorish2tr{µ(K)g′′(x)}/2andtheasymptoticvarianceis1121−12v0(x)=ν0(K)B(x)withB(x)=p1(x)Eσ(Xt,Yt)|Xt=x(8.64)(see,e.g.,MasryandFan,1997).Itisclearthatv1(x)=v0(x)ifXtandYtareindependent.IfXtandYtarecorrelatedandwhenσ2(x,y)isaconstant,itfollowsfromtheCauchy-Schwarzinequalitythatσ2Zp(y)σ2Zp2(y)B(x)=p1/2(y|x)2dy≤2dy=A(x),p1(x)p1/2(y|x)p1(x)p(y|x)whichimpliesthattheidealestimatorhasalwayssmallerasymptoticvariancethantheprojectionmethodalthoughbothhavethesamebias.Thissuggeststhattheprojectionmethodcouldleadtoanin-efficientestimationofg1(·)andg2(·)whenXtandYtareseriallycorrelated,whichisparticularlyrelevantforautoregressivemodels.Toalleviatethisshortcoming,Iproposethetwo-stageapproachde-scribednext.8.7.4Two-StageProcedureThetwo-stagemethodduetoLinton(1997,2000)isintroduced.Thebasicideaistogetaninitialestimateforgb2(·)usingasmallband-widthh12.Theinitialestimatecanbeobtainedbytheprojectionmethodandh12canbechosensosmallthatthebiasofestimat-inggb2(·)canbeasymptoticallynegligible.Then,usingthepartialresidualsZ∗=Z−µc−gb(Y),weapplythelocallinearregressiontt2ttechniquetothepseudoregressionmodel∗∗Zt=g1(Xt)+εt CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION320toestimateg1(·).Thisleadsnaturallytotheweightedleast-squaresproblemXn2∗TZt−β1−β2(Xt−x)Jh2(Xt−x),(8.65)t=1whereJ(·)isthekernelfunctioninℜpandh=h(n)>0isthe22bandwidthatthesecond-stage.Theadvantageofthisistwofold:thebandwidthh2cannowbeselectedpurposelyforestimatingg1(·)onlyandanybandwidthselectiontechniquefornonparametricregressioncanbeappliedhere.Maximizing(8.65)withrespecttoβ1andβ2cgivesthetwo-stageestimateofg1(x),denotedbyge1(x)=β1,whereccβ1andβ2aretheminimizerof(8.65).ItisshowninTheorem1,inwhichfollows,thatundersomeregu-larityconditions,theasymptoticbiasandvarianceofthetwo-stageestimatege1(x)arethesameasthosefortheidealestimator,providedthattheinitialbandwidthh12satisfiesh12=o(h2).SamplingPropertiesToestablishtheasymptoticnormalityofthetwo-stageestimator,itisassumedthattheinitialestimatorsatisfiesalinearapproximation;namely,1Xngb2(Yt)−g2(Yt)≈Lh12(Yi−Yt)Γ(Xi,Yt)δini=112′′+h12tr{µ2(L)g2(Yt)},(8.66)2whereδt=Zt−m(Xt,Yt)andΓ(x,y)=p1(x)/p(x,y).Notethatundersomeregularityconditions,byfollowingthesameargumentsasinMasry(1996),onemightshow(althoughtheproofisnoteasy,quitelengthy,andtedious)that(8.66)holds.Notethatthisassump-tionisalsoimposedinLinton(2000)foriidsamplestosimplifythe CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION321proofoftheasymptoticresultsofthetwo-stageestimator.Now,theasymptoticnormalityforthetwo-stageestimatorisstatedhereanditsproofisrelegatedtotheAppendix.THEOREM1.Under(8.66)andAssumptionsA1–A9statedintheAppendix,ifbandwidthsh12andh2arechosensuchthatqph12→0,nh12→∞,h2→0,andnh2→∞asn→∞,then,rp22Dnh2ge1(x)−g1(x)−bias(x)+oph12+h2−→N{0,v0(x)},wheretheasymptoticbiasish2h22′′12′′bias(x)=tr{µ2(J)g1(x)}−tr{µ2(L)E(g2(Yt)|Xt=x)}22andtheasymptoticvarianceisv0(x)=ν0(J)B(x).WeremarkthatbyTheorem1,theasymptoticvarianceofthetwo-stageestimatorisindependentoftheinitialbandwidths.Thus,theinitialbandwidthsshouldbechosenassmallaspossible.Thisisanotherbenefitofusingthetwo-stageprocedure:thebandwidthselectionproblembecomesrelativelyeasy.Inparticular,whenh12=o(h2),thebiasfromtheinitialestimationcanbeasymptoticallynegligible.Fortheidealsituationthatg2(·)isknown,MasryandFan(1997)showthatundersomeregularityconditions,theoptimalestimateofg(x),denotedbygb∗(x),byusing(8.65)inwhichthe11∗fpartialresidualZtisreplacedbythepartialerrorZt=Yt−µ−g2(Yt),isasymptoticallynormallydistributed,rh2p∗2′′2Dnh2gb1(x)−g1(x)−tr{µ2(J)g1(x)}+op(h2)−→N{0,v0(x)}.2This,inconjunctionwithTheorem1,showsthatthetwo-stagees-timatorandtheidealestimatorsharethesameasymptoticbiasandvarianceifh12=o(h2). CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION322Finally,itisworthtopointingoutthatundersomeregularityconditions,thenonlinearadditiveARXprocessesarestationaryandα-mixingwithgeometricdecaymixingcoefficient,seeMasryandTjøstheim(1997),sothatAssumptionsA6,A7,andA8intheAp-pendiximposedonthemixingcoefficientareautomaticallysatis-fied.Therefore,assumingthattheothertechnicalassumptionsofthispaperaresatisfied,theresultinTheorem1canbeappliedtothenonlinearadditiveARXmodels.8.7.5MonteCarloSimulationsandApplicationsSeeCai(2002)forthedetailedMonteCarlosimulationresultsandapplications.8.8ComputerCode#07-31-2006graphics.off()#cleanthepreviousgraphsonthescreen###################################################################z1=matrix(scan(file="c:\teaching\timeseries\data\w-3mtbs7097.txt"),byrow=T,ncol=4)#dada:weekly3-monthTreasurybillfrom1970to1997x=z1[,4]/100n=length(x)y=diff(x)#Deltax_t=x_t-x_{t-1}x=x[1:(n-1)]n=n-1x_star=(x-mean(x))/sqrt(var(x))z=seq(min(x),max(x),length=50)#win.graph() CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION323postscript(file="c:\teaching\timeseries\figs\fig-8.1.eps",horizontal=F,width=6,height=6)par(mfrow=c(2,2),mex=0.4,bg="lightblue")scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)",evaluation=60title(main="(a)y(t)vsx(t-1)",col.main="red")scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)",evaluatitle(main="(b)|y(t)|vsx(t-1)",col.main="red")scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)",evaluation=title(main="(c)y(t)^2vsx(t-1)",col.main="red")dev.off()#############################################################################################NonparametricFitting####################################################################################DefinetheEpanechnikovkernelfunctionkernel<-function(x){0.75*(1-x^2)*(abs(x)<=1)}################################################################Definethekerneldensityestimatorkernden=function(x,z,h,ker){#parameters:x=variable;h=bandwidth;z=gridpoint;ker=kernelnz<-length(z)nx<-length(x)x0=rep(1,nx*nz)dim(x0)=c(nx,nz)x1=t(x0)x0=x*x0x1=z*x1x0=x0-t(x1)if(ker==1){x1=kernel(x0/h)}#Epanechnikovkernel CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION324if(ker==0){x1=dnorm(x0/h)}#normalkernelf1=apply(x1,2,mean)/hreturn(f1)}################################################################Definethelocalconstantestimatorlocal.constant=function(y,x,z,h,ker){#parameters:x=variable;h=bandwidth;z=gridpoint;ker=kernelnz<-length(z)nx<-length(x)x0=rep(1,nx*nz)dim(x0)=c(nx,nz)x1=t(x0)x0=x*x0x1=z*x1x0=x0-t(x1)if(ker==1){x1=kernel(x0/h)}#Epanechnikovkernelif(ker==0){x1=dnorm(x0/h)}#normalkernelx2=y*x1f1=apply(x1,2,mean)f2=apply(x2,2,mean)f3=f2/f1return(f3)}####################################################################Definethelocallinearestimatorlocal.linear<-function(y,x,z,h){#parameters:y=response,x=designmatrix;h=bandwidth;z=gridnz<-length(z)ny<-length(y) CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION325beta<-rep(0,nz*2)dim(beta)<-c(nz,2)for(kin1:nz){x0=x-z[k]w0<-kernel(x0/h)beta[k,]<-glm(y~x0,weight=w0)$coeff}return(beta)}###################################################################h=0.02#Localconstantestimatemu_hat=local.constant(y,x,z,h,1)sigma_hat=local.constant(abs(y),x,z,h,1)sigma2_hat=local.constant(y^2,x,z,h,1)win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightyellow")scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)")points(z,mu_hat,type="l",lty=1,lwd=3,col=2)title(main="(a)y(t)vsx(t-1)",col.main="red")legend(0.04,0.0175,"LocalConstantEstimate")scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)")points(z,sigma_hat,type="l",lty=1,lwd=3,col=2)title(main="(b)|y(t)|vsx(t-1)",col.main="red")scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)")title(main="(c)y(t)^2vsx(t-1)",col.main="red")points(z,sigma2_hat,type="l",lty=1,lwd=3,col=2)#LocalLinearEstimatefit2=local.linear(y,x,z,h) CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION326mu_hat=fit2[,1]fit2=local.linear(abs(y),x,z,h)sigma_hat=fit2[,1]fit2=local.linear(y^2,x,z,h)sigma2_hat=fit2[,1]win.graph()par(mfrow=c(2,2),mex=0.4,bg="lightgreen")scatter.smooth(x,y,span=1/10,ylab="",xlab="x(t-1)")points(z,mu_hat,type="l",lty=1,lwd=3,col=2)title(main="(a)y(t)vsx(t-1)",col.main="red")legend(0.04,0.0175,"LocalLinearEstimate")scatter.smooth(x,abs(y),span=1/10,ylab="",xlab="x(t-1)")points(z,sigma_hat,type="l",lty=1,lwd=3,col=2)title(main="(b)|y(t)|vsx(t-1)",col.main="red")scatter.smooth(x,y^2,span=1/10,ylab="",xlab="x(t-1)")title(main="(c)y(t)^2vsx(t-1)",col.main="red")points(z,sigma2_hat,type="l",lty=1,lwd=3,col=2)###################################################################8.9ReferencesBowman,A.(1984).Analternativemethodofcross-validationforthesmoothingofdensityestimate.Biometrika,71,353-360.Cai,Z.(2002).Atwo-stageapproachtoadditivetimeseriesmodels.StatisticaNeerlandica,56,415-433.CAI,Z.andJ.FAN(2000).Averageregressionsurfacefordependentdata.JournalofMultivariateAnalysis,75,112-142.Cai,Z.,J.FanandQ.Yao(2000).Functional-coefficientregressionmodelsfornonlineartimeseries.JournalofAmericanStatisticalAssociation,95,941-956.CAI,Z.andE.MASRY(2000).NonparametricestimationofadditivenonlinearARXtimeseries:Locallinearfittingandprojection.EconometricTheory,16,465-501. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION327Cai,Z.andR.C.Tiwari(2000).ApplicationofalocallinearautoregressivemodeltoBODtimeseries.Environmetrics,11,341-350.CHEN,R.andR.TSAY(1993).NonlinearadditiveARXmodels.JournaloftheAmericanStatisticalAssociation,88,310-320.Chiu,S.T.(1991).Bandwidthselectionforkerneldensityestimation.TheAnnalsofStatistics,19,1883-1905.Engle,R.F.,C.W.J.Grabger,J.Rice,andA.Weiss(1986).Semiparametricestimatesoftherelationbetweenweatherandelectricitysales.JournalofTheAmericanStatisticalAssociation,81,310-320.Fan,J.(1993).Locallinearregressionsmoothersandtheirminimaxefficiency.TheAnnalsofStatistics,21,196-216.Fan,J.,N.E.Heckman,andM.P.Wand(1995).Localpolynomialkernelregressionforgeneralizedlinearmodelsandquasi-likelihoodfunctions.JournaloftheAmericanStatisticalAssociation,90,141-150.Fan,J.,T.Gasser,I.Gijbels,M.BrockmannandJ.Engel(1996).Localpolynomialfitting:optimalkernelandasymptoticminimaxefficiency.AnnalsoftheInstituteofStatisticalMathematics,49,79-99.Fan,J.andI.Gijbels(1996).LocalPolynomialModelingandItsApplications.London:ChapmanandHall.Fan,J.andQ.Yao(2003).NonlinearTimeSeries:NonparametricandParametricMeth-ods.NewYork:Springer-Verlag.Fan,J.,Q.YaoandZ.Cai(2003).Adaptivevarying-coefficientlinearmodels.JournaloftheRoyalStatisticalSociety,SeriesB,65,57-80.Fan,J.andC.Zhang(2003).Are-examinationofdiffusionestimatorswithapplicationstofinancialmodelvalidation.JournaloftheAmericanStatisticalAssociation,98,118-134.Fan,J.,C.ZhangandJ.Zhang(2001).GeneralizedlikelihoodteststatisticandWilksphenomenon.TheAnnalsofStatistics,29,153-193.Gasser,T.andH.-G.M¨uller(1979).Kernelestimationofregressionfunctions.InSmoothingTechniquesforCurveEstimation,LectureNotesinMathematics,757,2368.Springer-Verlag,NewYork.Granger,C.W.J.,andT.Ter¨asvirta(1993).ModelingNonlinearEconomicRelationships.Oxford,U.K.:OxfordUniversityPress.Hall,P.,andC.C.Heyde(1980).MartingaleLimitTheoryandItsApplications.NewYork:AcademicPress. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION328Hall,P.,andI.Johnstone(1992).Empiricalfunctionalandefficientsmoothingparameterselection(withdiscussion).JournaloftheRoyalStatisticalSociety,SeriesB,54,475-530.HASTIE,T.J.andR.J.TIBSHIRANI(1990).GeneralizedAdditiveModels.ChapmanandHall,London.Hjort,N.L.andM.C.Jones(1996).Betterrulesofthumbforchoosingbandwidthindensityestimation.Workingpaper,DepartmentofMathematics,UniversityofOslo,Norway.Hong,Y.andLee,T.-H.(2003).Inferenceonviageneralizedspectrumandnonlineartimeseriesmodels.TheReviewofEconomicsandStatistics,85,1048-1062.Hurvich,C.M.,J.S.SimonoffandC.-L.Tsai(1998).SmoothingparameterselectioninnonparametricregressionusinganimprovedAkaikeinformationcriterion.JournaloftheRoyalStatistical,SocietyB,60,271-293.Jones,M.C.,J.S.MarronandS.J.Sheather(1996).Abriefsurveyofbandwidthselectionfordensityestimation.JournalofAmericanStatisticalAssociation,91,401-407.Kreiss,J.P.,M.NeumannandQ.Yao(1998).Bootstraptestsforsimplestructuresinnonparametrictimeseriesregression.Unpublishedmanuscript.LINTON,O.B.(1997).Efficientestimationofadditivenonparametricregressionmodels.Biometrika,84,469-473.LINTON,O.B.(2000).Efficientestimationofgeneralizedadditivenonparametricregressionmodels.EconometricTheory,16,502-523.LINTON,O.B.andJ.P.NIELSEN(1995).Akernelmethodofestimatingstructurednonparametricregressionbasedonmarginalintegration.Biometrika,82,93-100.MAMMEN,E.,O.B.LINTON,andJ.P.NIELSEN(1999).Theexistenceandasymptoticpropertiesofabackfittingprojectionalgorithmunderweakconditions.TheAnnalsofStatistics,27,1443-1490.MASRY,E.andJ.FAN(1997).Localpolynomialestimationofregressionfunctionsformixingprocesses.ScandinavianJournalofStatistics,24,165-179.MASRY,E.andD.TJØSTHEIM(1997).AdditivenonlinearARXtimeseriesandprojec-tionestimates.EconometricTheory,13,214-252.Øksendal,B.(1985).StochasticDifferentialEquations:AnIntroductionwithApplications,3thedition.NewYork:Springer-Verlag.Priestley,M.B.andM.T.Chao(1972).Nonparametricfunctionfitting.JournaloftheRoyalStatisticalSociety,SeriesB,34,384-392.Rice,J.(1984).Bandwidthselectionfornonparametricregression.TheAnnalsofStatistics,12,1215-1230. CHAPTER8.NONPARAMETRICREGRESSIONESTIMATION329Rudemo,M.(1982).Empiricalchoiceofhistogramsandkerneldensityestimators.Scan-dinaviaJournalofStatistics,9,65-78.Ruppert,D.,S.J.SheatherandM.P.Wand(1995).Aneffectivebandwidthselectorforlocalleastsquaresregression.JournalofAmericanStatisticalAssociation,90,1257-1270.Ruppert,D.andM.P.Wand(1994).Multivariateweightedleastsquaresregression.TheAnnalsofStatistics,22,1346-1370.Rousseeuw,R.J.andA.M.Leroy(1987).RobustRegressionandOutlierDetection.NewYork:Wiley.Shao,Q.andH.Yu(1996).Weakconvergenceforweightedempiricalprocessesofdependentsequences.TheAnnalsofProbability,24,2098-2127.Sheather,S.J.andM.C.Jones(1991).Areliabledata-basedbandwidthselectionmethodforkerneldensityestimation.JournaloftheRoyalStatisticalSociety,SeriesB,53,683-690.SPERLISH,S.,D.TJØSTHEIM,andL.YANG(2000).Nonparametricestimationandtestingofinteractioninadditivemodels.EconometricTheory,Stanton,R.(1997).Anonparametricmodeloftermstructuredynamicsandthemarketpriceofinterestraterisk.JournalofFinance,52,1973-2002.Sun,Z.(1984).Asymptoticunbiasedandstrongconsistencyfordensityfunctionestimator.ActaMathematicaSinica,27,769-782.TJØSTHEIM,D.andB.AUESTAD(1994a).Nonparametricidentificationofnonlineartimeseries:Projections.JournaloftheAmericanStatisticalAssociation,89,1398-1409.Tjøstheim,D.andB.Auestad(1994b).Nonparametricidentificationofnonlineartimeseries:Selectingsignificantlags.JournaloftheAmericanStatisticalAssociation,89,1410-1419.vanDijk,D.,T.Ter¨asvirta,andP.H.Franses(2002).Smoothtransitionautoregressivemodels-asurveyofrecentdevelopments.EconometricReviews,21,1-47.Wand,M.P.andM.C.Jones(1995).KernelSmoothing.London:ChapmanandHall.