Multiple_Random_Variables

《Multiple_Random_Variables》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

NOTESONMULTIPLERANDOMVARIABLES1.1DISTRIBUTIONANDDENSITYFUNCTIONSTheconsiderationofmorethantworandomvariablesisbyandlargejusttheoftennaturalextensionsofthenotionsandresultsassociatedwiththeconsiderationoftworandomvariables.Thesefewpagesbrieflyrestatethemainpointsinthegeneralcase.GivenafinitecollectionofnrandomvariablesX1,X2,...,Xn,theirjointstatisticalpropertiesarefullydeterminedbyspecifyingtheirjointdistributionfunctiondefinedbyFX1X2···Xn(x1,x2,...,xn)=P(X1x1,X2x2,...,Xnxn).(1.1)Thefivepropertiesthatweregivenearlierforthejointdistributionfunctionoftworandomvariablesapply(intheanalogousformsfornrandomvariables)tothegeneralsituation:Property1.1.FX1X2···Xn(1,1,...,1),limFX1X2···Xn(x1,x2,...,xn)=1.(1.2)xi!1alliProperty1.2.FX1X2···Xn(x1,x2,...,xm1,1,xm+1,xm+2,...,xn),limFX1X2···Xn(x1,x2,...,xn)=0xm!1foranyrealnumbersx1,x2,...,xm1,xm+1,xm+2,...,xn,wheremisanyintegerfrom1ton.Property1.3.Foranyrealnumbersx1,x2,x3,...,xnandy1,y2,y3,...,ynsuchthatxiyiforalli2{1,2,...,n},FX1X2···Xn(x1,x2,...,xn)FX1X2···Xn(y1,y2,...,yn).(1.3)Property1.4.Foranyrealnumbersx1,x2,x3,...,xnandy1,y2,y3,...,ynsuchthatxiyiforalli2{1,2,...,n},P(x10)hastheobviousdefinitionF˜(x|M),P(X˜x|M).(1.17)XIfY˜issomeotherrandomvector,andM={Y˜y}fory=[y1,y2,...,ym],thenF˜(x;y)FXY˜˜(x|Y˜y),,(1.18)XF˜(y)Ywiththecorrespondingdensityfunctiongivenby@nf˜(x|Y˜y)=F˜(x;y)/F˜(y)X@x@x···@xXY˜Y12nRy1yRm···fX˜Y˜(x1,x2,...,xn;1,2,...,m)dmdm1...d111=(1..19)R1R1⇥Ry1yRm⇤······fX˜Y˜(x1,x2,...,xn;1,2,...,m)dmdm1...d1dx1dx2...dxn1111WeextendthisnotiontozeroprobabilityeventsoftheformY˜=yanalogoustothewayitwasextendedearlierforarandomvariableXconditionedonY=y:f˜(x;y)fXY˜˜(x|Y˜=y),.(1.20)Xf˜(y)YZxnZx1FX˜(x|Y˜=y),···fX˜(1,2,...,n|Y˜=y)d1d2...dn,(1.21)11OneparticularresultonconditionaldensityfunctionsthatwefindusefulisfX˜(x1,x2,...,xn)=fX1(x1|X2=x2,...,Xn=xn)·fX2(x2|X3=x3,...,Xn=xn)····fXn1(xn1|Xn=xn)·fXn(xn).(1.22)1.2FUNCTIONSOFRANDOMVECTORSLetZ˜=g(X˜),i.e.,Z1=g1(X1,X2,...,Xn),Z2=g2(X1,X2,...,Xn),...,Zm=gm(X1,X2,...,Xn).ThenZ˜willbearandomvectorifeachofthefunctionsg(X˜)areBorelfunctionsonRn.Forthecaseofin=mandg(·)acontinuousanddi↵erentiablefunction,wefindthatfX˜(x1)fX˜(x2)fX˜(xl)f˜(z)=++···+,(1.23)Z|J(x)||J(x)||J(x)|12lwherex1,x2,...,xlarethesolutionstothevectorequationz=g(x),(1.24)3 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013andwhereJ(x)istheJacobiandeterminantofthetransformationthatg(·):23@g1@g1@g16@x@x···@x7612n7676@g2@g2···@g27J(x)=J(x,x,...,x)=det66@x1@x2@xn77,12n6........766....774@gn@gn@gn5···@x1@x2@xnandweassumeJ(xi)6=0foralli.1.2.1IndependenceInthecaseoftworandomvariables,wedefinedtheindependenceofX,Yastheindependenceoftheevents{Xx},{Yy}forallx,y.Fornrandomvariables,wedefineindependenceofX1,X2,...,Xnastheindependenceofthenevents{X1x1},{X2x2},...,{Xnxn}forallx1,x2,...,xn.Thisisthecaseexactlywhenforallx1,x2,...,xnFX1X2···Xn(x1,x2,...,xn)=FX1(x1)FX2(x2)···FXn(xn),(1.25)orequivalentlywhenforallx1,x2,...,xnfX1X2···Xn(x1,x2,...,xn)=fX1(x1)fX2(x2)···fXn(xn).(1.26)Oneshouldnotethatitisnotenoughforeverypairofrandomvariablestobeindependenttoconcludethattheyareallindependent.Whenthislatterpropertyholdswesaytherandomvariablesarepairwiseindependent.Inthespiritofthedefinitionofindependenceofrandomvariables,wesaytworandomvectorsX˜andY˜areindependentifF˜(x;y)=F˜(x)F˜(y)(forallx,y),(1.27)XY˜XYorequivalentlyiff˜(x;y)=f˜(x)f˜(y)(forallx,y).(1.28)XY˜XYForacollectionofmrandomvectors,X˜1,X˜2,...,X˜m,wedefinethenotionoftheindependenceofthesemrandomvectorsasthesituationwhereFX˜1X˜2···X˜m(x1,x2,...,xm)=FX˜1(x1)FX˜2(x2)···FX˜m(xm)(forallx1,x2,...,,xm).(1.29)Aswithpairsofrandomvariables,itisnotenoughforeverypairoftherandomvectorstobeindependentforthecollectiontobeindependentifm>2.1.2.2ExpectationTheresultontheexpectationofafunctionoftworandomvariableshasthefollowinggeneralization:ifZ=g(X1,X2,...,Xn),thenwefindthatZ1Z1E{Z}=···g(x1,x2,...,xn)fX1X2···Xn(x1,x2,...,xn)dx1dx2...dxn(1.30)114 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013Fromthisresultwederivethegeneralformofthelinearitypropertyoftheexpectationoperator:E{↵1g1(X˜)+↵2g2(X˜)+···+↵mgm(X˜)}=↵1E{g1(X˜)}+↵2E{g2(X˜)}+···+↵mE{gm(X˜)}(1.31)providedalltheexpectationsexist.ThenotionoftheconditionalexpectationofarandomvariablethatisafunctionofX1,X2,...,XnconditionedonaneventMispreciselythesameasitwasinthecaseoftworandomvariablesexceptthatwenowmakeuseofthegeneraldefinitionofconditionaldistributions.ThusZ1E{X1|X2=x2,···,Xn=xn},x1fX1(x1|X2=x2,X3=x3,···,Xn=xn)dx1.(1.32)1LetX˜betherandomvector[X1,X2,...,Xn].WedefinetheexpectationofX˜asthevector(themeanvector)µX˜=E{X˜},[E{X1},E{X2},...,E{Xn}](1.33)(i.e.,expectationofavectortakencomponentbycomponent).ThecorrelationmatrixofX˜isdefinedtobe2⇤⇤⇤3E{X1X1}E{X1X2}···E{X1Xn}6⇤⇤⇤7T6E{X2X1}E{X2X2}···E{X2Xn}7R˜,E{X˜⇤X˜}=67.(1.34)X6.........74...5E{X⇤X}E{X⇤X}···E{X⇤X}|n1n{z2nn}matrixofvaluesr,wherer,E{X⇤X}isijijijthecorrelationofXiandXj(Thesuperscript⇤denotescomplexconjugationinthesenotes.)ThecovariancematrixofX˜isdefinedtobe231112···1n67T62122···2n7C⇤67X˜,E{[X˜µX˜][X˜µX˜]}=6........7,(1.35)4....5n1n2···nnwhere=E{(XE{X})⇤(XE{X})}isthecovarianceofcov(X,X).NotethatR⇤TijiijjijX˜=RX˜,andC⇤TX˜=CX˜(bothareHermitiansymmetricmatrices).WhentheelementsofX˜areallorthoganalrandomvariableswithrespecttoeachother,thenR˜isadiagonalmatrix.WhentheelementsofX˜areallXuncorrelatedrandomvariableswithrespecttoeachother,thenC˜isadiagnoalmatrixwiththevariancesXoftherandomvariablesasthediagonalelements.Theorem1.1.BoththecovarianceandcorrelationmatricesofarandomvectorX˜arepositivesemi-definiteHermitianmatrices.PnTProof:LetZ=i=1aiXi=X˜a.ThenXnE{|Z|2}=a⇤ar=a⇤RTijijX˜a.(1.36)i,j=1ButasE{|Z|2}0,itfollowsthata⇤RT˜a0,whichimpliesthatR˜ispositivesemi-definite.XXAsimilarconsiderationofvar(Z)=E{|ZE{Z}|2}givesusthatC˜ispositivesemi-definitematrix.X5 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013Wecanalsodefinethecross-correlationmatrixoftworandomvectorsX˜=[X1,X2,...,Xn]andY˜=[Y1,Y2,...,Ym]as2⇤⇤⇤3E{X1Y1}E{X1Y2}···E{X1Ym}6⇤⇤⇤7T6E{X2Y1}E{X2Y2}···E{X2Ym}7R˜,E{X˜⇤Y˜}=67,(1.37)XY˜6.........74...5E{X⇤Y}E{X⇤Y}···E{X⇤Y}|n1n{z2nm}matrixofvaluesr,wherer,E{X⇤Y}ijijijandthecross-covariancematrixofX˜andY˜as23c11c12···c1m67T6c21c22···c2m7C⇤67X˜Y˜,E{[X˜µX˜][Y˜µY˜]}=6........7,(1.38)4....5cn1cn2···cnmwherec=E{(XE{X})⇤(YE{Y})}=cov(X,Y).NotethatR⇤T⇤TijiijjijX˜Y˜=RY˜X˜,CX˜Y˜=CX˜Y˜,RT˜=R˜,andC˜=C˜.FurthermoreC˜=R˜µ˜µ˜.WheneveryrandomvariableinXX˜XXX˜XXY˜XY˜XYX˜isuncorrelatedwithrespecttoeveryrandomvariableinY˜(i.e.,ifCX˜Y˜=0,whichisthesameasRT˜=µ˜µ˜),wesaythatX˜andY˜areuncorrelatedrandomvectors.WheneveryrandomvariableinXY˜XYX˜isorthogonalwithrespecttoeveryrandomvariableinY˜(i.e.,ifRX˜andY˜areX˜Y˜=0),wesaythatorthogonalrandomvectors.ThecorrelationandcovariancematricesofZ˜=[X˜,Y˜]areclearlythe(n+m)⇥(n+m)matricesR˜R˜C˜C˜XXY˜XXY˜R˜=,andC˜=.(1.39)ZZR˜R˜C˜C˜YX˜YYX˜Y1.2.3CharacteristicFunctionsThecharacteristicfunctionofX˜=[X1,X2,...,Xn]isdefinedasj(!1X1+!2X2+···+!nXn)X˜(!1,!2,...,!n)=E{e},(1.40)|{z}⌦i.e.,Tj⌦X˜˜(⌦)=E{e}.(1.41)XIff˜(x)isthedensityfunctionofX˜,then˜(⌦)isitsn-dimensionalFouriertransformat⌦:XXZ1T+j⌦x˜(⌦)=f˜(x)edx(1.42)XX1i.e.,Z1Z1j(!1X1+!2X2+···+!nXn)X˜(!1,!2,...,!n)=···fX˜(x1,x2,...,xn)edx1dx2···dxn.(1.43)11ThenZ1Z11j(!1x1+!2x2+···+!nxn)fX˜(x1,x2,...,xn)=(2⇡)n···X˜(!1,!2,...,!n)ed!1d!2···d!n.(1.44)116 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013ThegeneralformofthemomenttheorembecomesthestatementthatifE{|X|j1|X|j2···|X|jn}<1for12nalljiki,then@k1+k2+···+kn(k1+k2+···+kn)k1k2knX˜(!1,!2,...,!n)!1=0.=jmk1,k2,...,kn,(1.45)@!1@!2···@!n..!n=0where,ofcourse,m,E{Xk1Xk2···Xkn}isajointmomentoforderk+k+···+k.k1,k2,...,kn12n12n1.3GAUSSIANRANDOMVECTORSLetX˜=[X1,X2,...,Xn],withXi⇠N(0,1)andallXiindependent.Then11(x2+x2+···+x2)11xIxTf˜(x1,x2,...,xn)=pe212n=pe2,(1.46)X(2⇡)n(2⇡)nwhereIdenotestheidentitymatrix.IfweconsiderarandomvectorY˜definedasanon-singularanetransformationofX˜Y˜=X˜A+µY˜,(1.47)whereAissomen⇥nmatrixandµ˜issomerowvector[whichwillformthemeanvectorofY˜,henceYthenotation],thenasforthecaseoftwojointlyGaussianrandomvariableswefindthatthecharacteristicfunctionofY˜canbegivenasT1T˜(⌦)=ej⌦µY˜e2⌦CY˜⌦,(1.48)XwhereC˜isthecovariancematrixofY˜.WhenC˜isnonsingular(soithasaninverse;thisoccursexactlyYYwhenAisnonsingular),wefindthatthedensityfunctionofofY˜canbeexpressedas111Tfp2[yµY˜]CY˜[yµY˜]˜(y)=e.(1.49)Y(2⇡)ndetC˜YDefinition:WhenY˜isarandomvectorwithacharacteristicfunctionthatcanbeexpressedasin(1.48)(orequivalently,inthenonsingularcase,withadensityfunctiongivenby(1.49)),thenwesaythatY˜isaGaussianrandomvectororthatY1,Y2,...,YnarejointlyGaussian.AswithtwojointlyGaussianrandomvariables,wemayshowthatthemarginaldensityfunctionsofjointlyGaussianrandomvariablesarealso(jointly)Gaussian.Also,ifwetransformjointlyGaussianrandomvariablewithalineartransformation,wecanshowthattheresultisalwaysacollectionofjointlyGaussianrandomvariables.ThusitfollowsthatanylinearcombinationofnjointlyGaussianrandomvariablesproducesaGaussianrandomvariable.ConverselywemayshowthatifalllinearcombinationsofnrandomvariablesX1,X2,...,XnareGaussian,thentherandomvariablesmustbejointlyGaussian.ThisleadsustotheimportantresultthatX1,X2,...,XnarejointlyGaussianifandonlyeverylinearcombinationoftherandomvariableproducesaGaussianrandomvariable.TheabovediscussionappliesonlytocollectionsofrealGaussianrandomvariables.Itcanbeex-tendedtoapplytocollectionsofcomplexGaussianrandomvariablesformingacomplexrandomvectorZ˜=[Z1,Z2,...,Zn],whereZiisacomplexrandomvariablewithrealpartXiandimaginarypartYi.WesaythatZ˜isacomplexGaussianrandomvector(orthatZ1,Z2,...,ZnarejointlyGaussiancomplexrandomvariables)ifthe2nrandomvariablesX1,X2,...,Xn,Y1,Y2,...,YnarejointlyGaussian.Thisis7 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013trueifandonlyifalllinearcombinations(withcomplexcoecients)oftheZ1,Z2,...,ZnformacomplexGaussianrandomvariable.CharacterizingacomplexGaussianrandomvectorissomewhatmorecomplexthanforrealGaussianrandomvectors.Unfortunately,themeanvectorandcovariancematrixofZ˜donotfullycharacterizeitsjointdistribution.Instead,besidesthemeanvector,weneedthecorrelation(orcovariance)matricesofX˜=[X1,X2,...,Xn]andY˜=[Y1,Y2,...,Yn],andthecross-correlationmatrixofX˜andY˜whichwillthengiveusthecorrelation(orcovariance)matrixof[X˜,Y˜]asnotedbefore.1.4RANDOMVECTOREXAMPLESIExample1.1TheChi-SquaredDistributionSupposeX˜=[X1,X2,...,Xn]aGaussianrandomvectorwithzeromeanvectorandcovariancematrixCX˜=I(i.e.,allXiindependentand⇠N(0,1)).Considertherandomvariable2,X2+X2+···+X2.12npThisrandomvariableistermedthe2[chi-squared]randomvariablewithn“degreesoffreedom”(,2isthe“chirandomvariable”).Itsdensityfunctionis(n1x/2(x/2)2e/2(n/2),x0;f2(x)=(1.50)0,otherwise.Thisrandomvariableoccursfrequentlyinthetheoryofexperimentalerrorsandespeciallyintheso-called2-goodness-of-fittestsofstatisticalhypothesistesting.Thedensityfunctionanddistributionfunctionof2fordi↵erentnisextensivelytabulatedinstatisticalreferencebooksandmosttextsonstatistics.NotethatifwehaveallX⇠N(0,2)andindependent,theni[X2+X2+···+X2]/2(1.51)12nisdistributedaccordingtothe2distribution.IExample1.2SampleMeanandSampleVarianceInattemptingtodeterminetheparametersofarandommodelforsomeexperimentfromsomeobser-vationsx1,x2,...,xn,onefrequentlyusesthesamplemeandefinedbyµs,(x1+x2+···+xn)/n.(1.52)Theideabehinditsuseisthatouroutcomescorrespondtoindependenttrialsofthesamequantity,oratleastofoutcomesdescribedbyindependentidenticallydistributedrandomvariables.However,asaconsequenceoftherandomness,µsisitselfisrandomandsowenaturallybecomeconcernedwithhowreliableµsisasanestimateofthemean.IfXiisarandomvariabledescribingxi(alltheXiareindependentidenticallydistributedrandomvariables),thentherandomvariabledescribingµsisX1+X2+···+Xnµs,.(1.53)nThemeanofthesamplemeanisthen11E{µs}=[E{X1}+E{X2}+···+E{Xn}]=·nE{Xi}=E{Xi}.(1.54)nnThusthemeanofthesamplemeancoincideswithmeanofeachoftheXi.Forthevarianceofµswefind1var(µs)=2[var(X1)+var(X2)+···+var(Xn)]n1=var(Xi).(1.55)n8 ELG3126RANDOMSIGNALSANDSYSTEMSWinter2013Thusthevarianceofµsisreducedbythefactor1/nwithrespecttovar(Xi),implyingthatanincreasinglyreliableestimateofthemeanofXiisobtainedasnincreases.AsanestimateofthevarianceofXifromtheobservationsx1,x2,...,xnonemightnaturallyconsiderthequantity(xµ)2+(xµ)2+···+(xµ)221s2snsˆs=.(1.56)nThemeanofthecorrespondingrandomvariableis(Xµ)2+(Xµ)2+···+(Xµ)21E{ˆ2}=E{1s2sns}=·nE{(Xµ)2}=E{(Xµ)2}.(1.57)s1s1snnNow(Xµ)2=X22Xµ+µ21s11ss2Xn1XnXn=X2XX+XX,(1.58)1n1jn2ijj=1i=1j=1whichimpliesthat2Xn1XnXnE{(Xµ)2}=E{X2}E{XX}+E{XX}1s1n|{z1j}n2|{zij}j=18i=1j=18