Sound Production and Modeling

《Sound Production and Modeling》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

TutorialSoundProductionandModelingPerryR.CookPrincetonUniversityoundinmultimedia,movies,games,vir-dentinwaterwaves,whichdisplacethesurfaceoftheStualreality,andhuman–computerinter-waterupanddown,yettravelperpendiculartothedis-facesisagrowingfieldthatencompassesthedisciplinesturbancemotion.Figure1showsatransversewaveinofanaloganddigitalsignalprocessing,physics,speech,thewavetravelingonastring.Thetransversewavemusic,perception,andcomputersystemsarchitecture.alsooccursinmanyothersolidssuchasmembranesThisoverviewofsoundproductionandmodelingtech-andplates.Othertypesofpropagatingsoundwavesniquessurveysthestateoftheartinsoundtechnology.arelongitudinal,inwhichthedisturbanceisintheSoundhasbecomeacriticalcomponentinmodernsamedirectionasthepropagation.Thishappensinair,multimediasystems,especiallymultispeakersurround-asFigure2shows.soundentertainmentsystems.ManyWavespropagateinphysicalobjects,butthevibra-ofthecomponentsandtechniquestioncansometimesappearmuchdifferentlytoThistutorialgivesabriefinthesesystemsarealsoapplicableobservers.Forexample,ifweliftupastringunderten-invirtualandaugmentedrealitysys-sionatthecenterandreleaseit,itappearstoflopupandoverviewofsound,tems.Withbasicsoundhardwaredown,ratherthantheappearanceofwavestravelingnowavailableformost(butnotall,downandbackalongthestring.Similarly,ifwestrikeadescribingsoundasaasinsomepalmtops)computersys-drumheadinthecenter,itappearstoflopupanddowntems,andincreasingprocessor(likethestring,butin2D).Ifweweretopickupastringphysicalphenomenon,speedsallowingdirectreal-timeneartheend,andifwecouldobservethevibrationinsoundmanipulation,enhancingslowmotion,wemightseethedisturbancetravelingcomputerrepresentationsofmultimodalcomputer–humaninter-downthestringandback.Itturnsoutthatwecanfacesbyusingthesonicchannelisdecomposeanyoscillationofapluckedstringintotwosound,andperceptionofbecomingcommonplace.Soundincomponents—onegoingleftandtheotherrightonthegamesandothergraphics-intensivestring—andtheobservedstanding-wavedisplacementsoundbyhumans.real-timeapplicationsisachievingisthesumofthesetwotraveling-wavecomponents(seehigherlevelsofsophisticationandFigure3,nextpage).Thisisalsotrueinmostacousticalpromisestoundergoevenfurtheradvancementsinreal-systems,suchasavibratingmembrane,orthelongitu-ismandresponsiveness.dinaloscillationsinatubefilledwithair.Thespeedofsoundinairisabout340meters(1,100Soundwavesinairandmaterialsfeet)persecond.Thisisanimportantnumbertoremem-Soundisawavephenomenoncreatedbyvibrationsberwhenthinkingaboutsound,especiallywhencom-ofphysicalobjectsorfluids.Foranygivenmedium,paringthehumanauditoryandvisualsystems.Becausesoundtravelsataconstantratedeterminedbytheoftheslowpropagationspeed,timeandtimedelayarepropertiesofthatmediumsuchasthedensityandoftenthecriticalaspectsofsound.Forexample,soundbending/compressionmodulus.Instiffmedia(suchaswavestravelingdownandbackinsideatrombonetakerigidbarsandplates),thespeedofsoundissometimesabout0.005secondstocompletetheirround-trip,caus-afunctionofthefrequencyofoscillation,withgreateringtheinstrument(withtheplayer’slip)tooscillatenat-speedforincreasingfrequency.Somepropagatingsoundwavesaretransverse,wherethedisturbanceisorthogonaltothedirectionofpropagation.Thisisevi-+−+1Transversewaveonastring.2Longitudinalwavesinapipe.0272-1716/02/$17.00©2002IEEEIEEEComputerGraphicsandApplications23 Tutorial2(a)16543(b)4Modesofavibratingstring.Eachmodeislabeledwithaharmonicnumber.Thefundamentalmodeis(c)number1andundergoeshalfofasinusoidalexcursionalongthelengthofthestring.Thesecondmodeunder-3(a)Left-and(b)right-goingtraveling-wavecompo-goesonecompletesinusoidalperiodalongthestring.nentssumtoform(c)acompositedisplacementwaveThethirdmodeundergoes1.5periodsofaspatialsineonstring.wave,andsoforth.Modesofvibration1,1AsIalreadymentioned,superimposedtraveling2,2wavescangivetheappearanceandbehaviorofstation-arystandingwaves.Anotherwaytovisualizetheoscil-lationofsystemssuchasstrings,membranes,and1,22,3enclosedtubesandspacesistolookatitasasuperposi-tionofstandingsinusoidalmodesofvibration.Asim-pledefinitionforasystem’smodesisthattheyarethat3,31,3system’snaturalfrequencieswhenitisexcitedand5Modesofavibratingrectangularmembrane.Theallowedtovibratefreely.modesintheleftcolumnarelabeled1,1,1,2and1,3Figure4showssomeofthesinusoidalmodesofacorrespondingtothefirst,second,andthirddisplace-vibratingstring,andFigure5showsthefirstfewvibra-mentmodesinthehorizontal(x)direction.Similartotionalmodesofastruckrectangularmembrane.Later,thevibratingstringinFigure4,thesemodesarespatialwe’lltalkabouttheFouriertransform,whichletsuscon-harmonics,undergoing0.5,1.0,and1.5periodsofavertanyshapeorwaveformintoasumofsuperimposedsinusoidaldisplacementfunctioninthexdirection,andsinusoidalmodes.Modesturnouttobeaneconomicalallundergo0.5sinusoidalperiodsintheydirection.methodtomodelandsimulatesomevibratingsystemsTherightcolumnshowsthe2,2,2,3,and3,3modesandarealsoimportantperceptually,aswe’lldiscusslater.correspondingtothesuperimposedsinusoidaldisplace-mentsinthexandydirections.DigitalsoundIncomputersystems,wecapture,store,transmit,ana-lyze,transform,synthesize,andplaybackaudioindig-Amplitudeitalform.Thismeansthattogetananalogsignalintothecomputer,wemustsampleit(measuringtheinstan-Quantumtaneousvalue)atregularintervalsintimeandthenquantizeit(roundingortruncatingtothenearestdigi-6Samplingtalnumber)todiscretevalues.Thedifferencebetweenandquantiza-quantizationstepsiscalledthequantum.Theprocessoftionofawave-Timesamplingawaveform,holdingthevalue,andquantiz-form.1ingthevaluetothenearestnumberthatcanberepre-Samplingratesentedinthesystemisananalog-to-digital(AtoD,orA/D)conversion.Codingandrepresentingwaveformsinthismanneriscalledpulse-codemodulation(PCM).Thedevicethatdoestheconversionisananalog-to-digitalconverter(ADC,orA/D).Thecorrespondingprocessofconvertingthesampledsignalbackintoanurallyatalowfrequencyofabout50cyclespersecond.analogsignaliscalleddigital-to-analogconversion,andAsanotherexample,thereflectionsofourownsoundsthedevicethatperformsthisisaDAC.Filteringisalsocomingbackfromawall25feetawayaredelayedbynecessarytoreconstructthesampledsignalbackintoaabout45milliseconds.Thismightseemlikeanimper-smoothcontinuoustimeanalogsignal,andthisfilter-ceptibledelay,butasanexercise,gofindabigisolatedingisusuallycontainedintheDAChardware.Figure6walloutdoors,step25feetawayfromit,andclapyourshowsawaveform’ssamplingandquantization.handsafewtimes.Walkclosertothewallandclap,thenAfundamentalmathematicallawofdigitalsignalfartheraway,andyou’llbeabletohearthedifferenceprocessingstatesthatifananalogsignalisbandlimit-indelay.Toboggleyourmindalittlefurther,thelastedwithbandwidthB,wecanperiodicallysamplethesectionwilltalkaboutdelaysontheorderofmicrosec-signalatsamplerate2Bandexactlyreconstructitfromondsthatareperceptibletothehumanauditorysystem.thesamples.Ifcomponentsarepresentinasignalat24July/August2002 frequenciesgreaterthanhalfthesamplingrate,theseSpectrogramcomponentswillnotberepresentedproperlyandwillaliasasfrequenciesdifferentfromtheirtrueoriginalvalues.Soifweproperlybandlimitsignalsbeforesam-plingthem,wecanexactlygetthemback,butnotreal-7Spectrogramly.Becauseofquantizationwhenthesignalvaluesareandwaveformroundedortruncated,thesmallroundeddifferencesofutteranceofbetweentheoriginalsignalandthequantizedsignalthewordsound.arelostforever.AruleofthumbforestimatingthenoiseWaveformintroducedbyquantizationis6NdB,whereNisthenumberofbitsweusetorepresentthesignal.This102030405060meansthatasystemusing16-bitlinearquantizationTime(ms)willexhibitasignal-to-quantizationnoiseratioofapproximately96dB.Younghumanswithnormalhearingcanperceivefre-quenciesfromroughly20Hzto20kHz,thusrequiringaboutsoundperceptionasafunctionofthespectralaminimumsamplingrateofatleast40kHz.Speechsig-properties.Oneisthenoisy“sss”sectionfrom0.0to0.01nalsareoftensampledat8kHzor11.025kHz,whileseconds.Thisischaracterizedbythefuzzyenergyathighmusicisusuallysampledat22.05kHz,44.1kHz(thefrequenciesandrelativelylessenergyatlowfrequencies.samplingrateusedonaudiocompactdiscs),or48kHz.AnotherthingtonoteisthatwecanseetheindividualOtherpopularsamplingratesinclude16kHz(oftenharmonic(integermultiplesinfrequency)modesoftheusedinspeech-recognitionsystems)and32kHz.Thepitchedvoicesourceinthe“ahh”and“ooo”and“nnn”maximumsamplingrateavailableinmostmultimediasectionsfrom0.012to0.05seconds.Wecanalsoclear-systemsis48kHz,butnewsystemsandstandardsarelyseedifferencesintherelativeenergyatdifferentfre-proposingorusingsamplingratesof96or192kHz.quenciesinthesedifferentvowelandliquid-consonantMostPC-basedmultimediaaudiosystemsprovidetwo(phoneticnameforthe“nnn”)sounds.Finally,wecanorthreebasicsizesofaudiowords.Sixteen-bitdataisseethestopconsonant“d,”followedbyanoisyburstofcommonbecausethisisthedataformatcompactdiscairsoundasthe“d”isreleased.systemsuse.Eight-bitdataisequallycommonandisusuallyusedtostorespeechdata.Twenty-four-bitdataSynthesis:Waveform,spectrum,andhasrecentlybecomemorepopular.physicsThemajorityofcomputeraudiocomesfromtheplay-PerceptionofsoundandtheFFTbackofstoredPCMwaveforms.Single-shotplaybackTheimportanceofmodesinmodelingsomevibrat-ofentiresegmentsofstoredsoundsiscommonforingsystemsisn’tjustamathematical/geometrictrickorsoundeffects,narrations,prompts,musicalsegments,convenience.Infact,amechanisminourinnerearper-andsoon.Formanymusicalsounds,it’scommontoformsthefunctionofturningtheeardrum’stimeoscil-storejustoneloop,ortable,oftheperiodiccomponentlationsintofrequency-dependentnervefiringsintheofarecordedsoundwaveformandplaythatloopbackbrain.Soingeneral,wecreatesoundinthetimedomainrepeatedly.Thisiscalledwavetablesynthesis.Formorebutperceivemanyaspectsoftheseobjectsandprocess-realism,theattackorbeginningportionoftherecord-esinthefrequencydomain.edsoundisstoredinadditiontotheperiodicsteady-Justaswecanpasslightthroughaprismtobreakitstatepart.Originallycalledsamplingsynthesisintheintotheindividuallightfrequenciesfromwhichit’scom-musicindustry,allsynthesisinvolvingstoredPCMposed,wecanseparatesoundintoindividualsimplefre-waveformshasmorecommonlybecomeknownasquencies(sinewaves).Amathematicaltechnique,calledwavetablesynthesis.Filtersandenvelopes(time-afrequencytransform,uniquelyconvertsatimedomainvaryinggainfunctions)areusuallyaddedtowavetablewaveformintoasetoffrequencycomponents.Thesetofsynthesistocontrolspectralbrightnessasafunctionofindividualamplitudesandphasesofthesinesthatmakeintensityandtogetmorevarietyofsoundsoutofaupasoundarecalledafrequencyspectrum.Usingthefre-givensetofsamples.Wecanonlypitchshiftagivenquencyspectrumtoinspectaspectsofasoundiscalledsamplesofarineitherdirectionbeforeitbeginstospectralanalysis.Theprocessofsolvingforthesineandsoundunnatural.Wedealwiththisbystoringmultiplecosinecomponentsofasignalorwaveformiscalledrecordingsofthesoundatdifferentpitches,andswitch-Fourieranalysis,ortheFouriertransform.Thedigitalver-ingorinterpolatingbetweentheseuponresynthesis—sionoftheFouriertransformisthediscreteFouriertrans-aprocesscalledmultisampling.Figure8(nextpage)form(DFT),andafastalgorithmforcomputingit(iftheshowstheinitialattackofapluckedstring,followedbylengthofthesignalbeingtransformedisapowerof2)istherepetitionofashortloopwithtime-varyingdecaythefastFouriertransform(FFT).tomodeltheremainderofthesound.Aspectrogram(orsonogram)isatime-varyingFFTSynthesisofsignalsbyaddingfundamentalwaveformplotshowingtimeontheabcissa,frequencyontheheightcomponentsiscalledadditivesynthesis.Becausewecanaxis,andintensityateachfrequencyintimeshownasauniquelyrepresentanyfunctionasalinearcombinationofbrightnessofcolor.Figure7showsaspectrogramofansinusoidalcomponents,thepowerfultoolofFourieranaly-utteranceofthewordsound.Figure7showsmanythingssisgivesrisetoacompletelygenericmethodofsoundIEEEComputerGraphicsandApplications25 TutorialLoop8Wavetablesynthesisofapluckedguitarsound.0.10.20.30.40.50.60.7Time(seconds)analysisandresynthesis.Whenonlyafewsinusoidalcom-Three-dimensionalaudioprocessingandmodelingponents(afewnaturalmodesofasystem)exist,additiveendeavorstouseheadphonesorspeakerstoplacesynthesiscanbeefficientandflexible.Inthiscase,calledsourcesatarbitraryperceivedlocationsaroundthelis-modalsynthesis,wecanusesinusoidaloscillatorsorreso-tener’shead.Ifweuseheadphonesandsynthesizethenantfilterstomodeltheindividualmodes.However,manyappropriatecuesintothebinauralsignalspresentedtosoundshaveasignificantnumberofcomponentsthatvarythetwoears,wecandirectlymanipulatevirtualsourcerapidlyinmagnitudeandfrequency.locations.Butifusersmovetheirhead,theimageofManysoundsyieldwelltosubtractivesynthesis,thevirtualsonicworldmovestoo.Headtrackingiswherewefilteracomplexsourcesignaltoyieldasoundrequiredtokeepthevirtualsonicworldstableintheclosetothedesiredresult.Thehumanvoiceisasub-observer’srealworld.Usingjustapairofstereospeak-tractivesynthesizerbecausethecomplexwaveformpro-ersrequiresmoresignalprocessingtocanceltheeffectsducedbythevocalfoldsisfilteredandshapedbytheofeachspeakersignalgettingtobothears.VRsystemsvocaltracttube’sresonances.Thus,thevoiceiswellusinghelmetswithvisionsystemsoftenhaveaninte-modeledusingsubtractive-synthesistechniques.gratedstereoaudiocapability,andwecanusetheOthersynthesistechniques,suchasfrequencymod-head-trackingsystemsusedtomanipulatethevisualulation(FM)andwaveshaping,usenonlinearfunctiondisplaysinthesesystemstomanipulatetheaudiodis-transformationsofsimplewaveforms(oftensinewaves)playsaswell.tocreatecomplexspectra.OthersynthesistechniquesResearchershavehistoricallyusedmultispeakersys-exploitthestatisticalpropertiesofsomesounds(espe-temssuchasquadrophonicsoundandAmbisonicsinciallynoisysounds).Manysystemssupportmixturesofcertainresearchandartisticsettings,butothercom-techniques,andwiththeincreaseofsoftware-basedmunities(suchasentertainmentandgaming)haven’tsoundsynthesis,weshouldexpecttoseemoreflexiblebroadlyadoptedthemforvariousreasons,includingsystemsinthefuturethatusehybridtechniques.economy,multiplecompetingstandards,andlackofPhysical-modelingsynthesisendeavorstomodelandcommerciallyavailableprogrammaterial.Recently,solvetheacousticalphysicsofsound-producingsystemsvarioussurroundsoundformatssuchasDolbyPro-tosynthesizesound.Unlikeadditivesynthesis,whichLogic,Dolby5.1Digital,DTS,AC3,andSonySuper-canuseonepowerfulgenericmodelforanysound,phys-CDareenteringthehome-entertainmentmarket,icalmodelingrequiresadifferentmodelforeachsepa-includingdedicatedgameboxes.Thesenewstandardsratefamilyofmusicalinstrumentsorsoundproducingpromisetogiveamultispeakercapabilitytomanyobjects.Oneofthemanypotentialbenefitstophysicalmoremultimediasystemsowners,makingimmersivemodelingisthenaturalexpressivecontrolavailableaudioexperiencesinthehomemorepervasive.Inwhenusingatruephysicalsimulation.additiontothemovieandaudiophilemusictitlesthathaveinitiallydriventhemarketforthesenewsystems,HearingsoundinspaceswenowseemanynewgamesandothermultimediaThehumanauditorysystemisremarkablyadeptatcontentemergingthattakeadvantageofmultipleusingonlythe(essentially1Dvibration)informationatspeakersystems.theeardrumstodeterminethelocationsofsound-producingobjects.ThestrongestperceptualcueforConclusionleft–rightsoundlocationisthetimedelaybetweenourSoundpervadesourlives,sometimesininvasiveandtwoears.Rememberingourearlierdiscussionofsoundirritatingways,butmoreoftenasanenhancementorpropagationspeed,andassumingthatthedistancecomplementtoothersensoryinformation.It’softensaidbetweenourearsisabout9inches,wecouldperformthatsoundguidestheeyes,butweonlyneedtoclosesomesimpleexperimentstoconvinceourselvesthatoureyesforamomenttoexperiencetheamazingvari-humanscandiscerninterauraltimedelaycuesofonlyaetyofinformationthatourearsprovideandoftenmorefewmicroseconds.Thenextstrongestcueforsoundquicklyandrichlythananyothersense.Activeresearchlocationisamplitudedifferencebetweenthetwoears.areasinsoundincludeOthercuesincludefilteringrelatedtotheshadowingeffectsoftheheadandshouldersaswellascomplexfil-efficientparametricalgorithmsforreal-timesynthe-teringfunctionsrelatedtothetwistsandturnsofthesisofsound;pinnae(outerears).exhaustive(butprovablyphysical)synthesismethods;26July/August2002 userinterfacesforsoundmanipulation;soundattheuserinterface;RecommendedFurtherReadingmodelingandrenderingofsonicenvironmentsEachoftheseworksiswhatIconsiderthe3Dsound;best,singlebookreferenceonitstopic.soundinvirtualandaugmentedreality;immersivesoundsystems;D.Begault,3-DSoundforVirtualRealityandMultimedia,soundinartsandentertainment;andAcademicPress,SanDiego,1994.verificationandtestingofsynthesis,rendering,andR.Bracewell,TheFourierTransformandItsApplications,interactionsystems.McGraw-Hill,NewYork,1986.P.Cook,Music,Cognition,andComputerizedSound,MITMostofthesetopicsareessentiallyopen-ended,withPress,Cambridge,Mass.,1999.muchworkstillremainingtobedoneinallareasofcom-P.Cook,RealSoundSynthesisforInteractiveApplications,putationalsound.Futurehardwarecapabilities,algo-AKPeters,Natick,Mass.,2002.rithmicadvances,andapplicationareasremaintobeK.Pohlmann,PrinciplesofDigitalAudio,McGraw-Hill,researchedandexploited.NewYork,2000.C.Roads,AComputerMusicTutorial,MITPress,Cam-bridge,Mass.,1996.C.RoadsandJ.Strawn,eds.,FoundationsofComputerMusic,MITPress,Cambridge,Mass.,1985.PerryCookisanassociateprofes-J.O.Smith,DigitalWaveguideModelingofMusicalInstru-sorintheComputerScienceDepart-ments,bookinprogressathttp://www-ccrma.ment,withajointappointmentinstanford.edu/~jos/waveguide/.theMusicDepartment,atPrincetonK.Steiglitz,ADigitalSignalProcessingPrimer,WithAppli-University.HismainresearchareascationstoDigitalAudioandComputerMusic,Addisonarephysics-basedmodelsforsoundWesley,Reading,Mass.,1995.synthesis,humanperceptionofsound,audioatthehuman–computerinterface,anddevicesandsystemsforinteractivesoundcontrolandartis-ticperformance.HehasaBAinmusic(studyingvoiceandReadersmaycontactPerryCookattheDept.ofCom-electronicmusic)fromtheUniversityofMissouriattheputerScience,PrincetonUniv.,35OldenSt.,Princeton,KansasCityConservatoryofMusicandaBSEEinengi-NJ08544-2087,emailprc@cs.princeton.edu.neeringfromtheUniversityofMissouri.HealsohasaPhDinelectricalengineeringfromStanfordUniversity,whereForfurtherinformationonthisoranyothercomputinghewastechnicaldirectoroftheCenterforComputertopic,pleasevisitourDigitalLibraryathttp://computer.ResearchinMusicandAcoustics.org/publications/dlib.CallforPapersCallforPapersGraphicsApplicationsforGridComputingCG&AMarch/April2003Submissionsdue:31August2002Gridcomputingproposestocreateaccesstodistributedcomputingresourceswiththesameeaseaselectricalpower.Inrecentyears,graphicsapplicationtoolsthatcantakeadvantageofgridcomputingenvironmentshaveemerged.Newmethodologiesandtechniquesthatharnesssuchresourcesforgraphicsandvisualizationapplicationsarecriticalforthesuccessofgridenvironments.Theabilitytotakeadvantageofthedisparateresourcesavailableingrid-enabledapplicationsisbothexcitinganddifficult.Wesolicitpapersthatdescribeinnovativeresultsinthisarea.GuestEditorsChuckHansen,UniversityofUtahhansen@cs.utah.edu™IEEEChrisJohnson,UniversityofUtahANDAPPLICATIONScrj@cs.utah.edu