资源描述：
《Formal grammar and information theory- together again》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。
1、Formalgrammarandinformationtheory:togetheragain?ByFernandoPereiraAT&TLabs{Research,FlorhamPark,NJ07932,USAInthelastfortyyears,researchonmodelsofspokenandwrittenlanguagehasbeensplitbetweentwoseeminglyirreconcilabletraditions:formallinguisticsintheChomsky
2、tradition,andinformationtheoryintheShannontradition.ZelligHar-rishadadvocatedaclosealliancebetweengrammaticalandinformation-theoreticprinciplesintheanalysisofnaturallanguage,andearlyformal-languagetheoryprovidedanotherstronglinkbetweeninformationtheorya
3、ndlinguistics.Neverthe-less,inmostresearchonlanguageandcomputation,grammaticalandinformation-theoreticapproacheshadmovedfarapart.Today,aftermanyyearsinthedefensive,theinformation-theoreticapproachhasgainednewstrengthandachievedpracticalsuccessesinspeech
4、recognition,informationretrieval,and,increasingly,inlanguageanalysisandmachinetransla-tion.Theexponentialincreaseinthespeedandstoragecapacityofcomputersistheproximatecauseoftheseengineeringsuccesses,allowingtheautomaticestima-tionoftheparametersofprobab
5、ilisticmodelsoflanguagebycountingoccurrencesoflinguisticeventsinverylargebodiesoftextandspeech.However,Iwillalsoarguethatinformation-theoreticandcomputationalideasareplayinganincreas-ingroleinthescienti cunderstandingoflanguage,andwillhelpbringtogetherf
6、ormal-linguisticandinformation-theoreticperspectives.Keywords:Formallinguistics;informationtheory;machinelearning1.TheGreatDivideInthelastfortyyears,researchonmodelsofspokenandwrittenlanguagehasbeensplitbetweentwoseeminglyirreconcilablepointsofview:form
7、allinguisticsintheChomskytradition,andinformationtheoryintheShannontradition.Chomsky(1957)'sfamousquotesignalsthebeginningofthesplit:(1)Colorlessgreenideassleepfuriously.(2)Furiouslysleepideasgreencolorless.:::Itisfairtoassumethatneithersentence(1)nor(2
8、)(norindeedanypartofthesesentences)haseveroccurredinanEnglishdiscourse.Hence,inanystatisticalmodelforgrammaticalness,thesesentenceswillberuledoutonidenticalgroundsasequally`remote'fromEnglish.Yet(1),thoughnonsensical,isgrammatica
