A framework for the fusion of visual and tactile modalities for improving robot perception.pdf

《A framework for the fusion of visual and tactile modalities for improving robot perception.pdf》由会员上传分享，免费在线阅读，更多相关内容在学术论文-天天文库。

1、SCIENCECHINAInformationSciences.RESEARCHPAPER.January2017,Vol.60012201:1–012201:12doi:10.1007/s11432-016-0158-2AframeworkforthefusionofvisualandtactilemodalitiesforimprovingrobotperceptionWenchangZHANG1,2,FuchunSUN1*,HangWU2&HaolinYANG11TheStateKeyLaboratoryofIntelligentTechnologyandSyste

2、ms,TsinghuaUniversity,Beijing100084,China;2InstitutionofMedicalEquipment,Tianjin300161,ChinaReceivedMarch8,2016;acceptedJune30,2016;publishedonlineNovember22,2016AbstractRobotsshouldideallyperceiveobjectsusinghuman-likemulti-modalsensingsuchasvision,tactilefeedback,smell,andhearing.Howeve

3、r,thefeaturespresentationsaredifferentforeachmodalsensor.Moreover,theextractedfeaturemethodsforeachmodalarenotthesame.Somemodalfeaturessuchasvision,whichpresentsaspatialproperty,arestaticwhilefeaturessuchastactilefeedback,whichpresentstemporalpattern,aredynamic.Itisdifficulttofusethesedataat

4、thefeaturelevelforrobotperception.Inthisstudy,weproposeaframeworkforthefusionofvisualandtactilemodalfeatures,whichincludestheextractionoffeatures,featurevectornormalizationandgenerationbasedonbag-of-system(BoS),andcodingbyrobustmulti-modaljointsparserepresentation(RM-JSR)andclassification,

5、therebyenablingrobotperceptiontosolvetheproblemofdiversemodaldatafusionatthefeaturelevel.Finally,comparativeexperimentsarecarriedouttodemonstratetheperformanceofthisframework.Keywordsmulti-modalfusion,robotperception,vision,tactile,classificationCitationZhangWC,SunFC,WuH,etal.Aframeworkfor

6、thefusionofvisualandtactilemodalitiesforimprovingrobotperception.SciChinaInfSci,2017,60(1):012201,doi:10.1007/s11432-016-0158-21IntroductionHumansperceiveobjectssyntheticallyusingvision,tactilefeedback,smell,hearing,andtaste;thefor-mertwosensoryinputsaremostimportant.Whenhumansencounterdi

7、fficultyinvisuallydiscriminatingobjectswithsimilarappearance,tactilesensingcanhelpdistinguishthembyreasoningandsynthesizing.Therefore,informationofeachmodalityandfeaturecanimprovetherobustnessandaccuracyofclassi-fication.However,thephysiologicalmechanismresponsiblefort