数据和知识双驱动的图像理解算法BeyondPureData-DrivenImageUnderstandingbyExploitingExternalKnowledge代季峰商汤研究院AboutMe•2005-2009,undergraduatestudent,DepartmentofAutomation,TsinghuaUniversity•2009-2014,Ph.D.student,DepartmentofAutomation,TsinghuaUniversity•2012-2013,visitingstudent,VCLAlab,UCLA•2014-2019,workingatVisualComputingGroup,MSRA•2019-now,workingatSenseTimeResearch•EditorialboardmemberofIJCV•AreachairofECCV2020,CVPR2021•SeniorPCmemberofAAAI2018ResearchonImageUnderstanding•R-FCN*(Daietal.,NIPS’16,2215citations)•DeformableConvNets*(Daietal.,ICCV’17,779citations)•Researchoninstancesegmentation:MNC(Daietal.,CVPR’16,699citations),FCIS(Lietal.,CVPR’17,395citations),CFM(Daietal.,CVPR’15,321citations)•1stplaceinCOCOchallenges2015&2016,3rdplaceinCOCOchallenges2017•AlgorithmsadoptedbywinningentriesofCOCOchallenges2017,2018,2019,andImageNetVID2017*WorksareintroducedatCIS680,taughtbyProf.JianboShiatUpennCurrentSuccessofImageUnderstanding•Puredata-drivenimagerecognitionwithdeepnetworks•Conformablezone:vastannotateddataforspecifictasksImageNetforimagerecognitionCOCOforobjectdetection&instancesegmentationKineticsforactionrecognitionCanwesimplyscaleup?ChallengesofPureData-DrivenImageUnderstanding(I)•LongtaildistributionofcategoriesChallengesofPureData-DrivenImageUnderstanding(II)•GeneralizationtounseensettingsisproblematicDeepNets:WhathavetheyeverdoneforVision?AlanL.YuilleandChenxiLiu.ArxivTechReport,2019.ChallengesofPureData-DrivenImageUnderstanding(III)•Manyentitiesandrelationsareinfeasibletodetectbytheirappearances,somedonotevenshowinanypixelshttps://vcla.stat.ucla.edu/dark/ChallengesofPureData-DrivenImageUnderstanding(IV)•ComplextasksatcognitionlevelFromRecognitiontoCognition:VisualCommonsenseReasoning.RowanZellers,YonatanBisk,AliFarhadi,YejinChoi.CVPR,2019.BeyondPureData-DrivenImageUnderstanding•Howdoesbabylearn?•“Theevidencethatchildrenarealreadybornknowingcertainthingsisextensive.Forexample,babiesseemtobeawarealreadyfrombirthofsomeof...
