× Research Publications Teaching Projects
☰ menu

Stefano Rosa

I am working on Robotics and Artificial Intelligence and a I am research fellow at University of Oxford (UK), under supervision of Prof. Niki Trigoni, working on the ESPRC Programme Grant "Mobile Robotics: Enabling a Pervasive Technology of the Future".

I was a research fellow at Politecnico di Torino (Italy), working in collaboration with Telecom Italia S.P.A. I was a PhD student in Robotics at Istituto Italiano di Tecnologia (IIT) and Politecnico di Torino, working on space robotics and service robotics.

My research interests include localization and mapping for mobile robotics, computer vision applied to robot navigation, and human-robot interaction. I also have an interest on vision-based assistive technologies.

Email  /  CV  /  Google Scholar  /  LinkedIn  /  Twitter  /  GitHub


Research

Spatial AI: Situational awareness requires autonomous agents to build and maintain a multi-layered model of the environment, including both a geometric model (useful for navigation and coordination) and a semantic level (useful to execute high-level tasks and to provide more succinct information to human operators). I work on using human experiences to improve semantic understanding of the environment for mobile assistive robots.

Generative Visual Models and Intuitive Physics Understanding: My research tries to bring common sense understanding to robotic perception. Interacting with the environment requires to perceive objects and understand how actions influence their movement ad shape. Generative perception models can make sense of partial and noisy observations and reconstruct their shape and semantics. On the other hand, understanding the intuitive physics of objects interacting with each other will provide next-generation AI agents with a common sense knowledge base that will enable human-level interaction with a complex, dynamical environment.

Interpretable Sensor Fusion: Recent developments in machine learning have made possible to learn end-to-end motion estimation from visual, inertial and ranging devices. I study reasoned ways to learn sensor fusion strategies in deep VIO frameworks. At the same time, I study how to integrate novel sensor modalities such as millimeter wave radar and thermal imaging into a single framework.

News
  • May 2020: Invited Talk at Italian Institute of Technology (IIT)
  • Mar 2020: 1 paper accepted at MobiSys 2020
  • Feb 2020: 1 paper accepted at CVPR 2020 (oral)
  • Jan 2020: 1 paper accepted at IEEE Robotics and Automation Letters / ICRA 2020
Preprints

SelectFusion: A Generic Framework to Selectively Learn Multisensory Fusion

Changhao Chen, Stefano Rosa, Chris Xiaoxuan Lu, Niki Trigoni, Andrew Markham
arXiv:1912.13077
[PDF]

We build a generic deep framework for state estimation from multiple sensor modalities, based on selective fusion of sensor streams. We show how selective fusion can help to increase robustness to noisy and misaligned data.


Publications
2020

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds

Quingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, Andrew Markham,
CVPR-2020, Conference on Computer Vision and Pattern Recognition, Seattle, WA
[PDF] [code] [video]

We show that random sampling combined with attention can achieve SOA performances in semantic segmentation while processing large point clouds in near real-time.

milliMap: Robust Indoor Mapping with Low-cost mmWave Radar

Chris Xiaoxuan Lu, Stefano Rosa, Peijun Zhao, Bing Wang, Changhao Chen, Niki Trigoni, Andrew Markham,
MOBYSIS-2020, The 18th ACM International Conference on Mobile Systems, Applications, and Services, Toronto, Canada, June 2020
[PDF]

We show how to build dense occupancy grid maps of indoor environments from sparse, noisy mmWave measurements, with cross-modal training.

DeepTIO: A Deep Thermal-Inertial Odometry with Visual Hallucination

Muhamad Saputra, Pedro Gusmao, Chris Xiaoxuan Lu, Yasin Almalioglu, Stefano Rosa, Changhao Chen, Johan Wahlstrom, Wei Wang, Andrew Markham, Niki Trigoni
RA-L, IEEE Robotics and Automation Letters
ICRA-2020, International Conference on Robotics and Automation, Paris, France, May 2020
[PDF]

In this RA-L work we try to hallucinate visual features from thermal images that can help first responders to navigate visually-denied scenarios.

2019

Selective Sensor Fusion for Neural Visual-Inertial Odometry

Changhao Chen, Stefano Rosa, Yishu Miao, Chris Xiaoxuan Lu, Wei Wu, Andrew Markham, Niki Trigoni
CVPR-2019, Conference on Computer Vision and Pattern Recognition, Long Beach, USA, June 2019
[PDF (2.6 MB)] [Bibtex] [Project Website]

We show how data-learned sensor fusion strategies can improve accuracy and robustness in deep VIO when dealing with noisy/corrupted data, while adding interpretability.

2018
prl

3D Object Dense Reconstruction from a Single Depth View

Bo Yang, Stefano Rosa, Andrew Markham, Niki Trigoni, Hongkai Wen
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2018, DOI: 10.1109/TPAMI.2018.2868195
[PDF] [Bibtex]

We propose an end-to-end approach to high-resolution reconstruction of 3D objects from a single depth image. We also release a real-world dataset for 3D reconstruction. We argue that real-world benchmarks for shape reconstruction are necessary for a thorough validation of future approaches.

Learning the Intuitive Physics of Non-Rigid Object Deformations

Stefano Rosa, Zhihua Wang, Andrew Markham
NeurIPS-2018 Workshops, Modeling the Physical World: Perception, Learning, and Control, Montreal, 2018
[PDF]

Neural Allocentric Intuitive Physics Prediction from Real Videos

Zhihua Wang, Stefano Rosa, Yishu Miao, Zihang Lai, Linhai Xie, Niki Trigoni
arxiv
[PDF]

We learn how to predict future video of interacting objects by decoupling the problem into appearence and dynamics and leaning invertible transformations from real domain to simulation domain and from egocentric view to allocentric view and vice-versa.

Semantic Place Understanding for Human-Robot Coexistence - Towards Intelligent Workplaces

Stefano Rosa, Andrea Patane', Chris Xiaoxuan Lu, Niki Trigoni
Transactions on Human-Machine Systems (THMS), 2018, DOI: 10.1109/THMS.2018.2875079
[PDF]

Robots and users can work synergistically by mutually learning over time, and benefitting from each other by exploiting each other's strengths. We show how detecting user activities can help robots to learn semantic understanding of the environment, while at the same time learning to better localise the user.

3D-PhysNet: Learning the Intuitive Physics of Non-Rigid Object Deformations

Zhihua Wang*, Stefano Rosa*, Bo Yang, Sen Wang, Niki Trigoni, Andrew Markham
IJCAI-2018, 27th International Joint Conference on Artificial Intelligence, Stockholm, SWE
[PDF] [code] [webpage]

We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.

Defo-Net: Learning Body Deformation using Generative Adversarial Networks

Zhihua Wang*, Stefano Rosa*, Bo Yang, Linhai Xie, Sen Wang, Niki Trigoni, Andrew Markham
ICRA-2018, IEEE International Conference on Robotics and Automation, Brisbane, AU
[PDF] [code] [video] [webpage]

We show that conditioning a generative model that predicts soft object deformations on real physical properties can improve prediction accuracy as well as enabling generalisation abilities.

Learning with Training Wheels: Speeding up Training with a Simple Controller for Deep Reinforcement Learning

Linhai Xie, Sen Wang, Stefano Rosa, Andrew Markham, Niki Trigoni
ICRA-2018, IEEE International Conference on Robotics and Automation, Brisbane, AU
[PDF] [video]

We propose a way to embed a switchable, simple controller into a deep reinforcement learning algorithm, to speed up training of mobile robot navigation in simulated environments.

CommonSense: Collaborative learning of scene semantics by robots and humans

Stefano Rosa, Andrea Patane', Chris Xiaoxuan Lu, Niki Trigoni
MOBISYS-2018 Workshops, 1st International Workshop on Internet of People, Assistive Robots and ThingS (IoPARTS), Munich, DE
[PDF]

2017
  • Lu X., Kan X., Rosa S., Wen H., Markham A., Trigoni N., Towards Self-supervised Face Labeling via Cross-modality Association, poster, SenSys 2017, The Netherlands [PDF]
  • Rosa S., Lu X., Wen H., Trigoni N., Leveraging User Activities and Mobile Robots for Semantic Mapping and User Localization., HRI 2017 late break reports [PDF]
  • Rosa S., Toscana G., Bona B. Q-PSO: Fast Quaternion-based Pose Estimation From RGB-D Images, Journal of Intelligent and Robotic Systems, 2017, DOI: 10.1007/s10846-017-0714-3 [PDF] [code]
  • Anjum M.L., Rosa S., Bona B. Tracking a subset of skeleton joints - An effective approach towards complex human activity recognition, Journal of Robotics, vol. 2017, doi:10.1155/2017/7610417 [PDF] [code]
  • 2016
  • Rosa S., Toscana G. Fast Feature-Less Quaternion-based Particle Swarm Optimization for Rigid and Articulated Pose Estimation From RGB-D Images, poster, ECCV 2016, Amsterdam, NL
  • Toscana G., Rosa S., Fast Feature-Less Quaternion-based Particle Swarm Optimization for Object Pose Estimation From RGB-D Images, BMVC 2016, York, UK [PDF] [video] GPU:[code] CPU:[code]
  • Toscana G., Rosa S., Bona B., Fast Graph-Based Object Segmentation for RGB-D Images, Intellisys 2016, London, UK [PDF] [video] [code]
  • Toscana G., Rosa S., Bona B., Vocal Interaction with a 7-DOF Robotic Arm for Object Detection, Learning and Grasping, HRI 2016 Late break reports [PDF] [video]
  • Russo L.O., Rosa S., Maggiora M., Bona B. A Novel Cloud Based Service Robotics Application to Data Center Environmental Monitoring, Sensors, 2016, DOI: 10.3390/s16081255 [PDF]
  • Ermacora G., Rosa S., Toma A. Fly4SmartCity: a Cloud Robotics Service for Smart City Applications, Journal of Ambient Intelligence and Smart Environments, 2016, DOI: 10.3233/AIS-160374 [PDF]
  • 2015
  • Rosa S., Russo L.O., Toscana G., Primatesta S., Kaouk Ng M., Bona B., Leveraging the Cloud for Connected Service Robotics Applications, Workshop on Robotics and Technology Transfer, ETFA 2015, Luxemburg, LU
  • B. de Gusmao P.B., Rosa S., Magli E., Lepsøy S., Francini L., Robotics Navigation Using MPEG CDVS, 17th International Workshop on Multimedia Signal Processing, MMSP 2015, Xiamen, China[PDF]
  • Lupetti M.L., Rosa S., Ermacora G., From a Robotic Vacuum Cleaner to Robot Companion: Acceptance and Engagement in Domestic Environments., HRI 2015 late break reports
  • 2014
  • Russo L.O., Farulla G., Pianu D., Salgarella A., Controzzi M., Cipriani C., Oddo C., Geraci C., Rosa S., Indaco M., A remote communication system for deafblind persons by means of gesture recognition, International Journal of Advanced Robotic Systems, 2014 [PDF]
  • Bona B., Carlone L., Indri M., Rosa S.,Supervision and monitoring of logistic spaces by a cooperative robotic team: methodologies, problems, and solutions, Intelligent Service Robotics, 2014, DOI: 10.1007/s11370-014-0151-0 [PDF]
  • Rosa S., Russo L.O., Bona B., Towards A ROS-Based Autonomous Cloud Robotics Platform for Data Center Monitoring. the 19th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Barcelona, Spain, 2014 [PDF]
  • G. Ermacora, A. Toma, S. Rosa, B. Bona, M. Chiaberge, M. Silvagni, M. Gaspardone, R. Antonini, A cloud based service for management and planning of autonomous UAV missions in smartcity scenarios, MESAS 2014, Rome, IT
  • Airo' Farulla G., Russo L.O., Pintor C., Pianu D., Micotti G., Salgarella A.R., Camboni D., Controzzi M., Cipriani C., Calogero M.O., Rosa S., Indaco M., Real-time single camera hand gesture recognition system for remote deaf-blind communication 1st International Conference on Augmented and Virtual Reality - Salento AVR 2014, Lecce, 17-20 September 2014 [PDF]
  • Ahmad O., Yin J., Bona B., Rosa S., Anjum M.L., Skeleton Tracking Based Complex Human Activity Recognition Using Kinect Camera, ICSR 2014, Syndey, AU [PDF]
  • Ermacora G., Toma A., Rosa S., Antonini R., Leveraging open data for supporting a cloud robotics service in a smart city environment. at IAS-13, July 15 - 19, 2014, Padova, Italy [PDF]
  • Rosa S., Russo L.O., AirĂ² Farulla G., Antonini R., Gaspardone M., Carlone L., Bona B., An Application of Laser-Based Autonomous Navigation for Data-Center Monitoring. at IAS-13, July 15 - 19, 2014, Padova, Italy [PDF]
  • Yuan Z., Rosa S., Russo L.O., Bona B., A Kinect-based Front-end for Graph-SLAM Using Plane Matching in Planar Indoor Environments. at IAS-13, July 15 - 19, 2014, Padova, Italy [PDF]
  • Yin J., Carlone L., Rosa S., Anjum M.L., Bona B., Scan Matching for Graph SLAM in Indoor Dynamic Scenarios.27th International FLAIRS Conference, May 21 - 23, 2014, Pensacola Beach, Florida, USA
  • Russo L.O., Rosa S., Matteucci M., Bona B., A ROS Implementation of the Mono-SLAM Algorithm. In: International Conference on Artificial Intelligence & Applications (ARIA-2014), 2014 [PDF] [code]
  • 2013
  • Abrate F., Bona B., Indri M., Rosa S., Tibaldi F.,Multi-robot map updating in dynamic environments, in Springer Tracts in Advanced Robotics, Volume 83, 2013, DOI: 10.1007/978-3-642-32723-0 [PDF]
  • Russo L.O., AirĂ² farulla G., Indaco M., Rosa S., Rolfo D., Bona B., Blurring prediction in Monocular SLAM, In: 8th IEEE International Design & Test Symposium 2013 (IDT), 2013 [PDF]
  • 2012
  • Abrate F., Bona B., Indri M., Rosa S., Tibaldi F., Multirobot Localization in Highly Symmetric Environments, Journal of Intelligent and Robotic Systems, 2012, DOI: 10.1007/s10846-012-9790-6 [PDF]
  • L. Carlone, J. Yin, S. Rosa, Z. Yuan, Graph optimization with unstructured covariance: fast, accurate, linear approximation. In: Simulation, Modeling, and Programming for Autonomous Robots (SIMPAR 2012), 2012. [PDF] [code]
  • Rosa S., Paleari M., Ariano P., Bona B., Object Tracking with Adaptive HOG Detector and Adaptive Rao-Blackwellised Particle Filter. In: SPIE 8301, Intelligent Robots and Computer Vision XXIX: Algorithms and Techniques, 2012. [PDF]
  • 2011
  • Paleari M., Margaria V., Rosa S., Ariano P., HExEC: hand exoskeleton electromyographic control, 4th International Workshop on Human-Friendly Robotics (HFR 2011) November 8th-9th, 2011, University of Twente, The Netherlands [PDF]
  • 2010
  • Macchia V.; Rosa S; Carlone L; Bona B., An Application of Omnidirectional Vision to Grid-based SLAM in Indoor Environments. In: Workshop on Omnidirectional Robot Vision, International Conference on Robotics and Automation (ICRA 2010), 2010. [PDF]
  • Abrate F; Bona B; Indri M; Rosa S.; Tibaldi F., Map updating in dynamic environments. In: ISR/ROBOTIK 2010, 2010. [PDF]
  • 2009
  • Brevi D., Fileppo F. , Scopigno R. , Abrate F., Bona B., Rosa S., Tibaldi F., Hybrid localization solutions for robotic logistic applications. In: Technologies for Practical Robot Applications (TePRA), 2009. [PDF]
  • Abrate F; Bona B.; Indri M; Rosa S; Tibaldi F., Three-State Multirobot Collaborative Localization in Symmetrical Environments. In: ROBOTICA 2009, 2009 [PDF]
  • 2008
  • Abrate F; Bona B; Indri M.; Rosa S; Tibaldi F.,Switching Multirobot Collaborative Localization in Symmetrical Environments. In: IROS 2008 2nd Workshop on Planning, Perception and Navigation for Intelligent Vehicles, 2008. [PDF]

  • Teaching
    pacman

    Assistant lecturer for Automatic Control, Politecnico di Torino, 2013
    Assistant lecturer for Basics of Automatic Control, Politecnico di Torino, 2013

    Introduction to ROS, Robotics, Politecnico di Torino, 2013-2015

    Lecturer for Ph.D. course: Research topics in computer and control engineering, Politecnico di Torino, 2010-2012


    Past projects

    CSS credits: jonbarron