School of Informatics and Computing Menu

Gian-Luca Mariottini, Ph.D.: Surgical Vision at the ASTRA Robotics Lab: Toward Long-term and Accurate Augmented-Reality Display for Minimally-Invasive Robotic Surgery


Abstract: Augmented-Reality (AR) displays increase surgeon’s visual awareness of high-risk surgical targets (e.g., the location of a tumor) by accurately overlaying pre-operative radiological 3-D model onto the intra-operative laparoscopic video. Existing AR systems lack in accuracy and robustness against frequent illumination changes, camera motions, and organ occlusions, which rapidly cause the loss of image(anchor) points, and thus the loss of the AR display after a few seconds.

In this talk, I will present our recent work at the ASTRA Robotics Lab @ UTA for the design and prototype development of a new AR system, which represents the first steps toward long term and accurate augmented surgical display.

This work is in collaboration with the Urology Dept. at UTSW. Our system can automatically recover the overlay by predicting the image locations of a high number of AR anchor points that were lost after a sudden image change. A weighted sliding-window least-squares approach is also used to increase the accuracy of the AR display over time. The effectiveness of the proposed strategy in recovering the augmentation has been tested over many real partial-nephrectomy laparascopic surgical videos from a DaVinci robot.


Bio: Gian Luca Mariottini (S’04-M’06) received the M.S. degree in Computer Engineering in 2002 and the Ph.D. degree in Robotics and Automation from the University of Siena, Italy, in 2006. In 2005 and 2007 he was a Visiting Scholar at the GRASP Lab (CIS Department, UPENN, USA) and he held postdoctoral positions at the University of Siena (2006-2007), Georgia Institute of Technology (2007-2008), and the University of Minnesota (2008-2010), USA. Since September 2010, he has been an Assistant Professor at the Department of Computer Science and Engineering, University of Texas at Arlington, Texas, USA, where he directs the ASTRA Robotics Lab. His research interests are in robotics and computer vision, with a particular focus on single- and multi-robot sensing, localization, and control, as well as on surgical vision and augmented-reality systems for minimally-invasive surgical scenarios.