I'm GROOT: a multi head multi GRaph netwOrk recognizing surgical actiOn Triplets✱.

Laparoscopic cholecystectomy is a widely performed minimally invasive surgical procedure that imposes many challenges to the operating surgeon. While we strive to understand and automate such surgeries, the key is to identify the actions involved in it. An action involves a set of tools and a target anatomy, together forming the action triplets. However, the relations between the triplets and their constituents are sparse, making it challenging to learn their relations. In this paper, we propose a graph neural network based approach to exploit these underlying sparse relations in the data. We portray the proposed method’s ability to uniformly learn multiple tasks and classify triplets with an mAP of 0.261. In addition, we experimentally show the inability of fully connected and convolution layers to learn these sparse relations when trained on 40 laparoscopic videos and validated using five videos. Codes will be available at : https://github.com/iitkliv/groot.