A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data

De Barrie, Daniel, Pandya, Manjari, Pandya, Harit , Hanheide, Marc and Elgeneidy, Khaled (2021) A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data. Frontiers in Robotics and AI, 8 . p. 631371. ISSN 2296-9144

Full content URL: https://doi.org/10.3389/frobt.2021.631371

Documents
A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data
Published Open Access journal
[img]
[Download]
[img]
Preview
PDF
A Deep Learning Method for Vision Based Force Prediction of a Soft Fin Ray Gripper Using Simulation Data _ Enhanced Reader.pdf - Whole Document
Available under License Creative Commons Attribution 4.0 International.

12MB
Item Type:Article
Item Status:Live Archive

Abstract

Soft robotic grippers are increasingly desired in applications that involve grasping of complex and deformable objects. However, their flexible nature and non-linear dynamics makes the modelling and control difficult. Numerical techniques such as Finite Element Analysis (FEA) present an accurate way of modelling complex deformations. However, FEA approaches are computationally expensive and consequently challenging to employ for real-time control tasks. Existing analytical techniques simplify the modelling by approximating the deformed gripper geometry. Although this approach is less computationally demanding, it is limited in design scope and can lead to larger estimation errors. In this paper, we present a learning based framework that is able to predict contact forces as well as stress distribution from soft Fin Ray Effect (FRE) finger images in real-time. These images are used to learn internal representations for deformations using a deep neural encoder, which are further decoded to contact forces and stress maps using separate branches. The entire network is jointly learned in an end-to-end fashion. In order to address the challenge of having sufficient labelled data for training, we employ FEA to generate simulated images to supervise our framework. This leads to an accurate prediction, faster inference and availability of large and diverse data for better generalisability. Furthermore, our approach is able to predict a detailed stress distribution that can guide grasp planning, which would be particularly useful for delicate objects. Our proposed approach is validated by comparing the predicted contact forces to the computed ground-truth forces from FEA as well as real force sensor. We rigorously evaluate the performance of our approach under variations in contact point, object material, object shape, viewing angle, and level of occlusion.

Keywords:Robotics, soft gripper, Finite Element Modelling (FEM), Deep learning
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
H Engineering > H671 Robotics
Divisions:College of Science
Related URLs:
ID Code:45569
Deposited On:02 Jul 2021 09:08

Repository Staff Only: item control page