Self-supervised Representation Learning for Reliable Robotic Monitoring of Fruit Anomalies

Choi, Taeyeong, Would, Owen, Salazar-Gomez, Adrian and Cielniak, Grzegorz (2022) Self-supervised Representation Learning for Reliable Robotic Monitoring of Fruit Anomalies. In: 2022 IEEE International Conference on Robotics and Automation (ICRA), 23-27 May 2022, Philadelphia (PA), USA.

Full content URL:

This is the latest version of this item.

Self-supervised Representation Learning for Reliable Robotic Monitoring of Fruit Anomalies
Accepted manuscript
With_Copyright_Notice.pdf - Whole Document

Item Type:Conference or Workshop contribution (Paper)
Item Status:Live Archive


Data augmentation can be a simple yet powerful tool for autonomous robots to fully utilise available data for self-supervised
identification of atypical scenes or objects. State-of-the-art augmentation methods arbitrarily embed "structural" peculiarity on typical images so that classifying these artefacts can provide guidance for learning representations for the detection of anomalous visual signals. In this paper, however, we argue that learning such structure-sensitive representations can be a suboptimal approach to some classes of anomaly (e.g., unhealthy fruits) which could be better recognised by a different type of visual element such as "colour". We thus propose Channel Randomisation as a novel data augmentation method for restricting neural networks to learn encoding of "colour irregularity" whilst predicting channel-randomised images to ultimately build reliable fruit-monitoring robots identifying atypical fruit qualities. Our experiments show that (1) this colour-based alternative can better learn representations for consistently accurate identification of fruit anomalies in various fruit species, and also, (2) unlike other methods, the validation accuracy can be utilised as a criterion for early stopping of training in practice due to positive correlation between the performance in the self-supervised colour-differentiation task and the subsequent detection rate of actual anomalous fruits. Also, the proposed approach is evaluated on a new agricultural dataset, Riseholme-2021, consisting of 3.5K strawberry images gathered by a mobile robot, which we share online to encourage active agri-robotics research.

Keywords:Computer Vision, Anomaly Detection, Self-supervised Learning, Deep Learning, Data Augmentation, Agri-robotics, Robotic Monitoring, Agriculture, Fruit Image Data, Robotic perception
Subjects:G Mathematical and Computer Sciences > G700 Artificial Intelligence
G Mathematical and Computer Sciences > G760 Machine Learning
H Engineering > H671 Robotics
D Veterinary Sciences, Agriculture and related subjects > D400 Agriculture
G Mathematical and Computer Sciences > G740 Computer Vision
Divisions:College of Science > Lincoln Institute for Agri-Food Technology
ID Code:48682
Deposited On:04 Apr 2022 15:26

Available Versions of this Item

Repository Staff Only: item control page