Abstract

Fruit counting and tracking is a crucial component of fruit harvesting and yield forecasting applications within horticulture. We present a novel multi-object, multi-class fruit tracking system to count fruit from image sequences. We first train a recurrent neural network (RNN) comprised of a feature extractor stem and two heads for re-identification and maturity classification. We apply the network to detected fruits in image sequences and utilise the output of both network heads to maintain track consistency and reduce intra-class false positives between maturity stages. The counting-by-tracking system is evaluated by comparing with a popular detect-to-track architecture and against manually labelled tracks (counts). Our proposed system achieves a mean average percentage error (MAPE) of 3% (L1 loss = 7) improving on the baseline multi-object tracking approach which obtained an MAPE of 21% (L1 loss = 41). Validating this approach for use in horticulture.