High-throughput phenotyping for breeding targets — Current status and future directions of strawberry trait automation

Strawberry breeders are faced with increasing demands by propagators, growers, retailers and consumers for particular agronomic traits. This and the volume of plants requiring assessment during selection constrain breeders to rapid and qualitative rating methods. High-throughput systems for assessing these traits automatically could indicate which families, or individual genotypes, should be singled out for further, more thorough evaluation, thus significantly increasing the selection intensity and accuracy. This review assesses the current status of and future potential for automated phenotyping in strawberry crops, highlighting key advances and the gaps which need to be addressed to facilitate the development of such technology.

The cultivated strawberry (Fragaria Â ananassa) is grown commercially throughout the temperate and subtropical zones of the world (Hancock, 2020), and its global cultivation has increased significantly over the past 20 years (for example, DEFRA (2021) illustrates this for the UK), with the development of superior genetics and advances in agricultural practices allowing for increased yields per hectare.
However, the availability and cost of labour pose a challenge to strawberry growers, as the price of fresh strawberries at the farm gate has remained largely static over the last decade (DEFRA, 2021).As a result, robotic operations in strawberry production have increased, automating tasks such as weed control, plant movement, sorting, stress detection and harvesting (Defterli et al., 2016), the lattermost of which is of critical need but is still in the development phase.
Strawberries are grown in several different commercial environments including glasshouses, polytunnels and open fields.In glasshouses and polytunnels, strawberry fruits hang down from tabletop systems, resulting in less occlusion of the fruit by the foliage which is beneficial for robotic interaction (Defterli et al., 2016).To keep harvest costs sustainable, new varieties must therefore have larger and betterdisplayed fruit which increases manual picking speeds (Diamanti et al., 2011) and, in time, will be amenable to robotic harvesting.
To meet grower and consumer demands, many breeding programmes have been established for cultivated strawberries throughout the world (Hancock, 2020).Although strawberry is a clonally propagated crop, the initial phases of strawberry selection are usually performed on large families of full-sibs with only single replicates of each genotype.To enable efficient selection, these programmes assess many agronomic traits, although the process is inefficient, as it requires significant time input from highly experienced domain experts.The breeder's equation (Kelly, 2011) predicts the mean changes that will be observed for a given trait (genetic gain) through the breeding and selection process over a given generation (time) in relation to the available genetic variation, the intensity and accuracy of the selection.With the limited evaluation window generally consisting of only a few weeks, the breeder's equation indicates that the intensity and accuracy of selection are limited by the seasonal nature of plant production.Due to this limitation, initial selection decisions are almost exclusively based on the qualitative evaluation of a genotype's characteristics, which may not be representative of the full potential of a genotype.Desired traits may thus be missed, and superior genetics may not be selected, despite their inherent genetic potential.Increasing the observation frequency as well as observation accuracy and precision in the field during selection would increase the efficiency of the breeding process, resulting in greater potential genetic gain.However, it is impractical to operate a breeding programme with large numbers of expertly trained personnel, meaning only a finite number of plants can be evaluated thoroughly and effectively in any given growing season manually.This leads to a bottleneck in the selection process that is difficult to overcome without automation.
The development of an automated system that could rapidly and

| Motivation, scope and contributions
Advances in robotics, machine learning and computer vision have begun to revolutionise data capture and analysis in real-world situations such as those found in plant phenotyping.A contemporary, complimentary review (Zheng et al., 2021) spans an array of applications for strawberry phenotyping broader than breeding, namely, yield forecasting, post-harvest monitoring, fruit quality assessment, stress, pest and disease detection.The review collates the past work done on combining remotely sensed data and machine learning methods, with computed traits related to fruit, leaves and canopy, as well as abiotic/ biotic stress detection, but does not address all relevant phenotyping traits for breeding, their automation status and potential for automation.High-throughput phenotyping has the potential to revolutionise the selection process in strawberry breeding by providing higher frequency, higher resolution, quantifiable measurements of traits.Here, we review the status of automation of those traits.Phenotypic traits may be broadly grouped as morphological, physiological and temporal (Choudhury et al., 2020).External morphological traits have the greatest automation potential as they can be directly measured through proximal/remotely sensed means.Physiological traits such as the percentage of soluble solids or chlorophyll content are typically assessed through destructive measurement, although some studies have attempted non-destructive analysis using NIR spectroscopy (for example, Mancini et al. (2020)).Temporal traits require multiple observations of the same trait over time, to measure traits such as leaf growth rate or flowering duration.Although temporal traits may measure external features, tracking components over time in real-world environments is challenging due to occlusion and movement of organs due to external factors, resulting in the rearrangement of organs (Magistri et al., 2020).
In this review, the current status of, and future potential for, the automation of external morphological traits of strawberry plants and fruit is examined, limiting the scope to external traits that are initially assessed qualitatively by breeders and are also important for future breeding targets such as ease of robotic harvest.We highlight relevant trends in methodology for the computation of these traits where high-throughput methods have been introduced and identify the remaining traits that are both of high relevance to breeding and are good candidates for future automation.This will highlight the gaps that need to be addressed to facilitate fully automated selection for traits that are critical in commercial cultivars and demonstrate how the automation of the phenotyping process will ultimately lead to richer, quantitative data that will better support breeders in their selection process.

| HIGH-THROUGHPUT IMAGE-BASED PHENOTYPING
Automated techniques using remotely sensed data are being used to address the phenotyping bottleneck both in breeding and in crop management (Jiang & Li, 2020), facilitating high-throughput phenotyping in crops including wheat, maize and soybean.Various sensing technologies have been utilised, each with different benefits and limitations, including resolution, cost and applicability for use in non-laboratory environments (Jin et al., 2021).Image-based phenotyping in particular shows great potential across the range of different spatial scales, ranging from individual organs to whole plants depending on the traits assessed (Li, Guo et al., 2020), although challenges relating to natural environmental variation and data acquisition, availability and analysis remain (Li, Guo et al., 2020;Minervini et al., 2015).
While tasks such as appearance assessment (which involves the identification or segmentation of particular parts of the plant or fruit) can be assessed using 2D images, 3D perception provides greater precision in localisation and shape analysis than 2D data, due to the geometrical representation of objects (Jiang & Li, 2020;Paulus, 2019).An example of a three-dimensional reconstruction of a strawberry, where surface points are arranged in 3D space, is shown in Figure 1.Point clouds, such as this, have potential for use in high-throughput assessment of morphological fruit quality attributes that would be too laborious for manual assessment (He et al., 2017).
Image analysis methods for automated phenotyping include both traditional image processing algorithms, which make use of 'handcrafted' (manually selected or engineered) features, as well as stateof-the-art data-driven techniques, where features are learnt from the supplied data, reducing the need for expert knowledge.Convolutional neural networks (CNNs) are one such family of algorithms that learn features directly from images and can be used for tasks such as the classification, detection and segmentation of image components (Figure 2) and have out-performed traditional methods in many applications, such as general visual object recognition and detection (LeCun et al., 2015).They have shown potential in recent years in image-based phenotyping, although data availability remains a challenge due to the requirement for a large number of manually annotated examples, which is resource-intensive and therefore costly (see Jiang and Li (2020) for a description of common CNN architectures and uses in phenotyping).
The variation in environmental conditions found in real-world agricultural settings poses a further challenge to image-based phenotyping and has been highlighted by several authors in the context of strawberry phenotyping.Illumination changes, due to variation with the time of day, weather and shadow (Heylen et al., 2021;Ilyas et al., 2021;Kirk et al., 2020;Lin & Chen, 2018;Yu et al., 2019;Zhou et al., 2020) pose one such impediment.However, CNNs are reasonably robust to variance in illumination, and this can be further addressed through methods such as merging features from different colour spaces (Kirk et al., 2020).The unstructured, complex environment also poses challenges in terms of occlusion of the organs under evaluation by other fruit, flowers, stems or leaves, and cluttered backgrounds make segmentation difficult (Fan et al., 2022;Kirk et al., 2020;Lamb & Chuah, 2018;Lin & Chen, 2018;Yu et al., 2019;Zhou et al., 2020), but imaging from multiple viewpoints (Kerfs et al., 2017) and 3D sensing have the potential to assist with this (Le Louëdec & Cielniak, 2021a).Fruit characteristics, such as the small size of the fruit and variation in appearance, have also been noted as further obstacles in agricultural settings (Fan et al., 2022;Kirk et al., 2020), along with sensor-related restrictions, such as available camera viewpoints, low contrast, variance in both colour balance and saturation and the interference of the sun on infra-red based sensors (Heylen et al., 2021;Kirk et al., 2020;Le Louëdec & Cielniak, 2021a).

| AUTOMATION OF MORPHOLOGICAL TRAITS CURRENTLY USED IN BREEDING
Strawberry cultivars are described and characterised through an extensive list of traits (Plant Variety Protection Office at Ministry of Agriculture, Foresty and Fisheries, 2011).However, some of these traits are only applied to certain circumstances, such as cultivar identification for protection, and from a breeding perspective, only a subset of these traits is considered as breeding targets.A standardised phenotyping protocol for strawberries was described in Mathey et al. (2013).The protocol outlines the different traits that were assessed on 890 genotypes of strawberries, with germplasm from multiple institutions worldwide, at different locations in the USA including the scoring criteria for these traits and outlines typical values found for the traits as a crop reference set.However, most of the criteria defined there can be applied to programs throughout the world.As external traits are the initial focus in selection, and also because non-destructive, remotely sensed techniques can be used for their estimation, of the 37 traits described in Mathey et al. (2013), only those 18 external traits related to the above-ground structure/ morphology are considered in terms of automation status.The definition of these traits, how they have traditionally been assessed, their automation status, breeding importance and their potential for automation are conveyed in Table 1, which is adapted and extended from Mathey et al. (2013).Very few of these traits have been automated, namely, only the external colour of the fruit, the estimated total yield per plant, the presence/absence of flowers, assessment of the fruit shape and measurement of the calyx size.
Plant productivity, or estimated yield per plant, is a crucial trait in breeding selection (Diamanti et al., 2011), and this measure of plant productivity is assessed through rough fruit and flower counts to place the estimated yield on a productivity scale.As fruit and flower counts are also vital for yield forecasting applications, the precise detection and counting of these components have been a research focus for high-throughput methods, with around half of the literature pertaining to automation of phenotypic traits centring around this.In image-based phenotyping, CNNs have been used to address fruit (Chen et al., 2019;Fan et al., 2022;Ilyas et al., 2021;Kerfs et al., 2017;Kim et al., 2020;Kirk et al., 2020;Lamb & Chuah, 2018;Yu et al., 2019;Zhang et al., 2022;Zhou et al., 2020) and flower (Heylen et al., 2021;Lin & Chen, 2018) detection in real-world agricultural conditions, as they offer greater robustness to the varying environmental conditions experienced than traditional machine learning methods that use manually defined features.However, a challenge for detection in real-world environments is occlusion, which can be minimised by selecting an appropriate viewpoint from which to collect data so as to maximise the prominence of the organ of interest in the image.For example, aerial imagery, or the use of both aerial and lateral perspectives, as in Kerfs et al. (2017), has been utilised in flower detection studies, as flowers are most visible from above the canopy.
A consideration when using CNNs for high-throughput phenotyping is how to balance the trade-off between accuracy (for example, using Faster R-CNN type networks (Chen et al., 2019;Lin & Chen, 2018;Zhou et al., 2020) and efficiency (such as the YOLO family of architectures (Fan et al., 2022;Kim et al., 2020;Zhang et al., 2022)) if realtime performance is of relevance in the system.The automation of fruit and flower counts is beneficial to the selection process as the productivity of genotypes can be assessed, allowing breeders to immediately disregard those that do not meet the required level and thus increasing the selection process efficiency.
Customer perception of fruit quality is associated with traits such as fruit size, colour, gloss, uniformity and skin toughness (Witaker et al., 2011).The external colour of a strawberry is additionally linked to the maturity of the fruit and is one of the key visual cues that attract consumers and help purchasing decisions; too light and the fruit does not look ripe, too dark and it is considered over-ripe.In the traditional approach described in Mathey et al. (2013), colour is scored according to a qualitative visual estimation of the colour in the range from white to dark red in nine steps.Expert knowledge is required to ensure repeatable colour classification and different genotypes may display different colours at full ripeness, with specific colours and shades appealing to consumers in different geographies.Automated methods approach this differently, removing subjectivity by numerically quantifying the colour through the calculation of the range of pixel values or their mean in different colour spaces, such as HSV (He et al., 2017) and CIELab (Liming & Yanchao, 2010;Zingaretti et al., 2021).
Since colour is also an indicator of potential maturity, regular automated assessment of seedlings or clonally propagated material can assist in filtering these in selection trials to focus on those which have potentially ripe fruit or to determine when it would be most beneficial for breeders to perform selections, such as when most plants have available fruit.
Quantified colour would further allow breeders to filter out any genotypes that do not meet the colour requirements for the selection target.
Fruit shape is another important trait for varietal success in the marketplace, with different shaped berries favoured by different cultural groups (Prescott & Bell, 1995), and it is thus important that selections meet the shape target for a particular market.This in turn results in breeders favouring particular shapes, such as oblate, globose, cordate, wedge, short-conic, long-conic or wedge (Jamieson, 2017).Furthermore, fruit shape is an important consideration for punnet packing and, along with fruit size variation and resistance to bruising, contributes greatly to the cost of packing activity (Herrington et al., 2012).Traditionally, shape has been classified ordinally in nine categories, and such categorisation has also been applied in automation studies.Shape categorisation through automated methods has been achieved by first extracting manually defined shape descriptors from the data (described hereafter) and supplying these to a machine learning algorithm for classification.These shape descriptors, extracted from images, can be simple measures of distance such as fruit length and width (Ishikawa et al., 2018;Rey-Serra et al., 2021), relate to the outline of the berry (Ishikawa et al., 2018), points around the perimeter of the fruit (Zingaretti et al., 2021), pixel-based descriptors, or a combination of one or more of these descriptor types (Feldmann et al., 2020).Shape classification is then performed either by a supervised machine learning algorithm (such as Random Forest) into a defined set of classes (Ishikawa et al., 2018) or using an unsupervised clustering algorithm (such as K-means) to find natural groupings within the data (Feldmann et al., 2020;Liming & Yanchao, 2010;Zingaretti et al., 2021).Automation of shape estimation can be used to highlight genotypes having a shape of interest, or disregard those that do not fall within a threshold of desired shape measurements, defined by the shape descriptors.
However, research around automated shape assessment has thus far been primarily focussed on fruit quality applications, where individual picked berries are imaged or scanned at high resolution in laboratory conditions, with un-occluded viewpoints.Such conditions lend themselves to 3D representation, as multiple unoccluded viewpoints are possible, allowing for a full representation of the fruit (Feldmann & Tabb, 2022;He et al., 2017;Le Louëdec & Cielniak, 2020;Li, Cockerton et al., 2020).To be practically useful to breeding and selection, however, shape needs to be assessed in real agricultural environments, for which there is potential although further research into this is needed (Le Louëdec & Cielniak, 2021a).
In addition to fruit size, shape and colour, the perception of fruit quality is also inferred from traits such as calyx size (He et al., 2017).
Traditionally, this trait has been assessed ordinally in relation to the fruit width.Automated methods developed to date allow for a quantitative measure of the maximum calyx dimension in laboratory-sourced 3D data (He et al., 2017), and automation of this trait still needs to be investigated in real-world conditions.While automated assessment of the calyx size on its own does not contribute significantly to increased throughput in selection, if combined with other traits, this could be used to identify genotypes of potential interest.
Considering each of the traits automated to date, automation can be seen to increase the quantitative nature of the phenotypic data collected, replacing categories with rankable values that ensure consistency across all selections, and allowing for quantitative descriptions of common traits in families of seedlings or clonally propagated material to be developed.Furthermore, high-throughput assessment across the potentially thousands of genotypes could allow for breeders to perform pre-selections, disregarding those that do not meet the criteria, so as to focus manual assessment on a smaller selection of those which are of greater potential importance.
To be of practical value for breeding, methods must be developed that are applicable in a real-world environment under a range of different environmental conditions; however, data collected for research into the automated phenotypic analysis of strawberries has been sourced from both laboratory and real-world conditions.The controlled conditions associated with laboratory-based phenotyping allow for reproducible results, but often, these cannot be directly translated to field applications due to the inherent variability in such environments (Araus & Carins, 2014), which has hampered the application of this research to real-world scenarios.

| POTENTIAL FOR AUTOMATION OF TRAITS AND IMPORTANCE TO STANDARD BREEDING TARGETS
Strawberry breeding requires the determination of many agronomic traits affecting the profitability of a final variety, followed by a ranking and weighting of the relative importance of those traits to the propagator, grower, retailer and consumer, based on the relative contribution of that trait to the economic success of strawberry sales.The breeder must then observe as many of these traits as possible within each plant during the growing season.Traits are observed as the breeder passes the plants in the field and following this rapid evaluation of traits and the mental calculation of their relative weightings, selection decisions are made and seedlings are either progressed to subsequent stages of evaluation or discarded.
All of the traits automated to date relate to the fruit and phenology of the plant.This is perhaps unsurprising, given that the fruit is the desired product from the plant and granular data pertaining to it is important for maturity estimation, yield forecasting, quality assessment and robotic harvesting applications (Zheng et al., 2021).Furthermore, traits describing the fruit may also be considered 'easy' traits for automated computation, as the fruit is less complex in structure than, for example, the canopy.However, in breeding programmes, phenological traits are not the only traits of interest, and there is a need for the automation of many other economically-important traits relating to the plant and non-fruit components (such as the number of runners or peduncle length).There are many traits for which automated phenotyping would be of significant value to the breeding and selection process, and the potential for this, along with the breeding importance, has been assigned a ranking out of 5 for both the breeding importance and automation potential (with 1 being of low importance/potential and 5 indicating high) in Table 1, with a justification for the assigned scores.A subset of this information is visually depicted in Figure 3 for ease of comparison, showing each trait as a function of its automation potential and breeding importance, with cases in the top-right indicating high importance to breeders and being most suited for automation.
Automation potential depends on a number of factors.Traits pertaining to the canopy structure, such as vigour, appearance, canopy density and volume, are important because they impact the ability of pickers to access the available ripe fruit on the crop, as well as impair air movement, which can lead to fungal disease infection, and affect disease control by impacting chemical spray penetration within the plant canopy (Sharpe et al., 2018).Such traits were traditionally characterised in a way that permits rapid assessment by humans using subjective classification, and, as such, traits like this are not ideally suited to automation as they stand.However, subjective traits like these could potentially be replaced by one or more automated quantitative traits that could provide a quantitative measure of the same Of the research articles focussing on the automation of these traits (listed in Table 1), only one (vigour) offers holistic assessment, with the remainder split between component (10/18) and sub-component for apples (Häni et al., 2020), blueberries (Patrick & Li, 2017), grapes (Rose et al., 2016) and tomatoes (Masuda, 2021), and potential lessons for strawberry phenotyping can be learnt from automated phenotyping of these fruit crops.High-throughput phenotyping research in viticulture has indicated that using 3D information enables the collection of more complete data through the reduction of occlusion and suggests that features based on both the colour and geometry of the scene are necessary for good reconstruction in 3D space (Rose et al., 2016).The problem of information loss due to the occlusion of berries by leaves also has the potential to be addressed through the use of generative adversarial networks to predict the hidden scenario (Kierdorf et al., 2022).Segmentation of plants into organs is an essential step prior to the extraction of phenotypic traits.Deep learning methods such as PointNet++ variants have shown promise for this in the context of tomato plants, performing 3D semantic segmentation of synthetically generated plants without prior knowledge of species and sensor setup (Heiwolt et al., 2021).In comparison to cereal crops, such as wheat, both the complex growth habit of the strawberry plants, as well as their relatively small size, increase the phenotyping complexity (Zheng et al., 2021).Developments in sensing technology and processing techniques are thus needed to address the highlighted challenges, targeting those traits most important to breeding, as only a very small fraction of these traits have been automated to date.
Some studies have computed new phenotypes using automated methods.These include traits such as achene number (Le Louëdec & Cielniak, 2021b), fruit volume (He et al., 2017) and canopy height (Abd-Elrahman et al., 2020), which are highly quantitative and have the potential to be useful for breeding applications, but would traditionally have been too costly and time-consuming to measure by hand.Features of the canopy, such as how the leaves are presented and the relationship between flowers and canopy, also have the potential to be important for novel breeding targets, such as suitability for robotic harvest.There is also the potential for the development of latent phenotypes that may more fully capture the underlying variations between individuals (Feldmann & Tabb, 2022), beyond the restriction of these traits to a few human-defined characteristics.
Looking forward, those traits that are as yet unautomated and have high importance for breeders should be the focus of future research.As high-throughput phenotyping for strawberries develops, relevance to breeding programs must be considered, to avoid the development of methods that are not relevant to the end-users (described in Lobet (2017)).To this end, a standardised phenotyping protocol or toolbox for automated phenotyping is needed to ensure that as we move into the new age of phenotyping using data-driven methods, phenotypic standards remain consistent between breeding programmes.The need for a unified approach has been alluded to in previous reviews such as Li, Guo et al. (2020) and Zhao et al. (2019), which highlight the need for a phenotyping database.Furthermore, to facilitate the uptake of automated phenotypes, it is essential that automated methods are well documented and presented in a way that allow breeders or domain experts without programming backgrounds to make use of them (Danilevicz et al., 2021).
High-throughput phenotyping, if deployed as part of a robotic sensor carrying system, has the potential to transform the strawberry breeding industry, assisting breeders by allowing for the rapid computation of a wide range of phenotypes at a level of granularity and temporal resolution not possible using manual methods.This will yield greater insights into individual plants within families and reduce the possibility of important features of individuals being missed due to the time constraints associated with manual phenotyping.

| CONCLUSIONS
In this review, we have sought to highlight the current status of automation for external morphological traits of strawberry plants.Across 23 identified sources found on the automation of these traits, only five of the 18 morphological traits listed in a standard phenotyping protocol had been automated, with all of those automated traits related to fruit or phenology.There is a real need for research into the automated calculation of phenotypic traits to address this gap in the automation of other economically important phenotypic traits for breeding.
The potential for automation of the assessed external traits varies depending on whether a holistic, component or sub-component view of the plant or its organs is taken and challenges relating to phenotyping in real agricultural conditions need to be overcome to achieve this.Both this potential and breeding importance were assessed for those traits for which research into high-throughput evaluation has not yet been conducted.Traits scoring highly in both these categories are candidates which would most readily contribute to increased efficiency in the selection process and these were identified to be truss complexity and peduncle length.

|
Phenotyping in the strawberry industry quantitatively evaluate individual plants would permit highthroughput observations to be made independently of human personnel within a breeding program in any given season.Quantitative data, captured multiple times per season, could indicate which families, or individual genotypes, should be singled out for further, more thorough evaluation, thus significantly increasing the selection intensity and accuracy of the breeding process.
cloud representation of a strawberry, reconstructed using data from He et al. (2017) The traits examined in Mathey et al. (2013) fall into the categories: (1) phenology and flower related, (2) plant characteristics, (3) external fruit characteristics, (4) internal fruit characteristics and (5) fruit chemistry and weight.When breeders conduct an initial assessment of plants, the external traits of the plants are evaluated first, and as they are limited by time constraints when assessing breeding populations of thousands of plants, most traits are scored to fall into ordinal categories.Although these provide some idea of scale or ranking for comparison, numerical quantification could provide better decision support through the delivery of more precise and detailed information to breeders, and automation has the potential to facilitate a shift in these assessments from being qualitative to quantitative.

F
I G U R E 2 Common tasks in machine vision are classification, detection and segmentation, in which a label is assigned to an image, portion thereof, or pixel.Each of these is illustrated here: (a) classification requires no localisation (i.e., identification of an object's position with an image), only an indication as to which of a set of classes the object in the image belongs to; (b) detection is a combination of classification and localisation, where the identified object is localised by identifying the object's bounding box; (c) segmentation (in this case, instance segmentation) goes a step further than detection, where individual pixels of each object are assigned classes T A B L E 1 External traits describing the fruit from Mathey et al. (2013) (which describes the standard phenotyping protocol for strawberries) and how these traits are traditionally scored Ordinal in the range 1:9 (dead: extremely vigorous) 3 -Plants need to be sufficiently vigorous to be healthy, but reduced vigour is desirable in protected systems, allowing for better spray penetration (hence pest and disease control) and better display of flowers and control) and better display of flowers and fruit for pollination and picking 1 -Classification, but subjective Crop estimate A measure of the amount of fruit on the plant Ordinal in the range 1:9 (no fruit: over-cropped) 5 -Yield potential is the key agronomic trait which makes the production of the crop profitable Automated -Kerfs et al. (2017), Lamb and Chuah (2018), Chen et al. (2019), Yu et al. (2019peduncle length allows flowers and berries to be well displayed.For berries under protected culture this can have a positive impact on picking efficiency and can reduce disease3 -Detection and measurement, some may be hidden within the held above the leaf canopy in protected cultures are better available for pollination and less likely to be affected by botrytis due to increased airflow and spray fruit is classed as unmarketable in most commercial situations and is therefore unsaleable, reducing profitable production to the grower.Increasing the proportion of marketable fruit from a cultivar is therefore advantageous, and this can be achieved by even fruit set and even fruit achenes give the impression of seediness (associated with storage) to the consumer, so even or protruding/raised achenes are desirable to avoid this and furthermore offer some protection to the skin during transit 2 -Detection may be difficult due to extremely small size of subpreferences in different markets incorporates how the calyx presents, although a raised calyx allows for better airflow and hence reduces the potential for disease4 -ClassificationNote: Traits that have already been the focus of automation research are not assigned an automation potential score; instead, articles focussed on the automation of the trait are listed.
component-for example, both canopy height and canopy volume allude to vigour.Subjective traits thus have low direct automation potential, falling towards the bottom of Figure 3. Computation of the phenotype requires assessment of the plant at a holistic (the whole plant), component (whole organs such as fruit, leaves and runners) or sub-component (e.g., achenes or the calyx) level.

( 7 /
18).In the case of traits at the component or sub-component level, detection or segmentation of the organ from the rest of the plant is a necessary prerequisite step.Subsequent steps would then involve classification, detection of other organs to facilitate the computation of the relative positioning, counts or measurement of an organ.Furthermore, small sub-component size increases the complexity of phenotyping, as the resolution of the image has the potential to pose a challenge.Thus, automation potential is affected by the number of steps needed to obtain the phenotype and by the size of the organ or component.The challenges posed by real-world environments, highlighted in Section 3, are also particularly pertinent for tasks where an unoccluded view is necessary, such as measurement of length, or for traits requiring the detection of multiple components.Considering Figure3, traits extracted from small sub-components are grouped as having lower automation potential than those from whole components.Considering those unautomated traits that are both important for breeding but are also amenable to automation, truss complexity (number of flowers per truss) and peduncle length are identified as prime targets for further research and development effort.Both traits impact the display of the berries (fruit) and are of importance to efficient picking (by humans and potentially by robots), with the number of flowers per truss additionally affecting berry size(Heide et al., 2013).These are thus essential from an economic perspective, so these traits are important breeding targets.In terms of automation, truss complexity requires detection and subsequent counting, while peduncle length requires quantification-the complexity for both of which is relatively low, provided there is minimal occlusion.Characterising genotypes automatically in terms of these important traits, in addition to those already automated, would reduce the effort required to identify individual genotypes or families of interest for closer inspection, thus increasing the efficiency of the selection process for strawberries.5| CURRENT STATUS, CHALLENGES AND PROSPECTS OF AUTOMATED STRAWBERRY PHENOTYPING IN AGRICULTURAL ENVIRONMENTSImage-based analysis has the potential to facilitate the highthroughput computation of phenotypic traits.The resulting higher level of quantification would lead to an increase in detail and precision of data available to breeders, which would result in more effective decision-making in the selection process.Automated image-based phenotyping is already widely used for a variety of crops at different spatial scales(Li, Cockerton et al., 2020), spanning the analysis of individual organs and plants within controlled laboratory conditions using fixed sensing platforms, through to field conditions where phenotyping is conducted using mobile ground-based or aerial vehicles carrying sensing payloads(Yang et al., 2020).F I G U R E 3The relative breeding importance and automation potential of the morphological traits of strawberries Translating methods from controlled to real-world environments remains a challenge(Araus & Carins, 2014;Li, Guo et al., 2020), but it is the only way that the potential of such technologies can be fully realised in breeding and selection.The challenges posed by real agricultural environments to strawberry phenotyping, highlighted in Section 3, are common to image-based phenotyping of other fruit crops and involve choices between the data source and representation type (2D or 3D), localisation of organs through detection or segmentation of the plant into organs and subsequent extraction of phenotypic parameters.Examples of such pipelines are demonstrated Automation will permit the collection of quantitative phenotypic data for traits of agronomic importance at an unprecedented level in the strawberry industry.A system providing automation such as this would allow breeding programmes to increase their selection intensity and address the phenotyping bottleneck that currently exists due to time limitations on domain experts, allowing breeders to spend more time focusing on genotypes of maximum interest.As a result, high-throughput phenotyping has the potential to transform the selection process in strawberry breeding, providing richer information for decision support.ACKNOWLEDGEMENTThis work was partially funded by the Collaborative Training Partnership for Fruit Crop Research.