Vegebot Harvests Lettuce Using Machine Learning

A University of Cambridge team developed the “Vegebot,” which was primarily trained to identify and harvest iceberg lettuce in a lab setting. It has recently been successfully tested in different field conditions in collaboration with G’s Growers, a local fruit and vegetable co-operative.

Machine Learning" />

Even though the prototype is not as quick or efficient as a human worker, it shows how using robotics in agriculture could be expanded, even for crops such as iceberg lettuce which are especially challenging to harvest mechanically. The results have been reported in The Journal of Field Robotics.

Crops such as wheat and potatoes have been harvested mechanically at scale for several years, but a number of other crops have so far resisted automation. Iceberg lettuce is a typical example. Despite the fact that it is the most widely grown type of lettuce in the United Kingdom, the iceberg is easily damaged and grows comparatively flat to the ground, posing a difficulty for robotic harvesters.

Every field is different, every lettuce is different. But if we can make a robotic harvester work with iceberg lettuce, we could also make it work with many other crops.

Simon Birrell, Study Co-Author, Department of Engineering, University of Cambridge

“At the moment, harvesting is the only part of the lettuce life cycle that is done manually, and it’s very physically demanding,” stated co-author Julia Cai, who worked on the computer vision components of the Vegebot while she was an undergraduate student in the lab of Dr Fumiya Iida.

First, the Vegebot identifies the “target” crop within its field of vision. It then establishes whether a specific lettuce is healthy and mature enough to be harvested. Finally, it cuts the lettuce from the rest of the plant without damaging it so that it is “supermarket ready.”

For a human, the entire process takes a couple of seconds, but it’s a really challenging problem for a robot.

Josie Hughes, Study Co-Author, University of Cambridge

The Vegebot has two key parts: a cutting system and a computer vision system. The overhead camera on the Vegebot captures an image of the lettuce field and initially identifies all the lettuces in the image. Then for each lettuce, it categorizes whether or not it should be harvested. A lettuce might be excluded if it is not yet fully developed, or if it has a disease that could infect other lettuces in the harvest.

The scientists created and trained a machine-learning algorithm on sample images of lettuces. As soon as the Vegebot could identify healthy lettuces in the lab, it was then taught in the field, in a range of weather conditions, on numerous real lettuces.

The Vegebot’s second camera is located close to the cutting blade and helps ensure a clean cut. The scientists could also modify the pressure in the robot’s gripping arm so that it gripped the lettuce secure enough not to drop it, but not very hard as to crush it. The force of the grip can be modified for other crops.

“We wanted to develop approaches that weren’t necessarily specific to iceberg lettuce so that they can be used for other types of above-ground crops,” said Iida, who leads the team behind the study.

Going forward, robotic harvesters could help solve issues with labor scarcities in agriculture and could also help curtail food wastage. At present, each field is normally harvested once, and any unripe fruits or vegetables are thrown away. However, a robotic harvester could be taught to pick just the ripe vegetables, and since it is capable of harvesting nonstop, it could carry out numerous passes on the same field. It can return at a later date to harvest the vegetables that were unripe during earlier passes.

We’re also collecting lots of data about lettuce, which could be used to improve efficiency, such as which fields have the highest yields. We’ve still got to speed our Vegebot up to the point where it could compete with a human, but we think robots have lots of potential in agri-tech.

Josie Hughes, Study Co-Author, University of Cambridge

Iida’s team at Cambridge is also part of the world’s first Centre for Doctoral Training (CDT) in agri-food robotics. In partnership with scientists at the University of Lincoln and the University of East Anglia, the Cambridge scientists will train the next generation of experts in robotics and autonomous systems for application in the agri-tech domain.

The Engineering and Physical Sciences Research Council (EPSRC) has granted £6.6m for the new CDT, which will aid a minimum of 50 PhD students.

(Video credit: University of Cambridge)