Brazilian scientists are working to develop a technology that allows agricultural machines to automatically recognize plants in the field. Research by Embrapa in the area of phenotyping aims to reconstruct plant species in three dimensions (3D) using computer knowledge and techniques such as robotics and artificial intelligence. The three-dimensional reconstruction of plants involves the automated capture of images of the agricultural crops and the generation of digital models that show the structures of the species, that is, leaves, stems, flowers or fruits, in 3D. Through this process, thousands of data for the classification and analysis of plant traits are collected, which can help breeding and genetic improvement. Equipment that autonomously makes diagnosis The results from this kind of research are useful to estimate the production of a given area, find areas with nutritional deficiency or identify pests and diseases in the plantation, contributing to the progress of precision agriculture. With the development of studies in robotics applied to agriculture, researchers expect that, in the future, autonomous agricultural machines can go into the fields to make the most varied observations. Experiments led with maize and wine grape crops integrate a research project focused on the generation of knowledge in digital agriculture led by Embrapa Agricultural Informatics (SP), in partnership with Embrapa Instrumentation (SP) and the State University of Campinas (Unicamp), cwith funds from the São Paulo State Research Support Foundation (Fapesp). The tests with robots and drones are going to be made in a vineyard in the state of São Paulo, which participates in the Precision Agriculture Network, and in a maize plantation area in Campinas. They are also going to use machine learning and pattern recognition techniques known as deep learning, deep neural networks that can learn complex patterns from a large number of observations. With the support of large databases and digital image processing softwares, the aim is to create robot prototypes that can identify the crops and differentiate fruits, grape bunches or corn ears, from other plant structures, like leaves and stems, for example. Technology used in autonomous cars “This can pave the way for a series of automations in agriculture”, reports the researcher from Embrapa Agricultural Informatics Thiago Teixeira Santos, leader of the research. The team plans to build a robot with connected cameras and a laser scanner to sweep the crop areas selected by the research. Thus it will be possible to see the three-dimensional structure with geolocation information based on Lidar technology - the same one used by autonomous cars that are being tested by the world's automobile industry. However, unlike the industry, where the environment is controlled, the robots designed to work in agribusiness face a much more complex environment and are subject to uncertainties, which requires a large investigation effort and countless simulations. The challenges range from overcoming land slope to climate factors and the need for high performance computer infrastructure for storage, processing and analysis. Therefore, the tests are being made in small cultivation parcels, with known characteristics and structures like defined plantation lines, so that the robots are trained and can recognize these environments. “The next generation of agricultural equipment will include small machines and robots that perform specific tasks. It is machinery that sees and makes decisions, that is, it is endowed with the capacity 'to reason' based on what it observes in the field”, Santos evaluates. The research project called “Ambient awareness in Agriculture: 3-D structure and reasoning in the crop field (AAcr3), was approved in a joint call by Fapesp and IBM and received US$ 60,000 in funds from the line of research aid Partnership for Technological Innovation (Pite). It is due to last two years, from April 2018 to March 2020. Computer vision and machine learning algorithms will be used to detect and classify objects of interest, such as land, plants, leaves and fruits. Moreover, information such as plants traits, spatial variation in the cultivation and other measurements will be estimated from 3-D point clouds. The researchers explain that parcels of three different crops, including grains and fruit, will be sensed and structured, capturing several stages of plant development. “The use of such tools and procedures for the recognition of the parts of a plant species that interests the farmer or technician will enable the automated obtainment of useful information, such as estimated production in an area, which parts of the area that can be more or less productive, pest and disease incidence levels in the plants in such area, among others”, undercores the researcher from Embrapa Instrumentation Luis Henrique Bassoi. “This could expedite data collection and analysis to obtain information that will inform decision-making regarding agricultural practices”, he affirms. The study is also going to integrate cutting-edge imaging, robotics and computer vision technologies within a full methodology for the acquisition of the 3-D structure for crop fields, approaching problems in terms of automation and high performance computing. It will also develop machine learning-based methods for the extraction of patterns and characteristics from such data, for comparison with methodologies traditionally used in agricultural research, opening a new field of scientific knowledge. Translation: Mariana Medeiros