Abstract
Machine learning and deep learning are subsets of Artificial Intelligence that have revolutionized object detection and classification in images or videos. This technology plays a crucial role in facilitating the transition from conventional to precision agriculture, particularly in the context of weed control. Precision agriculture, which previously relied on manual efforts, has now embraced the use of smart devices for more efficient weed detection. However, several challenges are associated with weed detection, including the visual similarity between weed and crop, occlusion and lighting effects, as well as the need for early-stage weed control. Therefore, this study aimed to provide a comprehensive review of the application of both traditional machine learning and deep learning, as well as the combination of the two methods, for weed detection across different crop fields. The results of this review show the advantages and disadvantages of using machine learning and deep learning. Generally, deep learning produced superior accuracy compared to machine learning under various conditions. Machine learning required the selection of the right combination of features to achieve high accuracy in classifying weed and crop, particularly under conditions consisting of lighting and early growth effects. Moreover, a precise segmentation stage would be required in cases of occlusion. Machine learning had the advantage of achieving real-time processing by producing smaller models than deep learning, thereby eliminating the need for additional GPUs. However, the development of GPU technology is currently rapid, so researchers are more often using deep learning for more accurate weed identification.
1. Introduction
The global population is steadily growing each year, and according to world statistical data of United Nations Department of Economic and Social Affairs, Population Division, 2022, it is projected to increase above 8 billion in 2022. This increase in population presents several challenges, particularly in the aspect of food resource availability. Furthermore, the demand for daily food ingredients is projected to increase (Talaviya et al., 2020), as more individuals require sustenance, water, and other agricultural resources to fulfill nutritional requirements. Consequently, the agricultural sector is required to address the daily needs of the growing population, which is also a primary source of income for farmers. However, farmers encounter difficulties in producing agricultural products due to the diminishing agricultural areas caused by the rise in residential constructions. Farmers are also faced with the dual challenge of ensuring the quality of crop and increasing production. A significant challenge in achieving these goals is the presence of weed, which compete with surrounding plants (Berquer et al., 2023; Nathalie et al., 2020; Sabzi et al., 2020).
Weed grows rapidly in unwanted locations, such as gardens, fields, and agricultural areas. Weed absorb essential nutrients including sunlight, water, and soil space from agricultural plants (Kubiak et al., 2022; Raja et al., 2020a). Consequently, farmers need to adopt various methods to eradicate weed around crop. For example, farmers can resort to using pesticides (Monteiro and Santos, 2022) or manually cutting the weed with tools (Scavo and Mauromicale, 2020). Both approaches have drawbacks, as pesticides may adversely affect crop when not accurately targeted, leading to residues and wastage (Kalyabina et al., 2021). Meanwhile, manual cutting consumes farmers’ time and energy, resulting in inefficiency. The recognition of these challenges causes studies to actively explore alternative methods or technologies for weed control (Liu and Bruch, 2020). Integrating technology into weed removal processes can aid farmers in reducing manual workloads, enhancing efficiency, and achieving better crop yields.
Aside from enhancing efficiency, the utilization of automation for weed removal brings about additional advantages, including cost-effectiveness and environmental friendliness. An example is the ability of farmers to conserve labor while effectively clearing weed (Rajmis et al., 2022). However, it is crucial to be aware that inaccurate pesticide application can lead to environmental pollution (Zaller et al., 2022). The environmental pollution of pesticides arises from their chemical properties, which contaminate water, soil, air, and the surrounding ecosystem. Consequently, the development of an automated weed-spraying system necessitates a meticulous classification process for both weed and crop (Liu and Bruch, 2020). On the other hand, accurate weed and crop categorization present unique challenges due to the uniform green color of leaves, which is further complicated by variations in color caused by lighting conditions (Kazerouni et al., 2019). The growth of weed often matches with that of crop, resulting in occlusion between leaves (L. Zhang et al., 2022b). Another factor is the similarity in size between the leaves of small crop and weed (Chen et al., 2020). Fig. 1 shows the frequent challenges faced when classifying weed and crop.
Numerous previous studies have explored the classification of weed and crop. An example is the works of Hasan et al., 2021 and Al-Badri et al., 2022b, which reviewed the application of deep learning and traditional machine learning for classifying weed. Both machine learning and deep learning were instrumental in efficiently and accurately identifying weed and crop. These technologies minimize human effort and improve crop management, particularly in the context of identifying weed and crop (Vasileiou et al., 2023). Typically, the main difference between machine learning and deep learning lies in the model complexity (Shah and Bhavsar, 2022; Siemers et al., 2022). Machine learning focuses on handcrafting algorithms and statistical models that enable computers to learn from data (Moshawrab et al., 2023), thereby facilitating predictions or decisions based on discerned patterns (Pandey et al., 2023). Meanwhile, deep learning mirrors the structure of the human brain (Bai et al., 2021), with interconnected neurons arranged in layers. This approach allows the model to learn automatically from the data by extracting features hierarchically (Banan et al., 2020) and producing an increasingly complex representation as the depth of the neural network layer increases (Samanta et al., 2023). Machine learning excels in simpler tasks with a limited number of features and uses less complex mathematical calculations, making it suitable for Internet of Things (IoT) devices (Phasinam and Kassanuk, 2022; Phasinam et al., 2022). On the other hand, deep learning is more effective in handling complex, data-intensive tasks, particularly those incorporating visual pattern recognition.
Previous survey papers have explored individual aspects of machine learning (Venkataraju et al., 2023; A. Wang et al., 2019; Z. Wu et al., 2021a) or deep learning (Hasan et al., 2021; K. Hu et al., 2024; Murad et al., 2023), but none have offered a comprehensive discussion of both in the context of this specific case study. Therefore, this review aimed to bridge the existing knowledge gap by conducting a comprehensive analysis of the various phases involved in machine learning and deep learning. It will include an in-depth exploration of the pre-processing, segmentation, feature extraction, and classification stages. The initial stage consists of pre-processing, where the dataset is prepared for subsequent processing. Following this process, segmentation is performed by separating leaf objects from the surrounding background, including soil and other elements. Subsequently, feature extraction is carried out to identify common features within the leaves. The final stage is classification, where weed and crop are categorized. At the end of this review, three challenges related to weed and crop detection are addressed, namely 1) the impact of lighting effects, 2) occlusion conditions, and 3) classification during the early growth phase. Additionally, an analysis is conducted on accuracy values and processing times, particularly in real-time applications. Implementations have been tested on Raspberries, computers (laptops), and robots. The primary goal of this review is to assist readers in navigating the optimal methods, particularly when faced with the complex challenges caused by classification and real-time processing under three specific conditions.
Based on the arrangement of this study, Section 2 describes the necessity of reviewing the classification of weed and crop. Furthermore, Section 3 explores weed detection and its categorization methods. Section 4 discusses the classification of weed and crop using machine learning. Section 5 explains deep learning applications for classifying weed and crop. Section 6 addresses the utilization of hybrid convolutional networks and machine learning methods in classification case studies. Section 7 also benchmarks the dataset to measure accuracy and speed. Section 8 contains future work and discussions, which explains problems commonly encountered during investigation. Finally, Section 9 concludes by providing insights into methods suitable for the classification process of weed and crop.
2. Related surveys
Several studies extensively used machine learning and deep learning for the detection and identification of weeds and crops, primarily focusing on weed control. Additionally, several review articles have delved into the utilization of these technologies, focusing particularly on either maximizing accuracy or enhancing the efficiency of processing speeds. In 2021, Hasan et al., 2021 conducted a comprehensive survey of deep learning methods for weed detection from image data, analyzing 70 papers across data acquisition, pre-processing, detection and classification, and evaluation. The result showed that fine-tuning deep learning models had very high classification accuracy, but a limitation was recorded in the form of computational speed. Another survey by Murad et al., 2023 explored the use of deep learning for weed detection, primarily using RGB-format datasets. To achieve optimal accuracy, Convolutional Neural Network (CNN) architectures were modified, but the survey showed extended training times, necessitating GPU devices. Additionally, K. Hu et al., 2024 contributed to the discourse by reviewing the use of deep learning, focusing on datasets, evaluation methods, and weed recognition techniques. The survey identified occlusion as a challenge in weed detection. Despite the high accuracy of deep learning, the architecture introduced complexities during training and deployment.
Machine learning methods for classifying weed and crop were also carried out by Venkataraju et al., 2023, which reviewed 35 papers on weed detection in corn plants. The frequently used method in this context was Support Vector Machine (SVM) and Neural Network. Similarly, Z. Wu et al., 2021a explored machine learning methods for computer vision-based weed detection, comparing features such as texture, shape, color, and spectral characteristics. The combination of these features was instrumental in achieving accurate results. A. Wang et al., 2019 also conducted a comprehensive analysis of machine vision techniques enhanced by machine learning, including steps such as pre-processing, segmentation, feature extraction, and classification. Their research delved into four main categories of features, namely biological morphology, spectral characteristics, visual textures, and spatial context. This integration of machine learning with image processing demonstrated promising capabilities for accurate and real-time identification of weeds and crops.
Despite several previous survey papers, a comprehensive review addressing the strengths and weaknesses of both machine learning and deep learning in the context of weed detection is lacking. Furthermore, an in-depth discussion on pre-processing stages, segmentation, feature extraction, and classification for weed and crop is presented. The exploration extends to common challenges in weed and agricultural plant detection, comprising the impact of light effects, occlusion conditions on leaves, the early-growth plant phase, and real-time processing for future field applications.
3. Weed control and categorization method
Weed consists of various forms, generally categorized as narrow-leaf and broad-leaf as shown in Fig. 2. The economic impact of weed is significant, leading to reduced crop yields (Kubiak et al., 2022), increased production costs (Sharma et al., 2021), and reduced crop quality (Monteiro and Santos, 2022). Consequently, effective weed control is an essential component in agriculture to sustain crop productivity and profitability. Currently, two predominant trends correspond to distinct methodologies for weed control in agricultural systems. While synthetic herbicides are prevalent in the conventional approach (Frimpong et al., 2018), the modern approach leans towards using mechanical, precision agriculture, and sensor-based strategies (Machleb et al., 2020).
Conventional weed control comprises manual labor, cultural practices, and herbicide application. Specifically, manual labor consists of the physical removal of weed by hand (Woyessa et al., 2022), a labor-intensive yet effective method suitable for small-scale farming. Cultural practices such as crop rotation, tillage, and mulching disrupt weed growth and prevent seed germination. Additionally, herbicides play a key role in conventional weed control with chemical substances applied to the soil or plant foliage for weed control (Strehlow et al., 2020). Herbicides can be selective, targeting only specific weed types, or non-selective, which affects all plants. While herbicides have revolutionized weed management, enabling large-scale control and increasing crop yields, overreliance can result in herbicide-resistant weed (Gaines et al., 2020), environmental pollution (S. Zhang et al., 2021), and health hazards for farmers and farmworkers (Shammi et al., 2020).
Modern weed control combines traditional methods with innovative technologies such as precision agriculture (Kent Shannon et al., 2018), remote sensing (Farooq et al., 2018b), and robotics (Raja et al., 2020b). Precision agriculture incorporates the use of sensors, GPS, and computer vision for site-specific weed management, reducing herbicide usage and enhancing crop yields. Meanwhile, remote sensing technologies, including satellite imagery (Rasmussen et al., 2021) and Unmanned Aerial Vehicles (UAVs) (Mohidem et al., 2021), detect and map weed infestations, which enables targeted control measures. Robotics, particularly autonomous robots equipped with computer vision and machine learning algorithms, offer real-time weed detection and removal, reducing reliance on manual labor and herbicides. It’s essential to recognize the transformative impact that these technologies may have on weed control. They offer a pathway towards a more efficient, sustainable, and eco-friendly method of controlling weeds.
Modern weed control is closely associated with Artificial Intelligence (W.-H. Su, 2020; Subeesh and Mehta, 2021). According to Fig. 3, weed detection is categorized into traditional machine learning, deep learning, and a combined approach. Machine learning techniques are categorized into three main groups, namely supervised learning, unsupervised learning, and semi-supervised learning. Supervised learning is labeled training and validation datasets (Chegini et al., 2019), where both the image and its accompanying labels are part of the input dataset for the machine learning model. However, unsupervised learning operates with an unlabeled training set, and the input dataset for the unsupervised model lacks annotations (J. Wang and Biljecki, 2022). This method enables the model to map the input to a specific output. Meanwhile, Semi-supervised learning occupies a middle ground as it is positioned between supervised and unsupervised learning (Chang et al., 2020). The utilization of traditional machine learning methods in weed and crop classification offers several advantages. Typically, traditional machine learning requires only a limited number of datasets for training, resulting in a relatively small model size. This compact model facilitates swift recognition of leaf objects and allows for real-time operation. Additionally, the features used in traditional machine learning are not mathematically complex. These characteristics make the method to be well-suited for real-time weed and crop detection (Tufail et al., 2021; Zhao et al., 2022). It is crucial to acknowledge that optimal feature combinations can further enhance accuracy in weed and crop detection (L. Zhang et al., 2022b). Consequently, this review provides examples of features as input to traditional machine learning methods, showing the importance of accurately classifying weed and crop species.
-
Deep learning approaches are further categorized into CNN, Fully Convolutional Networks (FCN), and Hybrid Networks. CNN model typically comprises two fundamental components, namely feature extraction and classification (Indolia et al., 2018). Various studies have implemented numerous CNN models, including VGGNet (Simonyan and Zisserman, 2014), DenseNet (G. Huang et al., 2017), MobileNet (Howard et al., 2017), ResNet (Shafiq and Gu, 2022), etc. These models incorporated diverse feature extraction and classification layers. Compared to CNN, FCN modifies the architecture by substituting the fully connected layers with convolutional ones and employs a transposed convolution layer to generate an output image that matches the input size in dimensions (J. Wu et al., 2021b). Additionally, hybrid architectures have been developed by combining the characteristics of two or more deep learning models (X. Zhang et al., 2020). For example, the hybrid approach combines convolutional feature extraction with machine learning as a classifier. It is essential to be aware that both machine learning and deep learning comprise data acquisition processes.
This review classifies images into three types, namely hyperspectral images (Farooq et al., 2019a; Farooq et al., 2019b; Farooq et al., 2018a; Gao et al., 2018; Y. Li et al., 2021), multispectral images (Farooq et al., 2019a; Liu et al., 2019; Louargant et al., 2018; J. Su et al., 2022), and digital images (Chen et al., 2020; Rahman et al., 2023; S. Zhang et al., 2021). Hyperspectral images are captured using sensors capable of collecting information across adjacent spectral bands. Hyperspectral imaging sensors often use a greater number of narrower bands to capture more specific spectrum information. The entire spectrum in each pixel of hyperspectral images has been applied in various agricultural contexts. However, a multispectral camera is suitable for examining larger areas of interest, as it is more portable and cost-effective with higher spatial resolution. Additionally, the multispectral camera used is equipped with more spectral bands and is less susceptible to environmental changes due to the availability of a reflectance calibration panel. Digital sensors or cameras produce digital images, which can be either color or black-and-white, with spatial resolution contingent on the sensor or camera quality. In general, the choice of image type also influences the input dataset used.
4. Weed detection using machine learning
Weed detection using image or video data leads to classification, which comprises distinguishing between species or differentiating weed from crop. In the context of machine learning, a crucial step is plant segmentation (Kitzler et al., 2022; Sakeef et al., 2023). This stage is very important for separating the background from the plants to be classified (K. Yang et al., 2020). Additionally, the features used for the classification process should be specifically related to the plant area. Land areas and other non-plant objects are typically removed or masked in black. Machine learning methods often use handcrafted features (Ghazal et al., 2021), manually designed and extracted from the object image based on knowledge to extract crucial information. In computer vision, these handcrafted features describe visual characteristics including color, texture, and shape (C. Yang, 2021)., capturing specific visual properties relevant to the case study, such as weed detection. The strengths and weaknesses of the traditional machine learning method are shown in Table 1. Machine learning models commonly follow the process shown in Fig. 4. Generally, machine learning consists of five stages, namely image acquisition, pre-processing, image segmentation, feature extraction, and classification. However, an image segmentation stage typically precedes the image feature extraction stage for enhanced precision (Ahmad et al., 2018). Typically, the segmentation stage is needed for preparing visual data for further analysis. By reducing data complexity, segregating objects from the background, and facilitating more effective feature extraction, segmentation contributes to constructing a focused and relevant data representation of the object under investigation.
