AGRICULTURAL PEST DETECTION AND PLANT DISEASE ANALYSIS USING DEEP LEARNING AND GEMINI API

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

AGRICULTURAL PEST DETECTION AND PLANT DISEASE ANALYSIS

USING DEEP LEARNING AND GEMINI API

G. Chenna Reddy¹, Mr. B. Rajasekhar Reddy², D. Tarak Reddy³

¹Student, Dept. of CSE (Data Science), Andhra Loyola Institute of Engineering and Technology, Vijayawada, Andhra Pradesh, India

²Assistant Professor, Dept. of CSE (Data Science), Andhra Loyola Institute of Engineering and Technology, Vijayawada, Andhra Pradesh, India

³Student, Dept. of CSE (Data Science), Andhra Loyola Institute of Engineering and Technology, Vijayawada, Andhra Pradesh, India ***

Abstract: Timely identification of plant diseases and pest infestations remains a critical challenge in modern agriculture, particularly in remote regions where expert consultation is limited. This paper presents a web-based intelligent agricultural diagnostic system that integrates deep learning models with a vision-language AI to deliver real-time, actionable insights from plant images. The proposed system employs EfficientNetB0 for plant disease classification across 38 categories using the PlantVillage dataset, achieving 94% accuracy. Additionally, EfficientNetB3 is fine-tuned for pest detection across 12 classes using a limited dataset of approximately 3,000 images, reaching 92% accuracy through a two-phase transfer learning strategy combined with extensive data augmentation. To enhance practical usability, the system integrates the Gemini Vision API to generate structured diagnostic reports, including disease severity assessment, progression forecasting, treatment recommendations, and pesticide suggestions. Unlike traditional single-task classification systems, the proposed approach provides a comprehensive decision-support solution accessible via a lightweight Flask web application. Experimental results demonstrate that the system achieves robust performance while delivering significantly improved real-world applicabilityanduser-centricoutputs.

Key Words: Deep Learning, EfficientNet, Transfer Learning, Plant Disease Detection, Pest Detection, Gemini Vision API, Precision Agriculture, Convolutional Neural Networks

1. INTRODUCTION

Agriculturecontinuestofaceincreasinguncertaintydueto evolving pest populations and unpredictable disease outbreaks. In many rural environments, farmers lack immediate access to expert guidance, resulting in delayed or incorrect interventions and significant crop loss. While deep learning has demonstrated high accuracy in plant diseaseclassificationtasks,mostexistingsolutionsremain confined to research settings or require specialized applicationsandinfrastructure.

This work addresses these limitations by designing a practical, deployable system that extends beyond classification to provide actionable agricultural guidance. The proposed system integrates two convolutional neural network models with a vision-language API to deliver a completediagnosticpipelinethroughawebinterface.

Theprimarycontributionsofthisworkareasfollows:

• Developmentofadual-modelframeworkforbothplant diseaseclassificationandpestdetection.

• Implementation of a two-phase transfer learning strategytoimproveperformanceonlimiteddatasets.

• Integration of a vision-language model to generate comprehensivediagnosticreports.

• Deployment of a lightweight, accessible web-based applicationrequiringnoinstallationortechnicalexpertise.

Although some prior works report higher classification accuracy under controlled conditions, this system prioritizes real-world usability, multi-task capability, and actionableoutput,whichintroducesadditionalcomplexity beyondsingle-taskmodels.

2. LITERATURE SURVEY

The advancement of deep learning has significantly improved automated plant disease and pest detection. Early methods based on traditional image processing techniques, such as color histograms and texture features with Support Vector Machines (SVM), performed well undercontrolledconditionsbutlackedrobustnessinrealworld agricultural environments due to variations in lighting,occlusion,andcomplexbackgrounds.

The adoption of convolutional neural networks (CNNs) marked a major breakthrough. Sharada P. Mohanty et al. (2016)demonstratedhighaccuracyusingthePlantVillage dataset, though performance dropped significantly when appliedtofieldimages,highlightingadomaingapbetween laboratoryandreal-worldconditions.

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

To improve generalization, transfer learning using pretrained architectures such as VGG16, ResNet50, InceptionV3, and EfficientNet has become standard. Among these, EfficientNet offers an optimal trade-off betweenaccuracyandcomputational efficiency, makingit suitableforreal-worlddeployment.

Pest detection remains less explored due to limited largescale datasets. Efforts using datasets such as IP102 have shown moderate success, often requiring data augmentation and class balancing to improve performance.

Recentstudieshavebegunintegratingvision-languageand large language models to provide descriptive, contextawareoutputsratherthansimpleclassifications.However, mostimplementationsremainattheprototypestage.

Overall, existing research highlights a key limitation: high accuracy on controlled datasets does not translate well to real-world environments. Additionally, most systems focus solely on classification without offering actionable insights. The proposed work addresses these gaps by integrating classification, diagnostic reporting, and practicaldeploymentintoaunifiedframework.

3. PROPOSED SYSTEM

Theproposedsystemisdesignedtoreflectthe real-world workflowofagriculturalusers.Afarmeruploadsanimage through a web interface, and the system processes it through three integrated components: disease classification, pest detection, and diagnostic report generation.

3.1 Plant Disease Classification

EfficientNetB0 is utilized for disease classification due to its optimal balance between accuracy and computational efficiency. A custom classification head is added and trained using a two-phase transfer learning strategy to ensure both stability and adaptability to domain-specific features. In the first phase, only the classification head is trained while the convolutional base remains frozen. In the second phase, the top layers of the base network are selectively unfrozen and fine-tuned with a reduced learning rate. This approach prevents catastrophic forgetting while enabling the model to adapt to agricultural image characteristics. The model achieves 94% validation accuracy on the PlantVillage dataset spanning38diseasecategoriesacross14plantspecies.

3.2 Vision-Language Diagnostic Extension

Toenhancepracticalutilitybeyondstandardclassification outputs, the system integrates the Gemini Vision API. When a disease or pest is identified, the classified image and associated label are submitted to the API, which generates a structured diagnostic report. This report includes disease identification confirmation, severity estimation on a standardized scale, progression analysis, recommendedtreatmentsteps,andsuggestedpesticideor

biological control options. These outputs were evaluated qualitatively based on relevance, clarity, and usefulness for end users, demonstrating meaningfully improved utilitycomparedtolabel-onlysystems.

3.3 Pest Detection

EfficientNetB3isemployedforpestclassificationduetoits higher input resolution capability, which is particularly importantfordetectingsmallorpartiallyoccludedinsects in natural settings. Given the limited dataset size of approximately 3,000 images across 12 pest categories, extensive data augmentation techniques are applied during training. These include random horizontal and vertical flipping, rotation, zoom, brightness and contrast variation, and shear transformation. The same two-phase transferlearningstrategyusedfordiseaseclassificationis applied, yielding 92% classification accuracy on the pest detectiontask.

3.4 System Architecture

The system is implemented using a Flask-based backend thatcoordinatesmodelinferenceandAPIcommunication. Upon image upload, the backend routes the input to both CNN models and retrieves predictions in parallel. The GeminiVisionAPIisqueriedonlywhenadiseaseorpestis identified with sufficient confidence, minimizing unnecessary API calls. The frontend dynamically renders results, including predicted class, confidence score, and thegenerateddiagnosticreport,providinganintuitiveand user-friendly interface accessible from any device with a modernwebbrowser.

Both trained models are loaded into memory at application startup and remain resident throughout the session, eliminating per-request model loading overhead and ensuring consistent low-latency inference. The backend is structured around two dedicated REST API endpoints, one for disease analysis and one for pest detection, each returning a standardized JSON response that decouples the inference logic from the presentation layer. A confidence threshold gate controls Gemini API invocation, reducing cost and latency. Session state is intentionally kept stateless on the server, making the architecture horizontally scalable and straightforward to deployonanycloudoron-premiseenvironment.

International Research

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

4. METHODOLOGY

The system development follows a structured pipeline encompassing data preparation, preprocessing, model training, evaluation, and deployment. Each stage is designed to address specific challenges associated with limited data availability and real-world deployment requirements.

For the disease classification model, the PlantVillage dataset is preprocessed by resizing images to 224 × 224 pixels and normalizing pixel values to the range [0, 1]. Classweightsarecomputedandappliedduringtrainingto address dataset imbalance across the 38 disease categories.ThemodelistrainedusingtheAdamoptimizer with an initial learning rate of 1 × 10⁻³ in the first phase and 1 × 10⁻⁵ in the fine-tuning phase. Early stopping and model checkpointing are employed based on validation accuracytopreventoverfitting.

Forthepest detection model,imagesare resized to 300 × 300 pixels to leverage EfficientNetB3's higher resolution capacity. The augmentation pipeline is applied online duringtrainingusingKerasImageDataGenerator,ensuring thateachepochexposes the model tosyntheticallyvaried samples. The same two-phase training approach is applied, with the number of unfrozen layers calibrated basedondatasetsizetobalanceadaptationandoverfitting risk.

Modelperformanceisevaluatedusingaccuracy,precision, recall, and F1-score computed on held-out test sets. Confusion matrices are generated to identify categories with elevated misclassification rates, informing targeted dataset improvements. The Gemini Vision API integration is validated through a qualitative evaluation involving structuredassessmentofgeneratedreportcontentagainst referenceagronomicguidelines.

5. RESULTS AND DISCUSSION

The disease classification model achieves 94% validation accuracyonthePlantVillagedataset,demonstratingstrong performance across the majority of the 38 disease categories. The pest detection model achieves 92% accuracy across the 12 pest categories, a result that is particularly noteworthy given the limited training data available. The two-phase transfer learning strategy is observed to provide consistent performance improvementsoversingle-phasetraininginbothtasks.

Comparative analysis indicates that while some benchmark studies report higher classification accuracy under idealized conditions, those results are typically obtained using balanced, high-quality datasets without deployment constraints. The proposed system is intentionally designed to prioritize practical utility over marginalaccuracygainsincontrolledsettings.

ThediagnosticreportsgeneratedbytheGeminiVisionAPI are assessed qualitatively and found to provide contextuallyappropriateandactionableinformationinthe majority of test cases. The structured report format consistently includes severity ratings, recommended interventions,andrelevant pesticideororganictreatment suggestions, significantly exceeding the informational valueofclasslabeloutputsalone.

Response latency for the full pipeline, including model inference and API query, averages approximately 3 to 5 seconds per image under standard network conditions. This is considered acceptable for the intended use case, where near-real-time feedback is more important than instantaneous response. The Flask web application maintainsstableperformanceundermoderateconcurrent usage.

5.1 Model Performance

Table 1: System Performance Summary

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net

5.2 Comparison with Baseline Architectures

Table 2: Comparison with Baseline Architectures

5.3 Comparison with Related Work

Table 3: Comparison with Base Papers Aspect Nigar et al. (2024) Butera et al. (2022) This Work

5.4 Output Screenshots

6. LIMITATIONS

Despite its effectiveness,theproposed systemhasseveral limitations. Model performance may degrade under poor image quality conditions, including low lighting, motion blur, and severe occlusion. The relatively small pest dataset restricts the model's ability to generalize to unseenorrarepestspeciesbeyondthetrainedcategories. Additionally, the diagnostic reporting module relies on external API integration, making it dependent on stable internetconnectivityandlimitingusabilityinlow-network environments. Furthermore, the system is restricted to a predefined set of crops and pest classes and does not support dynamic adaptation to newly emerging diseases orpestswithoutretraining.

7. FUTURE WORK

Futureresearchcanfocusonimprovingrobustnessunder challenging imaging conditions through advanced preprocessing and augmentation techniques. Expanding

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056

Volume: 13 Issue: 04 | Apr 2026 www.irjet.net p-ISSN: 2395-0072

the dataset to include a wider range of pest species and crop types will enhance generalization capabilities. Incorporating offline or edge-based inference can reduce dependency on internet connectivity and improve realworld applicability. Additionally, integrating continual learning mechanisms will enable the system to adapt dynamically to new diseases and pests. The inclusion of real-time detection and mobile deployment can further enhanceusabilityforpracticalagriculturalapplications.

8. CONCLUSION

This work presents a practical agricultural intelligence system that bridges the gap between deep learning research and real-world field application. By combining plant disease classification, pest detection, and AIgenerateddiagnosticreportingwithinaunifiedweb-based platform, the system delivers a comprehensive and accessible solution for farmers and agricultural practitioners. The integration of vision-language AI capabilities enables the system to move beyond classification into actionable decision support, representing a meaningful advancement over conventionalsingle-tasksystems.

Future work will focus on expanding dataset diversity to improve pest detection generalization, enabling offline inference through model quantization and edge deployment, and incorporating environmental and meteorological data to support predictive disease outbreakanalysis.

9. REFERENCES

[1] S. P. Mohanty, D. P. Hughes, and M. Salathé, "Using Deep Learning for Image-Based Plant Disease Detection," Frontiers in PlantScience,vol.7,p.1419,2016.

[2] M. Tan and Q. V. Le, "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks," in Proc. 36th Int. Conf. MachineLearning(ICML),2019.

[3]J.Liuetal.,"ADatasetforInsectPestRecognitionintheWild (IP102)," in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition(CVPR),2019.

[4] K. P. Ferentinos, "Deep Learning Models for Plant Disease Detection and Diagnosis," Computers and Electronics in Agriculture,vol.145,pp.311-318,2018.

[5] M. Brahimi et al., "Deep Learning for Tomato Diseases: Classification and Symptoms Visualization," Applied ComputationalIntelligenceandSoftComputing,2017.

[6] M. H. Saleem et al., "Automated Analysis of Crop Pest and Disease Detection Using Deep Learning," IEEE Access, vol. 7, pp. 189700-189715,2019.

[7]K.ThenmozhiandU.S.Reddy,"CropPestClassificationBased on Deep Convolutional Neural Network and Transfer Learning," Computers and Electronics in Agriculture, vol. 164, p. 104906, 2019.

[8] A. Kamilaris and F. X. Prenafeta-Boldú, "Deep Learning in Agriculture:ASurvey,"ComputersandElectronicsinAgriculture, vol.147,pp.70-90,2018.

[9] Google DeepMind, "Gemini: A Family of Highly Capable MultimodalModels,"TechnicalReport,2023.

[10] F. Chollet, "Keras: The Python Deep Learning Library," Available:https://keras.io