Weed Detech

Demo Video: here

Inspiration💡

Some of our project members come from farming families and we have personally seen farmers wasting hours of hard work in finding and removing harmful weeds from large fields. Our project aims to reduce their efforts by pin-pointing weeds in a field, thus enabling its faster removal.

While buying vegetables, one tends to enquire “Are these vegetables organic?”, “Are harmful pesticides used to grow them?”, “Where is the Organically Farmed produce?”. This concern is caused by the overuse of pesticides which prompted us to come up with Weed Detech (pronounced as weed-detekt). We aim to lower the consumption of pesticides by pin-pointing the weed in a field.

Subsequently, pesticides can be sprayed only around the weed, keeping the produce pure and organic!

What it does🔎

Weed Detech is a simple-to-use web application that allows crop growers or farmers to better find weeds in their fields and thus is a great way to promote the lesser and at the very lest a very local use of pesticides.

Here is a run of the show for how Weed Detech works:

• Take images of the field, could a few camera clicks or in the case of large fields a mapping of the field
• The images go through the machine learning model we have built which returns two outputs
  • Co-ordinates of boxes in which plants are detected
  • Information on whether the plant in the box is a weed or not
• Display the location of the weeds on the fields through an intuitive UI
• Improve and personalize the model in the long run by taking images from whoever opts in through Appwrite (yet to implement differential privacy through federated techniques for Weed Detech)

How we built it🔨

We majorly divide this project into two sub projects:

Weed Detech Machine Learning Model

The Weed Detech Machine Learning Model after thorough experimentation and also taking into account the data we have, we build on top of the ConvNeXt model using it as a backbone for our object detection model. We transfer learn from a pre-trained ConvNeXt model trained on ImageNet and then add quite a few layers to make it work for our specific tasks of weed detection. We trained this model on Google Cloud TPUs.

Having ended up with quite a large model, which we spent time training on and understanding the quirks for, optimizing this model was of the essence and we were also able to introduce optimizations to it using multiple optimizations from Grappler. All of this as shown in the demo comes together to give a great and quick usage experience for the project.

Weed Detech Web App

The Weed Detech Web App was built using Streamlit which simplified for us the labyrinth of managing data-heavy applications like ours, state management for a large model, and more. The web app is written in Streamlit allowed us to introduce optimizations easily for the data and memory intensive tasks we want to be able to run for this application.

Challenges we ran into⚠️

• Identifying and finding the right kind of data for the machine-learning model.
• Our first pass at integrating the model into the web application was quite bulky as well as took some time to infer from. Optimizing the model and finding or modifying the optimization techniques that work for our use cases was one of the big challenges.
• We also wanted to add support for federated learning for personalizing the model for users, however, this seemed to be a more time-involved task for us and we plan to work on this after the hackathon.

Accomplishments that we’re proud of🥇

• After multiple attempts and thousands of training images, we were finally able to produce a fully functional farm weed detector. Our model not only identifies the weed, but it is also able to precisely locate it by drawing a box around it.
• We are quite happy that we were able to piece together the UI components with the model we trained as well as a pipeline for retraining the model.

What we learned🧠

• Most of our project members were not experienced with techniques in the area of Machine Learning or modern machine learning model like the one we used.
• All our project members are first-year students at the University of Toronto, but our combined experience helped us complete this project in the allotted time. Our belief in teamwork also increased during the course of this hackathon. Most of all, it solidified our view that copious amounts of clean data are what drive the best artificial intelligence systems.

What’s next for Weed Detech💭

• Working more on having a federated learning system for personalizing the model for the users.
• With a bit more automation and personalization, Weed Detech is ready to go into the testing phase and help in keeping crop produce pure in real life.

Check it out here: GitHub Repo