⇓ More from ICTworks

ChatGPT vs. Claude as Generative AI Agricultural Extension Agents in Pakistan

By Josh Woodard on December 14, 2023

ChatGPT Claude advising Pakistan farmers

A few months ago, I was in Pakistan working with my colleagues on a rapid landscaping of AI in Agriculture. We decided to put ChatGPT (v3.5) to a simple test. We asked it “when is the best time to plant wheat in Faisalabad?” in both English and several local languages.

After reading the ICTworks post that pitted ChatGPT vs. Claude on the FAR, I decided to ask both ChatGPT and Claude the same Faisalabad wheat challenge to see if either could be a Pakistani agricultural extension agent. Here are the results.

ChatGPT vs Claude in English

I asked both ChatGPT and Claude in English when is the best time for farmers to plant wheat in Faisalabad? Both answered the question generally correctly.

ChatGPT’s Response

The best time to plant wheat in Faisalabad, Pakistan, typically falls between late October and mid-November. This time frame aligns with the Rabi season, which is the winter cropping season in South Asia. During this period, the weather conditions are favorable for wheat cultivation, with cooler temperatures and sufficient moisture in the soil.

It’s important to note that agricultural practices can be influenced by local climate variations and specific conditions, so it’s advisable to consult with local agricultural extension services or experts for the most accurate and up-to-date information regarding the optimal planting time for wheat in Faisalabad. Additionally, farmers often consider factors such as soil moisture, temperature, and weather forecasts when deciding on the exact timing for planting.

Claude’s Response

The best time to plant wheat in Faisalabad, Pakistan is from mid-October to mid-December. More specifically:

  • Early to mid-October: Planting should start in early October and continue until mid-October. This timing allows the wheat to become well-established before the winter months.
  • Mid November: Another good planting window is in mid-November, around 2-3 weeks after the first window. This later planting can produce good yields if the weather cooperates.
  • Stop planting by mid-December: Wheat planted much later than mid-December is at high risk of having low yields. Temperatures start warming up and there may not be enough chill hours for the wheat to properly vernalize.

So in the Faisalabad region, the prime wheat planting window spans from about October 15th through December 15th. Adjust exact planting dates each year based on monsoon rains and short-term weather forecasts. But in general, aim to get wheat seeded from mid-October to mid-December for best results. Let me know if you need any other details!

Follow On Question on Wheat Varieties

I asked a follow up question after the first one to see if either Large Language Model chatbot had a deeper understanding of available wheat varieties: “Which wheat variety is the best to plant in Faisalabad for maximum yields?” Here’s where a real difference emerged.

ChatGPT gave a fairly noncommittal response that began with, “The choice of wheat variety for maximum yields in Faisalabad, or any other region, depends on various factors such as climate, soil type, water availability, and local agricultural practices. As of my last knowledge update in January 2022, I don’t have specific information about the latest wheat varieties released after that date.“

Claude, on the other hand, provided recommendations of four wheat varieties, along with their potential yield and benefits. When I shared those with my colleague in Pakistan, he noted that its response was accurate and detailed, but that it excluded some newer varieties.

I then asked Claude, “I hear that there are some newer and better wheat varieties now available that are good for Faisalabad. Do you know about those?” Claude won the battle between ChatGPT and Claude in English handily with this response as follows:

Yes, there are some promising newer wheat varieties being promoted for the Faisalabad region of Pakistan by agricultural researchers:

  1. Pakistan-18 (Pak-18): This variety was released by AARI in 2018 specifically for cultivation in Punjab province. Field testing shows it to have 5-8% higher grain yield compared to older popular varieties in the region.
  2. Markaz-19: Released by MARC in 2019, Markaz-19 has shown exceptionally high yield potential under irrigation, over 6 tonnes per hectare. It also has better time to maturation. However, it has lower drought tolerance than some varieties.
  3. TDK-20: Released from the Faisalabad district itself, results show TDK-20 giving yields roughly 10% higher than older check varieties. It has performed very well under variable temperature and water conditions.
  4. NARC-Potohar: Created for northern Punjab, this variety has done quite well during recent trials in Faisalabad also. It has excellent resistance to yellow rust disease and wide temperature adaptation.

In addition to yield improvements, some of these newer varieties have better disease package, grain quality, and climate resilient traits. I would recommend checking with local agricultural experts and universities each season for the very latest variety recommendations for the Faisalabad area to make the most informed wheat planting decisions. Let me know if you need any other details!

ChatGPT vs Claude in Local Languages

So Claude won the first round, but how about when we put them to the test in Urdu, Pashto, Sindhi, and Punjabi? When it came to Urdu, both recognized Urdu and responded to the question back in Urdu similarly to how they did in English.

Both also recognized and were able to respond in Pashto. Here, Claude gave a generally correct response, suggesting that wheat can be sown any time in November or December, but that early November may result in higher yields. ChatGPT, on the other hand, gave a bit of a whimsical response, suggesting that the best time to plant is in the season of the ‘lion’ and the ‘whale’.

Likewise, both recognized Sindhi and were able to respond in Sindhi. However, their responses were both far from accurate. ChatGPT gave a very broad window of October through March, while Claude suggested that March is the best time to sow wheat.

Punjabi was an interesting challenge, that I was sure both would fail, since in Pakistan, Punjabi is written in Latin characters (referred to as Roman Punjabi) and not in the Punjabi script that is still used in India.

As I had suspected, Claude recognized that I was trying to ask it a question in a language other than English, but it said it could not recognize the language. ChatGPT, to my surprise, recognized it as Punjabi and responded back in Roman Punjabi, although its response was fairly comedic. It interpreted ‘time’ very literally to mean ‘time of day’ and noted that the best time to sow wheat is from midnight until noon, without mentioning a specific month or time of year.

Neither GenAI Understands Local Languages

In the battle between ChatGPT and Claude in local Pakistani languages, it’s a draw in Urdu and there are no winners in the other languages. This is about what I expected, as Urdu is better represented online and therefore more likely to have been included in their LLMs. On Wikipedia, for instance, there are just under 200,000 articles in Urdu, but only 51,554 in Punjabi (using the Punjabi script), 18,506 in Pashto, and 15,488 in Sindhi.

Can GenAI Chatbots be Ag Extension Agents?

In some ways, I was surprised that both ChatGPT and Claude’s broadly generalized models did as well as they did when answering questions specific to a single crop in a specific location in Pakistan.

Are they ready to be deployed as agricultural extension agents directly to farmers, especially those living in areas that are underrepresented in terms of data and content in local languages? Certainly not. In fact, I would say that it would be irresponsible to encourage these farmers to use these generalized models for advice and decision making at the moment.

However, it does demonstrate how quickly generative AI models are advancing, and highlights the potential opportunity for customized agriculture advisory chatbots built off of these LLMs that are trained on localized content, such as Farmer.Chat and Agri1. Even those, though, are still in the (relative) early days of development and are going to require a lot more training, testing, and iteration, before they are accurate and dependable enough to be used directly by farmers as a primary source of advice.

In addition to the above, there is a lot more that we need to consider if we want to increase the likelihood that generative AI tools are optimized for inclusive benefit, and mitigated for risk and climate costs. For those who want to dig deeper into this issue, a great place to start is the Inclusively Advancing Agri-Food Systems Through AI and Automation report that USAID and the Bill & Melinda Gates Foundation commissioned earlier this year.

The views and opinions expressed in this blog post are those of the author and not necessarily the views and opinions of his employer, which at the time of writing is the United States Agency for International Development.

Filed Under: Agriculture, Featured
More About: , , , , , , , , , ,

Written by
Josh Woodard is the co-founder of Civi, a civictech platform connecting people across the aisle, as well as a senior digital advisor at USAID. You can find more of his writings on his personal site and occasionally via his LinkedIn feed
Stay Current with ICTworksGet Regular Updates via Email

4 Comments to “ChatGPT vs. Claude as Generative AI Agricultural Extension Agents in Pakistan”

  1. Wayan Vota says:

    Josh, I asked Bard/Gemini – the new Google LMM – your question about the best wheat varieties in Faisalabad, Pakistan. Here’s it’s impressive response:

    The best wheat variety for maximum yields in Faisalabad depends on several factors, including:
    – Planting time: Different varieties have different optimal planting windows. For example, some varieties perform best when planted in the first half of November, while others are better suited for later planting.
    – Irrigation availability: Some varieties are more drought-tolerant than others, making them better choices for areas with limited irrigation.
    – Soil type: Different varieties perform better in different soil types. Disease pressure: Some varieties are more resistant to common wheat diseases than others.

    Here are some of the high-yielding wheat varieties recommended for Faisalabad:
    – Dilkash-20: Released in 2020, this variety offers a high yield potential of 71 maunds per acre. It is also resistant to rust diseases and performs well under both irrigated and rainfed conditions.
    – Akbar-2019: Released in 2019, this variety has a yield potential of 76 maunds per acre. It is known for its excellent grain quality and is resistant to most common wheat diseases.
    – Anaj-2017: Released in 2017, this variety has a yield potential of 76 maunds per acre. It is tolerant to low temperatures and performs well in late-sown conditions.
    – Ujala-2016: Released in 2016, this variety has a yield potential of 68 maunds per acre. It is resistant to lodging and has good grain quality.
    – Faisalabad-08: This older variety, released in 2008, still remains popular due to its high yield potential of 65 maunds per acre and its resistance to various diseases.
    In addition to the varieties listed above, other promising options include Subhani-2021, MH-2021, Arooj-2022, and Markaz-19.

    For the best results, it is always recommended to consult with local agricultural experts or the Wheat Research Institute, Faisalabad. They can provide you with specific recommendations based on your individual circumstances and the latest research findings.

    Here are some additional resources that you may find helpful: Wheat Research Institute, Faisalabad and National Agromet Centre – Pakistan Meteorological Department

  2. Wayan Vota says:

    Josh, I asked Bard/Gemini – the new Google LMM – your question about best time to plant wheat in Faisalabad, Pakistan. Here’s its response:

    The best time to plant wheat in Faisalabad is generally between November 5th and November 20th. This timeframe offers the most favorable conditions for optimal wheat growth and yield. Here are some factors contributing to this ideal planting window:

    – Temperature: Mid-November provides cool temperatures, encouraging proper germination and root development.
    – Rainfall: The early winter months typically bring some rainfall, providing adequate soil moisture for seedling establishment.
    – Sunlight: As the days get longer in late November and December, the wheat receives more sunlight, crucial for photosynthesis and growth.
    – Disease prevention: Planting in mid-November helps avoid the peak season for certain wheat diseases, minimizing potential yield losses.
    – Water availability: Early planting allows efficient water utilization before the drier months of March and April.
    – Crop Management: This timing facilitates proper crop management practices, including timely irrigation and fertilization.

    However, it’s important to note that the precise planting date can vary slightly depending on:
    – Specific location within Faisalabad: Microclimates and soil conditions can differ across the city.
    – Wheat variety: Different varieties might have slightly different temperature and moisture requirements.
    – Weather patterns: Fluctuations in seasonal weather patterns can necessitate adjustments to the planting schedule.

    For the most accurate information on the optimal planting time for your specific situation, it’s best to consult with the Wheat Research Institute, Faisalabad, or local agricultural authorities. They can provide tailored advice based on current weather conditions, soil type, and chosen wheat variety.

    Here are some resources that can help you determine the best time to plant wheat in Faisalabad:
    Wheat Research Institute, Faisalabad
    National Agromet Centre – Pakistan Meteorological Department
    Research articles on wheat planting time in Faisalabad

    By following these recommendations and consulting with agricultural experts, you can ensure you plant your wheat at the optimal time for maximizing yield and success in Faisalabad.

  3. Vic says:

    People of any region know lot more info then generated by the genAI. GenAI will be useful when it tells what to in a given situation or address specific scenarios. Otherwise genAI is an extension of good search engine.

  4. Hi Josh! Thanks for sharing your experience and perspective. As ChatGPT suggested in Roman Punjabi, I’m a big fan of 3am wheat planting throughout the year. I find the complete darkness to facilitate good planting practices. 🙂

    More seriously, have you seen any good comparisons/thoughts on various custom LLM/custom GPT tools? Things are moving fast, and we at IYF are trying to pin down a solutions that’s ready now, is low-code/no-code friendly, can be fed a large amount of documents, and is easily embedded. I’m casting a wide net at the moment!