Navigating AI Prototyping: Lessons from the Carter Center's Human-Centered Approach
How can organizations that rely on trust begin to prototype and adopt AI solutions?
In our June 2025 XD131 Applied AI course, Michael Scholtens, Manager of Data Analytics for the Carter Center shared his experiences from a multi-year journey with AI. The Carter Center's journey underscores that successful AI prototyping and adoption in mission-driven organizations hinges on a thoughtful, human-centered approach that prioritizes trust, adapts to user needs, and carefully manages data sensitivity.
For the Carter Center, an organization with a deep history in program work including election observation and conflict resolution, maintaining trust and credibility is paramount. As a result, any efficiencies gained from AI must not compromise trust built over decades, especially as AI outputs get closer to public consumption.
Three Pillars for Incorporating the Human Element
The Carter Center has developed an approach that ensures that human intuition, values, and principles remain central to the tools they select and build. This approach can be categorized into three broad, increasingly complex areas where human elements are incorporated.
Slide from XD131 presentation: Three Pillars for AI Trust
Thoughtful Prompting: As the initial step in AI engagement, this involves asking the right questions to ensure accurate outputs and maintain trust. The Carter Center prototyped a prompt library in Power Apps to help users share effective prompts for tools like Microsoft Copilot and responsible uses of ChatGPT. However, this library wasn't heavily used because user needs often varied significantly, leading them to realize that the overlap in prompt utility between users was small. They found more value in providing off-the-shelf AI tools and building pilot programs and user communities around them, especially in their country offices globally.
Reference Human Knowledge (Retrieval Augmented Generation - RAG): This mid-level complexity involves grounding AI tools in an organization's own vast repository of information. The Carter Center built an AI application using RAG to leverage their extensive historical documents, aiming to help users find information about their work's impact from cloud-stored reports and research. While the initial outputs were "interesting," they faced limitations when users asked questions requiring an understanding of relationships between documents, which the general chatbot architecture was not designed to provide. This led to user frustration and a key lesson: explicitly bounding AI tools for specific use cases results in better user satisfaction and trust.
Don’t Cut Out Humans: This represents the most complex and technically demanding stage of their prototyping. Currently, the Carter Center is prototyping workflows using LangGraph and Streamlit to improve AI results by soliciting human feedback. A detailed example was discussed from their work on analyzing data from the Syrian conflict to predict unexploded ordnance contamination. Initially AI models struggled with certain categorizations, which led to developing a system where humans review the model's reasoning and predictions, correct errors, and provide rationale for those corrections. The model then learns from these reflections, synthesizing them into improved prompts for future classifications, leading to significantly improved performance without constant human involvement. But at every stage, human oversight of autonomous AI agents is critical to maintain control and prevent unintended consequences, especially when they can make decisions or initiate actions within an organization's tech stack.
Building a Secure Internal AI Agent (AIDA)
Trust can be hard to maintain when relying on third-party AI tools, especially when it comes to sensitive data. The Carter Center created the AI Data Assistant (AIDA) as an internal AI data analysis toolkit. Built using LangChain and powered by Azure OpenAI for enhanced data protection, AIDA allows users to securely upload data files (CSV, Excel), or directly connect to ODK Central and SharePoint for analysis. This replicates functionalities of public tools like ChatGPT's data analysis feature within a controlled, secure environment. Despite the duplication of existing tools, the team felt that these steps were necessary to adhere to their data protection principles – and has been rewarded with enthusiastic internal use.
Slide from XD131 presentation: Prototyping lessons learned
Key Lessons Learned from Prototyping
Good technology doesn't always equate to a useful tool. Many technically impressive tools they built weren't adopted because they didn't meet user needs or were too rigid for flexible application.
Grounding AI models in human knowledge is crucial for internal adoption. If users feel ignored or overwritten by AI, they will not trust or use the tool, regardless of its potential benefits.
Data sensitivity dictates the prototyping approach. While AI as a service is often easier, handling sensitive information necessitates careful consideration of third-party trust, frequently leading to the decision to build in-house solutions.
AI adoption can improve communication and reduce language barriers. The broad-based training and adoption of AI tools across their global offices have significantly helped to close communication gaps between headquarters and country offices, often due to language differences. AI tools have increased confidence among field staff to communicate more directly and clearly, potentially leading to faster information flow and resolution of issues. This assistance in drafting communications helps users convey their intended message with the correct tone, reducing misunderstandings common in electronic communication across different languages and cultural contexts.
Even small prototypes can yield significant efficiency gains. In projects like the Syria conflict analysis, AI has drastically reduced the time needed for complex tasks from weeks or months to a matter of days, allowing the Carter Center to extend their valuable work.
This series features insights from our June 2025 Applied AI (XD131) course. Listen to the AI-generated podcast for an audio recap of this event.
