Creating Zebra-Llama, an open-source AI rare disease expert

ChatGPT and other LLMs are increasingly being used by rare disease patients and researchers, but this presents significant challenges around the reliability and safety of the responses. Zebra-Llama is an open-source AI model specifically developed for Ehlers-Danlos Syndrome (EDS) expertise. During the demo, I’ll discuss how we trained Zebra-Llama on an EDS-specific dataset we built from thousands of research papers and patient experiences to provide accurate responses and cite sources. I’ll walk through the model’s architecture, including LoRA fine-tuning of the Llama-3–8b-instruct model, vector database integration with RAG, and use of a Question-Context-Answer format to reduce hallucinations and improve reliability. Finally, I’ll show a live demo using the streamlit chat interface. Our team met and developed the model as part of the Rare Disease AI Hackathon organized by Stanford Medicine and Research to the People, and presented it at GitHub headquarters in San Francisco to a distinguished audience including Greg Brockman, President of OpenAI.