Building Brushi: A Local VLM Agent That Understands Kids’ Drawings

Brushi is a kids coloring and drawing app with a local multimodal AI companion that can observe what a child is creating and respond through a small, bounded interaction loop.

The project explores a question I have been thinking about in my recent work on edge AI: small VLMs and LLMs should not be treated as weaker versions of large cloud models. Like children are not simply “small adults,” edge models behave differently because the system around them has to provide structure, grounding, and feedback.

In the live demo, I’ll show the working Brushi app, the local VLM/agentic architecture, and the event loop that connects the child’s drawing activity to the AI mascot’s reactions. The demo will focus on the technical workflow: how drawing events are captured, how visual context can be passed to a local VLM, how the LLM layer interprets the learning or creative context, and how the app decides whether to encourage, guide, correct, or stay silent.

I’ll show the architecture, app workflow, logs/traces from the interaction loop, and examples of where the system succeeds or fails when trying to understand a child’s drawing.