Hackathon Showcase

ChatterBox

Team consisting of CIHI, Huawei, Lyft, RBC, EnerZam and TMU engineers—UofT/MEng grads—combining healthcare AI, 3D vision, LLM/agent integration, full‑stack, PyTorch/TensorFlow expertise.

6 members Watch Demo

YouTube Video

Project Description

Description:
ChatterBox is an agent who is added to your group chat just like a normal person.
It decides “when” to contribute to the chat. It doesn’t just contribute whenever your prompt it, but it engages with group members like a real person.
To increase the impact of ChatterBox, we added a helpful personality to it so it tends to help its friends plan outings or keep the order in the group.

Repo Structure:

frontend: frontend website and chat interface
backend: saves chats to a state file. triggers the agent whenever a new message is posted in the chat.
agent: core agentic flow.

Structure:
We have two main agents: summarizer and decider.
Summarizer gets the history of chats and summarizes the messages.
Decider gets the summary and the last message and decides how to contribute: either send a message or do nothing.

Potentially, we can add other tools like reading people’s calendars, map, etc. so it can help in organizing outings.
We can potentially add a moderator personality to it so if it feels the chat is getting out of order, it can take it back to order.

Gemini flash is used in both agents. We can potentially add multimodality to it so it can receive voice and video messages in the chat and process them.
We have chained the two agents to each other.
For evaluation, reactions on chat messages can serve as human feedback to the model.

Tools Used:
We used ADK, Vertex AI, Gemini, chaining.
It has potential for multimodal chat processing.
**