Members-Only
Recent Talks & Demos are for members only
You must be an AI Tinkerers active member to view these talks and demos.
GTA VAEs and I-JEPA Dynamics
Explore a custom VAE for GTA5 scene generation, examine KL‑weighted reconstruction loss, compare latent sizes, then demonstrate I‑JEPA learning dynamics to predict subsequent frames.
I’m working on using VAEs and I-JEPA to generate worlds you can move through. The goal is to teach a model to both imagine and understand/predict how they change as you explore. I will show a custom VAE that generates novel scenes from GTA5 and a custom implementation of I-JEPA that’s learning the world dynamics. The goal is to use them together to predict the next frame based on input.
The VAE I’ll show is custom and I will focus in on the loss calculation to show the reconstruction loss via the KL Divergence. I’ll also show the outputs of the model with different latent dimensions (i.e. too big vs too small vs just right).
For I-JEPA I’ll compare and contrast it against the VAE and why it’s different. I’ll use a Notebook to illustrate how the algorithm works and 🤞a decoder to show how what it has learnt about the world.