Lead Machine Learning Engineer | Victoria, BC
For the past 8 years I’ve been doing ML across a handful of companies. Right now I’m at Engine. I’m lucky to enjoy what I do enough to pick up some personal research projects when I’m not at my day job. I try to catalog those endeavors here, with some shorter posts landing on my X or LinkedIn.
I’m pretty focused on small vision/language models right now. I’m wondering how to improve the performance of visual-RAG from an engineering and cost standpoint.
I try to write everything by hand. Sometimes I want to get a post written up but don’t have the time to hand-write it entirely. It should be pretty obvious when AI is involved but I’ll mention it in my posts anyway.
Text RAG is manageable. Document RAG means bolting on OCR and layout models that work well but add real overhead. Visual RAG skips the extraction step entirely, but trades it for a cost and precision problem of its own.
A chronicle of a reinforcement learning project, from initial struggles with exploding losses to a successful pixel-based agent for LunarLander.
A very approachable jumping off point for video captioning. If you're GPU-poor (<24GB vram) this is for you.