How to Build a Flexible Robot Brain One Bit at a Time

Description

Your roomba can’t philosophize about the nature of dust, and your washing machine won’t debug your code while spinning your clothes - not because they lack ambition, but because running AI models on constrained hardware has traditionally meant sacrificing either performance or memory - usually both. Cortex.cpp is an open-source engine to run AI models on everything from Raspberry Pis to industrial robots without breaking the bank. This talk goes over how AI models work on different hardware from a C++ perspective, different ways of using Python from within C++ and vice-versa, and how to enable near-metal performance through aggressive bit manipulation, custom memory allocators, and SIMD optimizations. We will also go through the architecture of Cortex and compare it to different C++ implementations while highlighting the tradeoffs we have been making along the way. If you’re interested in pushing the boundaries between AI and constrained hardware or just want to see how far you can stretch a bit, this session will show you how we’re giving everyday devices the gift of intelligence, one carefully-aligned memory block at a time.