Friday 5.0: World Models, Transformers Transformed, Humanoids, and The Benefits of Bubbles
Some readings for your weekend if you're in the world of tech, especially if you're interested in the world of Compute, AI, and Robotics
Table of Contents:
Topics I’m Studying
Videos to Watch
Books to Read
Topics I’m Studying
WORLD MODELS: I strongly believe 2025 will be the “Year of AI-Powered Robotics”. World Foundation Models (WFMs) are specialized AI systems trained to understand and simulate physical environments, enabling robots and autonomous systems to interact with real-world dynamics. Unlike traditional foundation models focused on language or images, WFMs combine video data, physics simulations, and sensor inputs to create 3D-consistent, physics-aware environments for training AI agents. NVIDIA made a really cool announcement of their “COSMOS Platform” for training world models, and I think we’ve barely seen the beginning of an incredible acceleration here!
MATHS vs. PHYSICS - WHAT’S THE REAL BOTTLENECK FOR AI?: It’s hard to look anywhere without seeing “DeepSeek” and “tech market disruption” or “the short case for NVIDIA” these days. I’ll leave speculation and market forecasting to the pundits here, but understanding the technical improvements of DeepSeek vs. incumbent architectures I think will be a key source of insight and “alpha” in the weeks to come. Thus I’d strongly recommend reading “How Has DeepSeep Improved the Transformer Architecture?”. The real question here, will be to understand where the real constraints for AI will be this year? OpenAI recently said they were “power constrained, not compute constrained”. But are we all really just “Maths constrained” when it comes to how much AI systems can be optimized?
Videos to Watch
Humanoid Robot Trends to Watch in 2025: A lot of buzz around Humanoid Robots at CES this year (surprise, surprise), but it’s quite extraordinary to see how far some of these companies have come. And now with what seems to be a step-change in Foundation Model efficiency (looking at you, again, DeepSeek), as well as NVIDIA’s “Project DIGITS”, 2025 is shaping up to be quite the year in robotics. A lot of partnerships popping up between hardware companies and AI-providers (e.g. 1X and OpenAI), but I’m pondering where the value pools will sit for Humanoids - will it be in the hardware, the data collection, model development, compute, or somewhere else?
How does DeepSeek R1 work? DeepSeek Explained for Your Grandma
Books I’m Reading
“BOOM: Bubbles and the End of Stagnation”. This super interesting book examines how financial bubbles, often viewed negatively, have actually been the driving force behind major technological and scientific breakthroughs throughout history. It argues that by embracing the dynamics of bubbles—including excessive risk-taking, vast funding, and poor accountability—can actually accelerate innovation and end the current period of stagnation that has persisted since the 1970s. Really great read for those in Tech & Frontier investing
About Me
Working at the interface between frontier technology and rapidly evolving business models, I work to develop the frameworks, tools, and mental models to keep up and get ahead in our Technological World.
Having trained as a robotics engineer but also worked on the business / finance side for over a decade, I seek to understand those few asymmetric developments that truly shape our world
You can also find me on X, LinkedIn or www.andreasproesch.com