Nvidia GTC, Devin, and Neuralink – Live and Learn #38

✨ Quote ✨

Our bias is that technological progress is good, actually.

– Packy McKormick – (source)

🖇️ Links 🖇️

Devin by Cognition. Copilot and ChatGPT are already amazing tools for developers but they "only" seek to enhance the productivity of developers. Devin, on the other hand, tries to replace developers outright. And it is much much better at solving real-world, end-to-end problems than any system that came before. Sure it's still an early version and will only get better from here on, but watching Devin train another AI from scratch is simply insane. It creates an action plan and then executes it step by step: downloading the source from Github, writing some Python glue code, installing dependencies, fixing CUDA errors, etc... all on its own. And that makes me incredibly afraid that I lose my job much sooner than I thought.

Jensen Huangs GTC Keynote by Nvidia. To me, the biggest thing that happened in the last two weeks was the GPU technology conference hosted by Nvidia. The main keynote is two hours long but well worth watching in its entirety. It gives a glimpse into the future, of things yet to come. Nvidia is building out the infrastructure that will power the 4th industrial revolution. They aim to commodify and productize intelligence. Their thesis is simple: the cheaper they can make compute (and thereby intelligence), the more people will want to use it to solve their problems and Nvidia can create their own demand. Essentially they are aiming at disrupting every industry out there by selling them compute and advanced AI systems to solve whatever problems they face. To do this, Nvidia is building a lot of things right now, both software and hardware. Among a few of the things they are working on are Gr00t, Blackwell, OSMO, Isaac Perceptor, Isaac Manipulator, Isaac Sim, Isaac ROS, the Omniverse and NIMs. This is not even everything announced in the Keynote, it's just to give a sense of the scale of the ecosystem they are building. And they partner with basically every big company out there. In a nutshell, Nvidia aims to provide the tools necessary to build advanced robotics, biotech, weather prediction systems, AI agents, simulations, chip design, AI data centers, generative AI tools, and more. They even want to move the design of everything into the digital world, with something they call digital twins: prototypes that can be tested and iterated on digitally much faster than in the real world and then when done with designing be built first-try in the real world. If companies want to compete in their industries they will need AI, powered by Nvidia, and I don't see any other company that can reasonably compete with them and disrupt their gigantic flywheel. They simply innovate too much, too fast. Disrupting their own state-of-the-art chips every year, simply because they want to make computing even cheaper, even faster. And they are also developing the software that makes these chips so useful in the first place. And that's why Nvidia is single-handedly powering almost the entire AI revolution behind the scenes, selling shovels in a gold rush. This doesn't even do them justice, it's more like they are showing people what gold is and then teaching them how to mine it so that they can sell even more shovels. It's win-win thinking all the way down and watching the GTC keynote highlights that.

Sima AI by DeepMind. Essentially Google has created a generalist AI agent that can learn how to play different computer games with ease. This is huge, because earlier AI systems needed more finetuning to understand new games, while this approach can learn new games on its own and can then be instructed to do certain tasks within games by language only. The idea is to have an agent that can learn how to fulfill general tasks in any world (including the real one) by learning in virtual worlds first.

Figure 1 Update. I posted an update already in the last Edition of Live and Learn, but Figure 1 released an even more ridiculous update in the last week. What sets Figure 1 apart from other humanoid robots like Teslas Optimus, is that all the tasks are reinforcement learned and autonomously executed. This means that there is no human in the loop that needs to show the robot how to do something. It can understand commands in natural language and learn the appropriate actions on its own, even explaining what it does and why it "thinks" that this is the right thing to do. Figure 1 is by no means the only startup pursuing robotics at this point in time, as the GTC keynote shows, but for me it's one of the most exciting to follow.

EVE Video Editing by MetaAI. This AI by Meta can edit videos for you and it understands simple language instructions like "Make this red" or "Make it a panda". To me, the demos are simply mindblowing and fun to watch. And it's incredible how the AI understands enough about videos that it can go ahead and do even very involved edits easily and without artifacts. It honestly seems like magic.

Sam Altman Interview by Lex Fridman. I love the Lex Fridman podcast and his interviews with Sam Altman are among the best. To me, the craziest part of this show was how Sam Altman was teasing about how OpenAI has several bigger announcements than Sora to make this year. And that ChatGPT 4 will seem a bit meh in comparison to what they will release soon enough. I am so excited for whatever it is, that I don't even find words to describe how I feel. The future is going to be rad.

Neuralink Patient Update. Neuralink is trying to help people with all sorts of disabilities gain superpowers. And their human trials have only started. Seeing somebody control a chessboard on a computer with their brain is ridiculous and I cried tears of joy when I saw how happy the Neuralink implant made this person. Eventually, most of us will have systems similar to Neuralink in our heads and it will feel normal to use our brains to control computers and telepathically send messages to other people. As normal as it is today to use a smartphone. And that to me is absolutely bonkers. The Wizard Hat gets nearer and nearer every day.

Stability Video 3D (SV3D) Model by Stability AI. Stability AI has released another model that can generate 3D meshes from single image input and again improved on the current state of the art with their new model. I think that these image to 3D models will soon become good enough to be used in games and to me that is an interesting development because it will allow people to create games with particular art styles and a custom look on a much smaller budget than ever before.

Extropic Thermodynamic Future. Amidst all the hype around Nvidia dominating the world of AI, this company has a very different approach to computation. They are trying to build a computer chip that is closer to computronium than anything that I have ever seen anyone trying to build before. The main idea is simple: harness thermal fluctuations to do useful computation. The whitepaper releases some more details and there is also a First Principles Episode with the founders explaining what they are building. If they succeed they can speed up AI training and inference computations by several orders of magnitude. To me, one of the founders, Guillaume Verdon, is one of the most interesting and intelligent people alive today. He worked on quantum computing, and AI, and helped to create the e/acc movement with a meme-tweeting alt account named Beff Jezos. He's truly trying to accelerate the progress of technology and humanity and there's a whole interesting Lex Fridman Interview with him too, that I can only recommend watching.

🌌 Traveling 🌌

I am still enjoying my time in Guadeloupe, going diving every other day and staying in a small hostel by the name of Tikazalou. It’s cozy and beautiful around here. The people are nice, there is a beautiful cat and the WiFi is good, so I am quite happy.

🎶 Song 🎶

Cinnamon Crush by Jacob Collier

Youtube Music | Spotify

---

That's all for this time. I hope you found this newsletter useful, beautiful, or even both!

Have ideas for improving it? As always please let me know.

Cheers,

– Rico