All-In Podcast — Open Source Wins, AGI Is Here, and Scorsese's AI Toolkit with CEOs of Cerebras & Black Forest Labs

📋 한국어 요약 보기 →

If you have a chance to play with Fable or 5.6 from OpenAI, increasingly what you don't have to get the prompt just right — you don't have to be a prompt whisperer. Instead, you ask it and it says, "Well, here are some things, and by the way, maybe you wanted the chart to go two ways — you wanted a line and a bar." And it's like, "Well, that's exactly what I wanted. I didn't ask for it, but that is better." And so it's understanding intent, and that's a huge leap which if we were sitting here two years ago, the idea — we would never have been able to predict in a short 24 months that it would go from being a great summarizer researcher of web results to actually understanding your intent and then providing a solution and abstracting it all from you.

I mean, if you run these for 25 or 48 hours, you get amazing things now. And what if by using Cerebras we were 15 times faster, and then you ran it for 24 hours — you got weeks or months worth of thinking. And I mean it is extraordinary. Now Cerebras is at the center of this because this reasoning is inference. This reasoning is inference and it's computationally intensive. And so fast compute makes this sort of work fast and tractable — it doesn't [take] a huge amount of time to get a good answer. And so it's exactly the fact that this reasoning consumes a huge amount of tokens internally that allows a blisteringly fast machine like ours.

[Andrew brought a physical chip to the interview] I brought one because I'm never far without — you know, when one costs half a billion to make, you bring it everywhere with you. We were tossing this back and forth at Davos. This was in the first eight or 10. This has a special place — I mean my wife says it's like I'm a kid with a dirt bike for his 8th birthday. It was in his bedroom at night. I carry him with me.

But what we're looking at here is the ability to do that reasoning at scale. And what is Moore's law for inference and for Cerebras? Do you have something internally you discuss as — we're going to double this every x time period?

I think that early in an architecture, you have room to do much better than what was traditionally Moore's law. Now, if you've got a 20-year-old architecture like the GPU, it's much harder — you have to rely on things like smaller geometry, going to the next fab node. But in a newer architecture, you have a huge amount of room still to learn about the work that is being presented and make optimizations that give you huge gains.

How do you run the company? Like just being the CEO now in the age of AI — you have $25 billion in demand, you have to deploy at an incredible blistering pace, you have to hire people, you have to create a roadmap. You have to keep up with somebody like OpenAI who's moving so unbelievably quickly. Right, and they're competitive — you got to keep up. Your hardware, your software, your deployments have to keep up with some of the fastest moving organizations in history. They're demanding customers. They are not pushovers for sure. And also potentially competitors down the road.

I look — I think there is so much demand right now that there is no silicon that will go unused. But why isn't OpenAI releasing Jalapeno? Why is Amazon making their own chips? You see this reoccurring trend? Is it a way to let Jensen and Nvidia know, hey, we can do this too, so we need good pricing? Is it a little bit of a flex that way, or is that the future that they're going to be in your business?

And they wanted more customers, and so they set about to help fund these neoclouds. And I think that's a very reasonable thing. I think you don't have to sort of make the fastest chip — you just can't be entirely dependent on other people's chips.

📋 한국어 요약 보기 →

And that dependency has become a hot topic. Not sure if you caught the episodes over the last two weeks, but we've been talking over the last year about open source. I've been championing that a lot just because I was early into Open Claw and quickly started using Kimmy and was like, "Wait a second, I'm blowing out my Claude tokens, but this Kimmy, I can't tell the difference." And then we started smart routing it and suddenly this open source started to figure out reasoning and the gap has suddenly closed this year.

And if you think about what — well, we've been thinking a lot about it in DNA — a huge amount of DNA, all right, is not invention, and you may not need sort of the most sophisticated agents for this. And another card that's turned over recently is some folks maybe have concerns with the ambition of the frontier models and maybe sharing their data, data leakage, and sovereignty of intelligence, and they're saying, "Hey, our company is going to choose — maybe we're in a regulated industry, finance, healthcare, HIPAA, FINRA, all kinds of different regulations — we need to have this on prem, domestically," and we'd liken an open-source version where we have a little bit more control.

My understanding was Jensen was like, "Hey, we don't even want to talk about these open source models we have because our customers — we're now going to be competing with Sam, Dario, Elon, Sergey — like do we want to be in that position?" So, but we do need some more champions here and it's open source so people can fork it — but that puts you in a more neutral position. That's right. We run today — we run GLM, we run Kimmy, we run the Qwen set of models, and we run OpenAI's models, the closed source ones. We run models for say GlaxoSmithKline, which they wrote and developed. We run models for our partner in the UAE, G42 and MBZUAI — that are their models that they designed. So we have a wide variety.

So sovereignty is a trend. Sovereignty is a trend. And I think the government's actions with regard to Fable and 5.6, where they said, "Oh, whoa, let's think and then we can act" — I think particularly here in Europe was a bit of a wakeup call. And when you saw this going down, there's a layer of partisanship in our country right now. It's pretty fervent. Dario is pretty explicitly, you know, not part of this administration — they've been very adversarial. Both sides have admitted that they're starting to work it out now. So it's hard, I think, for us not being in the room with these parties to understand what's partisanship, what's games here. But do you believe that what they released was truly dangerous for cyber warfare, for cyber attacks?

Have we checked the infrastructure of the country, like of the NSA? Have we checked the infrastructure? And can you give us two or three weeks to patch any obvious holes that are found? This doesn't seem to be an unreasonable thing for the government to ask. But we — in this very polarized time — put on top of it, "well, oh my god, it's President Trump doing it." And then you have to think, well, what if it was President AOC or President anybody in between the two extremes? I think the polarization hurts a great deal. It hurts clear thinking. And in fact, what I found is that the people in the government are trying really hard, the rank and file are trying really hard and this is moving fast.

You know, I think not only are they racing hard, but they're inventing this as they go too. There's not a playbook. They're inventing — we say, "Oh, just put on guardrails." Well, they have to design the guardrails, and the guardrails have an impact. One of the things that Fast does is it makes the guardrails less painful. And so we discovered that in the last six weeks — the very guardrails can add time and make it feel slower, and so fast ships like ours can really help that.

I think we can also know that there will be a massive data leak. Of course, we know this. It's like Warren Buffett talked about the reinsurance industry — you know something bad's going to happen, you don't know when, but you got to save up for it, you put money away for reinsurance. But there will be a tornado, there will be a massive earthquake — we know this and we can do our best to plan, but there'll be a massive breach, and we have to steel ourselves in advance and think about the right response at the time. But even knowing that there's some unknown unknowns is a useful place to start.

📋 한국어 요약 보기 →

Which, you know, if you start thinking about AGI and super intelligence — they're just definitions, but they're important definitions I think to kind of keep in mind because they're waypoints.

I mean, you think about that — any period of time, sort of 10, 15, 20, 30, 40, 50 years ago — any definition we would have previously put forward, we've blown past it. And so which goes back to our previous point of, like, do we know the questions to ask? Twenty years ago, science fiction authors had their say, and we answered all their questions. If they were to look at this today, they'd be like, "Well, I'm out of questions. Sorry." And that's where sort of listening to people who sound sometimes like they're on the fringe — when Ilya was talking eight or 10 years ago about the need for safety, you're like, "What?" And dead right. When Elon was talking about building rockets and driving the cost to near zero of a launch vehicle, you're like, "What?" And there it is.

Talk a little bit about recursive and then the road to super intelligence. And do you have a way, Andrew, that you think about super intelligence and what it will mean for humanity and how we will define it and how we'll experience it?

You ask it a question, you learn from the results, you ask it to do it again, the results get better and more information is added, your answer gets better. You ask it to do it again, it covers more material. And these sort of loops are producing not a little bit better answers but vastly better answers. And that is enormously powerful because we don't quite know where it ends — you keep throwing compute at it — how much better does the answer get? We run out of tokens or our budget, but holy cow, I mean, when does the exponential stop, or does the answer keep going up and up and up to the right?

And when are the problems no longer sort of intellectual problems and they're now people problems? How to organize people to get done what the AI asked for. As you know, in running your company, a lot of your problems aren't hard intellectual problems, they're people-working-together problems. You spend a lot of time as a leader spraying WD40 on your team so friction is reduced.

How do we get behavioral insight from AI? And I think that's some of the things the world models are going to bring us as they begin to watch human behavior. When these things jump off the screens and they're in the real world and the recursiveness starts — not trying to solve math problems, humanity's most difficult ones, but hey, there's an incredible world out here, and here's the Palace of Versailles — you're just like, now we're like, "make me a new version of Salesforce," and we're like, "hey, you know what, I'd like a Palace of Versailles, I've got a hundred acres somewhere out Texas or Nevada, I'll just send a thousand [robots] out there, make me the Palace of Versailles."

Sounds fantastical, but the Palace of Versailles would seem fantastical to people who lived a thousand years before it, and it was fantastical I think to the people who built it — even to the builders, I think they were awed at it as they built it. They're compounding recursive learning. In all these large projects, often there were families who were specialists who apprenticed under your father, your uncle, and when you had a project that took 50 or 70 or 100 years, you might have three or four generations of the same family, the same stonemason family working on the same structure, and passing on the learnings, new innovations — which is what we've modeled with this new [technology].

I think the problem with human learning is it often moves at the pace of a generation — like elephants and other large mammals, we don't have generations but every 15 or 20 years. And if you want to move really quickly across generations, you want them happening more like drosophila, like fruit flies — you want two a day. And I think that what we're getting is that equivalent in AI. We're getting sort of learning so quickly over the equivalent of thousands of generations.

Unlimited energy, unlimited calories, unlimited knowledge, unlimited education, unlimited housing — and how we do it. Imagine we know how to teach children and we don't do it right. Aristotle was a tutor to Alexander the Great, Socrates was his tutor — we know that if you give a child a tutor and the tutor modifies the teaching for the child, they learn better. That's not how we do teaching — classes, factory farming. We teach to some sort of mid-level. Imagine if we built agents that taught children in their way of learning. And here's a way we've been doing it the same way for a thousand years, and during that entire time we knew how to do it better and we chose not to. So as long as we're sort of thoughtfully and fairly writing the good and the bad, I think it'll come out — and I think the ledger is heavily weighted towards abundance. It'll create massive abundance.

📋 한국어 요약 보기 →

Robin Rombach is the co-founder and CEO of Black Forest Labs. You are based in Germany, in Black Forest, which is a city — in Germany. A mountain range, actually. Where you grew up. And you are working on open-source image and video models. You worked at Stable Diffusion for a little bit — cut your teeth on that, and you're known for the open source model Flux, and maybe also for some closed source models. Tell us about the business of Black Forest Labs. What is the business and what is the goal?

One quick addition — we are based in the Black Forest, it's a town called Freiburg, and in San Francisco. We started a company two years ago. Me and my co-founders, as you said, we've worked on stable diffusion in the past. Before that we invented an algorithm called latent diffusion, which is basically the fundamental algorithm behind all of the generative models that are being deployed for image generation, video generation, even physical AI now.

And then, built on top of that, we built Stable Diffusion, and then on top of that, the generative models that we are developing today — and of course the technology has advanced, but we are now tackling models that are really made for understanding the whole world around us — multimodal visual models, pre-trained on images, audio data at the same time, and we are now entering a new paradigm which is combining that with something that's called action prediction, such that you can actually use the same model to make images, to make videos, to make audio, and to predict actions — which means you can ultimately deploy it on a robot in the real world.

So from the image to the video, the audio, and then eventually the real world with robotics and a real world model — because if you can make the image, and you can train the model, that means by default you understand the world. In order to make a video of the world you have to understand the world. I think that's a really good way to think about it. There's like these complementary forms of intelligence ultimately — there's intuitive intelligence and then there's a deep reasoning layer. Now ultimately you need both, for a kind of complete form, and you need them to interact, and I think we've been approaching it more from the intuitive side. Images is a very natural way to approach this whole field because it's not as computationally intensive as, let's say, video — but now I think we're combining it, it's converging into a multimodal model.

And with these models and the training, there's kind of been a limitation in creating videos and creating images where the criticism of generative AI is it's a bit of a slot machine — I give a prompt, it gives me something back. But how did it come up with that? The training data — but maybe I want a different style, maybe I want a different color, maybe I want a different aesthetic. Has that problem been solved? And do you actually understand what's happening when the image is being made under the hood?

Open source is kind of having a moment right now. We've been discussing it on the podcast a whole bunch recently, and people are also talking about sovereignty. You have companies that own incredible IP libraries — I mentioned Star Wars before, Disney owns an incredible library. What would your advice be to a company like Disney? Should they take your open source software, train their own models or work with you to train their own models to control it and then say "this is our IP"?

I think the most interesting use cases of this — if you think about content creation — is in generating something, making something that hasn't been there before, right, that's a fundamental interesting aspect of this technology. And then when it comes to IP, what we implement for example on our public facing tools is you cannot generate certain IP with these models, and I think that's a sensible approach, and then yes, we do work with certain IP holders to develop models together with them, some of them based on our open source models, some of them based on our more powerful proprietary models.

On the one hand, we're always looking for researchers who have experience in large scale model training, experience in diffusion model training, flow matching training. We're looking for engineers who want to be working with the customers to develop these customized physical AI solutions, or for example with an IP owner to develop these models jointly with them. We are looking for engineers who have experience in large scale compute infra, managing that, and making sure that the training runs run smoothly, that we maximize our MFU. We just crossed 100 people. We're hiring in Germany and in San Francisco.

📋 한국어 요약 보기 →

Let's talk about video. There's an announcement that you're working with the greatest director of all time, or living director, Martin Scorsese. Fantastic. But in a movie — this promise of being able to make a movie in which the camera angle, the sound, could be something that a Martin Scorsese would be proud to release to his fans — how close are we? And maybe tell us a little bit about this partnership, the technology being able to make an actual movie like Goodfellas, or a scene from Goodfellas, versus where it is today, where you can make interesting five or 10 second clips and then maybe people struggle making 10 of them and then they use some post-editing software to put them together, but you immediately understand this is not that. It's not a movie, it's AI slop, it's kludgy, it doesn't pass the uncanny valley.

So you sat in a room with Marty Scorsese and showed him your tools. Exactly. And what was his reaction? What did he key off of? What was the thing that he found most inspiring or interesting? I think it was really this idea of — he has clearly a vision in his head of a scene or a scenery where maybe a new movie will be shot, and he's trying to explore that, and we basically looked at the scenery of a village in Eastern Europe somewhere, and he was describing it. We saw some outputs, we iterated on the outputs. And ultimately I think that's what he said in the end — like getting the mental picture of something out of your head and communicating it in a visual way by making these images or the series of images is something that just makes it easier to communicate and convey an idea of what is actually in your head, and I think that's one of the very interesting and powerful ways to use this technology.

The brainstorming production level is so obviously a huge win — you can paralyze your brainstorming basically. And they have an analogy for this — they do storyboards, and some of the great directors, Ridley Scott of Aliens and Gladiator was known for making his own, I also believe Spielberg was also like to sketch Raiders of the Lost Ark and some of these. George Lucas was known for collaborating with many amazing artists, even making miniatures and storyboards for the Star Wars franchise.

And what else are people using the technology for? I understand there's a Bitcoin movie coming out — instead of using a green screen. I was talking to the woman who played the actress who played Wonder Woman, and she was telling me she just did a Bitcoin movie and they did it on a sound stage without green screens, but all the actors just worked in a sound stage, and then all of the scenery behind them was being done by generative AI.

Yeah, we see some use cases like that in production. I think high-end film production is kind of like one of the most demanding use cases, and I'm glad that it's being explored. It's important to see that this technology is on a trajectory and it's improving rapidly. If I look back at where we started a few years ago, when I was doing my PhD in this field, the only thing that you could do was images of 64x64 pixels — now you can do multi-input videos at a high resolution, but it's not going to stop there, it's going to continue to improve, and I think it's going to unlock even more of these high-end use cases.

I think the most interesting thing I've seen in this regard is fan films. So there's a category before generative AI — fan fiction, people would write their own Star Wars story. Then there came fan films where people would dress up as Jedi Knights and record their own films. And George Lucas said, "As long as you're not doing it commercially, you're not selling it, I give you permission to go make Jedi movies." Now, people are taking the stories that haven't been told from the Star Wars universe and they're recreating them using AI, and for the fans, they're becoming quite popular on YouTube — Star Wars Stories Untold is, I think, the biggest one, getting millions of views per video already.

No, 100%. I think if you find a model that works for the IP owners but then also can enable these super creative customization use cases, I think that's great. Like for myself, when I read a book or watched a movie, I had so many ideas how it could be done differently — this is so nice, that you can actually enable people to visualize these ideas.