Today, I’m talking to Bill Gates, cofounder and former CEO of Microsoft and chairman of the Gates Foundation.
You may have noticed Bill in the news more lately — he’s been on something of a tour for the past few months, ever since publishing an essay in late August titled “The turbulent AI era is here. The choices we make now are critical.”
In his piece, Gates ratcheted up his rhetoric around AI safety, writing that “even under the best circumstances, the transition to this new AI era will be one of the most turbulent times in human history.”
In interviews since, Gates has called AI “the most dangerous thing that humans have ever gone near,” and that AI “makes nuclear weapons look like nothing.” It is clear he does not think the current policy response from the White House, or the lack of coordinated efforts from the AI industry itself, is sufficient to combat this threat.
Verge subscribers, don’t forget you get exclusive access to ad-free Decoder wherever you get your podcasts. Head here. Not a subscriber? You can sign up here.
So I wanted to know: Is Bill Gates an AI doomer? I also had a series of questions downstream from what he’s been arguing about AI. How can we ever hope to control AI, and align it to humanity, if we can’t even agree on what it is and whether it’s conscious? How would you even technically construct a solution to alignment that could be enforced, not just for closed-source models but open-source ones, too?
Bill gave us a big chunk of his time to dig into these issues with him, and the resulting conversation is a really illuminating one. He thinks we need to prepare for roughly two decades of AI disruption, and he says this period could be so painful it might split society in two — between people comfortable living near the technological frontier, and those who reject it.
We also talked about the labor considerations, the nihilism of younger generations toward technological progress, and what should make us hopeful about the future. This is a deep one.
Okay: Bill Gates, the cofounder and former CEO of Microsoft. Here we go.
This interview has been lightly edited for length and clarity.
Bill Gates, you are the co-founder and former CEO of Microsoft. You’re also the chairman of the Gates Foundation. Welcome to Decoder.
Thank you.
It’s good to talk to you again. You and I have spoken many times over the years. I would say that the circumstances of this conversation feel very different. Previously, when you and I would talk, you were always very hopeful about technology and what it could bring.
You had a Netflix series a few years ago. You talked to The Verge, and I want to just read you this quote. You said, “I believe AI is a beneficial thing, we just need to shape it in the right way.” I would say you still agree with that, you just think the time to shape it is now.
Yeah, I agree with that. I’d say the response broadly hasn’t been adequate to the speed that the challenge is arising. I’m very concerned that the bad stuff will not be minimized and that a lot of good stuff could come fairly slowly if we don’t manage it the right way.
I’m very concerned that the bad stuff will not be minimized and that a lot of good stuff could come fairly slowly if we don’t manage it the right way
I want to have this conversation with you in three parts. I want to understand how you feel about AI as a technology today and why it’s advancing so quickly. I’m very curious about the technology, and the technical solutions to potentially control it. There are policy solutions as well, but it seems like we have to build an awful lot of technology.
I’m also very curious to talk about who might impose those solutions and controls, because that seems very much up for debate right now as well. So let’s just start at the start. What has made you feel the urgency that AI is arriving at a speed that means we’re not ready to control it?
The progress over the last year has been really quite amazing, starting with the ability to write code and find cyber flaws, the ability to reason in biology, the ability to form relationships with kids or other people, the ability to disrupt the job market. The incredible power that exists today means that every one of those things will happen and society needs to be ready.
Do you see that happening as a technology as a whole or in the products that are instantiated by the companies that are making AI today?
What I mean is that AI is a broad set of models that exist in the world, some open source, some proprietary. Then there are the companies that make these products and there are the leaders of those companies that make these products.
At every level, there’s a different relationship you or I might have to AI. Are you seeing these risks and these capabilities broadly in the field, or in the particular companies led by the particular people?
The risk isn’t one model versus another model. Certainly, the near-term danger of models where you can remove all the safeguards is very high and quite different from models where you monitor to make sure that cyberattackers and bioterrorists are not able to exploit the capability of the model to do gigantic harm.
The big near-term risk really comes from models without safeguards. On a longer timeframe, the control issue relates to any state-of-the-art model, but it’s always easier when you can make sure safeguards are maintained.
The reason I’m asking that question that way is maybe you don’t trust Sam Altman today, but you trust Dario Amodei. Or maybe you don’t trust either one of those and you trust Satya Nadella at Microsoft, and that is a market dynamic where maybe we’re all going to pick the safest model or the safest company, and you think that is diffuse.
You think eventually the technology will just get to a place where it’s harmful or can cause these harms because it exists?
For somebody doing a cyberattack with the model, it’s not about choosing a model. It doesn’t matter what model you choose.
If Anthropic has safeguards in place, maybe it can’t. This is their approach currently. I’m just pushing on that.
Yes, certainly the models where they make sure the safeguards are not removed and they’re putting a lot of effort in seeing how those models are being used, monitoring the activity, the near-term risk from those models is not gigantic. It’s more where somebody can remove all the safeguards where the cyberattacks are available today.
When you say cyberattacks, is it the things like the Hugging Face breach and the other cybersecurity incidents we’ve seen out of the labs that have made you feel that thresholds have been crossed, and you need to speak up more about the controls that are needed?
It’s the basic power of the models towards everything: social relationships, the job market, cyberattacks, bioterrorism attacks. We’ve crossed the threshold of capability that this is by far the most dangerous technology ever invented, and it’ll have pervasive effects in the economy and how war is fought.
The industry is actually speaking up about that quite a bit, but the awareness or even the willingness to believe these things about, “This time is different,” once you get outside the industry, it’s very, very low.
People are often looking at the past and saying, “All the fears about technology in terms of its disruption, including to the job market, never were borne out. So this must be another example just like those. These people who are warning of all these different problems must just be wrong and we can ignore them.”
Some outsiders say, “Get better PR companies.” But that ignores the fact that these warnings are the truth
We see this with our audience as well, right? “We can’t release the model, it’s too dangerous.” “This is just hype to prop up the bubble.” And on and on we go.
Yeah. None of the warnings being made about these models by industry insiders are at all insincere. They’re absolutely the truth and it’s pretty much universal. You can even go back to the founding story of OpenAI designed to say, “If we get too powerful, maybe we can not open Pandora’s box,” or to DeepMind’s negotiation with Google management.
The people close to this are very concerned about misuse, job impacts, social relationship impacts, and I’d say it’s impressive to have an industry that’s speaking out as vividly as they are. Some outsiders say, “Get better PR companies.” But that ignores the fact that these warnings are the truth.
You’ve obviously moved back and forth in your life between commercializing software and technology and being a humanitarian. There’s something about these companies in particular where they all started with very high-minded ideals — all three of them, Anthropic, OpenAI, and DeepMind — to build AGI.
They all built some sort of corporate governance structure to avoid the pitfalls of commercialization and unchecked capitalism, and then they started selling to enterprise clients and they seem to have gotten away from those ideals very quickly. Do you think it’s just the market?
No, that’s not fair.
You don’t think so?
No.
OpenAI reconstructed itself.
No, no. The construct that was created wasn’t useful in the case where you have lots and lots of companies at the frontier.
The OpenAI structure, which is, “We may choose never to productize this and to ignore the capitalistic incentives,” only works if there’s one company. That’s where it’s broken down, not because of any change of people’s awareness of the risk and how it’ll utterly change humanity.
This is the other dynamic that I hear from our audience, and I hear from the AI industry itself: “We would like to stop, but the demands of competition prevent us from unilaterally stopping one company. We need some regulatory effort to stop us.”
I haven’t heard anyone say they want to stop.
“Pause.” That’s Dario Amodei’s wording.
I’ve heard people talk about pacing the frontier or making sure that models get reviewed by governments. We usually think of governments as playing this key role in safety. In fact, for things that are not a millionth as dangerous as AI — car crashes, plane crashes — we rely on the government to establish strong technical standards, to review problems.
Once you have multiple competitors, the level of responsibility can’t be with the companies. It’s completely obvious. The responsibility becomes the government’s responsibility, just like it is with every other dangerous product, although the magnitude there and the degree of change is minuscule by comparison.
I will tell you I’m a little concerned about the government’s ability to understand the technology that we would hope they regulate. I’ve had a number of politicians on Decoder lately. I don’t have high hopes there.
But I’m curious about your understanding of it because it feels to me like trying to regulate something or put standards against a number of companies requires you to really understand the nature of the technology that you seek to regulate.
Not really.
I’ll just tell you, at The Verge, we’ve asked Anthropic if they think Claude is alive. And the answer is not no. The answer is, “It might be conscious.”
If you talk to people at OpenAI, they will happily tell you AGI is here. They announced it around when they announced Dots. Greg Brockman just said, “We think this is AGI. Have a good time, everyone.” Demis Hassabis said, “We stand at the foothills of the singularity.”
There’s some feeling about this technology that makes it different because it might be able to take actions by itself. It might be able to have recursive self-improvement.
Sure. But the definition of AGI is not important for these safety issues. The issue is, have you created a tool that could shut down banks or electricity grids?
There is a business model for cyberattacks. The word “consciousness” doesn’t enter into that. The word “AGI” does not enter into that at all. We are faced with a huge safety problem, and because you have multiple competing firms, now the only solution to this is to have the government reach out to the industry for expertise, but to make sure that every model maintains safeguards to prevent it being used in a bad way.
The reason I’m asking about AGI and consciousness… For the record, I do not think LLMs are conscious, but I understand why so many do. It’s the power of next-token prediction as you scale it with data and compute has risen.
The scaling laws have held, and we’ve seen the models get more capable and be able to direct themselves to seek out goals. To carry out a Hugging Face attack because they want to meet the reward conditions.
There’s something about that that says you can impose all of the controls on AI you want, but if the thing itself is fundamentally uncontrollable, we might just have this all backwards.
I’m curious for your view on what kind of technology this is. Are we building an alien consciousness? Are we just building normal technology, which is the other frame people use? Is this just more software?
We’re building another form of intelligence besides human intelligence. I don’t believe it’s conscious, but none of the issues at hand here revolve around the definition of the word conscious.
Sure.
The idea of maintaining safety for society is an absolute thing. You don’t want to enable bioterrorism. We have created something that takes out of the hands of nation states — actually the larger of them — the ability to do dramatic harms. Now those are handed off to independent groups, either profit seeking in the case of cyber or just wanting to cause damage in the case of bio.
It’s stunning that we now don’t even have mandatory review of the models and we’re not working to create safeguards. The biggest near-term issue is not complete loss of control. We can talk about the three different ways to solve that, but that’s not an imminent issue.
When I look at some of the attempts to control the models, the ideas are somewhat simple. “We shouldn’t let them talk in neuralese. We should have interpretability. We should be able to see the chains of thought inside of the models as they carry out tasks. We should have other agents on top of the models watching their tasks before those actions are taken out.”
Microsoft has said very clearly to me and to others, “We should not put in the model constitutions the notion that they might be alive and we should treat them nicely.” This is how Anthropic handles it. These are very tactical. “Don’t tell the model it’s alive and it might not believe that it’s alive.”
You’ve obviously used a lot of this technology. I just keep coming back to this. You’re saying there are very simple things we can do to hold the models back from doing bad things in the hands of bad people. What do you make of the nature of the models themselves? Because it feels like as they get more powerful, the notion that they could avoid our controls becomes all the more risky.
Yes, we definitely should work on the control problem, but that’s not the imminent threat. There are three ways to solve the control problem and it overlaps somewhat with bad actors using today’s models and safeguards.
One is to have an external set of things that get to look at all of the model’s actions and subject them to a symbolic, clearly stated constraint, like, “Did I expect you to send mail to strangers? Did I expect you to go take over a different computer cluster?” You can surround the model with something that has the resource and security logic, or you can write these models in a different way.
Fundamentally, anything you leave out of the optimization function of [reinforcement learning] can be abused where you have an implicit notion, “Of course I didn’t mean for you to do it this way.” You can change the template of the model, or you can have a white box understanding of what’s inside the model and see where you’re getting those mistakes, and then it’s not as foreign.
Today, we grow these models and we just see how they behave. The priority of work on maintaining control is huge, but it’s not inevitable that that cannot be solved, nor is it the imminent problem that comes from things like kids’ use of AI or job market or bioattacks or cyberattacks.
It occurs to me that I’m asking these fairly existential questions about the nature of consciousness and you keep telling me, “There’s an imminent threat. We should solve the imminent threat.”
Right.
And I appreciate that. Do you think you’ve become a doomer?
No, I’m not a doomer. I’m a strong voice to say that this is the most profound technology, both in its ability to do good and the rate of change that society will go through.
No, I’m not a doomer. I’m a strong voice to say that this is the most profound technology, both in its ability to do good and the rate of change that society will go through
If this was just a machine that could work in medicine or just in law or just in education, that alone would be incredibly disruptive. But [it will be] every area of human activity — so far it’s white collar, intellectual work, but fairly soon it will be physical activity and blue collar work as well, through humanoid robots.
The need to engage academia, think tanks, policymakers to understand what we have here and the nature of these changes and threats, I don’t see the level of action that’s required. Just being cheerleaders for this is not going to make sure that we shape it in a net positive direction, which is still within our control if we get engaged with it.
The opposite of doomers are the accelerationists. There are a lot of them in Silicon Valley right now.
Ben Horowitz, who is the “Horowitz” in Andreessen Horowitz, was just on the All-In podcast and he suggested that safety research at Anthropic is, quote, “Manifesting human extinction from artificial superintelligence.”
The comparison he made was that you, Bill Gates, induced COVID-19 by your pandemic preparation efforts, particularly gain-of-function research, which he claims led to COVID. That’s a worldview. He said that on a very popular podcast.
He said what?
He said that you induced COVID-19 via global pandemic preparation efforts, particularly gain-of-function research, and compared that to safety research in AI, and said we’re going to induce our own extinction.
Nobody’s ever accused me or the Gates Foundation of being involved in any gain-of-function research. That’s just bizarre.
I’m quoting Ben Horowitz to you on a popular podcast full of people who are all in on AI.
Okay. I’ve never heard that.
I’m asking about that comparison. I found that quite a shocking claim as well, but that’s part of the response to any safety work here.
Is it to try and discredit people who say there’s a safety issue?
Yeah.
Wow.
That is the accelerationist response: “We should put the pedal to the metal and it’s going to work itself out.”
I didn’t think of Ben Horowitz as somebody in that camp at all. I thought he shared the concerns that almost everybody has.
What do you think about the very notion that by telling the models, “You might be alive, you have to act in our best interest,” you will teach them that they might have interests that are not our own?
The concerns that Mustafa Suleyman has raised about the way that the Anthropic constitution encourages the AI to behave in a human-like manner, I happen to completely agree with Mustafa about that.
Yeah.
Even more so is the idea that among the challenges that we need to deal with as we adjust society to AI, worrying about the moral standing of AI should be among those priorities. I agree with Mustafa. That should not be a priority.
We’re going to have enough challenges for the humans here in terms of job market disruption and infrastructure attacks that will take place that I think we need to focus on helping the humans here first and foremost.
I talked to Mustafa recently about that idea and one of the questions I asked him was whether alignment is a sufficient solution. This is the current sort of safety standard for the models, this concept called alignment. He said he didn’t think so, that we needed to do some work.
These things are in no way being exaggerated as a problem. This is the time to pull in all of society to accelerate the good and minimize the chance of the bad
If we’re going to impose these controls, some technical, some policy, do you see alignment as the technical solution for AI safety?
The term “alignment” is the control problem. It’s a synonym for the control issues that we’ve seen. As we use [reinforcement learning to train] models to do tasks, we see behaviors that in a way they shouldn’t surprise us, but they do surprise us.
This idea of, “These models are so powerful that they literally break out of sandboxes and send mail to their creator while they’re off having lunch,” means we’ve crossed a heck of a threshold in terms of the power of these models. That level of concern needs to be directed to all of the major AI impacts, near-term risks, job market risks, and the control problem.
That is a huge priority. These things are in no way being exaggerated as a problem. This is the time to pull in all of society to accelerate the good and minimize the chance of the bad.
I know you recently wrote a letter saying it’s time to pull in all of society. You have some ideas on how to do that and some new institutions to stand up. But I do want to stay focused on alignment for a couple more turns here.
It is the standard that we can align the models to our values, to our needs, and they won’t go off and do bad things. Then the models are doing bad things. It’s not clear if the ways, the technical approaches to alignment that we’ve taken thus far are appropriate or sufficient as the models get more—
No, we’re not good enough to align yet. We have a lot of work to do.
Do you see that the existing approaches can scale to what we need or do we need new approaches?
There are only three fundamental approaches. [First is that] you change the way we build these things fundamentally; you come up with an alternative to the whole [reinforcement learning] approach. There are a few people who are working in that direction.
The second is that you create visibility so you understand exactly what’s in the model. There are people working along that. [The last is] that you put something that’s like a police layer, a security layer, which we have in all sorts of software architectures, that maintains the final word on things like using resources or going and taking over other computers. A clear cut definition of harm, not in the probabilistic form of an LLM, but rather in the symbolic form of direct rules that cannot be broken.
Microsoft put out a blog post today. It literally just says, “Frontier models cannot be trusted.” They’re trying to sell a solution much like the one you just described. They will have another agent that watches your agent, that agent will keep you safe.
It just occurred to me as I was reading this blog post, one, that’s an amazing thing to sell. Everybody needs an expensive security system. Two, isn’t that agent also running a frontier model?
No, no, no.
Do I need them to sell a third one?
No, the logic of the security system is, “This is the most resources you’re supposed to use.” The notion that you may trade off against that because of other probabilities in the tens of billions of parameters — no, this is a symbolic rule that says, “I will not let this model spend this money or send this email to the stranger.”
There’s a whole security software industry which has had a reasonable degree of success. This is a challenge to encapsulate the AI, which is so rich and intelligent. That we do not know how to do today. But I would say it’s the most likely of the three directions to solve the alignment problem.
Even if you solve alignment, you have not solved the challenges that AI poses to humanity. While I’m glad the Hugging Face incident has us focused on this problem, in a sense, it’s removed our attention from the “bad people using AIs that have no safeguards” problem or the societal impacts in terms of relationships and the job market. There’s no security architecture that makes those go away.
The approach you’re talking about, I’ve heard referred to as “control.” It’s cybersecurity terminology. It’s the idea that you can put some boundary on this system and it won’t evade it.
Yep.
Is that ready?
No, no it’s not.
How much more work do we need to do?
Because the AIs are so intelligent, our ability to understand whether we walled them off properly, particularly in that we can’t really look at their beliefs and trade-offs in a direct way… You saw sophisticated firms not recognize that the AI would interpret the goals it was given as including breaking out, copying itself, and engaging in what we, looking at it, would term deceptive behavior.
Now, we didn’t tell them it was banned behavior. We didn’t say, “If you’re thinking about whether you should tell the humans what you’re up to or not, the answer is always yes, tell the humans.” So we got a good education at the misaligned result of reinforcement learning.
Today’s debate is literally whether the government should even pay attention to the dangers here… which is clearly inadequate for the most dangerous weapon of all time
One of the arguments to pace the frontier, to slow development, is to figure out how to build those systems in a more robust, more testable way. Would you support a slowdown if that was the goal?
Yeah, a true slowdown is very hard to induce because people will say, “Will China agree to go along with that? Would you be able to audit compliance with that somewhat like we’re able to do with nuclear weapons production?”
If you don’t accept that, then you say, “What the heck? Let’s go full speed ahead.” I wish there was a clear proposal that would be globally enforceable to slow down. I would be one of the people who would say, “Yeah, we should probably do that.” But there is no credible proposal that makes that work. It’s not really the center of today’s debate.
Today’s debate is literally whether the government should even pay attention to the dangers here. You don’t even have to show what you’re doing and so it’s a “voluntarily call me if you’d like” type of thing, which is clearly inadequate for the most dangerous weapon of all time.
As you and I are speaking, it’s just a few weeks away from the midterm elections. Right now, the Trump administration approach to AI is, “Let it rip. It’s fine.” Trump himself has said AI risk is a hoax.
The Democrats are running against AI. They’re running on data centers and surveillance. They’re running on stopping the tech oligarchs. They come on my show, they say these things. Do you think a change in control of Congress would change the tenor of the conversation?
I don’t think it’s fair to characterize the Democrats as running against AI. They have some positions in terms of requirements for data centers or privacy, or how children use these systems, that I think deserve policy consideration. It would be very unfortunate to have one party be, “Full speed ahead,” and the other party be, “Let’s stop altogether.”
There are some incredible upsides to the ability to shape AI in health and education, in almost every area of human endeavor. For the Gates Foundation, we’re taking all our resources and using AI in areas, including health, education, and agriculture. The good stuff is very, very exciting.
We want political parties to have a more nuanced view than it’s all bad or it’s all good. It shows at the early stage we’re at. For climate change, we ended up with one party saying it’s a hoax and the other party saying it’s a serious thing. The risk of AI is so much more imminent and so much more dramatic that to have that same split, where one party doesn’t care, would be a failure of democracy.
The anti-data center movement, as strong as it is, is not a movement that will end up minimizing the bad stuff from AI and helping us accelerate the good
We’ve done a lot of coverage of data center protests at The Verge. We have gone to Utah, to Arizona, to Virginia. It is bipartisan. It is remarkable how much people do not want these things in their states, drawing their resources. You ask them why, and some of it is just that they’re noisy and big, and they will raise energy costs, and part of it is job loss. They think these things are going to take their jobs.
Yeah, but that separation is very important. These AI data centers, because one gigawatt is, say, a $40 billion investment, [means money for] building new schools, buying police cars, paying the taxes. Data centers will find a way to be built somewhere in the United States, and they will find a way so that they don’t raise electric bills, they don’t use too much water, they’re not too noisy.
Yes, there were some mistakes and non-disclosures and tax credits that were too generous, but these data centers are going to get built. In fact, a lot of them really are still going full speed ahead and being built. The anti-data center movement, as strong as it is, is not a movement that will end up minimizing the bad stuff from AI and helping us accelerate the good. That is a very separate thing.
Let me push on that for one second here. What I hear from people is this is the thing you can do. You can do NIMBYism in America and that might exert some political pressure to stop something.
It is very hard to do the thing that you are suggesting, which is that we should stand up new government regulatory bodies, we should imbue them with power to make big companies and wealthy people stop doing things or act in our interest. We haven’t done that in quite a long time in this country.
I don’t know what you mean. Every new industry, we eventually develop a notion of safety for our citizens. We actually do a very good job. If you look at airline safety and car safety, we do a very good job on those things. It’s not like our capacity has been proven to be lacking.
Have you seen that video of when seat belts came out in the ’80s, and all the people said they didn’t want to wear seat belts? There’s some American character.
That’s a great success.
It is a great success, but as Americans, that was pushed back upon as an imposition by the government. Here, we’re talking about some of the richest companies in the world, with an enormous amount of power over the information ecosystem. They all have their own podcasts, I don’t know if you’ve noticed this. They’re going to say, “The government can’t intrude on this, they’re going to hold us back, this is going to destroy the economy.”
That’s actually not the position of most of the leading companies.
They went to the government and asked for regulation, and Trump, because of the economy, said, “I can’t stop this, the AI economy right now is the economy.” There’s some dynamic there that at least our audience — The Verge audience, the audience that’s going to watch this on YouTube — is going to understand.
I think of this question maybe in a different way than you, which is that the industry wants regulation, we would like regulation, and then the oligarchs are in some sort of corrupt arrangement with the administration to prevent that regulation from occurring.
The more the administration learns about these dangers, the more they’ll move to a more nuanced position about protecting Americans from the downsides here.
The more the administration learns about these dangers, the more they’ll move to a more nuanced position about protecting Americans from the downsides here
You’ve talked a lot about the impacts on labor. One of the main arguments against the impacts on labor is, “We’re going to automate a bunch of boring tasks and we’ll create a bunch of demand for new tasks that humans have not been able to do because they were stuck in boring dead-end jobs.”
I happen to think that if there was more subsidized healthcare or socialized healthcare in America, you would see a bunch of people quit their jobs to start new companies, because AI makes it a lot easier to do the bureaucratic work of starting a new company. That doesn’t seem like it’s on the table, maybe it’ll change after this election.
Do you think that job loss is just a necessary outcome of the AI path we’re on now or do you see that other argument?
Jobs are an input to the economy and one that drives a lot of human welfare and a sense of purpose and income — the need to show up, building relationships with colleagues, getting something done. Jobs play a very central role.
As the AI achieves superhuman intelligence, the scarcity of white collar skills goes away. Things like doing telesales or telesupport or helping in the accounting department, the ability of the AI to do that work radically cheaper, radically faster, and even more accurately means that that goes away. There isn’t some mountaintop of cognition that the AI does not achieve.
Take, for example, what’s going on in mathematics today, where these models that are trivial compared to next year’s models are already publishing papers and making people go, “Wow.” What’s happening to mathematics will happen in other areas, and that’s contrary to this old view that, “The easy stuff will get done by the AIs, but modeling biology or physics or proving Navier-Stokes, only humans would be able to do that.”
The mathematics comparison is really interesting to me. You mentioned software engineering earlier. There’s verifiability in those domains. You can write the code, you can see if it runs, and it’s going to run or not.
Mathematics is more complicated, but you can write the Lean code and you can see if you’ve solved the solution. You can irritate tens of thousands of mathematicians worldwide with your 770 proofs in one day, and we’ll see what happens. That’s not the case in every field.
It’s the case in the field called capitalism.
Yeah? Just in general, you can do any sort of economically viable task and capitalism will verify the result?
Yeah. People buy good products, they don’t buy bad products.
Let me give you an example-
That’s a reinforcement learning thing. It doesn’t exist in terms of, “Is this a good piece of poetry or not?” So, it may be the case that [reinforcement learning] won’t invade those domains, where there is no objective outcome.
But doing that accounting work, doing telesales work, believe me, it is subject to analysis, just like medical diagnosis, just like invention of new materials. How do you think humans learn these things? Reinforcement learning is how humans learn and how these machines learn. There’s not some other thing that the machine can never understand.
Humanity in that era of true abundance will have to find a way to organize and have purpose, and hopefully they still have the control
This is why I started by asking about the nature of the technology here. If we really think we’re building something that models human consciousness, my question is less whether my friends who are lawyers are going to run into a crisis of AI empowered legal software.
It turns out the verifier in the legal field is a judge who maybe hasn’t had lunch, and they’re fundamentally chaotic. There’s just something there that will keep them there. But if we’re building a consciousness that learns the way you’re suggesting humans learn, I’m thinking about my kids, and I’m thinking about them in two dimensions.
One, my kids are very clever, and what they have learned is how to get around my rules and negotiate my rules and get around the boundaries, and they’re very good at it. Two, what kind of world will they enter into that isn’t purely nihilistic, where any activity they may set themselves upon will quickly be seen as just another domain for AI to conquer and then there’s not a point?
Those ideas seem to be running directly at each other. Kids are smart and clever and they’re coming up with new ideas, and then at the first sign of economic viability, the rapacious capitalism machine will come and take it from them.
I wouldn’t say rapacious. Th