But it also shows how far-reaching simple goals can be. I think you are correct that there are a lot of unknowns and big uncertainties. I'm now going to hold up another book called Nanomedicine by Robert Freitas, instead of Nanosystems by Eric Drexler. We are genetically almost identical to those stone ax swingers. If the government stomps around, and the dangers that the government--it's one of those very rare cases where the dangers that the government will interfere too little rather than too much. In some sense it's somewhat healthy in that it can be a deterrent under certain settings. Russ Roberts: Let's close with a quote from Scott Aaronson, which I found on his blog--we'll put a link up to the post--very interesting defense of AI. What if the world was so abundant and automated that we could all just do what ever we felt like all day every day and never have to work? My dog does that and seems to live quite a contented life but what would humans do, why would it be different from my dog? I suspect there is an obvious answer but maybe there isnt. Would we all sit around listening to more podcasts and commenting on them, we seem to enjoy spending some time of our lives doing that at the moment, what if all the commenters were AI bots that we couldnt tell the difference between them and humans would that be bad? Russ Roberts: There is a scene in Schindler's List, the Nazis, I think they're in the Warsaw Ghetto and they're racing--a group of Nazis are racing. Colon: What is your answer? It's actually helpful. In 2018, they received the Turing Award, often called the Nobel Prize of computing, for their work on neural networks. I have never seen something like this before on LessWrong. At the end of the day, there are many foundational assumptions built into these arguments that are just not known. Some of them may be true, others not, but it all gets lost because theyre assumed to be true and the conversation goes into the weeds with jargon. Right? Why don't we just run around chipping flint hand axes and outwitting other humans? So, there's a couple of steps in that. While AIs dont have evolutionary pressure in the same way, they are iterating to improve and grow that intelligence (much more rapidly than human biology can). So, your hands are made out of proteins that fold up, basically held together by the equivalent of static cling, Van der Waals forces, rather than covalent bonds. Instead of backing up that extraordinary claim, he began by asking you, Why dont you already believe that? Thats the attitude of someone whose mind is closed to any opinions that differ from their own. It is possible for you to know that one of their estimates is directionally mistaken and to know the direction. The trends are very different. But, you know, understand it may not be thinking about it correctly. He argues that evolution is essentially an optimization process, where the optimization objective is survival, and that despite this simple / singular objective, we went on to have our own ambitions that are seemingly detached from our original goal, like going to the moon. Similarly, optimizing models for next word prediction can produce new, emergent capabilities as a result of getting really good at this task. One solution to the this problem, after all, is true comprehension of the ideas being expressed. Or it might be 'What's a good restaurant in this place?' It's hard to know because we don't know what goes on in there. And yet, humans, despite being optimized exclusively for inclusive genetic fitness, want this enormous array of other things. I think people often underestimate how weird and hard this is to answer decisively in the relatively straight forward case of other people. In response to Yudkowsky's recent podcast interview about how the robot apocalypse is nigh, one brave soul has barged into LessWrong and done the unthinkable: patiently explain, in detail and citing relevant sources from the established academic literature, why Yudkowsky is wrong about everything. So, I have this black box and I don't understand--I put in inputs and the input might be 'Who is the best writer on medieval European history,?' https://sullivankevint.substack.com/p/on-artificial-intelligence. Thought that is simulated in enough detail is just thought. Russ Roberts: Okay. However, I also think he's wrong. So, if you are exploiting a design flaw like this, I can show you the code; and you can prove as a theorem that it cannot break the security of the computer, assuming the chips work as designed; and the code will break out of the sandbox that's in any ways because it is exploiting physical properties of the chip itself that you did not know about despite the attempt of the designers to constrain the properties of that chip very narrowly. Eliezer Yudkowsky OP points out that, actually, he already did exactly that in a previous LW post. This seems obviously, but is actually a really tough question how do we know there are other minds out there? (Going to the moon is an extreme way to signal fitness.) There's an inscrutability to the current structure of these models, which is, I found, somewhat alarming. Next you say that it is not known whether current ML programs could even calculate the algorithm of general intelligence. All rights reserved. So this might be one more reason why AI researchers dont stop doing what they do. That's a really important point. And it looks around and it starts a sentence and then finds its way towards a set of sentences that it spits back at me that look very much like what a very thoughtful--sometimes, not always, often it's wrong--but often what a very thoughtful person might say in that situation or might want to say in that situation or learn in that situation. They wanted to show you that having a very high advanced level of civilization does not stop people from treating other people--other human beings--like animals. So, this is ruin on a colossal scale. I am in this world, but I dont think he is a very effective communicator. Agree with earlier commenters that there may be danger, and if so we just heard from the wrong Cassandra. In particular, like many software people, he vastly underestimates the difficulty of doing anything complex in the physical world. And also overstates the utility of intelligence. Being smart doesnt solve all problems. I don't think people actually understood the research program that we were trying to carry out, but, yeah. This is how Eliezer Yudkowsky would defeat you if I wanted to do that--which to be clear I don't. 'Oh, yeah. Maybe it'll murmur better than I do. We need to stop this until we have an understanding of how to constrain it.'. Isnt there a possibility of AI destroying humanity not by killing us but by making us feel worthless? I see them driving around the city every day, NOW. Some specific problems I saw: Early on was a lot of talk about comparing AI machines to biological pseudo-random genetic change leading to selection of some genetics over others. The present generation of LLMs are only being trained to produce language, but there are already people who are using the same evolutionary approaches to train similar models to act in simulated worlds. I think you think that piece is almost certain? Singularity Summit Those two goals seems to be rather innate to all living organisms which would be exactly expected by how natural selection works. If you ask GPT-4 to write you a rap battle between Cyrano de Bergerac and Vladimir Putin, even if there's no rap battle like that that it has read, it can write it because it has picked up the rhythm of what are rap battles in general. Just like humans aren't just reflexive, unthinking hand-axe chippers and other human-outwitters: If you grind hard enough on the optimization, the part that suddenly gets interesting is when you, like, look away for an eye-blink of evolutionary time, you look back and they're like, 'Whoa, they're on the moon. I'm not asking for that. Malice is implied by just wanting all the resources of earth to yourself, not leaving the humans around in case they create a competing superintelligence that might actually be able to hurt you, and just, like, wanting all the resources and to organize them in a way that wipes out humanity as a side effect, which means the humans might want to resist, which means you want the humans gone. Not just merely doing it in virtual space? No one designed markets even to start with; and yet we have them. So, I think that's generally a really good point to make--that, putting ourselves inside the head of the paperclip maximizer is not an easy thing to do because it's not a human. The answer I would give is that biology has to be evolvable. He is obviously highly intelligent but was not a clear communicator of his ideas. With more than 20 years of experience in the world of AI, Eliezer Yudkowsky is the founder and senior research fellow of the Machine Intelligence What do you do if you're around humans who can potentially unplug you? I think the easier answer would be someone will place these AIs into physical machine that then will become out of control. But if the gun was pointed at the entire biome of the planet you live on? or 'What is so-and-so's view of that?' WebHarry Potter and the Methods of Rationality (HPMOR) is a Harry Potter fan fiction by Eliezer Yudkowsky, published on FanFiction.Net. It is just that people cannot stop it. But, carry on. Just check their arguments about that. That's the concept of gradient descent. Next, thinking about how, these things are already showing signs of developing prodigious skill at things seemingly orthogonal to the training objective. This, again, is just like us. Do you think there's any way we can restrain this AI phenomenon that's meaningful? But again, I could not understand the way Yudovsky described the AI getting out of the black box, escaping the pulled plug. Eliezer Yudkowsky: At least one U.S. So we should not assume that general intelligence for machines will be constrained in the same way we think traditional machines are constrained. This is true. The range of human intelligence is not that wide. Eliezer Yudkowsky is a research fellow of the Machine Intelligence Research Institute, which he co-founded in 2001. We have no idea exactly what's at stake here and how it's proceeding. Indeed, and more to the point, is a thing that beats a grandmaster at chess a grandmaster? So, you can tell them exactly what to do and they'll still be surprised at the end result because it exploits a law of the environment they don't know about. So, mostly, it seems to me that if we wanted to win this, we needed to start a whole lot earlier, possibly in the 1930s, but in terms of my looking back and asking how far back you'd have to unwind history to get us into a situation where this was survivable, but leaving that aside--. Explore audio transcript, further reading that will They don't know that's a law of nature. You have a beautiful essay we'll link to on 12 rules for rationality ["Twelve Virtues of Rationality".] I'm not creative enough. We'll come back to that later. I think this is intuitive if you think about talking to an AI that you know is an AI under current conditions. But they don't have to! Russ Roberts: So, in its current format, though--and maybe you're talking about the next generation--in its current format, it responds to my requests with what I would call the wisdom of crowds. As those simulated worlds get more complex, we can expect them to eventually produce agents that plan and have goals. Some experts worry that as researchers make these systems more powerful, training them on ever larger amounts of data, they could learn more bad habits. Because there were factual falsehoods that were pillars of the Nazi philosophy and that people would reliably stop believing as they got smarter. The soft version is it doesnt matter if theres a difference between doing a thing and simulating doing a thing if the outcome is the same. Another way AI could weaken humanity is by being a better than average partner in the sense of replacing romantic interst. I found the last section on how to stop AI very unconvincing. Not sure this one is worth most peoples time not for the lack of an interesting topic, but Eliezer evades Russs questions by using pedantic language and turning the questions back on Russ. But Taleb might be right and it's certainly not the case in my experience that bigger brains, higher IQ [intelligence quotient] means better decisions. - LessWrong Is Molecular Nanotechnology "Scientific"? What happens from an economic standpoint if 10% of those jobs disappear in the next 5 years? could one day destroy humanity. Eliezer Yudkowsky: If you literally--not just decisions where you disagree with the goals, but, like, false models of reality--models of reality so blatantly mistaken--that even you, a human, can tell that they're wrong and in which direction, these people are not smart the way that an efficient--a hypothetical, weak, efficient market is smart. He is the founder of the Machine Intelligence Research Institute, the founder of the LessWrong blogging community, and is an outspoken voice on the dangers of artificial general intelligence, which is our topic for today. It is far less jargon-y than, say, marginal utility. They have no idea what it's like to have come out of the KGB [Komitet Gosudarstvennoy Bezopasnosti (Committee for State Security)]--that they're totally clueless and dangerous because they think they can put themselves in the head of someone who is totally alien to them. Which we should not--which we would not if we were taking this seriously--get as close to as we possibly could because we don't actually know exactly where the level is. And right now, that schism is playing out online between two people: AI theorist Eliezer Yudkowsky and OpenAI Chief Executive Officer Sam Altman. 3. He is very intelligent in a specific way but he was very intellectually uninteresting due to his inability to play with contrary ideas. This episode left me with more questions than answers, the first time around it was difficult to follow but after listening to it for a second time I was able to pick up on Eliezers concepts much better. They can become more empathetic. To the extent AI-generated conversations are hidden behind a veil, those interested in self-preservation will grow to scrutinize ALL conversation even more, especially when unsolicited. Jacques Monod wrote: "A. Are you crazy? Cade Metz is a technology reporter and the author of Genius Makers: The Mavericks Who Brought A.I. It, what we tell it to could be dangerous, since we, t know what we want. What should I do to feel better about myself?' Yudkowsky may be someone whose understanding of AI and its potential lethality for humanity is based in a deeper understanding of advanced mathematics than even many well educated scientists and software developers possess. I only have a few hundred, so I don't murmur maybe as well. I hope I can point you to some answers to your objections. I don't know. I mean, Sam Altman write you a [?text?] Do you get people who are playing in this sandbox to write you and say, 'You've scared me. But theyve been light on the details. You've animated something or appeared to animate something that even a few years ago was unimaginable, and now suddenly it's suddenly--it's not just a feat of human cognition. Color key: Chat by Carl and Eliezer Other chat 9.14. Thank you for saying the obvious! Theres too much business incentive to make it so. Eliezer Yudkowsky: It is not a universal law on humans. Why is your hand not as strong as steel? I didn't even remember it was in there. They both move to neighboring spaces that are higher inclusive genetic fitness, lower in the loss function. Russ Roberts: Yeah. I think almost all evidence in this regard comes from transformers suddenly starting to work in 2016 or 2017 and not showing signs of slowing down. Of course, it would not surprise me if AI could be used by humans to create havoc. [Link to Gioia episode addedEconlib Ed.]. You can play against them and win. OP responds in turn; apparently he misunderstood Yudkowsky because Yud's intended meaning is so brazenly stupid that it never occurred to OP to interpret him in that way. Russ Roberts: Okay. At one point in the post OP asks, rhetorically. It is proposed that the next step in analyzing positions on the intelligence explosion would be to formalize return on investment curves, so that each stance can formally state which possible microfoundations they hold to be falsified by historical observations. could destroy humanity. Eliezer Yudkowsky So what would be an equivalent to AI machines? Humankind, like most creatures, has always sought to reach greener pastures. Russ Roberts: For those not watching on YouTube, it's a copy of a book called Nanosystems, but for those who are listening at home rather than watching at home, Eliezer, tell us why that's significant.
Cis Recruitment 2023 Name List,
Revolutionary War Unit,
School Nutrition Show Orlando,
What Is Embedded Analytics,
Articles E