https://www.youtube.com/watch?v=HN_hOuyXUkc

Guest: AI Artificial Intelligence has been one of our main topics today. Here comes another talk. It's called AI and the Megatrends Re-Shaping Game Development. And it will cover what impact AI has, not only on games, but also on any kind of interactive entertainment. It's held by John Raddorf. Welcome, John, CEO and co-founder of Beemobel. This is all yours. Enjoy.

Jon Radoff: Thank you. Thank you. Thanks for joining me here today. What I'm going to really be talking about is even bigger than games. It's really civilizational change, maybe even species change. When I talk about artificial intelligence and these other related trends, like UGC and whatnot, I want to hopefully give you some new tools for thinking about these things. Like, what is intelligence? What does that even mean? What is creativity? We talk about whether computers can be really creative or not. Maybe we should think about what is it for anyone to be creative. So I hopefully will challenge some of the things you think about in this. At the very least, give you some new ways to talk about these subjects. I'm going to be sharing a lot of things about technology with you. But the biggest thing about this talk is it's not really about technology. It's really about culture and social change and the way we are transforming as a species. Why should you care about what I have to say here? Well, I've been involved in game making and other things online for my entire career. I built one of the first online games. I've built other online games based on popular TV shows, like Star Trek and Game of Thrones. And now I'm operating a platform called Beamable, which makes it possible for people to very easily launch virtual worlds and experiences.

Unknown: And across this career, while the internet has been so important,

Jon Radoff: I remember thinking back in the 90s, when I got involved in this stuff for the first time, how transformative the internet was going to be for civilization. And of course, it has been. And across this career, as much as I've loved making games and bringing these experiences to people, until this point in time, there hasn't been another thing quite at the level of the internet, which is going to reshape society. And that, of course, is artificial intelligence. But the lens that I've always looked at these technologies through is not the tech for tech's sake. It's, how is it going to change humanity? How is it going to change culture? So I have to go back to a term that's gotten a little bit less popular over the last year. And that's metaverse. But I think metaverse is actually an interesting word to use to capture some of the social change. What do people mean when they use this word metaverse? Usually they're meaning some kind of augmented reality or virtual reality experience.

Unknown: Other people mean something like a Roblox type or a UEA.

Jon Radoff: They then type experience, or maybe they mean something, something blockchain, and then you figure out what the heck they're talking about. But usually those are the different categories of what metaverse means. And I think it's more interesting to look at it from the standpoint of, well, what do people actually do in the metaverse? It's really about shared imagination and storytelling. It's about exploring other planes of existence, having real-time interaction with each other, and even maybe transforming reality around us. I'll get a little more specific. But let's talk about why do we even have a metaverse. Well, for as long as humans have been crafting objects, art, culture, we have had games. So that's actually not a D20 from Dungeons and Dragons. It's a very ancient artifact. People were making game devices for a long time, and why did these things exist? Because they gave us ways of telling stories and creating abstractions and simulations of reality. But ultimately, that's about sharing an imaginary space with another person. I think of Dungeons and Dragons as the first real metaverse. And I'm not talking about Dungeons and Dragons in an online game. I'm referring to Dungeons and Dragons, the experience that I had growing up as a kid, where we sat around a table and shared that imaginative space. To me, the technologies that you build around this are really just ways to augment the experience of how you can share it, doing it across time and space, maybe filling in more of the graphics and the game system mechanical stuff. Video games, of course, were the first real attempt, after you get beyond Dungeons and Dragons, to digitize some of these experiences. So instead of having the stone die that we roll on a table, we now have computers that can do the random number generator. Think about how much of your life is defined by your online presence today. Compare it to the person that lived 20 years ago. It's really clear that so much of our lives are digitized now in digital. And whether that's eSports or playing online games, or online dating, live streaming something, whether you get involved with cryptocurrencies, if you think about something like personal branding, these are all aspects of what it is to live in a digital universe. That's what the metaverse is. The metaverse isn't a particular application or a set of technologies. It's the fact that we can take our identity, our expression, and live in this imaginary realm that we're creating, which, of course, games have been best at for many, many decades now. Part of that digital expression is our avatar, right? So the ability to project ourselves into digital space used to be that it was just a text-based interface, then it became game presence. Now you have the ability, for example, in Roblox, or Fortnite to take on a particular persona or look that you want. And there are now emerging technologies that may potentially span multiple games and multiple experiences online. That's just another form, though, of expressive ourselves. So when we talk about the online world and being in this shared imaginary space, an avatar is just a way to say, this is who I am in that space,

Unknown: and take it with me.

Jon Radoff: There's a schema that I use to think about this cultural trajectory that we're on, which is it started with digital identity. Who are we in these online spaces? That's things like the avatar. The next step beyond that, which we're very much in the midst of, is self-expression. What do we make online? So it started with posting photos on Facebook. Now it is shaping things in Minecraft. It might be doing UGC. In the future, it may be augmenting how we do things online, using artificial intelligence and actually projecting agents of our own creation online to work on our behalf. While we go into our digital spaces, we're also taking objects with us. And this is something that is creating a really interesting feedback loop between artificial intelligence and 3D realms. So on the right-hand side there, that's a scene that I made with AI. It looks like a photograph, but it's actually a picture of a gargoyle took from my garden using technology called Neural Rendering Fields Nerf, where you can take pictures and it figures out the 3D object from it, and then it can animate it. What's happening is lots of people are capturing these objects now. And the thing that artificial intelligence needs is lots and lots of data so that it can do generations of objects. Of course, there are already tons and tons of 2D objects that existed, which is why the first generative art stuff that you saw, such as stable diffusion, mid-journey, and so forth, were so effective, is because of that reservoir of 2D. But soon we're going to have a massive quantity of 3D and 3D object generation is going to be equally possible. We're also mirroring digital objects with the physical realm. So the first organization to really do this in a big way is NASA when they created spacecraft and created a digital twin of the spacecraft to look at all of the properties, all of the sensors, all of the parameters that were guiding that object in space. We're going to increasingly have digital twins that occupy all manner of our lives. Well, if you can have individual digital objects, you could scale that up to an entire city. So we have this idea called Smart Cities, which is essentially a digital twin at the scale of an entire city. So each object is present, each vehicle, each building, each part of a building. We're going to be able to transport ourselves back and forth between the physical and the virtual realm. And if you can do that, you can scale up all the way from cities to the entire earth. Nvidia has a project called Twin Earth, where they're actually trying to map and predict things like climate, as well as any number of other geophysical phenomena by looking at a twin of the earth, which requires massive compute. You can also enrich the world with physical information. When I trekked in Nepal about a year ago, I went around Mount Everest, and I wasn't sure of what all the other mountains are. Well, there's an app for that now. You can just hold up your phone, and it'll show you exactly what the mountains are. So as we project ourselves into digital space, I think it's also interesting to think about how we're not just creating digital spaces. We're also changing the way our physical space works as well. So for example, the whole phenomena of remote work, and now how that exists. The fact that our media rooms are different than they would have been if we didn't have the ability to interact with each other. We've talked about smart cities, we've talked about augmented reality. So there's going to be a feedback loop between how the real world and the physical world works. Something as simple as all the restaurants now where they have mobile ordering and the entire physical structure of many restaurants is now going to change because you don't need the same ordering and queue process. You'll order online and go through a different kind of process. Entire highway systems will be remapped because of autonomous vehicles, for example. A few examples. But let's pivot now to the piece that I was referring to earlier around. So we have this backdrop of immense creativity, the flow of information between the real world and the physical, the ability to train massive models because we're getting more and more data all the time. There's this idea that I think game developers have been stronger with than just about anybody over the decades, which is this idea of emergence. Emergent properties and games where you start with a relatively simple system but the complexity of it comes from the set of simple rules when humans interact with it and do interesting things with it. I want to start with one of the simplest games that has incredible emergent properties. Most people here who've been in game development, you're familiar with the game of life. Basically has three rules in it. The emergent aspect of it is really interesting because it's actually a creative game. It's fun because the player sits down and you pick which pixels you want to see come to life on that page and then you get to see what happens because of it. So game of life is a creative game. That's emergence. A really interesting thing to realize about game of life is a game of life is actually a touring, complete computer. Within game of life, if you know exactly where to put the pixels and you can read papers on this, you can actually allow it to process computer programs. It's not the most efficient computer program, but it shows you just how sophisticated some of these applications can be given that that is a game with three or four rules in it. You can create a touring, complete computer system. Let's talk about creativity because this is going to relate back to the artificial intelligence topic.

Unknown: Creativity is one of the squishy terms that people often don't know what they're even talking about.

Jon Radoff: Sometimes creativity strikes people as almost divinely inspired, like it's a special thing that only humans can do. Humans are really great at creativity, but creativity, I think, is a search. If we think of all the things that could possibly exist and all the hyper parameters of existence, all the different variables that you could tweak, creativity is about finding interesting paths to solutions in that infinity. And the better you are at finding those paths, the more, quote, unquote, creative you are. One interesting example of this is music. So, Aaron Gosi, who's now a professor at MIT, but before that, he was known as the CTO, who created the first guitar hero, rock band, so he's involved in music and game creation. He did a little paper on how many types of music, how many pieces of music are there going to be? So, just with a certain number, he found it's like 36 to 640, that's a really big number. It might as well be infinity when you get that big. But the idea, though, is music is really hard to do, but music creation is a creative process, is about figuring out what are the interesting things in that search space of infinity. There's a recent paper that came out from an astrophysicist who's using artificial intelligence to actually figure out how to solve for or identify formulas that exist within physics. And the problem with this is that if you use brute force methods to just try to find the equations that represent physical concepts, it's such a big search space that your computer can never complete that task. Maybe with quantum computers at some point, we'll have certain new ways to approach that. But even then, many, many of these are not really assailable through brute force methods, but if you combine artificial intelligence with the ability to find efficient routes through the search space, you can constrain your search and come up with results that actually are startling. And in this particular paper that I refer to here, it rediscovered general relativity by looking at a set of data inputs. Another form of emergence comes through social interactivity. Many people here, particularly, are probably familiar with the corrupted blood incident in the world of work craft where people got this disease in the game and they'd spread it to other people and then whole cities full of people would be killed. The interesting thing about the emergence of it isn't the fact that the disease was spreading. It's what were the human behaviors that were expressed as that went on. So you started to have some people who were very pro-social and they were like telling people where the disease was and where you could avoid it. There were the griefers who would go like intentionally get it and then find big pockets of people and spread it to them. That's emergent behavior, which is the world of work craft developers, as far as I know, didn't conceive of those particular gameplay patterns, but it was enabled because of how much you can express within something like world of work craft. As we go from the stage of simply playing games to expressing ourselves in games, from simply having an identity which is present in a virtual world to actually expressing ourselves within virtual worlds, we're going to see more and more of these emergent behaviors. And this is a really exciting time because people, number one, just love this and number two, it is really what it is to be human. Being human is expressing ourselves creatively, not just through an avatar, not just through digital fashion, but by shaping environments around us, creating social experiences, being leaders within these. And this form of emergent play through creativity is fun. So we don't think of mid-journey as a game, it isn't a game because it doesn't have the mechanics of a game, but it is fun. Just quick show of hands. Who has tried mid-journey in this room? I think it's almost 100% of the room has tried mid-journey. So number one, that's really interesting data point to think about when we consider is this stuff for real or not because there's a lot of stuff that gets hype, and I wouldn't find that people are raising their hands quite like that. But if you play with mid-journey, part of what's compelling about it is the fun factor of it helping you do that efficient search. And identifying things that maybe you didn't even think of. It's a search through art space in that case. Minecraft, I think, is a great story of constraints that can be implemented within a virtual world, and the brilliance of just making it voxel based is a way for people to start building things in a manner that is a lot simpler, but as a result, much, much more expressive. So creative emergence is one of the most important things that's defining this time in game development. And it's critical to understand not only as something people are having fun with, but I really have a passion about this because I feel like it's a blind spot that many people in our industry have, which is that in the future, many of the experiences that we might make in a traditional packaged game product may in fact be implemented far more easily in these environments and reach much larger audiences and just simply be much more commercially successful because of the structures that are available there. And it's a combination of the framework for UGC that many of these platforms have. It's going to be the convergence with generative AI. It's going to be supported by technologies that just speed you to market. Interesting thing back on the emergency thing you mentioned, you remember I mentioned that game of life is touring complete. Well, Minecraft is touring complete as well. So someone actually created a neural network in game of life that can do digit recognition. And they did it completely just using Redstone in Minecraft, no Java program. So it's amazing to think like what complexity level you can get to just in off the shelf creative platforms. So we're going to see more and more users are generated content. In the future, it may be that every game or every big game is going to be programmable. Maybe every game is essentially its own app store with its own content within it. Generative AI, we haven't quite seen it yet, but it's going to be part of Roblox, right? So you'll be able to not only build experiences in Roblox, but you'll be able to use prompted mechanisms to add content to change physics to change the art. We're starting to see convergence between UGC and generative AI. An interesting case for generative AI is when you have a game that's built around UGC, but you want it to be style consistent as lots of people add components to their world. This is one example from a company called Million on Mars that's working with scenario where the players are creating the content, but it goes through a style consistency pass using generative AI. We're also going to risk in reality. This is super cool. I talked about the physical continuum to the virtual before. Well, we can actually now, using currently available technologies, take a picture of a room in 3D, apply generative AI against it, and you see a different version of that room. The emergence in language models is happening because of a lot of quantitative improvements in these systems. Number one is these models have just gotten a lot larger. As they've gotten larger, they've made it possible to really uncover super interesting aspects of the latent space of these models. Sometimes I hear criticisms of language models, which is, oh, it's just a completion engine. First of all, I have is, well, my brain is just kind of a completion engine. When you say something to me, my brain is just prompted to respond in a certain way. Seems like that unlocks an awful lot of capabilities. But within these language models, they are getting more and more competent and capable of doing things that, I think, if you play with this stuff a lot, you will find that it does far more than just complete. It can help you ideate. And that's what gets back to the creativity that I was referring to, which is if it's helping you along an efficient search path, that's what creativity is. Another quick show of hands. Who has used chat GPD? I think 100% of the, well, a couple of people, I don't believe you. I think you just didn't want to raise your hand. But it was at least 95%. And who still uses it regularly? Almost the whole room. So, I mean, there's some statistics there. I think you just can't ignore the fact that people actually are using these things and deriving value from it, and they're no longer merely toys. You can apply these language models, though, in interesting new ways. I talked about chat GPD, but one of the really interesting frontiers for game development is not just using it to help you produce games faster, but actually putting generative AI into the loop of the game. Hidden Door is one example of a company that basically is creating a digital dungeon master for multiple players. By the way, that's a whole other area that's super interesting, which is multiplayer prompting. So, when you get multiple people together, and they're collectively prompting something, that's a game right there. Like, there's literally a billion game ideas just on multiplayer prompting. This is another super interesting one that incorporates language models. So, Facebook is actually about a year old now, but they have something called Cicero, and they got a game to play diplomacy, which is a notoriously difficult game to put on computers because of how it has to do essentially trickery deception negotiation with humans. Well, using language models diplomacy can be humans.

Unknown: Check this paper out.

Jon Radoff: Minecraft is another super interesting thing. This is VPT. VPT is called Video Pre-Training. They trained a AI model that does stuff in Minecraft by watching YouTube videos of people playing Minecraft.

Unknown: So, it's able to now do things like go and build simple structures, go hunting,

Jon Radoff: do the normal things that you do in Minecraft, which it learned from watching videos. So, just another super interesting innovation with crossover into games. We talked about avatars.

Unknown: Well, the next step beyond that is not only avatars, but virtual beings,

Jon Radoff: where I'm there as an avatar, but I'm interacting with other creatures who appear, who appear as avatars, but they're actually, you know, chatbots to full-blown virtual beings.

Unknown: This is an example of the code Niko, who is a Twitch streamer,

Jon Radoff: who appears usually on screen as an avatar. She played Minecraft with a virtual being called Lucy and they played together. Again, really interesting gameplay that can emerge out of this. And perhaps one of the most exciting areas, which gets us to the third stage that I was talking about earlier in this cultural evolution, is the idea of generative agents. So, currently, when you go to chat GPT, you ask it for something, you have a conversation. Well, we already start to have these things, some of them are called like BabyAgeI, and some other things where you tell it to actually perform actions for you. It's a language model to figure out a strategy for performing the action, and then it iteratively goes out and works on it. This is an incredible tool for game development. I've already this week seen a couple of demos from people who are building things that are basically generative agents, where you tell it the kind of game you want to make, and you work with it collaboratively. And I've also seen social versions of that. Again, multiplayer prompting, people who are working together with a generative agent to craft a game experience together. So, that's where we're going. From this period of it was expression and creating to now empowering using things like generative agents to go and carry out things on our behalf.

Unknown: And autonomous agent to AGI's, which includes some of the ones I just referred to,

Jon Radoff: this is another one if you want to get a really simple one. Some of them are a little tricky to install, but God mode is one that works just right in your web browser. That'll give you a sense of it. My team won a hackathon at the Andreson Horowitz Tech Week several weeks back, where in one day we built a multiplayer online role-playing game using a combination of an LLM art generation, and it's a multiplayer game where you occupy different virtual spaces and go on adventures together. One day, I actually run a software company during the day, so I didn't get to take it further than that. But if I had had a month or two, like you could actually build a pretty legit game off of that. So, that's where we're going. Started with digital identity. We're moving, we are very much in the stage of self-expression now, things like Minecraft and building things and Roblox. But next, it's about empowering us. It's about using artificial intelligence tools to carry out what we want and work as creative partners and be agents on our behalf. And that leads to the next battleground, which is, as we've talked about things like mid-Journey and OpenAI chat GPT, those have been the centralized model where you go to APIs that live up in the cloud. Here's the reason why this stuff is never going to get regulated, by the way. This is going to be fully decentralized. Right on your device, you're going to be able to do almost all of these things in the future. You can already do image generation on your own device. But language models, there's already quite a few coming. So, this is going to be technology that's pervasive everywhere. It's going to change all of society. Of course, it's going to change games. And that's super exciting time to be alive, because small teams are going to do amazing things. Thank you for having me here today and for listening to me.