Artificial Intelligence, Games, Metaverse and the Future of Civilization

Guest: AI Artificial Intelligence has been one of our main topics today and here comes another talk It's called AI and the Megatrends reshaping game development and it will cover what impact AI has not only on games But also on any kind of interactive entertainment. It's helped by John Raddorf Welcome John CEO and co-founder of Beemobl This is all yours, thank you. Thank you. Thank you

Jon Radoff: Thanks for joining me here today. What I'm gonna really be talking about is even bigger than games It's really Civilizational change maybe even species change When I talk about artificial intelligence and these other related trends like UGC and whatnot I want to hopefully give you some new tools for thinking about these things like what is intelligence? What does that even mean? What is creativity? We talk about whether computers can be really creative or not Maybe we should think about what is it for anyone to be creative? So I hopefully will challenge some of the things you think about in this at the very least give you some new ways to talk about these subjects I'm gonna be sharing a lot of things about technology with you But the biggest thing about this talk is it's not really about technology It's really about culture and social change in the way we are transforming as a species Why should you care about what I have to say here? Well, I've been involved in game-making and other things online for my entire career I built one of the first online games. I've built other online games based on popular TV shows like Star Trek and Game of Thrones And now I'm operating a platform called Beemobl Which makes it possible for people to very easily launch virtual worlds and experiences and across this career While the internet has been so important. I remember thinking you know back in the 90s when I got involved in this stuff for the first time How transformative the internet was going to be for civilization and of course it has been and Across this career as much as I've loved making games and and bringing these experiences to people Until this point in time there hasn't been another thing quite at the level of the internet Which is gonna reshape society and that of course is artificial intelligence But the lens that I've always looked at these technologies through is not the tech for tech sake It's how is it gonna change humanity? How is it gonna change culture? So I have to go back to a term that's gotten a little bit less popular over the last year and that's metaverse But I think metaverse is actually an interesting word to use to capture some of the social change What do people mean when they use this word metaverse? Usually they're meaning some kind of augmented reality or virtual reality experience other people means something like a Roblox type or a UE then type experience or Maybe they mean something something blockchain and then you figure out what the heck they're talking about But usually those are the different categories of what metaverse means and I think it's more interesting to look at it from the standpoint of what do people actually do in the metaverse? It's really about shared imagination and storytelling. It's about exploring other planes of existence having real-time interaction with each other and even maybe Transforming reality around us. I'll get a little more specific But let's talk about why do we even have a metaverse? Well for as long as humans have been crafting objects art culture We have had games, right? So that's that's actually not a D20 from Dungeons and Dragons. It's a very ancient artifact people were making Game devices for a long time and why did these things exist? Because they gave us ways of telling stories and creating abstractions and simulations of Reality, but ultimately that's about sharing an imaginary space with another person I think of Dungeons and Dragons as the first real metaverse and I'm not talking about Dungeons and Dragons in an online game I'm referring to Dungeons and Dragons the experience that I had growing up as a kid where we sat around a table and shared that Imaginative space to me the technologies that you build around this are really just ways to augment the experience of how you can share it doing it across time and space Maybe filling in more of the graphics and the the game system mechanical stuff Video games of course were the first real attempt after you get beyond Dungeons and Dragons to digitize some of these Experiences so instead of having the stone die that we roll on a table we now have computers that can do the random number generator Think about how much of your life is defined by your online presence today Compared to the person that lived 20 years ago It's really clear that so much of our lives are digitized now in digital and whether that's e-sports or playing online games or online dating Livestreaming something whether you get involved with cryptocurrencies if you think about something like personal branding These are all aspects of what it is to live in a digital universe That's what the metaverse is the metaverse isn't a particular application or a set of technologies It's the fact that we can take our identity our expression and live in this imaginary realm that we're creating

Unknown: Which of course games have been best at for many many decades now?

Jon Radoff: Part of that digital expression is our avatar right so the ability to project ourselves into digital space Used to be that it was just a text-based interface Then it became game presence now you have the ability for example in Roblox or fortnight to take on a particular persona or look That you want and there are now emerging technologies that may potentially Span multiple games and multiple experiences online That's just another form though of Expressing ourselves so when we talk about the online world and being in this shared imaginary space And avatars just a way to say this is who I am in that space and take it with me There's a schema that I use to think about this cultural trajectory that we're on which is it started with digital identity Who are we on in these online spaces? That's things like the avatar the next step beyond that Which we're very much in the midst of is self-expression what do we make online? So it started with posting photos on Facebook now it is shaping things in Minecraft might be doing ugc In the future it may be augmenting How we do things online using artificial intelligence and actually projecting agents of our own creation online to work on our behalf While we go into our digital spaces We're also taking objects with us and this is something that is creating a really interesting feedback loop between Artificial intelligence and 3d realms So on the right hand side there that's a scene that I made with AI it looks like a photograph But it's actually a picture of a gargoyle took from my garden using nerd technology called neural rendering fields Nerf where you can take pictures and it figures out the 3d object from it and then it can animate it What's happening is lots of people are capturing these objects now and The thing that artificial intelligence needs is lots and lots of data so that it can do generations of objects Of course, there are already tons and tons of 2d objects that existed Which is why the first generative art stuff that you saw such as Stable diffusion mid-jurn and so forth were so effective is because of that reservoir of 2d But soon we're going to have a massive quantity of 3d and 3d object generation is going to be equally possible We're also mirroring digital objects with the physical realm so the first Organization to really do this in a big way was NASA when they created spacecraft and created a digital Twin of the spacecraft to look at all of the properties all of the sensors all of the Parameters that were guiding that object in space. We're going to increasingly have digital twins that occupy All manner of our of our lives Well, if you can have individual digital objects you could scale that up to an entire city right So we have this idea called smart cities Which is essentially a digital twin at the scale of an entire city so each object is present each vehicle each building Each part of a building we're going to be able to transport ourselves back and forth between the physical and the virtual realm And if you can do that you can scale up all the way from cities to the entire earth Invidia has a project called twin earth where they're actually trying to map and predict things like client

Unknown: As well as any number of other geophysical phenomena by looking at a twin of the earth which requires massive compute

Jon Radoff: You can also enrich the world with physical information when I trekked in the paul about a year ago I went Around Mount Everest and I wasn't sure of what all the other mountains are well There's an app for that now you can just hold up your phone and it'll show you exactly what the mountains are

Unknown: So as we project ourselves

Jon Radoff: Into digital space. I think it's also interesting to think about how We're not just creating digital spaces We're also changing the way our physical space works as well So for example the whole phenomena of remote work and now how that exists the fact that our media rooms are different than they would have been if we didn't have The ability to interact with each other smart cities We've talked about augmented reality So there's going to be a feedback loop between how the real world and the physical world works Something as simple as all the restaurants now where they have mobile ordering and the entire physical structure of many Restaurants is now going to change because you don't need the same ordering and queue process You'll order online and go through a different kind of process entire highway systems will be remapped because of autonomous vehicles for example Few examples But let's pivot now to the piece that I was referring to earlier around so we have this backdrop of immense creativity The flow of information between the real world and the physical the ability to train massive models because we're getting more and more data all the time There's this idea that I think game developers have been stronger with than just about anybody over the decades Which is this idea of emergence emergence Emergent properties and games where you start with a relatively simple system But the complexity of it comes from the set of simple rules when humans interact with it and do interesting things with it I want to start with one of the simplest games that has incredible emergent properties Most people here who've been in game development you're familiar with the game of life Basically has three rules in it The emergent aspect of it is really interesting because it's actually a creative game It's fun because the player sits down and you pick which pixels You want to see come to life on that page and then you get to see what happens because of it So game of life is a creative game that's emergence A really interesting thing to realize about game of life is a game of life is actually a touring complete Computer with in game of life if you know exactly where to put the pixels and you've You can read papers on this you can actually allow it to process computer programs It's not the most efficient computer program But it shows you just how sophisticated Some of these applications can be given that that is a game with Three or four rules in it you can create a touring complete computer system Let's talk about creativity because this is going to relate back to the artificial intelligence topic I think creativity is one of the squishy terms that People often don't know what they're even talking about sometimes creativity Strikes people as almost divinely inspired like it's a special thing that only humans can do Humans are really great at creativity But creativity I think is a search If we think of all the things that could possibly exist and all the hyper parameters of existence all the different variables that you could tweak Creativity is about finding interesting paths to solutions in that infinity and the better you are at finding those paths The more quote-unquote creative you are One interesting example of this is music So Aron Agosi who's now a professor at MIT, but before that he was known as the CTO who Created you know the first guitar hero rock band. So he's involved in music and game creation He did a little paper on like how many types of music how many pieces of music are there going to be so You know just with a certain numbers he found it's like 36 to 640 that's a really big number It might as well be infinity when you get that big But the idea though is music is really hard to do But music creation as a creative process is about figuring out what are the interesting things in that search space of infinity There's a recent paper that came out from an astrophysicist who's using artificial intelligence to actually figure out how to solve for or identify formulas that exist within physics and the problem with this is that if you use brute force methods to just try to Find the equations that represent physical concepts It's such a big search space that your computer can never complete that task Maybe with quantum computers at some point will have certain new ways to approach that but even then many many of these are not really assailable Through brute force methods But if you combine artificial intelligence with the ability to find efficient routes through the search space You can constrain your search and come up with results that actually are startling and in this particular paper that I refer to here it rediscovered General relativity by looking at a set of data inputs Another form of emergence comes through social interactivity many people here Particularly are probably familiar with the corrupted blood incident in world of work raft where people got this disease in the game And they'd spread it to other people and then whole cities full of people would be killed The interesting thing about the emergence of it isn't the fact that the disease was spreading It's what were the human behaviors that were expressed as that went on so you started to have some people who were very Pro-social and they were like telling people where the disease was and you could where you could avoid it There were the griefers who would go like intentionally get it and then find big pockets of people and spread it to them

Unknown: That's emergent behavior which is the world of work raft developers as far as I know

Jon Radoff: Didn't conceive of those particular gameplay patterns, but it was enabled because of how much you can express within something like world of work raft As we go from the stage of simply playing games to expressing ourselves in games from Simply having an identity which is present in a virtual world to actually expressing ourselves within virtual worlds We're going to see more and more of these emergent behaviors and this is a really exciting time because people Number one just love this and number two. It is really what it is to be human Being human is expressing ourselves creativity creatively not just through an avatar not just through Digital fashion, but by shaping environments around us creating social experiences being leaders within these and This form of emergent play through creativity is fun So we don't think of mid-jurney as a game per se it isn't a game because it doesn't have the mechanics of a game But it is fun Just quick show of hands. Who has tried mid-jurney in this room? I think it's almost a hundred percent of the room has tried mid-jurney So number one that's really interesting data point to think about when we consider Is this stuff for real or not because there's a lot of stuff that gets hype and I wouldn't find that people are raising their hands quite like that But if you play with mid-jurney part of what's compelling about it Is the fun factor of it helping you do that efficient search and Identifying things that maybe you didn't even think of it's a search through art space in that case Minecraft I think is a great story of constraints that can be implemented within Of virtual world and the brilliance of just making it voxel based as a way for people to start building things In a manner that is a lot simpler, but as a result much much more expressive So creative emergence is one of the most important things that's defining this time in game development And it's critical to understand not only as something people are having fun with But I really have a passion about this because I feel like it's a blind spot that many people in our industry have Which is that in the future Many of the experiences that we might make in a traditional packaged game product May in fact be implemented far more easily in these environments and reach much larger audiences and just simply be much more commercially successful Because of the structures that are available there and it's a combination of the framework for UGC that many of these platforms have It's going to be the convergence with generative AI It's going to be supported by technologies that just speed you to market Interesting thing back on the emergence thing you mentioned you remember I mentioned that game of life is touring complete Well, Minecraft is touring complete as well So someone actually created a neural network in game of life that can do Digit recognition and they did it completely just using redstone in Minecraft no Java program So it's amazing to think like what complexity level you can get to just in off the shelf creative platforms So we're going to see more and more users are generated content In the future it may be that every game or every big game is going to be programmable Maybe every game is essentially its own app store with its own content within it Generative AI we haven't quite seen it yet, but it's going to be part of Roblox So you'll be able to not only build experiences in Roblox, but you'll be able to use prompted mechanisms to add content to change physics to change the art We're starting to see convergence between UGC and generative AI An interesting case for generative AI is when you have a game that's built around UGC But you want it to be style consistent as lots of people add components to their world This is one example from a company called million on Mars that's working with scenario Where the players are creating the content that it goes through a style consistency pass using generative AI We're also going to risk in reality This is super cool like I talked about the physical continuum to the virtual before Well, we can actually now using Currently available technologies take a picture of a room in 3d Apply generative AI against it and you see a different version of that room

Unknown: The

Jon Radoff: Emergence in language models is happening because of a lot of quantitative improvements in these systems Number one is these models have just gotten a lot larger as they've gotten larger. They've made it possible to Really uncover super interesting aspects of the latent space of these models So sometimes I hear criticisms of language models, which is oh, it's just a completion engine First of first thought I have is well my brain is just kind of a completion engine when you say something to me My brain is just prompted to respond in a certain way Seems like that on unlocks an awful lot of capabilities But within these language models they are getting more and more competent and capable of doing things That I think if you play with this stuff a lot you will find that it does far more than just complete It can help you ideate and that's what gets back to the creativity that I was referring to which is if it's helping you Along an efficient search path. That's what creativity is another quick show of hands. Who is use it? Who has used chat GPT? 100% of the well, okay a couple people I don't believe you. I think you just didn't want to raise your hand But it was at least 95% okay, and who still uses it regularly? Almost the whole room so I mean there's some statistics there like I think you just can't ignore the fact that people actually are Using these things and deriving value from it and they're no longer merely toys You can apply these language models though in interesting new ways we talked about chat GPT But one of the really interesting frontiers for game development is not just using it to help you produce games faster But actually putting generative AI into the loop of the game hidden doors one example of a company that basically is creating a digital dungeon master For multiple players by the way, that's a whole other area that's super interesting which is multiplayer prompting So when you get multiple people together and they're collectively prompting something that's a game right there Like there's there's literally a billion game ideas just on multiplayer prompting This is another super interesting one that incorporates language models. So Facebook This is actually about a year old now, but they had something called Cicero And they got a game to play diplomacy which is a notoriously difficult game to put on computers because of how It has to do essentially trickery deception negotiation with humans Well using language models diplomacy can be humans Check this paper out Minecraft is another super interesting thing. This is VPT VPT is called video pre-training they trained a AI model that does stuff in Minecraft by watching YouTube videos of people playing Minecraft So it's able to now do things like go and build simple structures go hunting do the normal things that you do in Minecraft Which it learned from watching videos. So just another super interesting innovation with crossover into games We talked about avatars. Well the next step beyond that is Not only avatars but virtual beings where I'm there as an avatar, but I'm interacting with other creatures who appear or people who appear as avatars But they're actually you know chatbots to full blown virtual beings. This is an example of the Of code meko who is a twitch streamer who appears usually on screen as an avatar She played Minecraft with a virtual being called Lucy and they played together again really interesting gameplay that can emerge out of this And Perhaps one of the most exciting areas which gets us to the third stage that I was talking about earlier in this cultural evolution is the idea of Generative agents so currently when you go to chat GPT you ask it for something you have a conversation Well, we already start to have these things some of them are called like baby agi and some other things where you Tell it to actually perform actions for you is as a language model to figure out a strategy for performing the action And then it iteratively goes out and works on it. This is an incredible tool for game development I've already this week seen a couple of demos from people who are building things that are basically Agents where you tell it the kind of game you want to make and you work with it collaboratively And I've also seen social versions of that again multiplayer prompting people who are working together with a generative agent to craft the game experience together So that's where we're going from this period of it was expression and creating to now empowering Using things like generative agents to go and carry out things on our behalf and autonomous agent to agi's Which includes some of the ones I just referred to this This is another one if you want to get a really simple one some of them are a little tricky to install But god mode is one that works just right in your web browser that'll give you a sense of it um My team won a hackathon at the and recent hero it's tech week several weeks back Where in one day we built a multiplayer online role playing game

Unknown: Using a combination of an lm art generation and it's a multiplayer game where you occupy different virtual spaces and go on adventures together one day

Jon Radoff: Um, I actually run a software company during the day so I didn't get to take it further than that But if I had had a month or two like you could actually build a pretty legit game off of that So that's where we're going started with digital identity We're moving we are very much in the stage of self-expression now things like minecraft and building things and roblox But next it's about empowering us. It's about using artificial intelligence tools to carry out what we want and work as creative partners and and be agents on our behalf And that leads to the next battleground which is as we've talked about things like mid-journey and open AI chat gpt Those have been the centralized model where you go to apis that live up in the cloud Here's the reason why this stuff is never going to get regulated by the way This is going to be fully decentralized right on your device You're going to be able to do almost all of these things in the future You can already do image generation on your own device But language models there's already quite a few coming So this is going to be technology that's pervasive everywhere. It's going to change all of society Of course, it's going to change games and that's super exciting time to be alive because small teams are going to do amazing things Thank you for having me here today and for listening to me