Generative Graphics Workflow for Games with Jussi Kemppainen

Jussi Kemppainen: The thing that is AAA today will be the indie of the future. And then the AAA of the future is something we haven't yet even seen. We haven't imagined that. I don't know what it's gonna be.

Jon Radoff: Yeah, yeah, it's going to be something that... So Indies will be making an Elden Ring in the future, because they're all the tools.

Jussi Kemppainen: Yeah, that's gonna be like a 25 people indie indie team making Elde Ring.

Jon Radoff: I'm with UC Companion, a veteran game developer, VFX artist and creative director from Household Names like Rovio. His recent work with generative graphics pipelines has captured the imagination of countless game

Jussi Kemppainen: developers. UC, welcome.

Jon Radoff: Thank you. Yeah, great to have you here. Can we begin by hearing about why you got so interested in generative AI? Like, what problem is did you want to solve?

Jussi Kemppainen: Something that's interested me always. Since I first saw any videos on YouTube about generative AI, I was captivated by it for... I can't really pinpoint why. It's maybe something that as a technology interests me. I've done a lot of procedural stuff. Everything I always do, I try to be as procedural as possible. So maybe that kind of was a trigger for me, because that's like the the Holy Grail of Procedural Art is his generative AI. And so, but the problem that it solves for me, like I think you'll solve for everybody, is just... It gives you... It enables you to work on projects that are out of your scope. As a small developer. I think that's the main thing that I'm most excited about. It's what it will do for the solar developers with individuals who want to make something, but they are maybe don't have the team or don't want the team or just want to work on something that's out of their scope.

Jon Radoff: Well, let's drill down into that a little bit, because people have been working on procedural systems and games for years and years and years. I think the first procedural game that I ever played was probably like NetHack. And I think that's like 40 years old or so. At this point, what's the difference between procedural technologies and what generative technologies offer, game developers, players, etc.

Jussi Kemppainen: The main difference between procedural is that it's a known entity. Somebody's controlling it. Somebody's precisely controlling it to do exactly what's required. It can be replicated. The process can be predictable. AI is not. It's something that's hard to replicate your results. It might be impossible. Or at least when talking about generative AI and diffusion models, it's very hard to get the same results wise. Even with the seed number, it's sometimes impossible. And also, there's a lot of just this black box where things happen, but nobody knows why.

Jon Radoff: That's interesting because I spoke to a game designer earlier today who had done a lot of work on chat-based systems using GTT and what not. He's been building these games for the last couple of years. He said when he first approached it, the mind-blowing aspect was when he got there and they said, well, you don't really code anything, you talk to it. That's how you explain to it what to do. What do you mean you talk to it? This is the holiday from Star Trek. Then you find out that it's a little bit more complicated and doesn't always behave as exactly like you want, which is what you're saying. Interesting way to think of this is a team member who joins up and maybe it'll become increasingly more that way as we end up with Star Trek.

Jussi Kemppainen: At the moment, it's a very stubborn teammate.

Jussi Kemppainen: You might want something, but they're always not going to make it.

Jussi Kemppainen: And always it's based on the data sets that use to teach the AI. It might be missing stuff. There might be blind spots in it that you just can't get it to do something you want. But it's still a teammate. And if you're willing to work with its limitations, it's very powerful.

Jon Radoff: It seems to me that the people who are having a lot of success with it right now are figuring out how to deal with that stubbornness and get the pieces working with each other. I think that's why some of the videos and whatnot that you've posted online have gotten very popular recently, which is because you've figured out the steps and how they link together. And by the way, we'll put links to some of this material down in the show notes so everybody can find it later. I highly recommend that if you're a any kind of game designer, a producer, a person who's involved in the graphics systems of a game, you check it out. But let's spend a little bit time on some of the things you've learned in your own projects. You've got this game that you're working on. Echoes of somewhere. That's what you call it, right? And you've been using Generative AI for that. And you've kind of been building in public sharing information about how the graphics system was set up. As I understand, you started with mid-journey and that was kind of like way to create concept art. Can we start there? What did you first learn about mid-journey and then what did you figure out about the limitations and how did you start building out a graphics pipeline from there?

Jussi Kemppainen: Yeah, the mid-journey, let's see. It was, I just stepped in queue for all of these Generative AI tools without knowing what I'm going to do with those. At the moment, the Echoes game was still a low polygon game like this retro style, very, very flat shading because that was something that I knew that I could create myself quite quickly. I wouldn't need to spend too much time on the assets. Low polygon graphics super easy to make, like this very, very simple solid colors. And I looked into AI because I was just interested in it and then I went to, I think I got the invitation to mid-journey first. I was able to work in there and try to figure it out. Mid-journey was super interesting. When I was working with it then I could see myself

Jussi Kemppainen: trying, maybe making a game using mid-journey. Like it was very good for backgrounds but horrible for characters. But I could get some kind of like characters out of it that were very messed up.

Jussi Kemppainen: But I thought maybe with clone stamping and brushing I could make the characters with mid-journey. And at this point the Echoes of Somewhere project was with a different name and it was, it had stalled because I just didn't have enough time in my free time to make the art.

Jussi Kemppainen: So this project was paused. But I was looking into mid-journey, looking into if I could use it to

Jussi Kemppainen: make the graphics and then stable diffusion came out that you could download on your computer and use locally. And I got that and it was better than mid-journey. And then I started creating assets

Jussi Kemppainen: and the graphics with stable diffusion. And then mid-journey four came out and then I went there

Jussi Kemppainen: and it was better again. So it's been kind of this back and forth going from tool to tool to tool. But now that I've started working with mid-journey version four, I'm kind of, I don't know if I'm stuck with it, but I kind of maybe shouldn't move somewhere else because now I can have this baseline of just acceptable graphics that I can work on on top of. So I think at least for the first couple

Jussi Kemppainen: of episodes of the game we're going to do them with mid-journey. And I tried Doli too, but Doli was,

Jussi Kemppainen: to me it was, the graphics is made were boring. It was very, they were very nice, but they didn't have the the solo that you could get from stable diffusion on the journey.

Jon Radoff: Yeah, hopefully we'll get some of the folks who've worked on stable diffusion and mid-journey on an episode coming up here on building the Metaverse. But yeah, mid-journey in particular, it just creates great results. Like it's at a point now and get things out of it that that rival, you know, people who have been working on concept art for a good chunk of their life. Like even someone like me can get a good result out of mid-journey. And it's it's pretty close to

Jussi Kemppainen: being game usable or at least inspiration. Yeah, yeah, very close. And my idea has been to embrace the AI for this for this echoes of somewhere just to build the project around AI graphics so that I will I won't make any decisions that prevent me from using AI made graphics because maybe I should go into detail why the AI graphics is the thing that I chose is that the project wouldn't happen otherwise. Like I tried it I tried to do it for years. The the game

Jussi Kemppainen: has been in my desk drawer for 10 years. And it hasn't moved. It hasn't it hasn't advanced.

Jussi Kemppainen: With family with kids you have to have to spend time with the family and then the work I do I love it. So I don't I don't have enough free time to come into a project that actually like requires so much effort. So with AI graphics they suddenly became possible to do.

Jon Radoff: Yeah, and you have a great job too. Right? You have a great job. You're built and also you're building a

Jussi Kemppainen: game for for a big studio along the way. So we'll come back to that but that I want to spend a

Jon Radoff: little bit more time on like your journey as an indie game developer essentially as well. So it sounds like you you've had this idea in your head that wants to burst forth and take shape on the screen and what the generative AI technologies have really enabled is for you to get some of these ideas out start accelerating the process. And and now it seems like you're well on your way

Jussi Kemppainen: to actually be able to ship this game. It's so nice when you get it work and you want to make the game and before I would go in and and I would have to maybe model a location, model trash cans, desks, drawers, cars, building walls like all of these things and build procedural sets out of them and I made I made a level designer I made a level tool that could build me interiors of houses procedurally but it still was a lot of work to to make all that happen. And with AI I just sit down I route write some words and then it gives me an option like how about this game screen and then I can just read like just reroll the same thing 20 times and it's like 15 minutes later I have a location and then the location is not necessary what I needed to be or what I wanted to be but if I'm accepting if I'm like open to being inspired by the location then they are perfectly usable. Like if I want a location with two doors and an a an a table with like some statues on it might not happen but I might get a location with one door and a window and a bed so then maybe we can write the story so that instead of having a desk there's a bed and so we kind of have to stay open to

Jussi Kemppainen: changes all the time like they we and it also gives a little bit of false impression of the how powerful AI is if we if you kind of embrace its stubbornness if you don't try to force it then

Jussi Kemppainen: you can get stuff done super fast and and with like an impressive speed. Comes back to that team

Jon Radoff: member metaphor and like that team member really wants a bed to be in that room. It wouldn't make

Jussi Kemppainen: a cupboard like yeah yeah yeah it's like okay okay all right then we will do a bed yeah I think at the moment I think it's it's something that people might not realize is when they look at all this great AI art and it's it's filling up every place and I don't I'm not I kind of I'm against AI art being pushed everywhere like it's it's so the volumes are so massive like I think they're it's gonna take a while for it to find its place where you can post AI stuff and it's it's fine but the thing that people kind of don't realize maybe is that they don't know what the artist's intention was or the one who made the prompt they don't know what they were after like it might seem threatening or it might seem something that's that's coming and it's unstoppable but at the moment it's mostly happy accidents I don't know if all of it is happy accidents but

Jussi Kemppainen: it's mainly happy accidents but I don't know how long that's gonna last because it tools the the

Jussi Kemppainen: control net and all those things are so advanced already so maybe this year already AI will stop

Jussi Kemppainen: being random generator and be something that you can totally control well let's come back to

Jon Radoff: things like control net and some of these technologies that enable you know more specific generations and and also some of the tools and that you've assembled into your pipeline but before we get to that let's spend a little bit of time on on people that are maybe just entering game development or VFX or any of these fields that you're super familiar with I've been sharing a lot of this content and and you can see some of the comments that I've received people people say stuff like OGs I've been like studying at college to learn shader programming or something and have I just been wasting my time or or or is there even going to be a job for people like me in the future I'm curious like what your overall take is on that like what do you think the impact of these technologies are on people that do graphics and design and game development specifically

Jussi Kemppainen: pre imagine that AI will take over the jobs of those junior level positions I think what it would mean is that people just start up one peg above so you wouldn't start up as a junior you would go in as as as a lead so instead of taking your first job in games company as as like a junior game like maybe junior shader artist you would lead a team of artificial intelligence shader artists maybe that would be the case and then what about all these junior positions like there's only so many companies but when the labor is cheaper because you don't have to pay everybody there's there's many more companies like I believe that the things it will just multiply the the like now we have one company with four junior artists and one lead in the future we might have four companies with one lead needs company working with AI to produce the arts it's like I kind of feel that life will find a way like there's been so many times when there has been disruption in technology where people have been worried that it will take our jobs and it's true it will take a job but it will produce many more jobs like steam engines internet maybe even just telephone car I don't know when people invented horse where people who walked were they're saying that they would never have to walk again like it's gonna take our job like for sure like everything always new things come and they will destroy jobs they will disrupt industries they will make positions in a company obsolete but they will open up so many new doors and open up so many new doors for possibilities like what if a game budget were slashed in half or in in in in like slashed in like to the fourth that would mean you wouldn't have to have so much you wouldn't have to earn so much with the game you would be able to yeah you wouldn't need to be you wouldn't need to have so high sales you could you could sell a game maybe four hundred thousand copies and that would be amazing with a team of four people and and then endless AI artists what I'm trying to come up it in come to here is that you could make you could take more risks you could make products that wouldn't please the major population you could make products that please a very niche part of the population so it would open up it would make people be more bold with their decisions and more crazy with the experiments

Jon Radoff: right because if you're if you're gonna spend 10 million or 20 million or g's and triple A

Jussi Kemppainen: a hundred million dollars to build a game it's you know going to attract a certain kind of capital

Jon Radoff: that's going to support that and while there are exceptions and people that do want to take big big risks with that the more capital you're raising and some ways it often results in lower risks being taken in the overall system design where if there's a lot more yeah if you look at the triple A market

Jussi Kemppainen: every game is pretty much the same game of course the story is different the characters are

Jussi Kemppainen: different there's maybe one new mechanic but at its core it's the same game usually

Jussi Kemppainen: even if it's third person or first person but it's similar game like you have a competition

Jussi Kemppainen: between FPS shooters but they're all basically the same game like if you have to describe how the

Jussi Kemppainen: game goes like just the base of it it's always the same but maybe in the future we can take more

Jussi Kemppainen: risks and veer out from the beaten path to more of this like these we are their directions and get

Jussi Kemppainen: like the games that we do in the future will be like one step above like the thing that it's triple A today will be the indie of the future and then the triple A of the future is something we haven't yet even seen we haven't imagined I don't know what it's gonna be yeah yeah it's going to be

Jon Radoff: something that so Indy will be making an Elden ring in the future because never all the tools yeah

Jussi Kemppainen: that's that's gonna be like 25 people in the indie team making Elden ring yeah or maybe even

Jon Radoff: fewer maybe it'll be you know echoes of somewhere will be the team that does that someday and it'll be

Jussi Kemppainen: and it's it's impossible to say but I I'm really interested in seeing like I kind of in my head I know what the indie games will be like what the low budget games will be they will be the same games that we play today but instead of being made by 250 people they're made by 10 but what will the 250 people do in the future that I really want to see like what crazy stuff is it's gonna come from the big studios that are gonna utilize AI to the full extent of it like it's going to be mind

Jon Radoff: blowing in your own area of expertise as a creative director and a VFX artist what would you tell the person who's maybe college age right now just thinking about or hoping to get into game development like what are the skills they should know like right now to enter a game development studio with relevant skill set given what you're seeing in generative AI it's quite easy today because you

Jussi Kemppainen: can get an engine like Unity or GoDot or Unreal and start learning so I think a really good

Jussi Kemppainen: option is just to make games just make games on your own at home and and you don't need to publish them you can just post them on YouTube maybe just have something to show and then don't shy away

Jussi Kemppainen: from junior positions or like I don't know there's this debate on this internship like free unpaid internship is that something that should be even considered is that like but that's a valid

Jon Radoff: path to a games company the number one thing I always told people when I was running a game studio before I was focused on the infrastructure of building games which is what we do at B-Mobile I always told people you have to have a portfolio of examples game development is very very hard to enter this field so it's really hard unless you're able to clearly communicate what you're capable of and that means sit down in front of Unity or Unreal and just start making things and don't worry about it so much don't worry about putting it online and worrying that someone is going to like it

Jussi Kemppainen: everybody knows that you're just starting in your career they'll be impressed that you're the one

Jon Radoff: out of 10 or maybe one out of 50 people that actually makes things and puts it online because

Jussi Kemppainen: that's what it immediately sets you apart. Yeah yeah and it's even if you want to be VFX artists do VFX learn how to do VFX in a program there are so many good resources to find education online if it's if it's video VFX you want to do there's there's YouTube tutorials for that it's mainly that YouTube is a great place to learn it's amazing place because it has so much content that you can just enjoy it like that didn't exist when I was learning these things that we didn't have YouTube and we could look into like you'd have to look at it yourself like find everything out yourself but kids these days have it so easy because everything is on YouTube or somewhere

Jussi Kemppainen: else on the net it's so easy to find it's just just go find that and then do the work and then

Jussi Kemppainen: publish it somewhere it doesn't need to be amazing it doesn't need to be the best you can do it

Jussi Kemppainen: just as long as you have stuff you you can show and then apply for realistic positions too of course

Jussi Kemppainen: like you are unlikely to get a lead level job in a company with your skills that you just kind of developing yourself at home but you can get your foot in and one option is also one thing that

Jussi Kemppainen: is very important you don't need to go to the right position immediately you can go into games

Jussi Kemppainen: company as an as an office assistant you can go into games company as a game tester or you can go into games company as a cleaning lady or whatever you want to go in once you're in there and you start networking with those people and then you start talking to them and you start showing your stuff at some point they will have an open position for a job that you're going to apply and then you have already your feet in the door you have been getting paid for this while you even if it wasn't

Jussi Kemppainen: your dream job like it might be possible to like that happens so much where people just shift

Jon Radoff: roles in a company hold a time yeah 100 that's I hired an office manager once and after a year or two she ended up getting hired on as the producer for a game we were working on I hired a customer service person who ended up going into QA and then from QA into a game design position so these things happen but you probably want to start by hanging out with the people that you want to emulate a lot and if the people you want to enter it's game developers start being with game developers all the time it'll it'll make a big difference in your life I guess the my take on this whole field as it emerges with AI is first of all you know everyone's always super optimistic about how these things are going to play out everything is always harder in game development than they initially look at anyway so this stuff is probably going to be harder to put into practice in some ways but I guess the biggest thing the biggest thing I guess from just my own standpoint and it matches what you were just kind of relating is you know there's so much material out there for you to learn the state of the art in a field so whether you want to be a VFX artist or a game designer or a coder or whatever it is that you love doing around game building metaverse simulations virtual worlds whatever it is like there's the things out there so that you can effectively speed run your way to the state of the

Jussi Kemppainen: art in any field I think that would be the aspiration of anyone right now get to the state of the art

Jon Radoff: fast start actually building stuff that is using state of the art the AI tools will then help you at the boundary of that you'll start to see where AI helps don't worry so much about like replacing it'll replace the things that are kind of not state of the art anyway you don't want to be doing those kind of tasks so you've been putting stuff online about the generative pipelines you've been creating so let's take a little bit of time just to step people through like a basic graphics pipeline that you could actually sit down and build for yourself today maybe a good place to start would be the work you did taking mid-journey assets bring it into F spy and then blender and then bring it into unity maybe talk through for people like what you could do if you're watching this and you want to try out some of the stuff that you've been demonstrating online what should people

Jussi Kemppainen: learn? Projection mapping is the name of the technique so it's an age old technique it dates back to I think it might even be one of the first things that you could actually do that if you want to do the effects for film I didn't know maybe like projection mapping has been how it's been here forever and it's basically what you do is you take your image and you imagine like if you had a video projecture placed exactly with a camera is and it would project a photograph on a surface and then you had another camera that you could move around and it would feel that there's actual depth to it of course of course there is actual depth to it but you kind of get for free all the lighting and all the little details in there I've been doing projection mapping for forever I think I started to do the technique in 2005 on some animation stuff we did the background for the animation with projection mapping where an artist would paint the background and then I would model it and then we could add some camera movement in it so it would feel 3D but it was actually just a 2D painting once I got the AI tools and I wanted to use it for games I didn't necessarily need that the project mapping first but I figured it would be easier to place the game content in the 2D image if it was in three dimensional space once you get the image out of mid-journey or any other tool it could be a photo it could be a drawing it doesn't need to be AI it can be it can be a watercolor painting or oil painting or whatever you want it to be

Jussi Kemppainen: all the techniques are the same after this point so then F's by is a tool that will

Jussi Kemppainen: it will figure out the vanishing points in the image like the perspective of it with a couple of guided to guidelines it will give you the camera position and the focal length and the F of E of the image so that you can start placing 3D meshes on it and F's by very nicely integrating to blender so if you go into blender then and you open up the file you will have the camera set up and you will have the images a background in your in your viewport and then you can just start placing boxes and squares and flats and whatever things in the scene and you just start flushing it I found a couple of tutorials online on this topic how you can do projection mapping and actually

Jussi Kemppainen: on on on a mid-journey image I think they were started they started popping up like a couple of

Jussi Kemppainen: weeks ago which they kind of outline my workflow exactly which was really nice so I could then just share those instead of having to do a full-on tutorial myself I could just point people to like hey this is this is a video online she does exactly the same thing that I do and she explained it explains it way better so those are really nice places to go if you're looking for the pipe it's it's very simple the pipeline is very simple you have a 2D image and then you figure out the vanishing point in the image so that you can start placing geometry into the image and then project the image on the geometry and if you don't move your camera at all you can get away with very simple geometry if you have one to have a moving camera then you need to model all the details in there and make sure that everything is detailed so that when the camera moves you don't get the the flat perspectives kind of revealing that it's just an image so you see you you covered a lot

Jon Radoff: about this the art and technique of projection mapping a lot of the work you're doing it seems in your indie game project is kind of bridging the gap between this 2D world and the 3D graphics world tell me a little bit more about that so that's something that I've done quite a lot in my career

Jussi Kemppainen: I've done I've done puppet animation I've done live action I've done hand-drawn animation pretty much any technique that you can think of I've done at some point maybe it's been a music video or tv ad or a game or for a film and most of the time you have to match techniques you have to have something that's a puppet animation and then you have something that's 3D or you have hand-drawn stuff but some things in there are 3D particles and you have to make sure that these different styles don't stand out from each other but they match together so with with echoes I really wanted to see how far I could push matching the AI backgrounds to anything 3D on it it would be reflections lighting how the depth of field effects work with the characters and the backgrounds and the atmospheric effects and little blinking light in the background try to bring the background to life so that it feels more than a painting so I've kind of approached this project as if it was like a video production instead of game production because I've been able to use compositing techniques and things that I've learned in a linear media like animation or movies to apply them in games to blend the background and the foreground together which has been a really nice kind of a marrying of these two facets of my professional life games and animations and kind of very nicely used techniques from

Jon Radoff: both of the professions maybe that's another role for generative AI is providing like a glue between some different domains that ordinarily you wouldn't be able to bring them together as easily because you'd always need that other person who like shuttles code and content between the two systems

Jussi Kemppainen: yeah it's possible and there's many steps in the in a course at the moment AI is only helping with the two these stuff but there's more tools coming and there's there's many layers to the things that I need to do for for the background image to be actually usable in the game I have to over paint stuff I have to do manual editing to it to to kind of isolate things that then I can use in the for the 3D rendering it would be nice if there was an AI that knew how to do those things just beat it up even more so that there would be less for me to do this groundwork and more I could just focus on the big picture and focus on selecting the background and then it would basically be just usable instantly in the 3D scene you could imagine a future where there's an AI that's

Jon Radoff: going to somehow figure out a lot of the projection mapping step for you and then maybe it even then figures out based on what it's seeing in the scenes like materials and lighting setup and stuff that could be automated it's code and unity like it based on what I'm seeing it looks like

Jussi Kemppainen: it's all coming and it's coming at us pretty fast the 3D projection mapping thing I think it's

Jussi Kemppainen: very close I how to derive depth information and then turn that into a mesh there's already tools for that the meshes are still quite poor quality they are maybe too low res or too high res so it's hard to get this perfect mesh but when the camera is not moving for thing like mine I think I think the ultimate tools could be quite usable already I haven't yet tested them because it's so fast to do the projection mapping it takes like 10 minutes per scene so that's it's it's not a step that yet needs to automate but for a complicated scene it might take an hour so then it would be

Jussi Kemppainen: nice to have an AI dude I've talked to a lot of modelers who if you could just get rid of the

Jon Radoff: complexity of UV unwrap they're so happy yeah we have to do that again or rigging a rigging is

Jussi Kemppainen: close but it's still you kind of still have to hand manually touch it but UV mapping is I hate it it's it's there's no joy in it it's not it's not something that if there is no one thing I doesn't really relate to anything but I figured out that to do UV mapping you can instead of doing the unwrapping in the UV editor you can just do a morph map and then just open up the mesh in the 3D editor using the 3D tools and then just do like a camera projection UV from the opened up version of the mesh and that to me it felt more fun I don't know to do it and just in the 2D space try to figure out the unwrapping but it still isn't fun it's it's it's tedious thing and I'd love

Jon Radoff: there to be an AI too to just well I know that a lot of my audience for for this is going to be people building some of those technologies so you you heard it here from a person who's a professional working on this stuff every day fix UV unwraps and you have a product to sell please please do it and that's a good reason why you've gone with this adventure game format as well you don't have to worry about the camera positioning and and we'll include a link to those tutorials that you were referring to in the show notes so the people can easily find that I think that'll be sure that's a good idea but one of the interesting comments that I heard when I shared some of the work that you've done is is people get kind of naively saying stuff like oh well people don't play adventure games anymore but I think that's so completely wrong like okay so that's not necessarily a AAA game it's not Eldon Ring but just because it isn't a game for a mass market of tens of millions of people that are going to buy that day one that goes back to your earlier point which is yeah it doesn't think would be there is a market for all kinds of games right so you can make this game now and realize it you don't need to sell 10 million copies I suspect you'd be super happy if you sold a few hundred thousand copies of a game or maybe even less as an indie game developer right you don't have to have those numbers great something good yeah yeah yeah yeah with this with with the way this goes

Jussi Kemppainen: it costs me pretty much nothing to make the game of course software licenses I have to pay for

Jussi Kemppainen: and that kind of stuff but otherwise it's it's free like I do it as a hobby and then once the game is

Jussi Kemppainen: out I'm gonna release the game as a freeware game so it's not gonna make a dime which is I think it's at the moment when AI is in its infancy I feel comfortable making a game that's free I think that's something that will also help me navigate the possible backslash that there might be in the community because sometimes like at the moment AI is some some AI is ethically in question like is it is it okay or not those issues will be solved real soon but while they haven't yet been and while I haven't kind of invested anything into the game it's fine to make it's a freeware project but but as you said even if I charged for the game if it was like two dollars or ten dollars or whatever the cost would be it wouldn't need to sell many copies to break even it would have to sell one copy and I'd be making profits from it and I think that's that's the that's the video of using being able to use AI in the future is to take risks make games that don't need to sell like you don't you don't need to make the game that will be like you kind of have to make enough money to be able to make the next game and if you have to make quarter of the money that a game today has to make you can make four times four times more meaning they can make that you can make how do you how do you even say this in English you can make four times the number of games you'd originally

Jussi Kemppainen: be able to make with the same same income some of the output will yeah and and why I guess my

Jon Radoff: question for the world is what why is all game development need to be quote unquote professional game developers anyway like maybe that'll be great for these the future 250 person studios they're doing these undremptive worlds but I think all these sandbox environments whether it's Roblox or Minecraft all that's shown that people just want to sit down and be creative so maybe these tools can really open up the way for anybody to kind of play with an idea in their mind that wouldn't

Jussi Kemppainen: necessarily be a commercial project yeah yeah sure you could make a game for yourself and just be happy like it wouldn't have to have any audience apart from just you like in the future when these tools are in this science fiction state where you just say to the computer I want to play a game that's like Super Mario but it will have a female lead character who's a cat and and then it will just make the game for you and you can play it like at some point that's gonna happen I don't know when in 10 years 100 years or five months who knows but it will the time will come those games won't be amazing but they will be pretty cool for somebody who just wanted to have accepted that game and I don't think those games will actually compete with the games that we have today the actual AAA titles and games like that but to be honest of course everything in entertainment competes with everything in entertainment it's about people's leisure time and so your your AAA game competes with your knitting and your Netflix competes with your throwing the ball at the wall or taking a brisk walk everything every leisure time activity of course is away from other leisure time activities that's

Jon Radoff: all right well so speaking of leisure time activities or any activities we've talked about a lot of

Jussi Kemppainen: things here that you're doing one of the things that you do in addition to all this other stuff

Jon Radoff: that we've just talked about is you're working at mainframe making a pretty interesting looking I call it a sandbox game I'm not sure that's the exact words that they use but it seems to be about straight you call it like tell us a little bit about the game that you're working on and why should people you know head off to steam and and wait list and solve for it right now yes so we're working

Jussi Kemppainen: on this massive massively multiplayer sandbox game I don't want to use it's social sandbox it's what it's called yeah so it's a game it's this like this world that we prepared for people to just live in their digital lives the major major many many many already but many of the founders of the company are from even online even online developers who have been just dreaming of this

Jussi Kemppainen: dreaming of a similar game but for fantasy so having this player economy like everything is

Jussi Kemppainen: player driven so in the game if you find a blacksmith who sells horseshoes it's not an NPC but it's an actual player who has just a little shop selling horseshoes so it's it's it's a very very nice idea of this like this just this world that you can living which I kind of find really

Jon Radoff: interesting yeah I can remember doing certain things like that in Star Wars galaxies we've we've talked to Raff Costa the the developer of Star Wars galaxies on building the metaverse and of course you did stuff like that then I ran a little shop when I was playing Star Wars galaxies and it had such a compelling sandbox environment for just building towns and things where people could hang out with each other and of course since then we've had Minecraft and you can build game experiences although not in like big worlds and things like Roblox it seems like a part of the market where there's just still a big gap even online I think of I played it a few times it I always remembered like the spreadsheet aspect of like figuring out all the mechanics is where you'd spend a spend a lot of time I'm looking forward to something where you get to kind of experience it more viscerally so yeah yeah yeah I don't think I don't think our game will have the spreadsheet

Jussi Kemppainen: any of that spreadsheet stuff that may matter yeah it's it's more it's more it's more more of like like an RPG where you actually get to get to pretend to be the character that you want to be

Jussi Kemppainen: get to play out the life of the character instead of instead of doing this complicated stuff like

Jon Radoff: I'm curious if the work you've done on that now as I understand it there's there's no generative AI in this it's a sandbox world but how has the work you've done on that game maybe informed you're thinking around the kinds of things you like to then work with as call it a hobbyist in any game developer how has it informed sort of gaps and opportunities in game development

Jussi Kemppainen: I'm not sure I kind of tend to do things outside of work that I oh yeah well yeah when it has informed then yeah what I get to do at work I don't usually do as a hobby so when if I'm if I'm doing something at work and that fulfills my need to do that stuff then when I start doing my hobby stuff when I go to my leisure time activity which at the moment is making a game using AI I I don't feel the need to do the same things so it's kind of just like it's almost like I have I have like my main course meal and then I just want to search and but if I have a dessert at work I don't I don't want to have a dessert after then I maybe have some salty snacks as a hobby afterwards so it's kind of that's how it's influencing my

Jon Radoff: in my work as an indie dev so I there was actually just a really interesting demo made by a guy who works at Unity showing maybe in the future shader graphs will be implemented with natural language what do you think of the prospect of more automation and generative AI maybe dropping down a level to these things that have been pretty technical domains like shader programming you know it's it's not exactly a newbie task when it starts working on graphic systems I think it's

Jussi Kemppainen: might be a good thing like it's it's such a delicate thing because you might make mistakes that make the game run real slow so I think it would require an oversight from a person and I think for me if I would have an AI that would do shaders for me I would just open it up and it wouldn't be perfect

Jussi Kemppainen: I would have to look at the shader graph that it made and I try to reverse engineer what they were

Jussi Kemppainen: doing or want they wanted to achieve with the shader and after like 15 minutes I give up and then I just scrap it and I do it from scratch because it takes me so long to try to understand what's happening in the shader but it's quicker just to do it from scratch maybe that's another use of

Jon Radoff: generative AI because I've seen uses of chat GPT for example where it's like explain what this

Jussi Kemppainen: program that was to me right so so if you're a shader programmer don't don't lose heart yet it's

Jon Radoff: so complicated you probably are going to spend some time learning optimizations and tweaking these things and really have to learn yeah yeah but this this will like I think I think the shader

Jussi Kemppainen: programmers that are working now will implore their job but this will make other games that don't

Jussi Kemppainen: have a shader programmer have much more cooler shaders because people who don't know how to do it

Jussi Kemppainen: can suddenly do it like it's not going to be perfect and then once you start to if you make a

Jussi Kemppainen: shader using AI and then you have to go in and fix it slowly you begin to understand what's happening in the shader slowly you begin to understand what's going on like how is it how's it working out like for me shaders were pretty straightforward because I did a lot of compositing with a compositing before I started to work on shaders for reals and and once I understood that shaders are just the same as you would be doing compositing on a new or or or other software like other node-based software for compositing it's just shader programming with nodes and then first I did shader programming with with just with just node pad but once nodes were added into the picture it was just easy as slipping on a like a slipping into something that you're done before

Jussi Kemppainen: like it was just going back to compositing in Nuke but in just in Unreal or Unity

Jussi Kemppainen: like I've done made like I've only done node-based shader stuff for two years now before that I only did just by writing code well that's another way that people could think of

Jon Radoff: these technologies is think of it as a learning tool maybe it won't build everything for you maybe it's like you said you'll throw away the output even that it generates but you'll learn so much in the process I think of like the current state of AI it's more like a collaboration it's similar to what you said about having another team member so you know you could probably learn a lot about shader programming even from a really bad shader programmer if you've never learned anything before because it'll teach you how the stuff works you'll also learn about some of these optimization problems that you're referring to where the game's going to grind to a halt and be three FPS

Jussi Kemppainen: and sometimes sometimes there's something that's so difficult to do that it will be nice to have an AI for echoes I wanted to make grass that would take a pixel of an image like for example this pixel of this video screen is white this is black so I wanted to have a shader that takes the people's position of a mesh which I had at the base of the of the grass blade and just colors the grass blade with that color fully so it's like a post-production post-process effect almost it takes the rendered pixel and it just stretches it upwards but to get the position from the to make a 3D point to screen space in unity was so difficult there was no I couldn't find anything online I tried to google it for hours and hours it took me eight hours to figure out the shader code how to get the 3D position to 2D space just by just trying it over and over and over again and I was just for the first three hours I went nowhere it just didn't happen and I just gained more and more convinced that it's not possible to do but I just kept at it and slowly little by little I made little breakthroughs and then in the end it's pixel perfect it gets the actual color of the on the screen and it can apply it on the 3D mesh and then you have this so I used it for when I AI does painting and it has grass in it I wanted the grass this way in the wind so now that I just paint with with the scattering tool I paint 3D grass blades on the mesh with the painting and so it just kind of extrudes these grass blades that match the painting perfectly and when the character walks into the grass the shadow that's projected on the ground is picked up by the shader and then the shadow color is on the grass and it's just an unlit material that takes no time to process on the GPU it's super fast and it looks real nice but it took me forever and I would have been so happy to ask an AI how do you transform 3D point into 2D space on the screen as a 2D coordinate and just to get an answer would have been so nice I would have saved eight hours AI is a creative partner that's

Jon Radoff: that's what people ought to be thinking of right now and embrace the technologies they're there they can really really help you and I think what you said earlier about your shader programming experience in this case is it's like a lot of game development a lot of it ends up being magic in the end and it starts out feeling very impossible at the beginning you just have to throw yourself into so many of these things and just start hacking away and try things so if AI can help speed up some of the hacking process to point you towards patterns or things people have figured out already all the better but the end of the day you just have to start learning opening your mind to it trying these things out and game development's hard right we're dealing with technology and art and craft and so many aspects if if we can make pieces of it a little bit easier so that you can focus on the creative vision you have for an idea yeah that's the opportunity in my mind

Jussi Kemppainen: yeah and maybe maybe one helpful note for people who will start making this stuff is that

Jussi Kemppainen: you don't need to know what you need what how to do stuff going in like you can start working on things that you have no idea how to finish like I love it when people ask me like when I get

Jussi Kemppainen: an assignment at work do something that you've never done before and somebody asks me like can

Jussi Kemppainen: you can you do this I always say yes of course I can like why wouldn't I be able to do it of course time might be an issue but like every time I get an assignment that I don't know how to finish I'm super pumped like after this thing I have a new skill like it is it's the best yeah keeps life

Jon Radoff: interesting life is not going to be boring especially with the rate of change that these technologies are experiencing now so you see this has been a really fun conversation I think everybody's going to really be inspired by a lot of what you had to share and we're going to provide links to some of the materials that you're referring to some your own work will point people at Pax Day it's a game you should definitely try out and some of these other tutorials that you referred to as well but generative AI is here it's here to stay it's going to change probably every part of the game development process and I'm so happy you were able to spend a little bit of time talking

Jussi Kemppainen: about your own experiments in this domain