Midjourney to Unity Game Development

Jon Radoff: Maybe a good place to start would be the work you did taking mid-Journey assets, bringing it into F spy and then Blender and then bringing it into Unity. Maybe talk through for people like what you could do if you're watching this and you want to try out some of the stuff that you've been demonstrating online, what should people learn? Projection mapping is the name

Unknown: of the technique. So it's an age old technique. It dates back to like the, I think it might even be one of the first things that you could actually do that if you want to do VFX for film.

Guest: I didn't know maybe a projection mapping has been out. It's been here forever and it's basically what you do is you take your image and you imagine like if you had a video projector placed exactly with a camera is and it would project a photograph on a surface and then you had another camera that you could move around and it would feel that there's actual depth to it. Of course there is actual depth to it but you kind of get for free all the lighting and all the little details in there. I've been doing projection mapping for forever. I think I started to do the technique in 2005 on some animation stuff. We did the background for the animation with projection mapping where an artist would paint the background and then I would model it and then we could add some camera movement in it. So it would feel 3D but it was actually just a 2D painting. Once I got the AI tools and I wanted to use it for games I didn't necessarily need that the projection mapping first but I figured it would be easier to place the game content in the 2D image if it was in 3D dimensional space.

Unknown: Once you get the image out of a mid-journey or any other tool it could be a photo, it could be a

Guest: drawing, it doesn't need to be AI. It can be a watercolor painting or oil painting or whatever you want it to be. All the techniques are the same after this point. So then F's by is a tool that will it will figure out the vanishing points in the image like the perspective of it. With a couple of guided guidelines it will give you the camera position and the focal length and the F of E of the image so that you can start placing 3D meshes on it. And F's by very nicely integrates into blender. So if you go into blender then and you open up the file you will have the camera set up and you

Unknown: will have the images a background in your viewport and then you can just start placing boxes and

Guest: squares and flats and whatever things in the scene and you just start flashing it out. I found a couple of tutorials online on this topic, how you can do projection mapping and actually on on on a mid-journey image. I think they were started they started popping up like a couple of weeks

Unknown: ago which they kind of outlined my workflow exactly. Which was really nice so I could then just

Guest: share those instead of having to do a full-on tutorial myself. I could just point people to like hey this is a video online she does exactly the same thing that I do and she explained it explains its way better. So those are really nice places to go if you're looking for the pipe. It's very simple the pipeline is very simple you have a 2D image and then you figure out the vanishing point in the image so that you can start placing geometry into the image and then project the image on the geometry and if you don't move your camera at all you can get away with very simple

Unknown: geometry.