Over the last couple of months, I've been working on transitioning to a 3D format making use of motion capture and Unreal Engine. Why? Because the possibilities are endless and I was limited in what I could do with the time and resources I had. Instead of having to rely on locations, sets, people, or equipment, I can make it all myself. The tools available now are pretty incredible, and it's only going to get better. I thought I'd share since I believe this is a big portion of the future, and others may see the same doors open up that I have.
Not going to beat around the bush, it's not that easy to get into and the learning curve can be steep for some. Getting the facial motion capture to work properly takes a lot of setup and knowledge to get right. Creating the environments takes a lot of time. Despite this, Unreal Engine (UE5) is shockingly intuitive and there are plenty of resources and tutorials to guide you. It's actually easier to create than I initially thought. Best of all, Unreal Engine is free and so are a lot of assets on it's marketplace. A lot of the built in tools allow to create photorealistic shots. This is a large part of how The Mandalorion was done, both the outdoor and indoor environments.
Also, there will have to be some significant investment to make it happen. Everyone knows about hardware, which you'll want to have a good system for. The software tools that are better than others come in at a significant cost. I'm creating some of the characters in Character Creator 4, and doing animations in iClone 8 (both by Reallusion). This alone was about $1000. While a lot of resources in UE5 are free, you are ultimately going to want to get more, and those will cost. To do basic motion capture (including facial), you'll need special equipment. This can be very expensive, but a lot can be done with a newer iPhone that has a LIDAR (which is what I'm using).
Truth be told, it's all not where I would like it to be... yet. Many pieces of software have to come together that depend on each other to make some simple things happen. It's more involved than I thought it would be, but I'm still glad I made the plunge. Though it's not at that ease of use that I'd like, it was still worth jumping into versus what it was maybe two years ago. The knowledge I'm gaining and the practice with today's tools I believe will make me better when they become even more evolved.
Here is a 3D version of one of my characters that I made with CC4 (the necromancer, Carl). I'm also including a link to a simple motion capture test I did with iClone using that character. I still have to do more work on tweaking the facial mo-cap, as well as working on other characters. Don't know when, but hopefully soon I can bring this all into UE5 to create the first YouTube video in the new format. If anyone has any questions about hardware, software, or processes, please let me know.
Carl mo-cap test
Not going to beat around the bush, it's not that easy to get into and the learning curve can be steep for some. Getting the facial motion capture to work properly takes a lot of setup and knowledge to get right. Creating the environments takes a lot of time. Despite this, Unreal Engine (UE5) is shockingly intuitive and there are plenty of resources and tutorials to guide you. It's actually easier to create than I initially thought. Best of all, Unreal Engine is free and so are a lot of assets on it's marketplace. A lot of the built in tools allow to create photorealistic shots. This is a large part of how The Mandalorion was done, both the outdoor and indoor environments.
Also, there will have to be some significant investment to make it happen. Everyone knows about hardware, which you'll want to have a good system for. The software tools that are better than others come in at a significant cost. I'm creating some of the characters in Character Creator 4, and doing animations in iClone 8 (both by Reallusion). This alone was about $1000. While a lot of resources in UE5 are free, you are ultimately going to want to get more, and those will cost. To do basic motion capture (including facial), you'll need special equipment. This can be very expensive, but a lot can be done with a newer iPhone that has a LIDAR (which is what I'm using).
Truth be told, it's all not where I would like it to be... yet. Many pieces of software have to come together that depend on each other to make some simple things happen. It's more involved than I thought it would be, but I'm still glad I made the plunge. Though it's not at that ease of use that I'd like, it was still worth jumping into versus what it was maybe two years ago. The knowledge I'm gaining and the practice with today's tools I believe will make me better when they become even more evolved.
Here is a 3D version of one of my characters that I made with CC4 (the necromancer, Carl). I'm also including a link to a simple motion capture test I did with iClone using that character. I still have to do more work on tweaking the facial mo-cap, as well as working on other characters. Don't know when, but hopefully soon I can bring this all into UE5 to create the first YouTube video in the new format. If anyone has any questions about hardware, software, or processes, please let me know.
Carl mo-cap test
