Andy Serkis showcases the future of motion capture

A hint at what PlayStation 5 can offer?

Naughty Dog through Uncharted and The Last of Us has shown how mo-cap (motion capture) can change game development and bring a level of dynamic realism never been seen in gaming. This level of detail not only means fancy graphics for trailers, but more intimacy between gamers and the characters they see on screen as well.

With PS4 Pro hitting the threshold of what’s possible on current hardware with the released Horizon, the soon-to-arrive God of War, and the likely 2019 The Last of US: Part II, articles are begining to pop up about what we can expect with PS5.

For the most part, I find articles like them nothing but clickbait and speculation and while this piece is a bit of the latter, there’s also real tangible tech of today on display which was unveiled at GDC that could be ready for prime time in just a few years’ time.

Brian Crecente for Rolling Stones:

Serkis, best known for his work as Gollum in the Lord of the Rings films and Caesar in the Planet of the Apes films, introduced new work being done by Epic and 3Lateral to better capture the look of an actor and recreate that person or any person using their recorded performance as a believable, near-human digital recreation.

To demonstrate the technology, 3Lateral founder and CEO Vladimir Mastilovic showed off a recreation of Serkis performing a piece of Shakespeare’s Macbeth. The fidelity, which was surprisingly human, was running in real-time using the new tech built on the back of Epic’s Unreal graphics engine. To prove that the video wasn’t simply a canned, pre-rendered performance, Mastilovic had the video run a second time, this time raising and lowering the digital Serkis’ eyebrows and moving his eyes around, manipulating where he was looking.

Vladimir Mastilovic, 3Lateral founder and CEO:

The implications of this technology are many. We have the digitized actions of Andy Serkis and it can be used to reshoot a scene, react it through a different person or even preserve it in case when he is older he wants to act out the scene as a younger man.

As for how they pulled off the whole thing:

According to Mastilovic and Serkis (who has absurdly not yet won an Oscar), we can expect to see this tech in future video games, films, and TV shows. As for the PlayStation 5 connection (and this is pure speculation on my part), with the PS4 successor still a few years off, is it that far fetched to assume we’ll see this engine’s SDK arrive on the next-gen console? After all, Unreal Engine powers the current PS4 and PS4 Pro, too.

You can read more about the real-time rendering over at the Unreal Engine website.

Thanks for reading! If you liked this post, let me and others know by 👏 below (you can 👏 more than once!) which also helps the site grow!