• First of all I’d like to credit the Institute for Creative Technologies for the amazing performance capture provided for the animation (http://ict.usc.edu/prototypes/digital-ira/). Their new capture technology enable photoreal facial animation performances together with extremely detailed skin features. The full team behind the capture, leaded by Paul Debevec, is the following: Oleg Alexander, Graham Fyffe, Jay Busch, Ryosuke Ichikari, Abhijeet Ghosh, Andrew Jones, Paul Graham, Svetlana Akim, Xueming Yu, Koki Nagano, Borom Tunwattanapong, Valerie Dauphin, Ari Shapiro and Kathleen Haase.
• Second, it has been quoted in numerous sources if this was related with the Nvidia FaceWorks presented in GTC. I’d like to clarify that we both use the same performance capture from the ICT, but the animation and render engine is completely different. In other words, same source data but different engine.
• Finally, I’d to clarify that the technology we presented runs in its higher quality preset at 93/74 fps at 720/1080p respectively, in a GeForce GTX 560 TI (a two-year old mid-range GPU).

Thanks to all the people who showed interest on our reseach, the slides will be available pretty soon online!

Our talk in GDC 2013, Next-Generation Character Rendering, is a few hours away. On it, we will present what represents to us the culmination of many years of work in photorealistic characters.

We will show how each detail is the secret for achieving reality. For us, the challenge goes beyond entertaining; it’s more about creating a medium for better expressing emotions, and reaching the feelings of the players.

We believe this technology brings current generation characters, into next generation life. At 180 fps in a Geforce GTX 680.

The team behind this technology consists on Javier Von Der Pahlen (Director of R&D), Etienne Danvoye (Technical Director), Bernardo Antoniazzi (Techical Art Director), Zbyněk Kysela (Modeler and Texture Artist), Mike Eheler (Programming & Support) and me (Real-Time Graphics R&D).

You have a teaser of the slides here:
Next-Generation-Character-Rendering-Teaser.pptx

A movie:

And the YouTube account of Activision R&D:
http://www.youtube.com/user/ActivisionRnD

We will show it running in our two-year old laptop, in live.

Hope to see you in our talk!

Off On

Off On

Off On

This wednesday I’ll talk about it in the Advances in Real-Time Rendering course (in SIGGRAPH 2012), I invite you to come by for all the gory details!

The R&D team behind this shader consists on Javier Von Der Pahlen (Technology Director R&D and Photographer), Etienne Danvoye (Technical Director) and me (Real-Time Graphics Researcher). Also participated Zbyněk Kysela (Modeler and Texture Artist), Suren Manvelyan (Photographer) and Bernardo Antoniazzi (Techical Art Director).

Performance wise, rendering the eyes takes a 2%-3% of the whole demo. This is work in progress, but here you have some shots showcasing our shader:

Eye Shader Off Eye Shader On

Eye Shader Off Eye Shader On

Eye Shader Off Eye Shader On Inspiration for this Shot

SMAA 1x natively integrated in latest ARMA 2: Operation Arrowhead beta, and in 1.05 update of Take On Helicopters!

These last months I’ve learned a very important lesson: efforts towards rendering ultra realistic skin are futile if they are not coupled with HDR, high quality bloom, depth of field, film grain, tone mapping, ultra high quality models, parametrization maps, high quality shadow maps and a high quality antialiasing solution. If you fail on any of them, the illusion of looking at a real human will be broken. Specially on close-ups at 1080p, that is where the real skin rendering challenge is.

As some subtleties (like the film grain) are lost on the online version, I encourage to download the original blu-ray quality version below, to better appreciate the details and effects rendered (but be aware that you will need a powerful computer to play it). Please note that everything is rendered in real-time; in fact, you can also download a precompiled version of the demo (see below), which shows the shot sequence of the movie, from its beginning to its ending. The whole demo runs between 80 and 160 FPS, with an average of 112.5 FPS on my GeForce GTX 580. But it can be run in weaker configurations by using more modest settings.

The main idea behind the new separable SSS approach is that you can get very similar results to the full 12-pass approach ([Eon07]) by just using a regular two-pass setup. It can be done in screen space and is really really fast (you may want to see this related post). I hope to write something about this in the future. However, the source code of whole demo is readily available on GitHub.

For the demo I’ve used SMAA T2x, which does a very good job at dealing with shader aliasing while avoiding pre-resolve tone mapping. The demo shows the average/minimum/maximum frame rate after running the intro, which hopefully will make it useful for benchmarking GPU cards.

I think there is still a lot work to do. Probably the most important one will be rendering realistic facial hair. It will be my dream if my skin research helps to improve the rendering of humans in games; I truly believe that more realistic characters will inevitably lead to deeper storytelling, and more emotionally-driven games.

Links:

• Precompiled demo [47.7 MB]: Download Torrent — Hit space to skip the intro, and go to the interactive part
• Source code: GitHub
• Blu-ray quality movie [693 MB]: Vimeo Mirror Download Torrent — On Vimeo, search for “Download this video”

The 3D head scan used for this demo was obtained from Infinite Realities. Special thanks to them!

Checkout the subpixel features section of the movie to see the new modes in action (or download the precompiled binary).

We are also happy to officially announce that all the nuts and bolts of the technique will be presented on EUROGRAPHICS 2012, in Cagliari (Italy). The paper is now much easier to follow, and has updated content. To all who tried to read the technical report and failed to deeply understand all the details, we definitely encourage them to give the EUROGRAPHICS version a try, as it does a much better job at explaining the core concepts and implementation details.

We’d like to thank all the community that is supporting SMAA, and give special thanks to Andrej Dudenhefner, whose hard work made InjectSMAA possible, allowing to use SMAA 1x in a lot of already published games. But bear in mind that SMAA 1x is just one third of the whole technique!

And to finalize this post I would like to remember some of the design decisions and key ideas of our technique, which were not completely understood on the past. I clarified them on a Beyond3D thread but I would like to give it another round here.

The main design decision behind SMAA is to be extremely conservative with the image. SMAA tries to analyze the possible patterns that can happen for a certain pixel, and favors no filtering when the filtering decision is not clear. As our paper shows, this is extremely important for accurately recovering subpixel features in the S2x, T2x and 4x modes. Additionally, super/multisampling will cover these cases, yielding smooth results even when the MLAA component is not filtering the pixel.

Furthermore, the MLAA component is enhanced, not only by improving the searches accuracy without introducing dithering or performance penalties, but by also introducing an stronger edge detection and extending the type of patterns detected, as shown in this example from Battlefield 3 (to easily compare the images, we recommend opening the images in browser tabs, and keep them at their original resolution):

SMAA MLAA FXAA

Finally, SMAA is not devised as a full MSAA replacement. Instead, the core idea behind it is to take the strengths of MSAA, temporal SSAA and MLAA, and make a very fast and robust technique, where each component backs up the limitations of the others, delivering great image quality in demanding real-time scenarios.

We joined forces with Tiago Sousa from Crytek, to deliver a very mature temporal antialiasing solution. It has been integrated into CryEngine 3, checkout the SMAA demo movie.

The goal of SMAA is to more accurately match the results of supersampling, by faithfully representing subpixel features, and by solving other common problems of filter-based antialiasing. This reduces the flickering seen in complex scenes.

On the other hand, we made really big efforts towards simplifying the usage of our shader; the source code is now reduced to a single SMAA.h header and two textures, with very detailed comments and instructions. We hope this will ease integration into game engines. The feedback so far has been very positive, with a Java/Ruby programmer with no experience in graphics integrating SMAA into Oblivion (using OBGE) in just a few hours.

SMAA, Subpixel Morphological Antialiasing

It solves the key weaknesses of MLAA by introducing the following features:

• Improved pattern searchs
• Improved pattern detection
• Diagonal handling
• Avoiding the general roundness introduced by MLAA
• And probably the most important: subpixel features.

We’ve decided to release it as a technical report, available here:
SMAA: Enhanced Subpixel Morphological Antialiasing

Please note that some information is outdated (for example the timings of FXAA and SRAA, as they are now faster); we will be updating it in the next weeks. The souce code builds on top of Jimenez’s MLAA v1.5, so we will be posting patches with each improvement we added, with all the nut and bolts details, so stay tuned!

We’ll also be talking a little about it in the SIGGRAPH course Filtering Approaches for Real-Time Anti-Aliasing, we invite you to come by!

The SSS effect is now fully customizable, including the generated gradients colors, the per-channel filter strength and more. We also setup various defines that allows to customize the quality, allowing to adapt the shader to diverse hardware targets. The shader takes 0.27ms@720 and 1.69ms@1080 in the lowest and medium settings respectively, in a GeForce GTX 295 (we used the medium setting for these shots). We are releasing the source code privately to interested game developers, with a public release being scheduled for 2011 Q4.

I recommend looking at the images at their native resolution to better appreciate the details.

Without SSS With SSS

Without SSS With SSS

I’ll be talking about this in SIGGRAPH 2011, in the session CG in Europe (Wednesday 10th from 11.30 to 12.30, in the International Center, Harbour Concourse). If you are attending SIGGRAPH 2011, I invite you to come by!

The 3D head scan used for the images was obtained from Infinite Realities.