News

Abstract

Cover

Virtual characters in games are becoming more and more realistic, with recent advances for instance in the fields of skin rendering [d'Eon and Luebke 07, Hable et al. 09, Jimenez and Gutierrez 10] or behavior-based animation [NaturalMotion 05]. To avoid lifeless representations and help the user engage in the action, more and more sophisticated algorithms are being devised that capture subtle aspects of the appearance and motion of these characters. Unfortunately, facial animation and the emotional aspect of the interaction have not been traditionally pursued with the same intensity. We believe this is an important missing aspect in games, especially given the current trend of story-driven AAA games and their movie-like real-time cutscenes.

We present a method to add expressive animated wrinkles to characters, helping enrich stories through subtle visual cues. Our system allows the animator to independently blend multiple wrinkle maps across regions of a character's face. We demonstrate how combining our technique with state-of-the-art real-time skin rendering can produce stunning results that bring out the personality and emotional state of a character.

This enhanced realism has little performance impact. In fact our implementation has a memory footprint of just 96 KB. Performance wise, the execution time of our shader is 0.31 ms, 0.1 ms and 0.09 ms on a low- end GeForce 8600GT, mid-range GeForce 9800GTX+ and mid-high range GeForce 295GTX respectively. Furthermore, it is simple enough to be easily added to existing rendering engines without requiring drastic changes, even allowing to reuse existing bump/normal textures, as our technique builds on top of them.

More details found in the GPU Pro 2 chapter. Buy the book!

Downloads

Shader

float3 WrinkledNormal(Texture2D baseTex,
                      Texture2D wrinkleTex,
                      Texture2D maskTex[2],
                      float4 weights[2],
                      float2 texcoord) {
  // Fetch and unpack base normal (3Dc format):
  float4 base;
  base.xy = baseTex.Sample(AnisotropicSampler16, texcoord).gr;
  base.xy = -1.0 + 2.0 * base.xy;

  #ifdef WRINKLES
  // Fetch and unpack wrinkles normal (3Dc format):
  // (a linear sampler is enough for the low-frequency wrinkles information)
  float2 wrinkles = wrinkleTex.Sample(LinearSampler, texcoord).gr;
  wrinkles = -1.0 + 2.0 * wrinkles;

  // We're going accumulate wrinkles on xy and zw simultaneously, so reset zw
  // to zero:
  base.zw = 0.0;

  // Fetch the first mask and apply the weights:
  float4 mask1 = maskTex[0].Sample(LinearSampler, texcoord);
  mask1 *= weights[0];

  // Accumulate the first mask wrinkles:
  base += mask1.rrgg * wrinkles.xyxy;
  base += mask1.bbaa * wrinkles.xyxy;

  // Fetch the second mask and apply the weights:
  float4 mask2 = maskTex[1].Sample(LinearSampler, texcoord);
  mask2 *= weights[1];

  // Accumulate the second mask wrinkles:
  base += mask2.rrgg * wrinkles.xyxy;
  base += mask2.bbaa * wrinkles.xyxy;

  // Accumulate the wrinkles:
  base.xy += base.zw;
  #endif

  // Normalize the result:
  base.z = 1.0;
  return normalize(base.xyz);
}
float3 WrinkledNormal(Texture2D baseTex,
                      Texture2D normalDiffTex,
                      Texture2D maskTex[2],
                      float4 weights[2],
                      float2 texcoord) {
  // Fetch and unpack base normal (3Dc format):
  float3 base;
  base.xy = baseTex.Sample(AnisotropicSampler16, texcoord).gr;
  base.xy = -1.0 + 2.0 * base.xy;
  base.z = sqrt(1.0 - base.x * base.x - base.y * base.y);

  #ifdef WRINKLES
  // Fetch and unpack normal diffrence (swizzled DXT5 format):
  float3 diff = normalDiffTex.Sample(LinearSampler, texcoord).agb;
  diff = -1.0 + 2.0 * diff;

  // Multiply by 2.0 as the difference wrinkle map was calculated using values
  // in the range 0..1, and now we are working in the range -1..1:
  diff *= 2.0;

  // Fetch the masks and apply the weights:
  float4 mask1 = maskTex[0].Sample(LinearSampler, texcoord);
  float4 mask2 = maskTex[1].Sample(LinearSampler, texcoord);
  mask1 *= weights[0];
  mask2 *= weights[1];

  // Accumulate the wrinkles from different zones:
  base += mask1.r * diff;
  base += mask1.g * diff;
  base += mask1.b * diff;
  base += mask1.a * diff;
  base += mask2.r * diff;
  base += mask2.g * diff;
  base += mask2.b * diff;
  base += mask2.a * diff;
  #endif

  return normalize(base);
}

Bibtex

@inbook{JIMENEZ2011_GPUPRO2B,
    author = {Jorge Jimenez and Jose I. Echevarria and Christopher Oat and Diego Gutierrez},
    editor = {Wolfgang Engel},
    title = {{GPU} Pro 2},
    chapter = {Practical and Realistic Facial Wrinkles Animation},
    publisher = {{AK} Peters Ltd.},
    year = {2011},
}
@article{JIMENEZ2009_TAP,
    author = {Jorge Jimenez and Veronica Sundstedt and Diego Gutierrez},
    title = {Screen-space perceptual rendering of human skin},
    journal = {ACM Transactions on Applied Perception},
    volume = {6},
    number = {4},
    year = {2009},
    pages = {23:1--23:15},
}
@inbook{JIMENEZ2010_GPUPRO,
    author = {Jorge Jimenez and Diego Gutierrez},
    editor = {Wolfgang Engel},
    title = {GPU Pro: Advanced Rendering Techniques},
    chapter = {Screen-Space Subsurface Scattering},
    publisher = {{AK} Peters Ltd.},
    year = {2010},
    pages = {335--351},
}
@article{JIMENEZ2010_IEEE,
    author = {Jorge Jimenez and David Whelan and Veronica Sundstedt and Diego Gutierrez},
    title = {Real-Time Realistic Skin Translucency},
    journal = {IEEE Computer Graphics and Applications},
    volume = {30},
    number = {4},
    year = {2010},
    pages = {32--41},
}
@article{JIMENEZ2010_TOG,
    author = {Jorge Jimenez and Timothy Scully and Nuno Barbosa and Craig Donner and Xenxo Alvarez and
              Teresa Vieira and Paul Matts and Verónica Orvalho and Diego Gutierrez and Tim Weyrich},
    title = {A practical appearance model for dynamic facial color},
    journal = {ACM Transactions on Graphics (Proc. SIGGRAPH Asia)},
    year = {2010},
    volume = {29},
    number = {6},
    pages = {141:1--141:10},
}
@article{JIMENEZ2015_CGF,
    author = {Jorge Jimenez and Károly Zsolnai and Adrian Jarabo and Christian Freude and Thomas Auzinger and
              Xian-Chun Wu and Javier von der Pahlen and Michael Wimmer and Diego Gutierrez},
    title = {Separable Subsurface Scattering},
    journal = {Computer Graphics Forum},
    year = {2015},
}

Related

Links