Texture aliasing

Originally posted to Shawn Hargreaves Blog on MSDN, Friday, April 29, 2011

Aliasing can occur any time you resample a texture, for instance to scale it, rotate it, or map it onto a 3D model. Thanks to our friend the Nyquist threshold, the resulting problems get worse the lower a frequency you sample at, ie. the smaller you shrink it.

Check out this awesome 32x32 test image:

Zoomed 4x so you can see in detail:

This test code draws six copies, first with no transform, then rotated, followed by straight and rotated versions at half and quarter size:

    spriteBatch.Begin(0, null, SamplerState.PointClamp, null, null);

    spriteBatch.Draw(testTexture, new Vector2(20, 24), null, Color.White, 0, new Vector2(16, 16), 1, 0, 0);
    spriteBatch.Draw(testTexture, new Vector2(60, 24), null, Color.White, 1, new Vector2(16, 16), 1, 0, 0);

    spriteBatch.Draw(testTexture, new Vector2(95, 24), null, Color.White, 0, new Vector2(16, 16), 0.5f, 0, 0);
    spriteBatch.Draw(testTexture, new Vector2(120, 24), null, Color.White, 1, new Vector2(16, 16), 0.5f, 0, 0);

    spriteBatch.Draw(testTexture, new Vector2(140, 24), null, Color.White, 0, new Vector2(16, 16), 0.25f, 0, 0);
    spriteBatch.Draw(testTexture, new Vector2(155, 24), null, Color.White, 1, new Vector2(16, 16), 0.25f, 0, 0);

    spriteBatch.End();

Note the use of SamplerState.PointClamp, which turns off the antialiasing SpriteBatch would normally apply by default.

Resulting image:

Zoomed 4x:

This has almost too many aliasing problems to mention! Some that I find particularly bothersome are:

• The black diagonal lines have pieces missing in the full size rotated image
• The half size non rotated image has lots of problems:
• The green vertical line, and also the red and blue horizontal lines, are entirely missing
• It ended up with just two rather than three sets of black diagonal lines
• The black border remains on the bottom and right, but is missing on the top and left
• The two quarter size images are entirely different:
• Non rotated version has only a blue line, no red or green at all
• Rotated version has scattered red and green pixels, but no blue
• Imagine how horrible it would look if we animated this sprite spinning, as it would flicker back and forth between different colors!

Remember how multisampling smooths triangle edges but does not affect their interior? We can confirm this by turning on 4x multisampling, and noting how the only difference is smoother borders on the rotated images:

With 4x supersampling, on the other hand, texture aliasing is indeed reduced:

But this is still far from perfect. The full size rotated and half size non rotated images look pretty nice, but the half size rotated and both quarter size versions have flaws. We'd still get nasty color pulsation if we animated a rotation of that quarter size version. This is because 4x supersampling only gives a 2x improvement in sampling frequency along each of the horizontal and vertical axes. That can reduce aliasing, but is not enough to keep us above the Nyquist threshold if we transform the texture by a factor of more than 2x.

Whatever is a poor graphics programmer to do?

To start with, we shouldn't have made life difficult for ourselves by specifying point sampling! If we turn off supersampling and multisampling, then change our SpriteBatch.Begin call to use the default SamplerState.LinearClamp, bilinear filtering produces a much nicer image than the point sampled version we started out with:

Unlike multisampling and supersampling, texture filtering does not smooth the border of the rotated images, but it does a nice job to reduce aliasing inside our full size rotated and half size non rotated images. Bilinear filtering lets us down in the same places as supersampling. Because the GPU only takes four filter taps per texture fetch (two horizontal and two vertical), we get 2x more headroom before we will run into aliasing problems, but a 2x improvement cannot entirely eliminate the Nyquist threshold, so our quarter size images remain nasty.

The solution is to combine bilinear filtering with precalculated mipmaps (plus anisotropic filtering if you can afford it). No matter how small the texture is scaled, the GPU can now simply select the appropriate mip level to sample from, always remaining just below the Nyquist threshold regardless of what transforms are applied.

With mipmaps enabled, even our smallest images now keep the same color intensity and hue, with no stray pixels or random changes of shape, so we can animate our sprite rotating or scaling with no aliasing problems at all: