Gouraud Shading

Published 2019-01-22

Let's compare and contrast the Saturn and the PC versions of Sonic R. I'll only need a screenshot, taken from an area I think has the most blatant difference.

Do you notice something very, very obviously missing in the PC version? Yes, the Saturn version has a broken background, but that's just my emulator not working properly. More importantly, it has vibrant colors that the PC version doesn't have. Why is this?

Saturn Analysis

Let's take a quick dive into the Saturn's video processor. It's a very strange but charming system. The Video Display Processor (VDP) is split into two units. The VDP1 handles all the foreground graphics, like characters, the course, and HUD elements, while the VDP2 handles the background, the floor map, and compositing the scene.

The VDP1 doesn't have polygons. What it does have are "distorted sprites", 2D bitmaps with a transformation applied to them. Each sprite can have a texture, a Gouraud table, half-luminance, half-transparency, and shadowing (post-processing half-luminance). Let's pull open Yabause to see how Sonic R renders it. (I'm using my own custom fork of Yabause, which includes a vertex color preview.)

It appears that every single polygon in Sonic R, without exception, is a distorted sprite with a texture and, if within the draw distance cutoff (as the game's lead programmer Jon Burton explains), a Gouraud table. (I am seriously left wondering if adding non-textured sprites in some places would have improved the draw distance.) How is Gouraud shading implemented on the Saturn? Looking at the leaked Saturn developer documentation, we find this:

The data interpolated for each of R, G, and B between the four points [of the sprite] are added to the original color of the part. Because each of the values of R, G, and B takes the values 00H to 1FH, the result of subtracting 10H from the complementary RGB data is added to the original color of the part. For example, if the value of RGB is 10H, the original color is left as is; if the value is 00H, the original color becomes 10H; and if the value is 1FH, then the original color becomes +0FH. If the value after color calculation becomes less than 00H, then 00H is used; if it is larger than 1FH, then 1FH is used.

Let's de-mystify that. Let's imagine we have a function that is designed to calculate the color for each pixel to be drawn. It takes in as input the original texture, four colors for each corner of the quad, and the x and y offset to render. We first have to determine the source color and the Gouraud color. We can simply look up the source color from the texture, but to get the Gouraud color we have to do linear interpolation.

In 1D, imagine you have a gradient. One side is red, the other blue. Suppose you want to get the color 33% of the way down the gradient. You simply multiply the left color by the position and the right color by (1 - the position). This is known as linear interpolation, or "lerp".

To do this in two dimensions (bilinearly), we run the 1D interpolation across the top-most and bottom-most rows. We then re-do the interpolation one more time vertically, using the two calculated values as the two values we're interpolating between.

Given our source and Gouraud colors, we now have to combine them. The Saturn subtracts half of the maximum possible Gouraud color and adds it to the texture color.

Testing our Hypothesis

I have a track exporter that exports the textures and the vertex colors. I navigated to the area and took two screenshots, one with just textures and one with just colors.

I then loaded these into Krita, the colors layed over the textures. Looking at the list of blend modes, it looks like Grain Merge was the same as the Saturn version. If I do that, it looks like this:

That's exactly what we need! How do we recreate that in the PC version?

PC Analysis

To answer that, we must first figure out how the PC version screws this up in the first place. Actually, in some versions of the game, it doesn't. The most common PC version of Sonic R from 1998 let you select between DirectDraw and Direct3D modes. (It also has a lot of issues running on modern versions of Windows. This analysis is based on the fixed 2004 version.) Here's a quick comparison of the two at our favorite spot.

The DDraw version looks just like the Saturn version! As it turns out, the DirectDraw version uses it's own software renderer to get the look just right. The Direct3D version instead sends the polygons to the GPU as any modern game would. The 2004 version doesn't have DirectDraw, and even if it did it's too limited and slow to be useful at high resolutions. We must find a way to fix the Direct3D mode.

Let's look at a byte in the file that corresponds to a color component in a texture on the track. After the track file gets loaded into memory, it goes through a function I've dubbed Track_TintGourauds. It attaches the colors to the track vertices so that the GPU knows what to apply to what polygon. However, this function acts differently depending on whether the render mode is set to Direct3D or DirectDraw. In DirectDraw mode, the colors are unmodified. However, in Direct3D mode, it adds 96 to each Gouraud value, causing most of them to saturate at 255. This gives the look that barely anything is shaded at all. If we remove that check, we get this:

It's pretty dark, but we can see that the color information is still there and being processed, just incorrectly. But how so? Let's use a GPU debugger to find out. My personal favorite is PIX for Windows, which comes with the DirectX 9 SDK. (The 2004 version of Sonic R uses DirectX 9. The more common 1998 version uses DirectX 5, for which almost no tools exist.) I do however wish to give a shoutout to apitrace, which is better at handling recordings but comes with less analysis options.

Looking at the results we find nothing particularly unusual. The scene is set up with some functions that set everything properly. The track is made up of texture-mapped triangle strips with vertex colors. The vertex colors are what they should be. What's going wrong?

I'll cut to the chase. There's a specific command, IDirect3DDevice9::SetTextureStageState, that needed to be called to set the color blending mode. However, it was only ever called to set the alpha blending operation. We would need to patch in a call to this that sets the blending mode to something like the Saturn. Looking at the list, D3DTOP_ADDSIGNED does exactly what we need to. By default it was D3DTOP_MULTIPLY, which gives the muddied out look here. I assume this was done to increase support with 3D accelerators from the late 90's, which we don't have any reason to target now. What does that look like when it's all patched in?

Woo! That looks great! It took a lot of work (patching that in was NOT easy, due to how many functions I had to patch), but the game finally looks as vivid as it's Saturn counterpart. The mod is up for download at GameBanana, with source hosted on GitHub. If you're interested in more Sonic R stuff, check out my Sonic R page!