Making a GB Game, Part 5: Adventures in Macro Land

Published 2018-12-04

So, remember that last part, when I did a whole fancy song and dance of register juggling? Yeah, I didn't want to do more of that. It's insanely hard to fix, add, or rearrange anything without the function breaking on me because I accidentally overwrote the wrong register. To hopefully resolve this, I wanted to implement some sort of system where I could reserve registers X and Y for some object, and be confident that I won't later touch anything involving that.

Additionally, I figured it would be really nice if I could seek back and forth within a struct without having to count bytes. I wanted a system where I could say that this register pair is currently at offset named X, and I needed to move it to offset named Y, and it would generate whatever code was needed to make that happen.

Brief warning that this entire post is nothing but code. No pretty pictures here, but I'll try to keep it somewhat readable.

Take 0: Psuedocode

First, let's define what the heck it is we're tying to do in a relatively normal language.

For the first task, the register registration system, I decided I wanted a variable for each register, and a variable for each registered object. There were two main tasks: registration and invalidation.

To register an object, I needed the object name and the registers it used. I would save the used registers as a variable under the object's name, and then mark each of those registers as used. To invalidate an object, I would look at the object, see what registers it uses, un-set them, and remove those registers from the object's name.

In a language like C, it would look something like this:

struct Object {
        Register * lo;
        Register * hi;
}
struct Register {
        bool set;
}

Object RegisterPair(Register * lo, Register * hi) {
        Object * result;
        result->lo = lo;
        lo->set = True;
        result->hi = hi;
        hi->set = True;
        return result;
}

void InvalidatePair(Object *obj){
        obj->lo->set = False;
        obj->hi->set = False;
        obj->lo = NULL;
        obj->hi = NULL;
}

Register a, b, c, d, e, h, l; // where set = false

which would allow the ASM to look something like this:

SomeFunction:
        Object *AnimPointer = RegisterPair(h, l);
        ld a, AnimPointer->lo
        ...
        InvalidePair(AnimPointer);
        ret

For the second task, the struct traversal system, is a bit easier. I need to create a function that takes in a register, a starting position variable and an ending position. It should increment or decrement the register and the starting position variable until the they are equal to the ending position. Again, in (hopefully mostly correct) C,

struct SomeStructDefn {
        short member1;
        short member2;
        char member3;
        short member4;
}

// Note: This is simple, but inefficient for large distances
void SeekStructAndSet(char * registerPair, void *start, void *end) {
        int offset = end - start;
        start = end;

        if(offset > 0) {
                for (int i = 0; i < offset; i++) {
                        'inc registerPair\`
                }
        } else if (offset < 0) {
                for (int i = offset; i > 0; i--) {
                        'dec registerPair\`
                }
        }
}

which allows for

SquareMember3:
        // hl points to the beginning of some struct
        // Note the somestruct variable doesn't contain anything; it's just for bookkeeping
        SomeStructDefn somestruct;
        SomeStructDefn * somestruct_pos = &somestruct;

        // we want to point at member 3
        SeekStructAndSet(hl, somestruct_pos, &(somestruct.member3))

        // And now we can load and store from member 3
        ld a, [hl]
        add a, a
        ld [hl], a

        // Unseek, and then return
        SeekStructAndSet(hl, somestruct_pos, &somestruct);
        ret

Notice that even if I added a member2.5, it would still seek to the right place. That would be fantastic to have as the game's object struct is still very much in flux, and I don't want to have to constantly rewrite code if it changes.

That all would be great, if C worked anything like that. But as it turns out, you can't (easily) use C like that. We need to come up with a solution to this problem.

Take 1: RGBDS Macros

RGBDS is, from what I understand, the standard GBZ80 assembler, with WLA-DX a close second. It's the one I've been using this entire time, thankfully. A neat thing about RGBDS is that it is what as known as a macro assembler.

For those out of the loop, a macro is a small command you define that expands into a series of commands. A well-used macro is NINTENDO_LOGO, defined in hardware.inc, a commonly used file that makes interfacing with the Game Boy hardware a bit nicer. The Nintendo Logo macro is called by just typing out it's name. It expands into the 128 byte long sequence that makes up the Nintendo logo the Game Boy needs to see for the game to boot.

That said, RGBDS macros are kind of dumb. They can take in arguments, and they place them in lines of code, but you can't do much more. Variables barely exist, loops aren't really a thing, and trying to do more complex things is difficult at best.

At the same time, they are the easiest tools to work with, being built into the language. So I tried anyways. I started with the second task, the struct manipulation macros. Here's what that looked like.

; \1 = Source address
; \2 = Dest address
; \3 = High register to increment
; \4 = Low register to increment
SeekStructAndSet: MACRO
offset = \2 - \1
\1 = \1 + offset

IF offset > 0
        .rept offset
                inc \3\4
        endr
ELIF offset < 0
        .rept offset
                dec \3\4
        endr
ENDC
ENDM

; Later
RSRESET
start   RB 0
member1 RB 1
member2 RB 1
member3 RB 2
member4 RB 1
sizeof  RB 0
next    RB 0

SquareMember3:
        SOMESTRUCT_POS = start;
        SeekStructAndSet h l SOMESTRUCT_POS member3
        ld a, [hl]
        add a, a
        ld [hl], a
        SeekStructAndSet h l SOMESTRUCT_POS start

As you can see, it's almost exactly what the C version was. However, there's some caveats with the macro's arguments. The registers are simply strings containing the name of the register, so that when `inc \3\4` is called, it concatenates \3 and \4 (say, h and l into hl) and emits an `inc` opcode. The addresses are based on a neat feature built into the assembler that it calls structs. Every time the RB command is used, the label to the left of it gets assigned the sum of every RB command before it (but after RSRESET).

To use it, we set SOMESTRUCT_POS to the start (wh ich also points to member1, due to start being 0 bytes long). We then increment hl using the macro until SOMESTRUCT_POS = member3, which is set to 2. This accomplishes exactly the same thing and works just as well. If the struct were part of a array, we could have also incremented until the next struct by moving past all the variables to the very start of whatever is next.

Much harder, though, was the first task. I needed some sort of way to have a variable named [input].hi equal a, and [input].hi.set equal to the value of a.set, which would be 0 or 1. RGBDS's macros don't have a concept of references or objects, just global variables.

I thought I could fake the objects by breaking up the macro expansion into steps. For example, ([input].hi).set -> a.set -> 0/1. This would have allowed me to "follow" a "reference" which is what I needed for this to work. This, annoyingly, turned out to be impossible. Because of this, RGBDS macros were unfortunately out of the question.

Take 2: Modding the Assembler

So, I'll admit up front that this was a dumb idea in retrospect. But at the time, I thought adding a couple of functions to the macro language was going to be relatively easy.

However, I didn't feel comfortable enough with the source code of RGBDS (which is written entirely in C and still undergoing very active development) to hack more macro features into that. Instead, I searched GitHub for "gameboy assembler" and came across the aptly named gbasm, a mostly RGBDS-compatible assembler written in JavaScript via Node.js, a much more flexible language. This was written for the development of BonsaiDen's Tuff, a very well put together game.

As it turned out, as well put together as the assembler was, there were a few features missing. A major one was the RB command for making the structs, as well as the ld a, [hl+] opcode needing to be ld a, [hli]. I fixed these issues in my own personal fork, I think. A much more critical issue, though, was that gbasm's macro system was completely incompatible with and had a lot less features than RGBDS.

I wasn't too deterred, though. I had a plan. Because JavaScript isn't a compiled language, it has an eval() function. If I could embed JavaScript code within my assembly source that returned a string containing the instructions I wanted to compile, I'd be able to basically make JavaScript a macro language. For instance, suppose I had a macro function

JMACRO SeekSructAndSet(src, dst, hi, lo)
        let offset = dst - src;
        src += offset;
        let retstr = "";
        let op = (offset > 0) ? "inc " : "dec ";
        while(math.abs(offset) != 0){
                retstr += op + hi + lo + "\n";
                if(offset > 0){offset -= 1;}
                else {offset += 1;}
        }

        return retstr;
ENDJMACRO

in place of the earlier SeekStructAndSet. Somewhere in the assembly code I could call that like I did before, which would then call this JavaScript function and replace that line with the return value of the function. Having the ability to use JavaScript would have been very nice, as I could use all of it's fancy features in my macros.

However, two things came up. First up technical issues, trying to insert a brand new macro system into an existing and quite solidified codebase was not easy. The way gbasm was set up meant that it was a bit hard to do an operation like "replace a line", or "read this chunk of text as-is and execute it as JavaScript", as it ran all the source code word-by-word. Additionally, look at the line src += offset. Where was src defined? I would have had to make another macro just to declare that variable, and that was possible but messy.

Secondly, while I was starting to work on this, college started back up and consumed about all of my free time. Overhauling this assembler was a massive task that I didn't have the time to take on. I let it sit like that until about November, when I had an even better idea.

Take 3: Use UNIX

I came to the realization that bolting a macro language onto an existing assembler was not a new problem. What I needed was some sort of generic macro preprocessor.

As it turns out every Linux system has a rather decent one built in: m4. It's an old beast, originally written in 1977 and mostly used these day for sendmail and those GNU autoconfig scripts that seem to come with every C project. But, reading over the manual, it seemed to be exactly what I needed.

m4 acts a lot like the C preprocessor. You insert m4 commands into your assembly source. When you run the source through m4, all the m4 commands are run and it returns a file that contains only assembly source. You take the new assembly source file and run it through the assembler as usual, and everything works.

Here's how I implemented the RegisterPair and InvalidatePair macros:

; Defines names for a pair of registers
; $1 = Prefix name, all caps
; $2 = first register in pair
; $3 = second register in pair
; Defines names $1_hi, $1_lo, $1_reg, $1_ref
; Checks against variables $2_inuse, $3_inuse

define(`RegisterPair', `
    ifdef(`$2_inuse', FAIL "$2 is already set!")
    ifdef(`$3_inuse', FAIL "$3 is already set!")

    define(`$2_inuse', 1)
    define(`$3_inuse', 1)

    define(`$1_inuse', 1)
    define(`$1_hi', $2)
    define(`$1_lo', $3)
    define(`$1_reg', $2`'$3)
    define(`$1_ref', [$2`'$3])
')

; Marks a macro pair as invalid
; $1 = Prefix name, all caps
define(`InvalidatePair', `
    undefine(`$1_inuse')
    undefine($1_hi`'_inuse)
    undefine(`$1_hi')
    undefine($1_lo`'_inuse)
    undefine(`$1_lo')
    undefine(`$1_reg')
    undefine(`$1_ref')
')

That is a LOT to deal with. Let's step through that. In m4, every macro is defined with define(name, definition). The use of quotes is a bit funky, but they basically mean "postpone evaluation". You need to put the macro name in quotes to avoid expanding it before passing it to the define function.

In RegisterPair, we first check in 2 and 3 are in use. If they're not, we emit an RGBDS macro command that tells the compilation to fail. We then define all of the variables we need to keep track of everything. To concatenate macros, we can use `', which runs the first half and then the second half. Here we use it to concatenate, for example, h and l into hl. For InvalidatePair, we simply use `undefine(macro)` to remove the definitions. Here we use `' to evaluate the first half, $1_hi, which will be the name of some register, and then add the second half.

This works incredibly well. With one caveat. m4 definitions and RGBDS definitions aren't compatible. If I wanted to use a registered pair name to seek through a struct, that would not be possible. So I rewrote the struct seek macro in m4 as well.

; Macro for moving to a specific element
; in a structure starting from a previous one
; $1 = Source address
; $2 = Dest address
; $3 = High register to increment
; $4 = Low register to increment
define(`SeekStruct', `
    define(`offset', eval(`$2 - $1'))
        define(`$1', $2)

    ifelse(eval(offset > 0), 1, `
        rept offset
            inc $3`'$4
        endr
    ')
    ifelse(eval(offset < 0), 1, `
        rept eval(0 - offset)
            dec $3`'$4
        endr
    ')

    undefine(`offset')
')

Again, this works, but I can't use the RB command. I just replaced those with define commands, and finally everything worked exactly as expected. The sprites are rendered as if nothing had changed. But now maintaining and adding features is a whole heck of a lot easier than it was before.

We are now in the present day. I thought I would have had more work on this done by now, but I don't. I think next up is going to be movement, as I've started working on that. I want to have something somewhat playable by the end of the year, but I don't realistically see that happening. (It didn't.)