Articles posted February 2006 Older >>

Yabba Dabba Doo

Been sitting on this one for quite a while. It's another redemption game that runs on the same hardware as Double Cheese and Lotto Fun 2. The main issue was that the sound and graphics ROMs were twice as big as the other games, so I had to figure out the banking. It took way longer than I think it should have, but sometimes these things are a little tricky.

Title ScreenLogoTicket valuesGameplay

Cleaning House

Every once in a while, you make a small change to the code, and then realize that you really should make a related change to keep things clean. Often, that second change cascades into a third, and a fourth, and then eventually you're making another big core change that everyone loves. All you can do at that point is keep going and hope you can put all the pieces back together again. Thus begins my story of Dr. Frankenstein's Latest Creation, or How I Ripped Apart the Core and Put it Back Together Again.

The change in question this week was making some sense of the ordering of initialization in mame.c. If you look at the code now (before any changes), it's pretty confusing. I even have a giant comment at the top of mame.c trying to explain all the steps and subsystems that are initialized and in what order. It's pretty nuts. And it doesn't have to be that complicated. So I flattened it all out. Now there is one init_machine() function which calls all the other init functions. Along the way, I documented why certain subsystems need to be initialized before other subsystems.

Then I looked at the exiting code with the thought of simplifying that as well.

Except that made even less sense. Only certain systems needed exit code, and the exit routines were not consistently called in the reverse order they were initialized. Aha, I can fix that, I thought. I added a new function add_exit_callback() which queues a function to be called at exit time. The idea is that each init routine can add its own exit callback if necessary. This nicely enforces the exit order to be the exact reverse of the init order (as it should be), and means that each init routine is free to decide whether or not it needs to be called at exit time.

I love consistency, so this appealed to me greatly. Along the way I also made sure that all the init routines were named similarly (subsystem_init()) and that they all returned error codes in a consistent fashion. I did this also the debugger system, and realized that with two debuggers in the source tree, there was no central place that defined the common interfaces. So I created debugger.h which defines the core debugger interfaces. It also added a new DEBUGGER_BREAK macro which you can insert into your code to break into the debugger (either old or new), and which nicely compiles to nothing on a non-debug build. Then I had the distinct pleasure of removing all direct accesses to debug_key_pressed which have accumulated over the years. And yes, I am serious that I found this immensely satisfying.

Along the way through this consistency kick, I noticed that the driver callbacks did not have any sort of consistency. We have DRIVER_INIT, which is called at init time. But was also have VIDEO_START, which is also called at init time. And then we have MACHINE_INIT, which is not called at init time. Furthermore, save state registrations should happen during init time, but more often than not, the most convenient place to put them is in the MACHINE_INIT callback, because that is shared among multiple game drivers.

To fix this required a bit of a re-think, and the addition of several more callbacks. I left DRIVER_INIT as-is. Then I added MACHINE_START, SOUND_START, and VIDEO_START callbacks, which are called right near the end of initialization (I'm still debating calling these MACHINE_INIT, SOUND_INIT and VIDEO_INIT instead, but they are not called at the same time as DRIVER_INIT, so this bothers me a bit). These three callbacks are specified in the MACHINE_DRIVER constructor, and are the ideal place to put save state registrations. The only one of the three that we currently have is VIDEO_START.

I also added three callbacks that are called at reset time. These are MACHINE_RESET, SOUND_RESET, and VIDEO_RESET. Note that MACHINE_RESET is the new name for what is currently MACHINE_INIT. A little search and replace magic made the change across existing files.

I originally thought about having MACHINE_STOP, SOUND_STOP, and VIDEO_STOP as well, but then realized that the corresponding _START routine can just call add_exit_callback to register a stop callback if it was necessary. Even better!

I could have just stopped there, but I noticed that a lot of the video initialization code in mame.c was pushed out to a separate function, and that in fact there was quite a lot of video code lurking in mame.c, as well as some additional bits in common.c. In fact, common.c is a giant pile of random crap that I've been dying to clean out (I already split out the ROM loading code previously into a new file romload.c). So I made a new file video.c and pushed all the video logic into there from mame.c and common.c. The resource management code in common.c really seemed fundamental to the core, so it got moved into mame.c. Most of the remaining common.c functionality is really generic stuff like coin counters and NVRAM that seemed more appropriate as a machine-level file, so I created machine/generic.c and put the rest of it there.

At this point, I was pretty much ready to do anything, so I took another to-do list item from my internal list and renamed driver.c to mamedriv.c. See, driver.c was a bad choice for two reasons. The first is that there is a file driver.h that defines a number of functions, which should be defined in driver.c, except that driver.c was already taken so we put them in mame.c. Now I get to fix that little annoyance. The second reason this is better is that derivative builds (MESS in particular) have their own driver lists. It is more logical to name the driver list source file based on the target you are building. In this case, mame.mak references mamedriv.c.

Once that was sorted out, I took a closer look at driver.h and discovered that there was a giant list of header files included there, most of which made sense, some of which were pretty useless. Since most every file in MAME includes driver.h, it's important to keep this list of headers to a reasonable minimum. While there I finally added state.h as a "standard" include header, and removed the explicit #include "state.h" calls that were floating around in the various source files.

Whew! I'm sure there are a few things I missed describing. I get a bit frenzied when I really start ripping through changes like this. But I'm much happier with the organization of the core and the initialization sequence. And I think I might even know where to put the call to render_init() now, which is the real reason I got started on this kick in the first place....

Save State Fundamentals, part 4

Warning: this is a long one!

In this final article in the series on adding save state support to MAME drivers, I will walk through the process of adding support to a basic driver. Although there were a number of great suggestions for which driver to look at, all the proposed drivers had a large number of games and some hidden complexities that would hide the fundamentals I am trying to illustrate here. So for this example, we will add save state support to the Return of the Jedi driver. To follow along, you will need MAME 0.104u2.

The first step in adding support to a driver is to identify what files are relevant. For Return of the Jedi we have the obvious file drivers/jedi.c. There is also a matching jedi.c in the vidhrdw directory, so we will need to look at that as well. These are our starting points.

From there, we should determine what the dependencies are. If a driver depends on code elsewhere in the MAME system, we need to make sure that that code also has save state support. The easiest way to do this is to look at the #includes at the top of each file. In drivers/jedi.c, we see that it references the m6502 CPU core (cpu/m6502/m6502.h), the TMS5220 sound core (sound/5220intf.h), the POKEY sound core (sound/pokey.h), and some generic video hardware functions (vidhrdw/generic.h).

Doing my homework, I searched the CPU and sound cores to see if there were any state_save functions being called. The 6502 and TMS5220 already had support built-in. I did this before 0.104u2 was released and found that the POKEY emulator was lacking save state support, so I added it quickly to make sure we would be able to make this work. We'll have to wait until we look more closely at the video system before we can determine what functions in vidhrdw/generic.c were used.

The next step is to give the game a try and see if there are any MAME-detected issues. We run "mame jedi -log" to start the game, and then hit Shift+F7 to do a save. After a second, you will discover that it fails due to "pending anonymous timers". It also says check out error.log for details. If you open up error.log, you will see a whole bunch of logging messages listing anonymous timers. The ones in cpunum_empty_event_queue are expected and don't happen in every case. The one that is causing us issues is that one at drivers/jedi.c line 150 (generate_interrupt). Let's look at the code:

static void generate_interrupt(int scanline)
    timer_set(cpu_getscanlinetime(scanline), scanline, generate_interrupt);

So, this is the real problem. Every time the generate_interrupt call is made, it requests another anonymous timer (via timer_set), meaning that there is always at least one anonymous timer running in the system. How does the first call to generate_interrupt happen? A little farther down the code we see:

static MACHINE_INIT( jedi )
    /* set a timer to run the interrupts */
    timer_set(cpu_getscanlinetime(32), 32, generate_interrupt);

So at MACHINE_INIT time, we set up our first timer to go off, and then a bit later, when the timer fires, its callback sets another timer, etc. This pattern isn't fundamentally flawed, but we need to avoid the use of anonymous timers to make it work. To solve this, we add the following declaration to the local variables at the top of driver/jedi.c:

static mame_timer *interrupt_timer;

Then we modify the code in MACHINE_INIT to allocate a timer explicitly, and then prime the first callback in a separate step:

static MACHINE_INIT( jedi )
    /* set a timer to run the interrupts */
    interrupt_timer = timer_alloc(generate_interrupt);
    timer_adjust(interrupt_timer, cpu_getscanlinetime(32), 32, 0);

Finally, we modify the code in the callback to also use timer_adjust to change the firing time of the timer:

static void generate_interrupt(int scanline)
    timer_adjust(interrupt_timer, cpu_getscanlinetime(scanline), scanline, 0);

At this point, we recompile, try saving again, and find that we get a different behavior. It now says it was able to successfully save the game, but warns that it might not work because save states are not officially supported. Time to fix that!

Once you can actually try saving the game, the next thing to look for is memory banking. Scan the code for calls to memory_set_bankptr. If that functon is called only at init time, you probably don't need to worry about it. But if it is called in any kind of write handler or other callback, you probably need to convert the memory banking system over to the new system I described in the previous part of this series.

In this case, driver/jedi.c does indeed make some calls to memory_set_bankptr in its rom_banksel_w callback:

static WRITE8_HANDLER( rom_banksel_w )
    UINT8 *RAM = memory_region(REGION_CPU1);

    if (data & 0x01) memory_set_bankptr(1, &RAM[0x10000]);
    if (data & 0x02) memory_set_bankptr(1, &RAM[0x14000]);
    if (data & 0x04) memory_set_bankptr(1, &RAM[0x18000]);

To fix this, we need to pre-configure all the banks with the memory system at init time. Since this driver doesn't have a DRIVER_INIT call, we'll add it to the MACHINE_INIT function:

static MACHINE_INIT( jedi )
    /* configure the banks */
    memory_configure_bank(1, 0, 3, memory_region(REGION_CPU1) + 0x10000, 0x4000);

This configures 3 separate banks starting at offset $10000 of the CPU1 memory region, each bank being $4000 bytes away from the previous one. Then we re-write rom_banksel_w to look like this:

static WRITE8_HANDLER( rom_banksel_w )
    if (data & 0x01) memory_set_bank(1, 0);
    if (data & 0x02) memory_set_bank(1, 1);
    if (data & 0x04) memory_set_bank(1, 2);

There -- memory banking issues solved!

Now on to global variables. The best place to look for global variables is at the top of each module. However, you need to be careful. Often people have added global variables down below in between functions, or even disguised as static variables declared within a function. In this case, the driver source is pretty clean, so all the variables we care about in drivers/jedi.c are at the top, neatly organized and identified:

/* local variables */
static UINT8 control_num;
static UINT8 sound_latch;
static UINT8 sound_ack_latch;
static UINT8 sound_comm_stat;
static UINT8 speech_write_buffer;
static UINT8 speech_strobe_state;
static UINT8 nvram_enabled;

The first thing we need to do is make sure that the save state functions are included and available to us, so at the top of drivers/jedi.c we add:

#include "state.h"

And then at the end of our MACHINE_INIT function, we register our global variables:

static MACHINE_INIT( jedi )
    /* set up save state */

Easy enough. And that pretty much covers it for drivers/jedi.c. Now onto the video side of things....

At the top of vidhrdw/jedi.c, we see all the global variable declarations:

/* globals */
UINT8 *jedi_backgroundram;
size_t jedi_backgroundram_size;

/* local variables */
static UINT32 jedi_vscroll;
static UINT32 jedi_hscroll;
static UINT32 jedi_alpha_bank;
static int video_off, smooth_table;
static UINT8 *fgdirty, *bgdirty;
static mame_bitmap *fgbitmap, *mobitmap, *bgbitmap, *bgexbitmap;

We'll have to examine each of these and understand whether or not they need to be saved. I glossed over this with the drivers/jedi.c file because all the globals happened to be relevant, but for the most part you should look through the code and understand whether or not the data needs to be saved.

Looking at the first items here, we see that jedi_backgroundram, jedi_backgroundram_size, and jedi_PIXIRAM are all referenced in the address map for the CPUs in drivers/jedi.c. When a pointer is filled in by the memory system (i.e., its address is passed into the AM_BASE or AM_SIZE macro), then generally this means that the memory system allocated the memory for it and told us where that memory lives. Since the memory system is responsible for saving all of its memory regions, it means that we don't need to worry about the memory pointed to by these variables -- it's all taken care of for us automatically.

The next three variables -- jedi_vscroll, jedi_hscroll, and jedi_alpha_bank -- are all written to directly by the memory handler code, and later used in the VIDEO_UPDATE routine, so they definitely need to be saved.

The next two variables (video_off and smooth_table) follow the exact same pattern: they are written to directly by the memory handler code and are later used in the VIDEO_UPDATE call. The difference here is that both of these variables are ints. Ints are a problem in save states because the size of an int variable is not guaranteed to be anything consistent across all platforms (this is the reason we have explicitly sized types in MAME like UINT8 or INT32.) Although you can save these variables as-is, doing so makes the save states less portable. So before we can save them, we should really take the time to convert them to something explicitly-sized.

Scanning for where these variables are used reveals that neither of them are used to store anything larger than 8 bits, so they can both be converted to UINT8s. If you're ever in doubt, however, changing an int to an INT32 for the purposes of adding save state support is the safe thing to do.

Saving these first five variables is easy. Just add the following code to VIDEO_START:

    /* register for saving */

All the remaining variables in vidhrdw/jedi.c are allocated in the VIDEO_START routine and represent the bitmap data of the various layers. This is an example of where you need to take the time to really understand what's going on in order to properly perform a save state. Here you have two options. One is just to blindly save everything. That is, you could save the full fgdirty and bgdirty arrays as well as the contents of all four bitmaps. And that would work just fine if you added code like this:

    state_save_register_global_pointer(fgdirty, videoram_size);
    state_save_register_global_pointer(bgdirty, jedi_backgroundram_size);

(Note the use of the new macro state_save_register_global_bitmap which was added in 0.104u2.) So this approach is the "heavyweight" but dumb approach. It is generally frowned upon because it is wasteful to store all that data when with a little bit of code examination, we can avoid saving any of that stuff at all! How, you ask?

Well, first let's look at fgbitmap. If you look in the VIDEO_UPDATE callback farther down in the file, you will see that fgbitmap is redrawn section by section depending on whether or not a corresponding entry in fgdirty is set. Which means that if all the entries in fgdirty were set after a restore, the entirety of the fgbitmap would be re-generated on the first VIDEO_UPDATE call. You can see a very similar pattern for bgbitmap and bgdirty. So, rather than saving any of these variables, we could register ourselves a postload callback:

static void jedi_postload(void)
    memset(fgdirty, 1, videoram_size);
    memset(bgdirty, 1, jedi_backgroundram_size);


Which means that on a restore, we can mark all of the foreground and background bitmaps "dirty" so that they get completely regenerated on the next VIDEO_UPDATE. Nifty.

This leaves us with bgexbitmap and mobitmap to consider. If you examine the code, you will see that as the bgbitmap gets updated, parts of the bgexbitmap are tagged dirty. Logically, if all of bgbitmap gets updated, all of bgexbitmap will be tagged dirty. Since we are already forcing bgbitmap to be fully updated, we don't need to worry about bgexbitmap at all -- it will just work.

Similarly, if you look at the logic for handling mobitmap, you will see that everything that is drawn into it during VIDEO_UPDATE is erased at the end, meaning that mobitmap never contains any useful data across refreshes; hence it does not need to be saved either.

So, you now have two approaches to saving the video state. I recommend giving them both a try, checking out the difference in save state file sizes, seeing what happens when you try to load a state saved one way after changing the code to work another way. Once you've made the changes, I also recommend playing the game to various points, saving, restoring back to different points, jumping around, and trying to break it. Hopefully everything will work solidly! Once you're sure things are looking good, then and only then can we add the flag to the GAME entry in drivers/jedi.c:

GAME( 1984, jedi, 0, jedi, jedi, 0, ROT0, "Atari", "Return of the Jedi", GAME_SUPPORTS_SAVE )


Save State Fundamentals, part 3

Consider this a "sidebar" to the save state fundamentals articles. Part 4 will work through the driver example. This brief article is meant to explain the new and improved memory banking changes that went into MAME a little while back.

In the "old days", let's say you had a banked ROM which you loaded like this:

ROM_LOAD    ( 0x0c000, 0x1000, CRC(...) SHA1(...) )
ROM_CONTINUE( 0x10000, 0x7000 )

This is a common way of loading a banked ROM. Bank 0, which lives in the first 4k, is loaded at the address where the CPU actually reads data from. The remaining banks are loaded outside of the CPU's address space (in this case, the CPU is an 8-bit CPU with a 16-bit address space, so it can't access memory beyond 0xFFFF).

Then you would need to map the memory where the ROM appears in the address space, as well as a handler for actually changing the banks. Let's say it looked something like this:

AM_RANGE( 0xc000, 0xcfff ) AM_ROMBANK(1)
AM_RANGE( 0xffff, 0xffff ) AM_WRITE( bank_select_w )

This configures bank #1 in the memory system to map to the 4k region 0xC000-0xCFFF, and configures a single byte address at the very end of the address space that programs write to in order to select which of the 8 banks in the ROM appears in the 0xC000-0xCFFF range.

Finally, the code for the bank_select_w function would look something like this:

WRITE8_HANDLER( bank_select_w )
    UINT8 *rombase = memory_region(REGION_CPU1);

    data &= 7;
    if (bank == 0)
        memory_set_bankptr(1, &rombase[0xc000]);
        memory_set_bankptr(1, &rombase[0x10000 + (data-1) * 0x1000]);

In this code, we tell the memory manager where the base of bank #1 starts. If it's bank 0, we point it back to the first bank, which was loaded at offset 0xC000 in the memory region. For banks 1-7, which were loaded at offset 0x10000 in the memory region, we perform a small calculation to determine the base of the bank.

This system has worked great for years, but is unfortunately not compatible with saving the state of such a driver. Well, it can be done, but it means that every driver which has banks like this would need to store the current bank number in a global variable, and after restore, it would need to have a postload function that called memory_set_bankptr to point the memory system to the correct bank. This is a pain.

In order to make memory bank saving automatic, the memory system was augmented with a slightly different way of setting the bank base. Rather than setting the base of the bank directly in bank_select_w, you instead tell the memory system up front where the base of all the banks are. Then in your bank select handler, you tell the memory system which bank number to switch to. The memory system can then manage the saving and restoring of the banks for you.

So how does this affect the code above? Well, your ROM loading and memory maps are identical. The first thing you need to do is to add, somewhere in your DRIVER_INIT, MACHINE_INIT, or VIDEO_START callbacks, code to register all the banks. Let's put it in DRIVER_INIT:

DRIVER_INIT( example )
    UINT8 *rombase = memory_region(REGION_CPU1);
    memory_configure_bank(1, 0, 1, &rombase[0xc000], 0);
    memory_configure_bank(1, 1, 7, &rombase[0x10000], 0x1000);

So what are all those parameters to memory_configure_bank? Well, the first parameter is which memory system bank you are configuring, in this case 1 because we are configuring memory system bank #1. The second and third parameters specify the range of banks you are configuring. In the first call, we are configuring 1 bank starting at bank #0, and in the second call we are configuring 7 banks starting at bank #1. The fourth parameter specifies the starting address of the first bank that is being configured. And the fifth parameter specifies the number of bytes between successive banks.

Breaking it down further, in the example above, the first call is configuring bank #0 to start at &rombase[0xc000]. And the second call is configuring banks #1-7 to start at &rombase[0x10000], &rombase[0x11000], &rombase[0x12000], etc.

Once you've done that, your bank selection code becomes simply:

WRITE8_HANDLER( bank_select_w )
    memory_set_bank(1, data & 7);

Understanding how this mechanism works is important if you want to cleanly add save state support to drivers with banked memory.

Save State Fundamentals, part 2

So, after reading part 1, you now know that in order to save the state of a driver, you need to register with the save state system all the bits of memory that need to be preserved. The next obvious questions is, how does it actually work?

As I mentioned before, all registration must occur in the window between early initialization and just after the driver’s MACHINE_INIT callback is called. The reason for this is simple: after that point, the game is actually up and running, and the user can request a save at any point thereafter.

Before a request to save the state can actually transpire, the state of the system must meet a few basic requirements. Usually these are met within a few milliseconds of the save state request, so it is usually not noticeable what is happening under the covers. To begin with, a save will only occur in between timeslices of CPU execution. That is, the state will only ever be saved at the start of a round-robin timeslice of the CPUs. This reduces the chances that any CPU will be in an odd state.

The second major requirement is that there can be no "anonymous" timers active in the system. What is an anonymous timer? It is any timer that is set using the timer_set call. Timers created this way are one-shot disposable timers and are intended primarily for things like timer_set(TIME_NOW)-style synchronization callbacks. Some games (including many I've written unfortunately) will cascade these anonymous timers by calling timer_set in the callback from a previous timer_set. This means that at any given instant, there is always at least one anonymous timer pending, and you will get an error after 1 second if the system can't get to a state where there are no pending anonymous timers. (The solution to this problem is to create a timer at init time using timer_alloc and then use timer_adjust on that timer to get the same behavior.)

Once these requirements are met, the actual save can occur.

To perform a save, the save state system first goes through the roster of data that was registered and computes a signature. This signature is a convolution of key data from each registration: the module name, the instance number, the data name, and the total size in bytes. The convolving function is simply a CRC, so the resulting signature is a 4-byte CRC that (mostly) uniquely describes the data that will be saved.

Once the signature is computed, it is written to the save state file, along with a simple header. After that, the save state system simply goes through every bit of data that was registered, and writes it to the save state file. Easy enough.

To restore the state of the system, it is important to realize that we can only perform a restore after all the parts of the system have registered their memory for saving. This means that we can only restore to a system that is fully up and running. If you initiate a restore from the command line, what actually happens is we set a flag to remember this request, and simply start running the game as usual. Before the first CPU timeslice, we check the flag, and if it is set, we begin the restore at that point.

To perform the actual restore, we first compute the signature of all the registered data, and compare it to the signature that was stored in the save state file. The two must match; if they don't, then something has changed in terms of what was registered between the time the game was originally saved and the current time. Usually this means that additional items were registered, or the names were changed, or something else is different. In this case, the game cannot be restored, and you will get an error popup in MAME.

If the signatures do match, then the save state system iterates through all of the registered data, finds that data in the save state file, and loads the memory. At that point, in theory, the game should just pick up where it left off.

Sadly, it's not always that easy.

Sometimes, the data as it is stored in memory is not formatted conveniently for saving as-is. In this case, what you need to do is convert the inconveniently-formatted data into something more readily saveable just before the save occurs. Fortunately, you can do this in MAME. This is what's known as a "pre-save" callback, and you can register them with the save state system at the same time you register all the data. There are three types of callbacks you can register: one that passes you back an int parameter (which you supply at registration time), one that passes you back a void * parameter, and one that passes you back nothing.

The way you would use it is like this. First, you would define up front the memory that will hold the saveable version of your data, and you would register that at init time with the save state system. At the same time, you would also register a pre-save callback. Then, when a save occurs, just before all the memory is read out, your callback is called. This gives you the opportunity to fill the previously-registered memory with converted data.

Given that, then, how do you un-do this conversion after a restore? Yep, you guessed it, there is also a "post-load" callback you can register. The same three types of callbacks are supported. These callbacks are called just after all the memory is read back from the save state file, and allow you to reconstruct your inconveniently-formatted data from what was saved.

One of the most common uses for a post-load callback is to update memory banks. At init time, you would register the global variable that specifies the current bank number, and you would also register a post-load callback. When that callback is called, you know that your global variable contains the bank number, but the memory system needs to be informed so that it is pointing to the right memory. So in your post-load callback, you call memory_set_bankptr to update your memory bank.

Of course, this is actually a bad example, because the memory system has an alternate way of handling banking now that is compatible with the save state system. But it is illustrative of the reason why you might want to have a post-load callback.

In the third and final part of this series, I will go through the steps necessary to add save state support to a driver by walking through the process on an existing MAME driver that doesn't yet have support. Stay tuned!