Scripting behaviour in a game engine - A small example

Hello there, today I’ll write a bit about scripting in game engine. Usually, game engines are written in low-level languages to run as fast as possible. It is a bit inconvenient for fast prototyping so on top of that, scripts are use to defined the gameplay. Think c# in Unity for example.

I’ve always been interested by this topic. In theory, you can separate the real performance-critical part of your engine (physics engine for example) from the gameplay parts. Different people can work independently. An engineer will optimize the renderer while another will implement a quest system for the game.

More often than not, people will answer negatively when you ask how to implement scripting in a game engine:

• Should you implement your own game engine? Of course not, use Unity
• Should you create your own GUI library? Why are you wasting you time? Use unity. But what the heck, we are here to learn and have fun so let’s get started!

The scripting language

Lua is used a lot for scripting language. It is easy to integrate with C and has relatively good performance compared to other dynamically typed languages. I am coding in Rust, and fortunately, Kyren from ChuckleFish has done most of the hard work to provide Lua integration in Rust. The module is rlua.

A simple example

Just playing with rlua examples, you can expose some data defined in Rust, modify it with Lua, and get it back to Rust.

use rlua::{Function, Lua, MetaMethod, Result, UserData, UserDataMethods, Variadic};

const CODE: &'static str = r#"
    position = position*2
"#;

#[derive(Copy, Clone, Debug)]
struct Vec2(f32, f32);

impl UserData for Vec2 {

    fn add_methods<'lua, M: UserDataMethods<'lua, Self>>(methods: &mut M) {
        methods.add_method("magnitude", |_, vec, ()| {
            let mag_squared = vec.0 * vec.0 + vec.1 * vec.1;
            Ok(mag_squared.sqrt())
        });

        methods.add_meta_function(MetaMethod::Add, |_, (vec1, vec2): (Vec2, Vec2)| {
            Ok(Vec2(vec1.0 + vec2.0, vec1.1 + vec2.1))
        });

        methods.add_meta_function(MetaMethod::Mul, |_, (vec1, scalar): (Vec2, f32)| {
            Ok(Vec2(vec1.0 * scalar, vec1.1 * scalar))
        });
    }

}

fn main() -> Result<()> {

    let lua = Lua::new();
    let mut v = Vec2(1.0, 0.5);
    lua.context(|lua_ctx| {
        let globals = lua_ctx.globals();
        globals.set("position", v)?;
        dbg!(lua_ctx.load(CODE).eval::<()>()?);
        v = globals.get::<_, Vec2>("position")?;

        Ok(())
    })?;

    dbg!(v);
    Ok(())
}

The data is just a simple Rust struct. Methods for this structure are exposed to Lua via the UserData trait. Now, let’s improve this a bit:

• I want to be able to load from a script, not from some hard-coded string.
• I want to loop and call my script’s method at each iteration.
• I want to load code in the lua context only once.

This is the function to load the script. Nothing too surprising here, but notice the use of <P: AsRef<Path>> that allows to convert the function’s parameter to a Path.

fn load_script<P: AsRef<Path>>(path: P) -> Result<String, Box<Error>> {
    let mut f = File::open(path).unwrap();
    let mut content: String = String::new();
    f.read_to_string(&mut content)?;
    Ok(content)
}

Then, the main function will be changed to:

fn main() -> rlua::Result<()> {

    let code = load_script("script.lua".to_string()).unwrap();
    let lua = Lua::new();
    let mut v = Vec2(1.0, 0.5);
    
    // Set up context
    lua.context(|lua_ctx| {
        let globals = lua_ctx.globals();
        globals.set("position", v)?;
        let vec2_constructor = lua_ctx.create_function(|_, (x, y): (f32, f32)| Ok(Vec2(x, y)))?;
        globals.set("vec2", vec2_constructor)?;

        dbg!(lua_ctx.load(&code).eval::<()>()?);
        Ok(())
    })?;

    loop {
        lua.context(|lua_ctx| {
            let globals = lua_ctx.globals();
            v = globals.get::<_, Vec2>("position")?;
            lua_ctx.load("update()").eval::<()>()?;
            Ok(())
        })?;

        dbg!(v);
        std::thread::sleep(std::time::Duration::from_millis(2000));
    }
    Ok(())
}

The Lua context is persistent, so the data I load in a lua.context block will be available in another block. The other subtleties (not so subtle though) here is that a function named update will be called at every loop iteration. This function is actually defined in my script.lua file. I guess defining conventions for Lua scripts cannot hurt.

function update()
        position = position * 2
end

If you run this code, v will double at each iteration and will be printed to the console. You can also change the code and run the program again to alter the behaviour without having to recompile! Bye-bye long compilation time :D

Hot-reload anyone?

It looks good, but why not adding hot-reload? I should be able to modify the script and my engine should reload the code.

Monitor for changes

To begin with, I’ll do it the hacky way, on Linux. The command stat -c '%y' script.lua will tell me the last time the file was modified. If I can monitor this in a separate thread, I will be able to know when I should reload my code.

The code that does the magic is:

    let (sender, receiver) = channel();

    thread::spawn(move|| {
        let mut cmd =  Command::new("stat");
        cmd.args(&["-c", "'%y'", "script.lua"]);
        let d = cmd.output().unwrap();
        
        let mut last_stat = String::from_utf8(d.stdout).unwrap();

        loop {
            let new_stat = String::from_utf8(cmd.output().unwrap().stdout).unwrap();
            if new_stat != last_stat {
                last_stat = new_stat;
                sender.send(true).unwrap();
            }

            std::thread::sleep(std::time::Duration::from_millis(1000));
        }
    });

It will just check the output of stat in another thread and send an event if the output differs from last time. Then, in the main loop, we can just check the receiver.

if let Ok(_) = receiver.try_recv() {
    println!("Reloading");
    eval_script(&lua, "script.lua".to_owned());
}

eval_script is just some code I extracted from main to read a file and evaluate its code.

It works, but I don’t feel so good about this solution. For example, just saving without modifying on vim will reload the script. Fortunately, watching for file modification is a common problem so there is a cross-platform crate for that.

A lot of work

Writing this small prototype has been a lot of fun. In reality, I expect a lot of repetitive codes as you need to specify the API that should be available in Lua file. There will be probably an impact on performance that need to be measured.

Again, Kyren sums it up nicely:

In the meantime, think really really hard before you add a scripting layer to your game engine. The problem is, I LOVE scripting layers in game engines (for modability and many other reasons), so I do this anyway, but it is not a decision to be taken lightly and it can eat up a lot of time and effort.

Deserialization with Rust

For my NES emulator, I want to be able to save the current state of the NES so that I can reload it later. This process is called serialization, and if you want to do it in Rust, the cool kid in the block is serde.

If you just want to serialize simple types, Serde pretty much provides everything out of the box. Things begin to get tricky when more dynamic types enter the scene. This post will explore how I implement serialization for mappers in the NES emulator. Mappers are not known at compile time and are determined by the ROM file header. Typically, you’d need 2 or 3 mapper’s implementation to get a lot of games running.

First iteration: Model the mapper as Box<dyn Mapper>

Coming from OOP, I modeled the mapper as a trait. The objects that contains a mapper would hold a pointer to the trait and the pointer is created from the ROM file header.

pub type MapperPtr = Box<dyn Mapper>;

pub trait Mapper {

    // Read ROM from cartridge
    // writing is needed for some mappers that have registers.
    fn read_prg(&self, addr: usize) -> u8;
    fn write_prg(&mut self, addr: usize, value: u8);

    // Read/Write pattern tables. Sometimes, it is RAM instead of ROM
    fn read_chr(&self, addr: usize) -> u8;
    fn write_chr(&mut self, addr: usize, value: u8);
    fn get_chr(&self, idx: usize) -> &[u8];

    fn get_mirroring(&self) -> Mirroring;
}

pub fn create_mapper(rom: &rom::INesFile) -> Result<MapperPtr, String> {
    let mapper_id = rom.get_mapper_id();

    match mapper_id {
        0 => {
            let nrom = nrom::Nrom::from(&rom)?;
            Ok(Box::new(nrom))
        },
        1 => {
            let mmc1 = mmc1::Mmc1::from(&rom)?;
            Ok(Box::new(mmc1))
        },
        2 => {
            let uxrom = uxrom::Uxrom::from(&rom)?;
            Ok(Box::new(uxrom))
        },
        _ => Err(String::from("Not implemented yet"))
    }
}

Using this model, the emulator works great! Except when trying to serialize this Box<dyn Mapper>. Serde does not know how to serialize this smart pointer. There are some crates to help (https://github.com/dtolnay/erased-serde) but deserialization becomes also complicated. The type of the mapper needs to be included in the serialized data in order to deserialize it to the correct structure, but Box<dyn Mapper> erases the type.

Second iteration: Use enumeration instead

The mapper is determined at runtime, but it is not that dynamic. In reality, I implemented the code for N mappers so there are only N variant of mappers. Using Rust enumerations to hold these variant, I can use serde to serialize the mapper: https://serde.rs/enum-representations.html.

As a bonus, the type is not erased and the mapper struct can be retrieved using pattern matching. It is also not necessary to use a trait anymore. The implementation becomes:

#[derive(Serialize, Deserialize)]
pub enum MapperType {
    Nrom(nrom::Nrom),
    Uxrom(uxrom::Uxrom),
    Mmc1(mmc1::Mmc1),
}

impl MapperType {
    pub fn read_prg(&self, addr: usize) -> u8 {
        match *self {
            MapperType::Nrom(ref x) => x.read_prg(addr),
            MapperType::Uxrom(ref x) => x.read_prg(addr),
            MapperType::Mmc1(ref x) => x.read_prg(addr),
        }
    }

    pub fn write_prg(&mut self, addr: usize, value: u8) {
        match *self {
            MapperType::Nrom(ref mut x) => x.write_prg(addr, value),
            MapperType::Uxrom(ref mut x) => x.write_prg(addr, value),
            MapperType::Mmc1(ref mut x) => x.write_prg(addr, value),
        }

    }

    // Read/Write pattern tables. Sometimes, it is RAM instead of ROM
    pub fn read_chr(&self, addr: usize) -> u8 {
        match *self {
            MapperType::Nrom(ref x) => x.read_chr(addr),
            MapperType::Uxrom(ref x) => x.read_chr(addr),
            MapperType::Mmc1(ref x) => x.read_chr(addr),
        }
    }
    
    pub fn write_chr(&mut self, addr: usize, value: u8) {
        match *self {
            MapperType::Nrom(ref mut x) => x.write_chr(addr, value),
            MapperType::Uxrom(ref mut x) => x.write_chr(addr, value),
            MapperType::Mmc1(ref mut x) => x.write_chr(addr, value),
        }
    }

    pub fn get_chr(&self, idx: usize) -> &[u8] {
        match *self {
            MapperType::Nrom(ref x) => x.get_chr(idx),
            MapperType::Uxrom(ref x) => x.get_chr(idx),
            MapperType::Mmc1(ref x) => x.get_chr(idx),
        }
    }


    
    pub fn get_mirroring(&self) -> Mirroring {
        match *self {
            MapperType::Nrom(ref x) => x.get_mirroring(),
            MapperType::Uxrom(ref x) => x.get_mirroring(),
            MapperType::Mmc1(ref x) => x.get_mirroring(),
        }
    }
}

pub fn create_mapper(rom: &rom::INesFile) -> Result<MapperType, String> {

    let mapper_id = rom.get_mapper_id();

    println!("MAPPER ID: {}", mapper_id);
    match mapper_id {
        0 => {
            let nrom = nrom::Nrom::from(&rom)?;
            Ok(MapperType::Nrom(nrom))
        },
        1 => {
            let mmc1 = mmc1::Mmc1::from(&rom)?;
            Ok(MapperType::Mmc1(mmc1))
        },
        2 => {
            let uxrom = uxrom::Uxrom::from(&rom)?;
            Ok(MapperType::Uxrom(uxrom))
        },
        _ => Err(String::from("Not implemented yet"))
    }
}

Using this code, serialization and deserialization work! On the other hand, it created a bunch of extra-code. Whenever you want to add a new variant, you will have to add it to all the pattern matching code…

Iteration 3: Use macros to reduce boilerplate code

Macros are the perfect fit to fix this inconvenience. A macro will generate the code for you. All you need to do is to write it (easier said than done :D).

The code becomes:

macro_rules! mapper_types {
    ($($name:ident: ($id: expr, $mapper:ty)),+) => {
        #[derive(Serialize, Deserialize)]
        pub enum MapperType {
            $(
                $name($mapper)
            ),+
        }

        impl MapperType {

            pub fn read_prg(&self, addr: usize) -> u8 {
                match *self {
                    $(
                        MapperType::$name(ref x) => x.read_prg(addr),
                        )+
                }
            }

            pub fn write_prg(&mut self, addr: usize, value: u8) {
                match *self {
                    $(
                        MapperType::$name(ref mut x) => x.write_prg(addr, value),
                        )+
                }
            }

            // Read/Write pattern tables. Sometimes, it is RAM instead of ROM
            pub fn read_chr(&self, addr: usize) -> u8 {
                match *self {
                    $(
                        MapperType::$name(ref x) => x.read_chr(addr),
                        )+
                }
            }

            pub fn write_chr(&mut self, addr: usize, value: u8) {
                match *self {
                    $(
                        MapperType::$name(ref mut x) => x.write_chr(addr, value),
                        )+
                }
            }

            pub fn get_chr(&self, idx: usize) -> &[u8] {
                match *self {
                    $(
                        MapperType::$name(ref x) => x.get_chr(idx),
                        )+
                }
            }

            pub fn get_mirroring(&self) -> Mirroring {
                match *self {
                    $(
                        MapperType::$name(ref x) => x.get_mirroring(),
                        )+
                }
            }

        }


        pub fn create_mapper(rom: &rom::INesFile) -> Result<MapperType, String> {
            let mapper_id = rom.get_mapper_id();
            match mapper_id {
                $(
                    $id => {
                        let x = <$mapper>::from(&rom).unwrap();
                        Ok(MapperType::$name(x))
                    },
                    )+
                    _ => Err(String::from("Not implemented yet"))
            }
        }
    }
}

mapper_types!(
    Nrom: (0, nrom::Nrom),
    Mmc1: (1, mmc1::Mmc1),
    Uxrom: (2, uxrom::Uxrom)
);

I admit, it is still a lot to swallow. However, the only thing to do if you want to add a new variant is to add a line to mapper_types!. If you compare to the previous iteration, where at least 6 methods had to be changed manually, that is a huge improvement.

A final word

In my code, changing Box<Trait> for enum delegation solved the serialization issue in a satisfactory way. However, as everything in programming, this solution has its trade-offs. This thread on the rust forum is really informative:

• An enum is as big as its biggest variant
• Enums keep data on the stack
• “Enums represent a closed set of type, trait objects represent an open set.”

For my use case, using an enumeration instead of trait objects for polymorphism really solved various issues I had. However, enumerations are hard-coded, so if you are writing a library it makes sense to use traits for polymorphism instead.

NES emulation journal: Implementing mappers

The first NES games were pretty simple. Arkanoid, Donkey Kong and Super Mario Bros for example are all running using the first generation of cartridge, which includes 16kb or 32kb of ROM (instructions) and 8kb of sprite/tiles data. The size and scope of a game were effectively limited.

Game developers began using different kind of chips for their game to extend those original limitations. Instead of having just 32kb of PRG-ROM (program ROM) data, a game could have much more and the chip would select what instruction area is currently in use. In NES emulation, the term mapper is used to refer to the different type of chips.

Mapper 0: NROM

This is the mapper used with the first NES games. It has not bank-switching capabilities so the game using it are pretty simple. It can have either one or two PRG-ROM of 16kb, that are mapped at ranges 0x8000-0xBFFF and 0xC000-0xFFFF of the CPU memory. It also has CHR-ROM which contains the tile and sprite data. This CHR-ROM is mapped to the PPU memory at addresses 0x0-0x2000.

You can still have fun with games using this mapper:

• Donkey Kong
• Mario Bros
• Super Mario Bros
• Arkanoid

The CPU cannot write to the PRG-ROM for this mapper. We’ll see later that other cartridge have registers that can be written to using the PRG-ROM addresses.

Mapper 2: UxRom

The easiest mapper to implement after NROM is Uxrom. It provide bank-switching capabilities. It means that the program can change the data that is mapped to the CPU memory. In the Mapper 2’s case, the CPU memory from 0x8000 to 0xBFFF can be changed by writing to the mapper’s register.

The PPU memory is mapped to the cartridge RAM. There are not ROM for sprites but instead the program will write the tiles to memory before starting the game.

By implementing UxROM, you can unlock a lot of pretty cool games. Notably:

• Contra
• MegaMan
• Castlevania

Contra: A game using mapper 2

Mapper 1: MMC1

Once you understand UxROM, a more complicated mapper is MMC1. The principle is similar. CHR and PRG data can be switched to allow a game to have more data. The bank switching mechanism is more complex that UxROM in that it allows to switch banks from multiple memory locations. You’ll need to implement this mapper if you wish to play:

• MegaMan 2
• The Legend of Zelda
• Metroid

Megaman 2: Glorious death

I still have a few bugs to fix though :D The Buggy Legend Of Zelda

Next steps

MMC3 is also a must to implement as it is required to play Super Mario Bros 3. I still have a lot of graphic glitches in some games so I’ll start with that first…But hopefully I should be ready soon to put the code on a RaspberryPi!

Other recent improvements

• Support 2 players
• Performance improvement: Turned out that reading from input at each cycle slows down the emulator a lot…

References:

http://wiki.nesdev.com/w/index.php/UxROM https://wiki.nesdev.com/w/index.php/MMC1

NES emulation journal: Do not abuse substances

Happy new year every body! My NES emulator is taking shape! I am currently implementing side-scrolling and the main target is Super Mario Bros.

The fine-x for smooth scrolling is set with the PPU register 0x2005. Fine-x is a 3-bit value (so, between 0 and 7) and will decide what bit to select from the tile byte when rendering a background pixel.

My implementation is not complete yet (I correctly fetch the low plane bit but not the high plane bit), which gives me something totally funky. Mario on mushrooms!

Enjoy!

NES emulation journal: Feels like I forgot something

Whoops!

I didn’t implement reading from VRAM, so the CPU cannot handle collision between sprites and background.