UAE Scripting Layer

[FrodeSolheim]

I've created this thread for discussion about the possibility of allowing UAE to be scripted with Lua. Exactly what can be scripted is yet to be determined, but Toni Wilen has considered adding Lua scripting to WinUAE, and I'm experimenting with Lua scripting in FS-UAE.

While some parts will probably be FS-UAE/WinUAE-specific, such as the ability to affect the FS-UAE user interface, other parts of the scripting layer could be kept general and shared between FS-UAE, WinUAE and other UAE implementations so people can create "cross-UAE-plugins" for some purposes.

Examples where script functions could be invoked:

- at each Amiga vsync
- when UAE config variables are changed
- triggered by input events
- triggered by Amiga display mode changes
- when UAE is contacted over a named pipe / unix socket (for IPC).
- when an Amiga frame is done and ready to be uploaded to the GPU (FS-UAE specific), the script can affect the cropping and processing of the video frame.
- when a frame is rendered (FS-UAE specific), the script can decide how/where to draw the frame, and draw "theme elements" (overlays, LEDs, text) and other user interface elements.

Some example functions scripts could use (the Amiga/UAE API):

- uae_read*(address) - read memory at a specific Amiga address like an Amiga program would do.
- uae_write*(address, value) - write to memory at a specific Amiga address like an Amiga program would do
- uae_peek*(address) - directly read memory at a specific Amiga address, but can read write-only registers and the read won't have side effects
- uae_poke*(address, value) - directly write memory at a specific Amiga address (?).
- uae_read_string(address, length) - read length bytes from memory into a Lua string (perhaps: length -1 = read until null-terminating byte).
- uae_get_config(option)
- uae_set_config(option, value)

(*) multiple variants for 8/16/32-bit and signed/unsigned read and writes.

Other UAE functions: send input events to the Amiga, read LED states, read/write data from/to files (through UAE, with security considerations), read/write data from/to IPC pipes.

Other FS-UAE functions: load font, render text, load bitmaps, render bitmaps, load/switch shaders, manipulate GUI menus.

Todo:

- Describe some usage scenarios
- Describe the API and "script hooks" (when are Lua script functions invoked).

(This continues a discussion started here: http://eab.abime.net/showthread.php?p=867380).

[Toni Wilen]

I think we need two kinds of read/write routines (not including byte/long/array? sized versions). "Normal" routines that do exact same as normal Amiga-side access would have done (including possible side-effects) and "special" that don't have side-effects and also allow reading of write-only registers.

Currently statefile code is used to "read" all custom registers, also used by debugger E-command. It should be good enough for scripting purposes too.

[FrodeSolheim]

Makes sense, perhaps a set of read*/write* functions for "normal access", and peek*/poke* functions for direct/non-side-effect memory access to clearly distinguish them (edit: updated the initial post).

Function set would at least include 8/16/32-bit unsigned and signed variants. Haven't looked at the possibility for something like byte array access. At least the ability to read a memory range into an immutable Lua string will be straightforward and useful. To manipulate memory, the best way is probably to use the write/poke functions.

[FrodeSolheim]

The scripting layer now has sufficient features to force unlimited lives for the floppy version of Rick Dangerous II (for a lack of a better example ):

function on_uae_vsync()
    uae_write_u8(96431, 6)
end

Another (silly) example: theme code can bounce a smaller live Amiga screen around on the host screen (like a cheap screensaver) :

x, y = 0, 0
dx, dy = 4, 4

function on_fs_emu_render_frame()
    fs_emu_set_scale(1920, 1080)
    w, h = 960, 540
    fs_emu_render_frame(x, y, w, h)
    x = x + dx
    if x + w > 1920 or x < 0 then
        dx = 0 - dx
    end
    y = y + dy
    if y + h > 1080 or y < 0 then
        dy = 0 - dy
    end
end

[Toni Wilen]

I have been thinking about this...

Custom registers/IO: uae_peek_xxxx(registername)

Name of register is needed (names are already in array used by debugger), not memory address (uae_write_x and uae_read_x use addresses only) because especially CIA chips have many hidden registers. Custom chipset also have some, for example was DIWHIGH written.

Custom chips example: uae_peek_custom(regname). Size of access is always either 16-bit or 32-bit (depending on register name), I don't see any need for size modifiers.

CIA: uae_peek_cia(regname). Access is always byte-wide.

We could even read/write CPU registers using exact same method. Probably quite pointless but may be useful in some weird future situations..

I always design for maximum flexibility

[xArtx]

Originally Posted by FrodeSolheim

The scripting layer now has sufficient features to force unlimited lives for the floppy version of Rick Dangerous II (for a lack of a better example ):
[CODE]
function on_uae_vsync()
uae_write_u8(96431, 6)
end

Hi
I've looked at your .cpp file, I just Googled this:"uae_write_u8",
and forgive me with LUA, but what part of it is actually looking at the Amiga's memory?
I'm trying to dump the entire chip memory, but not having any luck addressing it.
It seems to break down to this:

 byteput(addr, value);

so why is this commented out?

#define byteput(addr,b) (call_mem_put_func(get_mem_bank(addr).bput, addr, b))

Even then, it doesn't look like the best solution for reading a 1Mb chunk which is what I'd like to do.
Do you know how to get the start address in chip memory, and copy it to your own C array?

Cheers, Art.

[TCD]

It's a preprocessor directive and not commented out...

[Toni Wilen]

Originally Posted by xArtx

Do you know how to get the start address in chip memory, and copy it to your own C array?

Inside UAE code? Just use "chipmemory" pointer

[xArtx]

Originally Posted by Toni Wilen

Inside UAE code? Just use "chipmemory" pointer

I did that, and am seeing something.. it just looks like 'patterned' garbage, that at best, might be sound samples.
I expect to see some plain text strings in it if the running program contains them.
Maybe I need to disable expansion RAM, but I assumed chipmem would be populated first.

[xArtx]

Never mind, The trigger for snapshotting the memory was when the drive got to track 74
on the disk, which was when I thought the game had loaded, but the disk accesses that
track quickly beforehand, and I didn't notice it, so the snapshot was too early!



I've noticed I have to swap bytes, but it looks like all is well... It's the intro to MegaBall,
and I want to make a game trainer on the native hardware side, as you have discussed.

Looks like now a matter of taking snapshots between losing lives, and wasting a lot of
time that could be otherwise spent doing something productive

[Jerry]

It would be nice if you could set memory breakpoints ie. call a function everytime BLTSIZE is written to.

[Toni Wilen]

I finally bothered to include lua support in WinUAE and I'd like to add some more advanced features (like above debugger memwatch breakpoint -like operation)

Is it possible for lua script to register callback function, for example scripts says "call function x in this script when y happens" (y = read or write to some address, byte/word/long length etc, similar to debugger memwatch points).

When hit is detected, lua function x is called with parameters like address of hit, size and so on..

Possible? Any better or worse ideas?

[FrodeSolheim]

Such breakpoints would indeed be *very nice*... I did think a bit about this a while ago, but I can't see any portable way to do this other than to maintain a list of callbacks/breakpoints, and then manually call into lua when the breakpoint conditition is reached. Lua scripts would then register the callback functions on script initialization, This in itself should be straightforward to implement, but I assume there would be a quite significant performance hit, running these checks for every memory write...

One could of course optimize it, so it at least does not have any performance hit when the feature is not used (replacing the memory bank pointers comes to mind, having another set of functions to be used when lua callbacks are enabled and must be checked). And when lua callbacks are enabled, I suppose further optimizations could be done so only a minimal work is needed if the write is outside an address range which needs to be checked, but there'll at least be one conditional check in the best case...

On the other hand, I suppose stuff like this is already done by the debugger, so the size of the performance hit is already probably known... (?)

(And JIT / direct mode is another complication...)

[Toni Wilen]

Originally Posted by FrodeSolheim

On the other hand, I suppose stuff like this is already done by the debugger, so the size of the performance hit is already probably known... (?)

Yes, debugger already modifies memory banks when memwatch points are active. I don't see any reasons to duplicate debugger code, only need slightly modifications to debugger. (and possibly optimize it a bit and only modify "memory banks" that need watching)

My only problem is lua callback registeration/implementation, I didn't find any examples in fs-uae code yet

(And JIT / direct mode is another complication...)

There is no way around it. Also not my problem

[FrodeSolheim]

Originally Posted by Toni Wilen

My only problem is lua callback registeration/implementation, I didn't find any examples in fs-uae code yet

No worries, I can write an initial implementation with the Lua / C bridge

Originally Posted by Toni Wilen

There is no way around it. Also not my problem

Not mine either (FS-UAE does not currently work with direct JIT mode anyway).

[FrodeSolheim]

Hi, sorry - I forgot about the callback question... I have written a standalone sample program which allows lua code to register a callback, and with C code which subsequently calls it:

callback.lua:

function test_callback(arg)
    print("callback function called with arg", arg)
end

register_callback(test_callback, 42)

callback.c:

#include <stdio.h>
#include <lualib.h>
#include <lauxlib.h>

static int g_function_ref = 0;
static int g_function_arg = 0;

static int l_register_callback(lua_State *L)
{
    if (!lua_isfunction (L, 1)) {
        return luaL_error(L, "first parameter must be a function");
    }
    // store the function parameter in the registry and keep a handle to it
    lua_pushvalue(L, 1);
    g_function_ref = luaL_ref(L, LUA_REGISTRYINDEX);
    // also store the given integer parameter
    g_function_arg = luaL_checkint(L, 2);
    return 0;
}

int main(int argc, char *argv[]) {
    lua_State *L = luaL_newstate();
    luaL_openlibs(L);
    lua_register(L, "register_callback", l_register_callback);

    luaL_loadfile(L, "callback.lua");
    // execute the callback.lua script
    lua_pcall(L, 0, LUA_MULTRET, 0);
    // script execution done, script should have registered a callback now

    if (g_function_ref) {
        // push the callback function on the stack
        lua_rawgeti(L, LUA_REGISTRYINDEX, g_function_ref);
        // push the parameter
        lua_pushnumber(L, g_function_arg);
        // function call with one parameter and no return value
        lua_pcall(L, 1, 0, 0);
    }
    else {
    	printf("no callback function registered\n");
    }
    return 0;
}

This should output:

callback function called with arg	42

How to best associate callbacks with "UAE events" I'm not sure about. I guess it makes sense to try to integrate it with existing debugger triggers? All that is really needed is to associate the lua registry table ref (an integer - g_function_ref in the example) - and preferrably also the lua state (L) to allow for multiple scripts/states.