Thanks for the help, people, I got it working, woohoo!
There definitely are issues with the GUI debugger, in that it doesn't want to even start while my demo is running. It does start when you're in workbench, so I switched it to command line mode there. Fortunately, it remembers this mode, and after a restart I was able to get to the debugger while my demo was running. I'm using winUAE 4.0.1 (2018.07.16) on windows 10 pro.
Btw, are there any decent tutorials out there on this? I've read the help in the debugger itself, but most of that is rather cryptic to me, never having used an Amiga debugger before. If anybody knows of any stuff I could check to bring me up to speed more easily, I'll be glad to hear about it.
Edit: Oh and btw, it says "couldn't open amiga.lib". Not sure if that's something I should be worried about..