Feature request [Heatmap]....

[Galahad/FLT]

Never seen this before, but i think a cool idea would be for emulators to have a function to list the biggest loads (i.e. what routines within a game are the most system intensive for emulation).

Obviously its not a must have feature, but doing these ST to Amiga conversions, something like that would be awesome.

I will get back to dreaming

[ReadOnlyCat]

Originally Posted by Galahad/FLT

Never seen this before, but i think a cool idea would be for emulators to have a function to list the biggest loads (i.e. what routines within a game are the most system intensive for emulation).

Obviously its not a must have feature, but doing these ST to Amiga conversions, something like that would be awesome.

I will get back to dreaming

Interesting that you mention this! I have been caressing the idea of having some profiling system in UAE. An emulator is indeed perfectly suited for that since it has access to all the needed data.

Even something as simple as keeping a list of all the "memory hot spots" would be super useful to determine which code gets used the most.

[Toni Wilen]

I actually have some plans for some kind of memory "heat map" with different colors for different DMA accesses and CPU accesses. 8 bytes = 1 pixel or something like that.

[Galahad/FLT]

Originally Posted by Toni Wilen

I actually have some plans for some kind of memory "heat map" with different colors for different DMA accesses and CPU accesses. 8 bytes = 1 pixel or something like that.

[ReadOnlyCat]

Originally Posted by Toni Wilen

I actually have some plans for some kind of memory "heat map" with different colors for different DMA accesses and CPU accesses. 8 bytes = 1 pixel or something like that.

Nice!

Good that you mention that because I was starting to consider peeking at the code lately to see if I could hack in a quick and dirty version of that.

I cannot reliably promise I can offer time for such a feature but do you have any time horizon for it? If I end up actually needing it before you get to work on it I could make an effort to make the code cleaner and submit it. No promises though, my own code is already quite tough to schedule.

[Amiga1992]

[offtopic removed]

Big ups on the "memory heat map", sounds really useful for coders!

[alexh]

Will the "heat-map" have an option for capture pre-cache? That way routines which don't leave the cache can still be profiled

[Toni Wilen]

http://www.winuae.net/files/b/winuae.7z

- Very basic, as usual.
- Enable: "vh" in debugger. (cycle-exact mode must be enabled)
- Automatically adds "null" memwatch break point from 0 to 512k, used to capture all memory accesses.
- First 512k of chip ram only.
- 256x256 pixel matrix, 1 pixel = 8 bytes. (top/left = addresses 0 - 7, top/right = addresses 255*8 - 255*8+7 and so on..)
- Colors are same as in DMA debugger except CPU color was changed to brown from dark grey.
- CPU = all CPU accesses, no separation between opcode and data accesses.
- Each pixel has 32 shades, pixel gets darker each frame if it has not been "accessed".

At least it creates interesting images!

Will the "heat-map" have an option for capture pre-cache? That way routines which don't leave the cache can still be profiled

Too early for that.

EDIT: Separate color for CPU instruction (pink) and data accesses (brown).

"Heatmap" shows audio sample playing nicely (red lines). Also game/demo double/triple buffer usage is easy to see (flashing blue = bitplane DMA regions)

Not much CPU code activity, I guess most common routines are short loops except if program has or generates big unrolled loops.

[ReadOnlyCat]

Huh, I guess that explains the lack of answer to my proposition to write code.

Thanks Toni!

Will give it a look soon if I can port it to FS-UAE (or if I get un-lazy enough to download WinUAE on my PC ).

A bit related: do you have a roadmap of future features to be released? As I mentioned above there are modifications to UAE that I plan to do (for my own needs) and I would hate to start working on something that you are already working on.

Knowing which features are planned and approximately when they will be worked on would help me not waste time on them.
And conversely it would save you time if you could reuse well written code instead of starting from scratch. As well as reducing the "bus factor" of UAE, which is always a good thing.

[ReadOnlyCat]

Originally Posted by ReadOnlyCat

A bit related: do you have a roadmap of future features to be released? As I mentioned above there are modifications to UAE that I plan to do (for my own needs) and I would hate to start working on something that you are already working on.

Knowing which features are planned and approximately when they will be worked on would help me not waste time on them.
And conversely it would save you time if you could reuse well written code instead of starting from scratch. As well as reducing the "bus factor" of UAE, which is always a good thing.

I hope this is not a taboo topic?

Seriously though, it would really help me to have even just a very approximative roadmap.

[Toni Wilen]

I forgot and I don't do roadmaps, every time I even try to guess some time frame, it goes totally wrong

Data that current heatmap system collects can be easily used to show "top 10" list of addresses used by CPU instruction fetches or something like that.

[ReadOnlyCat]

Originally Posted by Toni Wilen

I forgot and I don't do roadmaps, every time I even try to guess some time frame, it goes totally wrong

Oh, I understand that. I really meant something like a simple prioritized list of things you think/planyou will be working on next.

Just having that on the WinUAE site or here in a sticky thread would be really useful to avoid stepping on your toes.

[Toni Wilen]

Statistics update: "vh" command after enabling it now lists 40 first addresses with most CPU instruction word fetch activity. (Data is cleared after command)

It first finds highest CPU usage in stat data. Then finds all other nearby addresses with same or slightly less (5% less) CPU usage and outputs single line to console with start, end and percentage of total used. Then repeat same 40 lines.

EDIT: Note that list is not fully sorted, the longer the detected address space the worse the list is..

EDIT2: now also accepts two parameters, first parameter: percentage value of how big range is accepted (default 95 = 95% of highest accepted, set to zero to list all address), second parameter is number of lines to output.

I quickly checked some simple intros and demos and most CPU time was nearly always used "loop: btst #6,$bfe001; bne.s loop" main loop, everything else was done in interrupts. How boring..

Next?

[ReadOnlyCat]

Originally Posted by Toni Wilen

Next?

A rudimentary sorted list of features you have in mind next for UAE so we do not accidentally start doing work which will be wasted because you will release your version by surprise.

Sorry if I am insistent but nobody likes to waste time and even just a tiny amount of predictability helps coordination efforts.

[Galahad/FLT]

Originally Posted by Toni Wilen

Statistics update: "vh" command after enabling it now lists 40 first addresses with most CPU instruction word fetch activity. (Data is cleared after command)

It first finds highest CPU usage in stat data. Then finds all other nearby addresses with same or slightly less (5% less) CPU usage and outputs single line to console with start, end and percentage of total used. Then repeat same 40 lines.

EDIT: Note that list is not fully sorted, the longer the detected address space the worse the list is..

EDIT2: now also accepts two parameters, first parameter: percentage value of how big range is accepted (default 95 = 95% of highest accepted, set to zero to list all address), second parameter is number of lines to output.

I quickly checked some simple intros and demos and most CPU time was nearly always used "loop: btst #6,$bfe001; bne.s loop" main loop, everything else was done in interrupts. How boring..

Next?

Awesome stuff, will give you something more challenging next time

[spud]

The .7z archive above seems to be asking for a password to extract. Is this correct? If so, what might it be?

[Toni Wilen]

Originally Posted by Galahad/FLT

Awesome stuff, will give you something more challenging next time

There is still possibilities to make it more challenging

Like collecting more data, like if code run runs only in interrupt (and which level).

Originally Posted by spud

The .7z archive above seems to be asking for a password to extract. Is this correct? If so, what might it be?

It isn't. Your unarchiver may be too old.

[spud]

Thanks, Toni, you were spot on. I've updated to a newer version.

[Galahad/FLT]

How about setting exclusions? Such as you know that a left button test is high on the heatmap which isnt really what people would necessarily be interested in.

Maybe have an option to isolate all audio or all DMA accesses or you get the idea.

[Toni Wilen]

Originally Posted by Galahad/FLT

How about setting exclusions? Such as you know that a left button test is high on the heatmap which isnt really what people would necessarily be interested in.

Probably not worth the trouble, it is easy to ignore manually.

Maybe have an option to isolate all audio or all DMA accesses or you get the idea.

Do you mean listing all addresses that were "hit" by copper only or audio only, blitter only (all, reads only, writes only) and so on? That could be useful.