[request] Not exactly an ST port.... Black Tiger Redux?

[touko]

Massive in the sense of many of colors have to be droped out(twice less colors + the 9 bits main palette), on YT is a bit difficult to notice, and seems to be close to the arcade, but in fact not really.
The colors choice due to the limited sub palettes was the main(maybe only) reproach of the arcade fans, and a good amount of sprites have simply not the same colors than the arcade ones .

But i must admit that the arcade gfx were poorly colored too,the colors use is really not optimal at all and makes an useless growth in colors count .

[h0ffman]

Originally Posted by chadderack

Welp; I bit the bullet and sent Mr. DMP Soft an email. Hope he agrees to share some src code.

I'm probably going to switch back to reversing the rest of the maincpu code and finish what graphics stuff is left. Kinda stubborn; I don't like to leave stones unturned, but I can't do too much with the audio at this point. I'll have to engineer a better understanding of how the arcade's byte codes for audio are transformed into actual YM2203 and SSG data.

I still have to figure out how the map tiles are indexed by the app. I know the size of the tiles and the screen positions; but I need to understand how the arcade's tile indexing relates to gfx1/gfx2/gfx3 ROM addresses. I can probably still work with the map tiles without that understanding, as all you have to do is piece the map together by hand. MAME is kind enough to give you the tiles.

The thing about the tiles in Black Tiger, though, is that in Level 3... some of the tiles are animated (palette animation).

I think giving my brain a bit of a break on the YM2203/SSG audio stuff for a bit might be a good idea.


Chad.

Loving this thread! I recently disassembled Knightmare on the MSX which is also Z80 but I was massively helped by porting Metal Gear this year as they are both MSX and Konami games.

As for the audio part, it will be without a doubt the single hardest task in the reversing process. Doing MG I did full enhanced sound, samples and ProTracker, but also ported the Z80 sound driver to 68k with a basic emulation / conversion layer into Paula on top. It was the hardest thing of all to get working and the disassembly by Manuel Pazos was fully commented!

In terms Amiga, here are some options...

AmigaKlang

It's a synth engine in that you generate a preset / patch for it and it will generate the sample for you. The sample still needs to sit in chip-ram to be playable so you aren't saving ram, only disk space. Also it will come at the cost of pre-calc time.

PreTracker

I've not used it enough myself to really comment here but I have a feeling there are some real-time options within the SFX here so could save you RAM.

PHX Replay and SFX

All ProTracker and samples. Benefit is that you'll easily find someone who can make the music. The draw back however is that its ProTracker and the pattern data is incredibly bloated. There is a bunch on things you can do here like split the song and sample data and have the song data in fast ram. You SFX will obviously be samples here but you and attenuate them to any sample rate thus offsetting size with quality to whatever degree you need. My fave benefit here is that the music can use 4 channels so when the game isn't kicking off SFX you get the full soundtrack.

Dutch Retro Guys mixed SFX Engine

Massive benefit here in that you can use 1 channel and play multiple SFX at the same time, leaving 3 Channels for music. Also all the samples can sit in fast ram saving you chip RAM. Draw backs are all the samples need to be at a fixed sample rate and also it uses some CPU time but its considerably optimised and well written. Believe this has also been integrated with PHX ProTracker replay.


Anyway, that's my 10p worth.

Keep it up, love seeing reversing projects like this.

[saimon69]

Originally Posted by h0ffman

In terms Amiga, here are some options...

AmigaKlang

It's a synth engine in that you generate a preset / patch for it and it will generate the sample for you. The sample still needs to sit in chip-ram to be playable so you aren't saving ram, only disk space. Also it will come at the cost of pre-calc time.

Since Chadderack is mainly concerned about fidelity to the YM chip i would say this might be one of the best options for him: use a YM chip emulator to generate the samples based on the sound data.

Am doing some tests on the samples-from-Hoot route; results are decent so far, however have to decide the storage method and how to organize songs in case.

The usual method for me is to group songs used in the same time; that would in case of a BT match involve:

- round music
- store
- dungeon
- boss
- round complete
- game over

however due to the size of samples and disk space i might just have either round music with dungeon and store, and a 'service mod' with all interactions like round complete, game over separated, or all song separated to reduce space in RAM and keep sizes small.
The round 1 music test so far not complete is around 55k but no dungeon no store.

[chadderack]

So ... Black Tiger again...

With a fresh day comes new understandings. I've gone back and looked at the data spit out by MAME when running the "Game Over" tune.

My findings are in source control in "game_over_decode_pdm.txt" in the main project.

...

RECAP FOR UNDERSTANDING

• Black Tiger has its own representation for each "sound". That data is nice and compact.
  
• MAME seems to have its own data format for the sound data. It is different from the data in Black Tiger--mostly in that it doesn't have timings. It is also "fluid" in that it doesn't reside anywhere... it is a stream of data. For timings, Black Tiger's audio chip must be controlling when each command is sent to either YM1/YM2.
  
  Apparently MAME stores addresses and data "passively" (meaning it holds on to the YM2203 addresses and data) until it should run them through the YM1/YM2 "chips." This is the biggest missing piece when debugging audio through MAME--when are the notes played?
  
• A VGM file is another type of MML file that has its own codes that aren't direct addresses and data for the YM2203 chip. I believe the format was created by "Valley Bell."
  
• There's also a native format for the YM2203 chip, which I haven't seen quite yet... but I don't believe that the sound chips themselves handle timing--so the codes from MAME are probably pretty close to the actual YM2203 hardware codes.

PROBLEM(s) TO SOLVE

1. Black Tiger audio data => HW data format for each sound
2. Implementation of YM2203 FM synthesis on Amiga

The second one should be a cinch if I just follow Aaron Giles' C++ source code. I should be able to implement that in the Amiga hardware in ASM.

The first one I can probably get to the hardware format by either new version of MAME where I dump out all of the data during playback. (Building MAME is usually pretty painful).

Or, I can (if the VGM version of the "Game Over" sound is lossless) use the VGM files we have for Black Tiger (under "audio" in the Github project) and build any number of VGM players (source code for a half dozen is available) and see just what makes it directly into the chip channels and operators.

...

Don't know why I was worried about reproducing Black Tiger's audio data compression for the Amiga port; it's an academic exercise to implement an encoder. Worst case, I can do simple RLE encoding for something like VGM (or a dictionary lookup version) and crunch it down quite a ways. Or use a very VGM-like format and crunch/decrunch.

If anyone else is interested in jumping into the task of FM synthesis for this new port, let me know. I could use another good programmer's "CPU"

Salmon69 is going to do the samples version. Hopefully one of the ways works out good. Maybe we could implement both?

[chadderack]

Originally Posted by h0ffman

Loving this thread! I recently disassembled Knightmare on the MSX which is also Z80 but I was massively helped by porting Metal Gear this year as they are both MSX and Konami games.

As for the audio part, it will be without a doubt the single hardest task in the reversing process. Doing MG I did full enhanced sound, samples and ProTracker, but also ported the Z80 sound driver to 68k with a basic emulation / conversion layer into Paula on top. It was the hardest thing of all to get working and the disassembly by Manuel Pazos was fully commented!

DANG! Replies during replies. Excellent!

In terms Amiga, here are some options...

AmigaKlang

It's a synth engine in that you generate a preset / patch for it and it will generate the sample for you. The sample still needs to sit in chip-ram to be playable so you aren't saving ram, only disk space. Also it will come at the cost of pre-calc time.

PreTracker

I've not used it enough myself to really comment here but I have a feeling there are some real-time options within the SFX here so could save you RAM.

PHX Replay and SFX

All ProTracker and samples. Benefit is that you'll easily find someone who can make the music. The draw back however is that its ProTracker and the pattern data is incredibly bloated. There is a bunch on things you can do here like split the song and sample data and have the song data in fast ram. You SFX will obviously be samples here but you and attenuate them to any sample rate thus offsetting size with quality to whatever degree you need. My fave benefit here is that the music can use 4 channels so when the game isn't kicking off SFX you get the full soundtrack.

Dutch Retro Guys mixed SFX Engine

Massive benefit here in that you can use 1 channel and play multiple SFX at the same time, leaving 3 Channels for music. Also all the samples can sit in fast ram saving you chip RAM. Draw backs are all the samples need to be at a fixed sample rate and also it uses some CPU time but its considerably optimised and well written. Believe this has also been integrated with PHX ProTracker replay.

Anyway, that's my 10p worth.

Keep it up, love seeing reversing projects like this.

Awesome. Thank you so much for the input and support
If you'd like to work on this too, let me know

Originally Posted by saimon69

Since Chadderack is mainly concerned about fidelity to the YM chip i would say this might be one of the best options for him: use a YM chip emulator to generate the samples based on the sound data.

That's the plan.

Am doing some tests on the samples-from-Hoot route; results are decent so far, however have to decide the storage method and how to organize songs in case.

The usual method for me is to group songs used in the same time; that would in case of a BT match involve:

- round music
- store
- dungeon
- boss
- round complete
- game over

however due to the size of samples and disk space i might just have either round music with dungeon and store, and a 'service mod' with all interactions like round complete, game over separated, or all song separated to reduce space in RAM and keep sizes small.
The round 1 music test so far not complete is around 55k but no dungeon no store.

Cool. Keep us posted

[chadderack]

The update for today is just to mention that:

The best way to figure out Black Tiger audio is to run the code step-by-step through the debugger. I load up the MAME Black Tiger test audio screen (yes, the audio test screen does work) in the manner mentioned in my docs (in source control).

I trigger the sound in the sound test screen, and my breakpoints in the audio cpu are hit.

Today I've been "translating" Black Tiger's internal audio codes (the audio data representing each sound) (as much as possible) to figure out what each code means. I'm seeing lookup tables, offsets, and it does seem to manage note durations. It will probably take me a few more very slow (and sometimes boring) sessions testing the sounds, running step-by-step, to get it all figured out.

Man, there is some funky math going on in that ASM code just to compute table offsets and offsets to offsets. Just gotta have some patience, which is a very big trick for me

[dlfrsilver]

Excellent, continue !

[vulture]

that's some dedication chadderack!

[chadderack]

Cheers for the encouragement, dlfrsilver and vulture.

So I'm in my "tinkering time" today away from the gf for a few hours, working on annotating some Black Tiger arcade audio code... when it occurred to me...

Black Tiger is monaural. There's only one speaker; meaning that all "channels" are actually mixed down to mono before output.

This is significant (potentially) because audio summing (something I know about) is something we can probably do anyway. In a parallel life, I went to audio engineer school (actually kinda did) and this seems doable.

In fact, I remember from some of the source readings on the YM2203 that output is essentially summed into a single channel.

If so, that's great news because we could sum both sets of virtual YM2203 devices (what MAME calls YM1 and YM2--the two YM2203 chips in Black Tiger) internally down to mono. That leaves 2 more audio channels free for Paula.

Anybody have different information? I think the restriction of "having" to use 6 channels of audio just because Black Tiger has two three-channel YM2203s isn't a restriction at all. When we're talking about YM2203--yes, the "channels" are separate. But that is not the same as AUDIO channels... the ones that actually play back.

In a sense, YM2203 "channels" are just data streams. All 6 of them are summed to mono audio in Black Tiger.

Summing to mono is no problem (essentially like mixing down to mono) because we aren't behind a mixing desk when playing the game; the relative levels are all set. The sound should be the same every time each sound is played. So we can "print" an audio track (stream) in much the same way as a mixdown would work in a DAW.

tl;dr -- we don't have to worry about audio channel restrictions like we thought, because YM2203 channels aren't the same thing as audio channels--of which Black Tiger (arcade) only has one.

[dlfrsilver]

Excellent. Damn U.S.Gold !

[h0ffman]

Originally Posted by chadderack

[*]Implementation of YM2203 FM synthesis on Amiga

I may have missed the target amiga you were aiming for but from my experience FM synthesis is multiplication heavy. I can't imagine that would run in realtime on a 68000 without fast ram, even pre-calcing that is going to be very slow.

However, I've not read up on that chip too much so who knows.

Looking forward to seeing how this "pans" out ( sorry for the audio pun ).

[chadderack]

Originally Posted by h0ffman

I may have missed the target amiga you were aiming for but from my experience FM synthesis is multiplication heavy. I can't imagine that would run in realtime on a 68000 without fast ram, even pre-calcing that is going to be very slow.

However, I've not read up on that chip too much so who knows.

Looking forward to seeing how this "pans" out ( sorry for the audio pun ).

Thanks man.

The Black Tiger arcade code seems to have lookups for note values and durations. MAME's implementation seems straightforward (and could be translated to 680x0), but MAME is meant to run on 32/64 bit machines.

I'll keep that in mind. Worst case, we'll get an old walkie talkie in there or something :P

No, I'll give it an old Engineering try. It'll be a challenge! Thanks h0ffman!

...

I've just finished reversing the SSG audio clips (completely) including all special Black Tiger-only "commands" (bytes that expand out to actual SSG addresses/data).

There's a few more things to reverse for the FM stuff, but if you check my Ghidra source project, you'll see there's not much left to finish. Almost done with the audiocpu code.

Gotta go watch some Amiga YT vids now (am only 20 minutes into Photon's new 2 hr video).

[chadderack]

salmon69 may get a kick out of this.

While working on finishing up the FM code, I realized that there are some "open gaps" of "RAM" (rom) in the audiocpu codespace. This made me wonder if I could write my own Black Tiger tune (!) and see if it played back(!)

Not quite there yet, but I decided to copy a short piece of music (the 7 notes of the "coin input" music) and modify it a little here and there.

In the past week or so, I've been parsing out the "codes" that Black Tiger intersperses into its sound data. These "codes" are very much like the "commands" Photon of Scoopex was just talking about in his YT video #45--scroller commands.

Anyway... there's a handful of these commands. One of them is 0x7 or hex 7.
This command apparently caused the OPs of each channel to get initialized with some data.

Ok, so I decided to try out my "open code space/own song" thing and copy/pasted the "Coin Added" sound bytes, fixing up all of the pointers (there are at least 6 pointers at the head of each piece of "music").

After doubling the byte after each "0" (or 0x0 zero command) (a different command--one dealing with note lengths) and seeing that this indeed did lengthen every note, I decided to play with the 7 command.

Seems I stumbled upon the patches! It was like finding 45 little settings on my Nord Stage 2 keyboard that played Black Tiger instruments.

With the memory window open and set to Region ':audiocpu' (not Zilog Z80 ':audiocpu' program space memory... that is read-only) i was able to copy/paste and patch each byte in the open rom area. Then all I had to do was set a breakpoint at 212h and switch the address in DE to 6380h (de = 6380).

I began to change each byte after each "7" command from 0xh through about 0x47 (about 70 of them) and heard differences each time. Based on what each sounded like, I named each patch.

I've given them all wonky names, but defined them in the code. Makes it easier to see which channels are percussion, etc.



....

Also... in light of this... and in light of the info that Bamboo Tracker can directly play YM2203 music (!) it would seem all I have to do is extract all 45 patches and import them into Bamboo Tracker. Voila! Instant ability to TRACK new (or existing) Black Tiger tunes!

That would still be pretty hard, though, because of note timings and delays that aren't directly encoded into the Black Tiger sound data. But I'm figuring the note duration piece out (still)... seems some of that code is twisted around itself.

Anyway, as you all were About to go see the gf and have some Sunday night wine and other things

[sparhawk]

Do I understand this right that you are using Ghidra for reversing? Do you use some special plugins?

[chadderack]

Originally Posted by sparhawk

Do I understand this right that you are using Ghidra for reversing? Do you use some special plugins?

Yep; Ghidra. No special plugins. I also had to define the overlays (or ROM banks) for the main CPU. Finding the bank switching routine in the maincpu was critical for figuring out where the banks were and how big they were. MAME was a big help describing a piece of the general memory map.

For the maincpu, there are 16 banks--only the first 8 of which have game code--and two of those are mostly data. The last 8 banks are all graphics.

The audiocpu has its own codespace and doesn't use any overlays.

[chadderack]

Just figured out most of the note timings.

Seems command:
0x6h = dotted note
0x30h = triplet note
0x2nh = hold note where 1h <= n <= 4h. The higher the n value, the longer the hold. 21h is 4x a normal note length. 22h is 5x, etc...

BT doesn't use command values above 24h.

I am able to (as mentioned in a previous post) play my own notes with my own patches in the "open ROM" area at 6380h on the audiocpu. That's helped tremendously in comparing note lengths and so forth.

A little more to do around decoding the "note" byte (certain bitfields control attack/decay)... but we know that the sound byte (XX & 1Fh) = note values offset from the current octave start value.

EX: 81 & 1F = 01. Note #1 in the current octave of the fm channel patch.
If the octave starts on C1, then 81 would be C#1. So would A1. But (as I said) the upper bits of the notes do something with attack/decay.

Also need to pull all of the patch data and computation tables out and make sure I understand how those are used.

Not sure there would be much more to do after that in terms of reversing the audio cpu.

I would think next steps are exporting the source of each piece of "music" (or sound effect) and translate my annotated ASM data into music sheets--or into a Bamboo Tracker track.

If the format is easy enough to understand, I could script something in C# (console app) that takes the BT input and outputs Bamboo Tracker files. I wonder how easily they would compress?

Another approach would be to encode them into my own (new) format (that I'd have to create)--a format that would be easy to use in Amiga 680x0 asm.

...

The year and a half has been rough; I'll be heading out of town next week with the gf to Disneyland. At 52, it's about time I went there before I die

Will pick things back up at that time. I'll probably have a few more commits before the end of this week.

[saimon69]

Hey Chadderack, Check your PM, i sent you a proof of concept in three channels

[saimon69]

If you can, also try to visit CLASS in Burbank on November ^__^ i might almost certainly be there
The Commodore Los Angeles Super Show

[chadderack]

Originally Posted by saimon69

If you can, also try to visit CLASS in Burbank on November ^__^ i might almost certainly be there
The Commodore Los Angeles Super Show

Hmmm Very interesting

We'll be in SoCal next week, but maybe I could convince the gf to go down.
It's about 6 weeks before we head to Jamaica--I may be able to convince her to get on an airplane

In terms of the stuff in PM... I'll check it when I get a chance. Have to finish up some stuff for work this week and there's not much time for a week or two.

...

In terms of the reversing project--just finished defining all of the music data in the audiocpu project. All of the notes and rests.

I'm thinking this should be relatively easy to build a "parser" from... in other words, a script that can take BT bytes as input and output something. Maybe MIDI. Maybe MML. If I can spit out MIDI, I could import that MIDI into Sibelius and really get silly--output some music score sheets

[chadderack]

Not sure how many more updates there will be before leaving town. Just logged out of work. Will be prepping for vacation tomorrow and Saturday.

A good update, though.
I decided to annotate the notes with their durations (wholenote, 1/2, 1/4 etc)
so it would read more like music.

Still need to update the triplets, hold notes and rests to have their durations.

Essentially, you could create a MIDI from the text listing. The top three bits of the note byte select the note length, and the rest of the bits are an offset from the note "C". So 81 would be a C# 1/8th note.

                             LEVEL_5_YM1_CH1_LOOP_02                         XREF[1]:     ram:2d4b(*)  
        ram:2d48 8a              1/8_note
        ram:2d49 0d 0a 48 2d     loop                 ; loops the above note 10 times (0a)

        ram:2d4d 89              1/8_note
        ram:2d4e 89              1/8_note
        ram:2d4f 89              1/8_note
        ram:2d50 89              1/8_note
        ram:2d51 89              1/8_note
        ram:2d52 02 c8           toneperiod
        ram:2d54 8a              1/8_note
        ram:2d55 80              rest
        ram:2d56 8d              1/8_note
        ram:2d57 80              rest
        ram:2d58 6b              1/16_note
        ram:2d59 60              rest
        ram:2d5a 66              1/16_note
        ram:2d5b 60              rest
        ram:2d5c a3              1/4_note
        ram:2d5d 8a              1/8_note
        ram:2d5e 8c              1/8_note
        ram:2d5f 8d              1/8_note
        ram:2d60 8a              1/8_note
        ram:2d61 6b              1/16_note

Here's what the data looks like (expanded)

                             LEVEL_5_YM1_CH1_LOOP_02                         XREF[1]:     ram:2d4b(*)  
        ram:2d48 8a              1/8_note
           ram:2d48 8a              byte:5    Ah                      offset                            XREF[1]:     ram:2d4b(*)  
           ram:2d48 8a              byte:3    4h                      period_factor                     XREF[1]:     ram:2d4b(*)  
        ram:2d49 0d 0a 48 2d     loop
           ram:2d49 0d              bt_fm_co  d_loop                  loop_type
           ram:2d4a 0a              db        Ah                      times_to_loop
           ram:2d4b 48 2d           addr      LEVEL_5_YM1_CH1_LOOP_02 loopback_point = 
        ram:2d4d 89              1/8_note
           ram:2d4d 89              byte:5    9h                      offset
           ram:2d4d 89              byte:3    4h                      period_factor
        ram:2d4e 89              1/8_note
           ram:2d4e 89              byte:5    9h                      offset
           ram:2d4e 89              byte:3    4h                      period_factor
        ram:2d4f 89              1/8_note
           ram:2d4f 89              byte:5    9h                      offset
           ram:2d4f 89              byte:3    4h                      period_factor
        ram:2d50 89              1/8_note
           ram:2d50 89              byte:5    9h                      offset
           ram:2d50 89              byte:3    4h                      period_factor
        ram:2d51 89              1/8_note
           ram:2d51 89              byte:5    9h                      offset
           ram:2d51 89              byte:3    4h                      period_factor
        ram:2d52 02 c8           toneperiod
           ram:2d52 02              bt_fm_co  set_tone_period         cmd
           ram:2d53 c8              db        C8h                     value
        ram:2d54 8a              1/8_note
           ram:2d54 8a              byte:5    Ah                      offset
           ram:2d54 8a              byte:3    4h                      period_factor
        ram:2d55 80              rest
           ram:2d55 80              byte:5    0h                      offset
           ram:2d55 80              byte:3    4h                      period_factor
        ram:2d56 8d              1/8_note
           ram:2d56 8d              byte:5    Dh                      offset
           ram:2d56 8d              byte:3    4h                      period_factor
        ram:2d57 80              rest
           ram:2d57 80              byte:5    0h                      offset
           ram:2d57 80              byte:3    4h                      period_factor
        ram:2d58 6b              1/16_note
           ram:2d58 6b              byte:5    Bh                      offset
           ram:2d58 6b              byte:3    3h                      period_factor