Sound overhaul for TKG

[abu_the_monkey]

as requested. Apollo 1240@40

Got Timer, tick frequency is 709379 Hz
Loaded sounds/airstrike.raw [60460 bytes] at 0x12bea80
Test case 0:
	Mix : None (data fectch only)
	Norm: None (data write only)
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    38262 ticks 189 packets
	Mixing  2 channel(s):    41067 ticks 189 packets
	Mixing  3 channel(s):    46106 ticks 189 packets
	Mixing  4 channel(s):    47412 ticks 189 packets
	Mixing  5 channel(s):    50888 ticks 189 packets
	Mixing  6 channel(s):    53516 ticks 189 packets
	Mixing  7 channel(s):    56062 ticks 189 packets
	Mixing  8 channel(s):    60356 ticks 189 packets
	Mixing  9 channel(s):    62955 ticks 189 packets
	Mixing 10 channel(s):    66026 ticks 189 packets
	Mixing 11 channel(s):    69822 ticks 189 packets
	Mixing 12 channel(s):    71968 ticks 189 packets
	Mixing 13 channel(s):    76486 ticks 189 packets
	Mixing 14 channel(s):    80697 ticks 189 packets
	Mixing 15 channel(s):    81467 ticks 189 packets
	Mixing 16 channel(s):    84157 ticks 189 packets
Test case 1:
	Mix : Multiplication
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68060

	Mixing  1 channel(s):   104047 ticks 189 packets
	Mixing  2 channel(s):   155481 ticks 189 packets
	Mixing  3 channel(s):   205045 ticks 189 packets
	Mixing  4 channel(s):   249343 ticks 189 packets
	Mixing  5 channel(s):   302113 ticks 189 packets
	Mixing  6 channel(s):   346093 ticks 189 packets
	Mixing  7 channel(s):   388344 ticks 189 packets
	Mixing  8 channel(s):   439071 ticks 189 packets
	Mixing  9 channel(s):   484781 ticks 189 packets
	Mixing 10 channel(s):   529004 ticks 189 packets
	Mixing 11 channel(s):   576807 ticks 189 packets
	Mixing 12 channel(s):   623998 ticks 189 packets
	Mixing 13 channel(s):   673350 ticks 189 packets
	Mixing 14 channel(s):   716279 ticks 189 packets
	Mixing 15 channel(s):   762594 ticks 189 packets
	Mixing 16 channel(s):   788144 ticks 189 packets
Test case 2:
	Mix : Shift Only
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040

	Mixing  1 channel(s):    93242 ticks 189 packets
	Mixing  2 channel(s):   129756 ticks 189 packets
	Mixing  3 channel(s):   158397 ticks 189 packets
	Mixing  4 channel(s):   189102 ticks 189 packets
	Mixing  5 channel(s):   221582 ticks 189 packets
	Mixing  6 channel(s):   254914 ticks 189 packets
	Mixing  7 channel(s):   284060 ticks 189 packets
	Mixing  8 channel(s):   318156 ticks 189 packets
	Mixing  9 channel(s):   345797 ticks 189 packets
	Mixing 10 channel(s):   379714 ticks 189 packets
	Mixing 11 channel(s):   408949 ticks 189 packets
	Mixing 12 channel(s):   442674 ticks 189 packets
	Mixing 13 channel(s):   474378 ticks 189 packets
	Mixing 14 channel(s):   505013 ticks 189 packets
	Mixing 15 channel(s):   535860 ticks 189 packets
	Mixing 16 channel(s):   554053 ticks 189 packets
Test case 3:
	Mix : Lookup
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    95102 ticks 189 packets
	Mixing  2 channel(s):   124686 ticks 189 packets
	Mixing  3 channel(s):   151468 ticks 189 packets
	Mixing  4 channel(s):   178837 ticks 189 packets
	Mixing  5 channel(s):   206412 ticks 189 packets
	Mixing  6 channel(s):   237385 ticks 189 packets
	Mixing  7 channel(s):   269127 ticks 189 packets
	Mixing  8 channel(s):   296815 ticks 189 packets
	Mixing  9 channel(s):   322307 ticks 189 packets
	Mixing 10 channel(s):   351619 ticks 189 packets
	Mixing 11 channel(s):   381851 ticks 189 packets
	Mixing 12 channel(s):   400917 ticks 189 packets
	Mixing 13 channel(s):   431904 ticks 189 packets
	Mixing 14 channel(s):   456465 ticks 189 packets
	Mixing 15 channel(s):   486514 ticks 189 packets
	Mixing 16 channel(s):   506035 ticks 189 packets
Test case 4:
	Mix : Delta Lookup
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    97014 ticks 189 packets
	Mixing  2 channel(s):   134346 ticks 189 packets
	Mixing  3 channel(s):   166873 ticks 189 packets
	Mixing  4 channel(s):   199198 ticks 189 packets
	Mixing  5 channel(s):   236140 ticks 189 packets
	Mixing  6 channel(s):   270004 ticks 189 packets
	Mixing  7 channel(s):   307443 ticks 189 packets
	Mixing  8 channel(s):   339725 ticks 189 packets
	Mixing  9 channel(s):   372852 ticks 189 packets
	Mixing 10 channel(s):   401830 ticks 189 packets
	Mixing 11 channel(s):   434775 ticks 189 packets
	Mixing 12 channel(s):   467363 ticks 189 packets
	Mixing 13 channel(s):   498165 ticks 189 packets
	Mixing 14 channel(s):   540029 ticks 189 packets
	Mixing 15 channel(s):   566729 ticks 189 packets
	Mixing 16 channel(s):   585569 ticks 189 packets
Preencoding sample data at 0x12bea80, 3779 frames...
Preencoding sample data at 0x13e8860, 3779 frames...
Test case 5:
	Mix : Delta Lookup (Pre-encoded source)
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040

	Mixing  1 channel(s):    94100 ticks 189 packets
	Mixing  2 channel(s):   124813 ticks 189 packets
	Mixing  3 channel(s):   150709 ticks 189 packets
	Mixing  4 channel(s):   177561 ticks 189 packets
	Mixing  5 channel(s):   203244 ticks 189 packets
	Mixing  6 channel(s):   232850 ticks 189 packets
	Mixing  7 channel(s):   262880 ticks 189 packets
	Mixing  8 channel(s):   287542 ticks 189 packets
	Mixing  9 channel(s):   311968 ticks 189 packets
	Mixing 10 channel(s):   346801 ticks 189 packets
	Mixing 11 channel(s):   361729 ticks 189 packets
	Mixing 12 channel(s):   389527 ticks 189 packets
	Mixing 13 channel(s):   415901 ticks 189 packets
	Mixing 14 channel(s):   439177 ticks 189 packets
	Mixing 15 channel(s):   467195 ticks 189 packets
	Mixing 16 channel(s):   481485 ticks 189 packets

[Karlos]

I think the hunch that there was too much to do per sample to do the delta analysis on the fly was on point.

Comparing the linear to the pre-encoded delta, what we are hopefully seeing is the improvement in cache accesses.

I am going to do a really dirty and crude calculation here. If we subract the null mixer time we should be removing the overhead of the source data fetch and chip write* and be left only with the actual mixing and normalisation time. For the 16 channel cases:

Multiplication: 788144 - 84157 = 703987
Shift Only: 554053 - 84157 = 469896
Linear Lookup: 506035 - 84157 = 421878
Delta Lookup: 585569 - 84157 = 501412
Preencoded Delta Lookup: 481485 - 84157 = 397328


I honestly expected the Shift Only version to be fastest, but based on that, it's 18.3% faster. I am going to have to look long and hard at that because I'm struggling to undersand how that can be the case. Shifting by a value in a register should cost almost nothing.

*note that we know the real workload hides work behind the writes.

[Karlos]

While I can't explain the shift only result (some sort of horrible pipeline stall I guess), the linear versus pre-encoded delta is more in line with what I was expecting. Worth noting that because all the channels are using the same source sounds with different offsets (all multiples of 16), the linear lookup case here is almost certainly better than it would otherwise be, since lookups for one channel will result in the data still being in the cache for another one soon after. That won't be the case normally.

[Karlos]

Since our mixing runs are only a cache line at once, it's probably worth unrolling some of these.

[paraj]

060 results for good measure.

Got Timer, tick frequency is 709379 Hz
Loaded sounds/airstrike.raw [60460 bytes] at 0x68bfc110
Test case 0:
	Mix : None (data fectch only)
	Norm: None (data write only)
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    28091 ticks 189 packets
	Mixing  2 channel(s):    32478 ticks 189 packets
	Mixing  3 channel(s):    34929 ticks 189 packets
	Mixing  4 channel(s):    37561 ticks 189 packets
	Mixing  5 channel(s):    39778 ticks 189 packets
	Mixing  6 channel(s):    42932 ticks 189 packets
	Mixing  7 channel(s):    45105 ticks 189 packets
	Mixing  8 channel(s):    48237 ticks 189 packets
	Mixing  9 channel(s):    50233 ticks 189 packets
	Mixing 10 channel(s):    52975 ticks 189 packets
	Mixing 11 channel(s):    55874 ticks 189 packets
	Mixing 12 channel(s):    58548 ticks 189 packets
	Mixing 13 channel(s):    60968 ticks 189 packets
	Mixing 14 channel(s):    64066 ticks 189 packets
	Mixing 15 channel(s):    65825 ticks 189 packets
	Mixing 16 channel(s):    67801 ticks 189 packets
Test case 1:
	Mix : Multiplication
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68060

	Mixing  1 channel(s):    47943 ticks 189 packets
	Mixing  2 channel(s):    64917 ticks 189 packets
	Mixing  3 channel(s):    79624 ticks 189 packets
	Mixing  4 channel(s):    94172 ticks 189 packets
	Mixing  5 channel(s):   108015 ticks 189 packets
	Mixing  6 channel(s):   122202 ticks 189 packets
	Mixing  7 channel(s):   136846 ticks 189 packets
	Mixing  8 channel(s):   151261 ticks 189 packets
	Mixing  9 channel(s):   165008 ticks 189 packets
	Mixing 10 channel(s):   179984 ticks 189 packets
	Mixing 11 channel(s):   193753 ticks 189 packets
	Mixing 12 channel(s):   208361 ticks 189 packets
	Mixing 13 channel(s):   222461 ticks 189 packets
	Mixing 14 channel(s):   236959 ticks 189 packets
	Mixing 15 channel(s):   250189 ticks 189 packets
	Mixing 16 channel(s):   260117 ticks 189 packets
Test case 2:
	Mix : Shift Only
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040

	Mixing  1 channel(s):    48941 ticks 189 packets
	Mixing  2 channel(s):    68683 ticks 189 packets
	Mixing  3 channel(s):    82186 ticks 189 packets
	Mixing  4 channel(s):    96897 ticks 189 packets
	Mixing  5 channel(s):   111798 ticks 189 packets
	Mixing  6 channel(s):   126199 ticks 189 packets
	Mixing  7 channel(s):   141283 ticks 189 packets
	Mixing  8 channel(s):   155747 ticks 189 packets
	Mixing  9 channel(s):   170288 ticks 189 packets
	Mixing 10 channel(s):   185352 ticks 189 packets
	Mixing 11 channel(s):   199467 ticks 189 packets
	Mixing 12 channel(s):   214552 ticks 189 packets
	Mixing 13 channel(s):   229121 ticks 189 packets
	Mixing 14 channel(s):   243713 ticks 189 packets
	Mixing 15 channel(s):   257308 ticks 189 packets
	Mixing 16 channel(s):   267358 ticks 189 packets
Test case 3:
	Mix : Lookup
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    50908 ticks 189 packets
	Mixing  2 channel(s):    73036 ticks 189 packets
	Mixing  3 channel(s):    91123 ticks 189 packets
	Mixing  4 channel(s):   109890 ticks 189 packets
	Mixing  5 channel(s):   127994 ticks 189 packets
	Mixing  6 channel(s):   145942 ticks 189 packets
	Mixing  7 channel(s):   165852 ticks 189 packets
	Mixing  8 channel(s):   186309 ticks 189 packets
	Mixing  9 channel(s):   202551 ticks 189 packets
	Mixing 10 channel(s):   222043 ticks 189 packets
	Mixing 11 channel(s):   239704 ticks 189 packets
	Mixing 12 channel(s):   258584 ticks 189 packets
	Mixing 13 channel(s):   275865 ticks 189 packets
	Mixing 14 channel(s):   294273 ticks 189 packets
	Mixing 15 channel(s):   312876 ticks 189 packets
	Mixing 16 channel(s):   323808 ticks 189 packets
Test case 4:
	Mix : Delta Lookup
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040/60

	Mixing  1 channel(s):    51653 ticks 189 packets
	Mixing  2 channel(s):    75233 ticks 189 packets
	Mixing  3 channel(s):    94751 ticks 189 packets
	Mixing  4 channel(s):   114344 ticks 189 packets
	Mixing  5 channel(s):   134937 ticks 189 packets
	Mixing  6 channel(s):   155077 ticks 189 packets
	Mixing  7 channel(s):   175402 ticks 189 packets
	Mixing  8 channel(s):   197675 ticks 189 packets
	Mixing  9 channel(s):   216831 ticks 189 packets
	Mixing 10 channel(s):   236820 ticks 189 packets
	Mixing 11 channel(s):   255212 ticks 189 packets
	Mixing 12 channel(s):   275194 ticks 189 packets
	Mixing 13 channel(s):   294581 ticks 189 packets
	Mixing 14 channel(s):   315643 ticks 189 packets
	Mixing 15 channel(s):   332874 ticks 189 packets
	Mixing 16 channel(s):   346663 ticks 189 packets
Preencoding sample data at 0x68bfc110, 3779 frames...
Preencoding sample data at 0x68c0ad60, 3779 frames...
Test case 5:
	Mix : Delta Lookup (Pre-encoded source)
	Norm: Multiplication/Shift
	Info: Move16 fetch, target 68040

	Mixing  1 channel(s):    50401 ticks 189 packets
	Mixing  2 channel(s):    72371 ticks 189 packets
	Mixing  3 channel(s):    90496 ticks 189 packets
	Mixing  4 channel(s):   109184 ticks 189 packets
	Mixing  5 channel(s):   127111 ticks 189 packets
	Mixing  6 channel(s):   146079 ticks 189 packets
	Mixing  7 channel(s):   164857 ticks 189 packets
	Mixing  8 channel(s):   184502 ticks 189 packets
	Mixing  9 channel(s):   203924 ticks 189 packets
	Mixing 10 channel(s):   221640 ticks 189 packets
	Mixing 11 channel(s):   239556 ticks 189 packets
	Mixing 12 channel(s):   256937 ticks 189 packets
	Mixing 13 channel(s):   277273 ticks 189 packets
	Mixing 14 channel(s):   294712 ticks 189 packets
	Mixing 15 channel(s):   314401 ticks 189 packets
	Mixing 16 channel(s):   325140 ticks 189 packets

[Karlos]

There's no question the 060 version should stick with multiplication based mixing.

I think that one obvious thing coming out here is that the maximum number of effective channels should be user definable, e.g. you may have a fundamental limit of 16 but a user might restrict it to a lower value than that.

[Karlos]

I think the next piece of the puzzle is music.

I have a dilemma here. I might want the music to be at a higher sample rate than the sounds, but that's just not how the mixer works right now. I am contemplating a simple scheme of full and half rate samples. Half rate could be trivially upsampled on the fly but I'd prefer to do that in a manner that doesn't duplicate a bunch of operations later, e.g. mix all the half rate, upsampled the intermediate, mix all the full rate. Something like that.

[paraj]

Originally Posted by Karlos

There's no question the 060 version should stick with multiplication based mixing.

I think that one obvious thing coming out here is that the maximum number of effective channels should be user definable, e.g. you may have a fundamental limit of 16 but a user might restrict it to a lower value than that.

Yes, just added results so there's timing for baseline case there as well. Also interesting to see "shift only" results. On 060 they're more in line with my intuition. Like you suggest there must be some weird pipeline thing in effect, but I just can't think of a good reason why that would be the case. I'd expect a mem lookup would need to result earlier in the pipeline, but maybe I should just forget about it

Originally Posted by Karlos

I think the next piece of the puzzle is music.

I have a dilemma here. I might want the music to be at a higher sample rate than the sounds, but that's just not how the mixer works right now. I am contemplating a simple scheme of full and half rate samples. Half rate could be trivially upsampled on the fly but I'd prefer to do that in a manner that doesn't duplicate a bunch of operations later, e.g. mix all the half rate, upsampled the intermediate, mix all the full rate. Something like that.

Right now "music" (at least in base game) is just one channel most (all?) of the time right? Does it actually use the mod player capability?


For the reworked engine ADPCM might even be more appropriate..


For low spec machines at least, you could consider giving music 2 channels (1L/1R) and only mixing for SFX. Just an idea.

[Karlos]

The current music engine is a single channel mod. It uses a full hardware channel for that, meaning that 4 and 8 channel mode are really 3 and 6 channel mode respectively. On the plus side, it does fully decouple the music from the sound effects allowing it to use a higher sample rate, which I make some use of with the remixed version of the music.

My plan for music with this system has always been to decode a stream directly into the accumulation buffers (instead of clearing them) and mix on top of it. One benefit being that we can theoretically do better than 8-bit for our stream. I am a bit worried about the overall quality though, particularly the sample rate. 16kHz is already a lot better for the sound effects (if using properly sourced data) but isn't great for music.

It's all a bit moot until I implement the actual output end mind. So far I'm just evaluating everything by dumping the buffers and using a quick script to reassemble them into 16 bit audio.