Generating an accurate Paula period table

[8bitbubsy]

Hi,

I'm creating a small, portable MOD player in C and I want to generate the period table (36 notes, spanned over 16 finetunes, padded with a zero before the new finetune, totalling in 592 values) but all the routines I've found on the net seem to be -1/+1 off on certain periods. The reason I want to do this is because the table is 1184 bytes long. It's not that big but it'll shave off some bytes for very small packed intros.

I found periodtable.bas by Lars Hamre (?) but converting it to C makes rounding errors. I've tried to use floor(val + 0.5) and what not, nothing seems to work. I've tried for hours to change variables. One time I managed to get some of the red values correct, but then some other ones went off.
I thought at first that maybe the periods must not be even values, but there are both odd and even values in the original table.

Here's the table I want to generate: http://pastebin.com/6xVK2msR
Here's PERIODTA.BAS: http://pastebin.com/3GcVZFG8
Here's the C code that generates the table: http://pastebin.com/2KTibxBk
Here's the (wrongly) generated period table from the C code: http://pastebin.com/fSRqDZRm

Using my C-converted code, the first 36 notes with no finetune added is like this:

    856,808,763,720,679,641,605,571,539,509,480,453,
    428,404,381,360,340,321,303,286,270,254,240,227,
    214,202,191,180,170,160,151,143,135,127,120,113,0,

It's supposed to be like this:

    856,808,762,720,678,640,604,570,538,508,480,453,
    428,404,381,360,339,320,302,285,269,254,240,226,
    214,202,190,180,170,160,151,143,135,127,120,113,0,

Some helpers on the constants in the QuickBasic file:
fclk = 4433618.75#*1.6
this is PAL color clock carrier * 1.6, which yields 7093790Hz which is the Amiga PAL CPU clock.

fr=258.9731 * b
258.9731 is a hand-rounded value from (7093790 / (2 * (856 * 16)) which yields 258.9730578271028. 856 is the mid-C period value.

[demolition]

The only thing I spot is on line 21 where you do an (int) cast. This is the same as a floor, so I'd add 0.5 to that line:

mt_PeriodTable[(i * 37) + j] = (int)(0.5+(1.0 / fr / pclk) / 16.0);

Not sure if this is what you meant by floor(val+0.5).

[StingRay]

The way I did it for one of my 4k intros was to generate a table and then in a 2nd pass I corrected the wrong values (5 or 6 IIRC) with another small lookup table. I think I never managed to create a 100% correct period table using code only, the 2nd LUT quite annoyed me but since it was very very small in the end I didn't care much about it.

[Predseda]

Sorry, but the thread title made me laugh

[8bitbubsy]

Yeah, I thought about having a 1-bit shift table to prevent errors, but if there is a formula that works instead then I'd much prefer that. I guess the original period table was fixed by hand and ear, since the period accuracy is not spot on. In fact, it's pretty bad.


Anyways, here's the replayer I made if anyone's interested (I converted the PT2.3a replayer from asm to C, it's very accurate): http://pastebin.com/5dPR3yiL
I added some bugfixes too. There's not many sanity checks on the loaded MOD tho', this player assumes you have a valid ProTracker module. For a more "safe" replayer, check out playptmod, which I also coded.

Originally Posted by Predseda

Sorry, but the thread title made me laugh

It sure is misleading, but at least it's posted on an Amiga forum.

[Leffmann]

I think the differences you're seeing are the results of an incorrect implementation of software floating point math in the BASIC interpreter he used.

If you trust his math to be correct, then just rounding each value to the nearest integer will probably give a better result than his original table, but just using a pre-computed table might save lots of space on certain platforms, since you want it to be really portable and as small as possible.

BTW, was 2.3 the one everyone used back in the day? What things are better in the 3.x versions? More player features, or just a bigger and better editor?

[StingRay]

Originally Posted by Leffmann

BTW, was 2.3 the one everyone used back in the day? What things are better in the 3.x versions? More player features, or just a bigger and better editor?

3.xx series used a hires screen so more things were visible on screen. It also had some better sampling features and some other improvements but 2.xx series was the most stable. AFAIR most musicians back then didn't like the 3.xx series much back then and kept using 2.xx versions.

[mc6809e]

I think I'd trust your C code (along with the suggestion that 0.5 be added before the INT cast) before I'd trust a basic interpreter.

[demolition]

When multiplying floats in a loop, round off errors can quickly become significant.

To test your results, I put this into MATLAB:

f = zeros(1,36);
for i = 1:36
    f(i) = 2 ^ ((i - 1) / 12);
end

round(856 ./ f)

The output gave exactly the same numbers as your C implementation.

[8bitbubsy]

Ok, so I assume it's an error in the BASIC interpreter's floating point that Lars (?) used... Or maybe PERIODTA.BAS actually generates a more correct table, like the C code, but that he edited the "errors" by hand to match the older tables found in Ultimate Soundtracker.
Ultimate Soundtracker's table is also wrong (and identic to the ProTracker one, but without finetune shifting), which dates back from 1987 (!):

	dc.w	856,808,762,720,678,640
	dc.w	604,570,538,508,480,453
	dc.w	428,404,381,360,339,320
	dc.w	302,285,269,254,240,226  
	dc.w	214,202,190,180,170,160
	dc.w	151,143,135,127,120,113
	dc.w	000

Maybe I can get in touch with Karsten Obarski and ask him some questions. He has by big chance forgotten all of this though, but it's worth a shot.
Anyways, the table *must* be like the one used in ProTracker, because I am looking up and comparing values, to get an index (which is the note ). These hard-coded period values are inside the MOD pattern data as well. With the "correct" table, I get wrong notes (not by a cent, but by a lot!).

[demolition]

Maybe you could use some kind of simple compression scheme on the original numbers?

You could also just store the difference from one number to the next. Then you could get away with using bytes for storage (and a word for the first value in every tuning), ending up with 555 bytes. You could even pack the data as no values would need more than 6 bits, ending up with 420 bytes + the space needed for a very simple unpacker.

[thomas]

Originally Posted by 8bitbubsy

Ok, so I assume it's an error in the BASIC interpreter's floating point that Lars (?) used... Or maybe PERIODTA.BAS actually generates a more correct table, like the C code, but that he edited the "errors" by hand to match the older tables found in Ultimate Soundtracker.
Ultimate Soundtracker's table is also wrong (and identic to the ProTracker one, but without finetune shifting), which dates back from 1987 (!):

	dc.w	856,808,762,720,678,640
	dc.w	604,570,538,508,480,453
	dc.w	428,404,381,360,339,320
	dc.w	302,285,269,254,240,226  
	dc.w	214,202,190,180,170,160
	dc.w	151,143,135,127,120,113
	dc.w	000

Maybe I can get in touch with Karsten Obarski and ask him some questions. He has by big chance forgotten all of this though, but it's worth a shot.
Anyways, the table *must* be like the one used in ProTracker, because I am looking up and comparing values, to get an index (which is the note ). These hard-coded period values are inside the MOD pattern data as well. With the "correct" table, I get wrong notes (not by a cent, but by a lot!).

Perhaps this table cannot be calculated. Perhaps the creator of this table used a tuning fork or a piano or an oscilloscope to calibrate each single value while his Amiga played a tone. This would include hardware tolerances which cannot be described in a mathematic formula.

[robinsonb5]

Has anyone actually tried running the BASIC programme in AmigaBASIC?

Looks like a great project, by the way - I've downloaded it and may well port it to run on one of my FPGA projects in due course.

[demolition]

Originally Posted by robinsonb5

Has anyone actually tried running the BASIC programme in AmigaBASIC?

Lines 21-30 doesn't ever seem to be executed.

[robinsonb5]

Originally Posted by demolition

Lines 21-30 doesn't ever seem to be executed.

Hmmm - so I see - but removing line 20 should fix that.

[mc6809e]

It's the algorithm itself that is the problem. Like demolition pointed out, the fr=fr*a in the loop is liable to introduce large errors. The frequency really needs to be recomputed each step by computing the appropriate power of the 12th root of 2 and multiplying that with the base frequency.

I ran this instead on a TRS-80 Color Computer emulator.

10 NTSC = 3579545
20 PAL = 3546895
30 LGTH = 16
40 BASEFREQ = 258.9731
50 FOR I = 0 to 35
60 FREQ = BASEFREQ*(2↑(I/12))
70 D1 = INT(NTSC/FREQ/LGTH + 0.5)
80 D2 = INT(PAL/FREQ/LGTH + 0.5)
90 PRINT D1;D2
100 NEXT I

It produced the same values as your C code. Most likely the long 64-bit (or possibly 80-bit during calculation) doubles in the C program are large enough to hide the errors introduced by the algorithm. The old algorithm running on the basic interpreter produced wildly different values.

One thing that hasn't been discussed is temperament. The old values might might actually sound better for certain music and it's possible they were chosen for that reason.

[demolition]

Originally Posted by mc6809e

One thing that hasn't been discussed is temperament. The old values might might actually sound better for certain music and it's possible they were chosen for that reason.

Since the music was created in Protracker using its 'inaccurate' table, if you then play it back with objectively more correct values, it will sound wrong since it was not created for that.

It could be temperament adjustments, but then you'd have to choose a base as it would sound wrong if you then made music with another base and it seems weird if Protracker would be designed mainly for music in A for example.

[mc6809e]

Originally Posted by demolition

Since the music was created in Protracker using its 'inaccurate' table, if you then play it back with objectively more correct values, it will sound wrong since it was not created for that.

It could be temperament adjustments, but then you'd have to choose a base as it would sound wrong if you then made music with another base and it seems weird if Protracker would be designed mainly for music in A for example.

Well I was thinking it might have been done to mask obvious beats when two notes are played simultaneously. The notes of chords in even temperament aren't perfect ratios of one another so beats are inevitable. Other temperaments are sometimes used to reduce these beats.

The correct temperament depends on the objective, of course.

EDIT:

Of course the computed values aren't even temperament either since they're rational approximations and 2↑(1/12) is not rational. I wonder what rounding/approximating does to the relationships between notes when chords are played. Maybe there should be an Amiga temperament!

[8bitbubsy]

Is it possible to do some BASIC (interpreter with errors in the floating-point calcs) work to find some of the "inaccurate" factor/variable values, and use that in the C calculation instead? Using inaccurate constants to simulate the process. It will of course not help for the dynamically non-const calculated values, but maybe there's a way around that too...

EDIT: Removed some non-sense babbling from this post. I was wrong.

EDIT2: Now I managed to make a "differentiate" table, halving the size of the period table section in the EXE. The generated table is 100% accurate with the original one, I compared each and every value in a loop.
The reason I have 14 extras zeroes at the end of the table is to prevent access violation on arpeggios on high notes with +7 finetuned samples. PT ignored this, you can imagine what kind of weird glitches you'd get.

New version of MOD player: http://pastebin.com/5dPR3yiL

[Photon]

Here's the code from 1Klång which calculates it realtime to save 10 bytes (i.e. no table at all). D1 contains note 0-11 << 16 + octave 0-n on entry. I might use this in P61_Init actually to save some horrible includes and compile options

_baseC	=54229
_baseA	=64489
_octBase=_baseC

	move.w #_octBase,d0
	swap d1
	bra.s .jmpin
.octl2:
	mulu #61858,d0			;exponential: 1/(2^(1/12))*65536
	swap d0
.jmpin:	DBF d1,.octl2
	swap d1				;!hi word NOT clear. d1=octave

	addq.w #4,d1			;shift 4 more (prec)
	lsr.w d1,d0			;shift down by #octaves.
	move.w d0,_prds(a2)		;prd