extended adf - what i read is not what i wrote

[a/b]

Doing some testing with extended adf and custom mfm layouts, ran into a strange problem. If I use 22x256 sectors (per track) instead of 11x512 I get corrupt data from the adf (but the adf itself contains exactly what I wrote).

OK, details...
1. winuae: 4.4.0 64-bit

2. configs
- a4k, aga, 040, ks 3.0/3.1, 2/8mb, no jit, floppy 3.5dd tried with 100% and 800% speed
- a500 ocs, 000, ks 2.04, 0.5/0.5mb, no jit, floppy 3.5dd tried with 100% and 800% speed
- exact and max compatible settings for pretty much everything
I can provide the config files if it turns out it's not reproducible (in my case it is 100%).

3. disk stuff
Nothing weird here except for custom layout, two versions: 512b and 256b sectors.
- adkcon: mfm, fast (2usec), sync for reads (4489), for writes 0ns precomp for tracks 0-80 and 140ns for 81+.
- dsklen is $1900 for reads, and $1b00 for writes (with a more than large enough $aaaa gap at the start of the buffer, and a single $aaaa at the end due to last 3 bits being lost).
- Using 2 diferent routines: trackdisk.device with TD_RAWREAD and TD_RAWWRITE, and a hardware banger, the same outcome in both cases.
All data mfm encoded start-to-end with 2 algorithms: rom (longwords with top bit correction) and winuae (disk.c, mfmcode()). Resulting data compared to make sure it matches 100%, and it does.

4. testing
Custom disk file (extended adf), created by winuae and completely empty.
I only check the first 8 bytes (1 data longword) after each syncword, for every sector in the written buffer (that is, I go through data I wrote and search for each sector start in data I read).
Codewise, it's all simple and the only difference is pretty much how many times is each dbf loop being executed.
Scenario 1: Write a track0 with 512b sectors, read it back, compare data. No problem. Do that for all 160 tracks, several times, no problem.
Scenario 2: Write a track0 with 256b sectors, read it back, compare data. Every time I get data corruption right after a syncword.

Here is a concrete example (all relevant files are in the attachment).
Track writen and then read, received data is as follows:
sync (manually inserted), sector20
sync, sector21
$aaaa (zero at the end, to counter lost bits)
gap
sync (track start)
sync, sector0
sync, sector1
sync, sector2
sync, and then corrupt data right after

	DC.W	$92A4,$4489,$2894,$9522,$5254,$9555,$48A2,$2AA2	; S3
; adf:		 92a4  4489  1251  292a  44a4  a92a  aa91  4455
; written:	      $4489,$1251,$292A,$44A4,$A92A,$AA91,$4455,$4545	; S3

I get $2894 right after a syncword and there isn't a single occurence of $2894 in neither data I wrote nor in adf hexdump. Both written and adf data is equal, $1251. And the rest is corrupted as well...
This is 100% reproducible for me.

Attached files:
- track.bin 5.5kb of random data that is then mfm encoded
- track_256.bin data mfm encoded as 22x256b sectors
- track_512.bin data mfm encoded as 11x512b sectors
- custom_TEST.adf Extended adf file (header, first track, the rest is all 0).
- mfm_written.txt Hexdump of track_256.bin.
- mfm_adf.txt Hexdump of track0 in adf.
- mfm_read.txt Hexdump of read buffer, look for "S3".
- mfm_test.S Test program to reproduce the problem, needs asm-one 1.20+ or compatible, and ks2.0+ (trackdisk.device version). Interactive version, to either run within asm-one or its debugger.
Hexdumps have comments for easier navigation.
In test program, set SMALL_SECTORS to either 0 or 1 to activate 512 or 256 bytes sectors, respectively. The rest is automatic. On completition, if d0 is 0 there were no problems, and -1 is fail due to corrupt data. Any other value means trackdisk error (never had any).

[a/b]

Oopsie, this was meant for support.WinUAE. Can admin move it please?

[ross]

Could you try writing a pair of sync words at the beginning of the sectors (not just at the beginning of the track) to see if it makes any difference for WinUAE?
[I have often wondered why it is practically always used two]

[Toni Wilen]

Are you sure there is no another sync word "hidden" in stream data? (Visible only after shifting data) Sync word matching is active continuously.

[ross]

Originally Posted by Toni Wilen

Are you sure there is no another sync word "hidden" in stream data? (Visible only after shifting data) Sync word matching is active continuously.

Could this also explain the usual double sync? (a sync hidden because 'concatenate' from the last word of the previous sector)

It would seem strange to me because in theory it should not be a valid MFM encoding, I try to check the raw stream.

[a/b]

I was also testing with 2 syncwords per sector as well as the OS style of a zero byte ($aaaa) + 2 syncwords, the same outcome. Sorry, forgot to mention it eariler.

I'm sure it isn't something that remained from earlier writes. As I mentioned, I overwrite the entire track, with a large buffer/gap to ensure no old data remains ($1b00 which is more than what ADF stores and more than should theoretically fit, which is 6000-6800 words based on disk speed etc, and I was also testing with up to $2000 words written). Also, I did several test with an entirely fresh ADF every time, so every write was a first write. Same problem every time.

EDIT: Track bit size in adf is a multiple of 16, so there is no sync problems after you are synced for the first time.. For example, I have 2 sync before the first sector and both are always present because you don't get out of sync and they are not consumed. There should be no hidden 4489 sync within data (see info about 2 methods of encoding in the original post).

[ross]

Ok there is a 7 bit gap on read stream, I keep investigating ...

EDIT: and not too casually the 7 bit pattern are 0001001
The previous bit is 1 so it make 89

EDIT2: and yes! before there are 01000100 (44), so you have a duplicated sync that shift the stream 7 bit

EDIT3: visually
1001001010100100 0100010010001001 0001001001010001
1001001010100100 0100010010001001 0001001001010001

EDIT4: so the section at the end is right

[a/b]

OK, I see the sequence.
Incorrectly encoded? MFM broken? I wasn't expecting anything like that, isn't MFM designed in such a way so that wouldn't be possible?

[ross]

Originally Posted by a/b

OK, I see the sequence.
Incorrectly encoded? MFM broken? I wasn't expecting anything like that, isn't MFM designed in such a way so that wouldn't be possible?

The MFM sequence is right, the encoder is 'broken'.
You are not free after a sync word, you need a proper sequence.

[I came up with a simple explanation for the double sync: since encoding/decoding is possible for longword then everything stays aligned]

[a/b]

OK, I was thinking about the same thing, I understand the problem now. For example, the $ff version byte (top byte in the 4-byte header) the OS is using that always comes right after 4489... $ff as a version doesn't make sense, but its main purpose is to prevent a subsequent hidden 4489.
Problem solved, thanks!

[ross]

Originally Posted by a/b

OK, I was thinking about the same thing, I understand the problem now. For example, the $ff version byte (top byte in the 4-byte header) the OS is using that always comes right after 4489... $ff as a version doesn't make sense, but its main purpose is to prevent a subsequent hidden 4489.
Problem solved, thanks!