English Amiga Board


Go Back   English Amiga Board > Coders > Coders. Asm / Hardware

 
 
Thread Tools
Old 05 June 2017, 11:53   #1
Paradroid
Rock Lobster
 
Join Date: Nov 2012
Location: Macclesfield
Age: 46
Posts: 39
Track/Sector Loader

All the trackmos I've made have used loaders based on those by other people, but for my next production I wanted to write my own and maybe solve some of the issues I've had with loaders in the past. One of those issues is I've never been able to run my demos on a real amiga via a usb floppy emulator (http://lotharek.pl/product.php?pid=113), which would be super useful during development, but after writing my own loader I found it also failed. It didn't take long to figure out that it was due to the individual sectors sometimes not being exactly where I thought they would be, as if their content has been shifted a random number of words, but considering how pretty much every none amiga-dos demo/game I've tried has also failed to run via this emulator I'm wondering if it's the emulator itself that is bugged/broken.

This is my first attempt at a loader so it's pretty basic in how it works, I just read 12800 bytes, find the first sector sync word(s) and then decode the sectors one at a time.

In winuae, and on my A500 running from a real disk, once I've found the first sector I can just add a fixed offset of 1088 bytes to get to the next one until I reach the gap and have to search past it for the remaining sectors. Is this expected to work on all functioning hardware, or should I be searching for the sync words even for the consecutive sectors? <-- this is my main question of the post as I found the documentation very unclear about this!

I added that extra sync search to my code and it helped a little with the emulator, but sometimes the sector data has shifted so far that the markers don't exist and not all the sector data is there. I got around this by reading a track over and over again until it managed to find all 11 sectors, but it can take a long time, which I'm guessing is what the OS file loader is doing as it can load via the emulator too with the same long pauses.

Has anyone else got one of these emulators and having no problems with track loaders? Or can a real floppy drive act like this, although less extreme, under normal conditions?

Thought it worth asking as I'm not sure how much effort to put into this when the emulator might just be borked to begin with :-/
Paradroid is offline  
Old 05 June 2017, 12:38   #2
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,970
It sounds like emulator creates too big gap. Compare number of words between end of last sector and start of first sector (how many words until 4489 is found again) between real drive (or UAE) and floppy emulator.

544 words (1088 bytes) is correct (except between last and first sector).

Does it work if you read more data? (Larger DSKLEN value). This would confirm too large gap.
Toni Wilen is online now  
Old 05 June 2017, 12:58   #3
a/b
Registered User

 
Join Date: Jun 2016
Location: europe
Posts: 190
Checking my 20+ years old stuff...
I was searching for a sync word after each sector, don't remember whether to play it safe or it simply didn't work in some cases without it. But the common sense would be to resync/search after each sector, performance hit is almost non-existant.
Keep in mind that there could be more than one consecutive sync word.
a/b is offline  
Old 05 June 2017, 18:51   #4
Paradroid
Rock Lobster
 
Join Date: Nov 2012
Location: Macclesfield
Age: 46
Posts: 39
Thanks for your answers. It sounds like my loader is working as it should for real hardware, and it's the emulator itself that is the issue, which would explain why so much other software is having issues too.

Quote:
Originally Posted by Toni Wilen View Post
It sounds like emulator creates too big gap.
If only that was the case... Every test I've done on real hardware results in the first word read being the second sync word at the start of a sector (although I do search for it and any repeats, just in case) and seeking 1088 byte forward from this will get me to the same sync word in the next sector (or the gap, but I'll ignore that for the purpose of this discussion), and looking back from here I can see the other 4489 and AAAA, AAAA.

With the emulator the first sector is always fine, exactly like with a real disk, but when I examine the following sectors the sync words aren't always there. Sometimes they might be located a word or two earlier, and sometimes it looks like it has shifted far enough back that it isn't even there (i.e., it's as if the read from the disk happened too late). The data doesn't look like random trash, it just seems to be in the wrong place, so maybe some sort of timing issue.

I probably won't get a real idea of what's going on until I've knocked something together to dump the mfm data back to a file (I'm currently using an action replay cart to examine by eye), allowing me to compare it with what should be there, but it'll be a while before I get around to that as I've got more interesting things to be coding in my spare time.

If anyone else has one of those emulators I'm keen to hear if it working for you or not when it's used with track loading demos/games. It could be the case that mine is broken, or I've got an issue with the software on the PC side.

Thanks
Paradroid is offline  
Old 05 June 2017, 23:49   #5
Kalms
Registered User
 
Join Date: Nov 2006
Location: Stockholm, Sweden
Posts: 226
Check out http://martin.blom.org/tmp/Amiga/hackdisk202/hackdisk.s, there is this section in there:

Code:
;This routine is trickier than it appears. The trick is that we must NOT
;assume a $4489 at the beginning of our buffer. This phenomenon occurs when
;the DMA starts in the middle of the first sync word. The second sync word
;is thrown away by the hardware. It sounds exotic, but it actually happens
;quite often!
Perhaps this is what is happening to you when booting from the emulator, but this happens to not be happening for you when you are testing off of a real floppy due to different timing there?

(My guess is that many other homemade loaders also have problems handling this situation, and they are effectively broken.)
Kalms is offline  
Old 06 June 2017, 08:44   #6
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,970
Yeah, it could be timing related, emulator being too accurate.

When I improved UAE floppy emulation, I noticed that some loaders failed if track length was 16 bit divisible and adding some random garbage bits at the gap end/start point fixed it (which happens with real floppy drives, track start/end position is not seamless). Same when stepping or starting motor: current bit position needs small (pseudo) randomization offset added.

If bit stream is already word "aligned" after gap when reading, both sync words will get written to memory and some loaders assume there is always only one and won't skip the second one. (Bad loaders that check the checksum will retry and continue normally. Except those that have broken retry logic..)

Logic in Paula is something like: if sync word is detected but bit counter is not zero = drop the sync word and reset the word bit counter.

I don't remember seeing loaders that properly handle no sync word case and expect to find one. I'd assume most of them retry if they can't find all sectors.
Toni Wilen is online now  
Old 06 June 2017, 10:15   #7
alpine9000
Registered User

 
Join Date: Mar 2016
Location: Australia
Posts: 805
I have a weird experience with my gotek using the track disk routines from Solid Gold where it will sometimes fail to save (a retry will normally work). Under emulation it will never fail.
alpine9000 is offline  
Old 06 June 2017, 13:39   #8
phx
Natteravn

phx's Avatar
 
Join Date: Nov 2009
Location: Herford / Germany
Posts: 1,690
I would also be interested in a solution. My trackdisk routines work on real hardware (not only on emulation). Only the Gotek makes problems, as Alpine9000 just indicated.
phx is offline  
Old 07 June 2017, 10:44   #9
hooverphonique
ex. demoscener "Bigmama"

 
Join Date: Jun 2012
Location: Fyn / Denmark
Posts: 1,122
Quote:
Originally Posted by Kalms View Post
Check out http://martin.blom.org/tmp/Amiga/hackdisk202/hackdisk.s, there is this section in there:

Code:
;This routine is trickier than it appears. The trick is that we must NOT
;assume a $4489 at the beginning of our buffer. This phenomenon occurs when
;the DMA starts in the middle of the first sync word. The second sync word
;is thrown away by the hardware. It sounds exotic, but it actually happens
;quite often!
Perhaps this is what is happening to you when booting from the emulator, but this happens to not be happening for you when you are testing off of a real floppy due to different timing there?

(My guess is that many other homemade loaders also have problems handling this situation, and they are effectively broken.)
As I understood the OP, the problem is not that the sync word is missing from the start of the buffer, but somewhere after the first sector (in the buffer).
hooverphonique is offline  
Old 07 June 2017, 11:41   #10
Photon
Moderator

Photon's Avatar
 
Join Date: Nov 2004
Location: Eksjö / Sweden
Posts: 4,819
There could be an issue with writing tracks on floppy emu, if so you could see if HxC firmware fixes it or vice versa. If it's a game it could be the write routine?

Re. question in OP: As long as you know you have a first proper sector it should be fine - an old sync word may have been left in the track write gap, and you don't know where the gap is in the data beforehand.

I might do a writeup for more detail, and my loader is on Coppershade if someone wants to have a look.
Photon is offline  
Old 07 June 2017, 13:41   #11
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 45
Posts: 23,970
Remains of old data can only happen (in real world) if writing is too short or aborted. Normal writes are always longer than track length to guarantee full overwrite. (reason why gap exists)

ADF images don't support any low level special cases so any properly working emulator should create normal looking low level structure when source is adf.
Toni Wilen is online now  
Old 07 June 2017, 15:56   #12
Paradroid
Rock Lobster
 
Join Date: Nov 2012
Location: Macclesfield
Age: 46
Posts: 39
Quote:
Originally Posted by hooverphonique View Post
As I understood the OP, the problem is not that the sync word is missing from the start of the buffer, but somewhere after the first sector (in the buffer).
Correct, although it was only a few extra instructions to handle the start of buffer special case Kalms mentioned, so I'm now handling that too. It hasn't happened in any of my tests since, but if it ever does it will save me having to load the track again

As for getting closer to figuring out what's going on mid-track, it'll have to wait for a few days as work is getting busy due to E3, urgh! My plan is to write bad raw tracks back to a real disk (the emu is read only) and then transfer that back to my PC for easier examination and comparison with good raw data.
Paradroid is offline  
Old 07 June 2017, 17:49   #13
Kalms
Registered User
 
Join Date: Nov 2006
Location: Stockholm, Sweden
Posts: 226
Quote:
Originally Posted by hooverphonique View Post
As I understood the OP, the problem is not that the sync word is missing from the start of the buffer, but somewhere after the first sector (in the buffer).
Yep, but: if you 'miss' the nonexistent sync word at the start of the buffer, you will skip over 1kb or so of real data, and then the remaining 12800-1024 bytes may not contain all 11 sectors any more (you need to read 12800+1kb+some extra bytes for that guarantee).
Kalms is offline  
Old 07 June 2017, 19:42   #14
a/b
Registered User

 
Join Date: Jun 2016
Location: europe
Posts: 190
Lots of interesting info on missing sync words. I'll have to add a few TODOs to my sources in case if/when I get back to them some day ;p.

Anyway, just thinking what I would do...
If there's no lead sync word (within the first few bytes) assume the actual data starts at the start of buffer, and check sector header checksum. If it matches, all good and we are in sync, otherwise either reload the track or keep searching for sync word until there isn't enough data to decode all 11 sectors.
a/b is offline  
Old 08 June 2017, 08:59   #15
Photon
Moderator

Photon's Avatar
 
Join Date: Nov 2004
Location: Eksjö / Sweden
Posts: 4,819
Quote:
Originally Posted by Toni Wilen View Post
Remains of old data can only happen (in real world) if writing is too short or aborted. Normal writes are always longer than track length to guarantee full overwrite. (reason why gap exists)

ADF images don't support any low level special cases so any properly working emulator should create normal looking low level structure when source is adf.
Yes. Possibly a game could save a too short track and sometimes get away with it on a real floppy drive. But I wonder how a floppy emu could make sense of it if such a thing happened? Reading the track beforehand and replace only the sectors contained in the write? Which would still not be correct/what one wanted. Anyway, that's speculation.
Photon is offline  
Old 08 June 2017, 19:12   #16
Paradroid
Rock Lobster
 
Join Date: Nov 2012
Location: Macclesfield
Age: 46
Posts: 39
I had a play with this again over lunch and this evening and have to admit my belief that the sector data was being offset by the emu was probably a mistake caused by me incorrectly parsing the data by eye. I've now written some bad raw mfm tracks back to a real disk and transferred them to my PC, then used a tool I wrote to compare them with known good tracks I'd ripped from winuae. I found the differences to have always been just a bad bit or two here and there - everything else was where it should be. I also confirmed this visually by loading the same image over and over again into an on-screen buffer and could see bad bytes appearing before being overwritten again by the retry attempts.

Thanks for all the input, it's helped me get a few steps closer to having a bulletproof loader. It's a shame my emu isn't as reliable (due to design or fault, I'm not sure) as the real drives, but I can live with it for development as waiting for retries to succeed is still a lot quicker than copying the image to a real disk.
Paradroid is offline  
Old 16 April 2020, 22:10   #17
Jeff_HxC2001
Registered User
 
Join Date: Sep 2008
Location: Paris / France
Posts: 640
Quote:
Originally Posted by Paradroid View Post
I had a play with this again over lunch and this evening and have to admit my belief that the sector data was being offset by the emu was probably a mistake caused by me incorrectly parsing the data by eye. I've now written some bad raw mfm tracks back to a real disk and transferred them to my PC, then used a tool I wrote to compare them with known good tracks I'd ripped from winuae. I found the differences to have always been just a bad bit or two here and there - everything else was where it should be. I also confirmed this visually by loading the same image over and over again into an on-screen buffer and could see bad bytes appearing before being overwritten again by the retry attempts.

Thanks for all the input, it's helped me get a few steps closer to having a bulletproof loader. It's a shame my emu isn't as reliable (due to design or fault, I'm not sure) as the real drives, but I can live with it for development as waiting for retries to succeed is still a lot quicker than copying the image to a real disk.

I am quite late i suppose (), but you probably have a buffer under-run issue at the PC side. Check the "sync lost" counter in the setting window. If this value is increasing, you may need to increase the floppy emulator buffer size. You also may need to increase the hxcfloppyemulator process priority.
Jeff_HxC2001 is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
ASM: Hardware Track Loader Vortex Coders. Tutorials 172 18 May 2013 20:44
Ripping the RNC sector loader... h0ffman Coders. General 13 07 September 2011 23:00
Can't transfer Supaplex cause of CSL track loader ! Vollldo support.Games 4 12 March 2011 21:51
Track & sector editor needed kipper2k request.Apps 9 01 May 2009 02:49
mfm/custom regs/track loader snyp Coders. General 9 06 June 2006 19:42

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 22:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.
Page generated in 0.09078 seconds with 15 queries