Could WinUAE make use of fdrawcmd.sys?

Hello everyone

I was trawling the net just now and totally at random fell upon this floppy driver http://simonowen.com/fdrawcmd/ basically it allows low level access to the floppy controller under xp/xp64.

As some of you may know the catweasel board is currently out of production due to germanys new enviromental laws (the catweasel pcb used some now banned substances) so until that situation can be solved no more catweasel

If someone here with uber programming skills has some spare time would they cast an eye over and see if it could be used to generate adf files or even allow some form of native access under WinUAE?

There was an old dos prog which allowed standard amiga ofs disks to be read on pc's using twin floppy drives and some weird timing hack so it may be possible but we all know how weird (and wonderfull) the amiga floppy system was.

[thomas]

I was trawling the net just now and totally at random fell upon this floppy driver http://simonowen.com/fdrawcmd/ basically it allows low level access to the floppy controller under xp/xp64.

Even if someone would add support for this to WinUAE, it would still not be able to read Amiga floppy disks. It is the floppy controller which has the PC format hardcoded in its firmware. There is no software whatsoever which could change this. You need a custom floppy controller (e.g. Catweasel) to read Amiga floppy disks.

I am sure, if it was a software problem, UAE would have got support for Amiga disks long ago, even without a special DLL. But it is not, it is a hardware problem.

There was an old dos prog which allowed standard amiga ofs disks to be read on pc's using twin floppy drives and some weird timing hack so it may be possible but we all know how weird (and wonderfull) the amiga floppy system was.

It's Disk2FDI and it's still around. You should read its docs, they describe very detailed why a PC floppy controller cannot read Amiga disks and what this program does to fool the controller. http://www.oldskool.org/disk2fdi/

As some of you may know the catweasel board is currently out of production due to germanys new enviromental laws (the catweasel pcb used some now banned substances) so until that situation can be solved no more catweasel

Individual Computers has found a circumvention for the law problem: http://ami.ga/news/news110_e.htm

Ta for the reply.

I did try Disk2FDI a long time back I had just forgotten the name of it I realise that it was an extreme hack where they basically twitched one of the drives to generate stepping signals while reading in the other (I never got it to work though). While not as flexable as the amiga controller the pc controller can be quite abused both fuji and sony brought out smart media/memory stick to floppy adapters which basically 'fired' the info at the floppy head (god knows what they were smoking when they came up with that idea) again that was an extreme hack which barely worked, but still the amiga floppy remains unbeatable.

Its good to see that the catweasel could be back in production one day as that still offers the best way for folk without an amiga to read the disks and such I will get round to buying one one day

[Dic_Ray]

I used the Disk2FDI prog to create *.adf of all my old amiga disks. It works great. More instructions will follow tommorow.

It would be really great, if fdrawcmd.sys would allow to use Amiga formatted disks under Windows 2000/XP.

Is there a possibility to make this wish come true?

[Adderly]

So does fdrawcmd.sys really works? Can you read real Amiga disks with it?

[Dic_Ray]

At least I guess I could create an .adf file under Windows XP. Till now with Disk2FDI, this was only possible under pure DOS or Windows 9x. But I didn't have an amigaformatted floppy disk.

So could someone else try it again together with the fdrawcmd.sys?

Under http://www.oldskool.org/disk2fdi/support.html , there is also a stand-alone Windows 95, 98 and ME frontend for Disk2FDI. The ADF creation of ADFOpus from http://adfopus.sourceforge.net/ should also work even under Windows NT,2k,XP.

Please give me a feedback!

Kind regards

fdrawcmd.sys works at the PC controller level, so it can read/write almost any PC protected disks, but won't work for Amiga disks... yet.

I'm currently experimenting with the same 2-drive technique as Disk2FDI (under W2K/XP/2003). I can already raw-read data+clock bits from double-density disks, which should be enough to read most Amiga disks. This is still at a very early stage though, and lacks the bitstream decoding needed to make sense of the track and create a working disk image from it. There'd be nothing to stop someone writing their own application to read the raw track using my driver, and decode it to something meaningful...

IIRC, the Amiga track decoding is all done by software anyway? In which case the raw track data might just what WinUAE needs

Si

[Toni Wilen]

Originally Posted by obo

fdrawcmd.sys works at the PC controller level, so it can read/write almost any PC protected disks, but won't work for Amiga disks... yet.

I'm currently experimenting with the same 2-drive technique as Disk2FDI (under W2K/XP/2003). I can already raw-read data+clock bits from double-density disks, which should be enough to read most Amiga disks. This is still at a very early stage though, and lacks the bitstream decoding needed to make sense of the track and create a working disk image from it. There'd be nothing to stop someone writing their own application to read the raw track using my driver, and decode it to something meaningful...

IIRC, the Amiga track decoding is all done by software anyway? In which case the raw track data might just what WinUAE needs

Si

Good job and you even have 64-bit version available

Long enough raw track starting from random position and ending to random position is basically everything UAE needs but preferably streamed so there is no need to wait for track and cause emulation slowdowns (nice extra would be timing info for some copy protections)

Unfortunately fully real-time and transparent disk access may not be possible. Some programs (usually games and demos) expect new data quite quickly after starting disk DMA (maybe in 1/3th of disk revolution or so), it may be good idea to have multiple "sector start markers" in second disk. And some programs don't use DMA but simple read disk byte register in tight loop. Some pre-buffering may be needed but it will cause slowdowns..

btw, doing decoding and adf-conversion is easy, just rip decoding code from uae or my rawread-utility (ADDED) I can do raw->adf-conversion utility quickly, I only need new enough driver (64-bit) and pointers to examples.

Originally Posted by Toni Wilen

Long enough raw track starting from random position and ending to random position is basically everything UAE needs

The 16K read gives 8K of data+clock bits, which gives plenty of overlap on a normal ~6K double-density track. The two drives aren't synced in any way, so the starting point is relatively random. I guess that's fine for the Amiga, but complicates things for lesser controllers that use index-synced formatting.

but preferably streamed so there is no need to wait for track and cause emulation slowdowns

The technique relies on switching the drive select to the 2nd drive after initiating a long sector read on the first. It could take up to the full 200ms rotation time to start reading the dummy sector (not including spin-up), and with the OS scatter/gather DMA buffering it'll be a further 256ms before the data is visible in your buffer. I think that rules out live use in WinUAE, unless there's another way to hold off the program using the drive. It's much easier with most higher-level controllers as you can indicate you're busy until you're done reading the track.

(nice extra would be timing info for some copy protections)

The data is read in DMA mode, so there isn't much timing information to provide. I could probably give a reasonable indication of an index-relative starting position and overall track length, but a seamless wrap will need to be determined by something that understands the track contents better.

btw, doing decoding and adf-conversion is easy, just rip decoding code from uae or my rawread-utility (ADDED) I can do raw->adf-conversion utility quickly, I only need new enough driver (64-bit) and pointers to examples.

Great! I'll package up a test driver and sample utility (+source) over the weekend, which should hopefully be all you need as input to your utility.

Si

[Toni Wilen]

Originally Posted by obo

I'll package up a test driver and sample utility (+source) over the weekend, which should hopefully be all you need as input to your utility.

Great, I can't wait

Quote:
Originally Posted by obo
I'll package up a test driver and sample utility (+source) over the weekend, which should hopefully be all you need as input to your utility.


Of course, I'm also interested! ;-)

Vincent.

The test program is about 95% done, but there's still a drive sync issue to improve before I upload it...

I'm doing the raw read, then checking for the longest run length in the data. Any run of 4 zero bits is treated as a sync failure (write splice or other problem), and ends the current run. If it's not long enough I'm retrying the read in the hope I get a better copy next time.

With Amiga tracks written in a single pass, as long as my raw read begins near the original track start, I'll get a long enough run to accept. With the drives spinning at approximately the same speed, after a successful read it's likely that the next track will also read successfully. However, when the slight variations in drive speed means the track isn't positioned correct to read a full track, it may take while before they're back to being in a suitable sync.

To help get Toni started with processing the data, I've dumped a single Amiga disk. The output is a dump of 80 cyls x 2 head x 16K per track, with each track starting with a run of at least 12400 data+clock bytes. The file is ordered Cyl 0 Head 0, Cyl 0 Head 1, Cyl 1 Head 0, ... You can get the data file from: http://obo.homeip.net/amidisk.zip (323K). I'll need another day or so to try and improve the test program itself, but will release it then even if I haven't.

I've noticed that Disk2FDI (Hi Vincent!) seems to have moved towards using a floppy->parallel cable for dumps. Was the 2-drive method not good/reliable enough? Or does it work well enough for Amiga disks, so it's still worth doing a Windows version? (the parallel method is a bit more timing critical, and may not be as suitable for use under a multi-tasking OS).

Si

Originally Posted by obo

I've noticed that Disk2FDI (Hi Vincent!) seems to have moved towards using a floppy->parallel cable for dumps. Was the 2-drive method not good/reliable enough?

Indeed (hi Obo! ;-) ), because the FDC interprets the data, and always decodes them as MFM, it is very difficult to recreate a good image of a track, in a general way.

Still, I'm working on a version that should work with 2 floppy drives, but will create images of a lesser quality, and probably won't take weak bits into account, since it seems it would be nearly impossible.

The other problem is that you can't read disks with strange bitrates (for example Mac GCR disks), so it's necessarily limited.

Or does it work well enough for Amiga disks, so it's still worth doing a Windows version?

Amiga disks are clearly the easiest to read (apart from sectors from PC disks of course! ;-) ), so it's probably a good idea to try to implement this feature for WinUAE. I would have always liked to see such a feature, but I don't know enough about Windows programming to do it.

There is something I can recommend though: you should not use the "double rate" (i.e. high-density) reading method, because it is far less compatible with many FDCs than reading with the normal (double-density) rate. Or maybe you can let the user choose.

Of course, when reading at double-density rate, you will read either the actual data, or the clock bits, which are not very useful in this case. That would mean only supporting standard Amiga sectors, since an MFM $4489 synchronization word (or any other sync word) cannot be reliably detected. But it is safe enough to assume a $A1A1 followed by an Amiga header with a valid CRC is really an Amiga sector (method used by Disk2FDI to dump to ADF).

By letting the user choose, you could have a "compatible" but "standard disks only" mode, and a "less compatible" but "some non-standard/protected disks support" mode.

(the parallel method is a bit more timing critical, and may not be as suitable for use under a multi-tasking OS).

Yes, I guess so too. A catweasel is probably a better choice in this case, since dedicated hardware takes care of the task.

BTW, do you think it would be possible to interface your driver with a DOS (Windows DOS box I mean) program? For example by calling a special INT number that would return data read with your driver at an address specified by DS:SI (that would be translated by your driver of course). That would easily allow at least a partial Disk2FDI for Windows version, which I'm sure many people would appreciate.

Vincent.

Originally Posted by ApH

Still, I'm working on a version that should work with 2 floppy drives, but will create images of a lesser quality, and probably won't take weak bits into account, since it seems it would be nearly impossible.

Covering standard Amiga disks as well as some difficult PC disks is still of great use. I started looking at the 2-drive method to help with the index blindness problem, which prevents even normal sectors being seen/read if they're too close to the index hole. I ran into this forum thread by accident - it's been a while since I was a regular Amiga user!

it's probably a good idea to try to implement this feature for WinUAE. I would have always liked to see such a feature, but I don't know enough about Windows programming to do it.

I'm still wondering whether it's possible, but don't know enough about the Amiga floppy hardware to know for sure. Presumably there is some busy-waiting needed during seeks and drive spin-up, which could be artificially extended to give Windows a chance to prepare the track contents? From what Toni said it sounds like the data reading expects immediate bitstream sampling, so there's no chance to stall there.

There is something I can recommend though: you should not use the "double rate" (i.e. high-density) reading method, because it is far less compatible with many FDCs than reading with the normal (double-density) rate. Or maybe you can let the user choose.

I'm currently using the double-rate method, which works very well on the 2 machines I've tried it on. I did wonder about the size=8 needed for the track reading, which is beyond the documented maximum of 7. Some controllers will definitely clip it to 3 bits - I've added a quick check for that in my utility. Using the safer size=7 makes it much less likely that a complete stream will be found (as I'm sure you remember!).

I haven't yet tried the normal rate reading, but I've left the option open in the driver. I'll be sure to confirm that works before I release a public update, just in case there's a problem.

Of course, when reading at double-density rate, you will read either the actual data, or the clock bits, which are not very useful in this case. That would mean only supporting standard Amiga sectors, since an MFM $4489 synchronization word (or any other sync word) cannot be reliably detected.

That'll be good enough for solving the index blindness on problem disks (certain retro platforms I've run into so far). If the clock bitstream is complete, I guess it may even be possible to match it up to suspected synch words in the data stream, for further track verification?

BTW, do you think it would be possible to interface your driver with a DOS (Windows DOS box I mean) program? For example by calling a special INT number that would return data read with your driver at an address specified by DS:SI (that would be translated by your driver of course).

I'm fairly certain the INT method won't work on the NT platforms, as it'll be handled entirely by the ntvdm.exe process that manages DOS/16-bit processes. It may be possible to thunk through to the Win32 API (as I've done from 16-bit Windows apps many times), but that's still not ideal. Your best option would be to extend your code to build as a 32-bit app, where things are much easier.

That would easily allow at least a partial Disk2FDI for Windows version, which I'm sure many people would appreciate.

Will the Windows functionality be freely available? My driver terms are for non-commercial use only, to ensure anything built on it is available to all. I might have to think about that...

I've got the sample utility into a reasonable alpha state to try, so I'll PM you and Toni when I'm done here.

Si

Originally Posted by obo

Covering standard Amiga disks as well as some difficult PC disks is still of great use. I started looking at the 2-drive method to help with the index blindness problem, which prevents even normal sectors being seen/read if they're too close to the index hole. I ran into this forum thread by accident - it's been a while since I was a regular Amiga user!

Ah, I didn't know about this index blindness problem. Indeed, using the 2-drive method could help, but keep in mind that reading PC disks with it is very difficult, because of the frequent write splices you mentioned, especially those occurring between a sector header and its associated sector data...

I'm still wondering whether it's possible, but don't know enough about the Amiga floppy hardware to know for sure. Presumably there is some busy-waiting needed during seeks and drive spin-up, which could be artificially extended to give Windows a chance to prepare the track contents? From what Toni said it sounds like the data reading expects immediate bitstream sampling, so there's no chance to stall there.

Yes, I think it's impossible to have a seemless integration. Either the emulation should be paused for each disk access on a track that has not yet been read, or the whole disk should be read when it is inserted, then only accessed virtually.

I'm currently using the double-rate method, which works very well on the 2 machines I've tried it on. I did wonder about the size=8 needed for the track reading, which is beyond the documented maximum of 7. Some controllers will definitely clip it to 3 bits - I've added a quick check for that in my utility. Using the safer size=7 makes it much less likely that a complete stream will be found (as I'm sure you remember!).

Some documentations indicate 8 as the maximum, but indeed I think many controllers take only 3 bits into consideration.
And yes, using 7 for high-density reading is a little too short. So that's one of the best reasons to switch back to double-density.

I haven't yet tried the normal rate reading, but I've left the option open in the driver. I'll be sure to confirm that works before I release a public update, just in case there's a problem.

Yes, the best would be to have a parameter for the read function where the density (including 300 Kbits/s) and data length can be chosen.

That'll be good enough for solving the index blindness on problem disks (certain retro platforms I've run into so far). If the clock bitstream is complete, I guess it may even be possible to match it up to suspected synch words in the data stream, for further track verification?

That's where things get very complicated. If you want to do it for some defined cases (for example assuming there are synch words in the signal), this is only slightly difficult, but doing it generally is extremely difficult, and requires additional information that would be too difficult (or impossible) to obtain under NT.

I'm fairly certain the INT method won't work on the NT platforms, as it'll be handled entirely by the ntvdm.exe process that manages DOS/16-bit processes. It may be possible to thunk through to the Win32 API (as I've done from 16-bit Windows apps many times), but that's still not ideal. Your best option would be to extend your code to build as a 32-bit app, where things are much easier.

Understood. So that means a complete rewrite in this case since Disk2FDI is 100% 8086 ASM...

Will the Windows functionality be freely available? My driver terms are for non-commercial use only, to ensure anything built on it is available to all. I might have to think about that...

I understand. It would be at least available along with the DOS Trial version, which is freely available. I could also make a stand-alone version with all references to registration removed.

Anyways, I'm not going to start doing this code rewrite until a few months from now, so we'll see how to proceed by then. If I do it, I'll probably start with ADF imaging only, since it's the easiest. Other formats require much more code because of the write splices.

Thank you for your wondeful driver!

Vincent.

Originally Posted by ApH

using the 2-drive method could help, but keep in mind that reading PC disks with it is very difficult, because of the frequent write splices you mentioned, especially those occurring between a sector header and its associated sector data...

I'm sure I'll look into that "fun" issue soon... For now, I'm concentrating on getting the core support in the driver, so it's available to anyone else wanting to have a go at it.

Yes, I think it's impossible to have a seemless integration. Either the emulation should be paused for each disk access on a track that has not yet been read, or the whole disk should be read when it is inserted, then only accessed virtually.

I have seamless integration into my own emulator (SimCoupe), which is based on the WD1772 controller like the Atari ST. It relies on the controller being able to hold BUSY for an arbitary amount of time during command execution, and the code running inside the emulator is happy to wait for it, without any drop in emulator speed. It doesn't seem quite as straight-forward for the Amiga, and may not be possible at all. Even if the head stepping could be delayed, there may be no way of knowing if the data on the new track needs to be read, until it's too late (the software could simply be stepping past it to a different track). Mmmm, sounds like disk images are still the best option!

Yes, the best would be to have a parameter for the read function where the density (including 300 Kbits/s) and data length can be chosen.

I've implemented that already, with the sample disk utility always passing 0 (500Kbs) for now. Hopefully passing 2 (250Kbps) will work for same-density reading. You can also pass the size code for the READ TRACK command, to vary the amount that is read (reducing it to 6 or 7 for for same-density reading). The final 2 parameters (currently) are the flags for the read track command (generally 0x40 for MFM) and the head to start reading from (this is the same head to sample after the DOR switch).

I also currently expect the PC-formatted drive to be positioned over the correct track for the READ TRACK. I may simplify it further and insist that track 0 is always used. That way I could check the TRACK0 line, and recalibrate back to track 0 if it's not already there.

That's where things get very complicated. If you want to do it for some defined cases (for example assuming there are synch words in the signal), this is only slightly difficult, but doing it generally is extremely difficult, and requires additional information that would be too difficult (or impossible) to obtain under NT.

The biggest FDC limitation under NT is that data transfers must be in DMA mode, as PIO is too timing critical (even with the 16-byte FIFO). The DMA monitoring techniques also introduce small variations, typically varying from bit-level differences to a few bytes on most modern machines.

Even with those limitations, there's still plenty of scope for interesting things to be done inside Windows. As well as the DOR switching, I support short-write commands for terminating data writes early. This allows ~6K of data to be written to 8K sectors, writing of sector errors, as well as some mixed-density track writing - the latter used to write 'weak sectors' for some Spectrum +3 / Amstrad CPC copy-protections.

It's all no substitute for a proper CatWeasel setup (I've got one of those too), but for a standard hardware setup it is still surprisingly flexible.

I understand. It would be at least available along with the DOS Trial version, which is freely available. I could also make a stand-alone version with all references to registration removed.

That would be great!

Thank you for your wondeful driver!

And thank you for sharing details of the ingenious 2-drive reading I've just implemented

Si

[Toni Wilen]

About Amiga disk reading:

OS will abort the read with a disk error if DMA transfer won't end quickly enough (2 disk revolutions?)

Games and demos can do anything.. Some will wait until DMA transfer is finished, some will complain about read errors, some will freeze, some will simply crash...

[Toni Wilen]

Not really about reading Amiga floppies but I just implemented new WinUAE floppy drive sound option: floppy sound from PC floppy drive (uses fdrawcmd.sys)

[alexh]

Do you need a disk in the drive for it to work?

Originally Posted by alexh

Do you need a disk in the drive for it to work?

No, it'll quite happily step an empty drive

Si