Analogue floppy disk reader project

[zero]

I've started work on a project to read floppy disks at the analogue level.

Stuff like Kryoflux uses the digital output of the drive only. I want to capture the analogue waveform, as you would see on an oscilloscope if you probed the output of the amplifiers on the drive.

The advantage of this technique is that even tricky copy protection schemes like Rob Northern CopyLock that use "weak" bits can be accurately imaged and preserved. It should also be useful for recovering data from damaged disks.

Of course the floppy drive will have to be modified so that the analogue signal can be read. Depending on the drive it might be possible to simply attach a probe to a test point or the leg of an op-amp, or otherwise add an attachment point with a bit of soldering.

I'm aiming for at least a 2msps sample rate, ideally 4msps which is the maximum that the XMEGA microcontroller I am using can capture. It has 12 bit ADCs but 8 bits is probably more than adequate for this task. There are faster ADCs available but this should be fast enough and is very cheap and easy to handle the timing.

The MCU has a DAC as well but it's only good for up to 1MHz. That is probably enough to write disks though. My understanding is you only need a 250KHz clock for standard density disks, although maybe some copy protection schemes went beyond that to create deliberately impossible to copy timing errors. In theory though it should be possible to write exact duplicates of copy protected disks.

I'm working on the hardware side now, component shortages mean prototypes can't be built yet. Interested in feedback on this idea. I know some people do analogue data recovery using custom hardware or oscilloscope waveform capture already.

[lesta_smsc]

Nice project though I assumed KF and GW can already manage all protected disks?

What would be interesting is a project whereby you can have an intermediary floppy controller that natively supports Amiga disks so you can boot original disks in emulation!

[zero]

Hmm, so it seems that KF works because the "weak" bit are actually just very short bits. Same signal level, just a signal that is right on the limit of what the Amiga FDC can read and combined with variable RPM that causes it to appear random.

So maybe I misunderstood how they were creating these weak bits.

The other issue with the KF is that it's not open source. Greaseweazle is though and seems like a decent solution.

A USB floppy controller would certainly be possible. Would need an ARM to get USB2.0 high speed support.

[Jeff_HxC2001]

Originally Posted by zero

So maybe I misunderstood how they were creating these weak bits.

Some main ways to generate weak bits :

- Unformatted surface : The drive's AGC circuit goes very quickly to its maximum gain. This generates electronic noises and finally generates random pulses on the read pin.
(This can also be done by removing a part of the magnetic surface)

- Long timing between flux reversals : Very similar to the previous process: The drive's AGC circuit goes very quickly to its maximum gain. This generates electronic noises and generates random pulses on the read pin.

- Modulated flux reversal : Modulate out of spec flux reversal to disturb the floppy controller : This generate random decoding errors. (Example : Dungeon Master : )

Some analogues captures at the oscilloscope :
[ Show youtube player ]

Video details :

Drive : YD-480 5"1/4 - 40 Tracks DD Floppy disk .

Channel 1: Preamplified head Analog signal (HA16642MP Diff in)
Channel 2: Drive Read Data output (/RDATA pin 30)

Oscilloscope synced on the index signal (Drive Pin 8).

00m00s : Track write splice. (see the random drive data output in the write splice area)
00m07s : Head moved between the disk track 0 and 1.
00m15s : Track 1 (with a write splice again).
00m23s : Head moved between the disk track 1 and 2.
00m35s : Head moved to the unformatted track 40 ( see the total random output data : the drive AGC's gain is set at its maximum value : The random data are generated directly from the AGC circuit).

[pandy71]

Had similar idea - substitute head motor control by dedicated uC (so micro-stepping will be possible - possibility to read difficult tracks), replace flywheel servo by external one so variable rotation speed floppies can be processed.
Cheapest sufficiently fast ADC should be available in modern ARM uC - go for something that support sample interleaving - alternatively some CMOS analog multiplexer (like 4051) with crude S/H (something on 4066) can be used to allow slower ADC to be used for signal capture.

Good luck with your project!

[zero]

Thanks Jeff, that's really helpful and interesting. So it looks like having an analogue capture would be of some use.

Question is if this idea will be adequate. pandy71, the part I am looking at will sample at 2MHz, but has two ADCs so I think I can do 4MHz by staggering them.

Rule of thumb is you want at least 5x the bandwidth of the signal, which for a 0.5MHz HD disk would be 2.5MHz, or 1.25MHz for a 0.25MHz DD disk. So 4MHz should be adequate. Having said that, you can just re-read the track several times and combine the results so it might not be an issue at all.

There is one other problem. As the bandwidth gets higher the analogue front end gets more complex. I was hoping that I could get away with something very simple but we will have to see. If the signal is only 500KHz then a very simple op-amp with say 5MHz+ zero gain bandwidth should be fine.

I was planning on taking the amplified signal in, but if there is an AGC then I might have to have an amplifier.

You might be surprised about ARMs, most of them have ADCs that can't even reach the 2MHz that an XMEGA one can. The main advantage is clock speed making everything easier, but my plan was to run the XMEGA at 48MHz (overclocked a bit but in my tests they are all stable at that speed) or if that fails 32MHz, which should make digital sampling stable at 24MHz or 16MHz respectively. Alternatively I could use the timers to capture transition time-stamps with a resolution of the clock speed, but that fails when you have transitions that are faster than the DMA can save the timestamps. For floppies it should not be an issue.

[Jeff_HxC2001]

10MHz seems to be the bare minimum sampling rate to have enough resolution and being able to have a usable waveform. You may need even more if you deal with HD or ED disks.

For Pauline ( [ Show youtube player ]) we just plan to use an low cost USB oscilloscope (Hantek6022BE) when the analogue dump is needed.

[zero]

I thought about using a USB scope like the Hantek or Picoscope. The goal was to produce something low cost that can be set up and used to churn out disk images.

I see Hantek offer an SDK. I have used the Picoscope one before, it was... Not great. The main issue is that the lower end models in particular had trouble streaming continuous data, IIRC. It was a few years ago that I tried it, and I ended up going the MCU/ADC route in the end.

So why do you need 10MHz sample rate?

[PurpleMelbourne]

Originally Posted by zero

The advantage of this technique is that even tricky copy protection schemes like Rob Northern CopyLock that use "weak" bits can be accurately imaged and preserved. It should also be useful for recovering data from damaged disks.

How extremely cool!
I'm working an a new A1200 motherboard with some enhancements to help FPGA developments of AGA replacement.

Part of my design is the separation of the main motherboard from the Floppy, Joystick and Mouse with room for expansion, first with STM32F for flash floppy. It would be great to incorporate your developments either directly or with provision for your designed add on module.

I was thinking there would be a small niche for disk/data recovery which Amiga could have as part of the revival campaign.

What signals would you need on a pin header?

[zero]

Originally Posted by PurpleMelbourne

How extremely cool!
I'm working an a new A1200 motherboard with some enhancements to help FPGA developments of AGA replacement.

Part of my design is the separation of the main motherboard from the Floppy, Joystick and Mouse with room for expansion, first with STM32F for flash floppy. It would be great to incorporate your developments either directly or with provision for your designed add on module.

I was thinking there would be a small niche for disk/data recovery which Amiga could have as part of the revival campaign.

What signals would you need on a pin header?

Well I was planning to just use USB to interface to the computer. You could just use SPI or something though, as long as it is reasonably fast. For this kind of data recovery application you need to either have a nice fast bus or enough RAM to cache the timing/analogue data. Some MCUs have a lot of RAM but in my design I have decided to add a small SDRAM chip (8MB).

Everyone seems to love STM32, but having used them I was less than impressed. The libraries and free IDE suck, the peripherals are not particularly great... They are cheap I suppose. If I were going ARM I'd probably go with Atmel, sorry Microchip SAM. I prefer XMEGA though.

[pandy71]

Originally Posted by zero

Rule of thumb is you want at least 5x the bandwidth of the signal, which for a 0.5MHz HD disk would be 2.5MHz, or 1.25MHz for a 0.25MHz DD disk. So 4MHz should be adequate. Having said that, you can just re-read the track several times and combine the results so it might not be an issue at all.

This rule of thumb apply to oscilloscopes when sinx/x interpolation is used for rectangular signals (so pulse will appear like pulse not sine) - for your application rule of thumbs is that signal shall be sampled twice faster than highest frequency present in signal. And because this is not 1 bit then phase accuracy is achieved trough amplitude resolution (so real 8 bit ADC offer sample period/256 phase accuracy) - this is opposite to brute force KryoFlux approach...

Originally Posted by zero

There is one other problem. As the bandwidth gets higher the analogue front end gets more complex. I was hoping that I could get away with something very simple but we will have to see. If the signal is only 500KHz then a very simple op-amp with say 5MHz+ zero gain bandwidth should be fine.


I was planning on taking the amplified signal in, but if there is an AGC then I might have to have an amplifier.

There is plenty of modern OPAMP's providing high slew rate but even ancient NE592 can be used to amplify FDD head signal.
At some point so called AFE can be used for this (like cheap optical scanner chips - they have very good ADC and usually fancy analog part but can be difficult to program.

Originally Posted by zero

You might be surprised about ARMs, most of them have ADCs that can't even reach the 2MHz that an XMEGA one can. The main advantage is clock speed making everything easier, but my plan was to run the XMEGA at 48MHz (overclocked a bit but in my tests they are all stable at that speed) or if that fails 32MHz, which should make digital sampling stable at 24MHz or 16MHz respectively. Alternatively I could use the timers to capture transition time-stamps with a resolution of the clock speed, but that fails when you have transitions that are faster than the DMA can save the timestamps. For floppies it should not be an issue.

Relatively cheap, fast uC large RAM and fast ADC (more than 12MSPS in 8 bit mode and interleaved sampling) is https://www.st.com/en/microcontrolle...2h725-735.html
Those uC have something that may be usable for this application - hardware delta sigma filters - so part of signal processing can be accelerated by HW.
Decent DAC can be build from resistors (R2R should be fast up to 15..30ns settle time if <1k R used) and some 74HC (74AC) buffer.
And overall i doubt that for FDD signal processing ADC/DAC with more than 4..6 bits is required - 8 bit is probably already overkill.

[zero]

That ST part is a beast. Would make sense to use something like that, but as a hobby thing I just don't have to time to work with it I think. Well, maybe I should look, perhaps I could just rip-off the reference design. I've used the F4 parts before and found the peripherals a bit jank, but could possibly cobble something together from the HAL libraries.

That would be an ideal system though, plenty of bandwidth and USB2.0 to offload it all to the PC in real-time. Ridiculously cheap too. It's just that there is a much higher ramp-up for knowledge and software tooling.

One other option would be something like a MAX1426. 10Msps with a parallel output so could easily sample it that fast using a timer and DMA. One thought there is that with a latch the digital signals could be sampled synchronously with it, although I suppose they might as well be sampled at a higher frequency.

I need to do a bit of prototyping anyway to check that the XMEGA is up to the task. It's a shame Atmel don't make any parts with fast ADCs, although I could go external.

[zero]

I've been thinking about this a bit more. There is a SAM3 ARM MCU that has USB 2.0 and an external RAM interface which could do this using an external high speed ADC.

It's still a bit bandwidth limited, if say you want to sample at 16MHz then you need to send 16MB/sec, possible for USB2.0 but you also need to handle the index signal ideally. Could either drop to 7 bits ADC resolution and 1 index bit, or try to hit 32MB/sec either to external SDRAM or over the USB bus. Going faster would mean being able to send other signals like the decoded read signal as well.

Another option would be to trigger sampling on the index signal so that it's position is implied rather than transmitted.

Anyway, the next issue is the analogue front end. I'm researching designs for a simple one that interfaces with 'scope probes.

The question arises why not just use a Picoscope or similar... Well, could do if there was a good one to use as a standard to develop software for, but Picoscope are not cheap and when I tried a few years back I found there were issues trying to stream continuous samples out of them. The lower end ones at least seem to be more geared towards periodical sampling and not continual long waveform recording. Also you still need something to control the drive.

[pandy71]

As quick capture USB frontend something like USB logic analyzer can be used (cheap Saleae clone should be more than fine - something like: https://sigrok.org/wiki/MCU123_Saleae_Logic_clone )

[pandy71]

Forgot about latest RPi - Pico is equipped with RP2040 MCU and it has similar to TI, dedicated HW to perform software I/O - this is called PIO and looks quite ok in terms of this project - with additional ADC should be more than fine - USB is quite slow but if some signal processing can be done on MCU itself then is should be sufficient.

[zero]

Originally Posted by pandy71

Forgot about latest RPi - Pico is equipped with RP2040 MCU and it has similar to TI, dedicated HW to perform software I/O - this is called PIO and looks quite ok in terms of this project - with additional ADC should be more than fine - USB is quite slow but if some signal processing can be done on MCU itself then is should be sufficient.

I looked at it but there was an issue... Maybe something to do with being able to synchronously sample a port?

At the moment I think the best option would be a SAM3 micro, which should be able to push ~30MB/sec over the USB port and sample at some decent rate like 20MHz. Those are used on the Arduino Due so are easy to come by.

When I have time I might grab one and try it out. I have used the Atmel USB stack before so not too much of a learning curve.

The other issue is the analogue front end. Wondering if a very simple fixed amplifier would be okay, so a 10x probe can be used. Maybe with a simple potentiometer to control gain. The actual voltages don't matter really, it's the waveform shape that is of interest so it doesn't need to be a precision calibrated system.

[pandy71]

Originally Posted by zero

I looked at it but there was an issue... Maybe something to do with being able to synchronously sample a port?

In theory with oversampling there is no need to synchronously sample port and i think you will use oversampling anyway.

But perfect synchronous sampling would require to generate system clock for Amiga itself so external clock generator connected to Amiga video port is absolute must for this - then you will have perfect synchronous sampling. It is quite important to be aware that Paula produce floppy bitstream synchronously with system clock and as such bitrate of the raw data is for PAL 506699.285714286bps and for NTSC 511363.636363636bps - it can be difficult to achieve such sample rates without substantial oversampling as i doubt that any uC will have this kind of flexibility in setting sample rate speed from normally used crystals.

More important is i think point where signal is sampled - if this will be raw magnetic head signal or for example analogue pre-procesed signal (so some filtering and differentiator applied to signal by specialized (ASP) floppy IC).

It would be nice to replace overall analogue processing by DSP but this may require a lot of RAM to store RAW analog flux stream and perhaps some DSP-like instructions.

Originally Posted by zero

At the moment I think the best option would be a SAM3 micro, which should be able to push ~30MB/sec over the USB port and sample at some decent rate like 20MHz. Those are used on the Arduino Due so are easy to come by.

When I have time I might grab one and try it out. I have used the Atmel USB stack before so not too much of a learning curve.

The other issue is the analogue front end. Wondering if a very simple fixed amplifier would be okay, so a 10x probe can be used. Maybe with a simple potentiometer to control gain. The actual voltages don't matter really, it's the waveform shape that is of interest so it doesn't need to be a precision calibrated system.

If you have experience with this uC then for sure this will be a good choice.

I would skip manual gain control and use one of PGA's - something like MAX9939 or perhaps something like PGA460 from TI (this is in theory for ultrasonic application but it has embedded 12 bit ADC and perhaps can be used for this application too).
Alternatively some VGA (like THS7530) can be used and gain can be controlled by DAC (DAC even like PWM/DeltaSigma, even in software on uC).

Of course everything depends on your approach to signal - if floppy default IC will be used then probably gain will be fixed.

[zero]

Originally Posted by pandy71

In theory with oversampling there is no need to synchronously sample port and i think you will use oversampling anyway.

I mean you need to be able to have a timer running in the MCU that samples a port periodically and saves the data into a buffer, which is then send over USB.

On an XMEGA it's really easy, you just use double buffered DMA triggered off a timer. You can reliably hit every other clock cycle that way, so 16Msps without overclocking.

Then you just connect an external DAC to an 8 bit port and output the clock on a pin via the PWM generator.

The SAM parts look like they can do it in a similar way, and STM32 probably can too.

More important is i think point where signal is sampled - if this will be raw magnetic head signal or for example analogue pre-procesed signal (so some filtering and differentiator applied to signal by specialized (ASP) floppy IC).

Every drive will be a bit different. Even two of the same model will have different signal levels coming from the heads. My plan was to take the amplified signal and use that, because as I mentioned the signal level itself doesn't really matter, only the shape of the waveform.

I would skip manual gain control and use one of PGA's - something like MAX9939 or perhaps something like PGA460 from TI (this is in theory for ultrasonic application but it has embedded 12 bit ADC and perhaps can be used for this application too).
Alternatively some VGA (like THS7530) can be used and gain can be controlled by DAC (DAC even like PWM/DeltaSigma, even in software on uC).

That's an interesting idea. I was think thinking of just using a few fixed gain settings or maybe an I2C potentiometer in the op-amp feedback loop, but it looks like I could just get a chip that integrates everything.

I think the first step is to get the MCU part working reliably. Need to be able to stream that data over USB. Then I can experiment with different analogue front ends.

[pandy71]

Originally Posted by zero

I mean you need to be able to have a timer running in the MCU that samples a port periodically and saves the data into a buffer, which is then send over USB.

On an XMEGA it's really easy, you just use double buffered DMA triggered off a timer. You can reliably hit every other clock cycle that way, so 16Msps without overclocking.

Then you just connect an external DAC to an 8 bit port and output the clock on a pin via the PWM generator.

The SAM parts look like they can do it in a similar way, and STM32 probably can too.

I think DMA controller (sometimes Flexible DMA - FDMA - DMA capable to process data similarily to Blitter is common in most of modern uC, especially for 32 bit ones as they usually don't deal very well with real time I/O tasks).
RPi Pico looks quite tempting to me as I/O can be served by some form of dedicated rudimentary but programmable state machine - something like Copper in Amiga - IMHO with this approach even real time CIA/s emulation is possible (so for example pin 8520 replacement with RP2040 should be feasible), similar for this kind of projects where floppy signal need to be processed - perhaps even MFM decoding/encoding done at the I/O level.

Originally Posted by zero

Every drive will be a bit different. Even two of the same model will have different signal levels coming from the heads. My plan was to take the amplified signal and use that, because as I mentioned the signal level itself doesn't really matter, only the shape of the waveform.

Unless you have this kind of issue:

https://www.kashifali.ca/2021/05/rec...-oscilloscope/

This is case where analog processing of the raw floppy head signal may be beneficial - so it was my question - are you using native floppy amplifier and differentiator analogue signal or rather you will replace it by your software so having VGA/PGA could be more than desired

Originally Posted by zero

That's an interesting idea. I was think thinking of just using a few fixed gain settings or maybe an I2C potentiometer in the op-amp feedback loop, but it looks like I could just get a chip that integrates everything.

Yep, those AFE are very nice solutions for many issues during conversion from analog to digital also price is more than OK (obvious if you know that some devices needs ten's or hundred's similar AFE's).

Originally Posted by zero

I think the first step is to get the MCU part working reliably. Need to be able to stream that data over USB. Then I can experiment with different analogue front ends.

True, from my perspective this is very nice project and i wish you all the best.

[zero]

Originally Posted by pandy71

I think DMA controller (sometimes Flexible DMA - FDMA - DMA capable to process data similarily to Blitter is common in most of modern uC, especially for 32 bit ones as they usually don't deal very well with real time I/O tasks).

Yes, the problem is that not all DMA is equal. Most of the time it's "best effort", so if something else is competing for bandwidth it can get delayed by a cycle or two.

To an extent some delay isn't an issue. For example if you are sampling at 1/4 the clock rate you would probably arrange it like this:

Cycle 0 - clock to ADC
Cycle 1 - Trigger DMA to sample port
Cycle 2 - DMA might be delayed until here
Cycle 3 - DMA might be delayed until here

On XMEGA in my tests you are guaranteed to get your sample in that timeframe. In fact you can go as low as clock/3. Problem is you need to do something with the samples and the USB FS interface on those parts isn't fast enough, it's about 1MB/sec max.

Of course you can just get an ARM with massive amounts of RAM, 500MHz and DMA, but the learning curve is kinda steep and the free tools are usually rubbish. The SAM3 looks like it might be the sweet spot.

RPi Pico looks quite tempting to me as I/O can be served by some form of dedicated rudimentary but programmable state machine

Problem is that it only has USB1.1 and about 256k of RAM, nowhere near enough.

The XMEGA parts support external SDRAM or SRAM, but in my tests it looks like the bandwidth may be too low.