Converting between SuperCard Pro and Kryoflux raw/stream file formats

[mark_k]

Is any software available to convert between the raw/stream file formats used by Kyroflux and SuperCard Pro?

If not, I'm hoping it wouldn't be impossibly hard to write my own converter...

[IFW]

You are going to alias data that has already been aliased during sampling by each hardware... if you really want to do it, you need to do proper sampling conversion, like an audio software does, otherwise you introduce drifts and shifts that are not there in the first place in either original, unconverted data.
So if you are familiar with audio processing or dsp then it's not too difficult, if you are not, it's easy to do something that seems to work, but is incorrect and may render any later analysis unfeasible, due to cell jitter, peak shifts, compression etc introduced during conversion.

[IFW]

Ideally, you read and represent the samples in a very high numerical resolution reading the original data from any of the formats, then you resample to the selected target format.
This way you could avoid most of the artefacts introduced by anything simpler.

[mark_k]

Ah okay. To avoid a complex not-entirely-lossless conversion process, would it be possible for a future version of the Kryoflux software (DTC?) to also support reading/processing .scp image files?

So if a Kryoflux owner obtains some .scp image files (of normal Amiga disks for this example), they could use DTC to convert them to ADF prior to writing back to a real floppy disk.

[JimDrew]

The HxC floppy emulator software can convert to/from SCP and KF image formats. No need to write anything, it's already done!

SCP can convert a .scp flux image into a .adf image. From there you can also use a variety of programs to write the adf back to a disk. I am testing the SCP's new .adf writing code right now. So, besides being able to go from disk->flux->adf, you will also be able to go adf->flux->disk. I assume you are just trying to write adf images back to disk?

[IFW]

I'll make note of this request.

[mark_k]

Originally Posted by JimDrew

The HxC floppy emulator software can convert to/from SCP and KF image formats. No need to write anything, it's already done!

I didn't know about the HxC software, thanks for that! If its conversion is good/accurate enough, it should allow excellent interoperability between Kryoflux and Supercard Pro software/hardware. So (in theory) it should be possible to:

• Dump a disk using SuperCard Pro (or download .scp file), convert to Kryoflux stream then analyse/process using Kryoflux software, or use with emulator which only supports IPF/Kryoflux images.
• Download/dump disk in Kryoflux stream format, convert to .scp for analysis/processing using SCP software.

[JimDrew]

That is correct, however, keep in mind that when going from KryoFlux to SuperCard Pro format there will be a loss of resolution. SuperCard Pro samples flux at a 25ns resolution and KryoFlux samples flux at a 41.66ns resolution.

[Supamax]

Originally Posted by JimDrew

That is correct, however, keep in mind that when going from KryoFlux to SuperCard Pro format there will be a loss of resolution. SuperCard Pro samples flux at a 25ns resolution and KryoFlux samples flux at a 41.66ns resolution.

Converting from 41.66ns res. to 25ns res. leads to resolution loss? Not vice versa?

P.S. I asked this before but got no clear answers: where can these .scp images be found? Is there a repository or something? Thanks

[JimDrew]

Originally Posted by Supamax

Converting from 41.66ns res. to 25ns res. leads to resolution loss? Not vice versa?

Correct. Going from 41.66ns to 25ns is a loss. 41.66ns is ~24 samples per microsecond, and 25ns is 40 samples per microsecond. When you convert a lower resolution to higher resolution, you will have to do something to make up for the missing data, like averaging it.

Originally Posted by Supamax

P.S. I asked this before but got no clear answers: where can these .scp images be found? Is there a repository or something? Thanks

I am sure you won't get clear answers from anyone about this publicly if they are hosting copyrighted images. I know there are sites that exist, but I don't keep track of them.

[demolition]

Originally Posted by JimDrew

Correct. Going from 41.66ns to 25ns is a loss. 41.66ns is ~24 samples per microsecond, and 25ns is 40 samples per microsecond. When you convert a lower resolution to higher resolution, you will have to do something to make up for the missing data, like averaging it.

Are both flux formats quantized to 2 steps?

If so, it will be hard to convert back and forth without issues. One time may work, but it will quickly deteriorate. If one of the two flux formats is analog (=a significant number of bits), it will be much better since much more information will be retained as you will also have timing information in the amplitude (if resampled properly). This timing information is thrown away when you quantize the output to 2 levels.

Notice that no matter how well it is done, resampling will always be lossy and converting back and forth several times will result in a successively degrading copy.

[IFW]

It is the timing between flux reversals that is being sampled by any of these devices, as that is the only thing you can access without modifying the drive hardware.
In other words, there is no amplitude information, it's all sampling an analog signal quantized to digital by the drive head filtering/AGC - that is the digital signal being timed by all of the devices. The quantization varies from drive to drive.
The sampling resolution simply refers to the aliasing factor in this case, not the number of samples taken per second. In other words, the time between two signals will be aliased to a multiple of whatever resolution is used for sampling plus the side effects of drive speed wobble etc.
There is absolutely nothing you can do about this, quite simply because you'd need a drive that was writing exactly at the same speed (to the nanosecond level) as is the drive you are using to sample the data as well as identical filtering circuitry... which is plain impossible due to how these drives work.
Any processing you do with the sampled digital data, such as resampling does introduce degradation, quality depending on how good the resampling you do is... but as you say each round trip would contribute to the loss of precision.

[Supamax]

Originally Posted by JimDrew

Correct. Going from 41.66ns to 25ns is a loss. 41.66ns is ~24 samples per microsecond, and 25ns is 40 samples per microsecond. When you convert a lower resolution to higher resolution, you will have to do something to make up for the missing data, like averaging it.

Thanks

I am sure you won't get clear answers from anyone about this publicly if they are hosting copyrighted images. I know there are sites that exist, but I don't keep track of them.

Ok, but I would gladly welcome a PM regarding those sites/images from any mate here in EAB .
I really don't understand why:
- exchanging .adf files of copyrighted material: ALLOWED
- exchanging .scp files of copyrighted material: NOT ALLOWED

Copyrighted material is copyrighted material, regardless the media used.

[Supamax]

Originally Posted by demolition

Are both flux formats quantized to 2 steps?

If so, it will be hard to convert back and forth without issues. One time may work, but it will quickly deteriorate. If one of the two flux formats is analog (=a significant number of bits), it will be much better since much more information will be retained as you will also have timing information in the amplitude (if resampled properly). This timing information is thrown away when you quantize the output to 2 levels.

Notice that no matter how well it is done, resampling will always be lossy and converting back and forth several times will result in a successively degrading copy.

Wouldn't it be easier (lossless) if both (SuperCard Pro and KryoFlux) sampled flux at the same resolution?

[IFW]

The sampling is already lossy for any device of this kind, but the "loss" is aliasing exact timings to nearby timings. The drive speed wobble contributes as well, not to mention the quantizing done by the filtering circuitry in the drive...
So no matter what you do, it's lossy by nature, just like saying sampling any analog source - while what you sample is the timing of a digital signal, but the digital signal has been converted from analog by the drive first and is subject to small deviations. The only thing you change with the sampling resolution is that where "rounding" (aliasing) will occur, ie every Nth sample - but again, this will constantly change.
e.g. to demonstrate this, some random numbers:
10 10 11 10 10 10... vs 9 9 9 10 9 9...
Or think about plotting a graph, where fractions are aliased to integer values and depending on the result they fall into a specific value.
Another example is the jaggies in computer graphics, where you need anti-aliasing to compensate for the rounding because they cause visual artefacts - it's exactly same case as plotting graphs; you try to represent fractional values in a system where only integers are present.
Notice the word "represent" - that does not imply that your data is incorrect; the original data is correct, and can be easily reconstructed.
What does matter is that only the values alias, the sampling hardware itself does not - so the sum of timings stored in the samples are identical, where rounding happens and in what way is different.

As for resampling, you can get it very good, if done properly. Thinking about it a bit more, if done at sufficiently high resolution, in the first round of the conversion you'll get some timings moving a bit (think in nanoseconds...) then you'll get a slight oscillating effect for each round-trip, but essentially two fixed variants of the second generation data as long as the resampling algorithm is consistent. A good example of this would be RGB <> HSV conversion where the exact values first don't match, but after a second conversion they become stable - as long as the conversion is done at least in floating point resolution, but ideally higher...

We are talking about differences oscillating in the nanoseconds realm here, and to put this into perspective the data is represented in microseconds where 1us is 1000ns... and e.g. in MFM DD encoding (Amiga) the smallest valid flux reversal is 4us, ie 4000ns - so we are talking about a very small error here, a few ns vs 4000ns (min).
So unless it is some next to unreadable data where the different recording bands are almost bleeding into each other, the conversion should make no difference to the actual content. I must stress the point here: "if done properly"

[demolition]

Originally Posted by Supamax

Wouldn't it be easier (lossless) if both (SuperCard Pro and KryoFlux) sampled flux at the same resolution?

Maybe, but if you can improve the sampling rate to create a better dump, shouldn't you? I think SCP is newer than Kryoflux, so it makes sense it would dump with a higher sampling rate. If only it was exactly double that of the Kryoflux, conversion would have been so much better.

Originally Posted by IFW

We are talking about differences oscillating in the nanoseconds realm here, and to put this into perspective the data is represented in microseconds where 1us is 1000ns... and e.g. in MFM DD encoding (Amiga) the smallest valid flux reversal is 4us, ie 4000ns - so we are talking about a very small error here, a few ns vs 4000ns (min).

When it is oversampled by that much, then a proper resampling shouldn't damage any data. It would be like resampling an audio signal and complaining about getting noise at 20 MHz. Who cares?

[JimDrew]

I have found that any sampling faster than 20ns doesn't improve the image quality, but there is a significant differnce between 25ns and 50ns sampling.

I chose 25ns because the PIC is used runs at 160MHz internal to give 40 samples per microsecond. I made a new version with <15ns sample rate, but it was a waste so I kept it the same.

[JimDrew]

Originally Posted by Supamax

Ok, but I would gladly welcome a PM regarding those sites/images from any mate here in EAB .
I really don't understand why:
- exchanging .adf files of copyrighted material: ALLOWED
- exchanging .scp files of copyrighted material: NOT ALLOWED

Copyrighted material is copyrighted material, regardless the media used.

You're right. Anyone posting copyrighted data without permission, regardless of the format it is in, is committing copyright infringement.

I don't keep track of these sites, because I don't want to know.

[IFW]

Originally Posted by demolition

When it is oversampled by that much, then a proper resampling shouldn't damage any data. It would be like resampling an audio signal and complaining about getting noise at 20 MHz. Who cares?

Yes, that pretty much sums up the effect you'd have with proper resampling

[Arnie]

Have I got this right, the op has Kryoflux (.raw) & SuperCardPro (.scp) images and wants to convert them to either format?

Why would you want to do that?