KryoFlux USB Floppy Controller (was: C2 DiskSystem)

[fiath]

Nice Andreas. You have no idea how many names we considered before that one!

WIP reports for 13th and 14th October have been posted:

• 2009-10-13 - KryoFlux - Stupid Timers
• 2009-10-14 - KryoFlux - Timing and Measurement

[andreas]

Originally Posted by fiath

Nice Andreas. You have no idea how many names we considered before that one!

Oh yes I do! Why else would people need software to find a good name these days? Because most of them are taken, obviously..

To a more personal note: nice to see YOU back on here! To cut it short ... you were keeping a "little bit (too?) low profile" all those past months...
Almost half a year...(let alone your 2 years (!) of absence before that! )

Nice to see you back in the biz!

[fiath]

Real life sucks doesn't it.

[fiath]

WIP reports for 15-26th October have been posted. This takes us up to a fantastic milestone, but it makes more sense if the updates are read in order.

http://softpres.org/news:2009-11-21

• 2009-10-15 - KyroFlux - Progress
• 2009-10-17 - KyroFlux - Problems with HD
• 2009-10-24 - KryoFlux - New Interrupt Encoding System
• 2009-10-26 - KyroFlux - Results!

IFW, you are a genius!

[fiath]

The WIP reports are now at the point where we were very happily sampling both DD and HD disks at a great resolution. This was especially pleasing for us, because we were not sure HD was even possible with this hardware. It's been a lot of work, but it's very exciting to see where this is going, and there is lots of good stuff to come.

http://softpres.org/news:2009-11-22

• 2009-10-27 - Optimisation Strategies
• 2009-10-28 - Transfer Rate Testing

[andreas]

"We could even make it faster, but the compiler thinks it’s smarter... and we’re not going to convert it to assembly code as it would not be maintainable anymore"

Wow. You still amaze me, guys. Still keeping to a high-level language despite all these (past and current) issues...frankly, I'd thought this could not be done without asm, let alone that stuff described in your end of October progress report.

I hope you won't ever have to pull the emergency brake (i. e. really be forced to use true asm)

Originally Posted by IFW

Yes, it is of course should be possible to write selected tracks. However unless you are 100% certain that your version is identical with an ipf version it is possible that you'd overwrite something you wouldn't want to...

"Unless identical with ipf".
Ouch. So the tool would think to be clever and decide, "hey there's some bytes off compared to the IPF" and write me back a Byte Bandit virus on block 0 of that target disk because the master had it too...
As you know, there ARE a half handful of IPFs known with a boot virus on.
Heh. That kind of software, which thinks to be (too) clever, can get you in a vice sometimes

[killergorilla]

Excellent stuff guys. Well done.

[IFW]

Originally Posted by andreas

"Unless identical with ipf".
Ouch. So the tool would think to be clever and decide, "hey there's some bytes off compared to the IPF" and write me back a Byte Bandit virus on block 0 of that target disk because the master had it too...
As you know, there ARE a half handful of IPFs known with a boot virus on.
Heh. That kind of software, which thinks to be (too) clever, can get you in a vice sometimes

No, it's up to you to decide - but any complaints about the outcome would be promptly ignored

[mr.vince]

Originally Posted by IFW

No, it's up to you to decide - but any complaints about the outcome would be promptly ignored

Giana v2 anyone?

[fiath]

The remaining WIP reports for October have now been posted.

http://softpres.org/news:2009-11-24

• 2009-10-29 - Out Of Band Data
• 2009-10-30 - Interrupts
• 2009-10-31 - Data Packets

[andreas]

Originally Posted by mr.vince

Giana v2 anyone?

Bingo!

[mr.vince]

Originally Posted by andreas

Bingo!

Nearly overwrote the existing copy of version 2 with working high score saving because there already was an IPF. It was the thread here on the forums that got my attention because of the problems with the IPF.

So in fact I'd say only overwrite when you really have to. Otherwise, just make a backup and use it instead.

[fiath]

First two WIPs from November are now up covering work done, and also detail an important detail - how the flux transition information streamed from the KryoFlux hardware will be persisted in a file - the DRAFT file format.

http://softpres.org/news:2009-11-26

• 2009-11-04 - DRAFT
• 2009-11-06 - Work To Do and Compression

[andreas]

"It may work with emulators, but functionality is not guaranteed and may be unreliable. Some types of protections will not work."

Heh...

But (lets say...) 90% of protections DID work with IPF, didn't they?
You guys are almost giving the impression of the DRAFT format being actually inferior to the well-established IPF one...?

[Pheonix]

In a way, DRAFT could be inferior. Even with a brand new factory disk, the DRAFT results will be raw data. This data can (and most likely always will,) have noise, minor flaws, etc... Just like any other analog form of storage. Meanwhile, from what I understand, IPF format is when the raw data is carefully examined on a disk by disk basis and produces an image of what the original floppy is supposed to contain & provide to the system. The only thing I can think of that would be superior to a properly constructed IPF would be the original "master" used to create the disks in the first place.

Then again, I don't know any real details on the IPF format itself. I could be wrong.

[mr.vince]

Originally Posted by andreas

Heh...

But (lets say...) 90% of protections DID work with IPF, didn't they?
You guys are almost giving the impression of the DRAFT format being actually inferior to the well-established IPF one...?

IPF is a scripted representation of the original master data, generated by the Softpres Analyzer. The IPF "knows" what's on the original disk (even in regard to data integrity) and how this data will be presented to the device by the drive electronics and controller. It is therefore possible to reconstruct ANY form copy protection, read: data representation. There just isn't support for 100% of all games available because of limited manpower.

DRAFT files are different. They are not meant to replace IPFs. They also contain all the data present on the original disk. There is nothing lost during the dumping process: "If it's on the disk, it's in the image" TM. What DRAFT files lack is the scripting part of IPFs, therefore it is impossible to mimic all forms of copy protection. Important: Although DRAFT files might not be able to mimic certain protections, all data is preserved at forensic quality. DRAFT files are what will be there in the first place, with ADFs, IPFs and other format derived from them. So from user's point of view, DRAFT is what makes sense. And you always have a chance to go back to the bare flux data in case you encounter any problems with the interpretation that has been made (e.g. ADF).

[IFW]

DRAFT files are going to replace the raw files submitted by contributors in the past.

DRAFT is not intended to replace any kind of format, it's a transient and archival format.
Whenever you have a DRAFT file it contains exactly what signals have been read from the disk at the creation of the DRAFT file - nothing added or taken, "no artificial ingredients"

How to interpret those signals is up to the application processing the file. Once you interpret data it is inevitable that something gets lost in the process, which may or may not be important.

If you need an ADF or an ST file chances are you only need the sector content, for example from a demo disk.

If you have an IPF file you have a file that represents the meaningful data of a factory new disk fresh out of duplication and guaranteed to be correct and to always read the same data as the data was intended to be read by the target platform(s) - ie not affected by reading problems unless such problems are intentional.

We'll provide a reference implementation of reading and processing a DRAFT file for some common cases, but there will be nothing stopping you to roll your own if feeling adventurous.

There is however no guarantee that what the DRAFT file contains is correct etc - they have to be properly analysed in the domain of the application. These targets are likely to be different for each application - but they can all rely on a format providing as much data as if the disk was available and read physically at the time of processing the DRAFT file.

[fiath]

We have added a page describing exactly what KryoFlux is. If you are unfamiliar with what we have being doing the last few months, this is the best place to start. It also has some pictures of one of the development boards we are using.

The WIP update for the 7th November has been posted, illustrating the information captured while dumping a disk, and why it is needed.

We've also written up the very good point, made here, of needing to clarify how the new DRAFT format relates to the existing IPF format.

http://softpres.org/news:2009-11-27

[andreas]

mr. vince & IFW, thank you very much for your detailed explanations - very appreciated.

fiath, that was about time we get a description of the (cor)relation between IPF and DRAFT. A very good read, thanks

Originally Posted by IFW

If you have an IPF file you have a file that represents the meaningful data of a factory new disk fresh out of duplication and guaranteed to be correct

Yep.
Though in contrary:

There is however no guarantee that what the DRAFT file contains is correct etc - they have to be properly analysed in the domain of the application.

Yeah. To put it bluntly, you're sort of playing a poker game...

So in terms of reliability (and absolute data correctness), IPF is still not expected to get sort of "deprecated" sooner or later - obvious, isn't it. Since their co-existence feels almost mandatory, considering the above facts and, in some way, "limitations" of the new format.

[mr.vince]

Originally Posted by andreas

Yeah. To put it bluntly, you're sort of playing a poker game...

So in terms of reliability (and absolute data correctness), IPF is still not expected to get sort of "deprecated" sooner or later - obvious, isn't it. Since their co-existence feels almost mandatory, considering the above facts and, in some way, "limitations" of the new format.

Well you're alway playing poker when saving data to any magnetic medium.

But to be honest... DRAFT is where it all starts, it replaces the RAW files we are currently using.

There is nothing better. You get an ultra high resolution flux stream with 5 consecutive revolutions of each track. And a documented format you can use to write your own converters for.

If you want to verify what you are doing, you could modify the host software to do decoding on the fly and read a single track over and over until the data matches a scheme, e.g. classic Amiga file system, FFS...

But you could as well take the DRAFT and us it. 5 Revolutions usually give enough source data. If not, you could redump. It's only a matter of file size...

IPFs will for sure be around for years to come. It's the format derived from RAW or DRAFT files, with data processed and verified as explained above.