Incorrectly sized ADFs in TOSEC

peo · 09 August 2020, 10:07

Hello,

Probably a stupid-question, but I'm working on something...

I have created a script that import the content from the dat files into MySQL. A quick check for oversized ADFs gives me:

Code:

MySQL [tosec]> select name,size from roms where size >923648 and name like "%.adf";
+--------------------------------------------------------------------------------------------------------------------+---------+
| name                                                                                                               | size    |
+--------------------------------------------------------------------------------------------------------------------+---------+
| Top Karaoke 1 (1995-03)(Saturne)(AGA)(Disk 1 of 2)[HD][extended adf].adf                                           | 2104892 |
| Top Karaoke 1 (1995-03)(Saturne)(AGA)(Disk 2 of 2)[HD][extended adf].adf                                           | 2104892 |
| Bloodwych (1989)(Image Works)[save disk][extended adf].adf                                                         | 2104892 |
| Cannon Fodder (1993)(Virgin)[a][save disk][extended adf].adf                                                       | 2025932 |
| Cannon Fodder (1993)(Virgin)[save disk][extended adf].adf                                                          | 2005594 |
| A-Mopoly v1.32 (1990-10-21)(R-H-S)(DE)[o][v Butonic v3.00 - Disaster Master 2].adf                                 |  936174 |
| Amberstar v1.96 (1992)(Thalion)(Disk 1 of 3)(A)[FD installed][extended adf].adf                                    | 2104892 |
| Amberstar v1.96 (1992)(Thalion)(Disk 2 of 3)(B)[FD installed][extended adf].adf                                    | 2104892 |
| Amberstar v1.96 (1992)(Thalion)(Disk 3 of 3)(C)[FD installed][extended adf].adf                                    | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 1 of 6)(Trailer Disk)[extended adf].adf                                              | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 2 of 6)(Main Disk)[extended adf].adf                                                 | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 3 of 6)(Level Disk 1)[extended adf].adf                                              | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 4 of 6)(Level Disk 2)[extended adf].adf                                              | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 5 of 6)(Level Disk 3)[extended adf].adf                                              | 2104892 |
| Apano Sin (2000)(V.A.K.)(Disk 6 of 6)(Final Disk)[extended adf].adf                                                | 2104892 |
| Arctic Fox (1985)(Electronic Arts)[b][extended adf].adf                                                            |  924560 |
| Arctic Fox (1985)(Electronic Arts)[extended adf].adf                                                               |  924560 |
| B.C. Kid (1992)(Ubi Soft)[extended adf].adf                                                                        | 2071552 |
| Beyond Dark Castle (1989)(Activision)(Disk 1 of 2)[extended adf].adf                                               | 2104892 |
| Beyond Dark Castle (1989)(Activision)(Disk 2 of 2)[extended adf].adf                                               | 2104892 |
| Champ, The (1989)(Linel)[cr Illustra][extended adf].adf                                                            | 2025932 |
| Deep Space v1.00a (1986)(Psygnosis)[extended adf].adf                                                              | 1413190 |
| Desert Strike - Return to the Gulf (1993)(Electronic Arts)(Disk 2 of 3)[cr PDX][o].adf                             |  934912 |
| Fate - Gates of Dawn v1.4 (1991)(reLINE)(DE)(Disk 1 of 2)[extended adf].adf                                        | 2051942 |
| Fate - Gates of Dawn v1.4 (1991)(reLINE)(DE)(Disk 2 of 2)[extended adf].adf                                        | 2051774 |
| Fighter Duel Pro Flight Recorder v1.0 (1993-03-01)(Jaeger)(Disk 2 of 3)[o][b fake track].adf                       |  934912 |
| Fighter Duel Pro Flight Recorder v1.0 (1993-03-01)(Jaeger)(Disk 3 of 3)[o][b fake track].adf                       |  934912 |
| Great Courts (1989)(Ubi Soft)[a][extended adf].adf                                                                 | 1987320 |
| Great Courts (1989)(Ubi Soft)[extended adf].adf                                                                    | 2104892 |
| Hard 'n' Heavy (1989)(reLINE)[cr QTX][t +6 BS1][o].adf                                                             |  934912 |
| Harley-Davidson - The Road to Sturgis (1990)(Mindscape)[cr TRSI][o][b dump].adf                                    |  934912 |
| Karate Kid Part II, The (1987)(Microdeal)[b startup-sequence, type kk][extended adf].adf                           | 1782924 |
| Masterblazer (1990)(Rainbow Arts)[extended adf].adf                                                                | 2025472 |
| Nitro (1990)(Psygnosis)[extended adf].adf                                                                          | 1942886 |
| Silkworm (1989)(Virgin)[cr ACC - TSK][t +1 ACC - TSK][extended adf].adf                                            | 2028458 |
| Super Ski II (1992)(Microids)(Disk 2 of 2)[cr FSN][o][b corrupt files].adf                                         | 1126400 |
| Terrorpods (1987)(Psygnosis)(M7)[extended adf].adf                                                                 | 1214634 |
| Turrican (1990)(Rainbow Arts)[extended adf].adf                                                                    | 1951744 |
| Turrican 3 (1993)(Factor 5)(beta)(Disk 1 of 2)[cr Heresy - ZNT][f trainer Heresy - ZNT][t +26 Heresy - ZNT][o].adf |  965632 |
| Turrican 3 (1993)(Renegade)[extended adf].adf                                                                      | 2342912 |
| Turrican II - The Final Fight (1991)(Rainbow Arts)[extended adf].adf                                               | 2166784 |
| Tusker (1990)(System 3)[b][extended adf].adf                                                                       | 1280380 |
| Tusker (1990)(System 3)[extended adf].adf                                                                          | 1585944 |
| Vigilante (1989)(U.S. Gold)[extended adf].adf                                                                      | 1994992 |
| Wings of Death (1990)(Thalion)(Disk 1 of 2)[extended adf].adf                                                      | 1990652 |
| Wings of Death (1990)(Thalion)(Disk 2 of 2)[extended adf].adf                                                      | 2004560 |
| Ziriax (1990)(Software Business)[a][extended adf].adf                                                              | 2018674 |
| Ziriax (1990)(Software Business)[extended adf].adf                                                                 | 2041480 |
| One, The #26 (1990-10)(EMAP Images)(GB)[Nov 1990][Car-Vup - The Final Frontier][extended adf].adf                  | 1439610 |
| One, The #29 (1991-02)(EMAP Images)(GB)[Feb 1991][Foundations Waste][extended adf].adf                             | 1440232 |
| One, The #30 (1991-03)(EMAP Images)(GB)[Mar 1991][Roboplex][extended adf].adf                                      | 1440076 |
| One, The #34 - for Amiga Games (1991-06)(EMAP Images)(GB)(Disk 2 of 2)[Jul 1991][extended adf].adf                 | 1180146 |
| One, The #48 - Amiga (1992-08)(EMAP Images)(GB)(Disk 2 of 2)[o][b][Sep 1992].adf                                   |  923649 |
| Zero #19 (1991)(Dennis)(GB)[May 1991][Artura][extended adf].adf                                                    | 1442574 |
| Zero #26 (1991)(Dennis)(GB)(Disk 1 of 2)[Dec 1991][extended adf].adf                                               | 1441168 |
| Zero #26 (1991)(Dennis)(GB)(Disk 2 of 2)[Dec 1991][extended adf].adf                                               | 1124056 |
| Zero #28 (1992)(Dennis)(GB)[Feb 1992][extended adf].adf                                                            | 1310342 |
| Zero #30 (1992)(Dennis)(GB)[Apr 1992][extended adf].adf                                                            | 1304824 |
+--------------------------------------------------------------------------------------------------------------------+---------+
58 rows in set (0.46 sec)

What are these oversized ADFs useful for ? I mean, a floppy can hold only 164 tracks (but normally the last used is 159, which is 160 in total), so as everybody knows, the ADF should be 901120 bytes, nothing less and nothing more except in some special cases (I have found some disks to be readable for the last 4 tracks, have not checked the extra data), so a maximum with 164 tracks would be as above, 923648 bytes.

Even more odd, one file above is 923649 bytes..

Continued below (20000-limit)..

peo · 09 August 2020, 10:08

Now these broken files (incomplete):

Code:

MySQL [tosec]> select name,size from roms where size <901120 and name like "%.adf";
+---------------------------------------------------------------------------------------------------------------------+--------+
| name                                                                                                                | size   |
+---------------------------------------------------------------------------------------------------------------------+--------+
| Animation Apprentice (1987)(-)(Disk 2 of 3)[cr TKT][u].adf                                                          | 506880 |
| Animation Apprentice (1987)(-)(Disk 3 of 3)[cr TKT][u].adf                                                          | 551936 |
| Art Department Professional v2.3.0 (19xx)(ASDG)(Disk 1 of 4)[u][HD].adf                                             | 889856 |
| Deluxe Paint IV v4.61 Update (1993)(Electronic Arts)(AGA)[u][update disk].adf                                       | 596992 |
| Reflections v2.5 (1994)(MSPI)(DE)(Disk 2 of 3)[u].adf                                                               | 709632 |
| PD Games Volume 029 (199x)(The Assassins)(PD)[u].adf                                                                | 889856 |
| PD Games Volume 202 (199x)(The Assassins)(PD)[u][b corrupt file].adf                                                | 889856 |
| Taifun 200 (1992-03-10)(Manewaldt, A.)(DE)(PD)(Disk 2 of 5)[u].adf                                                  | 889856 |
| Arcade Classics (1990)(Bithead Technologies)(NTSC)(Disk 2 of 2)[u].adf                                              | 889856 |
| Mint Condition (1991-06-06)(Warlock - 17-Bit Software)(Disk 1 of 2)[u][AMOS].adf                                    | 889856 |
| Absence of Light (1991-03-16)(Animators)[u].adf                                                                     | 687104 |
| Amigadeus (1990-07)(Desktop Music Workshop - 17-Bit Software)[u].adf                                                | 889856 |
| Best Sounds Around Vol.3, The (1991-05-20)(Patrick)[u].adf                                                          | 681984 |
| Chorus II (1990)(Byterapers)[u].adf                                                                                 | 225280 |
| Kylie Minogue - Made In Heaven (1989-03)(Kode and Muzak Ltd)(Disk 2 of 2)[u].adf                                    | 518144 |
| Vega Musicdisk 2 (1991-09)(Vega)[u].adf                                                                             | 867328 |
| Compilation Disk #08265 (1999)(The Scene Archives)[u][FFS].adf                                                      | 822272 |
| Old School (19xx)(-) & Waste of Breath (19xx)(Tulou)-[u][KS2.0].adf                                                 | 889856 |
| Zombie Ritual (1993-04-19)(Thrash Gang, The)(Disk 2 of 3)[u].adf                                                    | 376832 |
| 9 Fingers (1993-12-29)(Spaceballs)(Disk 2 of 2)[u][TP3#4].adf                                                       | 712704 |
| AMF II (1992-04-18)(Lynx Crew)[u].adf                                                                               | 889856 |
| Big Time Sensuality (1994-03-18)(Axis)(AGA)(Disk 1 of 2)[u][b dump].adf                                             | 833536 |
| Enigma (1991-03-29)(Phenomena)[u].adf                                                                               | 491520 |
| Eradication (1993-12-29)(Insane)[u].adf                                                                             | 664576 |
| Girls Need Love (1988)(Destiny 7)[u].adf                                                                            | 856064 |
| Hit Fido (1991-04-28)(KGB)(Disk 1 of 2)[u][SUM#4].adf                                                               | 529408 |
| Hoi Saga - Part 3, The (1994-01-19)(Team Hoi)(AGA)(Disk 2 of 3)[u].adf                                              | 844800 |
| Hoi Saga - Part 3, The (1994-01-19)(Team Hoi)(AGA)(Disk 3 of 3)[u].adf                                              | 799744 |
| Impulse 2 (1994)(Tilt)(Disk 1 of 2)[u].adf                                                                          | 833536 |
| Power and Agony - Back from Hell (1991-12-28)(Trackers)[u].adf                                                      | 630784 |
| S.O.S. - Smell Of Shit (1992-06-26)(Melon Dezign)[u][H92#5].adf                                                     | 461824 |
| Satisfaction Guaranteed (1993-12)(Pearl)[u].adf                                                                     | 901076 |
| Universal Intensity (1991-06-07)(Complex)[u][b][SOC#3].adf                                                          | 259072 |
| Vision (1995)(Oxygene)(AGA)(Disk 2 of 2)[u][TP5#2].adf                                                              | 889856 |
| Data Division Dox Disk No.2 (1996)(The Data Division)(AGA)[u].adf                                                   | 878592 |
| Legend Doc'n'Cheats Volume 1 (19xx)(Legend)[u].adf                                                                  | 889856 |
| LSD Sprint Docs 16 (1993)(LSD)[u].adf                                                                               | 833536 |
| Calendar Quiz (1991)(Helios)[u].adf                                                                                 | 889856 |
| Designasaurus v1.0 (1989)(Britannica)[h ESI - WOW][u][b corrupt files].adf                                          | 889856 |
| A.M.C. - Astro Marine Corps (1990)(Dinamic)(Disk 1 of 2)[extended adf].adf                                          | 712974 |
| A.M.C. - Astro Marine Corps (1990)(Dinamic)(Disk 2 of 2)[extended adf].adf                                          | 746764 |
| Adidas Championship Tie-Break (1990-06-21)(Ocean)[u].adf                                                            | 889856 |
| Alien Legion (1989)(Gainstar)[u][b].adf                                                                             | 816640 |
| American Gladiators (1992)(Gametek)(Disk 3 of 3)[cr LSD][u].adf                                                     | 901063 |
| AV8B Harrier Assault (1992)(Domark)(Disk 1 of 2)[h CSL][m doscopy][u].adf                                           | 889856 |
| AV8B Harrier Assault (1992)(Domark)(Disk 1 of 2)[h CSL][u].adf                                                      | 889856 |
| Bloodwych - The Extended Levels (1989)(Image Works)[cr ACC][u][b crack][data disk].adf                              | 473088 |
| Blue Angel 69 (1989)(Magic Bytes)(PAL)[cr VF - Bencor Brothers][u][b].adf                                           | 786432 |
| Cedric and the Lost Sceptre v1.0 (1995)(neo)(de-en)(Disk 1 of 6)[cr PDY][u][b].adf                                  | 900500 |
| Civilization v855.01 (1992)(MicroProse)(Disk 4 of 4)[u][b checksum error].adf                                       | 889856 |
| Cyber Empires v1.0 (1992)(SSI)(US)(Disk 1 of 2)[cr SR][u][b].adf                                                    | 573440 |
| Cyber Force - Zniewolony Umysl (1995)(Mirage Media)(PL)(Disk 3 of 3)[cr COR][u].adf                                 | 889856 |
| Cybernauts (1989)(Kingsoft)[cr QTX][u].adf                                                                          | 844800 |
| Dark Seed v1.0 (1993)(Cyberdreams)(Disk 3 of 7)[u].adf                                                              | 473088 |
| Dragonstone (1994)(Core)(M3)(Disk 1 of 5)(Boot)[cr RNX - TRD][u].adf                                                | 484352 |
| Drakkhen v1.1 (1990)(Infogrames)(Disk 2 of 2)[u][bootable].adf                                                      | 889856 |
| Drakkhen v1.1 (1990)(Infogrames)(IT)(Disk 1 of 2)[u].adf                                                            | 892928 |
| Explora II (1988)(Infomedia)(FR)(Disk 2 of 4)[u][b].adf                                                             | 901102 |
| F1 (1993)(Domark)[cr CSL][u].adf                                                                                    | 871424 |
| Fighter Bomber - Advanced Mission Disk (1990-06-25)(Activision)[h IMG][u][data disk].adf                            | 889856 |
| Forgotten Worlds (1989)(U.S. Gold)(Disk 2 of 2)[cr VF][u].adf                                                       | 889856 |
| Ghouls 'N Ghosts (1989)(U.S. Gold)(Disk 2 of 2)[cr Defjam - CCS][u][b checksum error].adf                           | 889856 |
| Grand National (1990)(Elite)(PAL)(Disk 2 of 2)[cr FLT][t +1 FLT][u][b].adf                                          | 473088 |
| Hunter (1991)(Activision)[cr CPY][h Pseudo-Ops][t +14 LGD][u][b checksum error].adf                                 | 764267 |
| Inve$t (1990)(Starbyte)(DE)(Disk 2 of 2)[cr Syndicate][u].adf                                                       | 811008 |
| James Pond - Underwater Agent (1990)(Millennium)[cr PDX][u].adf                                                     | 889856 |
| James Pond 2 - Codename RoboCod (1991)(Millennium)[cr SR][t +2 Soldiers][u].adf                                     | 901088 |
| Jupiter's Masterdrive (1990)(Ubi Soft)[cr SR][u][b].adf                                                             | 895184 |
| King's Quest V - Absence Makes the Heart Go Yonder v1.000.000 (1991)(Sierra)(Disk 7 of 8)(6)[o][b corrupt file].adf | 704512 |
| Leander (1991)(Psygnosis)(Disk 1 of 3)[cr SR][t +3 Goonies][u][b].adf                                               | 851968 |
| Lotus III - The Ultimate Challenge (1992)(Gremlin)(Disk 1 of 2)[cr FLT - Crack Inc][f AGA Ministry][u].adf          | 698368 |
| Millennium 2.2 (1989)(Electric Dreams)[cr Defjam - CCS - SP][u].adf                                                 | 698368 |
| North & South (1989)(Infogrames)(M5)[cr Phoenix][u][b doscopy].adf                                                  | 856064 |
| Power Pinball Expansion (1989)(KarmaSoft)[cr SR][u][b corrupt file][data disk].adf                                  | 889856 |
| Powerdrome (1989)(Electronic Arts)[cr TWG][u][b corrupt files].adf                                                  | 811008 |
| Push (1989)(Courbois)[cr LGD][u].adf                                                                                | 889856 |
| R-Type II (1991)(Activision)[cr SR][f AGA DCS][t +16 Slipstream][u].adf                                             | 889856 |
| Shadowlands (1992)(Domark)(Disk 2 of 2)[cr VF][u].adf                                                               | 687104 |
| Silent Service - The Submarine Simulation v825.01 (1987)(MicroProse)[cr New Age][u].adf                             | 822272 |
| StarRay (1988)(Logotron)(Disk 2 of 2)[u].adf                                                                        | 754688 |
| Statix (demo-playable) (1994)(Black Legend)[h Interactive][u][KS2.0].adf                                            | 878592 |
| Super Hang-On (1988)(Electric Dreams)[cr Defjam - CCS][u].adf                                                       | 856064 |
| Superfrog (1993)(Team 17)(Disk 3 of 4)[cr CSL][u][bootable].adf                                                     | 889856 |
| Thunder Blade (1989)(U.S. Gold)[cr Ackerlight][t +1 Ackerlight][u].adf                                              | 844800 |
| Toyottes, The (1990)(Infogrames)[cr QTX][u2].adf                                                                    | 709632 |
| Toyottes, The (1990)(Infogrames)[cr QTX][u].adf                                                                     | 709632 |
| Uninvited (1987)(Mindscape)[cr ECA][u].adf                                                                          | 900640 |
| Wacko Jr. in Wonderland (1990-06)(New Line)(preview)[cr RZR][u][unreleased].adf                                     | 867328 |
| World Games (1986)(Epyx)[u2].adf                                                                                    | 889856 |
| World Games (1986)(Epyx)[u].adf                                                                                     | 889856 |
| Workbench v1.1 rev 31.334 (1986)(Commodore)(Disk 1 of 2)(Workbench)[m icon positions][u].adf                        | 799841 |
| Workbench v3.1 rev 40.42 (1996)(ESCOM)(M10)(Disk 6 of 6)(Fonts)[u].adf                                              | 749568 |
| Coma - Issue 4 (1998-01-14)(High Voltage)(pl)(Disk 1 of 2)[u].adf                                                   | 833536 |
| Provocator - Issue 3 (1998)(Quality)(pl)[u][b corrupt file].adf                                                     | 889856 |
| Rave World - Issue 1 (19xx)(-)(pl)(Disk 1 of 2)[u][b read-write error].adf                                          | 608256 |
| Resident - Issue 1 (1992-06-05)(Deadline - ACE - SHI)[u].adf                                                        | 529408 |
| Silesia - Issue 07 (199x)(Illusion)(AGA)(pl)(Disk 2 of 2)[u].adf                                                    | 135168 |
| Stolen Data - Issue 10 (1992-12-26)(Anarchy)(Disk 1 of 2)[u][part 1].adf                                            | 450560 |
| Stolen Data - Issue 10 (1992-12-26)(Anarchy)(Disk 1 of 2)[u][part 2].adf                                            | 450560 |
| Berliner Spielekiste #464 (199x)(Berlin PD)(DE)(PD)[u][BSKCD].adf                                                   | 889856 |
| BugMine #3 (199x)(Noshtromo 2000)[f Skippy][h EMC-A][u][b corrupt files].adf                                        | 889856 |
| Expert Mine 1 (1992)(Leo - Stefan)[u].adf                                                                           | 889856 |
| Account Master v2.0 (1992)(Platt, Martin)[u].adf                                                                    | 889856 |
| Amiga Action #21 (1991-05)(Europress Interactive)(GB)(Disk 1 of 2)[u][b][Jun 1991][Disk 8].adf                      |  21504 |
| Amiga Fun issue 04 (1991-01)(MC Publications)(GB)[t +9 Defjam][u][Vyrus, Feb 1991].adf                              | 799744 |
| Bowl! v2.0 (1990)(Wacky Carrot Software)(PD)[u].adf                                                                 | 889856 |
| Real Popeye, The (1993)(Bignonia)(PD)[u][C64 conversion].adf                                                        | 878592 |
+---------------------------------------------------------------------------------------------------------------------+--------+
107 rows in set (0.41 sec)

Are these useless or not ? Have there been at least two copies (from different collections) with that same resulting dump ?

/PeO

thomas · 09 August 2020, 10:35

Extended ADFs hold the MFM bit streams rather than cooked data. 2MB size is usual. They cannot be read with trackdisk.device.

chip · 09 August 2020, 11:48

Interesting argument ..... i think extended ADF are OK

More weird is the presence of oversized and undersized ones

I suppose they could be removed when equivalent 880KB ADF is present too

But i am probably missing something

AMIGASYSTEM · 09 August 2020, 12:18

If the floppies are HD 1.76MB, and up to about 2MB with third party devices are Standard HD floppies that can be managed by AmigaOS, those posted above instead are Custom "UAE" floppies that are not visible by AmigaOS like those mentioned above.

---- Italiano ----

Se i floppy sono HD da 1,76MB, e fino a circa 2MB con device terzi sono floppy Standard HD gestibili da AmigaOS, quelli postati sopra invece sono Floppy Custom "UAE" che non sono visibili da AmigaOS come quelli ciatati sopra.

peo · 09 August 2020, 13:03

Using HD floppies on the Amiga as HD floppies are quite unusual. Wouldn't it just be better to have these (others) in KryoFlux CT RAW or IPF formats ?
The other way around, using HD floppies as DD on the Amiga were probably more common (but these will generate 901120 bytes dumps, so OK in this case - just makes a mess when preserving HD floppies which do not have the HD-hole covered)

A sneak-peak of what I'm into (unless this is already implemented in ADF-Workshop):
Besides the .dat to MySQL import I had since earlier (2.5 years), I'm now adding in checksums (crc32) of the individual blocks of each floppy. This will later (when I'm done thinking) make it possible to compare "unknown" floppy dumps against the database to try to find entries that are partial matches.

Don't know how useful this would be until I'm done.. A simple change in one file on a floppy might change too many places to then be able to recognize the disk (at least rootblock and fileinfo)..

So far (this is the part that will store the crc32 of every block of existing files, but I have added the search function to it too - used for filtering here):

Code:

[... /tosec-php] # php-cli block_checksum.php name asm-one|more
4970540b9c5c1f21ffdd2f2368fc185a
Found 36 files matching the search for name = 'asm-one'

Commodore Amiga/Applications/Public Domain/[ADF]

  Name: ASM-One v1.05 (1991-11-10)(The Flame Arrows).adf
   MD5: f175c7d940f25fb953873ab6ee659fac
  SHA1: 29cd227ae9ba54a1f0d8e2cd70a85ef95640173d

... and more until it found a file I have in my own TOSEC-sorted dumps ...

Code:

  Name: ASM-One v1.20 (1993-09-19)(The Flame Arrows)(Disk 1 of 4)(Install)[HD, KS2].adf
   MD5: fd7de16f1f506c51aba0f76d3f117948
  SHA1: 5b28de1e4b2c22807ae1b591a4e64ebc4ccba807
File exists. Reading..
901120 bytes read into buffer...
Track 000 combined CRC32: 0x17412E0FB2AA7578787141303E87D8667E515549E871EEB6738AE26ED1CA07038C8A279BECD8E95A2885529D
Track 001 combined CRC32: 0xA30E79941C79BCEFC49828C622C0DE678965FE98938C3324E6DDFEAA40B89F2E8EF301836AB9DFC69CAD6DFA
Track 002 combined CRC32: 0xCFFD1369612101485F9A8F3A8B2440D9EF37EFC7180067B5A02CC69FA52B75D69469F7BEDE2E7B5883145E00
Track 003 combined CRC32: 0x737330DA0519265B58BB581D31AEEA7FC68B7D232E88D69E2E9366654C5141BA544303D68E55544C778F3E15

...

Track 157 combined CRC32: 0xE0DBAFA783E9135DE8C5F767119DCF1384AA218DDBED41ADE7E0496AAB0B56BFD32A06DB8CE73D4DD15BCC45
Track 158 combined CRC32: 0x9E419CDD145A4FDFA3CD34FB8EA8B986C44C52BBFFC9DD5C4DDAFAA2D8B456B8C0D39A241D98327222FF5EE7
Track 159 combined CRC32: 0xB959043D53D612E756B116D8BC0AB3FE5CB9412658C8F9DA8BD4BC630B3FAE716B64E55711D7E30B91211598
Track 160 combined CRC32: 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Track 161 combined CRC32: 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Track 162 combined CRC32: 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
Track 163 combined CRC32: 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

This is repeated for every known "Asm-One" disk I have dumped in my test-collection.

It is also capable to search/filter by md5 (but really of no use here other than for limiting to checksumming only one floppy)

I have decided to limit this tool to handle up to 164 tracks, and ignoring checksumming those extra blocks in the oversized dumps (since these will not come from any private dumps anyway).

peo · 09 August 2020, 13:31

A short one about CRC32 un-uniqueness:

Code:

MySQL [tosec]> select crc,count(crc) as c from roms group by crc having c > 1;
Empty set (4.86 sec)

MySQL [tosec]> select count(crc) as c from roms;
+-------+
| c     |
+-------+
| 86397 |
+-------+
1 row in set (0.37 sec)

CRC is not duplicated over the 86397 files in the database, but there might be some false matches when doing this for up to 1804 blocks per disk (155 million+ entries)

mai · 09 August 2020, 13:52

Quote:

Originally Posted by peo

A sneak-peak of what I'm into (unless this is already implemented in ADF-Workshop):
Besides the .dat to MySQL import I had since earlier (2.5 years), I'm now adding in checksums (crc32) of the individual blocks of each floppy. This will later (when I'm done thinking) make it possible to compare "unknown" floppy dumps against the database to try to find entries that are partial matches.

There is a

Quote:

TOP 5 based on track identification

feature in ADF-Workshop:

Quote:

CRC32 based identification :
Clean disk______*[122321CA] ASM-One v1.05 (1991-11-10)(The Flame Arrows)

TOP 5 based on track identification :
1) 74.4% [7F94E6D3] ASM-One v1.02 (1991)(DMV Verlag)[h Venture]
2) 68.8% [C0D6B4FC] ASM106
68.8% [C79DF167] ASM-One v1.07 (1992-03-21)(The Flame Arrows)
68.8% [E21F79A7] ASM-One v1.08b (1992-08-04)(The Flame Arrows)
68.8% [1B2D47FE] ASM-One v1.06 (1991-12-07)(The Flame Arrows)[h Cobra]
68.8% [3E4D0C25] ASMONE7B
68.8% [571243AD] ASM-One
3) ---,-%

peo · 09 August 2020, 22:26

Quote:

Originally Posted by mai

There is a feature in ADF-Workshop:

... once you have the correct databases in place ...

I noticed that there should be updated ADF-Workshop databases for the latest TOSEC release, but couldn't find these (used those in uploads/Crashdisk)..

How come that the disk we both used as an example was not detected 100% identical to any in the db ?

Do the algorithm in ADF-Workshop do the same thing I'm trying to accomplish (or actually finished some hours ago), or is it a closed-source secret how it works (what it does) ? There is probably a lot I could optimize, but I think running my script on a real computer is the first step (now: old QNAP 420 NAS)..

Code:

# cat /proc/cpuinfo
Processor name  : Feroceon 88F6282 rev 1 (v5l) @ 1.6 GHz
BogoMIPS        : 1587.60
Features        : swp half thumb fastmult edsp
CPU implementer : 0x56
CPU architecture: 5TE
CPU variant     : 0x2
CPU part        : 0x131
CPU revision    : 1

Hardware        : Feroceon-KW ARM
Revision        : 0000
Serial          : 0000000000000000

With only 220 entries in the database (and searching for a known file, the wrong way - by comparing checksums of every disk block):

Code:

# time php-cli analyze.php analyze Commodore\ Amiga/Applications/\[ADF\>
901120 bytes read into buffer...
Sorting result..
100% match with 4f0a2d6d90442022db7e1f741115a523
Found 1 files matching the search for md5 = '4f0a2d6d90442022db7e1f741115a523'

Commodore Amiga/Applications/[ADF]

  Name: Adorage v2.0 (demo) (1993)(proDAD)(AGA)(DE)[WB].adf
   MD5: 4f0a2d6d90442022db7e1f741115a523
  SHA1: c0934c711c42dcef232f279f70d70467de8cdd0a

real    0m9.720s
user    0m6.650s
sys     0m0.040s

mai · 09 August 2020, 22:53

Quote:

Originally Posted by peo

... once you have the correct databases in place ...

I noticed that there should be updated ADF-Workshop databases for the latest TOSEC release, but couldn't find these (used those in uploads/Crashdisk)..

You have obviously not read whole new TOSEC release thread, especially the last post by me:
http://eab.abime.net/showthread.php?t=103344
However Crashdisk should also update ADF-Workshop thread first post!

Quote:

Originally Posted by peo

How come that the disk we both used as an example was not detected 100% identical to any in the db ?

You are probably not that practiced in dealing with ADF-Workshop!
The Disk is detected in ADF-Workshop, also with old databases!

again:

Quote:

File selected : ASM-One v1.05 (1991-11-10)(The Flame Arrows).adf
File size : 901.120 bytes
CRC32 : $122321CA

Quote:

CRC32 based identification :
Clean disk______*[122321CA] ASM-One v1.05 (1991-11-10)(The Flame Arrows)

Green crc32 means the disk is known, either known to current TOSEC database or any additional database!

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Searching my small collection for Non-TOSEC ADFs and zone them	RoC	support.Other	10	31 August 2018 20:51
Some non-TOSEC ADFs	ascp	project.Amiga File Server	2	16 March 2012 06:24
Burseg's non-tosec adfs	oldpx	Amiga scene	58	29 November 2006 13:53
missing 3 files from TOSEC Educational ADFs	high-5	request.Old Rare Games	0	09 November 2005 14:56
Want TOSEC'd ADFs	Methanoid	MarketPlace	6	26 June 2003 14:40

09 August 2020, 10:35	#3
thomas Registered User Join Date: Jan 2002 Location: Germany Posts: 6,985	Extended ADFs hold the MFM bit streams rather than cooked data. 2MB size is usual. They cannot be read with trackdisk.device.

09 August 2020, 11:48	#4
chip Registered User Join Date: Oct 2012 Location: Italy Age: 49 Posts: 2,942	Interesting argument ..... i think extended ADF are OK More weird is the presence of oversized and undersized ones I suppose they could be removed when equivalent 880KB ADF is present too But i am probably missing something

09 August 2020, 12:18	#5
AMIGASYSTEM Registered User Join Date: Aug 2014 Location: Brindisi (Italy) Age: 70 Posts: 8,248	If the floppies are HD 1.76MB, and up to about 2MB with third party devices are Standard HD floppies that can be managed by AmigaOS, those posted above instead are Custom "UAE" floppies that are not visible by AmigaOS like those mentioned above. ---- Italiano ---- Se i floppy sono HD da 1,76MB, e fino a circa 2MB con device terzi sono floppy Standard HD gestibili da AmigaOS, quelli postati sopra invece sono Floppy Custom "UAE" che non sono visibili da AmigaOS come quelli ciatati sopra.

09 August 2020, 13:31	#7
peo Registered User Join Date: Dec 2008 Location: Ursviken Posts: 137	A short one about CRC32 un-uniqueness: Code: MySQL [tosec]> select crc,count(crc) as c from roms group by crc having c > 1; Empty set (4.86 sec) MySQL [tosec]> select count(crc) as c from roms; +-------+ \| c \| +-------+ \| 86397 \| +-------+ 1 row in set (0.37 sec) CRC is not duplicated over the 86397 files in the database, but there might be some false matches when doing this for up to 1804 blocks per disk (155 million+ entries)

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)