new online TOSEC tool

[Steffest]

Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/

I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?

Ideally I would like the tool to return stuff like "This disk matches xxxx.yyyy.adf for 95% so it's probably this disk" instead of only the black&white match/no match.
That way you can actually see if you have found something rare or not.

[DamienD]

Originally Posted by Steffest

Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/

Wow!!!

In the past I've provide "combined" TOSEC .DATs to assist people but I'm no coder and alas these still need to be run the old fashioned way; via ClrMAMEPro:

Amiga - Latest Public DATS Combined (TOSEC-v2007-03-14) & (TOSEC-v2017-04-23)

[emufan]

nice tool

#1) ADF workshop is able to identify similiar disks,
it lists the correct tosec diskname, or lists disks close to it sorted by percentages.

[BippyM]

Originally Posted by Steffest

I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?

I seem to remember there was a tool that did this, however I am unsure of it right now.

I guess it wouldn't be that difficult to code something that does a byte comparison on the adf and let's you know the difference %, num bytes different etc... Worth pondering I suppose

Edit in fact yes I do remember see the screenshot

[chip]

How TOSEC team choose amongst the various slight variations of a disk ?

In other words, when a disk tagged with [a] is inserted in the archive ?

[Steffest]

Originally Posted by emufan

#1) ADF workshop is able to identify similiar disks,
it lists the correct tosec diskname, or lists disks close to it sorted by percentages.

Hmm, Can't seem to find that particular option.
Do you mean there are fingerprints other then the usual CRC/MD5/SH1 in those .db files? Or does it do a binary compare on 2 adfs ?
I guess I should ask Crashdisk about this.

[Steffest]

Originally Posted by BippyM

I guess it wouldn't be that difficult to code something that does a byte comparison on the adf and let's you know the difference %, num bytes different etc... Worth pondering I suppose

Edit in fact yes I do remember see the screenshot

Ah, thanks for the screenshot.
I would like to avoid a full binary compare though, as then the entire ADF would have to be transfered to/from the server.
And even then: I you do a disksalv or disk optimisation on a DOS disk, it still would be the same disk but it would be binary totally different.

A "simple" file list compare or a track hash compare would be much faster, but setting up these kinds of fingerprint databases for 80.000 disks would be a little out of scope for this project for now.
I guess I'll have to bug the guys on the TosecDev forums for this kind of data.

[mai]

Originally Posted by Steffest

Hmm, Can't seem to find that particular option.
Do you mean there are fingerprints other then the usual CRC/MD5/SH1 in those .db files? Or does it do a binary compare on 2 adfs ?

Alcatraz demo.adf within part 64

File selected : Alcatraz demo.adf
File size : 901.120 bytes
CRC32 : $BD7ED8A3
File loaded as an ADF : AmigaDOS OFS

CRC32 based identification :
Special disk____*[BD7ED8A3] I've Got the Power (1991)(Alcatraz)

TOP 5 based on track identification :
1) 99.4% [FFB491CB] I've Got the Power (1991)(Alcatraz)[b dump]
2) 76.3% [0E2842B5] alcatraz.power.music.

Its a TOSEC disk and there are a few known similar disks according to the database!
Isnt this, what you are looking for?

BTW: To be up to date you have to download database of current TOSEC release!

-----
Anyway, nice online tool for quick and easy checking ADFs.

[Steffest]

Originally Posted by mai

Its a TOSEC disk and there are a few known similar disks according to the database!
Isnt this, what you are looking for?

Ah ... yes indeed.
So that database DOES contain track based disk fingerprints!
Thanks, I'll dig into that.

[mai]

Database of current TOSEC release is here:
http://eab.abime.net/showthread.php?t=64668
TOSEC-v2017-11-01.db

[chip]

Your TOSEC tool is nice, but can i suggest you one thing ?

At the moment your tool miss the support for .dms files

This kind of disks are so common that i guess they should be supported by your tool

Anyway, no idea of the effort needed for such kind of improvement

[DamienD]

BS1 Collection - awesome looking haul of disks is the wrong thread for this discussion chip.

Moved accordingly

[Steffest]

Originally Posted by chip

At the moment your tool miss the support for .dms files

This kind of disks are so common that i guess they should be supported by your tool

Sure, and plain .zip, .adz and .7z files too ...
One step at a time :-)
It will never be a batch converter though, there are other/better options for that.
Dropping 200 .dms files in one go would probably kill your browser.

[chip]

Sure Steffest, one step at a time

And since we are talking of this, do you have intention to create also a standalone application for such kind of TOSEC check ?

[DamienD]

Originally Posted by chip

do you have intention to create also a standalone application for such kind of TOSEC check ?

If you are going to check thousands and thousands of files, I don't know why you don't just use ClrMAMEPro and the latest combined dats?

Again, see these threads:

... ADFs - renaming / comparing / finding out if you have anything rare!!!
... Amiga - Latest Public DATS Combined (TOSEC-v2007-03-14) & (TOSEC-v2017-04-23)

[chip]

No, i'm not going to check thousands and thousands of files, it's not that my intention

I only guessed that such kind of operation (checking few disks against TOSEC) could be done also with a standalone application

I'm still not able to use ADF-Workshop to do that, even if i know there's a way to do it ..... my fault

[mai]

Originally Posted by chip

I'm still not able to use ADF-Workshop to do that, even if i know there's a way to do it

You have disqualified yourself to get further Explanations how to use this tool, because of some statements!

[ascp]

Originally Posted by Steffest

Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/

I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?

Ideally I would like the tool to return stuff like "This disk matches xxxx.yyyy.adf for 95% so it's probably this disk" instead of only the black&white match/no match.
That way you can actually see if you have found something rare or not.

Handy tool, but at the moment it just stops at "checking tosec". I tried latest chrome and firefox and with a few different adf-files.

[Steffest]

Originally Posted by ascp

Handy tool, but at the moment it just stops at "checking tosec". I tried latest chrome and firefox and with a few different adf-files.

Fixed!

[demolition]

Originally Posted by Steffest

So I made one - all in the bowser, no installing needed:
http://www.amigaclub.be/tosec/

Thanks a lot for the lovely tool - this was just what I was looking for after having dumped a bunch of floppies.


One problem for me is that many of these disks that I dumped had been copied using something like Bamcopy so unused sectors were left untouched thus invalidating the checksum.. I know that this has nothing to do with your tool, but it does mean a lot of extra work to verify the disks against the versions in TOSEC.