English Amiga Board


Go Back   English Amiga Board > Other Projects > project.TOSEC (amiga only)

 
 
Thread Tools
Old 29 January 2018, 23:57   #1
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
new online TOSEC tool

Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/

I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?

Ideally I would like the tool to return stuff like "This disk matches xxxx.yyyy.adf for 95% so it's probably this disk" instead of only the black&white match/no match.
That way you can actually see if you have found something rare or not.
Steffest is offline  
Old 30 January 2018, 00:04   #2
DamienD
Global Moderator

DamienD's Avatar
 
Join Date: Aug 2005
Location: Australian in London
Age: 42
Posts: 12,575
Quote:
Originally Posted by Steffest View Post
Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/
Wow!!!

In the past I've provide "combined" TOSEC .DATs to assist people but I'm no coder and alas these still need to be run the old fashioned way; via ClrMAMEPro:

Amiga - Latest Public DATS Combined (TOSEC-v2007-03-14) & (TOSEC-v2017-04-23)
DamienD is online now  
Old 30 January 2018, 00:30   #3
emufan
Registered User
 
Join Date: Feb 2012
Location: #DrainTheSwamp
Posts: 4,546
nice tool

#1) ADF workshop is able to identify similiar disks,
it lists the correct tosec diskname, or lists disks close to it sorted by percentages.
emufan is offline  
Old 30 January 2018, 01:31   #4
BippyM
Registered User

BippyM's Avatar
 
Join Date: Nov 2001
Location: Nottingham, UK
Age: 43
Posts: 8,695
Quote:
Originally Posted by Steffest View Post
I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?
I seem to remember there was a tool that did this, however I am unsure of it right now.

I guess it wouldn't be that difficult to code something that does a byte comparison on the adf and let's you know the difference %, num bytes different etc... Worth pondering I suppose

Edit in fact yes I do remember see the screenshot
Attached Thumbnails
Click image for larger version

Name:	005.png
Views:	163
Size:	20.5 KB
ID:	56520  

Last edited by BippyM; 30 January 2018 at 01:37.
BippyM is offline  
Old 30 January 2018, 07:47   #5
chip
Registered User
chip's Avatar
 
Join Date: Oct 2012
Location: Italy
Age: 44
Posts: 769
How TOSEC team choose amongst the various slight variations of a disk ?

In other words, when a disk tagged with [a] is inserted in the archive ?
chip is offline  
Old 30 January 2018, 11:53   #6
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
Quote:
Originally Posted by emufan View Post

#1) ADF workshop is able to identify similiar disks,
it lists the correct tosec diskname, or lists disks close to it sorted by percentages.
Hmm, Can't seem to find that particular option.
Do you mean there are fingerprints other then the usual CRC/MD5/SH1 in those .db files? Or does it do a binary compare on 2 adfs ?
I guess I should ask Crashdisk about this.
Steffest is offline  
Old 30 January 2018, 11:58   #7
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
Quote:
Originally Posted by BippyM View Post

I guess it wouldn't be that difficult to code something that does a byte comparison on the adf and let's you know the difference %, num bytes different etc... Worth pondering I suppose

Edit in fact yes I do remember see the screenshot
Ah, thanks for the screenshot.
I would like to avoid a full binary compare though, as then the entire ADF would have to be transfered to/from the server.
And even then: I you do a disksalv or disk optimisation on a DOS disk, it still would be the same disk but it would be binary totally different.

A "simple" file list compare or a track hash compare would be much faster, but setting up these kinds of fingerprint databases for 80.000 disks would be a little out of scope for this project for now.
I guess I'll have to bug the guys on the TosecDev forums for this kind of data.
Steffest is offline  
Old 30 January 2018, 17:04   #8
mai
Moderator
mai's Avatar
 
Join Date: Feb 2008
Location: Stendal/Germany
Age: 50
Posts: 4,776
Quote:
Originally Posted by Steffest View Post
Hmm, Can't seem to find that particular option.
Do you mean there are fingerprints other then the usual CRC/MD5/SH1 in those .db files? Or does it do a binary compare on 2 adfs ?
Alcatraz demo.adf within part 64
Quote:
File selected : Alcatraz demo.adf
File size : 901.120 bytes
CRC32 : $BD7ED8A3
File loaded as an ADF : AmigaDOS OFS
Quote:
CRC32 based identification :
Special disk____*[BD7ED8A3] I've Got the Power (1991)(Alcatraz)

TOP 5 based on track identification :
1) 99.4% [FFB491CB] I've Got the Power (1991)(Alcatraz)[b dump]
2) 76.3% [0E2842B5] alcatraz.power.music.
Its a TOSEC disk and there are a few known similar disks according to the database!
Isnt this, what you are looking for?

BTW: To be up to date you have to download database of current TOSEC release!

-----
Anyway, nice online tool for quick and easy checking ADFs.

Last edited by mai; 30 January 2018 at 17:26.
mai is offline  
Old 30 January 2018, 17:33   #9
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
Quote:
Originally Posted by mai View Post

Its a TOSEC disk and there are a few known similar disks according to the database!
Isnt this, what you are looking for?
Ah ... yes indeed.
So that database DOES contain track based disk fingerprints!
Thanks, I'll dig into that.
Steffest is offline  
Old 30 January 2018, 17:58   #10
mai
Moderator
mai's Avatar
 
Join Date: Feb 2008
Location: Stendal/Germany
Age: 50
Posts: 4,776
Database of current TOSEC release is here:
http://eab.abime.net/showthread.php?t=64668
TOSEC-v2017-11-01.db
mai is offline  
Old 31 January 2018, 20:42   #11
chip
Registered User
chip's Avatar
 
Join Date: Oct 2012
Location: Italy
Age: 44
Posts: 769
Your TOSEC tool is nice, but can i suggest you one thing ?

At the moment your tool miss the support for .dms files

This kind of disks are so common that i guess they should be supported by your tool

Anyway, no idea of the effort needed for such kind of improvement

Last edited by chip; 31 January 2018 at 20:49.
chip is offline  
Old 31 January 2018, 20:46   #12
DamienD
Global Moderator

DamienD's Avatar
 
Join Date: Aug 2005
Location: Australian in London
Age: 42
Posts: 12,575
BS1 Collection - awesome looking haul of disks is the wrong thread for this discussion chip.

Moved accordingly
DamienD is online now  
Old 01 February 2018, 05:55   #13
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
Quote:
Originally Posted by chip View Post
At the moment your tool miss the support for .dms files

This kind of disks are so common that i guess they should be supported by your tool
Sure, and plain .zip, .adz and .7z files too ...
One step at a time :-)
It will never be a batch converter though, there are other/better options for that.
Dropping 200 .dms files in one go would probably kill your browser.
Steffest is offline  
Old 01 February 2018, 06:48   #14
chip
Registered User
chip's Avatar
 
Join Date: Oct 2012
Location: Italy
Age: 44
Posts: 769
Sure Steffest, one step at a time

And since we are talking of this, do you have intention to create also a standalone application for such kind of TOSEC check ?
chip is offline  
Old 01 February 2018, 12:09   #15
DamienD
Global Moderator

DamienD's Avatar
 
Join Date: Aug 2005
Location: Australian in London
Age: 42
Posts: 12,575
Quote:
Originally Posted by chip View Post
do you have intention to create also a standalone application for such kind of TOSEC check ?
If you are going to check thousands and thousands of files, I don't know why you don't just use ClrMAMEPro and the latest combined dats?

Again, see these threads:

... ADFs - renaming / comparing / finding out if you have anything rare!!!
... Amiga - Latest Public DATS Combined (TOSEC-v2007-03-14) & (TOSEC-v2017-04-23)
DamienD is online now  
Old 01 February 2018, 12:24   #16
chip
Registered User
chip's Avatar
 
Join Date: Oct 2012
Location: Italy
Age: 44
Posts: 769
No, i'm not going to check thousands and thousands of files, it's not that my intention

I only guessed that such kind of operation (checking few disks against TOSEC) could be done also with a standalone application

I'm still not able to use ADF-Workshop to do that, even if i know there's a way to do it ..... my fault
chip is offline  
Old 07 February 2018, 18:18   #17
mai
Moderator
mai's Avatar
 
Join Date: Feb 2008
Location: Stendal/Germany
Age: 50
Posts: 4,776
Quote:
Originally Posted by chip View Post

I'm still not able to use ADF-Workshop to do that, even if i know there's a way to do it
You have disqualified yourself to get further Explanations how to use this tool, because of some statements!
mai is offline  
Old 27 June 2018, 15:11   #18
ascp
Pastafarian

 
Join Date: Sep 2010
Location: Uppsala/Sweden
Posts: 283
Quote:
Originally Posted by Steffest View Post
Hey,
I've been busy the past week helping with the imaging of the Bamiga Sector One Collection from physical disks to ADF.
Doing that I had to dig around to find the correct TOSEC tools and .dat files to check if the disks where already known or not.
Installing all this is fine if you really are into collecting, but I missed a quick and easy way to quickly check a few disks.
So I made one - all in the bowser, no installing needed:

http://www.amigaclub.be/tosec/

I have a question though: Currently only the md5 hash of the disk is checked.
In the BS1 collection, this results in HEAPS of disks not already in TOSEC, while in most cases it's just a slight variation of a disk that is already present.
Is there a way to quickly identify these "alt" versions?
Maybe by comparing file hashes or track/sector hashes?
Are there any existing databases for this or should I just create one myself?

Ideally I would like the tool to return stuff like "This disk matches xxxx.yyyy.adf for 95% so it's probably this disk" instead of only the black&white match/no match.
That way you can actually see if you have found something rare or not.

Handy tool, but at the moment it just stops at "checking tosec". I tried latest chrome and firefox and with a few different adf-files.
ascp is offline  
Old 28 June 2018, 09:18   #19
Steffest
Registered User

 
Join Date: Jan 2017
Location: Antwerp / Belgium
Posts: 70
Quote:
Originally Posted by ascp View Post
Handy tool, but at the moment it just stops at "checking tosec". I tried latest chrome and firefox and with a few different adf-files.
Fixed!
Steffest is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Amiga - Latest Public DATS Combined (v2018-07-27) & (v2018-04-01) & (v2017-04-23) DamienD project.TOSEC (amiga only) 21 05 August 2018 13:35
Online "retro" image tool AnimaInCorpore Coders. General 71 23 May 2018 20:40
HEAVY BBS now online [Telnet] Over 700 ADF files online Dougal Retrogaming General Discussion 4 05 July 2016 02:53
TOSEC - DAT Pack - Complete (1111) (TOSEC-v2009-12-27) Aral project.TOSEC (amiga only) 8 30 January 2010 12:14
PD Tosec online! (come and download folks!) Ironclaw News 27 03 October 2007 22:34

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 22:58.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.
Page generated in 0.09615 seconds with 14 queries