ScanDeep 0.8

[hawui1]

ScanDEEP 0.8 beta2 - (c)2001 Ralf 'hippie2000'
Anybody has this utility?

here a brief description from the author

http://hippie2000.back2roots.org/sca...candeep.readme

it seems that with this utility you can automatically scan loads of ADF files and the utility will create a log where you can quickly find out if an .ADF image is corrupt, if it's infected etc.. I was unable to find it anywhere.. it should be (I guess) freeware..

I tried also to contact the author but no luck..

[emufan]

welcome.
just yesterday i came across this readme, funny somehow
unfortunatly i couldn't find it anywhere. since noone is writing such a readme for personal usage only, i still have hope, someone may have found it in the past.
well, not much help, but a *bump*

[1time]

I have been looking for this tool. Would make my life so much easier.

[Wiking67]

Have also been looking for it

[hippie2000]

Hello Well, yes, this is my tool I coded vor back2roots.org internal mass scanning.
It was never released so you can't find it in the net. I even forgot I put this readme online.
It is still unfinished and likely it will always be. It works fine for most disk images - but it fails on cue bin cd images
It can handle iso in zip if the amiga has enough memory to store *both* in memory - at the time ow writing it not even uae could do this.
I started with scannng for filesystems in mfm disks (ext adf) but this part is unfinished too...
But it works fine for everything which fits in amigas memory - used it to scan 100.000 dms and adf archives...
All disk images on our ftp have a txt file near them made by ScanDeep ftp://ftp.back2roots.org/pub/back2roots/disks/
ScanDeep can only scan archives and filesystems on r/w filesystem
Well - if theres a high interest i may redecide and release the current state - theres nothing like it thats why i wrote it
The idea was to index each and every file on any medium and make a cross reference search engine. Thats why it crc sums each file.
I coded a Perl parser to read the txt files into a mysql db but the indexing and search core is unfinished too (just local tool for daily bttr research) Since my interests are wide today updates are likely not to happen soon. I can release it here on an as is basis and would be interested in receiving your resulting txt collection

http://hippie2000.back2roots.org/scandeep/scandeep.zip

enjoy - greets - hippie2000

Ahh - and fyi - this was written in E and it was never testet on CPUs above 030

[Wiking67]

Thx

[1time]

Thanks!!!

[emufan]

nice x-mas gift. thanks hippie2000

[hippie2000]

quite some info is missing in the docs

path can not only be a folder or an adf, it should also handle a zip or lha file. best to be used as a dopus button (needless to say)

all scans are performed in alphabetically sorted order - and only files which lack a corresponding txt file are scanned. if it crashes it continues with the same file if called again.

crashes are likely on ndos disks with fragments of deleted compressed files. xfd tries to decrunch them and some decrunchers easily crash with invalid data. a planned option to defeat recovery scan on ndos disks isnt implemented yet (but this is no big issue to add).

in this case just put an empty txt file near the unscannable file and it should be skipped. and in opposite just delete a txt to rescan the corresponding file when you replace it with a clean version. a zero bytes txt can be used to trigger a rescan with lowered depth in future. the lowest possible would be to just scan sectors for ndos disks...

if i find more info to note i will post it here.

i was also working on automatted image and mod export and initial caps support but atm i dont remember on which computer this experimental version was

don't hesitate to ask if you find something out / need support

i have subscribed to this thread to get instant email - you may as well reach me at metamonk@yahoo.com

if you find free disk series not yet online on our ftp let me know

if you need space to make your scans public let me know too - google indexes .txt and we can search for rare files then...

remember how you found the readme (which is not linked at back2roots)

[cosmicfrog]

wow thanks hippie2000 may back2roots never dry up or fred will have to hop over to my pond

[emufan]

i uploaded some tools disks into the zone, i found on those BBS images and elsewhere.
they should fit into "/pub/amigascne/UtilityDisks/". but your folder is not uptodate. compare with: ftp.amigascne.org/pub/amiga/UtilityDisks/
need a sync.
the tool works very well. it does also scan diskimages within a given adf/dms. thats nice too.
too bad most programers didn't create a version string. would help on such menu driven disks with filenames 1, 2 ,3 ... but not a fault of your tool - it gives the version string where available. cool tool

[hawui1]

Hippie2000, what can I say !? Thanks thanks thanks thanks.. !!!!!
this tool will help me A LOT .. I have many .ADF I scanned with my Kryoflux and everytime I have to deal with my "friend" ByteBandit or with disks that were not dumped correctly.. so previously I had to test them one by one.. a very boring task.. with your tool it's a breeze to scan them in a few seconds and get immediately a clue..

I tried to contact you with your email address (but who knows, maybe it's no longer active) however in the end here it is the little jewel.. !

Too bad you decided not to upgrade it anymore, I think this tool could be *very useful* for the AMIGA community.. with all the thousands of .ADF floating around having a tool able to make massive scans preparing a log makes a LOT of difference..

Thanks once more.. I will put the little thing at work on my collection this evening..

[hippie2000]

thanks for the nice words!

hehe - cosmicfrog

before i forget it

ive started an index of the contents of the disks on our ftp at:

http://disks.back2roots.org/

its script generated (perl as entire back2roots) - however - it was planned to integrate scandeep results, which did not happen so far...

and yes - i try to keep all online and still upload stuff to the ftp which we lost during the last disk crash - 10+ years of work are worth to survive...

emufan - ohh yes the amigascne mirror is horribly outdated - and i had the intention to spend it txt files too - but im still restoring our stuff...

hawui1 - you see - you decided to join here and made something happen - even before your second posting - thats what i call quite efficient

it took me a year to code and test scandeep - and its nice to release unique amiga 68k software in december 2012

regarding update - its tested in depth and returns good results on sane disks - the update i abandoned for now was an integrated modripper and the integration of the portable amiga gfx converters i coded - see:

http://hippie2000.back2roots.org/ - just have a look at the supported formats: http://hippie2000.back2roots.org/gra...ilbmtoppm.html

btw - this is cgi capable installed on our apache server and can display nearly any amiga gfx (including icons) in a browser on the fly using mod_rewrite

however - this is a c code commandline executable and needs to be a library first to be added to scandeep - which is quite some work...

it also was designed to be included in imagemagick and netpbm - which is way less work-...

my problem iss i would need years stopping development to release all stuff i coded...

[hippie2000]

i had a look in the source code of the 0.8b2 compile and found some more info on implemented further libraries.

theres support for datamaster.library which is disabled since it reduced stability (thats already in the doc but can easily turn an option since already implemented).

3 further libraries are supported but disabled since they are replaced by newer libraries. these were filevirus and bootblock.libray (superceded by xvs) and unpack.library (superceded by xfd/xad/xpk). these libs are ignored even if you have them in libs. should you find any cruncher which unpack.library can handle but xfd/xad/xpk cant i can add an option to activate it again. its opensource pure asm - see http://lclevy.free.fr/packers/

theres an undocumented early support for the capsimage.device 1.1, which doesnt do much besides listing some trackinfo. its activated but highly experimental/pretty untested. theres also an early framework of an mfm level "filesystem" to enter caps and adf ext images for further integrity checks and virus testing but this is not working yet. this should not be a problem unless you scan a caps image. caps support should turn an option with default off.

one remaining external library was the mod ripper i named, based on xfrmaster.library (eXternal Format Rippers) - a xsystem library allowing scanning arbitary files for mods. see http://otaku.onlinehome.de/xfrmaster.readme

its written in pure asm and supports 29 external clients for module formats in the above version, see
http://www.psi5.com/~silva/afilter/#xfrclients

the intention was to enter whatever file like an archive and list mods in there, and then handle each mod again like an archive containing samples. this is pretty difficult since many formats lack a size calculation, rendering a crc of the remainder of a file useless.

the detection part already works but ive disabled the support since it horribly reduced performance (when scanning in its deep mode, which is pretty natural for what it does). besides this the archive alike lister is not implemented yet. this should turn another option to enable...

ahh - and if called with a filename as the first argument scandeep ignores all files which do not have one of these file extensions: adf dms zip lha ipf. this is not very elegant for a program which can detect zillions of amiga filetypes - i must admit

ahh2: amigados comments are listed in the error (###) / warning (!!!) slot prepended with colons (::: )

[1time]

hippie2000: Awesome tool. Thanks a lot for sharing it!

[hippie2000]

here's an example of a non bootable ndos disk (in fact not even a sane data disk) which is scanned in recovery mode:

ftp://ftp.back2roots.org/pub/back2ro...sionaga-01.txt

it shows numerous errors but still finds a lot of data - showing this disk could be salved - at least partially.

the track listing at the end shows the disk has the first track cleared on both sides, it obviously was bootable initially (like all other asi tools)

such disks likely crash scandeep on truncated crunched files (xfd). this didnt happen since they all were intact. you see xadmaster safely reports an error on the truncated lha file in there. no hunk errors are reported which means that no executable is truncated.

the block allocated twice errors are sorted to files which REused a block. theres no way to find out who owns it and who doesnt besides the other vality tests of a file. this means that the allocated twice errors dont show which file is the evil one in recovery mode. other block related errors do. like the last 4 here:

  Viewtek_2.0/                    Dir ----rwed 02-Sep-1992 13:07:50.24                                       
    DropBox                     14824 ----rwed 16-Oct-1993 13:54:01.22 ........                             
      !!! Blk 0013 (FileData) - block referred twice (after 0 bytes)                                         
      !!! Blk 0013 (FileData) - TYPE not T_DATA (after 0 bytes)                                              
      !!! Blk 0013 (FileData) - wrong DATASIZE (after 0 bytes)                                               
      !!! Blk 0013 (FileData) - wrong HEADER key (after 0 bytes)                                             
      !!! Blk 0013 (FileData) - wrong SEQNUM (after 0 bytes)

[emufan]

one sugestion: "Scan conditions" should always list all support libraries - so you can see what is missing.

[hippie2000]

the scan conditions are always listed at the end of each scan emufan including versions and amount of external clients, ie:

  Scan conditions                                                                  
  -------------------------------------------------------------------------------- 
  ScanDEEP 0.8b2           21.03.2004 -                                            
  xvs.library 33.42        16.05.2004 804 viri (326 boot, 374 file, 104 link)      
  xadmaster.library 12.1   28.09.2003 120 archivers (80 disk, 27 file, 15 filesys) 
  xfdmaster.library 39.15  09.03.2003 300 packers (141 int, 159 ext)               
  xpkmaster.library 5.2    08.09.1999 36 packers (34 pack, 0 arc, 10 crypt)        
  FileID.library 8.0       9.7.98     645 filetypes (645 by data)                  
  Idm.library 1.144        28.2.99    1161 filetypes, (1161 by data)               
                                                                                   
  --------------------------------------------------------------------------------

[emufan]

yes, but only if available. here is a scan using wb3.1 workbenchdisk, without any support libraries:

ScanDEEP 0.8 beta2 - (c)2001 Ralf 'hippie2000' Steines 
  For legal ADFs check out http://www.back2roots.org     


  File "PaNorAmA Toolkit - 26 November 1986.zip" (1.009 secs)                             
  --------------------------------------------------------------------------------------- 
  "PaNorAmA Toolkit - 26 November 1986.zip" 229675 ----rwed 17-Aug-2012 23:39:24.00 939925D4 d  
                                                                                          
  --------------------------------------------------------------------------------------- 


  Scan conditions                              
  -------------------------------------------- 
  ScanDEEP 0.8b2       21.03.2004 -            
                                               
  --------------------------------------------

this way, the user has no idea, which libraries are supported. you could add something like: "xpkmaster.library - n/a"

[hippie2000]

naah - i expect that little rtfm before using it - its absolutely modular and if someone decides to drop a feature its not missing just deactivated

besides that - those (few?) who take the time to seriously scan for viruses are likely not too lazy to read a list of 6 optional libraries