Req. CRC32 comparer

BippyM · 09 April 2006, 12:45

Now I have posted this in here (as opposed to the requests forum) as I am requesting a program (for the PC) that will help me with my tosec renaming. I need a program that will search through all my files and show me which use the same CRC32 (Clrmame isn't good enough for this).

I ask because I have thousands of adf's and I'm sure some share the same CRC32 (ie same game/program) and if I create a dat it'll have BOTH the named versions with the same crc32 in the same dat and wouldn't do anything :/

So can anyone help as I don't want to be renaming the same stuff and adding [a] when it isn't different!

killergorilla · 09 April 2006, 13:45

Try this, it seems comprehensive enough and is freeware, it also has a nice GUI too. Give it a look!

http://dff.nazrashid.com/index.php

BippyM · 09 April 2006, 13:59

Gave it a whirl and it is quite good tho not perfect!

Support for searching inside zip files needs adding.. but otherwise it seems quite useful

Thanks KG..

zardoz · 09 April 2006, 20:39

There's a c++ library on sourcesafe I use that's dead simple -

http://www.codeproject.com/cpp/crc32.asp

incidently, why do you need zip support? A zipped adf has a unique crc too (oh - but some of your adfs are unzipped?).

Ought to be easy to knock up something basic to do exactly what you want - pm me if you want something putting together.

BippyM · 09 April 2006, 20:41

basically my adf's are stored inside zips (tosec) so I need the crc of the adf not the zip!

zardoz · 09 April 2006, 21:48

yebbut... the crc alorithm doesn't care whether it's zipped or not. The only benefit of unzipping it first would be if there was a non-zipped adf you needed to compare your zipped adf with ('cos zipped, they'd have the same crc if they were the same file to start with).

[EDIT] but if the filenames in the zips were different then the crcs would be different even though the adfs are the same. Right. Enough said.

adolescent · 09 April 2006, 23:04

That also assumes (incorrectly) that the same compression, program used, Zip version, etc was used. Not a good idea if you want accurate results.

BippyM · 10 April 2006, 02:29

I don't want the crc of the zip file but the crc of what is in the zip file...

so unpack zip to temp folder... get crc checksum of adf thats been decompressed!

Duke · 10 April 2006, 03:02

I have attached a dupe finder I started two write about two years ago. It's written in java and surely has some bugs, but it can search inside zips (and zips inside zips etc.) and output all dupes to a logfile.

Use it like this:

Duplex.bat <directory to scan>

Example output:

c3239276 testdir\dir1\amberstar__1.zip|amberstar_1.zip|amberstar_1.adf
c3239276 testdir\dir2\empty4\amberstar_1.adf
c3239276 testdir\dir1\amberstar_1.adf

adolescent · 10 April 2006, 08:24

Quote:

Originally Posted by bippym

so unpack zip to temp folder... get crc checksum of adf thats been decompressed!

You can always use the CRC that is stored in the zipfile itself. That will save you a step.

BippyM · 10 April 2006, 12:54

Quote:

Originally Posted by adolescent

You can always use the CRC that is stored in the zipfile itself. That will save you a step.

Yeah but then i'd have to manually compare adf's... comparing 20000 adf's by hand doesn't sound pleasant.

@Duke.. cheers i'll give this a whirl a bit later

mr_0rga5m · 10 April 2006, 14:19

I've knocked summat up (in delphi) that will do just what you want bippym (including peeking zips) .. It scans then groups by same crc32's. I'll up it to ya later when i'm in from work.

BippyM · 10 April 2006, 15:18

Okay matey!

Sounds great

adolescent · 10 April 2006, 20:11

Quote:

Originally Posted by bippym

Yeah but then i'd have to manually compare adf's... comparing 20000 adf's by hand doesn't sound pleasant.

Of course. I just meant that writing a program that extracts the file and calculates it's CRC isn't necessary (unless you really hate your HD or want the program to take more time than necessary) since the CRC is already stored in the file header inside the zip.

mr_0rga5m · 10 April 2006, 23:41

Ok bippy .. Here it is .. See what you think.

And as adolescent said there is no need to calc the crc in a zip because its stored in the header. This proggy only calcs files crc's, Zips get peeked and use the header detail. The only downside i can see here is corrupt zips, but this should be far and few.

mr_0rga5m · 11 April 2006, 20:31

Any use bippy / anybody ?

Mods needed ?

BippyM · 11 April 2006, 22:39

yes it is useful.. I'm just thinking..

Maybe the option to select and then remove/delete or move to another folder

It was bloody quick.. does it check inside subfolders etc?

mr_0rga5m · 11 April 2006, 22:41

Subfolders in folders or zips .. neither atm .. but i suppose it could

Actually i just tested and by mis-design it does do subfolders in zips .. so i just need to add an option to do subfolders in folders.

BippyM · 11 April 2006, 22:44

yes.. that be why it was so quick lol

Subfolders/zips and support for removing files (So I don't have to search for them)

09 April 2006, 12:45	#1
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	Req. CRC32 comparer Now I have posted this in here (as opposed to the requests forum) as I am requesting a program (for the PC) that will help me with my tosec renaming. I need a program that will search through all my files and show me which use the same CRC32 (Clrmame isn't good enough for this). I ask because I have thousands of adf's and I'm sure some share the same CRC32 (ie same game/program) and if I create a dat it'll have BOTH the named versions with the same crc32 in the same dat and wouldn't do anything :/ So can anyone help as I don't want to be renaming the same stuff and adding [a] when it isn't different!

11 April 2006, 22:41	#18
mr_0rga5m Tik Gora :D Join Date: Oct 2001 Location: Round yo momma's Posts: 1,273	Subfolders in folders or zips .. neither atm .. but i suppose it could Actually i just tested and by mis-design it does do subfolders in zips .. so i just need to add an option to do subfolders in folders. Last edited by mr_0rga5m; 11 April 2006 at 22:49.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
CRC32 , MD5 , SHA - 1 Curiosity	Kyon	support.Other	38	09 June 2009 10:08
Unknown floppies: how to identify them using their checksums/CRC32?	Supamax	request.Other	5	13 May 2009 19:08
REQ: Hybris	Adderly	request.Modules	8	02 June 2006 20:27
REQ: Any of the following:	rob72	request.Old Rare Games	6	21 January 2004 22:50

09 April 2006, 13:45	#2
killergorilla Lesser Talent Join Date: Jan 2003 Location: UK Age: 42 Posts: 7,957	Try this, it seems comprehensive enough and is freeware, it also has a nice GUI too. Give it a look! http://dff.nazrashid.com/index.php

09 April 2006, 13:59	#3
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	Gave it a whirl and it is quite good tho not perfect! Support for searching inside zip files needs adding.. but otherwise it seems quite useful Thanks KG..

09 April 2006, 20:39	#4
zardoz Zone Friend Join Date: Oct 2004 Location: Wales Age: 53 Posts: 163	There's a c++ library on sourcesafe I use that's dead simple - http://www.codeproject.com/cpp/crc32.asp incidently, why do you need zip support? A zipped adf has a unique crc too (oh - but some of your adfs are unzipped?). Ought to be easy to knock up something basic to do exactly what you want - pm me if you want something putting together.

09 April 2006, 20:41	#5
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	basically my adf's are stored inside zips (tosec) so I need the crc of the adf not the zip!

09 April 2006, 21:48	#6
zardoz Zone Friend Join Date: Oct 2004 Location: Wales Age: 53 Posts: 163	yebbut... the crc alorithm doesn't care whether it's zipped or not. The only benefit of unzipping it first would be if there was a non-zipped adf you needed to compare your zipped adf with ('cos zipped, they'd have the same crc if they were the same file to start with). [EDIT] but if the filenames in the zips were different then the crcs would be different even though the adfs are the same. Right. Enough said.

09 April 2006, 23:04	#7
adolescent Powered by Motorola Join Date: Feb 2002 Location: Redondo Beach, CA Age: 52 Posts: 1,065	That also assumes (incorrectly) that the same compression, program used, Zip version, etc was used. Not a good idea if you want accurate results.

10 April 2006, 02:29	#8
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	I don't want the crc of the zip file but the crc of what is in the zip file... so unpack zip to temp folder... get crc checksum of adf thats been decompressed!

10 April 2006, 14:19	#12
mr_0rga5m Tik Gora :D Join Date: Oct 2001 Location: Round yo momma's Posts: 1,273	I've knocked summat up (in delphi) that will do just what you want bippym (including peeking zips) .. It scans then groups by same crc32's. I'll up it to ya later when i'm in from work.

10 April 2006, 15:18	#13
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	Okay matey! Sounds great

11 April 2006, 20:31	#16
mr_0rga5m Tik Gora :D Join Date: Oct 2001 Location: Round yo momma's Posts: 1,273	Any use bippy / anybody ? Mods needed ?

11 April 2006, 22:39	#17
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	yes it is useful.. I'm just thinking.. Maybe the option to select and then remove/delete or move to another folder It was bloody quick.. does it check inside subfolders etc?

11 April 2006, 22:44	#19
BippyM Global Moderator Join Date: Nov 2001 Location: Derby, UK Age: 48 Posts: 9,355	yes.. that be why it was so quick lol Subfolders/zips and support for removing files (So I don't have to search for them)

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)