English Amiga Board Amiga Lore



Go Back   English Amiga Board > Other Projects > project.TOSEC (amiga only)

 
 
Thread Tools
Old 28 March 2009, 21:18   #1
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2007-03-14_CM)

Following on from this thread, I've created these "combined" DAT files with "rebuildto directories" from the latest public TOSEC Commodore Amiga DAT files:


Commodore Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2007-03-14_CM).dat
- Contains all Commodore Amiga DATS.
- Download it from here.
- Note; see post #3 below.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [Excluding CD - CD32 - CDTV] (TOSEC-v2007-03-14_CM).dat
- Contains all Commodore Amiga DATS, excluding the CD / CD32 / CDTV DATS.
- Download it from here.
- Note; see post #3 below.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [ADF Only] (TOSEC-v2007-03-14_CM).dat
- Contains only the ADF Commodore Amiga DATS.
- Download it from here.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [SPS Only] (TOSEC-v2007-03-14_CM).dat
- Contains only the SPS Commodore Amiga DATS.
- Download it from here.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [CD - CD32 - CDTV Only] (TOSEC-v2007-03-14_CM).dat
- Contains only the CD / CD32 / CDTV Commodore Amiga DATS.
- Download it from here.

Last edited by DamienD; 28 March 2009 at 21:45.
DamienD is offline  
AdSense AdSense  
Advertisement:
Old 28 March 2009, 21:20   #2
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
Instructions

Here are the instructions you should follow:


Preliminary:
  • Download the latest version of ClrMAMEPro 32bit (zip) from here.
  • Extract it to your hard drive e.g. "C:\CLRMAMEPRO".
  • Download the desired file e.g. "Commodore Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2007-03-14_CM).zip" from above in post #1 and extract the .DAT file to ClrMAMEPro's datfiles directory e.g. "C:\CLRMAMEPRO\datfiles".
  • Create a directory called "___ORIGINAL" on your hard drive i.e. "C:\___ORIGINAL" and then copy all your files into this. You can have subdirectories as well if you like.
  • Create a directory called "___FOUND" on your hard drive i.e. "C:\___FOUND".

Running ClrMAMEPro:
  • Start ClrMAMEPro and click "OK" at the welcome screen.
  • Double-click on the downloaded / extracted .DAT file e.g. "Commodore Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2007-03-14_CM)" displayed in the right hand window and then click "Default".
  • Click the "Settings" button, click "Add...", then point to the directory where your files are stored i.e. "C:\___ORIGINAL", click "OK" and then click the "X" to close the "Settings" window.
  • Click the "Rebuild" button, click the "..." button under "Source", point to the directory where your files are stored i.e. "C:\___ORIGINAL" and then click "OK".
  • Click the "..." button under "Destination", point to the directory where you want your rebuilt files to go i.e. "C:\___FOUND" and then click "OK".
  • Tick "Remove Matched Sourcefiles" and then "Rebuild...".
  • Once totally finished the "Statistics" window will appear, click "OK" then "X" twice to exit ClrMAMEPro.

All renamed / identified files will be sitting in the "C:\___FOUND" directory.
Any rare / unrecognised files will be sitting in the "C:\___ORIGINAL" directory.

Last edited by DamienD; 28 March 2009 at 21:45.
DamienD is offline  
Old 28 March 2009, 21:20   #3
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
Note

When loading some of the above .DAT files you will receive messages like:



This will occur for the following:

Code:
Airball (1989)(Microdeal)[cr Subway - TDT][t +1 Subway - TDT]
Amoeba Invaders (1987)(LateNight Developments)(FW)
Archipelagos (1989)(Logotron)[cr Threat]
Army Moves (1987)(Imagine)[cr Champs][h Defjam - CCS]
Beyond the Ice Palace (1988)(Elite)[cr CCS][t CCS]
Black Lamp (1988)(Firebird)[cr Defjam]
Bloody Data (1990)(Abandon)
Bomb Jack (1987)(Elite)[cr DF]
Brataccas (1986)(Psygnosis)[cr Rectum Research]
Buggy Boy (1988)(Elite)
Cybernoid II - The Revenge (1989)(Hewson)[cr Subway - Dream Team][t +4 Subway - Dream Team]
Dark Side (1989)(Micro Status)[cr PNA]
Deflektor (1988)(Gremlin)[cr QTX]
Donkey Kong (1993)(Bignonia)(PD)[C64 conversion]
Dr. Mario (1994)(Trash)(PD)
E-Type (19xx)(Total Destruction)(PD)
F.O.F.T. - Federation of Free Traders (1989)(Gremlin)(M3)[cr QTX]
Fred Feuerstein (1988)(Grandslam)(De)[cr Defjam]
Gazza II (1991)(Empire)(M4)[cr HZ]
Gospel Karaoke (1992)(Complex)
IK+ (1988)(System 3)
Let's Rave (19xx)(Profecy)
Maxima (1993)(Scoopex)
Orange 8 (1995)(Abyss)
Peter Beardsley's International Football (1988)(Grandslam)[cr Defjam]
Rampage (1989)(Activision)
Screaming Wings (1987)(Red Rat)[cr Defjam][t +1 Defjam]
Second Demo (1989)(Beyond Force)
Sine Intro (1990)(Beyond Force)
Splitter v2.0 (1990)(Beyond Force)
Stunt Car Racer (1989)(MicroStyle)
Super Pacman 92 (1992)(Pendle Europa)
Super Twintris (1994)(-)
Surrealism (1993)(Andromeda)
Tales of a Dream (1991-08-25)(Dreamdealers)
Things Take Time (1991-04-15)(Artemis)
Yogi Bear and Friends in the Greed Monster (1990)(Hi-Tec)[cr CSL]
Yogi's Great Escape (1990)(Hi-Tec)[cr SCX][t +3 SCX]
This is due to there being identically named .ADF, .EXE or even .SPS files. Just select "No To All", everything will rebuild properly

Last edited by DamienD; 28 March 2009 at 21:45.
DamienD is offline  
Old 30 March 2009, 23:44   #4
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
"SPS (TOSEC-v2008-07-23_CM)" + "TOSEC CD Dumping Project (TOSEC-v2009-03-15_CM)"

Ok, so I've created some newer "combined" DAT files with "rebuildto directories". These are basically the above but the old SPS / CD / CD32 / CDTV parts these have been replaced with these:


Belgarath's latest SPS DATs:
Code:
... Commodore Amiga - Coverdisks - SPS (TOSEC-v2008-07-23_CM).dat 
... Commodore Amiga - Educational - SPS (TOSEC-v2008-07-23_CM).dat 
... Commodore Amiga - Games - SPS (TOSEC-v2008-07-23_CM).dat

The TOSEC CD Dumping Project DATs:
Code:
... Commodore Amiga - CD - Applications (TOSEC-v2009-03-15_CM).dat 
... Commodore Amiga - CD - Compilations (TOSEC-v2009-03-15_CM).dat 
... Commodore Amiga - CD - Coverdiscs (TOSEC-v2009-03-07_CM).dat 
... Commodore Amiga - CD - Games (TOSEC-v2009-03-09_CM).dat 
... Commodore Amiga - CD - Multimedia (TOSEC-v2009-03-15_CM).dat 
... Commodore Amiga CD32 - Applications (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Compilations (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Coverdiscs (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Educational (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Games (TOSEC-v2009-03-15_CM).dat 
... Commodore Amiga CD32 - Homebrew - Compilations (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Homebrew - Games (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Homebrew - Multimedia (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Multimedia (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CD32 - Samplers (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CDTV - Applications (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CDTV - Compilations (TOSEC-v2009-03-07_CM).dat 
... Commodore Amiga CDTV - Educational (TOSEC-v2009-03-07_CM).dat 
... Commodore Amiga CDTV - Games (TOSEC-v2009-03-07_CM).dat 
... Commodore Amiga CDTV - Homebrew - Compilations (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CDTV - Homebrew - Games (TOSEC-v2009-02-15_CM).dat 
... Commodore Amiga CDTV - Multimedia (TOSEC-v2009-03-09_CM).dat 
... Commodore Amiga CDTV - Samplers (TOSEC-v2009-03-15_CM).dat

Commodore Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2009-03-15_CM)
- Contains all Commodore Amiga DATS.
- Download it from here.
- Note; see post #3 above.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [Excluding CD - CD32 - CDTV] (TOSEC-v2008-07-23_CM).dat
- Contains all Commodore Amiga DATS, excluding the CD / CD32 / CDTV DATS.
- Download it from here.
- Note; see post #3 above.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [SPS Only] (TOSEC-v2008-07-23_CM).dat
- Contains only the SPS Commodore Amiga DATS.
- Download it from here.


Commodore Amiga - Latest Public DATS Combined rebuildto Directories - [CD - CD32 - CDTV Only] (TOSEC-v2009-03-15_CM).dat
- Contains only the CD / CD32 / CDTV Commodore Amiga DATS.
- Download it from here.
DamienD is offline  
Old 17 June 2009, 16:33   #5
andreas
Zone Friend
 
Join Date: Jun 2001
Location: Germany
Age: 40
Posts: 5,858
Send a message via ICQ to andreas Send a message via AIM to andreas
Quote:
Originally Posted by DamienD View Post
Here are the instructions you should follow:


Preliminary: [*]Download the latest version of ClrMAMEPro 32bit (zip) from here.
OK. I'm out. Bye.
andreas is offline  
Old 17 August 2009, 00:09   #6
peo
Registered User
 
Join Date: Dec 2008
Location: Ursviken
Posts: 21
Problem with 'rebuildto'

Hello,

I am having some problem with the Rebuilder (or with the .dat file) in clrmamepro.

Following the general guidelines about how to rename the ADF files using clrmame, I'm stuck when it comes to putting the files in their correct destination directories.

What I've done is:
Downloaded Commodore Amiga - Latest Public DATS Combined rebuildto Directories (TOSEC-v2009-03-15_CM) from EAB/Damien.
I have set it up in clrmamepro as described.
My source path is (something_long\adf), destination path is z:\ (mapped network location to get it short)

When I click 'rebuild', I get problems with a too long destination path for windows to handle, caused by what I think is an error in clrmame or the .dat file:

Checking through the destination path (for successfully moved files), I see that there is one directory for each file, like for example:

z:\Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat\4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL]\4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL].adf

The full 'rebuild' filename seems to be a combination of destination path ("z:\") + "rebuildto" ("Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat") + "name" or "description" ("4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL]") + rom name ("4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL].adf")

Since windos cannot handle these long filenames, some files are not moved, and the moved ones causes a terrible mess at the destination (which is simple to correct using the unix 'find' command to move the files to their correct destinations).

Is there some option to prevent the "name" or "description" to be used as a part of the destination filename, or could it even be easier (for me) to write a UNIX based 'rom' manager ?

/PeO
peo is offline  
Old 17 August 2009, 04:58   #7
PandMonium
Registered User
PandMonium's Avatar
 
Join Date: Sep 2008
Location: Portugal
Posts: 404
well i don't have time to test but there should be a option to use compressed sets where it would create something like:
z:\Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat\4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL].zip
...and inside the zip there would be the rom "4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL].adf"

other idea for damien to reduce the path lenght could be not using the dat version / using the tosec folder structure, so instead of:
z:\Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat\
if could be:
z:\Commodore\Amiga\Games\[ADF]\

anyway good luck with it and try using compressed sets or something like that to rebuild to zipfiles
PandMonium is offline  
Old 17 August 2009, 09:06   #8
TCD
Global Moderator

TCD's Avatar
 
Join Date: Sep 2006
Location: Germany
Age: 36
Posts: 23,606
Quote:
Originally Posted by peo View Post
The full 'rebuild' filename seems to be a combination of destination path ("z:\") + "rebuildto" ("Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat") + "name" or "description" ("4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL]") + rom name ("4D Sports Boxing v1.0 (1991)(Mindscape)(Disk 1 of 2)[cr CSL].adf")

Since windos cannot handle these long filenames, some files are not moved, and the moved ones causes a terrible mess at the destination (which is simple to correct using the unix 'find' command to move the files to their correct destinations).
Hi PeO,

can you give an example of the resulting name (and path) ? NTFS should support upto 255 chars for each part of the path and 32767 altogether. So it should handle these files correctly. I would rather say that it might be a limitation of ClrMamePro itself.
Edit : Have you tried it with a not mapped folder?

Last edited by TCD; 17 August 2009 at 09:13.
TCD is online now  
Old 17 August 2009, 10:51   #9
peo
Registered User
 
Join Date: Dec 2008
Location: Ursviken
Posts: 21
I get the same problem when setting the destination to a local disk (M:\ami_ok):

Error while rebuilding:
\\OPENSOLARIS-FS\amiga\ADF_och_CD\ah_compl\games #\18th Hole (1993)(S.Garside - Berlin PD)(SW)(Disk 2 of 2)(Course)[SK 477][AMOS].adf
To:
M:\ami_ok\Commodore Amiga - Public Domain Collections - Berliner Spielekiste (TOSEC-v2005-11-19_CM).dat\Berliner Spielekiste 477 (19xx)(Berlin PD)(De)(PD)(Disk 2 of 2)(Course)[BSKCD]\Berliner Spielekiste 477 (19xx)(Berlin PD)(De)(PD)(Disk 2 of 2)(Course)[BSKCD].adf

For successful renaming, a complete file path could be (from log file):
\\OPENSOLARIS-FS\amiga\ADF_och_CD\ah_compl\games #\20000 Meilen unter dem Meer (1988)(Infogrames)(De)[cr Champs][h Personal Crackings].adf
-> 20000 Meilen Unter dem Meer (1988)(Infogrames)(De)[cr Champs][h Personal Crackings]\20000 Meilen Unter dem Meer (1988)(Infogrames)(De)[cr Champs][h Personal Crackings].adf

To shorten the path a bit, I have even cut off the "Commodore Amiga - " from the rebuildto value (not in this test, but in a previous one). The problem seems to be the "double" path (last directory + the filename is the same, except from the .adf ending of the filename)
Attached Thumbnails
Click image for larger version

Name:	rebuilder.jpg
Views:	191
Size:	39.7 KB
ID:	22393   Click image for larger version

Name:	rebuilder_prob.jpg
Views:	198
Size:	72.5 KB
ID:	22394  
peo is offline  
Old 17 August 2009, 11:12   #10
peo
Registered User
 
Join Date: Dec 2008
Location: Ursviken
Posts: 21
Works using compressed files..

Rebuilding works when selecting the option to compress files at the destination, and the resulting archives just contains the .adf file, not the extra directory that was created without compression.

Problem solved, but this is a bug in clrmame, or something in the .dat file that causes the extra directories to be created.
peo is offline  
Old 17 August 2009, 11:17   #11
TCD
Global Moderator

TCD's Avatar
 
Join Date: Sep 2006
Location: Germany
Age: 36
Posts: 23,606
Well the file/path name lengths are pure theory as I just found out. There is indeed a 248 chars (empiric test ) limitation for the whole path...
Edit : Ah, good to hear it's sorted for now. Maybe Damien finds what causes the extra dir to be created.
TCD is online now  
Old 17 August 2009, 11:38   #12
peo
Registered User
 
Join Date: Dec 2008
Location: Ursviken
Posts: 21
Quote:
Originally Posted by PandMonium View Post
other idea for damien to reduce the path lenght could be not using the dat version / using the tosec folder structure, so instead of:
z:\Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat\
if could be:
z:\Commodore\Amiga\Games\[ADF]\

anyway good luck with it and try using compressed sets or something like that to rebuild to zipfiles
With compression the result was put in the destination without the extra directory..

As for path naming, I agree that it should be more logical than it is for now, maybe even a directory for each game/program, so that the disks for each title will be contained together:

z:\Commodore\Amiga\Games\[ADF]\Another World (1991)(U. S. Gold)
(Fr)[cr Company](Disk 1 of 2).adf
(Fr)[cr Company](Disk 1 of 2)[tr En].adf
(Fr)[cr Company](Disk 2 of 2).adf
(It)(Disk 1 of 2)[cr CSL].adf
[cr Amiga Box][tr Hu](Disk 1 of 2).adf
[cr Amiga Box][tr Hu](Disk 2 of 2).adf
[cr Company][h ATX](Disk 1 of 2).adf
[cr Company][h ATX](Disk 2 of 2).adf
[cr Company][h PRX](Disk 1 of 2).adf
[cr Company][h PRX](Disk 2 of 2).adf
[cr CSL](Disk 1 of 2).adf
[cr CSL](Disk 2 of 2).adf
peo is offline  
Old 17 August 2009, 15:32   #13
PandMonium
Registered User
PandMonium's Avatar
 
Join Date: Sep 2008
Location: Portugal
Posts: 404
if you go to the initial thread quoted in the first post i posted there a link for a version (here) that used the tosec folders structure like i described (if i remembered correct), you can test it at your own risk
For the other suggestion, grouping sets using a primary setname would give a bit more work and afaik there is no datfile for it atm.
PandMonium is offline  
Old 25 August 2009, 23:40   #14
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
Hey peo,

Sorry for the delayed response, only just saw the replies...

I did test using the full TOSEC set and everything worked perfectly before uploading all the DAT files

If you use the exact same settings + paths (same or shorter length anyway) that I posted in #2 everything will definitely work. Afterwards you will end up with directories named for each .DAT which contain the files .zipped individually e.g.

C:\___FOUND\Commodore Amiga - Compilations - Games (TOSEC-v2006-09-10_CM).dat
C:\___FOUND\Commodore Amiga - Games - [ADF] (TOSEC-v2006-10-19_CM).dat

When testing though I did try different things and noticed issues when paths are too long, hence that's why I suggest using C:\___ORIGINAL and C:\___FOUND. Also, as you found out you have the problem of "one directory for each file" when not having "Compress files" ticked... Unfortunately this is a Windows file system limitation.
DamienD is offline  
Old 26 August 2009, 00:26   #15
DamienD
Registered User
DamienD's Avatar
 
Join Date: Aug 2005
Location: Sydney / London
Age: 37
Posts: 5,058
Oh, I guess I should also mention a few things just to try to clear up the approach taken:

a) I used the "Compressed file" option as usually if you find a full TOSEC set it will be in the form of a directory name for each DAT and then contain the individual files zipped. Secondly as mentioned above, this also stops the behavour Peo discovered with errors due to extremely long paths i.e. "destination path + rebuildto + file name + file name".ADF (which is how ClrMAMEPro handles the rebuilding of uncompressed files and becomes a Windows file system limitation). Finally, when having all the DATs merged into one massive "Mega DAT" you need to have individual DAT name paths due to some files existing in more than one DAT.

b) I used the full name of the DATs instead of slighty shorter ones as it's easy to identify exactly what DAT the files belong to in terms of release date and name. As I'm sure most of you know there are loads of TOSEC Amiga DATs floating around on the internet, this helps you quickly identify exactly what ones are contained within the "Mega DAT".
DamienD is offline  
AdSense AdSense  
Advertisement:
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Memphis Amiga Group disks and dat for TOSEC public domain collection TroyDavis project.TOSEC (amiga only) 5 27 September 2007 11:49
does TIM auto download latest dats? gizmomelb project.TOSEC (amiga only) 4 06 September 2006 12:51
Jan 13 and more Amiga TOSEC dats Fissuras Retrogaming General Discussion 1 14 January 2003 00:51

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 22:14.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Page generated in 0.31560 seconds with 12 queries