DATs in Excel & Access format

[DH]

Renaming Dat's should be so much easier when this database is fully operational. All you do is correct the area in the input fields etc and it shows the full name below in the corrected field in yellow (not editable) you then can copy 'n' paste the correct info back into the Title name. Done!

For checking purposes it works well too.

Screenshot below hope it looks good 'cause it works well too

[PandMonium]

that is similar to what "Renamer Tool" part in TIM does/did
In my opinion it's a nice feature to have and you are missing some fields yet, if you want i can give you some tips, i think you miss things like dump flag counters for example but its going good , it could be a good alternative tool for guys renaming amiga stuff
About those kind of tools afaik almost none of the renamers used it much since they said it would take more time and prefered to rename files manually, TKaos was the only one that used it the last time i checked, don't know if he still uses it
The advantages in using that are that it avoids some mistakes and they would just need to care about filling the needed fields.

Anyway good work and have fun with your excel/access adventure

[DH]

Cheers PandMonium.

I'm hoping this will give renamers a chance to do the renaming much quicker with the added bonus of having the two (original name & new name) sandwiched together so you can see any differences.

Please do feel free to give me some tips on any flags or anything that may be required or missed, after all, I can't do this all on my own without the help from people that know exactly what needs to be present in the database. It's good to be able to tailor it around the needs of the renamers, hopefully they will give some tips too.

Any tips are welcome, by anybody too. I'm really hoping this will be an awesome database for all to use that's associated with EAB or the Amiga

[DH]

What are the most common errors regarding the TOSEC Dat file ?????

Where can I get corrected updates of renamed files ???

How many is there??? If anyone knows

I only need to know this as I can't expect this to work correctly unless the data in it is correct

[TCD]

Originally Posted by DemonHellraiser

What are the most common errors regarding the TOSEC Dat file ?????

Where can I get corrected updates of renamed files ???

How many is there??? If anyone knows

I only need to know this as I can't expect this to work correctly unless the data in it is correct

Pretty much question marks for an ordinary monday, don't you think?
What do you mean by 'corrected updates of renamed files'? Most common errors are wrong cracker/hacked info's I would say.

[DH]

Sorry about that. Monday blues I suppose

Basically all I need is the corrected data. That way I'm not repeating the incorrect stuff in the database

Would this thread be the one 'TOSEC Amiga Correction Thread' or are there more?

[TCD]

Originally Posted by DemonHellraiser

Sorry about that. Monday blues I suppose

Basically all I need is the corrected data. That way I'm not repeating the incorrect stuff in the database

Would this thread be the one 'TOSEC Amiga Correction Thread' or are there more?

You are not asking for the WIP dats, are you? I think there is little chance to get them.
The TOSEC correction thread would be the place to look for corrections, but do you really want to update the database manually? Also I don't think all the corrections are in the thread...

[DH]

Well I'm afraid with the database it will have to be done manually I can import the dat file but it will be a separate table and I can't see how I will be able to extract the correct info for the right fields, except manually.

Anyway, maybe I'll just pursue the original file from 2006 and get that working and looking right for now

[PandMonium]

...and now you can understand where we are and why we take time to release new dats

Of course there are also lots of times in tosec with almost 0 activity (at least from the non iso part), now we are having some activity but from few guys so it takes time and the TOSEC dats until the last official release have tons of errors, because they were not checked and renamers just rushed it up to add 100 new sets instead of 25 correct ones

Don't know if you are asking about all TOSEC dats from last release or only amiga dats, but answering for the first case:

First there are the TNC standard errors, in other words, setfiles that don't complain TNC rules (eg. missing mandatory flags, invalid values for some of the flags, invalid chars used, wrong flag order and so on).

For 2007-03-14 release those valuse were:
Invalid Sets: 5762
ZZZ-UNK: 4083

ZZZ-UNK doesn't mean invalid sets, but are sets in TOSEC that are not yet renamed and were added to the dats with a setname starting with "ZZZ-UNK", they don't follow TNC rules and were just dumped there so you can't parse those, the good thing would have a lower value of both.

Those errors were the first ones i tried to tackle and [idoru] fixed almost all so in wip dats the counter went down to only 17 TNC errors, last months i added some more checks (date checks / language+country flag order) and the number when up a bit to 285 errors. ZZZ-UNK are now 4055 but you have to remember that also a few new dats were added and some more were updated so new mistakes in sets not yet on last official dats appeared too

This last ~300 errors are yet to be fixed because its hard to coordinate fixes when there is few people working and some of them almost always busy (iduro ) + not having access to tosec.org (Grendel MIA forever i guess).

But these problems are just the start, TNC errors are easy to point and fix, the worst ones appear when you try to check the setname content. From titles, to dates, publishers, scene groups, descriptors for some flags (a, b, f, m) and so on, you soon understand that there are really lots and lots of errors and something needs to be done

To make things even more dificult, we are talking of tons of systems and 300.000+ sets, some values are valid in one system but not in the other or refer to different things that used the same name, and vice versa.
Currently we (mostly i, TKaos and gorski) are trying to check the scene groups in TOSEC sets (bippyM also helped a bit here on eab), there are a total of 6940 scene groups, and until now we barely passed 50% of them and 1559 (22.46 %) are already INVALID (typos or just some wrong thing renamers got) and 638 (9.19 %) need a bit of attention (are abbreviations, groups that are valid in one system/dat but invalid in other and so on).

Now take the current 1559 invalid ones * number of sets they have and thats a lot of setnames to fix, fortunately i have dedicated people helping out so i guess one day it will be done
The other good thing is that dump flags are not mandatory and since various renamers were really really lazy (to the point of just wanting to make the dats grow and adding lots of sets with "title (19xx)(-)" when other info is really easy to know or even shown at title screen) there are complete dats with only [cr] for example or not even not using any dump flags so the scene groups are not near the publishers number(29000+), the bad part is that those dats are not correct and one day someone has to check it better and rename lots of images better.

So, now that you are trying to get info from the dat and playing a bit with it, you just discovered what the problem is right now

Anyway if you just mess with Amiga - Games - [ADF] you shouldn't have much problems, its one of the most correct dats in terms of TNC errors and with so much help from lots of people and attention from [idoru] as maintainer i guess the content errors are not much (in % of sets )

[DH]

Yep! Sure have found out the problem PandMonium. It's such a shame that the renamers got too lazy and started adding stuff wrongly though 'cause it has created quite a lot of extra work that was unnecessary

I think for me, I'll just pursue the Amiga game dats or anything Amiga related. Seems like loads of things need fixing in the other dats and I really don't think this database will help in any useful way, except for checking for errors, or adding new sets.

At the end it could be quite useful, especially being able to convert it to XLM so it is available for many platforms, although this sounds good, it can only work if that is the direction the TNC/TOSEC Dats are heading.

Once again, thanks for your expert advice and I hope to have this database working fully soon.

[DH]

Well the database is fully working and looks like it is error free

But! There is some serious input to the database required, the Dump Info Flags need the input added This can be done reasonably quickly, but all and everyone of the 21,000+ files will need to be checked for correctness. Once this has been done I'm sure I can release v1.0 to some of you

[TCD]

Originally Posted by DemonHellraiser

Well the database is fully working and looks like it is error free

But! There is some serious input to the database required, the Dump Info Flags need the input added This can be done reasonably quickly, but all and everyone of the 21,000+ files will need to be checked for correctness. Once this has been done I'm sure I can release v1.0 to some of you

Well then better hurry DH, 'cause I want to see it under my X-mas tree
No really : Great work and I'm really looking forward to the first public version

[DH]

God dammit! Forgot about my poor database. Revived only because I'm creating a spreadsheet for cracking group names/abbreviations

Just been thinking, everything that is wrong where cracking group names/abbreviations are concerned, I can eliminate the errors very quickly by filtering the names/abb, checking for correctness and changing the ones that need changing, all in one go

If I filter 'Goonies' all I'll get is a column with say 20 or so 'Goonies', I then change the top one to 'The Goonies' and drag it down to the 20th entry and it'll replace all the others too. DONE!

Although this will change the name in *MY* set, it will not change the original name and will flag up when there is a difference between the two. So where there is no difference, you can skip by and only stop at the ones that have the big res X next to them.

btw. There is a small and annoying error in the database that needs sorting out. I just can't understand why I'm getting it after the first 50+ entries but non before it I'll sort it when I've finished my other project. Of course, don't forget, everything I do will *always* be WIP, so don't ever expect anything finalized

[TCD]

Originally Posted by DemonHellraiser

Of course, don't forget, everything I do will *always* be WIP, so don't ever expect anything finalized

Ah, I see there is still some life left in the project Any chance that you will release the thingy in a beta state someday?

[DH]

Once that damn problem is sorted out, I can't see why not

I'll give you an example of what it's doing:-

Worms - The Director's Cut (Disk 1 of 3)[h Ninja's AGA World].adf (Original Name)
Worms - The Director's Cut (Disk 1 of 3)[h Ninja's AGA World][][][][][][][][][][][][][][].adf (My Set)

It doesn't happen on the first 50+ but adds them thereon, except for those that are already correct

I must get back onto this and sort it all out properly so I can give you all a taster of what it's potential is

[TCD]

Hope you find the problem and looking forward to try the taster mate Also wouldn't mind if you bundle it with the cracking group list...

[DH]

Be patient my friend, there's still quite a lot of work to be done before it's finished

[DH]

Something I forgot to mention. Would it be a cool idea if I combined the group names/abbreviations into Access with the dats, maybe in another tab. Keeps them all together in a neat and useful way + it'll all be at hand

[DH]

Woohoo! Problem sorted from:-

Worms - The Director's Cut (Disk 1 of 3)[h Ninja's AGA World][][][][][][][][][][][][][][].adf

back to

Worms - The Director's Cut (Disk 1 of 3)[h Ninja's AGA World].adf

Now there's another problem, but this one is quite easy to sort out. Oh! and all the input that needed doing, has all been complete in the 21,852 entries from Amiga Games TOSEC Datfile This was all done yesterday, 'ALL DAY' practically

Now to sort out that little other problem that I noticed

Edit: Other little problem sorted, now I can concentrate on making sure my file names are identical to the originals, which is easy. I just need to cycle through all 21,852 entries and make sure there is a nice big green tick against my file names and not a great big red cross