A Preservation story

[viernez67]

Sorry for my Englich(or spaninglish), my native language is Spanish(The second most important language in the world, first is C++).

Now it's time for a Story...

Upon once a time, there was a poor man who wants to preserve MSX tapes, because other systems(C64, Amstrad, Spectrum...) have a format called TZX and some tools to do the preservation...

This man, as he has no time because he was(and is) working in other projects and he had no knowledge about Tapes, Copy protections... asked MSX community if somebody skilled can do the job. But MSX community doesn't take care of preservation...so this poor man was ignored...

But this poor man wants to see all MSX Tapes preserved, so he started to learn about how TZX tools do the job. Due to his knowledge, he knew that a Tape and a Disk is the same thing, with very little difference. So he read the web pages at Caps-project and also Disk2FDI tech docs... He read a lot and Talked with some guys too.

Then someday, he decided: "I have enough knowledge to do a tool to preserve MSX Tapes". Hacking VOC2TZX or Samp2CDT was a posibility, but was not accepted because he wanted to do a new tool and VOC2TZX/Samp2CDT code was a bit difficult to read and support.

So the poor men started his own tool...But after some lines of code, and as he read in the CAPS project's web pages, something about a TRACER machine...he thought, hey? Why not to do it for TAPES too. This man never read a FreeFrom script, and only saw a piece of a script in a CAPS' web page(AmigaDOS block). But that script gave him some ideas.

So after using his brain too much(With a risk of explosion), he found a way to describe any TAPE protection, but what was more incredible, is that this can be used to DISK, if he used more abstraction...(Or any other media: CDROM)

You can read a bit about that here(Zabado78 is the poor man): http://www.worldofspectrum.org/forum...rum=1&start=50

Well, after this story, what do i want to say? I want to ask CAPS team, what are your future plans?, I don't want to support DISKs, because it requieres a lot of time...But as you can see there're some guys out there that can do the same job, i know that you are an Amiga project.

You can work with some people out there...(Remenber that poor guy preserving C64 disks)

This post is only to change some mind in Caps Team, i don't need any help or code. I have done all myself, and i will continue in this way. I can do all that CAPS team does, but this needs time(which i don't have) and a lot of motivation(Which i don't have too). But why to duplicate the effort when you have it working...Also as coding/testing/completing my tool requires time, you know that for a preservation project, TIME is very important.

I hope you don't ignore this post like a private mail i sent you some time ago .

Regards

[IFW]

We are NOT ignoring ANY emails.
If you haven't received an answer it is because Fiath hasn't had the time to write a proper answer yet.
Alternatively the mail was deleted by the spam-filter.

Keep in mind that Fiath receives hundreds of emails a week, and normally answers most of them.
Someone who does not deal with a huge number of emails may think in his own terms, that answering his only emails takes no time. However when you have hunderds of "just one mail" a week or even a day the situation is somewhat more complex - everyone thinks he is the only one sending an email and waiting for an answer

As for your story:
Supporting hundreds of disk formats is not exactly the same thing with supporting a dozen or so known protection formats.
ANYONE who wanted to join the development phase of caps was free to do so. It was quite often advertised, yet nobody who wasn't an abolsute lamer has ever attempted to join in any way. A good example for the quality of people is suggesting storing the data on punch cards... and no I am not kidding, neither was the guy.
NOBODY competent was available, or was too busy to do things, basically contributing only "vocally".
Many of the people around were the ones who wanted to turn the project into their OWN financial gain, which is not in the interest of preservation, and leads to things like the company buying the c64 brand and so on.

A project like this needs:
- competence: very strong skills in reverse engineering, assembly, protections, industrial duplication and so on.
- commitment: most people will give up after a very short time, it is a very unrewarding experience
- interest: it should be interesting for the rest of the world too
- support: contributors, proper orgainzation, management, PR, web site
- money: buy rare items, equipment, collections - heaven forbid pay for the people working on it... You can't possibly imagine the amount of money you need to spend from your own wallet to get a project like this going. And unfortunately when the time comes for the people to put their money where their mouth is, most of them turn out to be just big faces.(btw, Thanks Tourn )
- time: a lot. Reverse engineering a format can take as little as a few minutes if you are good enough, but more than once it will be hours and sometimes days.
- sixth sense: there are many situations when you are left in the dark with seemingly unexplainable phenomenons, formats without checksums, no integirty information and so on. Sometimes however these formats do have integrity information recorded by the duplication firm in order to be able to verify the duplicated disks. You just don't know what and where. It is a good test of 6th sense skills to find and understand those.
- quality control: if you are not thinking in a hundred games, more like thousands quality is an important factor. You don't want to go back and re-release everything if a problem is found, possibly after thousands of images.
- think about the future: there are many shortcuts that could have been made, like running raw images under emulation, not developing a language to describe the formats and so on... All cheaper and faster than how it has happened, but would have damaged the goals of the project. It is not an easy task to always take the harder route.
- a competent system: something that is able to describe what is needed more over in a format that is
a, comparable, so images can be compared properly. You should remember that disk content varies depending on loads of factors for the very same disk written at two different times.
b, allows later rewriting of the data (disks)
c, allows virtual rewriting of the data (emulation)
d, compact (no, ipfs are not compressed)
e, easy to maintain and to enhance without losing compatibility with existing images
f, has a clean access method for programmers, so they don't have to reinvent the wheel every time
- legal support
- sooner or later a real world company or legal entity

If any of these are missing the project suffers and will eventually die.

[Interceptor]

glad someone made use of the technical information on the site, if they are able to start (and successfully implement) a project like ours for tapes, that's fantastic.

and are _you_ zebado78 by any chance?

[viernez67]

Originally Posted by Interceptor

glad someone made use of the technical information on the site, if they are able to start (and successfully implement) a project like ours for tapes, that's fantastic.

and are _you_ zebado78 by any chance?

Yes, i read all info that i can, all was usefull . And yes, i started my project and i was succesfull on some experiments... And yes, i'm the poor guy(Zabado78). The same guy of here http://www.msx.org/forumtopic3809.html(I ate some words, and i forgot some "not" words in some sentences...), Take in account the date, i know a lot more now. Also, Fiath was there, this guy is incredible, he is on all boards(Here, MSX, Spectrum, Atari...).

About doing a project like yours but for Tapes, well, it's basically the same project but for Tapes, just replace Bit Cell by Pulses, a flux when the Wave of the tape cross the X-asis, re-define some "transport layers"(In your terminology), and you can even reuse some of your encodings, and you can process Tapes. So simple.

You, like me, process Bit-streams, and you use a layer system, so a group of bits in a layer has some meaning in an upper layer...Maybe you do in another way(i know only what i read in your webpages). But I do on this way...

IFW:

I know all the problems of preservation, believe me, i read a lot and talked with some guys, and i understand why you don't release your tools(I will do the same until i get all MSX tapes preserved). Also my tool is becoming very complex(it outputs many information), so for a newbie, a MakeTZX is easier...I will follow your steps, i forced to some guys of the MSX community(It took me 4 months!!!!), to dump their tapes to WAVs. Also we are doing some organization, but it's a mess, because i'm the only guy who understand all. CAPS team is done by several menbers and all of you know what you want, and how to do all. My teammates only dump their waves, and i must do all the hard work, writting the tool and verify data integrity of each dump. So yes, i know how hard this is.

Even, i have not complete information about copy protections on MSX, but i think that are the same as Spectrum(Lazy programmers).
------------------------------------

What i want to say, is that i could do a simply TZX tool for MSX, this was the easy way, or better, just Hack Tomcat666's VOC2TZX... But imagine a boy who want to preserve all Spectravideo Tapes(Or any computer what uses Tapes). He will be in the same situation like me when i wanted a TZX tool for MSX. This is why i decided that a generic tool should be better, even if it takes me more time. Why? Because he can, for example, try to understand all Spectravideo's copy protections. And use my tool to describe it, so he doesn't need to write another program from scratch. And spectravideo tapes will be preserved. I will have no overhead, i can just send him the tool, explain him how it works and tell him, write the protections in this language, just like a Freeform script. So i have not wasted my time because i was not interested in Spectravideo Tapes.

Now(again). Imagine the poor guy of C64* disk preservation, he's working to identify copy protections, imagine that he has discover all, and he starts to do the preservation. First, i dont know if he is doing a RAW dump of a disk(Like A WAV is for me). Because perhaps he doesn't have the money to register DISK2FDI or a program like yours CT. But imagine again, that he is skilled and he do the job right(he dumps a disk rightly).

Second: What he should do? Waste his time writing a program which will do the same that your program does, and not so generic(like VOC2TZX is only for spectrum tapes)...Or just ask you for your program, teach him how to write a description of a protection, and do things rightly...

I hope that you understand what i mean(Sorry again for my english). I know that you(CAPS team) said that soon you will be a global preservation society, but i think that you still are working as a closed group(It's good to control all, but your project is so big, that if you want to control all, it will take years...so you're concentrated in some systems...not all ones).

I don't know what are your plans, but you(CAPS team) did a great job.

Regards


NOTE*: I'm using the Guy of C64 preservation project because(and i would like first MSX disk preserved than any other machine), because is the best example of: "I want to do it, but i dont know how..."

[IFW]

Ok, I understand what you meant much better now and I absolutely AGREE with you
There is no problem doing that, but only and only if it is based on caps principles.
I must invest time into it, and I am not investing into anything that is not beneficial for real world preservation.
That is:
- it is not a backup service it is a preservation service. Modified games etc are not allowed
- cannot be used for personal profit, no charge for media, distribution etc., no excuses
- quality control
- competent people
- people who are willing to invest time as well as money (a LOT of BOTH)
- people who are not backing out after a few months
- people who are not happy with shortcuts
- people must understand if they reach their limit and ask for help if needed, but this shouldn't be the norm, they must be able to work on their own most of the time
- it must work as a subsidiary of SPS
- it must follow all guidilines as set by SPS
- it must be legal

There is nothing here that cannot be followed through easily, but people with different intents are strongly discouraged and they are free to do whatever they want to do in their own free time and from their own money.

[fiath]

Originally Posted by viernez67

I hope you don't ignore this post like a private mail i sent you some time ago .

I do apologise! For some reason your private message got "missed". I had not even read it until today. I have only one theory of what happened, and that is that I had opened it, but for some reason my browser closed and so it was marked as "read" even though I had not replied. I do apologise for this!

I see that you emailed me to remind me, thanks for that! Unfortunately, I had not read that either, along with the *many* CAPS-related emails I have not answered yet.

Sorry! I hope things have now been worked out. I'll start reading this thread now.

[fiath]

Okay, I remember you now

Also, I realised that I did in fact read your private message, but I can't remember why I didn't reply. I think I was going to do some investigation and then get back to you, but, pretty obviously I haven't. Apologies once again. Your email would have reminded me, but I had not got to that one yet.

I may be getting a laptop in the next few months *hopefully* which will return CAPS email turn-around back to how it used to be, i.e. 2-3 days.