Seeking platform independant Crunchmania 'CrM2' data file depacking

28 September 2009, 17:01

I am looking for either source code or any suggestions as to how I can depack crunchmania packed data files outside of AmigaOS. Specifically I need to unpack files packed with the LZ-H algorithm ('CrM2' headed files.)

Can anyone help?

One possible solution: Am I right in thinking the registered versions of CrunchMania 1.9 included source code for depacking CrM files. Maybe this code is portable or can be made portable?

Leffmann · 28 September 2009, 18:23

I wouldn't want to implement LZH or read CrM2 format specs, but if you can find a manageable version (anything but raw 68K disassembly) of the decruncher source I'll give a go at rewriting it in portable C for command line or library use.

29 September 2009, 03:39

What a coincidence, I have just translated the decrunch routine from the crm.library into x86 code with the help of portasm 68ktox86.

Even for the demo version it still gets you past the tedious bit, and then most of the cleaning up deals with converting big endian to little endian on memory/packed file reads.

I have it running as a dll to which I have wrote a front end in Delphi, although still a little rough looking.

29 September 2009, 11:58

@serialthrilla
Wow, that's impressive. Allthough I probably should have specified I need portable C code for part of a cross platform project (a clean-room implementation of an Amiga game engine.) How does the disassembly look?, deceipherable? Allthough I guess dissasembly and reverse engineering would violate the terms of the crunchmania licence, (as old as it is).

That said, I can see this being really useful utility anyways, are you planning to release it?

@leffman
Thanks for the generous offer, hold that thought.

Skylight · 01 November 2009, 21:59

Here are the original CrM decruncher sources (68k ASM of course

) ...

Have fun!

Ciao, Rick ...

02 November 2009, 13:57

Wow, utterly awsome! You are a hero

Many thanks.

andreas · 02 November 2009, 16:16

Cool!

ubyte, when you're finished with your "C translation", please let me know!

It may help me a lot in practising my assembly reading skills (still very bad :P)

Leffmann · 02 November 2009, 23:58

Here's a C version of the optimized LZ-Huffman decruncher along with a command line tool.

It's a literal translation so the code looks awful, but it works and is portable. At least there were no errors on any of the files I tested on both 68K and x86 using different compilers.

Two functions are exposed, Decrunch(data) and GetSize(data). Decrunch does in-place decrunching and needs enough space to hold the decrunched file and the 14 byte header. GetSize returns the decrunched size of the data or 0 if the file appears invalid.

03 November 2009, 00:24

i can't thank you enough for this, cheers

andreas · 03 November 2009, 11:17

nice work Leffmann!

hint: (saves a lot of '0' typing)

not

Code:

d3l = (d3l&0xffff0000) | d3

but

Code:

d3l = (d3l&(~0xffff)) | d3

... should do the same thing

In int, ffff0000 is the 2's complement of 0000ffff

Not sure if you need those inner parentheses, even...

StingRay · 03 November 2009, 11:57

Quote:

Originally Posted by andreas

hint: (saves a lot of '0' typing)

Code is not about "avoid typing some characters", it should be readable in the first place! Don't you think &ffff0000 is much more intuitive? You can see at first glance that the lower bits are masked out. Just my 2 € cents.

Oh, and nice job Leffmann! =)

Leffmann · 03 November 2009, 13:04

Andreas,

Absolutely. There are a lot of space-savers and consistency fixes to be done now that I look through it again. I just translated it quickly to get it working since I don't know the algorithm itself, and when looking at a 1 to 2 instruction span you miss even the obvious things.

Feel free to optimize it. I wouldn't bother about code clarity with the code being what it is

Funny error message from VBCC: "Warning 175: this code is weird"

StingRay · 03 November 2009, 13:30

Quote:

Originally Posted by Leffmann

Feel free to optimize it. I wouldn't bother about code clarity with the code being what it is

Yeah, in this particular case I agree.

But my opinion still stands, readability first, specially when it comes to HLL! "Optimizing" code in a way that you have to type a few characters less, well, not for me!

pmc · 03 November 2009, 14:00

Quote:

Originally Posted by StingRay

But my opinion still stands, readability first, specially when it comes to HLL! "Optimizing" code in a way that you have to type a few characters less, well, not for me!

Gotta agree really - make the actual code execute quicker or take up less space in memory, yes.

Make the text file representing the unassembled / uncompiled code shorter by a few characters here and there, no.

The worst thing in the world, in my opinion, when trying to work out what code is doing by reading the source is when that source is difficult to fathom / read.

This has even happened to me when I've returned to old sources of my own where I used for example constants in the source rather than referencing equates. I end up thinking things like: Why did I put $02e0 in that register!

Leffmann · 03 November 2009, 15:32

Absolutely. Readability is a must for anything you might want to extend or other way come back to. I used to do all the magical numbers and ascii art like syntaxes too. Now I use symbolic names and expressions and comment the code in a tutorial like style, and it's just so much easier to follow.

This is btw one reason I moved away from ASM-One and similar assemblers. They only allow 1 source with little or no operand spacing = your program must be assembled in its entirety every run and just becomes one big lump of cluttered and incomprehensible code. I even used to think that splitting code into modules and linking them was some "high level stuff" that somehow added extra code to my executable and made it slower.

Separating the code within the file or into include files helps a good deal though.

StingRay · 03 November 2009, 15:39

Quote:

Originally Posted by Leffmann

This is btw one reason I moved away from ASM-One and similar assemblers. They only allow 1 source with little or no operand spacing = your program must be assembled in its entirety every run and just becomes one big lump of cluttered and incomprehensible code.

I wholeheartedly disagree

But that's for another thread I guess.

andreas · 03 November 2009, 18:02

Quote:

Originally Posted by StingRay

Code is not about "avoid typing some characters", it should be readable in the first place! Don't you think &ffff0000 is much more intuitive?

No. But I'm not going to argue with you about that now. (I even know well why I shouldn't ... let's say "from experience")

Quote:

Originally Posted by Leffmann

Feel free to optimize it.

Nah. I just wanted to give a quick hint, because I accidentally saw it. Since I can't think about everything myself - especially with time constraints - I'm always thankful for a good eye. For others ... well, can't really speak for the others, can I.

StingRay · 03 November 2009, 18:38

Quote:

Originally Posted by andreas

No. But I'm not going to argue with you about that now. (I even know well why I shouldn't ... let's say "from experience")

No what? You don't think it's more intuitive? If so, that's kinda strange but I don't have to understand everything anyway I guess.

Edit: And what do you mean with "I even know well why I shouldn't"? You know why you shouldn't argue with me? Dunno, the more often I read your post the more I fail to understand it... anyway, if you're happy obfuscating code that way just to "save some typing", feel free to do. After all, I don't have to review your code.

mr_0rga5m · 04 November 2009, 16:20

Quote:

Originally Posted by andreas

nice work Leffmann!

hint: (saves a lot of '0' typing)

not

Code:

d3l = (d3l&0xffff0000) | d3

but

Code:

d3l = (d3l&(~0xffff)) | d3

... should do the same thing

In int, ffff0000 is the 2's complement of 0000ffff

Not sure if you need those inner parentheses, even...

So by the time you put the (~ and closing ) you have saved yourself typing 1 character and your source is less readable to half the people. Not a great saving mate.

andreas · 05 November 2009, 00:04

Yay, keep on picking on those who can't fight back!

28 September 2009, 17:01	#1
ubyte Posts: n/a	Seeking platform independant Crunchmania 'CrM2' data file depacking I am looking for either source code or any suggestions as to how I can depack crunchmania packed data files outside of AmigaOS. Specifically I need to unpack files packed with the LZ-H algorithm ('CrM2' headed files.) Can anyone help? One possible solution: Am I right in thinking the registered versions of CrunchMania 1.9 included source code for depacking CrM files. Maybe this code is portable or can be made portable?

03 November 2009, 11:17	#10
andreas Zone Friend Join Date: Jun 2001 Location: Germany Age: 50 Posts: 5,857	nice work Leffmann! hint: (saves a lot of '0' typing) not Code: d3l = (d3l&0xffff0000) \| d3 but Code: d3l = (d3l&(~0xffff)) \| d3 ... should do the same thing In int, ffff0000 is the 2's complement of 0000ffff Not sure if you need those inner parentheses, even... Last edited by andreas; 03 November 2009 at 11:29.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
RTDD v1.9 (CrunchMania Data Decruncher). Does it exist?	BarryB	support.Apps	5	15 May 2015 21:46
Sprite data from file	Sparticle	Coders. General	2	25 August 2012 14:58
Maupiti island german version, is it possible to get the english data file?	paullawr	request.Old Rare Games	0	16 February 2009 23:06
Capture crash data in log file	DDNI	support.Apps	8	19 February 2007 20:17
RNC Data File Depacker v2.1	Nico	New to Emulation or Amiga scene	8	05 May 2002 18:05

28 September 2009, 18:23	#2
Leffmann Join Date: Jul 2008 Location: Sweden Posts: 2,269	I wouldn't want to implement LZH or read CrM2 format specs, but if you can find a manageable version (anything but raw 68K disassembly) of the decruncher source I'll give a go at rewriting it in portable C for command line or library use.

29 September 2009, 03:39	#3
serialthrilla Posts: n/a	What a coincidence, I have just translated the decrunch routine from the crm.library into x86 code with the help of portasm 68ktox86. Even for the demo version it still gets you past the tedious bit, and then most of the cleaning up deals with converting big endian to little endian on memory/packed file reads. I have it running as a dll to which I have wrote a front end in Delphi, although still a little rough looking.

29 September 2009, 11:58	#4
ubyte Posts: n/a	@serialthrilla Wow, that's impressive. Allthough I probably should have specified I need portable C code for part of a cross platform project (a clean-room implementation of an Amiga game engine.) How does the disassembly look?, deceipherable? Allthough I guess dissasembly and reverse engineering would violate the terms of the crunchmania licence, (as old as it is). That said, I can see this being really useful utility anyways, are you planning to release it? @leffman Thanks for the generous offer, hold that thought.

02 November 2009, 13:57	#6
ubyte Posts: n/a	Wow, utterly awsome! You are a hero Many thanks.

02 November 2009, 16:16	#7
andreas Zone Friend Join Date: Jun 2001 Location: Germany Age: 50 Posts: 5,857	Cool! ubyte, when you're finished with your "C translation", please let me know! It may help me a lot in practising my assembly reading skills (still very bad :P)

03 November 2009, 00:24	#9
ubyte Posts: n/a	i can't thank you enough for this, cheers

03 November 2009, 13:04	#12
Leffmann Join Date: Jul 2008 Location: Sweden Posts: 2,269	Andreas, Absolutely. There are a lot of space-savers and consistency fixes to be done now that I look through it again. I just translated it quickly to get it working since I don't know the algorithm itself, and when looking at a 1 to 2 instruction span you miss even the obvious things. Feel free to optimize it. I wouldn't bother about code clarity with the code being what it is Funny error message from VBCC: "Warning 175: this code is weird"

03 November 2009, 15:32	#15
Leffmann Join Date: Jul 2008 Location: Sweden Posts: 2,269	Absolutely. Readability is a must for anything you might want to extend or other way come back to. I used to do all the magical numbers and ascii art like syntaxes too. Now I use symbolic names and expressions and comment the code in a tutorial like style, and it's just so much easier to follow. This is btw one reason I moved away from ASM-One and similar assemblers. They only allow 1 source with little or no operand spacing = your program must be assembled in its entirety every run and just becomes one big lump of cluttered and incomprehensible code. I even used to think that splitting code into modules and linking them was some "high level stuff" that somehow added extra code to my executable and made it slower. Separating the code within the file or into include files helps a good deal though.

05 November 2009, 00:04	#20
andreas Zone Friend Join Date: Jun 2001 Location: Germany Age: 50 Posts: 5,857	Yay, keep on picking on those who can't fight back!

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)