crunchers with c and asm sources

[jotd]

The RNC packer works all right, but is really less effective than the official packer. So resulting packed files cannot replace original blocks in a diskfile. Still useful for file-based games, though.

I fiddled with the parameters (BLOCK_SIZE, etc.) and I only was able to make the compressed file bigger For a 16k file, it compresses to 6900 bytes whereas original packer produces a 6100 byte file size, so the difference is important.

[Galahad/FLT]

Originally Posted by jotd

The RNC packer works all right, but is really less effective than the official packer. So resulting packed files cannot replace original blocks in a diskfile. Still useful for file-based games, though.

I fiddled with the parameters (BLOCK_SIZE, etc.) and I only was able to make the compressed file bigger For a 16k file, it compresses to 6900 bytes whereas original packer produces a 6100 byte file size, so the difference is important.

If this is specifically for Magic Pockets, I can do a version that is ripped to files instead of sector loading and then there will be zero issues replacing files.

[temisu]

Originally Posted by jotd

The RNC packer works all right, but is really less effective than the official packer. So resulting packed files cannot replace original blocks in a diskfile. Still useful for file-based games, though.

This is rather surprising since the compression algorithm itself is very straightforward. Something is broken there, and the code is really bad, I wouldn't try to do non-trivial changes there.

Better to write a new implementation. It should not take more than one evening. I can think about that next week...

[jotd]

@Galahad: yes, using WRip provides all the crunched files & the offsets (all files are crunched) but I can also fix my whdload slave to do that / use files + diskfile. I wanted to be able to provide 1 diskimage so it can be used with emulators without whdload or even on a A500+floppy.

@temisu: yes the code is bad. For instance it computes a hash with 3 bytes to identify similar blocks, but what if there are a lot of stray 2-bytes similar blocks? I'm not a big compression expert, though... If you write a better implementation that would really be cool. I only need that to create a fully functional editor / disk packer for Magic Pockets. The rest is already working.

(edit: although I tried to pack my NEW1.PIN new level file with propack and it ended up way bigger than the original level, so it cannot fit on the diskfile, so maybe improving the packer is not worth it, there's a 5-10% size difference only)

[Foul]

No one tested the Cruncher i posted on #12 ?

[jotd]

Sorry Foul we kind of hijacked the thread with RNC.

[temisu]

Originally Posted by temisu

Better to write a new implementation. It should not take more than one evening. I can think about that next week...

I have to now admit that it actually isn't so easy.

Well, making the compressor was easy, but tuning it is hard. I made the code horrible by testing few things so it is yet another difficult to read implementation (it is also missing leeway calculation which is needed for some assembly decompressors)

(It is still better than the dungeon keeper utilities though )

Anyway. I dumped the current code here in case someone has some good idea how to improve it, or it might be useful for some things as is...

https://github.com/temisu/ancient_fo...e/master/extra

[jotd]

That's very good. Is it very difficult to add leeway? That would be cool if you could beat DK compression ratio, or even original compression ratio.

BTW Codetapper referenced our changes: http://codetapper.com/amiga/random-r...orthen-propack

it also contains a few nice benches for speed & efficiency against the original ProPack.

[idrougge]

Is the RNC format an IFF format?

[temisu]

Originally Posted by jotd

That's very good. Is it very difficult to add leeway? That would be cool if you could beat DK compression ratio, or even original compression ratio.

Well, if we take the test-file you sent earlier. RNC-propack does it in 6100 bytes, DK will do it in 6500. My thingy does it in 6350. So it is not entirely bad but still not good enough

I'll take a fresh look of it later this week and add the leeway as well. It would be waste of effort to leave it in half-working state.

(Probably there is some gotcha-moment to be had)

Originally Posted by jotd

BTW Codetapper referenced our changes: http://codetapper.com/amiga/random-r...orthen-propack

it also contains a few nice benches for speed & efficiency against the original ProPack.

This is cool

Originally Posted by idrougge

Is the RNC format an IFF format?

Nope, although in theory IFF could be used to contain anything, but I haven't heard that someone tried RNC...

[idrougge]

I ask because I was looking at the data files for UFO: Enemy Unknown yesterday, and they contain the familiar FORM header followed by chunks identified as RNC1…

[temisu]

Originally Posted by idrougge

I ask because I was looking at the data files for UFO: Enemy Unknown yesterday, and they contain the familiar FORM header followed by chunks identified as RNC1…

You can try the scan-functionality of the decompressor I wrote (link in the previous post) to see if it really validates as a RNC1 stream.

[temisu]

Ok, one last hijack of this thread. Since we already started on the RNC-topic

I (finally) fixed my implementation of the RNC1 compressor. Added leeway. Improved compression efficiency a bit. RNC seems to work best on short block sizes (f.e. 4096 bytes)

It's on my github, should someone want to give it a spin

[BippyM]

Hi temisu,

I am learning c++ and my first project requires me to decompress RNC packed files for SWOS. A couple of questions:

1). Am I able to use your compressor/decompressors? Full credit will naturally be given
2). Due to be very new to C++ what would be the easiest way to incorporate this into my own code. Note I am using QT due to having a GUI for my tool.

Thanks in advance

[temisu]

Originally Posted by BippyM

1). Am I able to use your compressor/decompressors? Full credit will naturally be given

I created the project under BSD 2-Clause license, which I think is the most permissive one. You can take the sources modify them as you wish, then combine it whatever you have and you dont even have to share the modifications if you do not want to.

The only requirements is to attach the disclaimer: https://github.com/temisu/ancient_fo...master/LICENSE

If you include the BZIP2 there is one special file (similar conditions):

https://github.com/temisu/ancient_fo...BZIP2Table.hpp

In short, do what you want, share what you want. Just give credit where it is due.

Originally Posted by BippyM

2). Due to be very new to C++ what would be the easiest way to incorporate this into my own code. Note I am using QT due to having a GUI for my tool.

Qt is good, just make sure you can do C++14 in your environment.

Depends what you want there are 2 ways to do it assuming you only want RNC. 1. The decompressor(s) can be taken in as a library and the compressor needs to bundled separetely.
2. just taking the bare minimum files needed for RNC, it is not much. Actual decompressor header, buffer, huffman stuff + actual compressor, decompressor.

If you want to use MSVC, you also need some patches that are not on the mainline but I can share later if you wish

[BippyM]

Hi temisu,

QT does support C++14, I just need to enable it in the .pro file. I only need RNC support (decrunch and crunch), so will look to use just the files required for that.

Thank you

[Tigerskunk]

Hey, guys... first off, thank you a lot.. I am using this, and it ->normally works like a charm.

I have one file, though, where I get an "Segmentation Error". It's not even the largest file I am packing (biggest one is 150kb, this one only around 50).

Any idea what may cause this?

[temisu]

Originally Posted by Steril707

Any idea what may cause this?

Well, smells like a bug. Is the file something you can share in the zone?

[Tigerskunk]

@Temisu: uploaded it into the zone. "Assetslevel1.bin"..
Thanks for having a look into it...

[Tigerskunk]

Now I have that problem also with another assets file.
The weird thing is, if I just add a pixel in those assets, the problem doesn't show up.

That doesn't work with the first assets file, unfortunately.

Super weird.