Fastest depacker
 

I'm doing a project that needs to have realtime depacking, but my usual goto choice of RNC Propack is too slow.

Each file is 3200 bytes in length, and whilst I appreciate the fastest depacker might not get the best results packing, faster depacking speed is more essential.

I'm after a data file packer not an executable one.

so has anyone ever done any experiments with packers and evaluated which one by far was the fastest to depack, even if its packing results were not among the best?

Did you try lzo ?

I generally don't have very strict constraints when it comes to depacking, so I stick with xkp: Nuke for general stuff, Duke for mods, Fast if I need a really fast decompression. So maybe Fast would qualify. All 3 are available in source form on aminet (xkp_source.lha).

You need to quantify how "fast" :)
There is so many packer..

But if you are not too concerned about packing ratio then LZ4 is for you.

A version by Leonard:
https://github.com/arnaud-carre/lz4-68k
[lz4_fastest.asm is 6.81 times faster than ARJ and 4.45 times faster than UPX]

Speaking of UPX.. I'm working in a new toolchain for nrv2b :D
There is still something to squeeze out from it..

Fastest depacker i know of is simple RLE, aka cmpbyterun1 (like used in iff ilbm images).
But the packing ratio, of course, is also the worse...

We used a combination of LZ4 & ARJ in De Profundis

LZ4 is incredibly quick to decompress.. the roto-zoomer images (32k each) are decompressed in the background on the fly using idle CPU time (about 30% of a frame)

Thanks guys, LZ4 gets the vote it seems, will try that out :)

If it was for gaming probably some basic hand written lossless would be best and could probably decompressed on the fly.

It is for a game to replace an interrupt loader. If LZ4 doesn't work I will have to tackle it a different way.

If LZ4 is not enough, there is also LZ4W.
Its word-based bitcode can reach terrific decompression speeds (>700kb/s on a bare A500).
However normally at the expense of compression ratio.
Anyway for small packets (like yours) can beat standard byte-based LZ4.

It's somewhere on SGDK (aka Sega Genesis Dev Kit).
https://github.com/Stephane-D/SGDK/b...r/bin/lz4w.txt

What are people using as an LZ4 packer then?

Just the reference Google LZ4 code probably, I think Anima posted all his LZ4 stuff on atari-forum (which I know you're a member of) - the packer is probably there somewhere!

This looks to be a win32 binary. https://github.com/lz4/lz4/releases/tag/v1.9.1

I use this:
https://github.com/encode84/lz4x

It's by a well known coder in compression scene.

I don't really want to hijack this thread but would it be possible for one or two people to state what they think the best packers are for compression as opposed to speed of unpacking?

I also use RNC ProPack, but i'm really now down to squeezing every remaining byte out of my project and it could be the difference of it going on to two disks as opposed to just one which I would very much prefer.

Geezer

Propack is usually very good, Shrinkler is better, but its VERY slow to depack, but Shrinkler was the deciding factor in me getting Nitro to fit on one disk like the original, Propack couldn't quite manage it.

Propack is useful for most applications, its just on this one i'm doing, I need realtime depacking because of the way the project works, otherwise I wouldn't bother at all.

Yeh, what Phill said. Shrinkler will usually get the best bang for your buck for compression, but it's very very slow (e.g. took ~3 minutes IIRC to decompress the 1.4MB Crazy Priest game exec on a standard A500 when I one-disked the game). Not surprising as Shrinkler was likely designed to compress demos.

Cranker is much the same, but will nicely decompress a file as it's loading and won't cost you any real additional memory to do so. Otherwise ProPack, XPK SHRI and PowerData are worth trying for A500 configs IMHO.

Thanks gents, I'll give these a try and post my results back into this thread when I've done it.

Geezer

I will also plug my own gzip-based exe packer. Compression ratio is generally better than Cranker and ProPack, it unpacks in place so memory overhead is low, and it's reasonably fast (in the same ballpark as the usual suspects; not as fast as Cranker which unpacks as it loads).

http://eab.abime.net/showthread.php?t=96129

Alternatively, you can try your luck with my own packfire: https://www.pouet.net/prod.php?which=54840