16 June 2019, 01:15 | #1 |
Going nowhere
Join Date: Oct 2001
Location: United Kingdom
Age: 50
Posts: 9,016
|
Fastest depacker
I'm doing a project that needs to have realtime depacking, but my usual goto choice of RNC Propack is too slow.
Each file is 3200 bytes in length, and whilst I appreciate the fastest depacker might not get the best results packing, faster depacking speed is more essential. I'm after a data file packer not an executable one. so has anyone ever done any experiments with packers and evaluated which one by far was the fastest to depack, even if its packing results were not among the best? |
16 June 2019, 03:43 | #3 |
Registered User
Join Date: Jun 2016
Location: europe
Posts: 1,053
|
I generally don't have very strict constraints when it comes to depacking, so I stick with xkp: Nuke for general stuff, Duke for mods, Fast if I need a really fast decompression. So maybe Fast would qualify. All 3 are available in source form on aminet (xkp_source.lha).
|
16 June 2019, 09:23 | #4 |
Defendit numerus
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 54
Posts: 4,488
|
You need to quantify how "fast"
There is so many packer.. But if you are not too concerned about packing ratio then LZ4 is for you. A version by Leonard: https://github.com/arnaud-carre/lz4-68k [lz4_fastest.asm is 6.81 times faster than ARJ and 4.45 times faster than UPX] Speaking of UPX.. I'm working in a new toolchain for nrv2b There is still something to squeeze out from it.. |
16 June 2019, 10:07 | #5 |
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,351
|
Fastest depacker i know of is simple RLE, aka cmpbyterun1 (like used in iff ilbm images).
But the packing ratio, of course, is also the worse... |
16 June 2019, 11:16 | #6 |
Lemon. / Core Design
Join Date: Mar 2016
Location: Tier 5
Posts: 1,213
|
We used a combination of LZ4 & ARJ in De Profundis
LZ4 is incredibly quick to decompress.. the roto-zoomer images (32k each) are decompressed in the background on the fly using idle CPU time (about 30% of a frame) |
16 June 2019, 11:56 | #7 |
Going nowhere
Join Date: Oct 2001
Location: United Kingdom
Age: 50
Posts: 9,016
|
Thanks guys, LZ4 gets the vote it seems, will try that out
|
16 June 2019, 11:57 | #8 |
Phone Homer
Join Date: Jun 2006
Location: 5150
Posts: 5,808
|
If it was for gaming probably some basic hand written lossless would be best and could probably decompressed on the fly.
Last edited by Retro1234; 16 June 2019 at 12:04. |
16 June 2019, 12:01 | #9 |
Going nowhere
Join Date: Oct 2001
Location: United Kingdom
Age: 50
Posts: 9,016
|
|
16 June 2019, 12:29 | #10 | |
Defendit numerus
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 54
Posts: 4,488
|
Quote:
Its word-based bitcode can reach terrific decompression speeds (>700kb/s on a bare A500). However normally at the expense of compression ratio. Anyway for small packets (like yours) can beat standard byte-based LZ4. It's somewhere on SGDK (aka Sega Genesis Dev Kit). https://github.com/Stephane-D/SGDK/b...r/bin/lz4w.txt |
|
16 June 2019, 14:12 | #11 |
Going nowhere
Join Date: Oct 2001
Location: United Kingdom
Age: 50
Posts: 9,016
|
What are people using as an LZ4 packer then?
|
16 June 2019, 18:11 | #12 |
Registered User
Join Date: May 2004
Location: Somewhere secret
Age: 50
Posts: 366
|
Just the reference Google LZ4 code probably, I think Anima posted all his LZ4 stuff on atari-forum (which I know you're a member of) - the packer is probably there somewhere!
|
16 June 2019, 18:35 | #13 |
OCS forever!
Join Date: Mar 2019
Location: Birmingham, UK
Posts: 418
|
This looks to be a win32 binary. https://github.com/lz4/lz4/releases/tag/v1.9.1
|
16 June 2019, 18:49 | #14 |
Defendit numerus
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 54
Posts: 4,488
|
|
16 June 2019, 22:29 | #15 |
Registered User
Join Date: Oct 2017
Location: Sunderland, England
Posts: 2,702
|
I don't really want to hijack this thread but would it be possible for one or two people to state what they think the best packers are for compression as opposed to speed of unpacking?
I also use RNC ProPack, but i'm really now down to squeezing every remaining byte out of my project and it could be the difference of it going on to two disks as opposed to just one which I would very much prefer. Geezer |
16 June 2019, 22:42 | #16 | |
Going nowhere
Join Date: Oct 2001
Location: United Kingdom
Age: 50
Posts: 9,016
|
Quote:
Propack is useful for most applications, its just on this one i'm doing, I need realtime depacking because of the way the project works, otherwise I wouldn't bother at all. |
|
16 June 2019, 23:22 | #17 | |
HOL / AMR Team Member
Join Date: Dec 2001
Location: Australia
Posts: 2,632
|
Quote:
Cranker is much the same, but will nicely decompress a file as it's loading and won't cost you any real additional memory to do so. Otherwise ProPack, XPK SHRI and PowerData are worth trying for A500 configs IMHO. Last edited by DrBong; 16 June 2019 at 23:33. |
|
17 June 2019, 21:29 | #18 | ||
Registered User
Join Date: Oct 2017
Location: Sunderland, England
Posts: 2,702
|
Quote:
Quote:
Thanks gents, I'll give these a try and post my results back into this thread when I've done it. Geezer |
||
18 June 2019, 14:30 | #19 |
Registered User
Join Date: May 2011
Location: Cambridge
Posts: 682
|
I will also plug my own gzip-based exe packer. Compression ratio is generally better than Cranker and ProPack, it unpacks in place so memory overhead is low, and it's reasonably fast (in the same ballpark as the usual suspects; not as fast as Cranker which unpacks as it loads).
http://eab.abime.net/showthread.php?t=96129 |
18 June 2019, 20:36 | #20 |
Registered User
Join Date: Jun 2008
Location: somewhere else
Posts: 523
|
Alternatively, you can try your luck with my own packfire: https://www.pouet.net/prod.php?which=54840
|
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
What depacker to use? Premier Manager II | pepsimaxman | Coders. General | 13 | 15 July 2019 11:17 |
Fastest Amiga Games | Djay | Nostalgia & memories | 73 | 09 June 2016 21:42 |
Fastest unZIP on 030? | Amiga1992 | support.Apps | 7 | 04 October 2010 01:15 |
fastest hardfile or directory ? | turrican3 | New to Emulation or Amiga scene | 10 | 06 June 2007 19:08 |
RNC Data File Depacker v2.1 | Nico | New to Emulation or Amiga scene | 8 | 05 May 2002 18:05 |
|
|