Best Compression Methods For...

[Lonewolf10]

Hi,

I know there are a few different compression methods (e.g. LZ77, LZW, RLE and Huffman encoding), but is there a general guide to which compression method is best (and not bound by copyright/licensing laws)?

I'm in the middle of making a game and the raw graphics data (so far) is 32KB in size, but storing it in .IFF format reduces it to around 15KB and using .PNG format it is 9KB! I'm thinking that .PNG is best.

Does anyone know which methods were used in the past to compress game data? Does anyone even bother compressing game data these days when making new Amiga games??

[pmc]

I presume, if you're storing your graphics in .png format, that you then have to have a routine to convert those graphics into raw bitplane data ready for display?

Why not just use raw data to start with and crunch it with a data cruncher prior to storing? Then just uncrunch the data as necessary as and when you load it.

[Thorham]

Originally Posted by Lonewolf10

Does anyone even bother compressing game data these days when making new Amiga games??

Depends on your target storage method. If you have a lot of data and want the game to run of floppies, then you'll probably need compression. If floppies aren't supported, then I probably wouldn't bother because of todays large HD's.

[Yakumo9275]

well, if its .IFF its probably ILBM or if its from the PC side, its PBM, they are just run length compression. PNG uses zlib/pkzip DEFLATE internally.. which is LZ77 + Huffman combined. it is not covered by active patents.

[Lonewolf10]

Originally Posted by pmc

I presume, if you're storing your graphics in .png format, that you then have to have a routine to convert those graphics into raw bitplane data ready for display?

Yeah. I'm currently working on that in AMOS Basic and once working will convert the code into ASM I'm on the IDAT bit now (the image data compressed using LZ77 + Huffman, as Yakumo9275 mentioned above), but Huffman code (especially variable length) always gave me a headache, which is why I avoided it in the past. However, it seems to be the best compression method available so I'm attempting to tackle it head on

Originally Posted by pmc

Why not just use raw data to start with and crunch it with a data cruncher prior to storing? Then just uncrunch the data as necessary as and when you load it.

I had thought of that, but it is much more convenient if I just have one image file for each image. In the past I'd have the image in multiple formats .PNG (created in MS Paint), .IFF (viewable on Amiga's) and .RAW
(for use inside my ASM programs).
NB: I didn't know until recently that Personal Paint code decode .PNG files

Originally Posted by Yakumo9275

well, if its .IFF its probably ILBM or if its from the PC side, its PBM, they are just run length compression. PNG uses zlib/pkzip DEFLATE internally.. which is LZ77 + Huffman combined. it is not covered by active patents.

Thanks for the info

[Mrs Beanbag]

I use a lot of compression in my code. My main port of call is a variant of LZSS with variable length offset fields.

For my graphics I started using AMOS's graphics compression command "spack", which does quite well when saved using powerpacker library ("ppsave"/"ppload" instructions from the AMOS Compiler extension). However I replaced powerpacker with my own routine later on. I'm still using the AMOS bank format for some of the graphics (the asm source code is available), but for tiles and bobs I do RLE, and everything followed by LZSS.

When compressing audio sample data, it is beneficial to "differentiate" the sample first. That is, store the bytewise difference between each sample and the next. This typically results in a narrower range of values.

I haven't touched Huffman yet although I'd like to. Also look up Range coding - might be a bit much for an Amiga but it's quite interesting.

[Lonewolf10]

Thanks for the tips, Mrs Beanbag

[Cyprian]

Originally Posted by Lonewolf10

Yeah. I'm currently working on that in AMOS Basic and once working will convert the code into ASM I'm on the IDAT bit now (the image data compressed using LZ77 + Huffman, as Yakumo9275 mentioned above), but Huffman code (especially variable length) always gave me a headache, which is why I avoided it in the past. However, it seems to be the best compression method available so I'm attempting to tackle it head on




I had thought of that, but it is much more convenient if I just have one image file for each image. In the past I'd have the image in multiple formats .PNG (created in MS Paint), .IFF (viewable on Amiga's) and .RAW
(for use inside my ASM programs).
NB: I didn't know until recently that Personal Paint code decode .PNG files




Thanks for the info

there you can find source code (asm 68k ) for PNG and LZ77 decompression http://dhs.nu/files.php?t=democreation

[MagerValp]

Hmm, I have a compression method I developed for the C64 that should port over quite easily. I wrote test unpacker in AmigaE as a proof of concept:

PROC backunpack(indata:PTR TO CHAR, inlen:LONG, outdata:PTR TO CHAR)
    DEF inpos = 0
    DEF outpos = 0
    DEF backpos
    DEF c
    DEF l
    
    WHILE inpos < inlen
        c := indata[inpos++]
        SELECT 256 OF c
            CASE 0
                RETURN outpos
            CASE $80 TO $ff
                l := c - 127
                REPEAT
                    outdata[outpos++] := indata[inpos++]
                    DEC l
                UNTIL l = 0
            DEFAULT
                l := c + 2
                backpos := outpos - 256 + indata[inpos++]
                REPEAT
                    outdata[outpos++] := outdata[backpos++]
                    DEC l
                UNTIL l = 0
        ENDSELECT
    ENDWHILE
ENDPROC

As you can see it's quite simple but it still performs better than LZ77.

[Cyprian]

MagerValp it looks like RLE compression

[MagerValp]

No, it does repeating patterns, its closest ancestor is LZSS.

[Lonewolf10]

Originally Posted by MagerValp

No, it does repeating patterns, its closest ancestor is LZSS.

I know what LZ is short for (Lempel-Ziv), but what is the SS for?

[StingRay]

Storer–Szymanski

[Lonewolf10]

Thanks, Stingray.

[Photon]

Do the gfx come from an Amiga or a PC paint program? How many colors are we talking? What is the average width/height of the image files?

If you're not cramped for memory, I recommend to just save in the best format available in the paint program and load it as that format. Speeds up development by skipping the conversion and compression steps.

[Lonewolf10]

Originally Posted by Photon

Do the gfx come from an Amiga or a PC paint program? How many colors are we talking? What is the average width/height of the image files?

Most of the graphics are currently drawn using Microsoft Paint (on my laptop), with 16 colours. The master image I have at the moment is something like 2000x1000, but once everything is drawn I will split the graphics into more manageable chunks (e.g. 320x200 or so) with each image storing graphics for one or two levels.

I'm considering using a CGI image for the title screen, and possibly making use of the super-hires mode. I already have some polygon corridors and towers created using Wings 3D (again, on my laptop), which I made last year for fun


I already have working code to inflate .PNG files in AMOS Basic, and am slowly converting it to ASM

[Photon]

Then you're set I'd say So unless you absolutely must do some cutting or editing on the Amiga end, it would be unnecessary to convert them to iff or add special compression or something like that.