English Amiga Board


Go Back   English Amiga Board > Coders > Coders. General

 
 
Thread Tools
Old 14 June 2017, 22:37   #1
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 389
Are there any compressors/tools that re-arrange/block-divide data(pictures)?

AFAIK (corrections welcome) both IFF and GIF store picture data on a line by line basis.
Colours are typically much more locally clustered, and this is also how JPEG (and others?) arrange pixel data to take advantage of local similarities.

Are there any compressors or other tools that will do such blocking or rearranging of data to increase compression ratios?
Can you affect the ratio for a standard IFF/GIF by re-arranging the palette and thereby the data combinations?
NorthWay is offline  
AdSense AdSense  
Old 14 June 2017, 23:10   #2
temisu
Registered User

 
Join Date: Mar 2017
Location: Tallinn / Estonia
Posts: 20
Well, all modern picture compressors have more context than a single scanline.

PNG has a pre-filter which takes input from 3 pixels from previous line and 1 previous pixel from same line. (after filtering it compresses the delta with deflate)

JPEG operates in blocks and JPEG 2000 and other wavelet compressors have naturally 2-dimensional context since the picture is filtered by both x and y axis.

IFF/ILBM is RLE-based. Only way to improve it by palette tricks would be to reduce the number of palette entries. (maybe to get rid of bitplanes)

GIF is LZW, same thing really. The encoding efficiency does not really depend on the ordering of the code words. Obviously dropping palette entries will have some effect since the initial dictionary is smaller.
temisu is offline  
Old 15 June 2017, 11:11   #3
chb
Registered User

 
Join Date: Dec 2014
Location: germany
Posts: 45
Quote:
Originally Posted by NorthWay View Post
AFAIK (corrections welcome) both IFF and GIF store picture data on a line by line basis.
Colours are typically much more locally clustered, and this is also how JPEG (and others?) arrange pixel data to take advantage of local similarities.

Are there any compressors or other tools that will do such blocking or rearranging of data to increase compression ratios?
Can you affect the ratio for a standard IFF/GIF by re-arranging the palette and thereby the data combinations?
For bitplane-based, run-length-encoded images, you can achieve some improvement by re-arranging the palette. Imagine e.g. a 4-bitplane image with a 2-color checkerboard pattern. If you'd use color 0 and color 15 for those two colors, you get the pattern in all bitplanes, so bad compressibility for RLE. If you use color 0 and 1, three out of for bitplanes are just constant -> perfect compressibility. In real world applications, the gains are of course lower, but I remember a thread here on EAB were a reduction of ~20% was given for real-world images.
chb is offline  
Old 15 June 2017, 11:44   #4
meynaf
son of 68k
meynaf's Avatar
 
Join Date: Nov 2007
Location: Lyon / France
Age: 44
Posts: 2,432
From what i've seen the best lossless image compression ratios are obtained by taking uncompressed chunky data and crunching that with good general purpose crunchers, e.g. you can take a windows bmp image and compress it with bzip2, you'll get a smaller file than with iff, gif or png.
meynaf is offline  
Old 15 June 2017, 12:13   #5
temisu
Registered User

 
Join Date: Mar 2017
Location: Tallinn / Estonia
Posts: 20
Quote:
Originally Posted by chb View Post
For bitplane-based, run-length-encoded images, you can achieve some improvement by re-arranging the palette. Imagine e.g. a 4-bitplane image with a 2-color checkerboard pattern. If you'd use color 0 and color 15 for those two colors, you get the pattern in all bitplanes, so bad compressibility for RLE. If you use color 0 and 1, three out of for bitplanes are just constant -> perfect compressibility. In real world applications, the gains are of course lower, but I remember a thread here on EAB were a reduction of ~20% was given for real-world images.
IFF/ILBM RLE operates per-pixel (chunky) instead of per bitplane. You'd have save the image as uncompressed and then compress it by some generic compression algorithm to do this...
temisu is offline  
Old 15 June 2017, 12:23   #6
meynaf
son of 68k
meynaf's Avatar
 
Join Date: Nov 2007
Location: Lyon / France
Age: 44
Posts: 2,432
Quote:
Originally Posted by temisu View Post
IFF/ILBM RLE operates per-pixel (chunky) instead of per bitplane. You'd have save the image as uncompressed and then compress it by some generic compression algorithm to do this...
No, IFF-ILBM is planar mode. It outputs first line for each plane, then next line for each plane, etc. Even IFF-24 is planar.
meynaf is offline  
Old 15 June 2017, 13:28   #7
invent
pixels

invent's Avatar
 
Join Date: May 2014
Location: Australia
Age: 46
Posts: 167
Hi Northway, I did notice a refinement in the reduction of file size by arranging the indexed colour palette, possibly darkest/lightest or most common colours or some other random method?

Not sure if it was IFF or gif/png but had some success, this was back in the day of dialup modems and end of Amiga Classics era.

A quick test for example of exporting a png-8 file via photoshop with no distinguishable differences in colours of some sprites has a file size changing from 10.03K and 10.13K file just by changing from either "Selective" or "adaptive" palettes.

The test file "Rygar" sprites has 20 colour and I was saving as a 32 png file.

I hope this is some use, if I find something else will add more detail, but back in the day when I descovered the reduction I was very surprised.

Jpg files, only success was using some extra tools with masking but was time consuming selecting which areas I wanted more compressed (usually flat colours)
invent is offline  
Old 15 June 2017, 20:53   #8
idrougge
Registered User
 
Join Date: Sep 2007
Location: Stockholm
Posts: 3,056
Quote:
Originally Posted by meynaf View Post
No, IFF-ILBM is planar mode. It outputs first line for each plane, then next line for each plane, etc. Even IFF-24 is planar.
Well, it's neither per-bitplane or chunky. It's per-line, per-bitplane. That's why it's called InterLeaved BitMap.
idrougge is offline  
Old 17 June 2017, 13:48   #9
Leffmann
 
Join Date: Jul 2008
Location: Sweden
Posts: 2,023
Quote:
Originally Posted by NorthWay View Post
AFAIK (corrections welcome) both IFF and GIF store picture data on a line by line basis.
Colours are typically much more locally clustered, and this is also how JPEG (and others?) arrange pixel data to take advantage of local similarities.
I played a bit with this a while back and found that rearranging the data according to a Hilbert-curve gave better compression for simple images, but it wasn't much, maybe 20% on monochrome images at best, and with higher bit-depths the difference was not worth it.

The compression gain is definitely there, but it's too little and too cumbersome to be of practical use.
Leffmann is offline  
Old 16 August 2017, 00:14   #10
Trachu
Registered User
 
Join Date: Dec 2015
Location: Poland
Posts: 172
Quote:
Originally Posted by NorthWay View Post
AFAIK (corrections welcome) both IFF and GIF store picture data on a line by line basis.
Colours are typically much more locally clustered, and this is also how JPEG (and others?) arrange pixel data to take advantage of local similarities.

Are there any compressors or other tools that will do such blocking or rearranging of data to increase compression ratios?
Can you affect the ratio for a standard IFF/GIF by re-arranging the palette and thereby the data combinations?
Try this:
http://www.pouet.net/prod.php?which=71288
Trachu is offline  
Old 16 August 2017, 14:39   #11
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 389
Thanks!
NorthWay is offline  
Old 01 September 2017, 11:20   #12
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 389
Did anyone try that tool? I might be missing what its point was.
"ZZ_MadMan" went from a 62K IFF to a 67K IFF when saved by the tool. I haven't tried converting to raw and compressing with some other tool if that was the purpose...
NorthWay is offline  
Old 01 September 2017, 12:51   #13
chb
Registered User

 
Join Date: Dec 2014
Location: germany
Posts: 45
Quote:
Originally Posted by NorthWay View Post
Did anyone try that tool? I might be missing what its point was.
"ZZ_MadMan" went from a 62K IFF to a 67K IFF when saved by the tool. I haven't tried converting to raw and compressing with some other tool if that was the purpose...
From the readme:
Quote:
About BPLOptimize: BPLOptimize is an IFF ILBM and DEGAS PI1 converter that optimizes bitplanes for better compression with LZ-like compression algorithms.
IFF-ILBM only applies RLE-compression, so yes, you should try with another (Lz-like) compressor, probably raw conversion is not necessary.

EDIT: the tool has an option ("-nc") to write uncompressed IFFs, not sure if the better compression ratio is also valid when LZ-compression is applied to already RLE-encoded files.

Last edited by chb; 01 September 2017 at 19:00.
chb is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
DivIDE Type Device for Amstrad CPC 464 manic23 request.Other 9 16 August 2014 11:55
3D import - Tools & Example source of 3D data tassel Coders. Asm / Hardware 12 09 June 2014 23:00
Romable XPK compressors Romanujan support.Other 2 23 January 2011 14:54
ERROR: Dalek Attack (Integer Divide By Zero) Hungry Horace project.Killergorilla's WHD packs 18 20 September 2009 23:17
Need program to auto arrange icons lopos2000 support.Apps 3 31 July 2005 22:04

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 15:07.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.18390 seconds with 14 queries