English Amiga Board


Go Back   English Amiga Board > Coders > Coders. General

 
 
Thread Tools
Old 18 March 2016, 16:50   #1
majikeyric
Registered User

majikeyric's Avatar
 
Join Date: Oct 2015
Location: France
Posts: 61
Compression method for real-time depacked bobs ?

Hi,

In my actual game project, I have a shitload of bobs to manage, mainly for the player.

I would want to keep enemies's bobs in chipmem and depack on the fly the current player's bob from slowmem to chipmem.

Which compression method could fit my needs ? acceptable ratio and ultra fast decompress...

I have read that RNC1 was good and would want to give it a try but I haven't found win32 binaries of ProPack nor sources I could compile.

Thanks
majikeyric is offline  
AdSense AdSense  
Old 18 March 2016, 17:26   #2
DanScott
Registered User

DanScott's Avatar
 
Join Date: Mar 2016
Location: Belgrade, Serbia
Posts: 217
Here's the depack source..

Code:
*------------------------------------------------------------------------------
* PRO-PACK Unpack Source Code (Compact Version) - MC68000, Method 1
*
* Copyright (c) 1991,92 Rob Northen Computing, U.K. All Rights Reserved.
*
* File: RNC_1C.S
*
* Date: 24.3.92
*------------------------------------------------------------------------------

*------------------------------------------------------------------------------
* Equates
*------------------------------------------------------------------------------

HEADER_LEN	EQU	18
RAW_TABLE	EQU	0
POS_TABLE	EQU	RAW_TABLE+16*8
LEN_TABLE	EQU	POS_TABLE+16*8
BUFSIZE		EQU	16*8*3

counts		EQUR	d4
key			EQUR	d5
bit_buffer	EQUR	d6
bit_count	EQUR	d7

input		EQUR	a3
output		EQUR	a4
output_hi	EQUR	a5

*------------------------------------------------------------------------------
* Macros
*------------------------------------------------------------------------------

getrawREP	MACRO
getrawREP2\@
		move.b	(input)+,(output)+
		dbra	d0,getrawREP2\@
		ENDM

*------------------------------------------------------------------------------
* PRO-PACK Unpack Routine (Compact Version) - MC68000, Method 1
*
* on entry,
*	a0.l = start address of packed file
*	a1.l = start address to unpack file
*	(note: a1 cannot be equal to a0)
*	stack space required: $1DC bytes
*
*	all other registers are preserved
*------------------------------------------------------------------------------
Unpack_RNC1:

		movem.l	d0-d7/a0-a5,-(sp)
		lea	-BUFSIZE(sp),sp
		move.l	sp,a2
		addq.w	#4,a0
		bsr	read_long
		lea	HEADER_LEN-8(a0),input
		move.l	a1,output
		lea	(output,d0.l),output_hi

		moveq.l	#0,bit_count
		move.b	1(input),bit_buffer
		rol.w	#8,bit_buffer
		move.b	(input),bit_buffer
		moveq.l	#2,d0
		moveq.l	#2,d1
		bsr	input_bits
unpack2
		move.l	a2,a0
		bsr	make_huftable
		lea	POS_TABLE(a2),a0
		bsr	make_huftable
		lea	LEN_TABLE(a2),a0
		bsr	make_huftable
		moveq.l	#-1,d0
		moveq.l	#16,d1
		bsr	input_bits
		move.w	d0,counts
		subq.w	#1,counts
		bra.s	unpack5		
unpack3
		lea	POS_TABLE(a2),a0
		moveq.l	#0,d0
		bsr.s	input_value
		neg.l	d0
		lea	-1(output,d0.l),a1
		lea	LEN_TABLE(a2),a0
		bsr.s	input_value
		move.b	(a1)+,(output)+
unpack4
		move.b	(a1)+,(output)+
		dbra	d0,unpack4
unpack5
		move.l	a2,a0
		bsr.s	input_value
		subq.w	#1,d0
		bmi.s	unpack6
		getrawREP
		move.b	1(input),d0
		rol.w	#8,d0
		move.b	(input),d0
		lsl.l	bit_count,d0
		moveq.l #1,d1
		lsl.w	bit_count,d1
		subq.w	#1,d1
		and.l	d1,bit_buffer
		or.l	d0,bit_buffer
unpack6
		dbra	counts,unpack3
		cmp.l	output_hi,output
		bcs.s	unpack2

		lea	BUFSIZE(sp),sp
		movem.l	(sp)+,d0-d7/a0-a5
		rts

input_value
		move.w	(a0)+,d0
		and.w	bit_buffer,d0
		sub.w	(a0)+,d0
		bne.s	input_value
		move.b	16*4-4(a0),d1
		sub.b	d1,bit_count
		bge.s	input_value2
		bsr.s	input_bits3
input_value2
		lsr.l	d1,bit_buffer
		move.b	16*4-3(a0),d0
		cmp.b	#2,d0
		blt.s	input_value4
		subq.b	#1,d0
		move.b	d0,d1
		move.b	d0,d2
		move.w	16*4-2(a0),d0
		and.w	bit_buffer,d0
		sub.b	d1,bit_count
		bge.s	input_value3
		bsr.s	input_bits3
input_value3
		lsr.l	d1,bit_buffer
		bset	d2,d0
input_value4
		rts

input_bits
		and.w	bit_buffer,d0
		sub.b	d1,bit_count
		bge.s	input_bits2
		bsr.s	input_bits3
input_bits2
		lsr.l	d1,bit_buffer
		rts

input_bits3
		add.b	d1,bit_count
		lsr.l	bit_count,bit_buffer
		swap	bit_buffer
		addq.w	#4,input
		move.b	-(input),bit_buffer
		rol.w	#8,bit_buffer
		move.b	-(input),bit_buffer
		swap	bit_buffer
		sub.b	bit_count,d1
		moveq.l	#16,bit_count
		sub.b	d1,bit_count
		rts

read_long
		moveq.l	#3,d1
read_long2
		lsl.l	#8,d0
		move.b	(a0)+,d0
		dbra	d1,read_long2
		rts

make_huftable
		moveq.l	#$1f,d0
		moveq.l	#5,d1
		bsr.s	input_bits
		subq.w	#1,d0
		bmi.s	make_huftable8
		move.w	d0,d2
		move.w	d0,d3
		lea	-16(sp),sp
		move.l	sp,a1
make_huftable3
		moveq.l	#$f,d0
		moveq.l	#4,d1
		bsr.s	input_bits
		move.b	d0,(a1)+
		dbra	d2,make_huftable3
		moveq.l	#1,d0
		ror.l	#1,d0
		moveq.l	#1,d1
		moveq.l	#0,d2
		movem.l	d5-d7,-(sp)
make_huftable4
		move.w	d3,d4
		lea	12(sp),a1
make_huftable5
		cmp.b	(a1)+,d1
		bne.s	make_huftable7
		moveq.l	#1,d5
		lsl.w	d1,d5
		subq.w	#1,d5
		move.w	d5,(a0)+
		move.l	d2,d5
		swap	d5
		move.w	d1,d7
		subq.w	#1,d7
make_huftable6
		roxl.w	#1,d5
		roxr.w	#1,d6
		dbra	d7,make_huftable6
		moveq.l	#16,d5
		sub.b	d1,d5
		lsr.w	d5,d6
		move.w	d6,(a0)+
		move.b	d1,16*4-4(a0)
		move.b	d3,d5
		sub.b	d4,d5
		move.b	d5,16*4-3(a0)
		moveq.l	#1,d6
		subq.b	#1,d5
		lsl.w	d5,d6
		subq.w	#1,d6
		move.w	d6,16*4-2(a0)
		add.l	d0,d2
make_huftable7
		dbra	d4,make_huftable5
		lsr.l	#1,d0
		addq.b	#1,d1
		cmp.b	#17,d1
		bne.s	make_huftable4
		movem.l	(sp)+,d5-d7
		lea	16(sp),sp
make_huftable8
		rts
DanScott is offline  
Old 18 March 2016, 17:34   #3
DanScott
Registered User

DanScott's Avatar
 
Join Date: Mar 2016
Location: Belgrade, Serbia
Posts: 217
http://aminet.net/package/util/pack/RNC_ProPack

PC packer in here.. just run from command prompt
DanScott is offline  
Old 18 March 2016, 17:34   #4
majikeyric
Registered User

majikeyric's Avatar
 
Join Date: Oct 2015
Location: France
Posts: 61
Thanks Dan (can be found on Aminet : http://aminet.net/package/util/pack/RNC_ProPack )
but I miss a win32 packer tool (The one in the archive is for DOS) .


*EDIT*: but that works with DOSBox!

Last edited by majikeyric; 18 March 2016 at 17:46.
majikeyric is offline  
Old 18 March 2016, 18:00   #5
DanScott
Registered User

DanScott's Avatar
 
Join Date: Mar 2016
Location: Belgrade, Serbia
Posts: 217
I'm actually using the amiga executable, run through WinUAE does the job it needs to do
DanScott is offline  
Old 18 March 2016, 20:32   #6
AnimaInCorpore
Registered User
 
Join Date: Nov 2012
Location: Willich/Germany
Posts: 154
Quote:
Originally Posted by majikeyric View Post
Which compression method could fit my needs ? acceptable ratio and ultra fast decompress...
LZ4 could be an option.
AnimaInCorpore is offline  
Old 19 March 2016, 10:48   #7
majikeyric
Registered User

majikeyric's Avatar
 
Join Date: Oct 2015
Location: France
Posts: 61
I did some tests:

RNC1 is too slow for real-time depacking but LZ4 is rather good!

@Anima: Is there a more recent depack routine maybe even more optimized than the one in the thread on atari-forum ?
Which version of the LZ4 compressor suits best with the depack routine for speed ? the latest ?
majikeyric is offline  
Old 19 March 2016, 14:38   #8
chb
Registered User

 
Join Date: Dec 2014
Location: germany
Posts: 48
Axis/Oxyron released the source for his demo "Rock Lobster", it includes a packer/depacker that he claims "is unbelievable fast (about 180KB/s on a stock Amiga 500)" at depacking (and given his track record, there's no reason not to believe this). Includes C source for the packer and asm for the depacker:
https://github.com/AxisOxy/Planet-Rocklobster
chb is offline  
Old 19 March 2016, 18:39   #9
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 391
I'd take a look at the XFD framework as it had a lot of good tools and performance tools to compare algorithms. Some of which were speed oriented while others were compression ratio focused.
NorthWay is online now  
Old 19 March 2016, 19:08   #10
DanScott
Registered User

DanScott's Avatar
 
Join Date: Mar 2016
Location: Belgrade, Serbia
Posts: 217
For a player "bob" or sprite with many frames of animation, you could probably write your own simple packer & depacker.

I would imagine there would be many parts of each bob that are the same in each frame of animation.

Some kind of char / 8x8 tile dictionary that you can construct the bobs from on the fly.

Also, if you are storing X flipped versions of your sprite, consider storing them the top half flipped one way, and the bottom half flipped the other.. you then either flip the top of the bottom depending on which way you want the sprite to face
DanScott is offline  
Old 19 March 2016, 21:20   #11
AnimaInCorpore
Registered User
 
Join Date: Nov 2012
Location: Willich/Germany
Posts: 154
Quote:
Originally Posted by majikeyric View Post
@Anima: Is there a more recent depack routine maybe even more optimized than the one in the thread on atari-forum ?
Which version of the LZ4 compressor suits best with the depack routine for speed ? the latest ?
There's a simple optimised version below in that thread.

The worst part is to deal with little endian data but that could be changed in your data afterwards if needed. Unfortunately they were focusing on that architecture for speed.

The current LZ4 depack routine is based on the LZ4 streaming format which is standard for every LZ4 program output so I would suggest to choose the HC option and the latest version for compression.
AnimaInCorpore is offline  
Old 19 March 2016, 22:34   #12
majikeyric
Registered User

majikeyric's Avatar
 
Join Date: Oct 2015
Location: France
Posts: 61
@chb: Thanks, I did some tests with the Doynax decompressor but it isn't fast enough in this case unfortunately.


@NorthWay: I didn't know the XFD system, I will give it a try!


@Dan: Thanks! good advices!
"8x8 tiles map" of main character's bob won't help because each frame is very different.
But I like your bob flipping method! How to perform the fastest x-flip ? with pre-calc values ?


@Anima: Thanks for the information!
majikeyric is offline  
Old 19 March 2016, 23:03   #13
DanScott
Registered User

DanScott's Avatar
 
Join Date: Mar 2016
Location: Belgrade, Serbia
Posts: 217
yes, you can have a 256 byte table of flipped bytes
DanScott is offline  
Old 20 March 2016, 00:55   #14
NorthWay
Registered User
 
Join Date: May 2013
Location: Grimstad / Norway
Posts: 391
Quote:
Originally Posted by majikeyric View Post
@NorthWay: I didn't know the XFD system, I will give it a try!
Sorry much, my Amiga touch is rusty - I was of course thinking of XPK...
NorthWay is online now  
Old 30 March 2016, 23:24   #15
pandy71
Registered User
 
Join Date: Jun 2010
Location: PL
Posts: 1,546
Maybe texture packing can be adopted https://en.wikipedia.org/wiki/S3_Texture_Compression
pandy71 is offline  
Old 16 April 2016, 18:41   #16
ReadOnlyCat
Code Kitten

 
Join Date: Aug 2015
Location: Montreal/Canadia
Age: 46
Posts: 1,012
Quote:
Originally Posted by pandy71 View Post
Maybe texture packing can be adopted https://en.wikipedia.org/wiki/S3_Texture_Compression
I doubt this will be usable for close to real time needs. Also this is lossy compression which does not work too well with palette based images. I would not exclude its use for static backgrounds but I doubt it has practical applications for bob storage.

Regarding speed, I doubt you can beat RLE. Sure, it is not highly packed but if the bitmaps are appropriate it can do a really good enough job and will unpack faster than any Lempel-Ziv or inspired scheme simply because it needs much less memory reads. And as someone mentioned (sorry, forgot your name! ), you can adapt the algorithm to particular features of your images if needed. This is particularly easy to do with RLE.

Moreover, even with a simple algorithm such as RLE, there is ample choice of methods for the compressing process and you can tweak that to your images to produce higher compression ratios.
Note that higher compression ratios automatically translate to faster decompression speed with RLE, this is not a general case for Lempel-Ziv packers (it depends highly on the data).

Hope this helps!
ReadOnlyCat is offline  
Old 13 June 2016, 16:15   #17
Stef_
 
Posts: n/a
I had recently to develop a *very fast* unpacker for the Sega Megadrive which also uses a 68000 CPU. The compressor i developed is based on LZ4 but optimized for unpacking speed so it does provide better unpacking speed than classic LZ4 as the cost of a lower compression ratio. Also i designed the codec (that i called LZ4W) to compress small packets of data (between 1KB and 8 KB) so it generally compresses better for small amount of data.

You can find more information about LZ4W here:
https://github.com/Stephane-D/SGDK/b...r/bin/lz4w.txt

If you want to give a try :
https://www.dropbox.com/sh/smgwbi8g6...NXo703vQa?dl=0

Hope it can be useful for you =)

Last edited by Stef_; 22 June 2016 at 15:09. Reason: fixing typo
 
Old 13 June 2016, 16:41   #18
ReadOnlyCat
Code Kitten

 
Join Date: Aug 2015
Location: Montreal/Canadia
Age: 46
Posts: 1,012
Quote:
Originally Posted by Stef_ View Post
I had recently to develop a *very fast* unpacker for the Sega Megadrive which also uses a 68000 CPU.
Oh, I guess you must also be Stef from the Spriteminds and Sega-16 sites?
I have noted your posts there as well as those of gasega68k.

Nice to see you there and thanks for the contribution!
ReadOnlyCat is offline  
Old 13 June 2016, 16:51   #19
Stef_
 
Posts: n/a
Quote:
Originally Posted by ReadOnlyCat View Post
Oh, I guess you must also be Stef from the Spriteminds and Sega-16 sites?
I have noted your posts there as well as those of gasega68k.

Nice to see you there and thanks for the contribution!
Yeah the same one I just found that topic looking for compression related discussions and realized my stuff for Megadrive could work in that case too
Gasega68k is doing wonders on Sega Megadrive, hope to have news from him soon.

Last edited by Stef_; 13 June 2016 at 16:57.
 
Old 13 June 2016, 20:45   #20
ReadOnlyCat
Code Kitten

 
Join Date: Aug 2015
Location: Montreal/Canadia
Age: 46
Posts: 1,012
Quote:
Originally Posted by Stef_ View Post
Yeah the same one I just found that topic looking for compression related discussions and realized my stuff for Megadrive could work in that case too
Gasega68k is doing wonders on Sega Megadrive, hope to have news from him soon.
Cool!

I won't be off-topic-ing the thread any longer but I really think you did great to intervene. The 16 bit micros and consoles have a great CPU in common and both scenes would profit a lot from inter-exchange of ideas and code!
I must say that your and gasega68k posts actually gave me a number of ideas for Amiga algorithms which I am looking forward to implement!

Cheers!
ReadOnlyCat is offline  
AdSense AdSense  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Amiga Real-Time 3D Graphics Jherek Carnelia Coders. Tutorials 12 27 October 2017 04:26
WTB: Amiga Real-Time 3d graphics Fridrik MarketPlace 0 27 September 2012 02:53
A1200 Real Time Clock Eclipse support.Hardware 4 22 March 2011 03:18
WinUAE time compression/speed up Neil support.WinUAE 12 11 July 2009 20:37
Reading the Real Time Clock girv Coders. General 5 04 September 2007 19:30

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 23:26.


Powered by vBulletin® Version 3.8.8 Beta 1
Copyright ©2000 - 2017, vBulletin Solutions, Inc.
Page generated in 0.57634 seconds with 12 queries