RNC Propack "in place" decompression does not always work

[roondar]

Originally Posted by ross

About RNC decoding speed: it's slower than NRV.

Yes indeed. I did some testing yesterday (replaced RNC with NRV2R) and it was notably faster for decrunching.

It also works with my troublesome file, so that's nice too

[roondar]

A small update: I've fully moved one of my long-term projects over to NRV2R now and I'm impressed!

Loading+decrunching times have decreased by about 25% and the compression rate seems to be around 3-5% better than RNC manages. Great stuff

[Antiriad_UK]

crazy that 30 year old tech can be improved. God bless retro nerds (also thanks ross - used this in my last intro. )

[jotd]

I think I found a better in-place RN decruncher in Heimdall code. Whdload fails to unpack some files, whereas embedded in-game decrunch routine works.

And in that code A0=A1 (in place) all the time.

; whdload built-in RNC decrunch chokes on that one
; for some reason on a particular file (unable to decrunch data)
decrunch
; this decrunch code is copied from french version
	MOVEM.L	D1-D7/A0-A6,-(A7)	;941f6: 48e77ffe
	MOVEA.L	A0,A1			;941fa: 2248
	BSR.W	.lab_0B2C		;941fc: 6100015a
	CMP.L	#$524e4301,D0		;94200: b0bc524e4301
	BNE.S	.lab_0B14		;94206: 6654
	BSR.W	.lab_0B2C		;94208: 6100014e
	LEA	4(A0),A4		;9420c: 49e80004
	LEA	0(A4,D0.L),A2		;94210: 45f40800
	ADDA.L	#$00000100,A2		;94214: d5fc00000100
	MOVEA.L	A2,A3			;9421a: 264a
	BSR.W	.lab_0B2C		;9421c: 6100013a
	LEA	0(A4,D0.L),A6		;94220: 4df40800
	MOVE.B	-(A6),D3		;94224: 1626
.lab_0B11:
	BSR.W	.lab_0B17		;94226: 61000044
	ADDQ.W	#1,D5			;9422a: 5245
	CMPA.L	A4,A6			;9422c: bdcc
	BLE.S	.lab_0B13		;9422e: 6f22
	BSR.W	.lab_0B1F		;94230: 61000090
	BSR.W	.lab_0B25		;94234: 610000c8
	SUBQ.W	#1,D6			;94238: 5346
	LEA	0(A3,D7.W),A0		;9423a: 41f37000
	EXT.L	D6			;9423e: 48c6
	ADDA.L	D6,A0			;94240: d1c6
	TST.W	D7			;94242: 4a47
	BNE.S	.lab_0B12		;94244: 6604
	LEA	1(A3),A0		;94246: 41eb0001
.lab_0B12:
	MOVE.B	-(A0),-(A3)		;9424a: 1720
	DBF	D6,.lab_0B12		;9424c: 51cefffc
	BRA.S	.lab_0B11		;94250: 60d4
.lab_0B13:
	MOVE.L	A2,D0			;94252: 200a
	SUB.L	A3,D0			;94254: 908b
	MOVEA.L	A3,A0			;94256: 204b
	BRA.W	.lab_0B2E		;94258: 6000010a
.lab_0B14:
	MOVEQ	#0,D0			;9425c: 7000
	BRA.W	.lab_0B2E		;9425e: 60000104
.lab_0B15:
	LSL.B	#1,D3			;94262: e30b
	BNE.S	.lab_0B16		;94264: 6604
	MOVE.B	-(A6),D3		;94266: 1626
	ROXL.B	#1,D3			;94268: e313
.lab_0B16:
	RTS				;9426a: 4e75
.lab_0B17:
	MOVEQ	#-1,D5			;9426c: 7aff
	BSR.W	.lab_0B15		;9426e: 6100fff2
	BCC.S	.lab_0B1D		;94272: 6444
	MOVEQ	#0,D5			;94274: 7a00
	BSR.W	.lab_0B15		;94276: 6100ffea
	BCC.S	.lab_0B1B		;9427a: 6432
	LEA	.lab_0B1E(PC),A0		;9427c: 41fa003c
	MOVEQ	#3,D1			;94280: 7203
.lab_0B18:
	CLR.W	D5			;94282: 4245
	MOVE.B	0(A0,D1.W),D0		;94284: 10301000
	EXT.W	D0			;94288: 4880
	MOVEQ	#-1,D2			;9428a: 74ff
	LSL.W	D0,D2			;9428c: e16a
	NOT.W	D2			;9428e: 4642
	SUBQ.W	#1,D0			;94290: 5340
.lab_0B19:
	BSR.W	.lab_0B15		;94292: 6100ffce
	ROXL.W	#1,D5			;94296: e355
	DBF	D0,.lab_0B19		;94298: 51c8fff8
	TST.W	D1			;9429c: 4a41
	BEQ.S	.lab_0B1A		;9429e: 6706
	CMP.W	D5,D2			;942a0: b445
	DBNE	D1,.lab_0B18		;942a2: 56c9ffde
.lab_0B1A:
	MOVE.B	4(A0,D1.W),D0		;942a6: 10301004
	EXT.W	D0			;942aa: 4880
	ADD.W	D0,D5			;942ac: da40
.lab_0B1B:
	MOVE.W	D5,-(A7)		;942ae: 3f05
.lab_0B1C:
	MOVE.B	-(A6),-(A3)		;942b0: 1726
	DBF	D5,.lab_0B1C		;942b2: 51cdfffc
	MOVE.W	(A7)+,D5		;942b6: 3a1f
.lab_0B1D:
	RTS				;942b8: 4e75
.lab_0B1E:
	DC.W	$0a03			;942ba
	DC.W	$0202			;942bc
	DC.W	$0e07			;942be
	DC.W	$0401			;942c0
.lab_0B1F:
	LEA	.lab_0B24(PC),A0		;942c2: 41fa0030
	MOVEQ	#3,D0			;942c6: 7003
.lab_0B20:
	BSR.W	.lab_0B15		;942c8: 6100ff98
	BCC.S	.lab_0B21		;942cc: 6404
	DBF	D0,.lab_0B20		;942ce: 51c8fff8
.lab_0B21:
	CLR.W	D6			;942d2: 4246
	ADDQ.W	#1,D0			;942d4: 5240
	MOVE.B	0(A0,D0.W),D1		;942d6: 12300000
	BEQ.S	.lab_0B23		;942da: 670e
	EXT.W	D1			;942dc: 4881
	SUBQ.W	#1,D1			;942de: 5341
.lab_0B22:
	BSR.W	.lab_0B15		;942e0: 6100ff80
	ROXL.W	#1,D6			;942e4: e356
	DBF	D1,.lab_0B22		;942e6: 51c9fff8
.lab_0B23:
	MOVE.B	5(A0,D0.W),D1		;942ea: 12300005
	EXT.W	D1			;942ee: 4881
	ADD.W	D1,D6			;942f0: dc41
	RTS				;942f2: 4e75
.lab_0B24:
	DC.W	$0a02			;942f4
	dc.w $0100
	DC.W	$000a			;942f8
	DC.W	$0604			;942fa
	dc.w $0302
.lab_0B25:
	MOVEQ	#0,D7			;942fe: 7e00
	CMP.W	#$0002,D6		;94300: bc7c0002
	BEQ.S	.lab_0B29		;94304: 672a
	MOVEQ	#1,D0			;94306: 7001
.lab_0B26:
	BSR.W	.lab_0B15		;94308: 6100ff58
	BCC.S	.lab_0B27		;9430c: 6404
	DBF	D0,.lab_0B26		;9430e: 51c8fff8
.lab_0B27:
	ADDQ.W	#1,D0			;94312: 5240
	LEA	.lab_0B2B(PC),A0		;94314: 41fa0036
	MOVE.B	0(A0,D0.W),D1		;94318: 12300000
	EXT.W	D1			;9431c: 4881
.lab_0B28:
	BSR.W	.lab_0B15		;9431e: 6100ff42
	ROXL.W	#1,D7			;94322: e357
	DBF	D1,.lab_0B28		;94324: 51c9fff8
	LSL.W	#1,D0			;94328: e348
	ADD.W	4(A0,D0.W),D7		;9432a: de700004
	RTS				;9432e: 4e75
.lab_0B29:
	MOVEQ	#5,D0			;94330: 7005
	CLR.W	D1			;94332: 4241
	BSR.W	.lab_0B15		;94334: 6100ff2c
	BCC.S	.lab_0B2A		;94338: 6404
	MOVEQ	#8,D0			;9433a: 7008
	MOVEQ	#64,D1			;9433c: 7240
.lab_0B2A:
	BSR.W	.lab_0B15		;9433e: 6100ff22
	ROXL.W	#1,D7			;94342: e357
	DBF	D0,.lab_0B2A		;94344: 51c8fff8
	ADD.W	D1,D7			;94348: de41
	RTS				;9434a: 4e75
.lab_0B2B:
	dc.w $0b04
	dc.w $0700
	dc.w $0120
	dc.l $00000020
	DC.W	$0000			;94356
.lab_0B2C:
	MOVEQ	#3,D1			;94358: 7203
.lab_0B2D:
	LSL.L	#8,D0			;9435a: e188
	MOVE.B	(A0)+,D0		;9435c: 1018
	DBF	D1,.lab_0B2D		;9435e: 51c9fffa
	RTS				;94362: 4e75
.lab_0B2E:
	MOVE.L	A0,D2			;94364: 2408
	SUB.L	A1,D2			;94366: 9489
	MOVE.L	D0,D1			;94368: 2200
	BEQ.S	.lab_0B31		;9436a: 670c
.lab_0B2F:
	MOVE.B	(A0)+,(A1)+		;9436c: 12d8
	SUBQ.L	#1,D1			;9436e: 5381
	BNE.S	.lab_0B2F		;94370: 66fa
.lab_0B30:
	CLR.B	(A1)+			;94372: 4219
	SUBQ.L	#1,D2			;94374: 5382
	BNE.S	.lab_0B30		;94376: 66fa
.lab_0B31:
	MOVEM.L	(A7)+,D1-D7/A0-A6	;94378: 4cdf7ffe
	RTS				;9437c: 4e75

After discussion with Wepl, it appears that this is the old RNC1 format. Quoting Wepl:

This is the old RNC1 format. Only used with a few games. The file format
does not contain an info how much distance is needed for inplace
decrunching. WHDLoad has used distance=0 which was enough for the Moonstone
files. The file you sent needs a distance=1. I have now set a distance=16.
I don't like to make this distance larger then necessary because it may
create problems with other games.

So in-place decompression can fail on some old RNC1 files. Just repack the file with propack (which creates "new" RNC1 files) and problems may be fixed.

[girv]

Where can you get hold of the ARJ + mode 7 packer and 68k depacker source?

[Giants]

Originally Posted by roondar

A small update: I've fully moved one of my long-term projects over to NRV2R now and I'm impressed!

Loading+decrunching times have decreased by about 25% and the compression rate seems to be around 3-5% better than RNC manages. Great stuff

Hi, so you have the NRV2R compressor binary ?
I would be very interested to have a version that works on Amiga 500.
Source and binary if it possible

[girv]

Originally Posted by Giants

Hi, so you have the NRV2R compressor binary ?
I would be very interested to have a version that works on Amiga 500.
Source and binary if it possible

I hate to be the +1 guy ... but +1

[Giants]

I really need an alternative of 'RNC propacker' which really works 'in place'.
I try Crunchmania, it's work good but don't give me a rate equivalent of RNC.
But, for me, RNC it's not my solution because it don't really work 'in place'.
and I don't have enough space for 'buffer'.
So maybe NRV2R it's a solution for me.
IF, i can compress files of course, LOL.
no matter if the 'packer' work on Intel or 68000.
I prefer of course a 68000 solution.

Well, I am stuck waiting for your answers about NRV2R.
Maybe, one day.

[mcgeezer]

Originally Posted by Giants

I really need an alternative of 'RNC propacker' which really works 'in place'.
I try Crunchmania, it's work good but don't give me a rate equivalent of RNC.
But, for me, RNC it's not my solution because it don't really work 'in place'.
and I don't have enough space for 'buffer'.
So maybe NRV2R it's a solution for me.
IF, i can compress files of course, LOL.
no matter if the 'packer' work on Intel or 68000.
I prefer of course a 68000 solution.

Well, I am stuck waiting for your answers about NRV2R.
Maybe, one day.

There are two versions of Pro Pack. One supports in place the other doesn't (if I recall).

[roondar]

Originally Posted by Giants

Hi, so you have the NRV2R compressor binary ?
I would be very interested to have a version that works on Amiga 500.
Source and binary if it possible

Originally Posted by girv

I hate to be the +1 guy ... but +1

Originally Posted by Giants

I really need an alternative of 'RNC propacker' which really works 'in place'.
...
Well, I am stuck waiting for your answers about NRV2R.
Maybe, one day.

The links to the NRV2R packer and unpacker are in this very thread on page 1
They're not mine to distribute, so all I can offer is those same links

See here for the unpacker: http://eab.abime.net/showpost.php?p=...2&postcount=14
And see here for the packer: https://github.com/jonathanbennett73...ster/ToolChain

All courtesy of ross

[Giants]

mcgeezer : It's not a 'real' in place.
Sometime it's work, sometime not.

roondar : Exact, I see now.
Arf, it's a intel version... -_-'
better than nothing, I'm going to test this.

Thks

Last Time : I just tested the compression...
Ouaaaa...Better than RNC Propack for my first file
and identical for my second file.
impressive.

Going to test the decrunch in WinUae now

[Giants]

roondar : Do you use the engine decompression in a amiga ?
I just triedthe 'second' code available in you link (http://eab.abime.net/showpost.php?p=...2&postcount=14)
and for me, it's don't work.

[mcgeezer]

Originally Posted by roondar

I've been using RNC Propack using it's built in support for "in place" decompression for a while now and never had any issues. But yesterday I came across a file that would not unpack properly if the source and destination were the same address. It would work properly if I used a different destination address. As I expected, merely changing one or two bytes in the source file also made it work again. I also tried increasing BUFSIZE in the source, but that didn't change anything.

For the past 8 weeks I've been dealing with an issue with one of Super Sprint files that would intermittently not unpack using the RNC in place unpacker.

Tonight I decided to delve deeper into it and have found that reading the packed file into the end of the allocated buffer and then uncompressing has resolved the issue. I have ran 40 tests and every one worked, previously it would fail in at least 10 attempts.

Thanks for this thread Roondar, it has helped solve a bug for me.

[roondar]

Originally Posted by Giants

roondar : Do you use the engine decompression in a amiga ?
I just triedthe 'second' code available in you link (http://eab.abime.net/showpost.php?p=...2&postcount=14)
and for me, it's don't work.

IIRC it just worked without issues.

If you have problems, I'd suggest asking ross for help - he wrote the code, he will have a better chance of helping you figure out the problem than I do as I merely used the code as provided

Originally Posted by mcgeezer

Thanks for this thread Roondar, it has helped solve a bug for me.

Happy to help!

This in-place decompression stuff can cause some very odd bugs and it certainly was a bummer when it happened to me.

[Giants]

Humm.
If it's work for you, it should work for me.
I must have been wrong somewhere, I'm watching this tomorrow.

[Giants]

Ok,it's working
I have some audio bug 'it's a animation I crunch'.

I don't exactly understand the D0 value
;d0 offset inside buffer for packed data

I'v juste put 0 for D0, maybe it's not a good idea

[roondar]

Originally Posted by Giants

I don't exactly understand the D0 value
;d0 offset inside buffer for packed data

I'v juste put 0 for D0, maybe it's not a good idea

Ok, you point A0 to the buffer that contains the packed data.
D0 then is the value that is added to A0 to get to the start of the packed data.

This is there so you can do stuff like read like a file with a header attached, or a file with several different packed things in it or have a file read so the end of the file is the at end of the buffer, but the buffer is the size of the unpacked file rather than the packed one (many coders like to do this one).

An example: if you have a file that has a ten byte header you've put in there for whatever reason and then the packed data starts, you'd put a 10 in D0.

If you don't do stuff like the above and just read a packed file to a certain address, the offset can be 0.

[ross]

The use of d0 also depends on the type of decoder used.

They are both in-place, but differ in the memory layout:
- forward in memory (nrv2s, standard) : slightly faster, but requires data to be loaded at the end of the buffer
- backward in memory (nrv2r, reverse), slightly slower, usually compresses a few bytes better and allows data to be loaded at the beginning of the buffer (therefore d0 can be set = 0).

Call the command line help for the packer, it indicates the parameters to be used.

You will see that if you use the forward mode it give to you the value to be entered in d0 to load the file in the final part of the buffer.

You can also use d0 as roondar suggested.
I have often managed the data in separate block to be unpacked when and where I needed it, with custom headers.

Warning: it can happen that some files can be compressed in only one of the two modes. The generic decompressor supports both, but you need to adapt your loader.
Another warning: this is not a general purpose packer, you need to know what you do and how you do it

[alpine9000]

I’ve used RNC in place for thousands of different files over the past couple of years and have yet to have one that doesn’t work. Are you guys sure you have enough free stack space?

[Giants]

Ross : Roondar :
ok, Big Thks, now I understand better the role of D0.
I've juste check again (for my audio probleme after decrunch), it's NOT a decrunch problem.
I checked, decrunched data is identical to my origina file (so don't crunched)
Maybe in my code, it don't like we use some registry (like D0).
I'll take a closer look and adapt the decrunch if necessary and/or my code !

In any case, it works super well. great rate.
Very good job 👍

Big Thks