Imploder compression algo

[Exl]

I've been working on a level\image editor for Superfrog for a while now. The only thing standing in the way of making it complete is a way to compress levels using the File Imploder algorithm, with which all the Superfrog game data is compressed (both floppy and CD32 versions). It can already decompress such files, but aside from a dissasembly of the original FImp tool, I have not been able to find any code that can compress. I don't have the skills to write such a thing myself, even going from the decompression code... Has somebody come across it, can somebody do this or does anyone know of any other solution?

As a proof of concept so to speak, here is the current state of the editor;
http://members.home.nl/meuwissenth/froggy_05a.zip
It works, you can open levels from the CD32 version (L*M* files), select a tileset (L*BM files) and sprite graphics (L*BO) files. I've only added specials for the first three worlds so far, but other worlds and even Project-F will load with other configs and are fully editable. If you save the level, compress it with FImp and then change the first 4 bytes to "ATN!" Superfrog will be able to load the level and play it. You can move around by holding the right mouse button and moving about and select tiles from the map by holding down shift while dragging over them.

Again, I hope somebody might have some leads or something into finding compression code or something, it would make the editor almost complete. Thanks.

[WayneK]

The compression code is included in the "Beermon" monitor/debugger program, which was released with full source. Search the web, or pm me with an email addy and I'll send you a copy.

[Exl]

I found it, but unfortunately the compression code is written in 68k assembler. Thanks though.

[demoniac]

Can't you just use Turbo Imploder and select the library option to compress data files?

[WayneK]

You could do what some other coders have done... take the really cheap way to port a packer from another architecture - emulation
I've seen some ZX Spectrum packers made into Windows cmdline versions, by simply running a z80 core + writing a tiny shell around it to start up the emulation in the correct way - surely you could do the same with starscream or some 68k core... probably no more than 20-30 lines of code.

[jotd]

Originally Posted by WayneK

You could do what some other coders have done... take the really cheap way to port a packer from another architecture - emulation
I've seen some ZX Spectrum packers made into Windows cmdline versions, by simply running a z80 core + writing a tiny shell around it to start up the emulation in the correct way - surely you could do the same with starscream or some 68k core... probably no more than 20-30 lines of code.

Using awk scripts & macros, you could even create C code from asm source. I've done so on 6502 oric games. it works.
I am interested in your project (since I'm a game remaker/extender myself).
Post the 68k source code of the imploder packing routine and I'll try to convert it to C for you.
And it would nicely fit in your editor this way

[Exl]

Originally Posted by demoniac

Can't you just use Turbo Imploder and select the library option to compress data files?

The editor is a Windows app (VB6 for the UI and C for some low-level rendering stuff), so no. Of course right now what you describe is the only way to compress such files that I know.

Originally Posted by WayneK

You could do what some other coders have done... take the really cheap way to port a packer from another architecture - emulation
I've seen some ZX Spectrum packers made into Windows cmdline versions, by simply running a z80 core + writing a tiny shell around it to start up the emulation in the correct way - surely you could do the same with starscream or some 68k core... probably no more than 20-30 lines of code.

I looked into Starscream, and it's very interesting indeed. I can get as far as emulating a CPU in my helper DLL, beyond that I don't know how to continue. It's certainly an option to look into further though.

Originally Posted by jotd

Using awk scripts & macros, you could even create C code from asm source. I've done so on 6502 oric games. it works.
I am interested in your project (since I'm a game remaker/extender myself).
Post the 68k source code of the imploder packing routine and I'll try to convert it to C for you.
And it would nicely fit in your editor this way

If you could at least try, I'd be very grateful! The source code is in the archive I linked to, under renderlib/explode.c. Some auxillary functions of it are in deimp.c. See http://members.home.nl/meuwissenth/explode.c and http://members.home.nl/meuwissenth/deimp.c

[jotd]

Originally Posted by Exl

If you could at least try, I'd be very grateful! The source code is in the archive I linked to, under renderlib/explode.c. Some auxillary functions of it are in deimp.c. See http://members.home.nl/meuwissenth/explode.c and http://members.home.nl/meuwissenth/deimp.c

Those are C functions for unpacking. I'd need the asm function that packs data, so I can try converting it.

[Exl]

Sorry, it's getting late here See imploder.s in this archive, ftp://ftp.cs.tu-berlin.de/pub/atari/...r/imploder.lzh It's a "port" of Imploder for the Atari, but 68k code nonetheless.

[jotd]

Originally Posted by Exl

Sorry, it's getting late here See imploder.s in this archive, ftp://ftp.cs.tu-berlin.de/pub/atari/...r/imploder.lzh It's a "port" of Imploder for the Atari, but 68k code nonetheless.

After a short analysis, rewriting it from scratch is not really a piece of cake !

I guess I should write a pseudo emulation core like I did for 6502 instructions.
That would allow to port other 68k only code (not games!)

[Toni Wilen]

Originally Posted by jotd

I guess I should write a pseudo emulation core like I did for 6502 instructions.
That would allow to port other 68k only code (not games!)

I have been trying to do transparent XFD slave decompression library support to winuae, it is already included in source archive but inactive, only missing part is very basic 68000 emulator

[Exl]

Originally Posted by jotd

After a short analysis, rewriting it from scratch is not really a piece of cake !

I guess I should write a pseudo emulation core like I did for 6502 instructions.
That would allow to port other 68k only code (not games!)

That's a shame that it's too difficult. I didn't have much luck tinkering with the Starscream CPU core either, it is beyond me how I should get a particular piece of code to run on it. Are you planning on really making such a pseudo-emulation core?

[jotd]

I have started my 68k -> C translater. I think it will work on small pieces of code, with some manual rework.
Hopefully we can make a portable ATN cruncher in C without using emulation.

Example of code:

L000 MOVE.L D0,6(A0)
LEA L245(PC),A0
MOVEQ #0,D0
LEA 10(A1),A0
LEA $10(A1,D0.W),A2

becomes in C:

L000:
MOVE_D_IND(D[0],6 + A[0],4);
LEA(L245,A[0]);
MOVEQ_IMM_D(0,D[0]);
LEA(10 + A[1],A[0]);
LEA(0x10 + A[1] + D[0] & 0xFFFF,A[2]);

run-time symbolic 68k emulation will do the rest (hopefully)

[Exl]

That looks good! I found an attempt someone else was doing; http://www.devmaster.net/forums/showthread.php?t=5276
Sadly the link to his source doesn't work anymore, but it might be of interest.

And maybe the assembler code used in Beermon to Implode files is more useful, I've uploaded it here. Imploder routine starts at line 9967.

Don't know how else I can help out. Thanks though for the effort you're putting into it.

[jotd]

Originally Posted by Exl

That looks good! I found an attempt someone else was doing; http://www.devmaster.net/forums/showthread.php?t=5276
Sadly the link to his source doesn't work anymore, but it might be of interest.

And maybe the assembler code used in Beermon to Implode files is more useful, I've uploaded it here. Imploder routine starts at line 9967.

Don't know how else I can help out. Thanks though for the effort you're putting into it.

Yes, Beermon source is cleaner (no memory allocation, compress in place, etc..) that's a better place to start. I'll start this week-end. Hopefully I can produce something within a few weeks and give you a blackbox C library.

I have already made a simple bubble sort 68k code work with my technique. I've got to test my code on simple sources first, or bugs will be hard to find from imploder source simulation!

[Exl]

Sounds good! Let me know if there is anything I can do to help.

[jotd]

I've got the "emulated" code compiling and running, but something is not right: nothing gets compressed in the end. I'm going to compare with an execution on WinUAE + AsmPro to try to find which instructions are wrongly emulated.
Sooooo close...

[crabfists]

Sounds very interesting what you guys are up to. I am reverse engineering Ruff'n'Tumble with plans to hopefully write a cross platform engine and I have thought about going down the 68000->C converter route. (Saying that, I'm getting a sick enjoyment from disassembling the game which in theory wouldnt be entirely necessary if I went down the 68000->C route) In my travels around the net looking into it I found this guy who wrote a 68000->C converter for running Frontier.

http://www.tomatarium.pwp.blueyonder...lfrontier.html

The source code for the converter is on the site. Might be useful?

[jotd]

Originally Posted by crabfists

Sounds very interesting what you guys are up to. I am reverse engineering Ruff'n'Tumble with plans to hopefully write a cross platform engine and I have thought about going down the 68000->C converter route. (Saying that, I'm getting a sick enjoyment from disassembling the game which in theory wouldnt be entirely necessary if I went down the 68000->C route) In my travels around the net looking into it I found this guy who wrote a 68000->C converter for running Frontier.

http://www.tomatarium.pwp.blueyonder...lfrontier.html

The source code for the converter is on the site. Might be useful?

Very interesting indeed. But I have compiled the assembler and I cannot get past line 2 of my assembler code! I changed stuff, but it cannot even convert a simple "move". Maybe I should have a look at the source code a little closer, seems very complete.

On the other hand I have made some progress on my converter. It now creates ATN files, but they are corrupt. I suspect that some LSR or ROR operation is buggy. Will debug this week-end. I hope that it will work soon.

[hit]

hi,

hxxp://lclevy.free.fr/packers/
hxxp://lclevy.free.fr/packers/ulib4271_src.zip
source code of unpacker.library which includes Imploder.S. maybe somewhat helpfull.

regards,
hit