Bad Apple Amiga Version - Page 3

chiark · 14 July 2020, 10:08

True. But doing a lot of pre-processing will be the answer

@robinsonb5, exactly my thought: using the blitter to fill part of a rasterised image as that'd get us down to low memory usage at the expense of a lot of filling. Picking a complex frame and timing it would be a simple check - will have a look...
@stevelord, I think the tile approach was used by quite a few implementations and if I recall correctly it was the BBC Model B implementation that started me looking at that path... That's on github I think?

ross · 14 July 2020, 11:43

Quote:

Originally Posted by Yannick

I’m storing all points per frame, inter frame compression is very poor as only two consecutive identical frames are detected and optimized.
Detecting changes in between frames could bring improvements, but the algorithm to detect and encode point movements between frames is not trivial at all.

Do you have any statistics available for the vector matix?
Just to understand how many maximum lines per frame we are talking about.
I don't think blitter filling for the frame is a stumbling block, but the polydraw.

Furthermore, scaling (and ratio tranform) must be avoided.
You have to decide in advance the resolution (I would opt for one at 60Hz in order to render the frames from the originals at 30FPS, to avoid temporal artifacts).

By trying exclusively on AGA (I have some ideas about some overlay and dithering effects) you could also explore medres-interlace (well, maybe I'm aiming too high

)
However, the format must be increased from 8x8 bit to 9x8 or 10x9 (depends on the choice of resolution).
Of course there will be an increase from 850Kb to >900Kb in the first case or >1000Kb in the second case but it is to be verified, given the different positioning of the bits the same compression ratio is not guaranteed, especially for the huffman part.

robinsonb5 · 14 July 2020, 12:37

I've been tinkering with this a little this morning - I've picked a couple of busy-looking frames and done a horizontal edge-detect to find the boundaries for blitter fill.

I'm assuming here that we'd want to use a separate buffer for even and odd frames, and would blitter-fill into separate destination buffers, leaving the source unchanged. We can avoid clearing buffers by just overwriting what's changed since two frames previously.

So taking this random pair of frames, two frames apart in the animation, and finding the differences between them, the second one can be encoded as a series of 1637 word writes over the top of the first one. In a horribly inefficient text-based format, in which the first number is the number of words to skip and the second is the literal word to be written, the second frame looks like this:

Code:

Once encoded more efficiently, and converted to binary, that strikes me as something that will compress pretty well.

sovenyimre · 14 July 2020, 14:08

I had the same 16x16 block (and a faster 32x32) idea and already did it in amos. It stores and refresh only the frame movements.
File size 4mb stored as tome map+12mb as icon (fixed for black or white background) tiles....+4mb audio.
Slow at present state but it is a work in progress.
I will also do a color mode and try to detect movement in pixel quantity to reduce size.

https://electricblacksheep.itch.io/bad-apple

chiark · 14 July 2020, 14:21

Fantastic both @robinsonB5 and sovenyimre!

To fit on 6,600 frames on 2 floppies the target is around 256 bytes a frame... Loving the innovation that is going on here, and on other platform approaches too.

2020 is already the year Amiga got a bad apple implementation thanks to SteveLord, it looks like it'll get a few more!

pandy71 · 14 July 2020, 14:53

Years ago i've experimented a bit with ffmpeg and potrace trying to create silhouettes video and later vectorize it - script below:

Code:

@setlocal
@if not exist %cd%\pot (mkdir %cd%\pot)
@rem for %%1 in (*.avi *.mp4 *.webm *.mkv *.mpg *.m2v) do (ffmpeg.exe -y -i "%%1" -an -vf hqdn3d=luma_spatial=16:chroma_spatial=16:luma_tmp=8:chroma_tmp=8,decimate=cycle=2,decimate=cycle=2,colorkey=color=0x00D800:similarity=0.5:blend=0.1,alphaextract,format=pix_fmts=monow,removegrain=m0=4 -vsync 0 "%cd%\bmp\%%~n1_%%06d.bmp")
@rem ffmpeg -i %1 -an -vf hqdn3d=luma_spatial=16:chroma_spatial=16:luma_tmp=8:chroma_tmp=8,decimate=cycle=2,decimate=cycle=2,format=pix_fmts=rgb24,colorkey=color=0x00D800:similarity=0.5:blend=0.1,alphaextract,format=pix_fmts=monow,removegrain=m0=4 -vsync 0 "%cd%\bmp\%~n1_%%06d.bmp"
@rem pause


@rem http://potrace.sourceforge.net/#downloading
@rem for %%1 in (*.bmp) do ("potrace.exe" -b pdf -P A4 -k 0.5 -o "%cd%\pot\%%~n1.pdf" "%%1")
@rem for %%1 in (*.bmp) do ("potrace.exe" -s --flat -k 0.5 --tight --longcurve --opttolerance 0.2 --alphamax 1 --turdsize 2 -o "%cd%\pot\%%~n1.svg" -- "%%1")
@for %%1 in (*.bmp) do ("potrace.exe" -s --flat -k 0.5 --tight --longcurve --opttolerance 5 --alphamax 5 --turdsize 5 -o "%cd%\pot\%%~n1.svg" -- "%%1")
@pause

To extract silhouettes some additional video processing may be required.

Above script should work with videos like: [ Show youtube player ] without any modification.

ross · 14 July 2020, 14:54

Quote:

Originally Posted by robinsonb5

So taking this random pair of frames, two frames apart in the animation, and finding the differences between them, the second one can be encoded as a series of 1637 word writes over the top of the first one. In a horribly inefficient text-based format, in which the first number is the number of words to skip and the second is the literal word to be written, the second frame looks like this:

Probably this is really an intense frame but 1600 words are too much.
Even if the average is 800 and pack it at 1/3 you have a requirement of 800*6000 = 4.8Mb.

You probably need to decrease the resolution and details to make the tiles change much less.
Well, many tests will be needed to find a compromise, however it is a road that it's worth being explored

Yannick · 14 July 2020, 15:34

Quote:

Originally Posted by ross

Do you have any statistics available for the vector matix?
Just to understand how many maximum lines per frame we are talking about.
I don't think blitter filling for the frame is a stumbling block, but the polydraw.

Furthermore, scaling (and ratio tranform) must be avoided.
You have to decide in advance the resolution (I would opt for one at 60Hz in order to render the frames from the originals at 30FPS, to avoid temporal artifacts).

By trying exclusively on AGA (I have some ideas about some overlay and dithering effects) you could also explore medres-interlace (well, maybe I'm aiming too high

)
However, the format must be increased from 8x8 bit to 9x8 or 10x9 (depends on the choice of resolution).
Of course there will be an increase from 850Kb to >900Kb in the first case or >1000Kb in the second case but it is to be verified, given the different positioning of the bits the same compression ratio is not guaranteed, especially for the huffman part.

Some statistics from the shown video:
- All frames are rendered (6535 frames)
- Total of 570401 polygons
- Average points / polygon = 10.5
- Average points / frame = 13
- Max points / frame = 653
- Max points / polygon = 253
- Max polygon / frame = 93

I did some expiriments by reducing maximum number of points / polygon to 126 max, i doesn't reduce size nor quality a lot.

The maximum number of polygons / frame might be a good way to reduce complexity without too much impact on visible quality. Only 150 frames have more the 20 polygons.

Yannick

robinsonb5 · 14 July 2020, 16:06

Quote:

Originally Posted by ross

Probably this is really an intense frame but 1600 words are too much.
Even if the average is 800 and pack it at 1/3 you have a requirement of 800*6000 = 4.8Mb.

Agreed - however, if you look at the data, for very many of those words only one bit is set, so I'd expect a much more efficient encoding to be possible. Actually, I should see how many unique words are within that dataset.

ross · 14 July 2020, 16:23

Quote:

Originally Posted by Yannick

- Max points / frame = 653
- Max points / polygon = 253
- Max polygon / frame = 93

Interesting

If I find the time I do some tests with the blitter.

Yannick · 14 July 2020, 17:16

Quote:

Originally Posted by ross

Interesting

If I find the time I do some tests with the blitter.

Tell me what you can reach at 30fps, we can then optimize

robinsonb5 · 15 July 2020, 00:37

A little bit more data:
If I output the whole animation in a simplistic binary format, 1 byte with the number of words to skip, 2 bytes with a literal word to write to memory, I end up with 12,626,385 bytes of data, which gzips to 3,634,175 bytes.

Here's where it gets interesting:
I only see 5878 unique values within that data, so it's using less than 10% of its encoding space.
If I analyze how many bits are set within each unique word, I get this:

0 bits set (i.e. a zero is written): 1158688 in total, 1 unique values
1 bits set: 2619397 in total, 16 unique values

Which means of that 12-meg dataset, nearly 11 meg represents writing one of just 17 values.

2 bits set: 337667 in total, 120 unique values
3 bits set: 72624 in total, 560 unique values
4 bits set: 14866 in total, 1797 unique values
5 bits set: 4200 in total, 2301 unique values
6 bits set: 1012 in total, 763 unique values
7 bits set: 280 in total, 259 unique values
8 bits set: 45 in total, 45 unique values
9 bits set: 11 in total, 11 unique values
10 bits set: 5 in total, 5 unique values

So... clearly scope for a much more efficient encoding.

Also, the source animation is 320x240@30fps. Presumably because of the frame rate we'd be targetting an NTSC screen resolution, so it would make sense to downsample to 320x200 - so that should reduce the size by about 16%, too.

Yannick · 15 July 2020, 08:58

After some optimization I came to this without compromising quality too much:
- Max points / frame = 535
- Max points / polygon = 126
- Max polygon / frame = 93

Only 151 frames (2.3%) have more than 200 points.
73.3 % have less than 100 points.

Data is now :
- 1084272 bytes uncompressed
- 808378 bytes compressed

ross · 15 July 2020, 09:04

Quote:

Originally Posted by Yannick

After some optimization I came to this without compromising quality too much:
- Max points / frame = 535
- Max points / polygon = 126
- Max polygon / frame = 93

Only 151 frames (2.3%) have more than 200 points.
73.3 % have less than 100 points.

Good, this allows you to remove the check for% x1111111 and the sum.
How much does the compression degrade with the coordinates in xyxyxy .. instead of xxx..yyy ..?
The index applied to all y extraction slow it.. (I'm short of Ax registers

)

Yannick · 15 July 2020, 09:08

Quote:

Originally Posted by ross

Good, this allows you to remove the check for% x1111111 and the sum.
How much does the compression degrade with the coordinates in xyxyxy .. instead of xxx..yyy ..?
The index applied to all y extraction slow it.. (I'm short of Ax registers

)

Yes the goal of limiting paths to 126 points is to allow removing the test and the sum.

The test shows that the gain of having X and Y data stored independently is now marginal:
- 1084272 bytes uncompressed
- 808378 bytes XXX, YYY data compressed
- 813405 bytes XYXYXY data compressed

You can go this way for decoding, I'll modify the data generation

ross · 15 July 2020, 09:44

Quote:

Originally Posted by Yannick

Yes the goal of limiting paths to 126 points is to allow removing the test and the sum.

The test shows that the gain of having X and Y data stored independently is now marginal:
- 1084272 bytes uncompressed
- 808378 bytes XXX, YYY data compressed
- 813405 bytes XYXYXY data compressed

You can go this way for decoding, I'll modify the data generation

Yes, acceptable.

But the main concern is to avoid the scaling.
x need to be absolutely 9 bit.

x,y in 16 bit, compressed, what file size do they give you?

Yannick · 15 July 2020, 09:58

What resolution do you want to come up with.
Original video is 512*384, scaling it to 320 wide, gives 240 in height
Should we go for this 320*240 or decrease is further to 320*200?
That would fit in 9bits for x and 8bits for y, but this is not easy to handle at all.
I will try something with 9 bits for X and 7 bits for Y.
Y will need to be shifted by one bit to scale.
X will be used as is.

Let me check what it looks like.

/Yannick

ross · 15 July 2020, 10:17

Quote:

Originally Posted by Yannick

What resolution do you want to come up with.
Original video is 512*384, scaling it to 320 wide, gives 240 in height
Should we go for this 320*240 or decrease is further to 320*200?
That would fit in 9bits for x and 8bits for y, but this is not easy to handle at all.
I will try something with 9 bits for X and 7 bits for Y.
Y will need to be shifted by one bit to scale.
X will be used as is.

Let me check what it looks like.

/Yannick

Well, final resolution is secondary.
Could also be some overscan resolution and not a standard one.
Main idea is to use 9x8 bit or 10x9 (yes not easy..)

If we go for a 60Hz resolution then the most suitable is lores NTSC (or for us PAL60

).
A 10% overscan give 352x220 (9x8), that are good values.
So a 0.6875 scale for x and 0.5729 for y.

In any case first I have to see something moving on the screen, later in the day I will try.

Yannick · 15 July 2020, 11:31

I've made it with 9x7bit on a single 16bit word per point.
The result is good.
X is the high 9 bits, and it scaled from 0 to 319.
Y is the lowest 7 bits, and is scaled from 0 to 127, that shall be shifted by 2 for a scale of 0 to 254.

End resolution can be modified with this concept to go up to 512 in X and 254 in Y.

chip · 15 July 2020, 11:33

We are looking at the born of Bad Apple for Amiga ..... that's fantastic !

Thanks to all involved

15 July 2020, 08:58	#53
Yannick Registered User Join Date: Jul 2020 Location: France Posts: 25	After some optimization I came to this without compromising quality too much: - Max points / frame = 535 - Max points / polygon = 126 - Max polygon / frame = 93 Only 151 frames (2.3%) have more than 200 points. 73.3 % have less than 100 points. Data is now : - 1084272 bytes uncompressed - 808378 bytes compressed Last edited by Yannick; 15 July 2020 at 09:04.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Bad Apple - Amiga shadow animation	voyager	Nostalgia & memories	37	10 November 2020 11:31
Is Amiga the only system without a Bad Apple demo?	Glen M	support.Demos	11	17 July 2020 18:07
Bad Apple mod	Zarchos	request.Modules	5	07 May 2017 04:35
SWIV unsupported version or bad dump?	BarryB	project.WHDLoad	13	08 January 2017 12:04
Last Ninja Apple 2GS version longplay	laffer	Retrogaming General Discussion	4	04 June 2008 17:50

14 July 2020, 10:08	#41
chiark Needs a life Join Date: Jan 2008 Location: England Posts: 1,707	True. But doing a lot of pre-processing will be the answer @robinsonb5, exactly my thought: using the blitter to fill part of a rasterised image as that'd get us down to low memory usage at the expense of a lot of filling. Picking a complex frame and timing it would be a simple check - will have a look... @stevelord, I think the tile approach was used by quite a few implementations and if I recall correctly it was the BBC Model B implementation that started me looking at that path... That's on github I think?

14 July 2020, 14:08	#44
sovenyimre Registered User Join Date: Jul 2020 Location: Hungary Posts: 94	I had the same 16x16 block (and a faster 32x32) idea and already did it in amos. It stores and refresh only the frame movements. File size 4mb stored as tome map+12mb as icon (fixed for black or white background) tiles....+4mb audio. Slow at present state but it is a work in progress. I will also do a color mode and try to detect movement in pixel quantity to reduce size. https://electricblacksheep.itch.io/bad-apple

14 July 2020, 14:21	#45
chiark Needs a life Join Date: Jan 2008 Location: England Posts: 1,707	Fantastic both @robinsonB5 and sovenyimre! To fit on 6,600 frames on 2 floppies the target is around 256 bytes a frame... Loving the innovation that is going on here, and on other platform approaches too. 2020 is already the year Amiga got a bad apple implementation thanks to SteveLord, it looks like it'll get a few more!

15 July 2020, 00:37	#52
robinsonb5 Registered User Join Date: Mar 2012 Location: Norfolk, UK Posts: 1,153	A little bit more data: If I output the whole animation in a simplistic binary format, 1 byte with the number of words to skip, 2 bytes with a literal word to write to memory, I end up with 12,626,385 bytes of data, which gzips to 3,634,175 bytes. Here's where it gets interesting: I only see 5878 unique values within that data, so it's using less than 10% of its encoding space. If I analyze how many bits are set within each unique word, I get this: 0 bits set (i.e. a zero is written): 1158688 in total, 1 unique values 1 bits set: 2619397 in total, 16 unique values Which means of that 12-meg dataset, nearly 11 meg represents writing one of just 17 values. 2 bits set: 337667 in total, 120 unique values 3 bits set: 72624 in total, 560 unique values 4 bits set: 14866 in total, 1797 unique values 5 bits set: 4200 in total, 2301 unique values 6 bits set: 1012 in total, 763 unique values 7 bits set: 280 in total, 259 unique values 8 bits set: 45 in total, 45 unique values 9 bits set: 11 in total, 11 unique values 10 bits set: 5 in total, 5 unique values So... clearly scope for a much more efficient encoding. Also, the source animation is 320x240@30fps. Presumably because of the frame rate we'd be targetting an NTSC screen resolution, so it would make sense to downsample to 320x200 - so that should reduce the size by about 16%, too.

15 July 2020, 09:58	#57
Yannick Registered User Join Date: Jul 2020 Location: France Posts: 25	What resolution do you want to come up with. Original video is 512384, scaling it to 320 wide, gives 240 in height Should we go for this 320240 or decrease is further to 320*200? That would fit in 9bits for x and 8bits for y, but this is not easy to handle at all. I will try something with 9 bits for X and 7 bits for Y. Y will need to be shifted by one bit to scale. X will be used as is. Let me check what it looks like. /Yannick

15 July 2020, 11:31	#59
Yannick Registered User Join Date: Jul 2020 Location: France Posts: 25	I've made it with 9x7bit on a single 16bit word per point. The result is good. X is the high 9 bits, and it scaled from 0 to 319. Y is the lowest 7 bits, and is scaled from 0 to 127, that shall be shifted by 2 for a scale of 0 to 254. End resolution can be modified with this concept to go up to 512 in X and 254 in Y.

15 July 2020, 11:33	#60
chip Registered User Join Date: Oct 2012 Location: Italy Age: 49 Posts: 2,942	We are looking at the born of Bad Apple for Amiga ..... that's fantastic ! Thanks to all involved

Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)