09 July 2018, 09:15 | #21 | |||
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,323
|
Quote:
Quote:
Quote:
|
|||
09 July 2018, 13:26 | #22 |
Natteravn
Join Date: Nov 2009
Location: Herford / Germany
Posts: 2,496
|
Coming back to Mr.Huffman on Deltas, followed by an LZ-compression. That seems to be the best approach.
The Huffman algorithm is straight-forward, but as I understand you have to save a table of bit-patterns in the output file, which were used to encode every byte. This requires some additional space, so the gain on small samples will be limited, right? I'm currently thinking about the optimal format for such a Huffman encoded file... |
09 July 2018, 13:57 | #23 | ||
Defendit numerus
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 53
Posts: 4,468
|
Quote:
Much better what usually do LZH's family compressors: in addition to the 256 symbols, add LZ tokens reference in the huffman tree. If you want you could chop into blocks and regenerate (all or in part) the htree to optimize output code but this 'split point' is compression art per se. - EDIT, to try to give a quick explanation: compressing a file is a form of redundancy reduction. Two possible methods are: use statistic to better encode the symbols (huffman) or use a better encode for the dictionary (pardon the over-semplified lz specification..). Obviusly if used together they influence each other retroactively. But what has more impact, since it transforms symbols into a variable number of 2^n bits, is huffman and as LZ requires groups of repeated bytes to work, if applied later, it finds a destroyed and completely unusable context. Therefore all the compressors that also uses an entropic pass (huffman, arithmetric/range, ANS) should use it last, perhaps inserting among statistical data also those obtained from LZ tokens, giving them significant weight compared to normal symbols. - But instead of complicating your life much better use an already available algorithm, also because subject would become very complex.. LHA, ZIP, ARj, LZX.. all are of the LZH family. Quote:
And yes, you need extra space for the tree in output file (search Canonical Huffman for optimal code). Last edited by ross; 09 July 2018 at 18:34. |
||
09 July 2018, 14:00 | #24 | ||
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,323
|
Quote:
Quote:
That's a tricky task. LZX seems to be state of the art in regard to this, but probably too complex. |
||
09 July 2018, 15:03 | #25 | |
Defendit numerus
Join Date: Mar 2017
Location: Crossing the Rubicon
Age: 53
Posts: 4,468
|
Quote:
A great enhancemed was the previous match concept (a fast selection for a queue of 3 matches already encountered). This is extended in LZMA, with other great improvements (moreover entropy encoding with arithmetic/range coding, not huffman). This concept is also used in aplib and many others (ex. our Blueberry's Shrinkler , but this is a CM compressor so another planet..). Unfortunately here we enter the world of NP-hardness problems, so a perfect solution is not computable in usable times. But some optimal solutions exist . |
|
17 July 2018, 08:20 | #26 |
Registered User
Join Date: Jan 2014
Location: Poland
Posts: 168
|
We do not know how works TTA 2.0 at 68000, because there is no port.
|
17 July 2018, 08:51 | #27 | |
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,323
|
Quote:
It would be really naive to think IIR (Infinite Impulse Response model) is doable in small enough time for 68000. This kind of computation needs many multiplies even for a single sample (check the formulas on the TTA website). Then come Rice codes, which are enough to take 100% cpu time and more. |
|
17 July 2018, 09:39 | #28 |
Registered User
Join Date: Jan 2014
Location: Poland
Posts: 168
|
[ Show youtube player ]
|
17 July 2018, 11:09 | #29 |
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,323
|
Do you think it will magically solve the lack of power for the 68000 ?
50Mhz 68030 is much faster than this, and is barely able to do Flac which is simpler than TTA (because IIR is tougher than LPC). Anyway this is not at all the OP's target. |
17 July 2018, 11:17 | #30 |
Registered User
Join Date: Jan 2014
Location: Poland
Posts: 168
|
But you probably think about playing 16bit 44k music files, and I did not write anything like that. It's about compressing 8bit samples.
|
17 July 2018, 11:21 | #31 |
son of 68k
Join Date: Nov 2007
Location: Lyon / France
Age: 51
Posts: 5,323
|
You will see little or no speed change going from 16 to 8 bits. Ok, you can get it 4 times faster going to 10khz samples, but 68000 is still out of league.
And again this is not the OP's target. |
17 July 2018, 12:51 | #32 |
Natteravn
Join Date: Nov 2009
Location: Herford / Germany
Posts: 2,496
|
I have to say that I didn't make any further experiments with Huffman encoding for now. I have still about 40K left on the disk and will probably only need some more samples for the menu. Also Meynaf's hint that I have to merge serveral small sample file to get the best compresion ratio out of Huffman is a problem for me, as I have multiple small samples (usuall 6-10K) which must be loaded in various arrangements for every game world.
|
17 July 2018, 14:34 | #33 |
Computer Nerd
Join Date: Sep 2007
Location: Rotterdam/Netherlands
Age: 47
Posts: 3,751
|
|
14 October 2018, 05:09 | #34 |
Registered User
Join Date: May 2013
Location: Grimstad / Norway
Posts: 839
|
How did the compression work out for you?
I just looked at a few of my MODs and saw that SQSH reduced the size by 35-40%, though I don't know how much comes from non-sample data. |
Currently Active Users Viewing This Thread: 1 (0 members and 1 guests) | |
Thread Tools | |
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Playing samples | Lonewolf10 | Coders. Tutorials | 8 | 18 February 2013 22:33 |
Best way to get samples onto the amiga... | ElectroBlaster | Amiga scene | 4 | 21 October 2012 17:13 |
MOD with Highlander samples? | Steve C | request.Modules | 2 | 20 July 2012 19:36 |
Does anyone have any protracker samples adf's? | CaptainNow | Amiga scene | 7 | 26 June 2011 13:36 |
Need Samples from Turrican 2 | hipoonios | request.Other | 9 | 07 April 2010 18:03 |
|
|