Is it possible to reduce an AllocMem() chunk?

[hooverphonique]

Originally Posted by a/b

Exec is tracking free and allocated chunks with linked lists. Free chunks have an 8-byte header at the start, and allocated chunks at the end.

Is this new? I'm sure back in the days, exec would only track free memory.

[a/b]

Did the same test on KS1.3, the same outcome (3 headers).

[Thomas Richter]

No, just incorrect information. Exec only tracks free chunks.

However, you should really not depend on the internals of the allocator, especially on the rounding policy. What you get is sufficiently aligned to make all members of the 680x0 happy, with the exception of MOVE16 which you should not use anyhow.

The rule is very simple: What you allocate with AllocMem you release with FreeMem, exactly the same pointer, the same length. What you allocated with AllocVec you release with FreeVec, same address, exec keeps care of the length. What you allocated with AllocAbs you release with FreeMem, namely the address you requested and the size you requested *AND NOT* the address you received from AllocAbs(!) which may be slightly off, but contains the requested range *if* allocation was successful.

Do not depend on the implementation details of the allocator. There are custom allocators out in the wild that may operate (more or less) different.

[a/b]

Originally Posted by Thomas Richter

No, just incorrect information. Exec only tracks free chunks.

Ugh, I understand that you have a lot of free time on your hands (e.g. the "Any reason to use assembly on the Amiga rather than C" thread :P) and you are now wasting mine, but anyway...
These were done before I made my previous post, so here they are redone on A500+KS1.3 and documented just to show I didn't pull it out of my... back socket.
1. preliminary run to get the approx. address (lets call it X) of an allocated 1024 bytes block
2. zero out all free memory
3. doublecheck a few KB around the address X, all zeroes
4. run again, breakpoint right after X=AllocMem(1024)/Forbid() and filling 1024 bytes at X with a pattern
5. contents at X: pattern 1024 bytes, $0.L, $732a8.L
6. call FreeMem(X,1024)
7. contents at X: 0.L, $736a8.L (=$732a8+1024), pattern 1016 bytes, 0.L, $732a8.L
8. clear 8 bytes at X+1024 (last 2 .L from step 7)
9. contents at X: 0.L, $736a8.L, pattern 1016 bytes, 0.L, 0.L
10. call first AllocAbs(X,512), returned d0 == passed a1
11. contents at X: (X+512).L, $736a8.L, pattern 504 bytes, 0.L, $734a8.L (=$736a8-512), pattern 504 bytes, 0.L, 0.L
12. call second AllocAbs(X+512,512), returned d0 == passed a1
13. contents at X: (X+512).L, $736a8.L, pattern 504 bytes, (X+1024).L, $734a8.L (=$736a8-512), pattern 504 bytes, 0.L, $732a8.L (=$734a8-512=$736a8-1024)

So, again. After the first AllocAbs() those 8 bytes that magically appeared at X+512 sure look like a private tracking info, why was I able to AbsAlloc() at that very same address. Genuinely confused. Especially (shame on me) because the 4 instructions at label aa_splitsucc in memory.asm line 1405 look just like some kind of tracking.

[Docent]

Originally Posted by a/b

So, again. After the first AllocAbs() those 8 bytes that magically appeared at X+512 sure look like a private tracking info, why was I able to AbsAlloc() at that very same address. Genuinely confused. Especially (shame on me) because the 4 instructions at label aa_splitsucc in memory.asm line 1405 look just like some kind of tracking.

Indeed these bytes are sort of tracking - they are the remains of MemChunk struct that was created in the free area to track free space.
When you did the first AllocAbs, the block of free memory that contained the requested start address had to be split into three blocks - one allocated and two free blocks (one before allocated block and the second after allocated block). Free blocks were linked into free blocks list, so at the beginning of each free block the MemChunk struct was initialized. The MemChunk struct is only valid for free blocks, so you were able to allocate the second free block with this struct at the beginning.

[a/b]

Yes, that makes sense. I realized it when I woke up today and thought that allocating the second half first would've made a more interesting test case.
And the autodocs stating "The 8 bytes past the end of an AllocAbs will be changed by Exec relinking the next block of memory." certainly didn't help. I mean, it's correct but could've been a little more informative (e.g. "next free block").

[Rotschi]

@a/b and Olaf Barthel
Thanks for the replies and your time. Hope, I have not opened a can of worms with my naive question :-|
Anyway, the good news are: The cat is still alive :-)

[Docent]

This scheme of keeping information on free memory is also the reason why the minimal allocated block size is 8 bytes - when freeing allocated chunk must have enough space to hold the struct MemChunk, which is 8 bytes in size ...

[8bitbubsy]

Oh man, what have I done with making this thread...

[NorthWay]

Originally Posted by Docent

This scheme of keeping information on free memory is also the reason why the minimal allocated block size is 8 bytes - when freeing allocated chunk must have enough space to hold the struct MemChunk, which is 8 bytes in size ...

I once hacked Exec in my 4000 to round all allocations up to a multiple of 4K. I couldn't understand where the bug in my code was as it would not work.
Well, it worked just fine; the only problem was that there are so many small allocs during boot that the 16M was not enough to complete bring-up.

[Thomas Richter]

Things should be better in 3.2 now as the two most important sources of fragmentation are gone. These are the cliprects of the layers library, and the region and region rectangles of graphics. Both are responsible for window and layer clipping.

[a/b]

Originally Posted by 8bitbubsy

Oh man, what have I done with making this thread...

We just took a (not-really-)shortcut to get back to the start. Based on your description in #7, I don't see why my suggestion in #2 wouldn't work. So, again...
E.g. you allocmem a 256KB buffer, free it, then allocabs 2x 128KB. Now you hold the same completely consecutive 256KB, no gaps or anything, and you can start filling them with data and then you process it. There was nothing there before nor is there anything inbetween now to get corrupted. Then you normalize 16-bit to 8-bit, basically shrink down data to a half, so now it completely fits into the first absolute buffer, and you free the second one, as far as I understand your description.
Of course, you have to ensure that the sizes and the second address are 64-bit aligned (=> 128-bit align the full size so it's divisible by 2).
If any allocabs fails, you can fall back to a full buffer and set a flag so you know what to do later on. You could also check if a system has KS >= 2.0 and more than 512KB (or whatever you think is a minimum) and if so you just work with a full buffer (to cover a theoretical case of your app running on some imaginary tripple next-gen Amiga with a weird non-compatible memory pool handler, because I doubt something like a 512KB KS1.3 system will have one).
I don't see a problem with this approach, if all alloc calls succeed, and they should, it will work.

[hooverphonique]

Originally Posted by Thomas Richter

Things should be better in 3.2 now as the two most important sources of fragmentation are gone. These are the cliprects of the layers library, and the region and region rectangles of graphics. Both are responsible for window and layer clipping.

This only affects chipmem on a system without 3rd party patches, right!?

[Thomas Richter]

No, it only affects regular memory. All these structures are relatively tiny administration structures. The bitmaps, if needed, are allocated in addition.

[8bitbubsy]

Originally Posted by a/b

We just took a (not-really-)shortcut to get back to the start. Based on your description in #7, I don't see why my suggestion in #2 wouldn't work. So, again...
E.g. you allocmem a 256KB buffer, free it, then allocabs 2x 128KB. Now you hold the same completely consecutive 256KB, no gaps or anything, and you can start filling them with data and then you process it. There was nothing there before nor is there anything inbetween now to get corrupted. Then you normalize 16-bit to 8-bit, basically shrink down data to a half, so now it completely fits into the first absolute buffer, and you free the second one, as far as I understand your description.
Of course, you have to ensure that the sizes and the second address are 64-bit aligned (=> 128-bit align the full size so it's divisible by 2).
If any allocabs fails, you can fall back to a full buffer and set a flag so you know what to do later on. You could also check if a system has KS >= 2.0 and more than 512KB (or whatever you think is a minimum) and if so you just work with a full buffer (to cover a theoretical case of your app running on some imaginary tripple next-gen Amiga with a weird non-compatible memory pool handler, because I doubt something like a 512KB KS1.3 system will have one).
I don't see a problem with this approach, if all alloc calls succeed, and they should, it will work.

I decided to go with the slow method, like said. So max 128kB allocation (for a 128kB sample). It's faster on A1200 and expanded Amigas anyway.