AllocVec returning zero - why?

[buzzybee]

Hi guys!

Working on RESHOOT PROXIMA 3 for some months now. From time to time, the executable fails and returns to WB, because AllocVec returns 0 in d0-register. The values in d0 and d1 given to the function are all fine and there is plenty of free memory. Happens at different sections of the game, here and there. Code is pretty straightforward:

move.l #copperGameMaxSize,d0		; copperlist max. size
move.l	#Chip!Mem_Clear,d1
CALLDOS AllocVec 
tst.l d0
beq .noMemory
...

CALLDOS	MACRO
move.l	Execbase,a6
jsr	_LVO\1(a6)
ENDM

Any idea what could cause this behavior?

[alkis]

Hmm, I'd save d1 prior to allocvec, restore immediately after and on fail call availmem and examine d0

[meynaf]

If AllocVec returns zero, then not enough memory. I don't see any other reason.

There could be plenty of memory overall but not enough chipmem.
It is possible total is enough but memory is fragmented.

Typing

avail

command could give more info.

[robinsonb5]

Where you do define Chip and Mem_Clear? (I though the official includes call them MEMF_CHIP and MEMF_CLEAR, respectively?)
Make sure you're using the flag definitions, not the bit definitions.
i.e. flag is (1<<bit)

[Hedeon]

Even if defined correctly, not sure if you wanted ! instead of |

[buzzybee]

Defs look like this

MEMF_ANY        =   0
MEMF_CHIP		=	1<<1
MEMF_FAST		=	1<<2
MEMF_CLEAR   	=   1<<16
MEMF_TOTAL		=	1<<19

(cleaned that up a bit right now)

Used Debugger to check that
move.l #Chip!Mem_Clear,d1
or
move.l #Chip|Mem_Clear,d1
both translate to
MOVE.L #$00010002,D0
Which should be fine.

[StingRay]

Where/how is "copperGameMaxSize" defined?

[robinsonb5]

Is it a fragmentation issue?
If you're randomly allocating and freeing larger numbers of memory chunks, and not freeing them in reverse order of allocation, you could end up without a large enough contiguous chunk to satisfy an allocation.
Also, could you be overrunning an allocation and corrupting the memory lists?
(I don't know exactly where AllocVec stores the size that's been allocated, but if I had to guess I'd say immediately before or immediately after the allocated chunk.)

[buzzybee]

Originally Posted by StingRay

Where/how is "copperGameMaxSize" defined?

copperGameMaxSize	=	10500&$fff8	; align to size of 8

Originally Posted by robinsonb5

Is it a fragmentation issue?
If you're randomly allocating and freeing larger numbers of memory chunks, and not freeing them in reverse order of allocation, you could end up without a large enough contiguous chunk to satisfy an allocation.
Also, could you be overrunning an allocation and corrupting the memory lists?
(I don't know exactly where AllocVec stores the size that's been allocated, but if I had to guess I'd say immediately before or immediately after the allocated chunk.)

The error happens when I ran the game a couple of times, so yes – fragmentation might be causing this, especially as I do not yet free memory in reverse order. Should I?

Also, yes, corruption of memory lists might be the case. But the game does compare free memory at the very start and very end of startup, and throws an error if both numbers are not the same. Implemented this to identify memory leaks. If pointers / size markes are overwritten within game code, should this not result in memory leaks?

[Exodous]

Do you need to keep allocating and freeing the memory?

Could you allocate it all at the beginning, use it when necessary, then free it at the end when exiting, thus preventing fragmentation?

[Thomas Richter]

Sounds more like an issue of a bad pointer, damaging memory and hence the freelist of exec. MuGuardianAngel may be helpful to hunt down the issue.

BTW, I think it is a bit strange to call a macro CALLDOS, which actually calls exec. Not dos.

[buzzybee]

Originally Posted by Thomas Richter

Sounds more like an issue of a bad pointer, damaging memory and hence the freelist of exec. MuGuardianAngel may be helpful to hunt down the issue.

My stomach tells me the same. How would you check for such an issue?

Originally Posted by Thomas Richter

BTW, I think it is a bit strange to call a macro CALLDOS, which actually calls exec. Not dos.

True. Did that macro years ago and stuck with it. Changed it right now.

[buzzybee]

Originally Posted by Exodous

Do you need to keep allocating and freeing the memory?

Could you allocate it all at the beginning, use it when necessary, then free it at the end when exiting, thus preventing fragmentation?

Well... I do most memory allocation like that, and de-alloc / re-alloc one memory chunk only when switching between stages. Should not cause too much fragmentation.

[Thomas Richter]

Originally Posted by buzzybee

My stomach tells me the same. How would you check for such an issue?

As said. Run MuGuardianAngel.

[Galahad/FLT]

Originally Posted by buzzybee

Well... I do most memory allocation like that, and de-alloc / re-alloc one memory chunk only when switching between stages. Should not cause too much fragmentation.

Better idea is to figure out how much chipmem you need to allocate in total for a stage and allocate as one block that will take care of lots of smaller allocations.

Will likely fix your memory issues.

[hooverphonique]

Originally Posted by buzzybee

Defs look like this

MEMF_ANY        =   0
MEMF_CHIP        =    1<<1
MEMF_FAST        =    1<<2
MEMF_CLEAR       =   1<<16
MEMF_TOTAL        =    1<<19

But those are not the symbols you used in the code you showed

Originally Posted by buzzybee

copperGameMaxSize    =    10500&$fff8    ; align to size of 8

If your copper list can grow to 10500 bytes, you need to round up: (10500+7)&fff8

Considering you didn't mention any memory list-related gurus, I suspect memory fragmentation as well..

[buzzybee]

Originally Posted by Galahad/FLT

Better idea is to figure out how much chipmem you need to allocate in total for a stage and allocate as one block that will take care of lots of smaller allocations.

Will likely fix your memory issues.

Makes sense. Okay, will modify memory alloc structure in such a way and see what happens; also will see what MuGuardianAngel can do – never really worked with that tool before.

Guys, thanks a lot for inspiration. Will keep you posted!

[BastyCDGS]

Memory allocation can also fail if the memory list gets damaged.

Depending on the damage done, largest free memory block can shrink dramatically.

I have experienced largest free blocks suddenly dropping down from 98% to less than 10% of total memory, just because of memory corruption (actual memory allocated and used was in few KB range).

If in doubt, consider C:AVAIL it will show both free and largest free!

Accidentally overwritten memory is really evil!!!

[buzzybee]

Originally Posted by BastyCDGS

Memory allocation can also fail if the memory list gets damaged.

Depending on the damage done, largest free memory block can shrink dramatically.

I established a number of checks in "debug mode" which should make sure that I get notifications in case unallocated memory gets written to. Had no more strange memory alloc errors any more since then.

[DMWCashy]

While allocating all the memory you require in one go does make life easier to manage memory, It relies on the fact that no other running application has memory allocated. Even a small application can have allocated ram that splits up the largest memory block, this is sadly the hazard of non paged memory.

So allocating in blocks that you require may still find blocks of adequate size, where allocating the full amount is highly likely to fail, if not called right after a clean boot.

AllocVec will allocate an extra memory block chunk (I believe this is 8 bytes) for the memory details followed by the required memory. This makes it easy for FreeVec() to verify the block, by simply moving back one block ( Same as modern PC based memory heaps work)

When Memory is freed, the first block is used as a node for the free memory list (Size and Next pointer). If this memory is written to after freed it will destroy the free memory list.

Though I would suspect that they would have to be zeroed, as any other data would most likely result in a crash.

If the allocated memory block is altered, i.e a negative offset writing over the information block, this would cause the FreeVec() function to free an incorrect amount of memory.

You could check this size in the info block before freeing to see if this has been overwritten.