Different Results for malloc(0) Without/With Optimisations with VBCC

[tygre]

Hi there!

I stumble upon a problem with malloc() when the size is 0. I have this piece of code that can receive different sizes (depending on the HTTP answer from a Web server). Essentially:

	hresp->body     = malloc(number_of_bytes);
	if(hresp->body  == NULL)
	{
		temp_string = NULL;
		log_print_error("_http_request(), could not allocate response body\n");
		goto _RETURN_ERROR;
	}

When compiling without optimisation, when number_of_bytes == 0, malloc returns a non-NULL value (which was fine for the rest of my code). When compiling with optimisation, when number_of_bytes == 0, malloc returns a NULL value (which broke my code, but I fixed it ).

Without optimisation:

all :  COFLAGS += -O=1 -g -hunkdebug -DFORTIFY -DUNRECOGNISED
all :  ASFLAGS +=
all :  LDFLAGS += -S -s -sc -sd

With optimisation:

dist : COFLAGS += -O=3 -size -final
dist : ASFLAGS += -opt-size
dist : LDFLAGS +=

I wonder if that's expected, with VBCC v0.9h?

Cheers!

[coldacid]

Why are you allocating 0 bytes, though? Would it be possible to just skip malloc() when you aren't allocating anything?

[tygre]

Hi ColdAcid!

Yes, I know have a if that checks for zero value but I preferred my code before, it was simpler, more "linear" .

The reason for have a zero value is that the body of a HTTP response may be of 0-length, for example in the case of a redirect (status code 302).

Still, I wonder if the difference of behaviours is expected wrt. the difference compiler optimisations?

Take care!

[patrik]

Content-Length is not required in the answer and might be missing for unknown reasons. An example where it is always missing is for streaming sources.

[Karlos]

Have you checked what the C library says about it? It sounds like something that might be undefined behaviour.

[Thomas Richter]

Both is actually allowed.

The malloc() function allocates size bytes and returns a pointer to the allocated
memory. The memory is not initialized. If size is 0, then malloc() returns either
NULL, or a unique pointer value that can later be successfully passed to free().

[Karlos]

Personally I'd never try to allocate zero bytes. Sounds like a logical error to me.

[hooverphonique]

Originally Posted by Thomas Richter

Both is actually allowed.

I like the second one where you don't (as a malloc client) need to treat 0 size as a special case.


On the other hand, if it changes with optimization levels, then not so much.

[tygre]

Hi all!

Exactly my point, HooverPhonique: I liked the case "a unique pointer value that can later be successfully passed to free()" because it made my code more straightforward

I'm just wondering if it's expected from VBCC (or something else?) to have two different behaviours depending on the compilation options?

Cheers!

[Karlos]

Originally Posted by tygre

Hi all!

Exactly my point, HooverPhonique: I liked the case "a unique pointer value that can later be successfully passed to free()" because it made my code more straightforward

I'm just wondering if it's expected from VBCC (or something else?) to have two different behaviours depending on the compilation options?

Cheers!

I don't remember the last iteration of libc I've used were free(0) wasn't a no-op.

[tygre]

PS. Here is the problem is with malloc(0)

[vbc]

Originally Posted by tygre

When compiling without optimisation, when number_of_bytes == 0, malloc returns a non-NULL value (which was fine for the rest of my code). When compiling with optimisation, when number_of_bytes == 0, malloc returns a NULL value (which broke my code, but I fixed it ).

What target configuration are you using? I think the vbcc libraries for m68k-amigaos and m68k-kick13 both explicitly check for 0 and return 0.

With optimisation:

dist : COFLAGS += -O=3 -size -final

Btw. -O=3 is probably not what you want. For high optimization level use option -O3 with vc or -O=-1 when calling vbcc directly (although using the frontend is recommended as it takes care of some other stuff).

[tygre]

Hi VBC!

I'm targetting m86k-amigos, compiling against NDK v3.2R4.

Thanks for the tip about the optimisation!
Update: I tried O=-1 and the resulting object files are broken: they are between 28 and 40 bytes instead of "normal" sizes

Cheers!

[vbc]

Originally Posted by tygre

I'm targetting m86k-amigos, compiling against NDK v3.2R4.

Strange. I double-checked the vc.lib from the target-archive and it does contain a check for 0 in malloc(). Are you sure about the malloc(0)?

Update: I tried O=-1 and the resulting object files are broken: they are between 28 and 40 bytes instead of "normal" sizes

Without knowing your build commands, I would suggest removing -final. This option requires that the entire source code is passed to vbcc in a single invocation and everything that is not reached from main() or objects marked __entry is removed. With separate compilation it will pretty much remove everything. For most normal cases you can omit it and the linker will do an adequate job.

[tygre]

Originally Posted by vbc

Strange. I double-checked the vc.lib from the target-archive and it does contain a check for 0 in malloc(). Are you sure about the malloc(0)?

Yes, absolutely sure
It bit my behind when I released the "release" version of my code (with the optimisations), in which malloc(0) was now returning NULL instead of a "valid" pointer.

Originally Posted by vbc

Without knowing your build commands, I would suggest removing -final. This option requires that the entire source code is passed to vbcc in a single invocation and everything that is not reached from main() or objects marked __entry is removed. With separate compilation it will pretty much remove everything. For most normal cases you can omit it and the linker will do an adequate job.

Exactly that, thanks

[vbc]

Originally Posted by tygre

Yes, absolutely sure
It bit my behind when I released the "release" version of my code (with the optimisations), in which malloc(0) was now returning NULL instead of a "valid" pointer.

Yes, what seems strange to me is that the non-optimized version would have returned non-zero for malloc(0). Are you sure that there are no other differences between those builds?

[phx]

Originally Posted by tygre

I'm just wondering if it's expected from VBCC (or something else?) to have two different behaviours depending on the compilation options?

Like vbc I see no reason why it should happen with vbcc's vclib. But you should expect different behaviours! For example if you want to compile your source with different compilers or clibs.

Maybe some of your defines in the no-optmisation build (like -DFORTIFY) overload vclib's malloc() and implement your own functions for debugging purposes?

Otherwise, prepare a small example for reproduction and I will happily analyse it.

[tygre]

Hi there

Indeed it could be something like Fortify... let me prepare this MWE and see if I can reproduce this difference!

Will keep you posted!

[tygre]

Hi there!

So I created a MWE and, indeed, it's Fortify that changes the behaviour of malloc(0)! With Fortify enabled, malloc(0) returns a non-NULL pointer ("0 bytes in 1 blocks with 104 bytes overheard") while, with Fortify disabled, malloc(0) returns NULL.

Mystery solved!

(I joined this MWE, just (un)comment -DFORTIFY in the makefile to see the difference.)

Cheers!

[Olaf Barthel]

Originally Posted by tygre

Hi there!

I stumble upon a problem with malloc() when the size is 0. I have this piece of code that can receive different sizes (depending on the HTTP answer from a Web server).

Sorry for being late: the effect of malloc(0) is basically implementation-defined. The standard (this being POSIX and the ISO 'C' standard: visit https://pubs.opengroup.org/onlinepub...9.2018edition/ and enter "malloc" into the search form) offers two alternative behaviours. The first being to return NULL and the second returning a pointer to a small memory allocation.

The second behaviour is very common on POSIX systems and the malloc/calloc/realloc implementations in my clib2 code feature this behaviour if you use libunix.a instead of the libc.a variant (which uses the first behaviour).

Small computer systems such as the Amiga tend to have a different view here with regard to what the standard library functions might do. malloc(0) might not yield option #1 (a NULL pointer), option #2 (a pointer to a small memory allocation), but option #3477: memory corruption with or without a crash later or maybe sooner.

All this goes to say that you should never ever put your trust into the 'C' standard library to do right by you under circumstances which challenge the specification of the respective library function. If you read the historic documentation for it, e.g. Manx Aztec 'C' of yore, you will find that it mostly describes the "happy case" and leaves the likely undefined behaviour outside the specified use to the imagination of the reader.

Bottom line is that you need to be most careful on the Amiga, reviewing code you are porting for gotchas such as the malloc(0) jack-in-the-box. Defensive programming is called for, not just for your own code but also for somebody else's...