-fwhole-program woes

[Nightfox]

Hi all

I'm using a dockerised Bebbo's gcc 6 toolchain and I've noticed something weird with our favourite C function printf(). I have noticed this same issue with both libnix (-noixemul) and clib2 (-mcrt=clib2).

Making a simple hello world program in a tiny main.c with the lines

#include <proto/exec.h>
#include <dos/dos.h>
#include <proto/dos.h>
#include <stdio.h>
#include <workbench/workbench.h>

int __initlibraries=1; /* Disable auto-library-opening */

struct DosLibrary *DOSBase=NULL;

int main(void) {
	SysBase = *((struct ExecBase**)4UL);
	if ((DOSBase=(struct DosLibrary *)OpenLibrary("dos.library",37)) != NULL) {
		printf("Never gonna give you up\n");
      	CloseLibrary((struct Library *)DOSBase);
	}
	return 0;
}

When running the program it will freeze the system when it reaches the printf() line. Commenting out the printf() it will no longer freeze.

If however it is compiled with the -fwhole-program option the program doesn't freeze and it successfully rickrolls me.

I would normally have no issue just using this compiler option (I use this option in my AmigaGPT code) however since I am in the process of porting AmigaGPT from ReAction to MUI, for some reason if I have -fwhole-program enabled I get a linker error "libmui.c:60: undefined reference to `MUIMasterBase'" despite me making MUIMasterBase a global variable in the same file where I am using MUI.

Also, attempting to use libjson-c without -fwhole-program enabled will cause a crash.

It freezes in AmigaOS 3.2 within AmiKit. Unsure if it freezes in other versions.

I also get freezes with the alternates "Printf()" and "Write()".

So I need to know:

1) What exactly does fwhole-program do?
2) Why do print statements cause the program to crash without it enabled?
3) Why does MUI refuse to link with it enabled?

My whole weekend was ruined with my frustration trying to get this all to play nice.

For reference here is the Makefile with it enabled. I borrowed the Makefile from AmigaGPT and made some tweaks to remove linking unused libs.

# to generate assembler listing with LTO, add to LDFLAGS: -Wa,-adhln=$@.listing,--listing-rhs-width=200
# for better annotations add -dA -dP
# to generate assembler source with LTO, add to LDFLAGS: -save-temps=cwd
OS := $(shell uname)

subdirs := $(wildcard */) $(wildcard src/*/) $(wildcard src/*/*/)
VPATH = $(subdirs)
BUILD_DIR := build/os3/obj/
cpp_sources := $(wildcard *.cpp) $(wildcard $(addsuffix *.cpp,$(subdirs)))
cpp_objects := $(addprefix $(BUILD_DIR),$(patsubst %.cpp,%.o,$(notdir $(cpp_sources))))
c_sources := $(wildcard *.c) $(wildcard $(addsuffix *.c,$(subdirs)))
c_sources := $(filter-out src/test/%, $(c_sources))
c_objects := $(addprefix $(BUILD_DIR),$(patsubst %.c,%.o,$(notdir $(c_sources))))
s_sources := $(wildcard *.s) $(wildcard $(addsuffix *.s,$(subdirs)))
s_objects := $(addprefix $(BUILD_DIR),$(patsubst %.s,%.o,$(notdir $(s_sources))))
vasm_sources := $(wildcard *.asm) $(wildcard $(addsuffix *.asm, $(subdirs)))
vasm_objects := $(addprefix $(BUILD_DIR), $(patsubst %.asm,%.o,$(notdir $(vasm_sources))))
objects := $(cpp_objects) $(c_objects) $(s_objects) $(vasm_objects)

AUTOGEN_FILE := src/version.h
AUTOGEN_NEXT := $(shell expr $$(awk '/#define BUILD_NUMBER/' $(AUTOGEN_FILE) | tr -cd "[0-9]") + 1)

# https://stackoverflow.com/questions/4036191/sources-from-subdirectories-in-makefile/4038459
# http://www.microhowto.info/howto/automatically_generate_makefile_dependencies.html

PROGRAM_NAME = AmigaGPT
EXECUTABLE_OUT = out/$(PROGRAM_NAME)
CC = m68k-amigaos-gcc
VASM = vasmm68k_mot

LIBDIR = /opt/amiga/m68k-amigaos/lib
SDKDIR = /opt/amiga/m68k-amigaos/sys-include
NDKDIR = /opt/amiga/m68k-amigaos/ndk-include
INCDIR = /opt/amiga/m68k-amigaos/include

ifeq ($(OS),Darwin)
	SED = sed -i "" 
else
	SED = sed -i
endif

CCFLAGS = -MP -MMD -m68020 -Ofast -Wextra -Wno-unused-function -Wno-discarded-qualifiers -fwhole-program -Wno-int-conversion -Wno-volatile-register-var -fomit-frame-pointer -fno-tree-loop-distribution -fno-exceptions -D__AMIGAOS3__ -DPROGRAM_NAME=\"$(PROGRAM_NAME)\" -noixemul
CPPFLAGS= $(CCFLAGS) -fno-rtti -fcoroutines -fno-use-cxa-atexit
ASFLAGS = -Wa,-g,--register-prefix-optional,-I$(SDKDIR),-I$(NDKDIR),-I$(INCDIR),-D
LDFLAGS =  -Wl,-Map=$(EXECUTABLE_OUT).map
VASMFLAGS = -m68020 -Fhunk -opt-fconst -nowarn=62 -dwarf=3 -quiet -x -I. -D__AMIGAOS3__ -DPROGRAM_NAME=\"$(PROGRAM_NAME)\" -I$(INCDIR) -I$(SDKDIR) -I$(NDKDIR)

.PHONY: all clean copy_bundle_files

all: copy_bundle_files $(EXECUTABLE_OUT)

$(BUILD_DIR):
	@$(info Creating directory $@)
	@mkdir -p $@

$(EXECUTABLE_OUT): $(objects)
	$(info Linking $(PROGRAM_NAME))
	$(CC) $(CCFLAGS) $(LDFLAGS) $(objects) -o $@ $(LDFLAGS) 

clean:
	$(info Cleaning...)
	@$(RM) -f $(EXECUTABLE_OUT)
	@$(RM) $(BUILD_DIR)*

-include $(objects:.o=.d)

$(cpp_objects) : build/os3/obj/%.o : %.cpp | build/os3/obj/%.dir
	$(info Compiling $<)
	$(CC) $(CPPFLAGS) -c -o $@ $(CURDIR)/$<

$(c_objects) : build/os3/obj/%.o : %.c | $(BUILD_DIR)
	$(info Compiling $<)
	$(SED) 's|#define BUILD_NUMBER ".*"|#define BUILD_NUMBER "$(AUTOGEN_NEXT)"|' $(AUTOGEN_FILE)
	$(CC) $(CCFLAGS) -c -o $@ $(CURDIR)/$<

$(s_objects): build/os3/obj/%.o : %.s |$(BUILD_DIR)
	$(info Assembling $<)
	$(CC) $(CCFLAGS) $(ASFLAGS) -c -o $@ $(CURDIR)/$<

$(vasm_objects): build/os3/obj/%.o : %.asm | $(BUILD_DIR)
	$(info Assembling $<)
	$(VASM) $(VASMFLAGS) -o $@ $(CURDIR)/$<

copy_bundle_files:
	$(info Copying bundle files...)
	mkdir -p out
	cp assets/AmigaGPT_OS3.info bundle/$(PROGRAM_NAME)/$(PROGRAM_NAME).info
	cp -R bundle/$(PROGRAM_NAME)/* out/

[giantclam]

Have a read here -> https://gcc.gnu.org/onlinedocs/gccint/WHOPR.html

[phx]

Originally Posted by Nightfox

When running the program it will freeze the system when it reaches the printf() line.

Most likely the clib's stdio subsystem was not correctly initialised. But it doesn't explain why the dos.library

Printf()

fails.

Do you really know what

__initlibraries=1

is doing? From its name it seems strange that it disables automatic library opening. Other compilers do not open libraries by default, unless you link with

-lauto

or similar.

1) What exactly does fwhole-program do?

I didn't know this option before. It seems to allow some extra-aggressive optimisations based on the knowledge that your source more or less represents the whole program.

Generally I wouldn't use any options where you don't know exactly what they are doing. Also do not start optimising with strange options before a first version of your program is finished and works. Your makefile has a lot of

-f

.

Theoretically, a simple DOS-based helloworld with

Printf()

should neither need a startup-code nor any linker-library (maybe

-lamiga

if the inline-calls do not work).

gcc -c helloworld.c
ld -o hello helloworld.o

In your case: set a breakpoint at

main()

and debug the mystery!

[Nightfox]

Do you really know what __initlibraries=1 is doing? From its name it seems strange that it disables automatic library opening.

Sorry that was a typo. Either way it still crashes when it is set to 0.

Yeah I might have to get around to setting up my amiga environment for debugging. Right now my only debugging has been printf() but right now I can't even use that haha

[alkis]

The following works (in vamos) both with -fwhole-program and without.

eab.c

#include <dos/dos.h>
#include <proto/dos.h>
#include <proto/exec.h>
#include <stdio.h>
#include <workbench/workbench.h>

void __nocommandline() {}; /* Disable commandline parsing  */
void __initlibraries() {}; /* Disable auto-library-opening */

struct DosLibrary *DOSBase = NULL;
struct ExecBase *SysBase = NULL;

struct WBStartup *_WBenchMsg;

int main(void) {
  SysBase = (struct ExecBase *)*((struct ExecBase **)4UL);
  if (_WBenchMsg == NULL) {
    if ((DOSBase = (struct DosLibrary *)OpenLibrary("dos.library", 37)) != NULL) {
      printf("Never gonna give you up\n");
      CloseLibrary((struct Library *)DOSBase);
      return 0;
    }
  }
  return 5;
}

void exit(int n) {}
int puts(const char *s) {
  PutStr(s);
  PutStr("\n");
}

mystart.s

        jmp     _main

m68k-amigaos-gcc -noixemul -nostartfiles -nostdlib -fwhole-program -o eab mystart.s eab.c

Doesn't "feel" right. But it compiles and runs.

[bebbo]

1) What exactly does fwhole-program do?

From the gcc help:

Assume that the current compilation unit represents the whole program being
compiled. All public functions and variables with the exception of @code{main}
and those merged by attribute @code{externally_visible} become static functions
and in effect are optimized more aggressively by interprocedural optimizers.

This option should not be used in combination with @option{-flto}.
Instead relying on a linker plugin should provide safer and more precise
information.

2) Why do print statements cause the program to crash without it enabled?

Using -fwhole-program affects your variables and your zero intialized variable DOSBase becomes a common symbol and __initlibraries is no longer global visible. Thus the DOSBase gets initialized early - as needed by other startup code - and it does not crash.

The other way, your botchering results in an unitialized DOSBase, which is used before main is entered.
To avoid this you also have to botcher __nocommandline away.

But who wants to do that? What are the real benefits of botchering with a library?

3) Why does MUI refuse to link with it enabled?

Remove the botch and it will link and run.

=> You can use -fwhole-program as long you have only one source file. Sometimes you can write a file and include all the other files, if these are prepared to do so. If not, use -flto. That's a different story.

EDIT:
If you want smaller programs, you might test

#include <amistdio.h>

instead of the normal stdio header. It's not 1:1 but it's usable (see amigassh).
- use BPTR instead of FILE *
- don't use fopen/close use Open/Close
- use %ld in printf for int, %d is for short, which is not supported by the gcc calling convention.
- the most used functions are there...

[Nightfox]

By botch I assume you mean my use of __initlibraries=0

Indeed removing that fixes the issue however it introduces a new issue if I apply the same solution to my AmigaGPT app. It uses ReAction gadgets which in my code it opens them with OpenLibrary() as usual but now that this __initlibraries() thing is allowed to run, it tries to auto open the various GUI gadgets and the program exits with an error saying failed to load button.library, clicktab.library etc which of course fails because the library files have the .gadget extension not .library

[bebbo]

Originally Posted by Nightfox

By botch I assume you mean my use of __initlibraries=0

Indeed removing that fixes the issue however it introduces a new issue if I apply the same solution to my AmigaGPT app. It uses ReAction gadgets which in my code it opens them with OpenLibrary() as usual but now that this __initlibraries() thing is allowed to run, it tries to auto open the various GUI gadgets and the program exits with an error saying failed to load button.library, clicktab.library etc which of course fails because the library files have the .gadget extension not .library

I'd say: your library is too old. I'm pretty sure there are still bugs... but the code for the mentioned libraries reads:
- gadget/button.gadget
- gadget/clicktab.gadget
- ...


These stubs where once generated by a script and that bug was discovered and fixed around February 2024. (https://github.com/bebbo/libnix/comm...2d58758404b6d4)


Nevertheless, there is a good way to open libs by yourself, which is e.g. needed if a library is optional. Provide it as an initialized variable:

struct Library * SelfOpenedBase = 0;

[Nightfox]

Ah perfect thanks! Gosh I wish these little tricks were documented. Or if they are I was not able to find them. I had declared the bases in the file as globals but were uninitialised