VBCC: building a TextEdit syntax plugin for OS3.2

[guy lateur]

I'm trying to build a plugin for some decent M68K ASM syntax highlighting for TextEdit on OS3.2. My code is based on the example by Camilla Boemann found here: https://gitlab.com/boemann/texteditfiletypeplugins

I can get her code to build, but it's using SAS/C, and I'd rather use vbcc/vasm for my future developments. I'm using the default vc.config for the m68k amigaos 2/3 target, and this is my makefile (using gnu make):

name	= exampleplugin
cc	    = vc
objs	= example.o pluginheader.o m68k.o util.o

cflags    = -INDK32:Include_H -c99
lflags    = -LNDK32:lib -lamiga -ldebug

retest:         $(objs)
	          $(cc) $(lflags) -o $(name) $(objs)

example.o:      example.c m68k.h util.h
	          $(cc) $(cflags) -c example.c
pluginheader.o: pluginheader.c
	          $(cc) $(cflags) -c pluginheader.c
m68k.o:         m68k.c m68k.h util.h
	          $(cc) $(cflags) -c m68k.c
util.o:         util.c util.h
	          $(cc) $(cflags) -c util.c

Here's the output:

vc -LNDK32:lib -lamiga -ldebug -o exampleplugin example.o pluginheader.o m68k.o util.o
vc.lib(_main.c): In "___main":
Error 21: vc.lib(_main.c) (CODE+0xac): Reference to undefined symbol _main.
m68k.o: In "_FormatTest2":
Error 21: m68k.o (CODE+0x46): Reference to undefined symbol _HighlightSetFormat.
vlink failed returncode 20
vlink -bamigahunk -x -Bstatic -Cvbcc -nostdlib -mrel vlibos3:startup.o "example.o" "pluginheader.o" "m68k.o" "util.o"   -LNDK32:lib -lamiga -ldebug -s -Rshort -Lvlibos3: -lvc -o exampleplugin failed

The first error is about

_main

being undefined. I'm guessing the linker is trying to generate an executable rather than a library. I've been playing around with some linker settings, but I can't seem to shake this error. Does anybody know what switches I need to set to have it generate a library? I need a file of the type produced by Camilla's code -- which is whatever TextEdit needs for its plugins.

The second error is about

_HighlightSetFormat

. This error is slightly more alarming, as this function is declared in a file included from

proto/texteditor.h

. This file is full of checks of which compiler is being used, like

LATTICE, __SASC, __STORM__ and __GNUC__

and so on. No mention of anything resembling VBCC, though. Does this mean this is not (yet) supported by VBCC? Is there anything I can do to fix this? She does mention making sure to build with data=far and not data=near, but I have no idea what that means, let alone how to convince vc/vlink to do that.

I've attached my code folder. You can test it by downloading Camilla's repository and putting my stuff in the example folder (or any folder at the same level, I presume). TIA for any help!

[Samurai_Crow]

Shared libraries in AmigaOS 68k ARE executables. The main is usually just a return 0.

[alkis]

You have to skip linking with normal c startup. I have no experience with vbcc but you have to tell it to not use any startup.

[guy lateur]

Originally Posted by alkis

You have to skip linking with normal c startup. I have no experience with vbcc but you have to tell it to not use any startup.

This indeed seems to solve the first problem. I've changed the second line in my makefile to

cc = vc -nostdlib

. This is what the docs say about this option: "Do not link with standard-startup/libraries. Useful only for people who know
what they are doing." I'm not sure about that last part, but hey, it seems to work. The same option can also be given to the linker ("Ignore default library search path, if one was compiled in."), which is a little confusing if you ask me.

So the error about

_main

being undefined has disappeared, but I still have the second error about

_HighlightSetFormat

being undefined. I've included all relevant header files I could find, including the one containing the pragmas -- which I've been told I shouldn't do, but hey, I don't know what else to try. To no avail, however; the error remains. Does anybody have any tips on that?

[Warty]

I think (blind leading blind here!) that you have to rebuild the protos from the .fd or .sfd files, and put those in your vbcc target folder. That got me as far as a build that works (depending). Depending on what, you ask? I'm not sure, but I think maybe some of the link libs for VBCC need updating. As soon as I added some reaction stuff, I hit some compiler errors saying this or that was defined differently. But again, I have a very superficial understanding of how anything outside my own code actually works.

[guy lateur]

Originally Posted by Warty

I think (blind leading blind here!) that you have to rebuild the protos from the .fd or .sfd files, and put those in your vbcc target folder. That got me as far as a build that works (depending). Depending on what, you ask? I'm not sure, but I think maybe some of the link libs for VBCC need updating. As soon as I added some reaction stuff, I hit some compiler errors saying this or that was defined differently. But again, I have a very superficial understanding of how anything outside my own code actually works.

Yeah, I saw your post about that, and that's what I thought as well. I really don't feel comfortable doing that, though. Like you, I'm having enough trouble understanding my own code and compilation/linking process, let alone anything system related.

But as I understand it, phx is working on a new target release that resolves these kinds of issues? If that is the case, I think I'll wait for him to finish his work, rather than trying to 'roll my own'. I'm not in any particular hurry; I've got plenty of other stuff to go through in the meantime..

[phx]

Originally Posted by guy lateur

But as I understand it, phx is working on a new target release that resolves these kinds of issues?

Yes, I am. And I'm also in contact with the OS3.2 team to include vbcc support in the next SDK release, which will hopefully appear soon.

For your plugin it is not only important to link without startup code, but I guess also the order of object files matters. The SAS/C example project links pluginheader.o first, which makes sure the file starts with the

PluginHead

structure.

Then I noticed that you didn't define any Library base. You just declared them as external references everywhere. This might work when having a startup code to initialize

SysBase

and

DOSBase

, and

auto.lib

to automatically open other libraries. But without startup code you have to do everything yourself. Remove the

extern

in front of the bases in pluginheader.c.

There are still some SAS/C specific attributes in

example.c

:

__asm

is not needed with vbcc. And a register argument

a0

is defined with

__reg("a0")

instead of

register __a0

.

[bubbob42]

The following is useful to write compiler-independent source:

#ifdef __VBCC__
#define INTERRUPT __amigainterrupt __saveds
#define LIBFUNC __saveds
#define REG(reg, arg) __reg(#reg) arg
#endif

#ifdef __SASC
 #define INTERRUPT __interrupt __saveds __asm
 #define LIBFUNC __saveds __asm
 #define REG(reg,arg) register __## reg arg
#endif

You’d use REG() for both compilers.

[guy lateur]

Originally Posted by phx

Yes, I am. And I'm also in contact with the OS3.2 team to include vbcc support in the next SDK release, which will hopefully appear soon.

Alright, that's good to hear! It would be great if vbcc support was just included in the SDK. Can't wait for that update to be released!

Originally Posted by phx

For your plugin it is not only important to link without startup code, but I guess also the order of object files matters. The SAS/C example project links pluginheader.o first, which makes sure the file starts with the

PluginHead

structure.

Ok, I see, thanks for the heads-up, I put

pluginheader.o

first.

This does make me wonder how the order works for include paths. For instance, if I

#include <proto/texteditor.h>

. This file can be found via

vincludeos3:

(in cc setting in

vc.config

) as well as via

NDK32:Include_H

(passed to vc via the -I switch; see my makefile). So which file gets included in this case? I suppose I should always try to have the vbcc includes take priority over the NDK ones, right?

Originally Posted by phx

Then I noticed that you didn't define any Library base. You just declared them as external references everywhere. This might work when having a startup code to initialize

SysBase

and

DOSBase

, and

auto.lib

to automatically open other libraries. But without startup code you have to do everything yourself. Remove the

extern

in front of the bases in pluginheader.c.

Yeah, in the original source, the

extern

weren't there, but I had to put them there when switching to vbcc because the bases were already there, for the reasons you mentioned. I did already take them out again after removing the startup code.

Originally Posted by phx

There are still some SAS/C specific attributes in

example.c

:

__asm

is not needed with vbcc. And a register argument

a0

is defined with

__reg("a0")

instead of

register __a0

.

Ok, I've changed that signature to this:

ULONG __saveds highlightBlock(__reg("a0") struct Hook *hook, __reg("a2") VOID *obj, __reg("a1") VOID *msg)
{
	return FormatTest2(hook, obj, msg);
}

This is correct, right? Thanks to bubbob42 for the tip on how to handle these kinds of differences gracefully, btw.


So, after all this, I'm still having the error. I suppose I better just wait for those updates if I don't want to mess with the header files myself? Or is there anything else I could easily try to get this linking properly?

For your inspiration, here's my current m68k.c file. As mentioned earlier, I included all kinds of header files I thought might be relevant, including the inline one. I also have no idea what those defines near the top mean, btw, so if anybody wants to enlighten me..

#include <gadgets/texteditor.h>

#define __NOLIBBASE__
#include <proto/exec.h>

#define __CLIB_PRAGMA_LIBCALL
#include <proto/alib.h>
#include <proto/texteditor.h>
#include <clib/texteditor_protos.h>
#include <inline/texteditor_protos.h>
#include <pragmas/texteditor_pragmas.h>

#include "util.h"
#include "m68k.h"

extern struct Library *SysBase;
extern struct Library *UtilityBase;
extern struct Library *TextFieldBase;

extern struct MessageInfo messageInfo;
struct MessagePartsInfo messagePartsInfo;

// void to indicate empty argument list is optional
void StaticInitMessagePartsInfo(void)
{
	messagePartsInfo.messageInfoPtr = &messageInfo;
}

ULONG FormatTest2(struct Hook *hookPtr, VOID *objectPtr, VOID *messagePtr)
{
	InitMessageInfo(hookPtr, (APTR) objectPtr, (struct HighlightMessage *)messagePtr);

	// if line is longer than 10 chars we paint part of the line with color x
	if (messageInfo.end > 10)
		HighlightSetFormat(objectPtr, 1, 6, (34<<8)|TBSTYLE_SETCOLOR|TBSTYLE_ITALIC);

	return messageInfo.state + 1L;
}

[phx]

Originally Posted by guy lateur

This does make me wonder how the order works for include paths. For instance, if I

#include <proto/texteditor.h>

. This file can be found via

vincludeos3:

(in cc setting in

vc.config

) as well as via

NDK32:Include_H

(passed to vc via the -I switch; see my makefile). So which file gets included in this case?

When you look into your config file you see a line on top for calling the compiler, which looks like this:

-cc=vbccm68k -quiet -hunkdebug %s -o= %s %s -O=%ld -Ivincludeos3:

The first %s is the source file, the second the target file and the third contains any compiler options you pass, like

-I

. So paths, which you specify on the command line, do always have priority. Which is a problem here, because the

proto

directory from the NDK 3.2 has no vbcc support and must not replace vbcc's protos.

Under AmigaOS you should add the NDK path to the

vincludeos3:

assignment, so it is searched after

targets/m68k-amigaos/include

. On other host systems just add a second

-I

at the end of the

-cc

and

-ccv

lines in the config file.

I suppose I should always try to have the vbcc includes take priority over the NDK ones, right?

True.

ULONG __saveds highlightBlock(__reg("a0") struct Hook *hook, __reg("a2") VOID *obj, __reg("a1") VOID *msg)
{
    return FormatTest2(hook, obj, msg);
}

This is correct, right?

Looking good.

So, after all this, I'm still having the error

The error about

HighlightSetFormat

being undefined? This has three reasons:

1. NDK3.2 support is still missing in vbcc. So you have to take the texteditor_lib.sfd (which is not even included in the last NDK release) and generate proto, inline and amiga.lib stub routines from it.
2. vbcc support is missing in NDK3.2. So including the NDK3.2 proto headers doesn't give you any inline calls for the library.
3. The last released NDK3.2 amiga.lib does neither have stub functions for all libraries, nor does it have all FD or SFD files.

   HighlightSetFormat()

   is a V47 function and is definitely missing. So it also doesn't help you to link against the NDK's amiga.lib.

Or is there anything else I could easily try to get this linking properly?

Here is a first test version of the updated m68k-amigaos target:
http://sun.hasenbraten.de/~frank/TES...8k-amigaos.lha

You may try to replace all your files in

targets/m68k-amigaos

with the upated ones. But no guarantee! I didn't have enough time for testing. Do backups!

For your inspiration, here's my current m68k.c file. As mentioned earlier, I included all kinds of header files I thought might be relevant, including the inline one.

Usually neither including

clib

,

inline

nor

pragmas

should be needed as

proto

should take care of that. Also pragmas are for SAS/C and will be ignored for vbcc. The inline header file exists in the old vbcc distribution, but is lacking

HighlightSetFormat()

as a new V47 function.

I also have no idea what those defines near the top mean, btw, so if anybody wants to enlighten me..

__NOLIBBASE__

makes sure that the proto headers do not declare their library base as externally defined. You will have to do it yourself then.

I don't think that

__CLIB_PRAGMA_LIBCALL

has any function. I don't see it in any NDK 3.2 header file. If it exists, it is probably SAS/C-specific.

[guy lateur]

Originally Posted by phx

..

Thanks for elaborating on these various questions, much appreciated!

Originally Posted by phx

Here is a first test version of the updated m68k-amigaos target:
http://sun.hasenbraten.de/~frank/TES...8k-amigaos.lha

You may try to replace all your files in

targets/m68k-amigaos

with the upated ones. But no guarantee! I didn't have enough time for testing. Do backups!

Ok, this does finish the linking process and produces the plugin, but when I then start TextEdit, I get a

Program failed (error #8000000A)

. When I tried some more, I also got a

Program failed (error #80000006)

at some point.

The first time I tried to build the plugin, I also got an error in free, something about a corrupt end of file or something. This was during the compilation process. I only got that error once, and I unfortunately can't remember the exact message. I also cannot seem to reproduce this error, even if I restart the entire build process from clean sources.

Anyway, seems like some more testing will indeed be required. If you want me to try out some things, please let me know; I'll be happy to help wherever I can.

[alkis]

Does the example plugin you linked above compile/link and run ok? (i.e. only your plugin bombs?)

[guy lateur]

Originally Posted by alkis

Does the example plugin you linked above compile/link and run ok? (i.e. only your plugin bombs?)

Both plugins (mine and Camilla's example) behave in exactly the same way:
1) They both compile & link and work ok when using SAS/C
2) They both fail to link (

Error 21: Reference to undefined symbol _HighlightSetFormat

) when using the latest officially released VBCC m68k-amigaos target
3) They both compile & link when using the experimental VBCC m68k-amigaos target release provided by phx here earlier. But when starting TextEdit with those plugins, it crashes immediately (

Program failed (error #80000006)

)

[phx]

Originally Posted by guy lateur

1) They both compile & link and work ok when using SAS/C

Which is surprising, or quite lucky.

As far as I understand the first code executed in both plugins is the

init()

function. And there is an

AllocVec()

before calling

openLibs()

. For vbcc this would guarantee that

AllocVec()

crashes, because

SysBase

is not initialised. Don't know about SAS/C, but there shouldn't be any startup code either.

[guy lateur]

Originally Posted by phx

Which is surprising, or quite lucky.

As far as I understand the first code executed in both plugins is the

init()

function. And there is an

AllocVec()

before calling

openLibs()

. For vbcc this would guarantee that

AllocVec()

crashes, because

SysBase

is not initialised. Don't know about SAS/C, but there shouldn't be any startup code either.

Hey, you're right, I hadn't noticed that. TBH, I just left the code in

example.c

alone as much as possible, and just had it hook into my syntax parsing code (well, stub really, atm).

I can't really explain why it works with SAS/C either; I can only report that it does. If you like, you can easily test this for yourself by downloading the github repo, going into the example drawer, and doing a

smake -f smakefile.debug

. Copy the resulting

exampleplugin

file into

AOS32:Tools/TextEditFileTypes

and start TextEdit.

I've pasted the SAS/C smakefile below. There's a reference to assert.lib in the linking process there. Maybe that does the necessary setup for this to work?

Anyway, we do now have a reason why the plugin will crash. As soon as I have some more time, I will make sure

openLibs()

is called before

AllocVec()

and rebuild the plugin with your experimental amigaos target. I will report back here with that result.

# smake -f smakefile.debug

#
# :ts=8
#
# An example filtype plugin for AmigaOS 3.2 TextEdit
#

.asm.o:
	asm $(AFLAGS) $<

.c.o:
	sc $(CFLAGS) $<
#	@ctags >tagfiles/$* $<

###############################################################################

OPTIMIZE =	optimize opttime
CPU =		any
#DEBUG =		line
DEBUG =	symbolflush noopt

###############################################################################

CFLAGS =	idlen=64 comnest streq strmerge nostkchk \
		$(OPTIMIZE) cpu=$(CPU) debug=$(DEBUG) \
        params=register strsect=code mccons data=far \
		IDIR=NDK32:Include_H
AFLAGS =	-d
LFLAGS =	addsym smallcode noicons batch smalldata

###############################################################################

OBJS = pluginheader.o example.o util.o m68k.o
LIBS = lib:scnb.lib NDK32:lib/amiga.lib NDK32:lib/debug.lib

###############################################################################

all: tagfiles exampleplugin

tagfiles:
    makedir $@

exampleplugin: $(OBJS)
	slink $(OBJS) to $@.debug lib $(LIBS) assert.lib $(LFLAGS) \
		map $@.map,fhx fwidth 32 pwidth 32 swidth 32
	slink $@.debug to $@ noicons nodebug
	@type tagfiles/\#? >t:tags
	@copy t:tags ""
	@delete >nil: t:tags

###############################################################################

###############################################################################

clean:
	-delete \#?.(o|lib) \#?/\#?.(o|lib) DAControl(%|.debug)

realclean: clean
	-delete tags tagfiles \#?.map all

###############################################################################

mkid:
	mkid -v \#?.(c|h)

[guy lateur]

Originally Posted by guy lateur

I will make sure

openLibs()

is called before

AllocVec()

and rebuild the plugin with your experimental amigaos target. I will report back here with that result.

Ok, I did this but it's still not working. In fact, it brings down the whole system now. As soon as I double click the TextEdit icon, everything freezes.

I've attached the current version of the plugin in case anyone wants to have a go at it. Remember to add the NDK includes & libs to the VBCC assigns, as mentioned by phx earlier. So you'll need something like this in your startup sequence:

assign >NIL: vincludeos3: vbcc:targets/m68k-amigaos/include
assign >NIL: vincludeos3: NDK32:Include_H ADD
assign >NIL: vlibos3: vbcc:targets/m68k-amigaos/lib
assign >NIL: vlibos3: NDK32:lib ADD

[phx]

The executable doesn't look too bad on first sight. The plugin header is correct and

init()

should work. In which environment did you test it? You have to copy it into some plugin folder before starting TextEdit?

I don't have OS3.2 myself yet, so I cannot really debug it, but what I would do is either take a good AmigaOS debugger (BDebug, MonAm, etc.) or the UAE-debugger and add a breakpoint at the start of the

init()

function. Is it even executed?

You can do it like this (for UAE, then

fi cf4f

in its debugger):

static void init(APTR userData)
{
    struct Filetype *fType = (struct Filetype *)userData;
    struct Hook *hook;

    __asm("\tdc.w\t$cf4f");
    openLibs();

[alkis]

Originally Posted by guy lateur

Ok, I did this but it's still not working. In fact, it brings down the whole system now. As soon as I double click the TextEdit icon, everything freezes.

I've attached the current version of the plugin in case anyone wants to have a go at it. Remember to add the NDK includes & libs to the VBCC assigns, as mentioned by phx earlier. So you'll need something like this in your startup sequence:

assign >NIL: vincludeos3: vbcc:targets/m68k-amigaos/include
assign >NIL: vincludeos3: NDK32:Include_H ADD
assign >NIL: vlibos3: vbcc:targets/m68k-amigaos/lib
assign >NIL: vlibos3: NDK32:lib ADD

I guess this is the experimental VBCC. Either it has already the 3.2 staff in (in which case 3.2 additional includes/libs are not needed) or you need to reverse the order in includes and libs. (i.e. 3.2 first)

There is another option, that I don't know what I am talking about, but hey

[guy lateur]

Originally Posted by alkis

I guess this is the experimental VBCC. Either it has already the 3.2 staff in (in which case 3.2 additional includes/libs are not needed) or you need to reverse the order in includes and libs. (i.e. 3.2 first)

There is another option, that I don't know what I am talking about, but hey

Well leaving out the NDK includes doesn't work. There are a lot more header files in the NDK than there are in the target include folder. I'm assuming not all those files have to be modified in order to make them work with VBCC, so the target only includes those that do need modfications and falls back to the NDK files for those that are usable as they are.

Changing the order in the startup sequence produces the same result, ie the system freezes as soon as I double click the TextEdit icon. I'm going to revert back to the order that phx told me to use -- what with him being a developer of VBCC and everything, I'd like to think he might at least have a slight idea what he's talking about..

Originally Posted by phx

In which environment did you test it? You have to copy it into some plugin folder before starting TextEdit?

I've been testing this on my Vampire 4 StandAlone installation of OS3.2. And yes, you indeed have to copy the produced plugin binary into a specific folder. When TextEdit starts, it scans this folder for plugins and does some initialization on them. You can read more about that in the readme provided by Camilla, under 'How do the plugins work?' here: https://gitlab.com/boemann/texteditfiletypeplugins

Originally Posted by phx

I don't have OS3.2 myself yet, so I cannot really debug it, but what I would do is either take a good AmigaOS debugger (BDebug, MonAm, etc.) or the UAE-debugger and add a breakpoint at the start of the

init()

function. Is it even executed?

You can do it like this (for UAE, then

fi cf4f

in its debugger):

static void init(APTR userData)
{
    struct Filetype *fType = (struct Filetype *)userData;
    struct Hook *hook;

    __asm("\tdc.w\t$cf4f");
    openLibs();

Ok, I do have Amiga Forever, so I should be able to get this working. It hás been over 30 years since I last installed any version of AmigaOS, though, and I've never (properly) used an Amiga debugger before, so it may take me a while to get this going. But it will be a good thing to have this up and running, if only to have that debugger at hand. I'll report back here when I have this set up.

[alkis]

Originally Posted by guy lateur

Changing the order in the startup sequence produces the same result, ie the system freezes as soon as I double click the TextEdit icon.

Sorry, it was a long long shot. Without having the environment to check against (3.2) it's pretty hard.

Originally Posted by guy lateur

I'm going to revert back to the order that phx told me to use -- what with him being a developer of VBCC and everything, I'd like to think he might at least have a slight idea what he's talking about..

Solid strategy