VBCC: building a TextEdit syntax plugin for OS3.2

[guy lateur]

Originally Posted by alkis

Sorry, it was a long long shot. Without having the environment to check against (3.2) it's pretty hard.

There's absolutely no need to apologize -- au contraire, thanks for thinking along! It's hard enough to debug stuff as it is, let alone if you don't have the targeted environment.

And anyway, it was worth a shot. It's not really obvious exactly what happens when you have multiple include paths that may lead to the same files, so I'm glad we got this sorted out.

[phx]

Originally Posted by guy lateur

There are a lot more header files in the NDK than there are in the target include folder. I'm assuming not all those files have to be modified in order to make them work with VBCC, so the target only includes those that do need modfications and falls back to the NDK files for those that are usable as they are.

The NDK will even work as it is, when you use

clib

instead of

proto

. Which results in linking stub routines from

amiga.lib

for all OS calls.
The vbcc target headers only replace the

proto

directory from the NDK, to be able to use its own assembler inline code for OS calls (which are a little bit faster than stub routines).

You can read more about that in the readme provided by Camilla, under 'How do the plugins work?' here: https://gitlab.com/boemann/texteditfiletypeplugins

She writes that you have to use the

far

option with SAS/C and not

faronly

. I don't know what the difference is. Does it mean A4 must not be modified? vbcc will always save and restore A4, but on intermediate callbacks into TextEditor the register may have a different value. Not sure if such calls even exist.

and I've never (properly) used an Amiga debugger before

Ok, that might be a problem here. You need a good understanding of your debugger and also m68k assembly to follow my recommendation.

An alternative, which takes a lot more time, is to insert code which tells you that it was executed. But it shouldn't be as complex as a

printf()

, because such a call has many dependencies which also could cause a crash. I would just replace the

dc.w $cf4f

by a

trap #0

. So you will immediately know that the instruction has been executed when a software failure

#80000020

appears. Then you can move the location of this breakpoint until your other crash happens, which will narrow it down in your code.

You should also make your plugin as simple as possible. The

init()

function shouldn't do much more than initialise the

highlighterHook

and leave everything else

NULL

. The hook itself should do nothing. When this works, you can slowly start adding more code to it.

[alkis]

Originally Posted by phx

She writes that you have to use the

far

option with SAS/C and not

faronly

. I don't know what the difference is. Does it mean A4 must not be modified? vbcc will always save and restore A4, but on intermediate callbacks into TextEditor the register may have a different value. Not sure if such calls even exist.

ULONG __saveds __asm highlightBlock(register __a0 struct Hook *h,register __a2 VOID *o, register __a1 VOID *msg)

__saveds is supposed to restore the value of a4 when this is called from hooks.

What confuses me is that she writes

It is important to build with data=far
not data=near and not data=faronly
"near" will make the compiler use a4 as base, which will mess up things
"faronly" will free a4 for other use by the compiler which will also mess up things

I don't understand what a4 is used for. (She doesn't want the faronly option cause that would 'free' a4)

[boemann]

First of all great that you are trying to build a better highlighter for m68k. Much approved.

Secondly apologies for not discovering this thread and answering sooner.

Originally Posted by phx

Which is surprising, or quite lucky.

As far as I understand the first code executed in both plugins is the

init()

function. And there is an

AllocVec()

before calling

openLibs()

. For vbcc this would guarantee that

AllocVec()

crashes, because

SysBase

is not initialised. Don't know about SAS/C, but there shouldn't be any startup code either.

AllocVec is an Exec function so if I can call OpenLibrary to open the libraries I can also call AllocVec so this doesn't matter or shouldn't at least as it depends how those functions are actually called

[boemann]

Originally Posted by alkis

ULONG __saveds __asm highlightBlock(register __a0 struct Hook *h,register __a2 VOID *o, register __a1 VOID *msg)

__saveds is supposed to restore the value of a4 when this is called from hooks.

What confuses me is that she writes


I don't understand what a4 is used for. (She doesn't want the faronly option cause that would 'free' a4)

far means don't use a4 but also don't use a4 for anything else
faronly means don't use a4 but you are free to use it as any other register

We need it to be build with "far" so we don't trash a4 in the plugin

[alkis]

Originally Posted by boemann

far means don't use a4 but also don't use a4 for anything else
faronly means don't use a4 but you are free to use it as any other register

We need it to be build with "far" so we don't trash a4 in the plugin

A4 and all other registers besides a0-a1 and d0-d1 are preserved in all C routines.
A4 is not used as a base register.
highlightBlock isn't even referencing (or I missed it?) any variables external to highlightBlock in the example.c. (but even if it did a4 is not base register)

so...what magic does a4 hold in its value?

[alkis]

Originally Posted by boemann

First of all great that you are trying to build a better highlighter for m68k. Much approved.

Secondly apologies for not discovering this thread and answering sooner.




AllocVec is an Exec function so if I can call OpenLibrary to open the libraries I can also call AllocVec so this doesn't matter or shouldn't at least as it depends how those functions are actually called

Shouldn't init start with something like:

struct ExecBase *SysBase = (*((struct ExecBase **) 4));

then? SAS/C nostartup example3 starts like that, which makes sense for code that doesn't use any startup.

[alkis]

https://gitlab.com/boemann/texteditf...mple/SCOPTIONS
doesn't contain NOSTARTUP.

Now I am REALLY confused.

[boemann]

Originally Posted by alkis

https://gitlab.com/boemann/texteditf...mple/SCOPTIONS
doesn't contain NOSTARTUP.

Now I am REALLY confused.

The sas/c pragmas I use make it use 4 (or does it - i'm not sure anymore)

But yeah

Feel free to make a pullrequest for this and any other improvement to make it work with more compilers

[guy lateur]

Originally Posted by boemann

First of all great that you are trying to build a better highlighter for m68k. Much approved.

Well let me start with a little disclaimer here. I'm pretty new to Amiga programming, whether it be in C or in ASM. I especially still have a lot to learn about how the system and the hardware actually work. Also, I'm far from an expert in M68k ASM, and it's been about a quarter of a century since I last programmed anything in C. So let's just hope I didn't bite off more than I can chew here.

My aim is to have the syntax highlighting look something like the C syntax highlighting in TuiTED, or like M68k does in Sublime Text (on Windows, using an M68k plugin). So I'll be differentiating between labels, instructions & their qualifiers, operands and comments to begin with. I might also highlight the registers and literal values in a second phase, and maybe also the preprocessor directives at some point. But once I get this plugin actually working properly, I'll probably start another thread about the actual implementation of the highlighting.

Originally Posted by boemann

Secondly apologies for not discovering this thread and answering sooner.

No need to apologize for that, I'm just glad to you're here now. Better late than never, eh.

I probably shouldn't speak before my turn, but it doesn't look like that far/A4 thing will be the cause for the problems. So if you can think of anything else that might be causing issues with this plugin, please let us know. Any other caveats we should to be aware of when compiling/linking this?

Originally Posted by boemann

Feel free to make a pullrequest for this and any other improvement to make it work with more compilers

Well as suggested by phx I will first focus on getting your basic example working using VBCC. I will be testing this in WinUAE and use its debugger to try and narrow down where the problem occurs. Once I have this working, I'll gladly share my code & VBCC settings, so you can include it in your github repo. Not sure about pull requests and all that; I somehow always end up in trouble when trying to do things using git. But I'm getting ahead of myself; we'll solve that problem when it arises.

[boemann]

Okay sounds like very good goal you have there. I'm sure you will get there

The a4 thing I think is the culprit, as I had the exact same symptoms when trying to create the example and not using far but faronly. The highlighters in the OS use another buildsystem so I had to go recreate the example almost from scratch, which is why i stumbled into the problem.

Also the API is not fixed so if we figure out that I need to make changes to make it work better with VBCC et al I'm all for it. The part about the hook and calling HighlightSetFormat is fixed though and the idea of the way the GUI is used is fixed too, so don't worry it will not be big changes, from a coding point of view if we do decide to change things a bit.

As for contributing to that example code, yes let's wait until you are ready but please make my life as easy as possible. My time is better spent improving the OS itself

[boemann]

Have you tried to not do anything in the init function except setting the names. So don't set up any callback og highlighter hook and don't call any functions

It should be safe as they are all set to a safe NULL when init is called

at least this should mean it will appear in the textedit menu

[guy lateur]

Originally Posted by boemann

Have you tried to not do anything in the init function except setting the names. So don't set up any callback og highlighter hook and don't call any functions

It should be safe as they are all set to a safe NULL when init is called

at least this should mean it will appear in the textedit menu

I haven't, but it will definitely be the first thing I try! Phx suggested something very similar, ie to make the plugin as simple as possible. Once it appears in TextEdit, I can begin adding stuff back in until it breaks, thus narrowing down where the problem occurs. I hope to get some work done on it later today.

[guy lateur]

Originally Posted by boemann

Also the API is not fixed so if we figure out that I need to make changes to make it work better with VBCC et al I'm all for it.

That sounds great! This is stuff that's way above my head, though, so that'll be up to you and phx to decide if anything can be done there. But it's nice to hear you're open for these kinds of changes.

Originally Posted by boemann

As for contributing to that example code, yes let's wait until you are ready but please make my life as easy as possible. My time is better spent improving the OS itself

Well I'll do my best. TBH I was just planning to have an example working in VBCC, next to your SAS/C example, so people at least have a choice which framework they're going to develop under. We can always look into merging those into one later. I doubt I'm the ideal person to do this kind of merging operation, TBH, but we'll see. First we'll need something worth merging.

[phx]

Originally Posted by boemann

AllocVec is an Exec function so if I can call OpenLibrary to open the libraries I can also call AllocVec so this doesn't matter or shouldn't at least as it depends how those functions are actually called

True. But without any startup code setting up

SysBase

I wouldn't expect you to be able to call either of them. In your example

SysBase

is initialised in

openLibs()

, which would be after

AllocVec()

.

Originally Posted by alkis

https://gitlab.com/boemann/texteditfiletypeplugins/-/blob/m
aster/example/SCOPTIONS
doesn't contain NOSTARTUP.

Now I am REALLY confused.

So this example plugin has an SAS/C startup code? This could explain why

SysBase

is already valid, but the Plugin Header structure will no long be at the beginning of the executable. Maybe that doesn't matter?
But: would a startup-code even by executed? TextEdit probably looks for the Plugin Header and executes the

init()

function from there. Nothing more.

Originally Posted by boemann

The sas/c pragmas I use make it use 4 (or does it - i'm not sure anymore)

Yes, that would be an explanation! Maybe the inline code, generated by pragmas for exec.library calls, load the base from

(struct ExecBase *)4

directly, instead of using SysBase? This would make all exec calls unnecessarily slow, though.

This morning I got a mail from somebody who seems to know some SAS/C internals and he wrote something that I would translate into: SAS/C (probably the linker) automatically generates a reference to address 4, when a reference to

_SysBase

cannot be resolved. Therefore you don't need to define it anywhere.
He also wrote that you can define

__USE_SYSBASE

before including the exec-pragmas to force using

SysBase

instead of

4

in the pragmas. Which is more in line with what you said.

Anyway, this seems to be a quite compiler-specific feature and should better be avoided in public example code. Just move

AllocVec()

behind

openLibs()

and you are clean.

Originally Posted by alkis

A4 and all other registers besides a0-a1 and d0-d1 are preserved in all C routines.
A4 is not used as a base register.
highlightBlock isn't even referencing (or I missed it?) any variables external to highlightBlock in the example.c. (but even if it did a4 is not base register)

so...what magic does a4 hold in its value?

I'm asking myself the same question. Most likely TextEdit is compiled for the small data model (

near

)?

Originally Posted by boemann

The a4 thing I think is the culprit, as I had the exact same symptoms when trying to create the example and not using far but faronly. The highlighters in the OS use another buildsystem so I had to go recreate the example almost from scratch, which is why i stumbled into the problem.

I'm still brooding and try to imagine a situation where this will be a problem. When TextEdit calls a function from the plugin all non-volatile registers will be saved and restored on exit. So A4 is never modified from its perspective. The only dangerous situation would be when the hook calls a function from within the TextEdit context in return, while it has already used A4, and TextEdit relies on A4 being unmodified when one of its functions is called. But then such an "exported" function should be protected by

__saveds

in the TextEdit source.

Requiring that all plugin authors don't touch A4 would restrict them to use SAS/C. I'm not sure if any other compiler has a

far

option which takes A4 away from the code generator.

[boemann]

Originally Posted by phx

Yes, that would be an explanation! Maybe the inline code, generated by pragmas for exec.library calls, load the base from

(struct ExecBase *)4

directly, instead of using SysBase? This would make all exec calls unnecessarily slow, though.

This morning I got a mail from somebody who seems to know some SAS/C internals and he wrote something that I would translate into: SAS/C (probably the linker) automatically generates a reference to address 4, when a reference to

_SysBase

cannot be resolved. Therefore you don't need to define it anywhere.
He also wrote that you can define

__USE_SYSBASE

before including the exec-pragmas to force using

SysBase

instead of

4

in the pragmas. Which is more in line with what you said.

Anyway, this seems to be a quite compiler-specific feature and should better be avoided in public example code. Just move

AllocVec()

behind

openLibs()

and you are clean.

Yes that is it, and we do have two set of proto includes for exec, however since the legacy doesn't use sysbase but 4 we didn't change it by default. Some old code may depend on it.

But I fully agree new code shouldn't and my example in particular shouldn't expose such bad practice

[boemann]

Originally Posted by phx

I'm asking myself the same question. Most likely TextEdit is compiled for the small data model (

near

)?


I'm still brooding and try to imagine a situation where this will be a problem. When TextEdit calls a function from the plugin all non-volatile registers will be saved and restored on exit. So A4 is never modified from its perspective. The only dangerous situation would be when the hook calls a function from within the TextEdit context in return, while it has already used A4, and TextEdit relies on A4 being unmodified when one of its functions is called. But then such an "exported" function should be protected by

__saveds

in the TextEdit source.

Requiring that all plugin authors don't touch A4 would restrict them to use SAS/C. I'm not sure if any other compiler has a

far

option which takes A4 away from the code generator.

Yes the released version of TextEdit is built with near model - I've since changed it but too late for the release but it will be fixed in 3.2.1

[alkis]

__saveds will mess A4 in the TextEdit near model context. __saveds will rebase A4 on the plugin's variables.

Some weird workaround could be to have a dummy register a4 parameter on the hook function highlightBlock

[boemann]

Originally Posted by alkis

__saveds will mess A4 in the TextEdit near model context. __saveds will rebase A4 on the plugin's variables.

Some weird workaround could be to have a dummy register a4 parameter on the hook function highlightBlock

Could you elaborate about your insight into saveds

[alkis]

Originally Posted by boemann

Could you elaborate about your insight into saveds

__saveds makes C compiler call __geta4() or something similar (see http://franke.ms/cex/z/7736WY for gcc)

That makes a4 having a value than can address 64K data for the current project.

My understanding from your previous post is that TextEdit needs A4 to point to TextEdit's variables.

If you do __saveds in another C program (this plugin) you restore A4 for that program and not TextEdit.

There are two different programs __saveds will restore a4 to two different values. Plugin's functions' __saveds with set A4 to say value1 and __saveds functions in TextEdit will set A4 to value2.