testrunner-68k: a test runner for 680x0 code

[Kalms]

Here is my latest hobby project: a test runner for 680x0 code.

This is like a test runner for high level languages, except that it comes with a built-in 68000 emulator. The primary intention behind this tool is to get a better dev-test cycle when coding at home, but should also allow for CI test coverage for any projects I put onto GitHub in the future.

Let me know if you find it useful.

(Current limitations: 68000 only, no AmigaOS support, no Amiga-hardware emulation, no integration to IDEs, C code may not work.)

[phx]

Interesting. Sounds a little bit like Vamos, which can execute 68k with an emulator in your shell. Could be a useful tool for automated tests in a Makefile or quick tests in your shell.

It only could have been more portable, by using a different language. How about C? But I guess you absolutely wanted to do something with Rust?

[Locutus]

Rust is perfectly portable to any modern development host.


Also, suggestion: if you want to use this for CI integration, take a look if some bare bones xUnit generation is feasible or worth the effort to you.

[Kalms]

I'm using Rust for this for learning purposes + it produces small, self-contained executables.

I would like to get some sort of OS call support in there. Vamos looks nice, except that it is in Python and is difficult to distribute (well, it needs the Python runtime - or a pre-packaged version of python+vamos).

I'd like to export results to a standard format like xUnit in the future. To motivate that I'd first like to see a CI platform or 68k-compatible IDE that supports displaying the results.

[Kalms]

@phx On the topic of CI and tests...

It is problematic for me to use vasm & vlink in automated tests, due to how vasm/vlink are distributed and their licenses.

In order to use vasm/vlink in a CI system, my basic need is to be able to fetch & install either the latest-stable or a fixed version with a few shell commands on a Linux host. The current official distribution model ("go to sun.hasenbraten.de/vasm & /vlink, and click on the appropriate links, depending on your host system") does not satisfy those requirements;

I could take a snapshot of the vasm/vlink source code, and include it within every single repository/project. That way, I can build vasm/vlink from source and then use those tools to build the remainder of the project. This requires me to do a bunch of manual work if I want to update to newer vasm/vlink versions, multiplied by number of projects. The "normal" way to handle this for other opensource-or-sort-of-opensource projects is to offer the source code on a hosting platform like GitHub, which offers standardized URLs for downloading either a fixed or latest-stable version. They also take the hosting costs.

I could also build vasm & vlink and put it into the repository itself. That will work, but, updating to newer versions of vasm/vlink will be a manual process, and if I want to support running the tests under multiple different OSes (when testing locally) it multiplies the effort. All this work needs to be repeated for every single repository/project. The "normal" way of handling this for other opensource-or-sort-of-opensource projects is to offer download via apt-get / brew / chocolatey, some of which there are hosting solutions that cover the hosting costs, some of which that don't.

I could also build vasm / vlink into a Docker image. This is similar to "build yourself from source" but building vasm/vlink only once. To minimize efforts I'd like to automate the building of the Docker image whenever there is new vasm/vlink release. That ties back to the problem of how to fetch source code again. The Docker distribution format is also less flexible than the two other (CI systems want all tools in a single Docker image, and there is no standard way to merge two Docker images - so if I need more than just vasm/vlink I need to include those tools at well during image build time).

Do you have an opinion on how you would like to see this work in the future?




I am also not sure whether I am allowed to distribute home-built vasm & vbcc binaries myself according to the licenses.

Taken from the vasm manual:

"vasm is copyright in 2002-2019 by Volker Barthelmann.
This archive may be redistributed without modifications and used for non-commercial purposes.
An exception for commercial usage is granted, provided that the target CPU is M68k and
the target OS is AmigaOS. Resulting binaries may be distributed commercially without
further licensing.
In all other cases you need my written consent.
Certain modules may fall under additional copyrights."

Taken from the vlink manual:

"vlink is copyright 1995-2019 by Frank Wille.
This archive may be redistributed without modifications and used for non-commercial purposes.
An exception for commercial usage is granted, provided that the target OS is AmigaOS/68k.
Resulting binaries may be distributed commercially without further licensing.
In all other cases you need my written consent."

I am allowed to redistribute the vasm & vlink source code, without modifications. However, what does 'resulting binaries' refer to? is that a reference to vasm & vlink binaries, or to applications created using vasm & vlink?

[phx]

Originally Posted by Kalms

In order to use vasm/vlink in a CI system, my basic need is to be able to fetch & install either the latest-stable or a fixed version with a few shell commands on a Linux host.

Like wget? Sure. That's possible.

Do you have an opinion on how you would like to see this work in the future?

Since four years all tagged releases are stored with a defined name under https://server.owl.de/~frank/tags/.
There is also the menu item "Tagged source archives" on the vasm home page, although it didn't work over the last weeks, because the administrator of server.owl.de started to enforce SSL connections. I still have to recompile a new PHP with SSL enabled on sun.hasenbraten.de. Nevertheless, the tagged source archives will always be there!

I am also not sure whether I am allowed to distribute home-built vasm & vbcc binaries myself

Certainly you are, as long as you didn't modify them.

I am allowed to redistribute the vasm & vlink source code, without modifications. However, what does 'resulting binaries' refer to? is that a reference to vasm & vlink binaries, or to applications created using vasm & vlink?

I hate this licensing stuff and usually try to stay away from it. The new vasm and vlink licenses are the result of a deal between Volker and Individual Computers. Individual Computers generously sponsored us to make sure that all programs created with vbcc, vasm and vlink for the AmigaOS/68k platform may also be distributed commercially - as a gift to the community. The original license only allowed to create free software with it, as long as there was no written consent from us which permits that.
So your last statement is true: the license is refering to applications created by these development tools.

[Kalms]

Originally Posted by phx

Like wget? Sure. That's possible.

Since four years all tagged releases are stored with a defined name under https://server.owl.de/~frank/tags/.
There is also the menu item "Tagged source archives" on the vasm home page, although it didn't work over the last weeks, because the administrator of server.owl.de started to enforce SSL connections. I still have to recompile a new PHP with SSL enabled on sun.hasenbraten.de. Nevertheless, the tagged source archives will always be there!

Thanks, good to know. I wasn't sure whether those URLs were guaranteed to be persistent, and whether you would be happy with CI systems downloading one of those packages every time a new build starts.

Originally Posted by phx

Certainly you are, as long as you didn't modify them.

Ok. Again, good to know, as the license does not state this: it only states that I am allowed to distribute the source archive (if unmodified).

Originally Posted by phx

I hate this licensing stuff and usually try to stay away from it. The new vasm and vlink licenses are the result of a deal between Volker and Individual Computers. Individual Computers generously sponsored us to make sure that all programs created with vbcc, vasm and vlink for the AmigaOS/68k platform may also be distributed commercially - as a gift to the community. The original license only allowed to create free software with it, as long as there was no written consent from us which permits that.
So your last statement is true: the license is refering to applications created by these development tools.

Noted, thanks.

[phx]

Originally Posted by Kalms

and whether you would be happy with CI systems downloading one of those packages every time a new build starts.

No problem. This server is running at a big provider in Berlin and has lots of bandwidth.

Ok. Again, good to know, as the license does not state this: it only states that I am allowed to distribute the source archive (if unmodified).

I agree that the license could be more precise. Will talk with Volker about it.

[Kalms]

Originally Posted by Locutus

... suggestion: if you want to use this for CI integration, take a look if some bare bones xUnit generation is feasible or worth the effort to you.

Good idea. JUnit XML output is now working, and CircleCI can display the results. It is not much more than a pass/fail list with core dump info atm - in the future I hope to include source file:line information and callstacks.

[Kalms]

Originally Posted by phx

No problem. This server is running at a big provider in Berlin and has lots of bandwidth.

I have gone with the 3rd option; first I build a Docker image with make/vasm/vlink/testrunner-68k, and then I use that image for any projects that need tests run during CI. Works alright, and it avoids hardcoding download URLs for vasm/vlink into every project.