English Amiga Board


Go Back   English Amiga Board > Coders > Coders. System

 
 
Thread Tools
Old 12 June 2022, 08:44   #1
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Some questions about devices and compilers

I have finally started work in this idea I had for a super simple Amiga 500 SPI controller. The hardware for this is now done so I am turning my attention to the software. The SPI controller has two (shared) ports that I want to use for an SD card and an ENC28J60 ethernet controller.

I was looking at Mike Stirling's excellent K1208 drivers https://github.com/mikestir/k1208-driversand I plan to adapt those for my hardware. It will involve some extra work as I will need some mechanism to arbitrate access to the shared SPI bus and I need to make the ethernet controller a polling one as I have no interrupt but those are things I can handle.

What I cannot handle though is the whole GCC/Linux/Cygwin/make-file/help!!! business though. I found Niklas's SPI parallel port projecthttps://github.com/niklasekstrom/ami...to-spi-adapter and it appears he has the same problem that I have and has thus diverted to VBCC which seems much more straightforward. However, he has also converted the main driver into a threaded one by diverting all IO requests to a helper task.

And here is my first question about that:
Why does Mike's original driver work without a helper task? My understanding is that BeginIO() could be called at any time by any task so in theory BeginIO could be interrupted while handling an SD-card operation? Is it maybe that the OS handles subsequent calls to BeginIO so that it waits until the previous call returns before doing another call?

Question 2:
Mike's driver uses some macro's to generate the romtag structure and jump tables it seems. He also uses the -lamiga-dev GCC option that I assume handles this table generation. However, I cannot find any description of this option anywhere on the net. Jorgen Bilander has published a skeleton driver that generates the table using some inline assembly https://github.com/jbilander/SimpleDevice. That seems more obvious to me. As i can't get GCC to run on my PC and as ADE from aminet (which is GCC 2.95 based!) does not recognize the -lamiga-dev option, can't I use that method instead? But what is the linker option then to disable standard linking? Is it -nostartfiles? And will it work on the Amiga hosted ADE as well?


I know these are many questions but I hope someone can help me here. I was hoping to just change some low-level code and hit "build" but as this is not a standard piece of software and because of the whole compiler trickery this is more complex than I hoped.....
Mathesar is offline  
Old 12 June 2022, 09:37   #2
Thomas Richter
Registered User
 
Join Date: Jan 2019
Location: Germany
Posts: 1,776
Quote:
Originally Posted by Mathesar View Post
Why does Mike's original driver work without a helper task? My understanding is that BeginIO() could be called at any time by any task so in theory BeginIO could be interrupted while handling an SD-card operation? Is it maybe that the OS handles subsequent calls to BeginIO so that it waits until the previous call returns before doing another call?
No, the Os does nothing like this. BeginIO() is called for every SendIO() and DoIO(), both of which are only thin wrappers around BeginIO().



Note that BeginIO() is supposed to be asynchroniously, unless the IOF_QUICK flag is set in which direct execution in the caller task is permitted. There are multiple strategies how it could be done without a helper task. The worst is by simply ignoring IOF_QUICK and always performing the operation synchronously, but this makes the whole point of SendIO() absurd and limits performance. Another one would be to queue up the IORequests in a list and work them off in some other kind of asynchronous operation such as an interrupt.


One way or another, the device task approach is considered the gold standard, so look somewhere else for a suitable example.


What BeginIO() typically does is that it checks the operation requested. If it is something very simple and the IOF_QUICK flag is set, then handle the operation immediately and return. Otherwise, clear the flag, and queue the IORequest up at the message port of the device task, and let it perform the operation. Such "quick" operations should only be those that require little to no time, and do not have to wait for hardware to complete an operation - the whole point of BeginIO() is to allow asynchronous I/O such that the caller *does not* have to wait for the operation to complete.



Concerning using gcc, I afraid I'm not of big help. For such very Amiga specific stuff, I would always recommend SAS/C or assembler stubs.
Thomas Richter is offline  
Old 12 June 2022, 10:58   #3
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Quote:
Originally Posted by Thomas Richter View Post
The worst is by simply ignoring IOF_QUICK and always performing the operation synchronously, but this makes the whole point of SendIO() absurd and limits performance.
Thanks for your reply.
I think this is exactly what the Mike's driver does. As does the A500IDE driver from Mika. But then there should be some mechanism in the driver to prevent recurring calls from another task right? I mean, is it possible that BeginIO() gets called again from another process while it is still processing another request? For example, reading some blocks of data from the SD card takes quite some time. Another, higher priority, process could theoretically pre-empt that and also call BeginIO(). However, i cannot see any mechanism inside the driver to handle that case.
Mathesar is offline  
Old 12 June 2022, 11:03   #4
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Now that I think of it, the SD card driver would be called by DOS right? Maybe that is the reason it works as maybe DOS will make sure BeginIO is not called from multiple processes?
Mathesar is offline  
Old 12 June 2022, 11:47   #5
Thomas Richter
Registered User
 
Join Date: Jan 2019
Location: Germany
Posts: 1,776
BeginIO() can surely be called by multiple processes, but it is then up to the processes to ensure that the data is consistent. So for example, if you have two partitions on an SD card, there will be two processes accessing the SD card simultaneously, one for each partition. They won't interfere because they operate on different parts of the SD card, i.e. address different sectors, but it is up to the device to "sequence" operations, that is, perform one after another.

One way of doing this is to ensure (e.g. by a semaphore) that there can be only one calling task within the critical part of the device driver, but this is very sloppy as it prevents the other task calling from progressing. Instead, you queue up the operations at a work queue (a message port) and then the device task works on them one by one.

This is not a "hypothetical" setting, but happens every time there are more partitions on a device, and any well written file system such as the ones delivered with the Os perform IO asynchroniously, that is, they initiate an operation by SendIO(), and are then free to do something else and react on the user while the device is busy with other queued operations.

Only lazy file systems such as FAT95 "hang" as long as a device is busy performing the IO, but the Amiga is a multitasking operating system, and the device and the file system should take advantage of it and free the task or caller to do something else while the IO is operating.

BTW, there is no "DOS" process in AmigaOs that performs this operation. The "dos" is just a thin "virtual file system" which dispatches commands to file systems, i.e. other processes, to perform the actual operation. This is very close to the way how exec interacts with devices, just that the call is not "DoIO()" or "SendIO()" but "DoPkt()" and "PutMsg". So a file system works very much like a device driver, by very similar principles, but due to some historical accident, it uses a different set of functions for it.
Thomas Richter is offline  
Old 12 June 2022, 14:06   #6
phx
Natteravn

phx's Avatar
 
Join Date: Nov 2009
Location: Herford / Germany
Posts: 2,178
Quote:
Originally Posted by Mathesar View Post
Mike's driver uses some macro's to generate the romtag structure and jump tables it seems.
On first sight I didn't find them in the linked github repository, but you are probably talking about the macros from
exec/initializers.h
? Or did he make his own?

Quote:
He also uses the -lamiga-dev GCC option that I assume handles this table generation.
I doubt that a linker library handles table generation. The -l option specifies a library to link with and I guess that libamiga-dev is a modified version of libamiga?

Quote:
Jorgen Bilander has published a skeleton driver that generates the table using some inline assembly https://github.com/jbilander/SimpleDevice. That seems more obvious to me.
A simple method is certainly to do the library/device init and Romtag stuff in a separate assembler source. But you should be able to do it in C as well. But note that the
OFFSET
macro in
exec/initializers.h
is not really compatible with vbcc and should rather be implemented with
offsetof()
from
<stddef.h>
.
phx is offline  
Old 12 June 2022, 20:17   #7
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Quote:
On first sight I didn't find them in the linked github repository, but you are probably talking about the macros from
exec/initializers.h
? Or did he make his own?
The line is here:
https://github.com/mikestir/k1208-dr.../device.c#L205
To me it suggest this Macro is used to generate the romtag table, but I can't find where the macro comes from. It's not from initializers.h. But yeah, I think I am gonna try my luck with VBCC anyway which can be hosted on Amiga. The seperate assembler file is way more comprehensible to me.
Quote:
I doubt that a linker library handles table generation. The -l option specifies a library to link with and I guess that libamiga-dev is a modified version of libamiga?
Ah yes, that is probably it.
Mathesar is offline  
Old 12 June 2022, 20:49   #8
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Quote:
BeginIO() can surely be called by multiple processes, but it is then up to the processes to ensure that the data is consistent. So for example, if you have two partitions on an SD card, there will be two processes accessing the SD card simultaneously, one for each partition. They won't interfere because they operate on different parts of the SD card, i.e. address different sectors, but it is up to the device to "sequence" operations, that is, perform one after another.

One way of doing this is to ensure (e.g. by a semaphore) that there can be only one calling task within the critical part of the device driver, but this is very sloppy as it prevents the other task calling from progressing. Instead, you queue up the operations at a work queue (a message port) and then the device task works on them one by one.

This is not a "hypothetical" setting, but happens every time there are more partitions on a device, and any well written file system such as the ones delivered with the Os perform IO asynchroniously, that is, they initiate an operation by SendIO(), and are then free to do something else and react on the user while the device is busy with other queued operations.
I am pretty sure that Mike's driver and all it's derivatives like the SDbox driver do not use any semaphores (or any other mechanism) to prevent multiple IO requests from corrupting the data transfer. So, if one would create multiple partitions on the SD using that driver it could go wrong. Niklas's version uses a helper task and properly queues requests to that task so that one would work correctly. I guess most people just create a single FAT partition. However, my controller will be an internal controller to act as an Amiga HDD (and a network interface ) so I will use Niklas's version as a base. Ultimately I would like to make it autobooting as well....
But first things first!


Another quick question: The SPI bus will be shared between the network and SD device. I could make the SPI part a device on it's own but that would make things very complicated as SPI accesses are often very complex. So, the plan is to include the SPI code in each driver but make sure that once a driver asserts the chip select line for it's device (SD for example), the other device (network) will wait untill the chip select lines are all de-asserted (and thus "free") again. I was thinking of using a public semaphore for that. The first driver that comes up will check if the semaphore exists (during device init) and if not create the semaphore and make it public. The second driver will then find that semaphore and not create one of it's own. I think that will work right?


Quote:
BTW, there is no "DOS" process in AmigaOs that performs this operation. The "dos" is just a thin "virtual file system" which dispatches commands to file systems, i.e. other processes, to perform the actual operation. This is very close to the way how exec interacts with devices, just that the call is not "DoIO()" or "SendIO()" but "DoPkt()" and "PutMsg". So a file system works very much like a device driver, by very similar principles, but due to some historical accident, it uses a different set of functions for it.
Thanks for the clarification! Now that you mention it, it probably has to do with the fact that AmigaDOS is actually Tripos right? It has been too long since I have done some Amiga coding...
Mathesar is offline  
Old 13 June 2022, 01:57   #9
alkis
Registered User

 
Join Date: Dec 2010
Location: Athens/Greece
Age: 51
Posts: 670
Quote:
Originally Posted by Mathesar View Post
The line is here:
https://github.com/mikestir/k1208-dr.../device.c#L205
To me it suggest this Macro is used to generate the romtag table, but I can't find where the macro comes from. It's not from initializers.h. But yeah, I think I am gonna try my luck with VBCC anyway which can be hosted on Amiga. The seperate assembler file is way more comprehensible to me.
Ah yes, that is probably it.
The macro comes from stabs.h
In modern m68k-amigaos-gcc it's at m68k-amigaos/sys-include/stabs.h
alkis is offline  
Old 13 June 2022, 11:45   #10
Thomas Richter
Registered User
 
Join Date: Jan 2019
Location: Germany
Posts: 1,776
Quote:
Originally Posted by Mathesar View Post
Another quick question: The SPI bus will be shared between the network and SD device. I could make the SPI part a device on it's own but that would make things very complicated as SPI accesses are often very complex. So, the plan is to include the SPI code in each driver but make sure that once a driver asserts the chip select line for it's device (SD for example), the other device (network) will wait untill the chip select lines are all de-asserted (and thus "free") again. I was thinking of using a public semaphore for that. The first driver that comes up will check if the semaphore exists (during device init) and if not create the semaphore and make it public. The second driver will then find that semaphore and not create one of it's own. I think that will work right?
I would recommend another approach, and use the trackdisk.device as blue-print for a design. The trackdisk device uses the disk.resource to allocate access to the actual hardware, and thus, you could create an SPI.resource for that, which potentially, could also handle the lower-level hardware abstraction and leave the media-specific handling to the device.
At bare minimum, the resource would just provide the semaphore, but from a software design perspective, it's probably good to have an abstraction around SPI available for multiple devices.


Quote:
Originally Posted by Mathesar View Post

Thanks for the clarification! Now that you mention it, it probably has to do with the fact that AmigaDOS is actually Tripos right? It has been too long since I have done some Amiga coding...
Well, that's certainly part of the Tripos design, but it is also part of the exec-Design and its devices. There is some parallelism between exec devices and tripos handlers. If you look into the tripos specs, its low-level device drivers work pretty much like exec devices. Potentially Carl took the exec design also from this document, or added this later to ease porting of Tripos to AmigaDos.
Thomas Richter is offline  
Old 13 June 2022, 13:36   #11
Olaf Barthel
Registered User
 
Join Date: Aug 2010
Location: Germany
Posts: 311
Quote:
Originally Posted by Mathesar View Post
IJorgen Bilander has published a skeleton driver that generates the table using some inline assembly https://github.com/jbilander/SimpleDevice. That seems more obvious to me.
You do not need to resort to inline assembly. The same data structures can be built using 'C' only. However, there are no public 'C' data structures defined in the Amiga operating system header files which would allow you to replace the auto_init_tables data.

As for the simple device code, it indeed is simple to the point of leaving the entire device BeginIO function implementation as an exercise to the reader. This is hard, but very easy to get subtly and fatally wrong

The device initialization function init_device ought to verify that it is running on the operating system version it expects. As written, it will accept anything, including Kickstart 1.0. While this may sound picky, you can save yourself hours of debugging by testing for operating system versions which your driver does not support. Even if it does not crash on your system, somebody else might see her or his machine freeze for no apparent reason.

The device function open would benefit from checking if the I/O request size matches what the device expects. If it happens to be shorter than expected (check the IORequest->io_Message.mn_Length field; it ought to be at least as large as sizeof(struct IORequest)) all kinds of random side-effects may occur, making it much harder to diagnose the problem.

Last edited by Olaf Barthel; 13 June 2022 at 16:04.
Olaf Barthel is offline  
Old Yesterday, 20:58   #12
Mathesar
Registered User

Mathesar's Avatar
 
Join Date: Aug 2014
Location: Netherlands
Posts: 567
Thanks all for the helpful comments so far. A small update:
I have managed to install vbcc on my A3000. It compiles Niklas's version of the driver just fine after adding a missing header define. I now have a development flow going where I edit my code using stormC and then compile with vbcc using a script that automatically copies the compiled binaries to a floppy with which I can boot the A500 for testing.
This way I was able to verify that the hardware indeed works as I hoped it would. For me, that means I have at least 3 milestones done (hardware designed, compiler setup, hardware verified).
The next step is to replace Niklas's SPI code by my own.
I'll keep you posted with my progress.
Mathesar is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
Earlier C++ Compilers Anubis Coders. General 25 23 October 2019 00:48
C Compilers? Pheonix support.Apps 7 05 December 2016 18:06
C++ Compilers where/what to get? Spadger request.Apps 18 05 May 2006 05:10
c compilers? kruwi request.Apps 1 25 April 2006 18:30
Intel Compilers, anyone ? guest support.WinUAE 0 10 December 2002 13:04

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 04:22.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Page generated in 0.07734 seconds with 15 queries