68030 write-allocate mode

[TheDarkCoder]

Hi,

during my holydays I am enjoying a refresch of my knowledge about computer architechtures. While reading about caches, maybe stimulated by the recent Photon statement about poking every bit and byte of the Amiga, I recalled the following.
The 68030 has a "write-allocate" mode (writes in memory cause the written to location to be put in cache). In the ninethies, I tried to activate such a mode and found out that the Amiga did not worked as expected. In particular a blitter-rendered 3D scene was not correctly rendered. I don't remmber exactly the errors, but the rendered image has small but noticable missing pixels. The tests were done on a A4000 with a C= 3630 (030 25Mhz) CPU card.
I think that at that time I also discovered some official C= note saying that the 68030 write-allocate mode was not compatible with the way Amiga bus works.
I apologize for not recalling more details, but basically that was what I know about the subject.

I would like to know a more detailed reason for the impossibility to use this mode on the Amiga, and I wonder if any of the many hardware/low-level gurus in this forum can help.

[Toni Wilen]

Second time today I post this link: http://amigadev.elowar.com/read/ADCD.../node0161.html

Quick explanation: 68030 data cache always caches long aligned writes, even if destination address is supposed to be uncacheable. Following read(s) from same address come from cache, not from memory that may have been modified by some DMA device.

AFAIK this can be only fixed by using MMU to mark required memory regions as uncacheable. (if CPU is non-EC)

[TheDarkCoder]

Originally Posted by Toni Wilen

Second time today I post this link: http://amigadev.elowar.com/read/ADCD.../node0161.html

Quick explanation: 68030 data cache always caches long aligned writes, even if destination address is supposed to be uncacheable. Following read(s) from same address come from cache, not from memory that may have been modified by some DMA device.

AFAIK this can be only fixed by using MMU to mark required memory regions as uncacheable. (if CPU is non-EC)

Thanks for the interesting link, Toni. Your quick answer is not correct (with respect to my question). Indeed, uncorrect has been my memory. If you read carefully the doc you linked, you'll find out that it is the opposite: write-allocate is required for the Amiga to work correctly. It is also written in the C= include file where the cache macros are defined.
I have found several threads in the natami forum where they speak about this but it's not clear if and why it is *required* that Write-Allocate mode is turned on.
More in detail, Thomas Ritcher gives a partial answer:
"Ehem. No. It only works with write allocation enabled. Turning write allocation *off* is even more of a problem. We had a thread on this - reason for this is that the cache is logically indexed, not physically indexed, and that the function codes are part of the cache index. IOW, turning write allocation *off* breaks even more things."
since Thomas sometimes writes here too, I wonder if he can explain things in more detail.

[Shadowfire]

No, I read through that and I have to agree with Tony.

The 68030 data cache has a mode called write-allocate. In this mode,
the 68030 not only updates the data in the cache, but, in the case of
a cache miss, the 68030 can also allocate a new cache entry. While
in this mode, if a data write causes a cache miss, the CPU first
marks the corresponding cache entry as invalid (or, if in burst mode,
the CPU marks the entire cache line as invalid).

While in write-allocate mode, the 68030 will also allocate a cache
entry during certain write operations. If, while in write-allocate
mode, the 68030 writes a long word to a long word aligned address,
the 68030 will write to that address and will allocate a cache entry
for that address. This provides a loophole where the 68030 will
allocate a cache entry for a non-cachable memory address. If the CPU
does a long word write to a Zorro III hardware register that happens
to be aligned on a long word address, the 68030 will put that address
in the cache. If the CPU attempts to read from that address again
and that address happens to still be in the data cache, it will see
the value in the cache and will not attempt to read the hardware
register.

This here is the crux of the matter. Even though the device asserts the /CINH (cache inhibit) line, the 68030 is still caching the write. Caching must be stopped at the MMU level for this to work properly.

[TheDarkCoder]

Originally Posted by Shadowfire

No, I read through that and I have to agree with Tony.





This here is the crux of the matter. Even though the device asserts the /CINH (cache inhibit) line, the 68030 is still caching the write. Caching must be stopped at the MMU level for this to work properly.

No, Shadowfire. It's not that Toni is wrong, he correctly summarized the document, but he does not answer my question. Problem of course is that I remembered the other way around.
What caused my code not to render correctly was not turning write-allocate ON (as is by default in AmigaOS). Instead problems came out turning write-allocate OFF. Why? Which is the problem with write-allocate OFF ?

Please note that the linked document is aimed mainly at hardware developers. It describe the problem caused by the 68030 write-allocate mode (which is of course the one quickly explained by Toni), and the solutions that have to be provided at hardware-level (i.e. putting the 32 bit Zorro registers at non-long-aligned adress, or splitting read/write Zorro registers in a read-only and a write-only pair). Note that it does not suggest the simple solution to turn write-allocate OFF. Instead in says in parenthesis that write-allocate is required by AmigaOS.
I find strange that a cache mode is required by the OS. In any case I would like to know why it is required, just to improve my Amiga knowledge

[Toni Wilen]

Bug in program but it worked accidentally if write-allocate was on? For example incorrect read from write-only custom chipset register that was written previously = read does not go outside of CPU (which would mess up register contents) because it is already in data cache.

Something being buggy but accidentally still working is not that rare.

[TheDarkCoder]

Originally Posted by Toni Wilen

Bug in program but it worked accidentally if write-allocate was on? For example incorrect read from write-only custom chipset register that was written previously = read does not go outside of CPU (which would mess up register contents) because it is already in data cache.

Something being buggy but accidentally still working is not that rare.

maybe, I'll have to recover the source and inspect it.
From your answer I deduce that according to your (huge) knowledge it is possibile for the Amiga to work with write-allocate OFF on a 68030.
Still don't know why, according to Mike Sinz AmigaOS "requires" write-allocate ON (the word require should imply that AmigaOS does not work without write-allocate, not just that it is less efficient)

[TheDarkCoder]

I am still digging the issue, more exactly I am trying to understand why AmigaOS REQUIRES write-allocate enabled.
I have found several confirms of this fact googeling the newsgroup comp.sys.amiga.hardware.
I have found the following affirmation from Dave Haynie, which I don't fully understand but it is related to the issue:

"Actually, it pretty much does work as well under AmigaOS as UNIX, except for
a few caveats. AmigaOS doesn't support I=D anymore than UNIX, though it's
possible to write code in AmigaOS that sets I=D (of course, you CAN do just
about anything in AmigaOS, since there's no MMU protection, but that doesn't
make it proper). The super=user issue isn't an I-Cache issue, though it is
in the 68030's data cache. The '030's Write Allocate mode is designed to
handle this, and it does, but unfortunately this hits upon what we consider
a bug in the data cache mechanism: a longword write to a longword port gets
cached, regardless of the hardware caching mode, and will hit when read.
That's not true with write allocate off, or with caching managed by the MMU.
So A3000's, the only C= sponsored Amigas with 68030s and uncachable longword
ports, need the MMU around to work real well (the MMU isn't the only solution,
but it is the most efficient). "

Moreover, Ralf Babel claims, in answering a question from Christian Stiens, that in its book the question is explained:
"
Christian Stiens wrote:
> why must this bit be set in the Amiga?


Explained in detail in section 2.7.6.3 ("MC68030 and write-
allocation") of the Amiga Guru Book. (Pardon the plug.) :-)
"

I have a copy in german of the Amiga Guru Book, but I don't understand german

Is there anyone who can read the book and summarize to me what is in this section?

... I so much would like an english language copy of the Amiga Guru Book, second edition...

[Jherek Carnelia]

For what it's worth here's the section:

"2.7.6.3 MC68030 and write-allocation

The MC68030's data cache stores the function-code bits along with parts of the address as tag information for each cache entry. The Amiga OS, however, uses the same logical address space for user- and for supervisor- mode accesses, so changing a data cacheable location from supervisor mode must invalidate a possibly corresponding user-mode entry in the data cache and vice versa. But since a write access with different function codes is not considered a cache hit, without write-allocation mode the entry is not replaced. If the write-allocation bit is set, however, then the corresponding cache line is invalidated and possibly updated. Although this new entry is not considered a cache hit either when read from a different mode (e.g supervisor-mode entry read from user mode), it will at least force a reread from memory instead of using the (unreplaced) stale data cache entry, as would be the case without write-allocation.

Enabling write-allocation has an undesired side effect, though: as the NOT CIIN signal is ignored during write-accesses, a longword-aligned write of a longword will create a valid cache entry that may cause a cache hit during subsequent read accesses, regardless of the caching mode for that memory location as indicated by the external hardware. This behaviour can be corrected only by using the MMU to specify caching modes on a page-by-page basis, e.g. by disabling data caching in Chip memory completely (code is not supposed to be located in Chip memory anyhow if Fast memory is available) or by differentiating between code and data accesses using the function-code bits, which - however - causes more frequent misses in the ATC (address translation cache). Drivers and similar system software relying on longword I/O can avoid this problem by using nonaligned transfers or different (e.g. mirrored) addresses for read and write accesses. Refer to [25, sections 5.7.1 and 6.1.2.1] for all the gory details."

[TheDarkCoder]

10x Jherek!

The following explains which is the problem when WA is OFF

Originally Posted by Jherek Carnelia

For what it's worth here's the section:

"2.7.6.3 MC68030 and write-allocation

The MC68030's data cache stores the function-code bits along with parts of the address as tag information for each cache entry. The Amiga OS, however, uses the same logical address space for user- and for supervisor- mode accesses, so changing a data cacheable location from supervisor mode must invalidate a possibly corresponding user-mode entry in the data cache and vice versa. But since a write access with different function codes is not considered a cache hit, without write-allocation mode the entry is not replaced. If the write-allocation bit is set, however, then the corresponding cache line is invalidated and possibly updated. Although this new entry is not considered a cache hit either when read from a different mode (e.g supervisor-mode entry read from user mode), it will at least force a reread from memory instead of using the (unreplaced) stale data cache entry, as would be the case without write-allocation.

There is just one missing detail in the previous explaination: why does turning WA on ensures that the write access invalidates the cache entry allocated by the read at the same address but in a different address space?
Answer: because 030 caches are direct mapped so even if address spaces are different, equal phisical addresses are mapped into the same cache entry.
I am almost sure that my code suffered from the same problem

The following passage expains the problem caused when WA is ON, i.e. the problem already explained by Toni at the beggining of the thread.

Originally Posted by Jherek Carnelia

Enabling write-allocation has an undesired side effect, though: as the NOT CIIN signal is ignored during write-accesses, a longword-aligned write of a longword will create a valid cache entry that may cause a cache hit during subsequent read accesses, regardless of the caching mode for that memory location as indicated by the external hardware. This behaviour can be corrected only by using the MMU to specify caching modes on a page-by-page basis, e.g. by disabling data caching in Chip memory completely (code is not supposed to be located in Chip memory anyhow if Fast memory is available) or by differentiating between code and data accesses using the function-code bits, which - however - causes more frequent misses in the ATC (address translation cache). Drivers and similar system software relying on longword I/O can avoid this problem by using nonaligned transfers or different (e.g. mirrored) addresses for read and write accesses. Refer to [25, sections 5.7.1 and 6.1.2.1] for all the gory details."