English Amiga Board


Go Back   English Amiga Board > News

 
 
Thread Tools
Old 08 September 2019, 20:02   #1
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
UAE CPU core based 680x0 tester/validator

UAE CPU Tester

http://www.winuae.net/files/b/cputester.7z (Windows generator executable + Amiga m68k tester executable. Sources in WinUAE github)

I finally wrote utility (This was my "Summer 2019" project) that can be used to verify operation of for example software emulated or FPGA 680x0 CPUs.
It is based on UAE CPU core (gencpu generated special test core). All the CPU logic comes from UAE CPU core.

[More detailed information in WinUAE forum: http://eab.abime.net/showthread.php?t=98746]

Verifies:

- All CPU registers (D0-D7/A0-A7, PC and SR/CCR)
- All FPU registers (FP0-FP7, FPIAR, FPCR, FPSR)
- Generated exception and stack frame contents (if any)
- Memory writes, including stack modifications (if any)
- Loop mode for JIT testing. (generates <test instruction>, dbf dn,loop)
- Supports 68000, 68020, 68030 (only difference between 020 and 030 seems to be data cache and MMU), 68040 and 68060. (I don't currently have real 68010)

Tests executed for each tested instruction:

- Every CCR combination (32 tests)
- Every FPU condition combination (4 bits) + 2 precision bits + 2 rounding bits (256 tests)
- Every addressing mode, including optionally 68020+ addressing modes.
- If instruction generated privilege violation exception, extra test round is run in supervisor mode (Total 64 tests).
- Optionally can do any combination of T0, T1, S and M -bit SR register extra test rounds.
- Every opcode value is tested. Total number of tests per opcode depends on available addressing modes etc. It can be hundreds of thousands or even millions..

Test generation details:

Instruction's effective address is randomized. It is accepted if it points to any of 3 test memory regions. If it points outside of test memory, it will be re-randomized few times. Test will be skipped if current EA makes it impossible to point to any of 3 test regions.
If 68000/68010 and address error testing is enabled: 2 extra test rounds are generated, one with even and another with odd EAs to test and verify address errors.

Notes and limitations:

- Test generator is very brute force based, it should be more intelligent..
- Address error testing is optional, if disabled, generated tests never cause address errors.
- RTE test only tests stack frame types 0 and 2 (if 68020+)
- All tests that would halt or reset the CPU are skipped (RESET in supervisor mode, STOP parameter that would stop the CPU etc)
- Single instruction test set will take long time to run on real 68000. Few minutes to much longer...
- Undefined flags (for example DIV and CHK) are also verified. It probably would be good idea to optionally filter them out.
- Instruction cycle order or timing is ignored. It is not possible without extra hardware.
- 68000 bus errors are not tested but there are some plans for future version with custom hardware. (Hatari also uses UAE CPU core)
- FPU testing is not yet fully implemented.
- Sometimes reported old and new condition code state does not match error report..

Tester compatibility (integer instructions only):

68000: Complete. (Bus errors are not yet verified)
68010: Not supported yet (Don't have real 68010, at least not yet).
68020: Almost complete (DIVS.W/DIVS.L V-flag weirdness).
68030: Same as 68020.
68040: Almost complete (Weird unaligned MOVE16 behavior which may be board specific).
68060: Same as 68040.

Not implemented or only partially implemented:

68010+:

- MOVEC: Only MOVEC to An/Dn is tested and read value is ignored. Basically only verifies if correct and only correct CPU model specific control registers exist.
- RTE: long frames with undefined fields are skipped. Basic frame types 0 and 2 are verified, also unsupported frame types are tested, error is reported if CPU does not generate frame exception.
- 68020+ undefined addressing mode bit combinations are not tested.

All models:

- Interrupts (stack frames and STOP)
- MMU instructions (Not going to happen)
- 68020+ cache related instructions.
- FPU FSAVE/FRESTORE, FPU support also isn't fully implemented yet.

Build instructions:

- buildm68k first (already built if UAE core was previously compiled)
- gencpu with CPU_TEST=1 define. This creates cpuemu_x_test.cpp files (x=90-94), cpustbl_test.cpp and cputbl_test.h
- build cputestgen project.
- build native Amiga project (cputest directory). Assembly files probably only compiles with Bebbo's GCC.


Test generator quick instructions:

Update cputestgen.ini to match your CPU model, memory settings etc...

"Low memory" = memory accessible using absolute word addressing mode, positive value (0x0000 to 0x7fff). Can be larger.
"High memory" = memory accessible using absolute word addressing mode, negative value (0xFFF8000 to 0xFFFFFFFF)

If high memory is ROM space (like on 24-bit address space Amigas), memory region is used for read only tests, use "high_rom=<path to rom image>" to select ROM image. Last 32k of image is loaded.

Use "test_low_memory_start"/"test_high_memory_start" and "test_low_memory_end"/"test_high_memory_end" to restrict range of memory region used for tests, for example if part of region is normally inaccessible.

"test_memory_start"/"test_memory_size" is the main test memory, tested instruction and stack is located here. Must be at least 128k but larger the size, the easier it is for the generator to find effective addresses that hit test memory. This memory space must be free on test target m68k hardware.

All 3 memory regions (if RAM) are filled with pseudo-random pattern and saved as "lmem.dat", "hmem.dat" and "tmem.dat"

Usage of Amiga m68k native test program:

Copy all memory dat files, test executable compiled for target platform (currently only Amiga is supported) and data/<cpu model> contents to target system, keeping original directory structure.

cputest all = run all tests, in alphabetical order. Stops when mismatch is detected.
cputest tst.b = run tst.b tests only
cputest all tst.b = run tst.b, then tst.w and so on in alphabetical order until end or mismatch is detected.

If mismatch is detected, opcode word(s), instruction disassembly, registers before and after and reason message is shown on screen. If difference is in exception stack frame, both expected and returned stack frame is shown in hexadecimal.

Example output:

Code:
CHK2.L:
data/68020/chk2.l/0000.dat. 0...

0: 04d0 0800    chk2.l   (a0),d0
SR: expected 0000 -> 0009 but got 0001 (ffff)
Registers before:
D0: 00000010 D1: 00000000 D2: ffffffff D3: ffffff00
D4: ffff0000 D5: 80008080 D6: 7fff7fff D7: aaaaaaaa
A0: 00000000 A1: 00000080 A2: 00008000 A3: 00007fff
A4: fffffffe A5: ffffff00 A6: 0883ff00 A7: 08877ff8
SR:*0000   PC: 08840000 ISP: 0887ff80 MSP: 08880000
T1=0 T0=0 M=0 S=0 X=0 N!0 Z=0 V=0 C*0
Registers after:
D0: 00000010 D1: 00000000 D2: ffffffff D3: ffffff00
D4: ffff0000 D5: 80008080 D6: 7fff7fff D7: aaaaaaaa
A0: 00000000 A1: 00000080 A2: 00008000 A3: 00007fff
A4: fffffffe A5: ffffff00 A6: 0883ff00 A7: 08877ff8
SR:*0001   PC: 08840000 ISP: 0887ff80 MSP: 08880000
T1=0 T0=0 M=0 S=0 X=0 N!0 Z=0 V=0 C*1
OK: Generated exception 6
Toni Wilen is online now  
Old 08 September 2019, 22:29   #2
supaduper
Registered User
supaduper's Avatar
 
Join Date: Oct 2012
Location: Birmingham UK
Posts: 879
Fantastic work Toni

And I just donated to you as well, for I would be lost without Winuae, and I hope other Amigans throw some funds your way too,as its well deserved

Supaduper
supaduper is offline  
Old 08 September 2019, 23:10   #3
DamienD
Global Moderator

DamienD's Avatar
 
Join Date: Aug 2005
Location: London / Sydney
Age: 43
Posts: 15,418
Quote:
Originally Posted by supaduper View Post
And I just donated to you as well, for I would be lost without Winuae, and I hope other Amigans throw some funds your way too,as its well deserved
Yup, I too would be absolutely lost without WinUAE so try to donate to Toni as often as I can
DamienD is offline  
Old 13 September 2019, 15:11   #4
ceztko
Registered User
 
Join Date: Aug 2006
Location: Italy
Posts: 89
Quote:
Originally Posted by Toni Wilen View Post
UAE CPU Tester

I finally wrote utility (This was my "Summer 2019" project) that can be used to verify operation of for example software emulated or FPGA 680x0 CPUs.
Cool! Can you put link of this thread on the currently locked WinUAE beta changelog thread?
ceztko is offline  
Old 13 September 2019, 18:57   #5
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
Done. And thanks.

I know this has very limited use so I don't really expect anything but perhaps at least someone could test if first post "weird stuff" can be confirmed
Toni Wilen is online now  
Old 15 September 2019, 09:46   #6
gururise
Registered User

 
Join Date: May 2016
Location: Los Angeles
Posts: 63
Would be interesting to see the results from the Amiga emulation in Mister FPGA
gururise is offline  
Old 15 September 2019, 13:41   #7
rare_j
Zone Friend

rare_j's Avatar
 
Join Date: Apr 2005
Location: London
Posts: 939
Quote:
68010: Not supported yet (Don't have real 68010, at least not yet).
I have a 68010 to donate if you want it. Let me know if interested.
Attached Thumbnails
Click image for larger version

Name:	IMG_20190915_135507.jpg
Views:	146
Size:	546.1 KB
ID:	64479  

Last edited by rare_j; 15 September 2019 at 15:58. Reason: added photo
rare_j is offline  
Old 15 September 2019, 17:04   #8
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
Quote:
Originally Posted by gururise View Post
Would be interesting to see the results from the Amiga emulation in Mister FPGA
Run test generator and run the generated tests on that hardware. cputestgen.ini defaults should work, if ROM is A500 3.1 and system has 8M of Z2 Fast RAM. Generated test files are not included because they take almost few hundred megabytes for basic full 68000 test set.

It is almost guaranteed address error tests are not correct because it is totally undefined behavior in 68000 and probably no program cares.. (Some old copy protections do but they mainly care about stacked PC, not flags)

Quote:
Originally Posted by rare_j View Post
I have a 68010 to donate if you want it. Let me know if interested.
I am already getting 68010 soon.
Toni Wilen is online now  
Old 23 September 2019, 04:33   #9
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
Quote:
Originally Posted by Toni Wilen View Post
Run test generator and run the generated tests on that hardware. cputestgen.ini defaults should work, if ROM is A500 3.1 and system has 8M of Z2 Fast RAM. Generated test files are not included because they take almost few hundred megabytes for basic full 68000 test set.

It is almost guaranteed address error tests are not correct because it is totally undefined behavior in 68000 and probably no program cares.. (Some old copy protections do but they mainly care about stacked PC, not flags)



I am already getting 68010 soon.
This is really good stuff, thank you for creating it. I've started using it to test my attempts at implementing a couple of missing instructions on MiSTer FPGA.
apolkosnik is offline  
Old 23 September 2019, 07:05   #10
turrican3
Moon 1969 = amiga 1985

turrican3's Avatar
 
Join Date: Apr 2007
Location: belgium
Age: 43
Posts: 3,609
it's hard stuff !!!
winuae 4.3.0 won't be a piece of cake too !!!
thanks toni !!!
turrican3 is offline  
Old 28 September 2019, 10:31   #11
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
Important changes since last release:

- Fast test mode (CCR=0, CCR=31 only)
- Improved MOVEC test (read/write all ones, all zeros, verify, restore.
- SR/flag state was sometimes incorrect in error output (Visual bug only)
- More detailed error output. Depending on instruction type, source address, destination address and/or stack memory contents are listed. Contents before and after if instruction modified data. Only output "after" registers if they changed.
- Atari ST support (thanks Nicolas)

Last edited by Toni Wilen; 28 September 2019 at 12:09.
Toni Wilen is online now  
Old 03 October 2019, 18:47   #12
asymetrix
Registered User

 
Join Date: Jul 2009
Location: UK
Posts: 99
This project is very useful. Great work ! If I may, maybe add an MD5 check to a specific group of ROMS, to keep consistency in testing.

Its exciting - does this mean we can have a open source PCIE i7 CPU card emulating a 68060 in 64bit ASM ?

Last edited by asymetrix; 03 October 2019 at 19:04.
asymetrix is offline  
Old 04 October 2019, 19:35   #13
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
Quote:
Originally Posted by asymetrix View Post
This project is very useful. Great work ! If I may, maybe add an MD5 check to a specific group of ROMS, to keep consistency in testing.
I can CRC32 of the ROM image used during generating which gets printed when tester runs but I am not going to add any checksum check code to tester because it should not be too slow even on slow 7MHz 68000 hardware.

Quote:
Its exciting - does this mean we can have a open source PCIE i7 CPU card emulating a 68060 in 64bit ASM ?
What does this have to do with tester?

It is an automatic tester, it can only make compatibility testing faster and less boring than manual testing. Nothing more.

It won't make developing actual emulator any easier.
Toni Wilen is online now  
Old 19 October 2019, 17:57   #14
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
Toni, thank you again for this great tool! I think that its brute-force approach is an important feature, perhaps leaving it as an option would make sense, at least for people that might want to try to do some fuzzing on their new implementations.

Your cputest tool helped me to get some bugs related to unsupported addressing modes (for instructions using register bits in the second word) out of the TG68K core on MiSTer. Now, the mystery behind the texture corruption in the Gouraud Pulse in Nexus 7 demo has been solved! :-)
apolkosnik is offline  
Old 19 October 2019, 20:15   #15
Toni Wilen
WinUAE developer
 
Join Date: Aug 2001
Location: Hämeenlinna/Finland
Age: 44
Posts: 23,249
Quote:
Originally Posted by apolkosnik View Post
Toni, thank you again for this great tool! I think that its brute-force approach is an important feature, perhaps leaving it as an option would make sense, at least for people that might want to try to do some fuzzing on their new implementations.
I am not going to remove it, as you said, it can reveal weird undocumented cases by creating bit combinations that no one is supposed to use.. (Note that currently some invalid combinations are never generated because some of them confuse the CPU, some parts of calculations use "hidden" address register, some ignore the bit and use matching data register..)

"target ea mode" is the alternative, non-random one. It only generates EAs listed in ini (one or more, comma separated). Mainly meant for testing address and bus errors.

Quote:
Your cputest tool helped me to get some bugs related to unsupported addressing modes (for instructions using register bits in the second word) out of the TG68K core on MiSTer. Now, the mystery behind the texture corruption in the Gouraud Pulse in Nexus 7 demo has been solved! :-)
Which instruction? I am quite sure UAE didn't support those undocumented A/D field instructions and the demo did work fine.
Toni Wilen is online now  
Old 20 October 2019, 00:46   #16
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
Quote:
Originally Posted by Toni Wilen View Post
I am not going to remove it, as you said, it can reveal weird undocumented cases by creating bit combinations that no one is supposed to use.. (Note that currently some invalid combinations are never generated because some of them confuse the CPU, some parts of calculations use "hidden" address register, some ignore the bit and use matching data register..)

"target ea mode" is the alternative, non-random one. It only generates EAs listed in ini (one or more, comma separated). Mainly meant for testing address and bus errors.



Which instruction? I am quite sure UAE didn't support those undocumented A/D field instructions and the demo did work fine.
The patch link is below, I'll list them when I get home.
https://github.com/MiSTer-devel/Minimig-AGA_MiSTer/pull/23/files
apolkosnik is offline  
Old 20 October 2019, 01:39   #17
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
For the Nexus7 demo, I think that some loop was getting bfins instruction tripped up to generate bad output, I think. I've even found it to be the case on the Alpha release of Vampire AGA core. (http://www.apollo-core.com/knowledge...21853&z=1RQMVi). lol.

Some concrete samples:

Bitfield instructions:

bfchg eac0 31fe ... should be 01fe since bits 15-12 of the second word should be 0. No dest in this opcode.
bfclr ecc2 ec68 ... should be 0c68 since bits 15-12 of the second word should be 0. No dest in this opcode.

bfset eec0 8fc4 ... should be 0fc4 since bits 15-12 of the second word should be 0. No dest in this opcode.
bftst e8c3 3a9a ... should be 0a9a since bits 15-12 of the second word should be 0. No dest in this opcode.

bfexts ebc0 f094 ... should be 7094 since bit 15 of the second word should be 0 - makes it write into An.
bfextu e9c0 116b ... bfextu d0{#5:d3},d1 ... if Dw(bit 5)=1 bits 4,3 should be 0. Also, if Do(bit 11)=1 bits 10,9 should be 0.
bfffo edc0 d771 ... should be 5771 since bit 15 of the second word should be 0 - makes it write into An.

bfins efc0 7ee1 ... if Do(bit 11)=1 bits 10,9 should be 0. Also, if Dw(bit 5)=1 bits 4,3 should be 0.

Division and multiplication:
divul.l d0,d6:d0 - 4c40 000e ... Result written into A6 rather than D6.
mulu.l d0,d1:d0 - 4c00 05e9 ... A1: modified 00000080 -> 00000000 but expected no modifications

eorsr/orsr/etc:
8194: 0a7c 0800 eori.w #$800,sr -- In this case the bit12 of SR is supposed to be 0 and is not supposed to be used for anything at least on 68000-68030.


I'm not sure about callm/cas/cas2/chk/chk2 as these are not really implemented on TG68K

Last edited by apolkosnik; 20 October 2019 at 01:45.
apolkosnik is offline  
Old 20 October 2019, 09:36   #18
grond
Registered User

 
Join Date: Jun 2015
Location: Germany
Posts: 651
Quote:
Originally Posted by apolkosnik View Post
For the Nexus7 demo, I think that some loop was getting bfins instruction tripped up to generate bad output, I think. I've even found it to be the case on the Alpha release of Vampire AGA core.
The Vampire AGA core you are mentioning is from 2017. The 080 bitfield instructions were fixed long ago. If you look at the V4, you'll see that there is no problem.
grond is offline  
Old 20 October 2019, 15:03   #19
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
Quote:
Originally Posted by grond View Post
The Vampire AGA core you are mentioning is from 2017. The 080 bitfield instructions were fixed long ago. If you look at the V4, you'll see that there is no problem.
I could technically check the 2.11 or 2.12 core on my Vampire 500.
apolkosnik is offline  
Old 23 October 2019, 20:08   #20
apolkosnik
Registered User

 
Join Date: May 2019
Location: US
Posts: 12
I must be doing something wrong, but I get from cputestgen.exe the following:
...
data/68020/MOVE.L
Loop mode didn't end!?

and it just stops right there.
apolkosnik is offline  
 


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Similar Threads
Thread Thread Starter Forum Replies Last Post
680x0 tester JimDrew support.WinUAE 3 28 October 2014 21:44
Cheap dual core mobo/cpu/ram Smiley MarketPlace 0 16 May 2012 15:23
Hatari now uses WinUAE CPU Core Ian Retrogaming General Discussion 7 15 January 2012 11:36
Free : 680x0 CPU's alexh MarketPlace 23 28 November 2009 13:29
Usung UAE CPU core gbr New to Emulation or Amiga scene 3 12 November 2003 22:29

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +2. The time now is 10:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.
Page generated in 0.08762 seconds with 14 queries