FS-UAE / WinUAE x86-64 JIT Compiler

[Enverex]

Long time without any posts in here so I hope this is the right place.

If I enable JIT and the JIT cache and then try to open fs-uae, I get...

arcade@ken ~ $ fs-uae
FS-UAE 2.7.14dev (Built for Linux x86-64)
Copyright 1995-2002 Bernd Schmidt, 1999-2015 Toni Wilen,
2003-2007 Richard Drummond, 2006-2011 Mustafa 'GnoStiC' Tufan,
2011-2015 Frode Solheim, and contributors.

This is free software; see the file COPYING for copying conditions. There
is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE. See the README for more copyright info, and the source code for
a full list of contributors

UAE: Initializing core derived from WinUAE 3220
LOG: /home/arcade/Documents/FS-UAE/Cache/Logs/fs-uae.log.txt
# could not open dir
UAE: KS ROM v3.1 (A4000) rev 40.68 (512k)
JIT: JIT: 64-bit pointer (0x560fcca97c48) at src/jit/compemu_support.cpp:3775 (fatal)
Aborted (core dumped)

I installed FS-UAE by compiling directly from GIT. Am I missing something essential here for it to work?

[FrodeSolheim]

Originally Posted by Enverex

I installed FS-UAE by compiling directly from GIT. Am I missing something essential here for it to work?

Have you in any way overridden / added gcc compiler/linker options?

[Enverex]

I'm compiling it through Arch Linux's typical setup which will be adding my normal systemwide compiler options of...

CARCH="x86_64"
CHOST="x86_64-pc-linux-gnu"
CFLAGS="-march=native -Ofast -pipe -w"
CXXFLAGS="${CFLAGS}"
LDFLAGS="-Wl,-O1,--sort-common,--as-needed,-z,relro"
MAKEFLAGS="-j5"

[FrodeSolheim]

Hmm, I don't think those particular options should cause any issues (but GCC could be built with other "default options" causing the problem as well).

In order for the JIT to work, the fs-uae executable's "data section" must be located below 0xffffffff (in the first 4 GB of virtual address space). The error you got indicates that the data section (think "global variables in C") is loaded above this. So, I still suspect FS-UAE is built as a position-independent executable, or with other "hardened" options causing the problem.

Maybe Arch is adding some options of its own? What's the final link command for fs-uae, as echoed to your terminal?)

[Enverex]

This is the last one...

g++ -I/usr/include/libdrm -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -DLUA_USE_POSIX -I/usr/include/AL -I/usr/include/libdrm 
-I/usr/include/libpng16 -D_REENTRANT -I/usr/include/SDL2   -fno-strict-overflow -fno-strict-aliasing -Wmissing-declarations -Waddress -fvisibility=hidden 
-I/usr/include/mpeg2dec -march=native -Ofast -pipe -w  -Wl,-O1,--sort-common,--as-needed,-z,relro -o fs-uae src/fs-uae/config.o src/fs-uae/config-accelerator.o 
src/fs-uae/config-common.o src/fs-uae/config-drives.o src/fs-uae/config-graphics.o src/fs-uae/config-hardware.o src/fs-uae/config-model.o 
src/fs-uae/config-paths.o src/fs-uae/config-sound.o src/fs-uae/input.o src/fs-uae/joystick.o src/fs-uae/keyboard.o src/fs-uae/main.o 
src/fs-uae/menu.o src/fs-uae/mouse.o src/fs-uae/paths.o src/fs-uae/plugins.o src/fs-uae/recording.o src/fs-uae/uae_config.o src/fs-uae/video.o 
src/ppc/ppcd.o src/sndboard.o  src/moduleripper.o  libuae.a libcpuemu.a libfsemu.a   libmanymouse.a   libudis86.a  libprowizard.a libsoftfloat.a  
-lm -ldl    -lGLEW -lGLU -lGL -lgthread-2.0 -pthread -lglib-2.0 -lopenal -lGL -lpng16 -lSDL2 -lX11 -lz  -lmpeg2 -lmpeg2convert

[FrodeSolheim]

Can you please include the result of running:

objdump -h fs-uae

[Enverex]

As requested...

/bin/fs-uae:     file format elf64-x86-64

Sections:
Idx Name          Size      VMA               LMA               File off  Algn
  0 .interp       0000001c  0000000000000238  0000000000000238  00000238  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  1 .note.ABI-tag 00000020  0000000000000254  0000000000000254  00000254  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .note.gnu.build-id 00000024  0000000000000274  0000000000000274  00000274  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  3 .gnu.hash     00000234  0000000000000298  0000000000000298  00000298  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  4 .dynsym       00003360  00000000000004d0  00000000000004d0  000004d0  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  5 .dynstr       00002178  0000000000003830  0000000000003830  00003830  2**0
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  6 .gnu.version  00000448  00000000000059a8  00000000000059a8  000059a8  2**1
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  7 .gnu.version_r 000001b0  0000000000005df0  0000000000005df0  00005df0  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  8 .rela.dyn     001a4210  0000000000005fa0  0000000000005fa0  00005fa0  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  9 .rela.plt     00002d60  00000000001aa1b0  00000000001aa1b0  001aa1b0  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 10 .init         00000017  00000000001acf10  00000000001acf10  001acf10  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 11 .plt          00001e50  00000000001acf30  00000000001acf30  001acf30  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 12 .plt.got      00000018  00000000001aed80  00000000001aed80  001aed80  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 13 .text         00a0ea42  00000000001aeda0  00000000001aeda0  001aeda0  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 14 .fini         00000009  0000000000bbd7e4  0000000000bbd7e4  00bbd7e4  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
 15 .rodata       000550b0  0000000000bbd800  0000000000bbd800  00bbd800  2**5
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 16 .eh_frame_hdr 000468e4  0000000000c128b0  0000000000c128b0  00c128b0  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 17 .eh_frame     001a8c34  0000000000c59198  0000000000c59198  00c59198  2**3
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 18 .gcc_except_table 00000bd7  0000000000e01dcc  0000000000e01dcc  00e01dcc  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
 19 .init_array   00000048  00000000010036c8  00000000010036c8  00e036c8  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 20 .fini_array   00000008  0000000001003710  0000000001003710  00e03710  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 21 .jcr          00000008  0000000001003718  0000000001003718  00e03718  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 22 .data.rel.ro  001035a8  0000000001003720  0000000001003720  00e03720  2**5
                  CONTENTS, ALLOC, LOAD, DATA
 23 .dynamic      000002d0  0000000001106cc8  0000000001106cc8  00f06cc8  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 24 .got          00000068  0000000001106f98  0000000001106f98  00f06f98  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 25 .got.plt      00000f38  0000000001107000  0000000001107000  00f07000  2**3
                  CONTENTS, ALLOC, LOAD, DATA
 26 .data         00036139  0000000001107f40  0000000001107f40  00f07f40  2**5
                  CONTENTS, ALLOC, LOAD, DATA
 27 .bss          06217f20  000000000113e080  000000000113e080  00f3e079  2**5
                  ALLOC
 28 .comment      00000034  0000000000000000  0000000000000000  00f3e079  2**0
                  CONTENTS, READONLY

[jbl007]

-Ofast

makes crazy optimizations (enables many unsafe flags). Better stick with

-O2

as default options. https://wiki.gentoo.org/wiki/GCC_optimization

But I admit, it could be interesting to compare JIT-compiler performance with different gcc optimizations flags.

[Enverex]

I'm familiar, but if a specific program is broken by "more serious" flags then that program should be setting the lower flags at compile time. I assume FS-UAE's makefile doesn't specify an optimization level, so it's using my defaults.

My entire system (well, everything that's compiled by source which is a sizable amount) is using these flags and I've not actually run into any other issues.

[FrodeSolheim]

The presence of the ".data.rel.ro" section can indicate that FS-UAE is compiled with position-independent code. Maybe you are using a brand new GCC and it defaults to creating position-independent executables? (or GCC on Arch Linux is built with PIE enabled by default).

Please try to add -no-pie to LDFLAGS, reconfigure/recompile FS-UAE and see if that fixes it...

[Enverex]

I was using GCC 6.1 and have since updated to 6.2 with no change. I'll try that flag tomorrow and report back.

[Enverex]

"-no-pie" wasn't valid in LDFLAGs, but after adding it to my C(XX)FLAGs and rebuilding, the error went away. Looks like that may need to be forced in the makefile.

[FrodeSolheim]

Originally Posted by Enverex

"-no-pie" wasn't valid in LDFLAGs, but after adding it to my C(XX)FLAGs and rebuilding, the error went away. Looks like that may need to be forced in the makefile.

According to documentation, "-no-pie" is a linker flag, and it works fine if I run ./configure LDFLAGS="-no-pie"

But yes, maybe that option needs to be enabled by configure (if/when supported by gcc).

Originally Posted by Enverex

I'm familiar, but if a specific program is broken by "more serious" flags then that program should be setting the lower flags at compile time. I assume FS-UAE's makefile doesn't specify an optimization level, so it's using my defaults.

My entire system (well, everything that's compiled by source which is a sizable amount) is using these flags and I've not actually run into any other issues.

Enabling optimizations past "O2" is definitively not supported. You are free to do so, but remember to mention that if filing bug reports. Actually, test with an fs-uae compiled with default optimizations before filing a bug report...

(I am not going to force a specific optimization level though)

Originally Posted by jbl007

But I admit, it could be interesting to compare JIT-compiler performance with different gcc optimizations flags.

It will have next to no impact on performance (The reason is that the JIT-generated code is generated by the UAE code, not the C compiler, so GCC is not in any way involved in the generated JIT code).

In addition, previous tests (done by me) have shown very little general benefits for FS-UAE, if any at all, from enabling more aggressive optimizations than -O2.

[Enverex]

Yeah, I read that it was a linker option which is why I was surprised it didn't like being in LDFLAGs, but for some reason CFLAGS made it happy.

Originally Posted by FrodeSolheim

In addition, previous tests (done by me) have shown very little general benefits for FS-UAE, if any at all, from enabling more aggressive optimizations than -O2.

To be honest, I'd force that then for the reasons you've stated. If there's no benefit past O2 and higher levels are prone to breaking FS-UAE, it's the safe thing to do.

[Enverex]

BTW, it still crashes in the same way when using just "-O2". Only manually adding -no-pie gets JIT working (sorry, forgot to mention this before).

EDIT: -no-pie also makes FS-UAE 1.64MB smaller as well, curiously enough.

EDIT 2: I take that back. A1200 and A4000 work with -no-pie, but CD32 still crashes with "JIT: JIT: 64-bit pointer (0x7f9a3883dee0) at src/jit/compemu_support.cpp:2981 (fatal)" so it looks like it kinda works, but not completely.

[jbl007]

Originally Posted by Enverex

CD32 still crashes with "JIT: JIT: 64-bit pointer (0x7f9a3883dee0) at src/jit/compemu_support.cpp:2981 (fatal)"

Confirmed!
But CD32+JIT can't work at all because it has a 68EC020 (24-bit addressing). If you set the cpu to "real" 68020, it works.

[Enverex]

Originally Posted by jbl007

Confirmed!
But CD32+JIT can't work at all because it has a 68EC020 (24-bit addressing). If you set the cpu to "real" 68020, it works.

Aaaah, silly me. That makes sense, thanks.

I've adapted the last fs-uae release to libretro; Libretro handles emulators as shared libraries.

Once JIT is enabled, the 32 bits address range limitation on linux x86_64 is an issue: We can set the executable segments inside the proper address range using: export LD_PREFER_MAP_32BIT_EXEC=1 But some data part must be loaded in the 32 bits memory too. A modified malloc using mmap MAP_32BIT is likely required here: This is the proper method; Turning off position independent code generation using -no-pie is not a good idea.

Do you know if someone is trying to fix this?

[FrodeSolheim]

Yes, some global variables are referenced by the JIT compiler, and therefore the data section must also be within "32-bit memory". Yes, allocating memory manually and using pointers instead of global variables would fix that. I don't think anyone is planning to do anything about that.

(Btw, libretro is used mostly for games, right? If so, you generally don't want to use JIT anyway...)

Yes indeed about libreto; But it works fine on a full computer, and adding the full support is worthy.

UAE4ARM has a working ARM JIT; Has someone tried to merge both JIT ?