FEX-2202

Changes

FEXCore

• AArch64

  • Single instruction optimization for AESKeyGenAssist (6744a06)

• CPUID

  • Fixes ASAN problem with reading midr (6022715)

• OpcodeDispatcher

  • Fixes CRC32 decoding in 0F_38 table (1c24206)
  • Implements PCMPGTQ (caffad8)
  • Implements CRC32 instruction (b7a76af)
  • Removes extraneous extract in VFCMP (4c92b56)
  • Fixes ptest flags calculation. (0a8717d)
  • Fixes signed displacement wraparound on 32-bit (e2798e3)

• IR

  • Fixes some wrong offsets in passes (c252f8b)
  • Fixes SSA ID printing (a2aa9f3)

• Softfloat

  • Stop doing special handling for FREM (9d08451)

• Misc

  • Defer a significant number of ALU flag calculation (832d013)

Linux

• Fixes for older build environments (ade0d0f)
• Fixes 32-bit getsockopt and setsockopt (d914954)
• Handles the remaining 32-bit siginfo_t usage (7768f69)
• Fixes emulatedpath with symlink following (5dda3a1)

Thunks

• Add meta data to libvulkan_device (a0026c9)

Tools

• SocketLogging

  • Fixes MsgHandler not syncing with Assert level (9eb5ba5)

• FEXLogServer

  • Stop duplicating and dropping messages (eb17ede)

• FEXRootFSFetcher

  • Fixes some edge case behaviours (13f35f7)
  • Check if curl is installed and fail before running (784cbdd)

• FEXLoader

  • Resolve the absolute path to rootfs if possible (023aaaa)

CMake

• Always use local xxhash to statically link (5978143)
• Adds TUNE_ARCH option (e36beb8)

FMT

• Update fmt to 8.1.1 (140e982)

Vixl

• Point vixl submodule towards FEX's fork (a3a138e)
• Update vixl (8e3893d)

Unit tests

• Revise POSIX tests known failures and disabled (ea76de0)
• Fixes ROOTFS needing to be defined prior to cmake (7c08b67)
• Adds a BCD unit test (609587f)
• Fixes InterruptableConditionVariable test to use the syscal… (82e5977)

Misc

• Fixes FEXConfig build (d308a43)
• Adds an option to disable the IR interpreter (11a5105)
• Fix pressure vessel hangs (334a8ef)
• Fixes some of the warnings that cropped up (bf581de)

FEX-2201

Changes

A large amount of changes this month.

• We now support BMI2, RDTSCP, CLZero.
• We tell the guest what ARM CPU name we are running on.
• We expose the hybrid CPU flag flag on big.LITTLE ARM SoCs.
• Bunches of syscall fixes, JIT compile time speed increases, usability improvements.
• Tons more.

A new FEXRootFSFetcher tool for downloading prebuilt RootFS images.
A Python script for automatically setting up an Ubuntu PPA and RootFS.

• See our quick start guide to see how here
• Or see the Ubuntu PPA here

FEXCore

Arm64

• Fixes vixl assertions around ubfm usage (5654f9a)
• Fixes MapSelectCC FGT flag (13087f8)

CPUID

• Fixes another memory overflow issue (0e45b98)
• Fixes crash on unknown CPU (be3e3a3)
• CPUID Expose Hybrid flag and CPU names (9394e49)

Context

• Take some arguments as pointer-to-const (9d43904)

Dispatcher

• Disables unsync context message (154468c)
• Adds more state reconstruction to state restore (fa6f1b1)

OpcodeDispatcher

• Handle PDEP (0a3a270)
• Handle PEXT (9198400)
• Implements RDTSCP (ee821b9)
• Implements CLZero instruction (5f0dfcd)

X86Tables

• Build Unknown op definition tables at compile time (e8670bb)
• Speed up initialization of X86Tables (e99a23c)

HostFeatures

• Detect if the host CPU suports float exceptions (117cbde)
• Consolidates HostFeatures flags (e24eb7a)

IR

• Fixes sized constant mask (565d1e2)

Misc

• Don't use Run() inside RunUntilExit() (0bf5776)
• Fixes a memcpy overflow in processor brand (340699c)
• Fixes FEXCore_Base config dependency (95bd309)
• Implements recoverable INTO instruction (cd4269f)
• Reorganizes some AOT related code (d4655fb)

Linux Emulation/Syscalls

• Fixes pidfd_send_signal for 32-bit (ac63a2b)
• Finish 32-bit msqid_ds usage (98210fd)
• Implements the remaining 32-bit syscalls (bc120b9)
• Fixes semid_ds_64 definition (f2fd9d9)
• Fixes 32-bit shmctl (45430e9)
• Adds the safe syscall unimplemented gap for x86-64 (e05d116)
• Enable warnings (75e5df5)
• Socket
• Resolve sign comparison mismatch cases (097f48f)

Misc

CMake

• Adds an OVERRIDE_VERSION option (0808599)
• Adds a TUNE_CPU option (5146651)
• Adds experimental libc++ option (f3a27a5)

Build fixes

• Fixes build on Ubuntu 20.04 take 3 (8f83567)
• Fixes build on Ubuntu 20.04 take 2 (059e8a9)
• Fixes build on Ubuntu 20.04 (02005e7)
• Fixes Ubuntu 20.04 compilation on AArch64 (47bd479)

Config

• Enables all host threads by default (edce981)
• Sanitize Core option (9bffaee)

Docs

• Adds documentation about the FEX monthly release process (a65e7a0)

GDBServer

• Fixes long string packet encodings (88ce9b5)

Misc

• Stop hardcoding /usr/bin paths (1d5bd15)
• Improves compile ability for older libraries (c691d70)
• More AOT code movement (43dc232)
• FEXMountDaemon squashfs fixes. (ea73c9d)
• Update jemalloc for fixed libc overrides (c43af0e)
• Implement thunk library generation using libclang (425d932)
• {x32, x64}/Thread: Make use of .data() instead of .at(0) (b6499ac)
• Adds a new ReentrantMutex to use for FEXCore (b9c4902)
• NetStream: Move NetBuf definition into cpp file (9860e8b)

Tests

gvisor

• Disables flaky test (e62cb41)

unittests

• Fixes a missing dependency on ASM tests (b078f51)
• Enables remaining unaligned atomic tests on ARMv8.0 (fd04247)
• Fixes known failures (5c4112f)

RootFS

• Stop trying to retry rootfs after five times (a45bc75)
• Be more robust against stale lock files (d4b31cd)

Tools

FEXConfig

• Removes jemalloc usage (c65be9f)

FEXRootFSFetcher

• Remove 0x prefix on hash (b72245a)
• Adds a new tool to help set up a new RootFS (72e8a99)

Scripts

• Adds a python script that can hand hold a user through FEX install (b276750)

FEX-2112

Arm64

• Reduce the chance of hanging on reentrant allocations (8ab6830)
• Adds an inline syscall optimization (f746870)
• COND_FGT should map to gt, not hi, as it is not FGTU (b45603d)
  • Fixes steamwebhelper burning all CPU cores!

FEXCore

• Fixes CompileService race condition on thread creation (5345f7f)
• Centralize alignment utility functions in one header (966272c)
• Fixes ERROR_AND_DIE (fbd14b6)
• Supports guest SIGILL (6e1ea92)
• CPUID
  • Call handler functions directly (ffee22d)
• OpcodeDispatcher
  • Remove debug message (d883b4b)
  • Implement BZHI (50c8d9e)
  • Amend return types of BitSize helpers (c63c5fb)
  • Implement MULX (498a845)
  • Resolve sign mismatches in loops (ad864e0)
  • Add helper for getting bit sizes (3a3cbf2)
  • Handle RORX (51d21d8)
  • Implement BMI2 SARX/SHLX/SHRX (e37a0bb)
  • Removes usage of SignalFrame stack on JITs (d81d5e2)
  • Use std::vector as the underlying container for std::stack (61ccec3)
  • Partial support for RIP adjust in signal handler (77093a7)
  • Resolve unused variable warnings (a04cc4d)
• OpcodeDecoder
  • Shorten up GPR and MM base offset retrieval (59ed91c)
  • Convert enums into enum classes where applicable (d1fa8bc)
  • Add support for ADX (d3cd354)
• Frontend
  • Handle VEX RXB bits (d47182b)
  • Mark modrm function LUT as static (c6662a4)
• GDBStub
• Fixes a few hangs and crashes (e027b83)
• Interpreter
  • Build op handler table at compile-time (63f9e0b)
  • Mark F80CMP array as static constexpr (13ad5b6)
• JIT
  • Eliminate redundant jump target map lookups (8ef0278)
  • Ensures signals in compileservice JIT space is handled (97e3a36)
• JitSymbols
  • Make use of fmt (db0740b)
• IR
  • Make some interface functions accept pointers to const (8c38f93)
  • Convert NodeID into a strong type (b93871f)
  • Add type alias for Node IDs (1f306d6)
• RegisterAllocationPass
  • Resolve sign comparison mismatch in CalculateNodeInterference() (597e4e1)
  • Reduce usages of NodeIDs where applicable (19a0651)

Linux

• Updates syscalls to 5.16 syscalls (c4d05fb)
• Fixes sched_getaffinity (d5bf641)
• Fixes MAP_32BIT error case (92ae6ae)
• Fixes sigprocmask with new and old set being the same address (7bb7891)
• Passthrough 32-bit syscalls that can be (3dc938b)
• Emulate MAP_32BIT on mmap (081d016)
• SignalDelegator update signal mask on return from sigsuspend (90511dd)
• Updates 32-bit DRM emulation (a92233c)

Thunks

• Fail loudly if thunking is enabled for a library that isn't installed (b1b6160)
• ThunksDB
  • Updates file to include local libs (e0d40dd)

Tests

• Enables 32-bit host runner (cabb58a)
• Minor cleanup (2a894bc)
• Various thunk library cleanups (3730a42)

Misc

• BucketList
  • Minor API touchups (baadf0b)
• CMake
  • Adds option to enable time-trace compile option (919e2bc)
• CompileService
  • Store WorkItem instances as unique_ptr (ef772c3)
• ELFCodeLoader
  • Fixes BRK allocation on some ELF loading (b327daf)
• EnumUtils
  • Remove shift operators from helper macro (977d92b)
• General
  • Migrate logging over to fmt where possible (513b02f)
• RootFS
  • Keep rootfs around while in container (16f1ad4)
• IR.json/json_ir_generator: Minor touchups to generated IR utilities (9b43f94)
• Various packaging improvements (23e583e)

Frontend

• Adds Socket logger and tool (6c93b6f)
• FEXLoader
  • Change 32-bit memory check error to warning (82ebdf5)
  • Print memory map when 32-bit intersect happens (e6d285a)
• Syscalls
  • Move construction of syscall name map into GetSyscallName() (0560e9b)

FEX-2111

Core

• Mark relevant Interpreter/JIT functions as [[nodiscard]] (43454ab)

• IR

  • Fixes memory ops having a duplicate size field (56e5e78)
  • Add handling for ANDN operations (34b2f93)

• Frontend

  • Handle VEX source operands (7e9201c)

• Interpreter

  • Splits ops in to separate files (e9937d9)

• JIT

  • Fixes asserts added to the JIT (28d084b)

• JITSymbols

  • Allow grouping JIT symbols by guest named regions (ff74e0a)
  • Change over to runtime enablement of symbols (8b3c461)

• Arm64

  • Fixes paranoid TSO mode (8b64546)
  • Consolidate HandleSIGBUS (babb81a)
  • Don't fall back to native (a3b39af)
  • Make sure to spill static FPRs on guest signal (98ba0bf)
  • Be more explicit about x87 ABI usage (bdc66a3)

• OpcodeDispatcher

  • Implement BLSR/BLSMSK (b4a71a2)
  • Implement handling for BLSI (285ef38)
  • Handle BMI1 BEXTR (76538be)
  • Deduplicate OpToIndex definition (031fa8a)

CPUID

• Adds support for hybrid flag (a9d3122)

Linux Emulation

• Fixes execve on softlinks in rootfs (a393d66)
• Implements virtio ioctls for 32-bit (e034364)
• Fixes FEXLoader argument passing (f9078f8)
• Follow more symlinks in emulation (09ee6d3)
• No longer do magic on SIGCHLD (95457bc)
• Fix 32-bit nanosleep always passing valid remainder (9e8af23)

Thunks

• Fix missing libdrm include path (fa1648c)
• Respect DESTDIR environment variable (6cd73a6)
• XCB Add missing header file (8dfe305)
• vulkan
  • Suppress compiler warnings about unknown attributes (64aa4f0)

Misc

• Fixes environment loader not hooked up to ArgumentLoader (8f170d4)
• Fix left-over printf specifier in fmt log (aa1c47c)
• X86Tables
  • Make flag helper functions constexpr (d09706a)
• Telemetry
  • Adds telemetry for when an application tears (235367b)
• DeadContextStoreElimination
  • Fix missing printf specifier entry (b1ab252)
• RAPass
  • Add debug compile option to disable spill slot reuse (3661223)
• TestHarnessRunner
  • Make argument check more strict (eb8a8bf)
  • Convert LOGMAN_THROW_A into error log and exit (c9c3526)
• Cmake
  • Change static-pie message to indicate compiled without it (e547f0c)
• FEXConfig
  • Fixes timeout in select causing 100% CPU load (8aebbbd)
• Allocator
  • Reserve upper 128TB of VA on 64-bit process (cae2f8c)

FEX-2110

• AOTIR
  • fix use after free (327c4d5)
• Config
  • Check for container-manager and redirect (56bddd3)
• Core
  • Minor documentation and code splitting (fb01e8b)
• FEXConfig
  • Change shortcut for opening application profile (a56463a)
• FEXLoader
  • Flush log output (360c4a2)
• FEXMountDaemon
  • Fixes shutdown race conditions (c9a6265)
• FileManager
  • Allow reading real root (ba174e5)
• Interpreter
  • Changes basic loadstores to sized accesses (08f5654)
• Linux
  • Implements support for POSIX message queues on 32-bit (21ff433)
  • Implements support for timer_create (f232dce)
  • Fixes timex definition for 32-bit syscalls (368095f)
  • Fixes rlimit syscalls for 32-bit (e91061f)
• OpcodeDispatcher
  • Ensure some x87 templates get passed long constants (1fa3afd)
• Scripts
  • Adds a new script for extracting function definitions (9d460e8)
• Softfloat
  • Allow forcing use of some x87 on x86 host (d004fee)
• StructVerifier
  • Fix struct match and minor fixes (2b757a9)
• Thunks
  • Vulkan thunks (b3efb1d)
  • Support versioned libraries (0dc8e23)
  • Install a global thunksDB for our current thunks (72125c9)
  • Adds a few missing libGL thunk functions (9a07b55)
  • Some minor X related thunk changes (e0b878f)
  • Expands Xext thunked functions listo (073224f)
  • Expands what asound thunking supports (00511c1)
  • Adds a new Thunks database config file (66c7fdb)
  • Makes file searches a bit easier (a48ed37)
  • Adds init helpers with function call (98bee5a)
  • ThunkHelpers script improvements (8d13261)
  • Minor fixes to the config and loading (d93edb2)
• Misc
  • Adds support for setting host environment variables from config (d5f9f43)
  • Build fix for ENABLE_JITSYMBOLS (591fc00)
  • RA validation (3744ec2)

FEX-2109

• Arm64

  • Fixes SRA spilling on signal (35c6642)
  • Reimplements support for binfmt_misc without update-binfmts (5d73ac3)

• Arm64Emitter

  • Resolves some IWYU warnings (c206942)

• Config

  • Moves non-OS specific configuration loading to FEXCore (6da3330)

• EmulatedFiles

  • Fixes openat for emulated files not using FDCWD (109c42a)

• FEXBash

  • Allow creating a bash instance easily (9140ba2)

• FEXConfig

  • Load application config and advanced tab (3e05544)

• FEXCore

  • Return the ParentThread with InitCore (97f413c)
  • Minor symbol visibility fixes (5debdf8)
  • Use GetCursorAddress when able (8eb0df9)

• FEXLoader

  • Fixes potential bug in log output to stdout/stderr (1092229)

• FEXMountDaemon

  • Fixes dangling mounts problem (dd34316)
  • Fixes some minor issues (ad34cdd)
  • Early fork to deparent child (7795078)

• Hangover

  • Initial support for the syscall handling. (77db25f)

• Linux

  • Setup signal mask correctly to block signal-in-signal situations (70931bf)
  • sigaltstack ignore SS_ONSTACK (c74620f)
  • Handle fpstate in the signal delegator correctly (1c4503e)
  • Fixes 32-bit interval timers (425ee98)
  • Fix V3d and VC4 ioctl definitions (0c484ac)
  • x86: Initial V3D and VC4 ioctl emulation (127d7c1)
  • Fixes accidental execve escape (1c3be54)
  • Fixes 32-bit syscalls that use 64-bit values (6afc3ca)
  • Implements support for clone with namespaces (7aae9b7)
  • Fixes readlinkat for self (8dd41e7)
  • Implements pivot_root syscall (cce3f36)

• RA

  • Add max NoteCount assert (c645d86)

• SignalDelegator

  • More splitting and cleanup (097b3ad)

• Misc

  • Fix unaligned CASPair on ARMv8.0 (2c02dca)
  • Syscall fixes (befd0aa)
  • Properly implement single CAS on ARMv8.0 (25e9585)
  • Move BucketList into it's own file (4f66d3e)
  • Gvisor fixes (d600b34)
  • Massive amount of IWYU cleanup (10793e8)
  • Fixes Callback interface to take a thread argument (fa224a3)
  • Fixes jemalloc library ordering (b373d0f)
  • Proton 6.3 32-bit fixes (8099dfc)
  • OpcodeDispatcher: Implements undocumented repne on string ops (926ddab)
  • 32-bit wine fixes (6a08587)
  • Updates jemalloc to fix missing alias posix_memalign (63af80f)
  • Adds new FEXGetConfig program (43052a5)
  • Implements support for offline only telemetry (b120a8e)
  • Rebase skmp/no sra (0be16ba)

• unittests

  • Hotfix for older nasm (9563b5a)

• x86

  • Fixes siginfo_t si_addr for SIGBUS/SIGSEGV (6b87839)

FEX-2108

• 64BitAllocator

  • Convert std::vector table into std::array (ec6cc5c)

• AArch64

  • Workaround static-pie crashing (4c43151)

• Arm64

  • Fixes fill and spill slot offset calculation (6ac2300)

• CPUID

  • Update TM and TM2 CPUID bits (d142d7c)

• CPack

  • Update package name to remove conflict (2d11df0)

• FEXLoader

  • stderr on immediate failure (84dee03)

• FEXMountDaemon

  • Make squashfs mounting more robust (ff98a43)

• FlexBitSet

  • Use non-template type parameter for index parameters (7931dc2)

• GdbStub

  • Fix memory leak in GdbServerLoop() (749f1eb)

• General

  • Remove redundant string constructions in log calls related to IR op name retrieval (80caf90)

• Jits

  • Migrate logs over to fmt where applicable (7052d02)

• Linux

  • Implements support for vsyscall (a5b142c)
  • Implements rt_{tg,}sigqueueinfo (1a64c90)
  • Implements a base implementation of signalfd{4,} (2db44cf)
  • Remove logs about parent or child stack usage (336eb9a)

• OpDispatcher

  • Fixes imul flags calculations (7dd7003)

• OpcodeDispatcher

  • Split the opcode handling to multiple files (2fd41de)

• RCLSE

  • Fixes an assumption in RCLSE (896aa52)

• Misc

  • Linux: Implements support for signals 32 and 33 (7e8577a)
  • Wine fixes (de0337d)
  • Cleanup compile service on fork (a48e41e)
  • Adds an option to stall processes on launch (0315964)
  • Adds an option to disable jemalloc from cmake (dec8221)
  • binfmt_misc: Support loading ELFs from FD and support preserve (007b1cb)
  • Fixes bug in RAPass (b0a28e5)
  • Works around static-pie crashing (6277516)
  • Adds cmake option ENABLE_STATIC_PIE (aa767b3)
  • Fixes some issues I found when running under tsan and asan (14480f3)
  • Fixes binfmt_misc install when not installed to /usr (04aacea)
  • Validation Passes: Remove unused variables (7f5b352)
  • Fixes flag setting for 8bit and 16bit LOCK ALU ops (3a0da68)
  • Adds a convenience of named thunk configs (f4e044a)
  • Update man page to show JSON key for config options (108081e)
  • Fixes Host thunks lib not being able to GL (6ff9b96)
  • Allocator fixes (a24d63b)
  • Atomic improvements (53417e5)
  • Actually fix jemalloc for FEXConfig (35a4f52)
  • Switch some memory allocations over to mmap (1eee2dd)
  • Update jemalloc to fix glibc override. (0d9ab9c)
  • Fix jemalloc malloc replace (a9bb33b)
  • Fixes old kernel defines for sockios (8f09bd9)
  • Use externals xxhash if not found installed (0a8ee43)

FEX-2107

Compatibility & Bug Fixes

• Fixes bugs in unaligned atomic signal handlers
• CPUID cleanups
• Fixed a memory leak in Register Allocation
• Several syscall fixes (pidfd_send_signal, arch_prctl, fcntl, send(m)msg, shmdt, recvmsg, *chown32, edge cases around time syscalls, 32/tmpfile)
• Implement more syscalls (setfsuid32, setfsgid32, getgroups32, setgroups32, settimeofday, futimesat, utimes, 32/sigpending, 32/truncate64)
• Deferred signal handler registration, fixes bash (pts handover)
• Fixed handling of some rare elf files
• Implemented 32 bit iret

Usability

• Adds support for squashfs based rootfses
• cpack support for debian packaging

Performance

• Huge page support in our VA allocator

Misc

• Reduce warnings
• Several assorted Cleanups
• Relocate ELF handling logic to the os frontend
• Improve logging SNR

FEX-2106

Compatibility & Bug Fixes

• Several syscall bug fixes (pselect, epool, semctl, msgctl, pidfd_getfd, poll, fadvise64, sigaction, mmap, epoll, uname, openat, ...)
• Implemented SSE4.1

Performance

• Multi threaded AOTGen
• Optimized ioctl/drm marshaling

Misc

• Several minor cleanups & refactors

FEX-2105

Compatibility & Bug Fixes

• New, simpler ELF Loader
• ioctl32 marshaling for several devices
• Handle more cases of an application pinging self
• binfmt_misc fixes to support AppImage
• Several system call fixes (eventfd, eventfd2, openat, creat, trucnate, clock_nanosleep, pselect6, sendmsg, recvmsg)
• Signal fixes
• Better handling of /proc/self/, /proc/pid-self/
• Implements support for CLFLUSH

Performance

• Faster AOTIR file loading
• AOTIR offline generation
• Switch to xxhash from fasthash
• IR structure optimizations
• Adds Long Divide removal pass
• Zero-cost asserts

Misc

• Fixes host thread stacks ending up in lower 32-bit VA
• Remove reliance on librt and libnuma
• Default hidden visibility and strip symbols
• FEX now uses jemalloc