megboyzz/nfsmw-online
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

arm64: flat guest mapping, audio, crash reporting, self-extracting data

Browse Source 
The run of work that took the translated build from "boots" to "playable".

Engine:
- Flat guest mapping replaces the software MMU on aarch64 hosts. A 4 GiB
  PROT_NONE reservation lets a 32-bit guest address zero-extend safely, so
  tcg_out_qemu_ld/st short-circuit before tcg_out_tlb_read and the prologue
  materialises the base into X28. Measured 1.24x (51 vs 41 fps, interleaved
  A/B). Note the base must be set BEFORE UC_INIT - Unicorn inits lazily, and
  setting it after uc_open silently falls back to softmmu.
- num_get<char> facet implemented, which was the real cause of the crash after
  the prologue race; a full race is now playable end to end.
- Thread-stack free list + ReleaseThreadEngine, fixing the arena leak that
  showed up as a black screen when entering a race. kMaxGuestThreads 16 -> 64.
- Real ARM32 FMOD now runs in-engine via the Java FMODAudioDevice bridge, with
  a per-thread JNIEnv. Two of the three blockers were our own single-image-era
  guards.

Host/app:
- Native crash handler: async-signal-safe, decodes the host fault back to a
  guest address, writes a report file and nothing else. CrashReportActivity
  picks it up on the NEXT launch, zips it, and offers to share. No backend, no
  automatic upload.
- Game data ships inside the APK and self-extracts on first launch, so a tester
  installs one file and plays. Copy-to-.part-then-rename, with a free-space
  check up front.
- EGL context preserved across pause, fixing black textures on resume.
- Navigation bar hidden and re-hidden on focus gain; volume keys reported as
  system keys, checked before the loading-state gate.
- x86_64 added to abiFilters: the ARM32 guest runs under tcg/i386 with no
  houdini in the path. The flat mapping is aarch64-only, so that host falls
  back to the software MMU - commented at the abiFilters line.

Ignore rules added for app/translated/ (611 MB of signed release APK, which
also carries the bundled OBB) and ostream_repro/build/.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
megboyzz
2026-09-22 23:17:47 +03:00
co-authored by Claude
parent 80768652ea
commit 725ffbd8ed
44 changed files with 3096 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
mpcore/build.gradle.kts
+5 -1
View File
@@ -11,7 +11,11 @@ android {
minSdk = 27
ndk {
abiFilters += listOf("armeabi-v7a")
// ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md):
// mpcore itself is now plain native ARM64 code that drives an
// embedded ARM32 CPU-emulation core (Unicorn) instead of being
// ARM32 code injected into a natively-loaded libapp.so.
abiFilters += listOf("arm64-v8a", "x86_64")
}
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
mpcore/scripts/test_on_device.sh
Executable
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env bash
# Repeatable on-device smoke test for the ARM32-in-ARM64 prototype (see
# /ARM64_TRANSLATION_LAYER.md). Replaces the manual adb-shell archaeology
# this project's live debugging sessions have needed up to now: install,
# launch, watch logcat for a bounded window, then report the same signals
# those sessions kept hand-checking - onCreate timing, MEM FAULT count (see
# guest_engine.cpp's mem_fault_hook_cb), and process-death cause (ANR / OOM /
# MIUI SwipeUpClean / other) - as one command, so every future interface-
# layer change gets checked against the same baseline instead of a fresh
# investigation each time.
#
# Usage: mpcore/scripts/test_on_device.sh [device-serial] [watch-seconds]
# device-serial defaults to the first device `adb devices` lists.
# watch-seconds defaults to 180 (most observed onCreate runs finish in
# 100-120s; this leaves headroom to also see the post-onCreate lifecycle
# calls that previously kept faulting).
set -uo pipefail
# Script lives at mpcore/scripts/ - project root (where app/ and mpcore/ are
# siblings) is two levels up.
cd "$(dirname "$0")/../.."
PKG="com.ea.games.nfs13_arm"
ACTIVITY="com.ea.ironmonkey.GameActivityMain"
# "translated" (the emulated arm64-v8a path) - see app/build.gradle.kts's
# product flavors, added for the native ARM32 tracing investigation
# (ARM64_TRANSLATION_LAYER.md). The pre-flavor-split path
# (app/build/outputs/apk/debug/app-debug.apk) is now a STALE, never-updated
# file left over from before that split - silently reinstalling it here
# instead of the real current build wasted real debugging time confirming
# this fix, so this path is now the single source of truth.
APK="app/build/outputs/apk/translated/debug/app-translated-debug.apk"
SERIAL="${1:-$(adb devices | awk 'NR==2{print $1}')}"
WATCH_SECONDS="${2:-180}"
if [[ -z "$SERIAL" ]]; then
echo "No adb device found. Connect the device and retry." >&2
exit 1
fi
ADB="adb -s $SERIAL"
if [[ ! -f "$APK" ]]; then
echo "APK not found at $APK - build it first (./gradlew :app:assembleDebug)." >&2
exit 1
fi
echo "== Installing $APK on $SERIAL =="
$ADB install -r "$APK" || { echo "install failed" >&2; exit 1; }
echo "== Launching =="
$ADB shell am force-stop "$PKG"
$ADB logcat -c
$ADB shell monkey -p "$PKG" -c android.intent.category.LAUNCHER 1 >/dev/null
sleep 2
PID=$($ADB shell pidof "$PKG" | tr -d '\r')
if [[ -z "$PID" ]]; then
echo "Process did not start." >&2
exit 1
fi
echo "pid=$PID"
echo "== Watching logcat for ${WATCH_SECONDS}s =="
LOG="$(mktemp)"
timeout "${WATCH_SECONDS}s" $ADB logcat --pid="$PID" > "$LOG" 2>/dev/null || true
echo ""
echo "===================== RESULTS ====================="
ONCREATE_LINE=$(grep "GameActivityMain onCreate took" "$LOG" | tail -1)
if [[ -n "$ONCREATE_LINE" ]]; then
echo "onCreate: $(echo "$ONCREATE_LINE" | grep -oE '[0-9]+ms')"
else
echo "onCreate: did not complete within ${WATCH_SECONDS}s"
fi
FAULT_COUNT=$(grep -c "MEM FAULT" "$LOG" || true)
echo "MEM FAULT lines: $FAULT_COUNT"
if [[ "$FAULT_COUNT" -gt 0 ]]; then
echo " first: $(grep "MEM FAULT" "$LOG" | head -1)"
echo " last: $(grep "MEM FAULT" "$LOG" | tail -1)"
fi
CRASHED_LINE=$(grep -c "refusing to run - engine already crashed" "$LOG" || true)
echo "engine crashed (fail-fast tripped): $([[ "$CRASHED_LINE" -gt 0 ]] && echo yes || echo no)"
STILL_ALIVE=$($ADB shell "pidof $PKG" | tr -d '\r')
if [[ -n "$STILL_ALIVE" ]]; then
echo "process status: alive (pid=$STILL_ALIVE) at end of watch window"
else
echo "process status: DEAD"
DEATH_LINE=$($ADB logcat -d 2>/dev/null | grep -E "Killing $PID:|$PID.*died->background" | tail -3)
if [[ -n "$DEATH_LINE" ]]; then
echo " cause:"
echo "$DEATH_LINE" | sed 's/^/ /'
else
echo " cause: unknown (no ActivityManager kill/death line found - check for a FATAL crash instead)"
$ADB logcat -d --pid="$PID" 2>/dev/null | grep -iE "FATAL|AndroidRuntime" | sed 's/^/ /'
fi
fi
echo "====================================================="
echo "Full log: $LOG"
mpcore/src/main/cpp/CMakeLists.txt
+1
View File
@@ -34,6 +34,7 @@ add_subdirectory(third_party/unicorn)
add_library(${CMAKE_PROJECT_NAME} SHARED
main.cpp
crash_handler.cpp
game_lifecycle_stubs.cpp
game_lifecycle_stubs_extra.cpp
game_lifecycle_stubs_extra2.cpp
mpcore/src/main/cpp/crash_handler.cpp
+283
View File
@@ -0,0 +1,283 @@
#include "crash_handler.h"
#include "util/util.h"
#include <cerrno>
#include <csignal>
#include <cstring>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/ucontext.h>
#include <unistd.h>
// ---- Everything below the handler boundary must be async-signal-safe ----
//
// The rule is narrow and unforgiving: inside the handler, only functions
// POSIX lists as async-signal-safe may be called. write(), open(), close(),
// _exit(), sigaction(), raise() are; snprintf(), malloc(), any C++ container,
// anything that takes a lock, and every JNI call are NOT.
//
// This project already paid for ignoring that once - see
// ARM64_TRANSLATION_LAYER.md's note that mmap() inside a SIGSEGV handler
// deadlocked on bionic. So: every string is built HERE, at install time, and
// the handler only appends bytes to a pre-opened path and formats integers
// with the hand-written helpers below.
namespace {
constexpr size_t kPathMax = 512;
constexpr size_t kStampMax = 256;
// Pre-built at install time. `g_installed` also guards against a second
// install and against the handler running before setup finished.
char g_reportPath[kPathMax];
char g_buildStamp[kStampMax];
bool g_installed = false;
// Guest memory window, for decoding a host fault address back to the guest
// address that produced it. Plain integers written once from the engine
// thread and only read afterwards.
uint64_t g_guestFlatBase = 0;
uint32_t g_guestRegionSize = 0;
// The alternate stack. A stack-overflow SIGSEGV cannot be handled on the
// stack that just overflowed, so the handler gets its own - allocated here,
// at install time, never inside the handler.
constexpr size_t kAltStackSize = 64 * 1024;
char g_altStack[kAltStackSize];
// Previous dispositions, so the default handler still runs afterwards and
// Android still produces its own tombstone. Our report is extra evidence, not
// a replacement for the system's.
struct sigaction g_prevActions[NSIG];
// ---- async-signal-safe output helpers ----
void SafeWrite(int fd, const char* s, size_t len) {
while (len > 0) {
ssize_t n = write(fd, s, len);
if (n <= 0) {
if (n < 0 && errno == EINTR) continue;
return; // nothing useful left to do inside a handler
}
s += n;
len -= (size_t)n;
}
}
void SafeStr(int fd, const char* s) {
if (!s) return;
size_t len = 0;
while (s[len]) len++;
SafeWrite(fd, s, len);
}
void SafeHex(int fd, uint64_t v) {
static const char kDigits[] = "0123456789abcdef";
char buf[19];
buf[0] = '0';
buf[1] = 'x';
int pos = 18;
buf[pos] = '\0';
if (v == 0) {
buf[--pos] = '0';
} else {
while (v && pos > 2) {
buf[--pos] = kDigits[v & 0xF];
v >>= 4;
}
}
SafeWrite(fd, "0x", 2);
SafeStr(fd, buf + pos);
}
void SafeDec(int fd, long v) {
char buf[24];
int pos = 23;
buf[pos] = '\0';
bool neg = v < 0;
unsigned long u = neg ? (unsigned long)(-v) : (unsigned long)v;
if (u == 0) buf[--pos] = '0';
while (u && pos > 0) {
buf[--pos] = (char)('0' + (u % 10));
u /= 10;
}
if (neg && pos > 0) buf[--pos] = '-';
SafeStr(fd, buf + pos);
}
const char* SignalName(int sig) {
switch (sig) {
case SIGSEGV: return "SIGSEGV (bad memory access)";
case SIGBUS: return "SIGBUS (misaligned or unmapped access)";
case SIGABRT: return "SIGABRT (abort - often a failed assertion or a JNI error)";
case SIGILL: return "SIGILL (illegal instruction)";
case SIGFPE: return "SIGFPE (arithmetic fault)";
default: return "unknown signal";
}
}
void WriteRegisters(int fd, void* ucontextRaw) {
if (!ucontextRaw) return;
auto* uc = static_cast<ucontext_t*>(ucontextRaw);
#if defined(__aarch64__)
const mcontext_t& mc = uc->uc_mcontext;
SafeStr(fd, "registers (host arm64):\n");
for (int i = 0; i < 31; i++) {
SafeStr(fd, " x");
SafeDec(fd, i);
SafeStr(fd, i < 10 ? " = " : " = ");
SafeHex(fd, mc.regs[i]);
SafeStr(fd, (i % 2) ? "\n" : " ");
}
SafeStr(fd, "\n sp = ");
SafeHex(fd, mc.sp);
SafeStr(fd, " pc = ");
SafeHex(fd, mc.pc);
SafeStr(fd, "\n");
// x28 is where the flat guest mapping keeps its base (task #61). Printing
// it makes the guest-address arithmetic below checkable by hand from the
// report alone.
SafeStr(fd, " x28 (guest base register) = ");
SafeHex(fd, mc.regs[28]);
SafeStr(fd, "\n");
#else
(void)uc;
SafeStr(fd, "registers: not captured on this architecture\n");
#endif
}
void CrashSignalHandler(int sig, siginfo_t* info, void* ucontextRaw) {
if (g_installed) {
// O_TRUNC, not O_APPEND: one pending report at a time. If a second
// crash happens before the first is collected, the newer one is the
// one worth having - it is the one the tester just saw.
int fd = open(g_reportPath, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd >= 0) {
SafeStr(fd, "=== NFSMW arm64 engine - native crash report ===\n\n");
SafeStr(fd, "build: ");
SafeStr(fd, g_buildStamp);
SafeStr(fd, "\n\nsignal: ");
SafeStr(fd, SignalName(sig));
SafeStr(fd, " (");
SafeDec(fd, sig);
SafeStr(fd, ")\n");
const uint64_t faultAddr = info ? (uint64_t)(uintptr_t)info->si_addr : 0;
SafeStr(fd, "fault address (host): ");
SafeHex(fd, faultAddr);
SafeStr(fd, "\n");
// The number that actually helps. A host fault address inside the
// guest window is really a guest pointer; reported raw it is
// meaningless, and nobody reading a tester's report will subtract
// the base by hand.
if (g_guestFlatBase && faultAddr >= g_guestFlatBase &&
faultAddr < g_guestFlatBase + 0x100000000ull) {
uint64_t guestAddr = faultAddr - g_guestFlatBase;
SafeStr(fd, "fault address (GUEST): ");
SafeHex(fd, guestAddr);
if (g_guestRegionSize && guestAddr >= g_guestRegionSize) {
SafeStr(fd, " <- BEYOND the mapped guest region (");
SafeHex(fd, g_guestRegionSize);
SafeStr(fd, ") - a wild pointer, not a real guest object");
}
SafeStr(fd, "\n");
} else if (g_guestFlatBase) {
SafeStr(fd, "fault address is OUTSIDE the guest window - this is a fault in the "
"engine's own native code, not in emulated guest code\n");
}
SafeStr(fd, "thread id: ");
SafeDec(fd, (long)gettid());
SafeStr(fd, "\n\n");
WriteRegisters(fd, ucontextRaw);
SafeStr(fd,
"\nnote: Android's own tombstone for this crash has more detail "
"(/data/tombstones). This file exists because a tester cannot reach that.\n");
close(fd);
}
}
// Chain to whatever was installed before us so the platform still writes
// its tombstone and the process dies the way it would have. Restoring the
// previous action and re-raising is the portable way to do that.
if (sig >= 0 && sig < NSIG) {
sigaction(sig, &g_prevActions[sig], nullptr);
}
raise(sig);
}
} // namespace
void SetCrashHandlerGuestBase(uint64_t flatBase, uint32_t regionSize) {
g_guestFlatBase = flatBase;
g_guestRegionSize = regionSize;
}
void InstallCrashHandler(const char* crashDir, const char* buildStamp) {
if (g_installed) return;
if (!crashDir || !*crashDir) {
Log("crash_handler: no crash directory given - native crashes will NOT be reported");
return;
}
// mkdir here, at install time. Doing it lazily from the handler would mean
// a filesystem call on a broken process, which is exactly what this file
// exists to avoid.
if (mkdir(crashDir, 0755) != 0 && errno != EEXIST) {
Log("crash_handler: could not create %s (%s) - native crashes will NOT be reported",
crashDir, strerror(errno));
return;
}
// Fixed filename. The handler cannot safely format a timestamp, so Java
// renames this to something unique when it collects it on the next launch.
static const char kFileName[] = "/crash_pending.txt";
size_t dirLen = strlen(crashDir);
if (dirLen + sizeof(kFileName) >= kPathMax) {
Log("crash_handler: crash directory path is too long (%zu) - not installing", dirLen);
return;
}
memcpy(g_reportPath, crashDir, dirLen);
memcpy(g_reportPath + dirLen, kFileName, sizeof(kFileName));
if (buildStamp) {
size_t n = strlen(buildStamp);
if (n >= kStampMax) n = kStampMax - 1;
memcpy(g_buildStamp, buildStamp, n);
g_buildStamp[n] = '\0';
} else {
memcpy(g_buildStamp, "(unknown)", sizeof("(unknown)"));
}
stack_t ss{};
ss.ss_sp = g_altStack;
ss.ss_size = sizeof(g_altStack);
ss.ss_flags = 0;
if (sigaltstack(&ss, nullptr) != 0) {
// Not fatal: without it, a stack-overflow crash goes unreported, but
// every other kind still works. Worth saying out loud rather than
// discovering the gap from a missing report later.
Log("crash_handler: sigaltstack failed (%s) - stack-overflow crashes will not be "
"reported, other crashes still will", strerror(errno));
}
struct sigaction sa{};
sa.sa_sigaction = CrashSignalHandler;
sa.sa_flags = SA_SIGINFO | SA_ONSTACK | SA_RESTART;
sigemptyset(&sa.sa_mask);
static const int kSignals[] = {SIGSEGV, SIGBUS, SIGABRT, SIGILL, SIGFPE};
for (int sig : kSignals) {
if (sigaction(sig, &sa, &g_prevActions[sig]) != 0) {
Log("crash_handler: could not hook signal %d (%s)", sig, strerror(errno));
}
}
g_installed = true;
Log("crash_handler: installed - native crashes will be written to %s", g_reportPath);
}
mpcore/src/main/cpp/crash_handler.h
+33
View File
@@ -0,0 +1,33 @@
#pragma once
// Native crash reporting (task #66 / BETA_TELEMETRY_PLAN.md).
//
// Most crashes in this project are NATIVE - a SIGSEGV inside JIT-generated
// code - so Java's uncaught-exception handler never sees them and the tester
// sees only "the game closed". This writes a report the moment it happens.
//
// What it does NOT do, deliberately: show any UI. A signal handler runs on a
// process that has already gone wrong, where only async-signal-safe calls are
// legal - no malloc, no JNI, no Activity. It writes one file with write() and
// then lets the process die. The report screen is shown by Java on the NEXT
// launch, which is how every serious crash reporter handles native crashes.
//
// The report lands on external storage (Android/data/<pkg>/files/crashes), so
// a tester can reach it over USB or a file manager without any permission.
#include <cstdint>
// Call once, early, with the directory reports should be written to (the app's
// external files dir). Creates the directory if needed, pre-builds every string
// the handler will need, installs an alternate signal stack so a stack-overflow
// crash can still be reported, and hooks the fatal signals.
//
// Safe to call more than once; only the first call installs anything.
void InstallCrashHandler(const char* crashDir, const char* buildStamp);
// Tells the handler where guest memory starts, so a host fault address can be
// reported as the GUEST address that caused it. Without this a report says
// "fault at 0x6f464c459b", which means nothing to anyone; with it the report
// also says "guest 0x464c459b", which is the number worth reading. Called by
// GuestEngine once its region is mapped.
void SetCrashHandlerGuestBase(uint64_t flatBase, uint32_t regionSize);
mpcore/src/main/cpp/emu/guest_engine.cpp
+339 -12
View File
@@ -4,11 +4,14 @@
#include "zlib_accel.h"
#include "name_lookup_accel.h"
#include "../util/util.h"
#include "../crash_handler.h"
#include <cstdio>
#include <cstring>
#include <cerrno>
#include <cstdlib>
#include <sys/mman.h>
#include <dlfcn.h>
#include <set>
#include <array>
#include <string>
#include <thread>
@@ -133,6 +136,13 @@ constexpr uint32_t kGuardPageSize = kPageSize;
// TCG to re-translate (tb_invalidate_phys_page_fast_arm, 3.15%). If this
// reports zero, the invalidation comes from somewhere else and that lead
// is dead.
// Task #61: bypass the software MMU and let generated code reach guest memory
// as `[X28, Wn, UXTW]`. Kept as an explicit switch because it trades away real
// safety nets (UC_PROT_* enforcement, UC_HOOK_MEM_* callbacks, self-modifying
// code detection) - when chasing a memory-corruption bug, turn it OFF to get
// MEM FAULT diagnostics back, then turn it on again.
constexpr bool kUseFlatGuestMapping = true;
constexpr bool kCountStubDispatches = false;
constexpr bool kCountTextWrites = false;
// kCountArenaWrites - task #54. tb_invalidate_phys_page_fast_arm is 5.56% of
@@ -169,7 +179,13 @@ constexpr uint32_t kControlArenaSize = 256 * 1024;
// engine is not expected to spawn anywhere near this many worker threads -
// generous headroom, not a measured requirement. CarveThreadStack() fails
// (logged, not fatal) if this is ever exceeded.
constexpr uint32_t kMaxGuestThreads = 16;
// Raised 16 -> 64 (2026-09-21) alongside ReleaseThreadEngine's free list.
// The free list is the actual fix for the exhaustion that showed up as a black
// screen entering a race; this is headroom for the genuinely-concurrent peak,
// since a single thread can hold one primary stack plus up to
// kMaxNestedEngines nested ones at the same time. The arena is lazily-mapped
// address space (see MapSegments), so unused entries cost no resident memory.
constexpr uint32_t kMaxGuestThreads = 64;
constexpr uint32_t kThreadStacksArenaSize = kMaxGuestThreads * kStackSize;
// AllocMmap's arena - backs real anonymous mmap() (see libc_shims.cpp's
@@ -186,6 +202,25 @@ constexpr uint32_t kArmLdrPcPcMinus4 = 0xE51FF004u;
struct ImportStubEntry {
std::string name;
ImportShimFn shim; // nullptr if not implemented - see guest_engine's own unresolved-import log-once behavior
// Task #67. Non-zero means "this symbol IS implemented, by real guest code
// in a secondary image, and we want to watch the call". The dispatcher
// logs the arguments, forwards to that address, and logs what comes back.
// Used to find where the game's FMOD initialisation chain stops: the three
// entry points libapp.so imports all return an FMOD_RESULT, so the first
// non-zero one names the failure.
//
// Deliberately on OUR side of the boundary. The same thing attempted from
// outside, via an LD_PRELOAD interposer on the A9, could not resolve the
// real symbols and ended up standing in for them - silencing audio on the
// reference device. Here the real address is known, so forwarding cannot
// silently degrade into replacing.
GuestAddr tracedGuestTarget = 0;
// How many arguments the traced function really takes. AAPCS32 puts the
// first four in r0-r3 and the REST ON THE STACK, so forwarding only the
// registers silently truncates the call. The first version of this probe
// did exactly that and produced error codes that were its own doing -
// FMOD_Memory_Initialize takes six arguments, EventSystem::init five.
int tracedArgCount = 4;
bool loggedUnresolved = false;
// Task #56 instrumentation (2026-09-19, temporary). Live measurement on
// the Xiaomi 14 showed shim crossings/sec pinned near a ceiling (~170k)
@@ -2289,7 +2324,29 @@ void import_stub_dispatch_cb(uc_engine* uc, uint64_t address, uint32_t /*size*/,
}
uint32_t result = 0;
if (entry->shim) {
if (entry->tracedGuestTarget) {
GuestEngine& eng = GuestEngine::Instance();
// Rebuild the WHOLE argument list, stack arguments included, before
// forwarding - see tracedArgCount's own comment for what truncating it
// cost the first time.
uint32_t args[8] = {r0, r1, r2, r3, 0, 0, 0, 0};
int n = entry->tracedArgCount;
if (n < 1) n = 1;
if (n > 8) n = 8;
for (int i = 4; i < n; i++) {
args[i] = eng.ReadIncomingArg(i, r0, r1, r2, r3, sp);
}
std::string argText;
for (int i = 0; i < n; i++) {
char buf[24];
snprintf(buf, sizeof(buf), "%s0x%x", i ? ", " : "", args[i]);
argText += buf;
}
Log("GUESTCALL %s(%s) - entering real guest code at 0x%x",
entry->name.c_str(), argText.c_str(), entry->tracedGuestTarget);
result = eng.CallGuestFunction(entry->tracedGuestTarget, args, n);
Log("GUESTCALL %s -> 0x%x", entry->name.c_str(), result);
} else if (entry->shim) {
result = entry->shim(GuestEngine::Instance(), r0, r1, r2, r3, sp);
} else {
if (!entry->loggedUnresolved) {
@@ -2424,6 +2481,19 @@ bool GuestEngine::LoadImage(const char* path) {
// addresses by the time any GOT slot references them.
for (auto fn : data_symbol_setup_fns_) fn(*this);
EnsureThreadEngine();
// Sibling guest libraries, loaded HERE - after the engine exists (they
// need one to build their own import stubs) but BEFORE this image's
// relocations are processed. The ordering is the whole point: a GOT
// slot resolved against a shim cannot be un-resolved later, so anything
// that should reach real code has to be exported before the first
// relocation is applied (task #67).
//
// Failure is not fatal. A missing library leaves fmod_shims.cpp's
// no-ops in charge, which is exactly the behaviour that shipped before
// this - the game runs, silently, rather than not at all.
if (t_state_.uc) LoadSiblingLibraries(path);
ok = t_state_.uc != nullptr && ProcessRelocations(fileData, fileSize);
}
free(fileData);
@@ -2509,13 +2579,42 @@ bool GuestEngine::MapSegments(const uint8_t* fileData, size_t fileSize) {
mmap_end_ = mmap_cursor_ + kMmapArenaSize;
region_size_ = AlignUp(mmap_end_ + kPageSize, kPageSize);
void* backing = mmap(nullptr, region_size_, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
// Reserve the FULL 4 GiB a 32-bit guest address can name, then commit the
// part we actually use at its start (task #61).
//
// Why reserve the whole span rather than just region_size_: with the flat
// mapping enabled (uc_set_flat_map_base), generated code reaches guest
// memory as `[X28, Wn, UXTW]` - the guest address is zero-extended and
// added with no bounds check whatsoever. A wild guest pointer beyond
// region_size_ (we have hit real ones, e.g. 0x3d3d3d3d in task #33) would
// otherwise land on whatever unrelated host mapping happens to sit there
// and be read or WRITTEN silently. Holding the whole 4 GiB as PROT_NONE
// turns every such access into an immediate, honest SIGSEGV instead.
//
// This costs address space, not memory: PROT_NONE|MAP_NORESERVE commits no
// pages, and the host is 64-bit so 4 GiB of VA is unremarkable.
constexpr size_t kGuestAddressSpace = 4ull * 1024 * 1024 * 1024;
void* reservation = mmap(nullptr, kGuestAddressSpace, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
if (reservation == MAP_FAILED) {
Log("GuestEngine: reserving the 4GiB guest address space failed (%s) - cannot continue",
strerror(errno));
return false;
}
void* backing = mmap(reservation, region_size_, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
if (backing == MAP_FAILED) {
Log("GuestEngine: mmap(0x%x) for guest region failed", region_size_);
Log("GuestEngine: mmap(0x%x) for guest region failed (%s)", region_size_, strerror(errno));
munmap(reservation, kGuestAddressSpace);
return false;
}
host_region_ = static_cast<uint8_t*>(backing);
// The crash handler needs this to turn a host fault address back into the
// guest address that caused it - see crash_handler.h. Set as soon as the
// region exists, so a crash during the rest of loading is already decodable.
SetCrashHandlerGuestBase((uint64_t)(uintptr_t)host_region_, region_size_);
// No uc_engine created here anymore - each host thread gets its own,
// mapped onto this same host_region_, the first time it needs one (see
// EnsureThreadEngine). This call (LoadImage's caller) will get its
@@ -2686,6 +2785,36 @@ bool GuestEngine::ProcessRelocations(const uint8_t* fileData, size_t fileSize) {
// of a clean crash - a small, self-contained, independently-readable
// function is safer here than a shared helper parameterized just enough to
// serve both call sites' different assumptions.
void GuestEngine::LoadSiblingLibraries(const char* primaryImagePath) {
// libfmodex FIRST: libfmodevent lists it in its own DT_NEEDED, so its
// relocations only resolve to real code if libfmodex's exports are already
// recorded. Load order here IS dependency order.
static const char* kSiblings[] = { "libfmodex.so", "libfmodevent.so" };
std::string dir(primaryImagePath ? primaryImagePath : "");
size_t slash = dir.find_last_of('/');
dir = (slash == std::string::npos) ? std::string(".") : dir.substr(0, slash);
// Every instruction in libfmodex that loads FMOD_RESULT 33 into a register
// (found by scanning the disassembly for the constant - 39 sites, far too
// many to reason about by hand). Watching them all at once turns "which
// one fires" into an observation instead of an argument. Offsets are
// file-relative and biased by the load address below.
//
// Temporary, for the audio bring-up (task #67) - delete the list to remove
// the hooks entirely.
for (const char* lib : kSiblings) {
std::string full = dir + "/" + lib;
if (LoadSecondaryImage(full.c_str(), nullptr)) continue;
// Say which one and what it costs. A silent miss here would present
// later as "the game runs but makes no sound", with nothing pointing
// back to the real cause.
Log("GuestEngine::LoadSiblingLibraries: could not load %s - its entry points stay on the "
"no-op shims in fmod_shims.cpp, so expect no audio from it", full.c_str());
}
}
GuestAddr GuestEngine::LoadSecondaryImage(const char* path, const char* entrySymbol) {
if (!host_region_) {
Log("GuestEngine::LoadSecondaryImage(%s): no primary image loaded yet (host_region_ null) - "
@@ -2890,24 +3019,67 @@ GuestAddr GuestEngine::LoadSecondaryImage(const char* path, const char* entrySym
for (size_t s = 0; s < symCount; s++) {
if (syms[s].st_shndx == SHN_UNDEF) continue;
const char* name = strs + syms[s].st_name;
if (strcmp(name, entrySymbol) == 0) {
entryAddr = base + syms[s].st_value; // st_value already carries the Thumb bit, same convention as everywhere else
break;
if (!name || !*name) continue;
// st_value already carries the Thumb bit, same convention as everywhere else.
GuestAddr addr = base + syms[s].st_value;
// Record the export so a later import of this name reaches the
// real code. First definition wins, matching how a real dynamic
// linker resolves against load order - and a collision is worth
// saying out loud rather than silently preferring one.
auto existing = secondary_image_exports_.find(name);
if (existing == secondary_image_exports_.end()) {
secondary_image_exports_[name] = addr;
} else if (existing->second != addr) {
Log("GuestEngine::LoadSecondaryImage(%s): symbol '%s' already exported at 0x%x by an "
"earlier image - keeping the first, ignoring 0x%x", path, name, existing->second, addr);
}
if (entrySymbol && !entryAddr && strcmp(name, entrySymbol) == 0) {
entryAddr = addr;
}
}
}
}
free(fileData);
if (!entryAddr) {
if (entrySymbol && !entryAddr) {
Log("GuestEngine::LoadSecondaryImage: entry symbol '%s' not found in %s's .dynsym - "
"was it declared extern \"C\" with default visibility?", entrySymbol, path);
return 0;
}
Log("GuestEngine::LoadSecondaryImage: loaded %s at base=0x%x span=0x%x - entry '%s'=0x%x",
path, base, span, entrySymbol, entryAddr);
return entryAddr;
secondary_image_ranges_.push_back({base, span});
Log("GuestEngine::LoadSecondaryImage: loaded %s at base=0x%x span=0x%x - entry '%s'=0x%x, "
"%zu exported symbol(s) now available to later imports",
path, base, span, entrySymbol ? entrySymbol : "(none requested)", entryAddr,
secondary_image_exports_.size());
return entrySymbol ? entryAddr : 1;
}
// Symbols whose calls are logged and forwarded rather than branched to
// directly (task #67, temporary). Empty this list to remove the overhead -
// each entry costs a host crossing per call, so only initialisation-time
// functions belong here, never anything on a per-frame path.
// Argument counts come from FMOD's public headers, not from guesswork:
// FMOD_Memory_Initialize(poolmem, poollen, useralloc, userrealloc,
// userfree, memtypeflags) - 6
// FMOD_EventSystem_Create(eventsystem) - 1
// EventSystem::init(this, maxchannels, flags, extradriverdata,
// eventflags) - 5
static int TracedGuestSymbolArgCount(const std::string& name) {
if (name == "FMOD_Memory_Initialize") return 6;
if (name == "FMOD_EventSystem_Create") return 1;
if (name == "_ZN4FMOD11EventSystem4initEijPvj") return 5;
// System::init(this, maxchannels, flags, extradriverdata) - libfmodevent
// calls this INTO libfmodex, so it crosses an image boundary and goes
// through import resolution like any other. Watched to separate "the event
// layer refused" from "the core system refused".
if (name == "_ZN4FMOD6System4initEijPv") return 4;
return 0; // not traced
}
static bool IsTracedGuestSymbol(const std::string& name) {
return TracedGuestSymbolArgCount(name) > 0;
}
GuestAddr GuestEngine::ResolveOrCreateImportStub(const std::string& symbolName) {
@@ -2925,6 +3097,44 @@ GuestAddr GuestEngine::ResolveOrCreateImportStub(const std::string& symbolName)
return dataIt->second;
}
// A symbol DEFINED by a secondary guest image is real code - branch
// straight to it, no stub and no host crossing. Checked before the shim
// table on purpose: once the game's own FMOD is loaded, its entry points
// must win over fmod_shims.cpp's no-op stand-ins.
auto exportIt = secondary_image_exports_.find(symbolName);
if (exportIt != secondary_image_exports_.end()) {
if (registered_shims_.count(symbolName)) {
Log("GuestEngine: '%s' resolves to real guest code at 0x%x from a secondary image - "
"the registered shim for it is now unused", symbolName.c_str(), exportIt->second);
}
// Watched symbols get a stub that forwards; everything else branches
// straight to the real code with no stub and no host crossing.
if (!IsTracedGuestSymbol(symbolName)) {
import_stub_by_symbol_[symbolName] = exportIt->second;
return exportIt->second;
}
auto* traced = new ImportStubEntry();
traced->name = symbolName;
traced->shim = nullptr;
traced->tracedGuestTarget = exportIt->second;
traced->tracedArgCount = TracedGuestSymbolArgCount(symbolName);
{
std::lock_guard<std::mutex> lock(g_importStubEntriesMutex);
g_importStubEntries.push_back(traced);
}
GuestAddr tracedStub = AllocCodeStub(import_stub_dispatch_cb, traced);
if (!tracedStub) {
Log("GuestEngine: could not allocate a tracing stub for '%s' - calling it directly, "
"untraced", symbolName.c_str());
import_stub_by_symbol_[symbolName] = exportIt->second;
return exportIt->second;
}
Log("GuestEngine: watching '%s' - calls will be logged and forwarded to 0x%x",
symbolName.c_str(), exportIt->second);
import_stub_by_symbol_[symbolName] = tracedStub;
return tracedStub;
}
auto* entry = new ImportStubEntry();
entry->name = symbolName;
{
@@ -2960,6 +3170,30 @@ uc_engine* GuestEngine::CreateConfiguredEngine() {
Log("GuestEngine::CreateConfiguredEngine: uc_open failed: %d", (int)err);
return nullptr;
}
// Task #61: tell the JIT it can reach guest memory by plain addition.
//
// Must happen here, before anything is translated on this engine. Our
// guest address space is already exactly what the flat path requires - one
// contiguous host block where G2H(a) == host_region_ + a - so the software
// TLB is pure overhead: a nine-instruction check per access, measured at
// 29.8% of all generated host code.
//
// The price, paid knowingly: UC_PROT_* is no longer enforced on data
// accesses, UC_HOOK_MEM_* stops firing (so MEM FAULT diagnostics go
// quiet), and guest stores no longer invalidate translated code. The
// 4 GiB PROT_NONE reservation in MapSegments is what keeps a wild guest
// pointer from silently touching unrelated host memory.
if (kUseFlatGuestMapping) {
uc_err flatErr = uc_set_flat_map_base(newUc, (uint64_t)(uintptr_t)host_region_);
if (flatErr != UC_ERR_OK) {
// Not fatal - the software MMU still works - but it means this
// engine silently runs several times slower than its siblings,
// which is exactly the kind of thing that must not pass in silence.
Log("GuestEngine::CreateConfiguredEngine: uc_set_flat_map_base failed: %d - this "
"engine falls back to the software MMU and will be much slower", (int)flatErr);
}
}
// uc_mem_map_ptr shares OUR host buffer directly as the guest's backing
// store, instead of Unicorn allocating its own - this is what makes
// G2H/H2G plain pointer arithmetic (see guest_engine.h's class comment).
@@ -3419,6 +3653,15 @@ void GuestEngine::ReplayHooksOnEngine(uc_engine* newUc) {
GuestAddr GuestEngine::CarveThreadStack() {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
// Reuse a returned stack before growing the arena. The guest does not read
// a fresh stack before writing it, so no scrubbing is needed here - and
// leaving the previous thread's bytes in place has caught real
// uninitialized-read bugs before.
if (!thread_stack_free_list_.empty()) {
GuestAddr top = thread_stack_free_list_.back();
thread_stack_free_list_.pop_back();
return top;
}
if (thread_stacks_cursor_ + kStackSize > thread_stacks_end_) {
return 0;
}
@@ -3427,6 +3670,37 @@ GuestAddr GuestEngine::CarveThreadStack() {
return base + kStackSize; // top = highest usable address, same convention the old stack_top_ used
}
void GuestEngine::ReleaseThreadEngine() {
// Nested-call engines first: each one holds its OWN dedicated stack
// (GetOrCreateNestedEngine), and those leaked exactly like the primary
// one did - a single guest thread could consume several stacks out of the
// arena and give none of them back.
for (uint32_t i = 0; i < kMaxNestedEngines; i++) {
if (t_state_.nestedEngines[i]) {
uc_close(t_state_.nestedEngines[i]);
t_state_.nestedEngines[i] = nullptr;
}
if (t_state_.nestedStackTop[i]) {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
thread_stack_free_list_.push_back(t_state_.nestedStackTop[i]);
t_state_.nestedStackTop[i] = 0;
}
}
if (!t_state_.uc) return;
uc_close(t_state_.uc);
t_state_.uc = nullptr;
if (t_state_.stackTop) {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
thread_stack_free_list_.push_back(t_state_.stackTop);
}
t_state_.stackBase = 0;
t_state_.stackTop = 0;
t_state_.callDepth = 0;
t_state_.lastCallHiWord = 0;
}
// Task #56: every shim crossing (libc, JNI and GLES alike) passes through
// here, so one relaxed increment gives a total that gles_shim's PERF line can
// print alongside fps and draws in a SINGLE log line - which is the whole
@@ -4055,6 +4329,59 @@ void GuestEngine::RegisterDataSymbol(const char* symbolName, GuestAddr address)
registered_data_symbols_[symbolName] = address;
}
namespace {
void watch_address_cb(uc_engine*, uint64_t address, uint32_t, void* user_data) {
auto* label = static_cast<const char*>(user_data);
// Once per address: these sit on error paths, and an error path that runs
// in a loop would otherwise bury everything else in the log.
static std::mutex seenMutex;
static std::set<uint64_t> seen;
{
std::lock_guard<std::mutex> lock(seenMutex);
if (!seen.insert(address).second) return;
}
Log("WATCH reached guest 0x%llx - %s", (unsigned long long)address, label ? label : "?");
}
} // namespace
void GuestEngine::WatchGuestAddress(GuestAddr addr, const char* label) {
if (!addr) return;
char* owned = strdup(label ? label : "");
{
std::lock_guard<std::mutex> lock(hook_registrations_mutex_);
hook_registrations_.push_back({watch_address_cb, owned, addr});
}
if (t_state_.uc) {
uc_hook h;
uc_err err = uc_hook_add(t_state_.uc, &h, UC_HOOK_CODE, (void*)watch_address_cb, owned,
addr, addr);
if (err != UC_ERR_OK) {
Log("GuestEngine::WatchGuestAddress(0x%x): uc_hook_add failed: %d", addr, (int)err);
}
}
}
bool GuestEngine::IsGuestImageCode(GuestAddr addr) const {
if (addr && addr < image_end_) return true;
for (const auto& r : secondary_image_ranges_) {
if (addr >= r.base && addr < r.base + r.span) return true;
}
return false;
}
GuestAddr GuestEngine::LookupSecondaryExport(const char* symbolName) const {
if (!symbolName) return 0;
auto it = secondary_image_exports_.find(symbolName);
return it == secondary_image_exports_.end() ? 0 : it->second;
}
const char* GuestEngine::NameForDataSymbol(GuestAddr address) const {
for (const auto& entry : registered_data_symbols_) {
if (entry.second == address) return entry.first.c_str();
}
return nullptr;
}
std::string GuestEngine::DescribeAddress(GuestAddr addr) const {
char buf[128];
if (addr >= region_size_) {
mpcore/src/main/cpp/emu/guest_engine.h
+71 -1
View File
@@ -126,8 +126,24 @@ public:
// other resolved address in this engine) ready to hand straight to
// CallGuestFunction, or 0 on any failure (bad ELF, arena exhaustion,
// symbol not found - all logged).
// Loads an additional guest ELF alongside the primary image and records
// everything it EXPORTS, so a later import of one of those names resolves
// to the real code instead of a shim (see ResolveOrCreateImportStub).
//
// Pass entrySymbol = nullptr when the caller only wants the library loaded
// and does not need one specific address back - which is the case for the
// game's own FMOD libraries, whose entry points are reached through
// libapp.so's ordinary imports rather than by name from the host.
// Returns the entry symbol's address, or 1 for "loaded, no entry requested",
// or 0 on failure.
GuestAddr LoadSecondaryImage(const char* path, const char* entrySymbol);
// Loads the guest libraries that sit alongside the primary image in the
// same directory - currently the game's own FMOD. Called from LoadImage at
// the one point where it is both possible and still useful; see the call
// site's own comment for why the ordering is not negotiable.
void LoadSiblingLibraries(const char* primaryImagePath);
bool loaded() const { return host_region_ != nullptr; }
// ---- Address translation ----
@@ -289,6 +305,37 @@ public:
// ResolveOrCreateImportStub BEFORE falling back to a code stub.
void RegisterDataSymbol(const char* symbolName, GuestAddr address);
// Reverse of RegisterDataSymbol: which symbol, if any, was given this
// address. Exists so a shim that cannot honour a request can NAME what it
// was asked for instead of printing a bare arena address - the facet
// addresses handed out by AllocPermanent mean nothing on their own, and
// resolving one otherwise costs a rebuild and a reproduction (see
// Shim_use_facet). Linear scan: this is an error path only.
// Returns nullptr when the address is not a registered data symbol.
const char* NameForDataSymbol(GuestAddr address) const;
// Address of a symbol defined by a secondary guest image (see
// LoadSecondaryImage), or 0 if no loaded image exports that name. Lets
// host-side JNI entry points forward into real guest code - see
// game_lifecycle_stubs_extra2.cpp's FMOD audio bridge.
GuestAddr LookupSecondaryExport(const char* symbolName) const;
// Logs once when execution first reaches `addr`, without altering control
// flow - a pure observation point, unlike InstallTrampolineHookRaw which
// displaces instructions. Used to answer "which of these N places produced
// the error code" by watching every candidate at once instead of reasoning
// about which one is reachable.
//
// `label` is copied and used verbatim in the log line.
void WatchGuestAddress(GuestAddr addr, const char* label);
// True if `addr` lies inside ANY loaded guest image - the primary one or
// a sibling loaded by LoadSecondaryImage. Callers that need to tell "real
// guest code" from "one of this engine's own arenas" must ask this rather
// than comparing against image_end(), which only ever described the
// primary image and silently rejected every sibling.
bool IsGuestImageCode(GuestAddr addr) const;
// Registers a callback GuestEngine invokes exactly once per LoadImage
// call, right after MapSegments succeeds (so host_region_/
// AllocPermanent are usable) but strictly BEFORE ProcessRelocations
@@ -354,6 +401,13 @@ public:
// before that thread can run any guest code at all.
void EnsureThreadEngine();
// Tears down everything EnsureThreadEngine (and any nested-call engine)
// set up for the CALLING host thread, returning its guest stacks to the
// arena for reuse. Must be called by any thread that created an engine
// and is about to exit - pthread_shim's thread body does. Safe to call on
// a thread that never had one.
void ReleaseThreadEngine();
// Reserved, never-fetched guest PC used as the "return to caller"
// target for every CallGuestFunction invocation (see .cpp for why a
// fixed constant is safe to reuse across nested calls).
@@ -515,7 +569,15 @@ private:
GuestAddr thread_stacks_cursor_ = 0, thread_stacks_end_ = 0; // CarveThreadStack arena
GuestAddr mmap_cursor_ = 0, mmap_end_ = 0; // AllocMmap arena - see its own comment
std::mutex thread_stacks_mutex_; // guards thread_stacks_cursor_ (concurrent pthread_create calls)
std::mutex thread_stacks_mutex_; // guards thread_stacks_cursor_ and the free list below
// Stacks handed back by ReleaseThreadEngine, ready to be reused.
//
// Without this the arena was a one-way bump allocator: every guest thread
// that finished kept its stack forever, and so did every nested-call
// engine, so a long session simply ran out. Confirmed live - the game
// created its race thread, CarveThreadStack returned 0, the thread's
// start routine never ran, and the race rendered as a black screen.
std::vector<GuestAddr> thread_stack_free_list_;
std::mutex control_mutex_; // guards control_cursor_
std::mutex mmap_mutex_; // guards mmap_cursor_
@@ -523,6 +585,14 @@ private:
std::unordered_map<std::string, ImportShimFn> registered_shims_;
std::unordered_map<std::string, GuestAddr> registered_data_symbols_; // see RegisterDataSymbol
// Symbols DEFINED by secondary guest images (see LoadSecondaryImage).
// Consulted by ResolveOrCreateImportStub ahead of the shim table, so that
// once the game's real ARM32 FMOD is loaded its own entry points win over
// the no-op stand-ins fmod_shims.cpp registers.
std::unordered_map<std::string, GuestAddr> secondary_image_exports_;
// [base, base+span) for each image LoadSecondaryImage has loaded.
struct LoadedImageRange { GuestAddr base; uint32_t span; };
std::vector<LoadedImageRange> secondary_image_ranges_;
std::vector<DataSymbolSetupFn> data_symbol_setup_fns_; // see RegisterDataSymbolSetup
std::unordered_map<std::string, GuestAddr> import_stub_by_symbol_; // dedupe: one stub per symbol name
std::unordered_map<GuestAddr, ImportShimFn> shim_by_stub_addr_;
mpcore/src/main/cpp/emu/jni_shim.cpp
+25 -5
View File
@@ -317,13 +317,12 @@ std::string GuestCStr(GuestEngine& eng, uint32_t guestPtr) {
} // namespace
std::atomic<uint32_t> JniHandleTable::g_callEpoch{0};
thread_local uint32_t JniHandleTable::t_callEpoch = 0;
uint32_t JniHandleTable::Alloc(void* real, bool isGlobal) {
if (!real) return 0;
std::lock_guard<std::mutex> lock(mutex_);
table_.push_back(Entry{real, std::this_thread::get_id(), isGlobal,
g_callEpoch.load(std::memory_order_relaxed)});
table_.push_back(Entry{real, std::this_thread::get_id(), isGlobal, t_callEpoch});
return (uint32_t)(table_.size() - 1);
}
@@ -344,8 +343,10 @@ bool JniHandleTable::IsSafeToUseFromCurrentThread(uint32_t handle) const {
// nativeOnRunLoopTick calls SetRealEnv/BumpCallEpoch on every single
// entry, so a handle from an earlier tick on the SAME thread is just
// as stale as one from a different thread).
return e.owner == std::this_thread::get_id() &&
e.epoch == g_callEpoch.load(std::memory_order_relaxed);
// Both halves are per-thread now: the ref must have been created by THIS
// thread, during the native call this thread is currently inside. Another
// thread entering or leaving a native call cannot affect either test.
return e.owner == std::this_thread::get_id() && e.epoch == t_callEpoch;
}
thread_local JNIEnv* JniShim::real_env_ = nullptr;
@@ -1734,6 +1735,11 @@ uint32_t Impl_NewDirectByteBuffer(GuestEngine& eng, uint32_t, uint32_t r1, uint3
g_directBufferGuestAddrs[h] = r1;
return h;
}
void RegisterDirectBufferGuestAddr(uint32_t handle, GuestAddr guestAddr);
void RegisterDirectBufferGuestAddr(uint32_t handle, GuestAddr guestAddr) {
g_directBufferGuestAddrs[handle] = guestAddr;
}
uint32_t Impl_GetDirectBufferAddress(GuestEngine&, uint32_t, uint32_t r1, uint32_t, uint32_t, uint32_t) {
auto it = g_directBufferGuestAddrs.find(r1);
if (it == g_directBufferGuestAddrs.end()) {
@@ -2143,6 +2149,20 @@ uint32_t Impl_RegisterNatives(GuestEngine& eng, uint32_t, uint32_t r1, uint32_t
} // namespace
uint32_t JniShim::NewGuestBackedDirectBuffer(GuestEngine& engine, GuestAddr guestAddr,
uint32_t capacity) {
if (!guestAddr || !capacity || !RealEnv()) return 0;
jobject buf = RealEnv()->NewDirectByteBuffer(engine.G2H(guestAddr), (jlong)capacity);
if (!buf) {
Log("jni_shim: NewGuestBackedDirectBuffer(0x%x, %u) - NewDirectByteBuffer failed",
guestAddr, capacity);
return 0;
}
uint32_t handle = handles().Alloc(buf);
RegisterDirectBufferGuestAddr(handle, guestAddr);
return handle;
}
GuestAddr JniShim::BuildGuestJNIEnv(GuestEngine& engine) {
if (guest_env_) return guest_env_;
mpcore/src/main/cpp/emu/jni_shim.h
+29 -3
View File
@@ -81,15 +81,30 @@ public:
// IsSafeToUseFromCurrentThread now checks BOTH the owning thread AND
// the owning epoch.
bool IsSafeToUseFromCurrentThread(uint32_t handle) const;
static void BumpCallEpoch() { g_callEpoch.fetch_add(1, std::memory_order_relaxed); }
//
// Corrected 2026-09-21 (task #67): the epoch used to be ONE process-wide
// counter. That was indistinguishable from correct while only one host
// thread ever crossed into JNI - but a local ref's lifetime is scoped to
// a native call ON ITS OWN THREAD, and a bump from a different thread has
// no business invalidating it.
//
// It stopped being academic the moment the FMOD audio bridge began
// calling in from FMODAudioDevice's AudioTrack thread every 100 ms: each
// of those bumps invalidated the refs GLThread was holding mid-call, and
// the process aborted with "JNI DETECTED ERROR IN APPLICATION: jfieldID
// was NULL" inside GLThread. Per-thread is both the fix and the more
// accurate model - no atomics needed either, since a thread's own epoch
// is only ever read and written by that thread.
static void BumpCallEpoch() { ++t_callEpoch; }
static uint32_t CurrentCallEpoch() { return t_callEpoch; }
private:
static std::atomic<uint32_t> g_callEpoch;
static thread_local uint32_t t_callEpoch;
struct Entry {
void* real = nullptr;
std::thread::id owner;
bool isGlobal = false;
uint32_t epoch = 0; // g_callEpoch's value at Alloc() time - see IsSafeToUseFromCurrentThread
uint32_t epoch = 0; // the OWNING THREAD's epoch at Alloc() time - see IsSafeToUseFromCurrentThread
};
mutable std::mutex mutex_;
std::vector<Entry> table_{Entry{}}; // index 0 reserved for guest NULL
@@ -104,6 +119,17 @@ public:
// function that expects a JNIEnv*.
GuestAddr BuildGuestJNIEnv(GuestEngine& engine);
// Wraps a range of GUEST memory in a real Java direct ByteBuffer and
// returns the guest handle for it, registered so that the guest's own
// GetDirectBufferAddress resolves it back to `guestAddr`.
//
// Needed because GetDirectBufferAddress can only answer for buffers this
// shim created (see its own comment): a ByteBuffer that Java allocated
// lives at a host address the guest cannot reach. Anything handing guest
// code a Java-allocated buffer therefore has to bounce through one of
// these - the same shape as the AndroidBitmap_lockPixels fix.
uint32_t NewGuestBackedDirectBuffer(GuestEngine& engine, GuestAddr guestAddr, uint32_t capacity);
// Builds a minimal guest JavaVM (8-slot JNIInvokeInterface) - only
// GetEnv and AttachCurrentThread are real (both just return the same
// guest JNIEnv from BuildGuestJNIEnv - this shim only ever has one
mpcore/src/main/cpp/emu/libc_shims.cpp
+269
View File
@@ -17,6 +17,7 @@
#include <functional>
#include <algorithm>
#include <unistd.h>
#include <sys/syscall.h>
#include <fcntl.h>
#include <dirent.h>
#include <dlfcn.h>
@@ -667,8 +668,63 @@ uint32_t Shim_vsnprintf(GuestEngine& eng, uint32_t dst, uint32_t bufSize, uint32
}
// ==================== unistd.h / fcntl.h (POSIX file I/O) ====================
// /proc/cpuinfo, rewritten for a 32-bit reader (task #67).
//
// This is the root cause of "no audio", traced all the way down. FMOD detects
// CPU capabilities by reading /proc/cpuinfo and string-matching the Features
// line for "vfp", "vfpv3" and "neon" (libfmodex sub_BFFD8). If it finds none
// of them it leaves its capability mask at zero, and sub_A9AF8 then returns
// FMOD_RESULT 48, which propagates out through System::init and
// EventSystem::init - measured live as exactly that chain.
//
// On an arm64 kernel the same silicon is described with the AArch64 names:
//
// Pixel 6a: Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 ...
//
// "fp" IS VFP and "asimd" IS NEON - the hardware has both, the 64-bit format
// simply spells them differently, and a 32-bit library from 2012 cannot know
// that. So this is not a lie told to the guest: it is the same CPU, described
// in the vocabulary the guest was built to read. Everything reported here is
// genuinely present on any ARMv8 core running this engine.
//
// Kept minimal on purpose - only the fields FMOD's parser actually looks at
// ("CPU architecture", "Processor", "Features"), plus enough shape to look
// like a real file.
static const char kGuestCpuInfo[] =
"Processor\t: ARMv7 Processor rev 1 (v7l)\n"
"processor\t: 0\n"
"BogoMIPS\t: 38.40\n"
"Features\t: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt\n"
"CPU implementer\t: 0x41\n"
"CPU architecture: 7\n"
"CPU variant\t: 0x0\n"
"CPU part\t: 0xd03\n"
"CPU revision\t: 1\n"
"\n"
"Hardware\t: ARMv7 Processor\n"
"Revision\t: 0000\n"
"Serial\t\t: 0000000000000000\n";
uint32_t Shim_open(GuestEngine& eng, uint32_t path, uint32_t flags, uint32_t mode, uint32_t, uint32_t) {
const char* p = (const char*)eng.G2H(path);
if (p && strcmp(p, "/proc/cpuinfo") == 0) {
// memfd_create keeps this a perfectly ordinary fd, so read/lseek/close
// need no special cases and the guest cannot tell the difference.
int memfd = syscall(__NR_memfd_create, "guest_cpuinfo", 0);
if (memfd >= 0) {
size_t len = sizeof(kGuestCpuInfo) - 1;
if (write(memfd, kGuestCpuInfo, len) == (ssize_t)len && lseek(memfd, 0, SEEK_SET) == 0) {
Log("libc_shims: open(\"/proc/cpuinfo\") -> %d, serving an ARMv7-format copy "
"(the host's arm64 kernel calls VFP \"fp\" and NEON \"asimd\", which no 32-bit "
"library recognises - see kGuestCpuInfo)", memfd);
return (uint32_t)memfd;
}
close(memfd);
}
Log("libc_shims: could not build the ARMv7 /proc/cpuinfo replacement (%s) - falling through "
"to the host's own, which will make FMOD report no VFP/NEON and fail to initialise",
strerror(errno));
}
int fd = open(p, (int)flags, (mode_t)mode);
Log("libc_shims: open(\"%s\", flags=0x%x, mode=0%o) -> %d%s", p, flags, mode, fd,
fd < 0 ? " [FAILED]" : "");
@@ -1692,7 +1748,220 @@ uint32_t Shim_glGetBufferPointervOES(GuestEngine& eng, uint32_t target, uint32_t
} // namespace
// ---- Runtime support the game's own ARM32 FMOD needs (task #67) ----
//
// libfmodex.so is a real, shipped ARM32 shared library, and the plan is to run
// it under this engine rather than reimplement 63 FMOD entry points. Comparing
// its 117 undefined symbols against everything already registered left exactly
// 37 gaps, and every one of them is compiler-runtime or libm - no reverse
// engineering, just forwarding. They live here rather than in fmod_shims.cpp
// because none of them is FMOD-specific; libapp.so may well want them too.
//
// The __aeabi_* helpers are the ARM EABI's own arithmetic routines. They use a
// softfp register convention (a double is an r0:r1 pair), which is exactly what
// ReadDoubleArg/ReturnDouble above already handle.
// Integer division. ARM has no integer divide instruction in this profile, so
// the compiler emits calls to these instead.
uint32_t Shim_aeabi_idiv(GuestEngine&, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
int32_t n = (int32_t)a, d = (int32_t)b;
return d ? (uint32_t)(n / d) : 0; // divide-by-zero: EABI leaves it undefined, 0 is as good as anything
}
uint32_t Shim_aeabi_uidiv(GuestEngine&, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
return b ? (a / b) : 0;
}
// ...mod variants return quotient in r0 AND remainder in r1 - the same r0:r1
// pair convention ReturnU64 writes.
uint32_t Shim_aeabi_idivmod(GuestEngine& eng, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
int32_t n = (int32_t)a, d = (int32_t)b;
int32_t q = d ? n / d : 0, r = d ? n % d : 0;
return ReturnU64(eng, ((uint64_t)(uint32_t)r << 32) | (uint32_t)q);
}
uint32_t Shim_aeabi_uidivmod(GuestEngine& eng, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
uint32_t q = b ? a / b : 0, r = b ? a % b : 0;
return ReturnU64(eng, ((uint64_t)r << 32) | q);
}
// 64-bit unsigned divide: quotient in r0:r1, remainder in r2:r3. The remainder
// half needs registers the shared dispatch contract does not cover, so it is
// written directly, the same way ReturnU64 writes r1.
uint32_t Shim_aeabi_uldivmod(GuestEngine& eng, uint32_t nlo, uint32_t nhi, uint32_t dlo, uint32_t dhi, uint32_t) {
uint64_t n = ((uint64_t)nhi << 32) | nlo;
uint64_t d = ((uint64_t)dhi << 32) | dlo;
uint64_t q = d ? n / d : 0, r = d ? n % d : 0;
if (uc_engine* uc = eng.uc()) {
uint32_t rlo = (uint32_t)r, rhi = (uint32_t)(r >> 32);
uc_reg_write(uc, UC_ARM_REG_R2, &rlo);
uc_reg_write(uc, UC_ARM_REG_R3, &rhi);
}
return ReturnU64(eng, q);
}
// Double arithmetic and conversions.
uint32_t Shim_aeabi_dadd(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) + ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp));
}
uint32_t Shim_aeabi_dmul(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) * ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp));
}
uint32_t Shim_aeabi_dcmplt(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) < ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp) ? 1u : 0u;
}
uint32_t Shim_aeabi_f2d(GuestEngine& eng, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
return ReturnDouble(eng, (double)v);
}
uint32_t Shim_aeabi_ui2d(GuestEngine& eng, uint32_t v, uint32_t, uint32_t, uint32_t, uint32_t) {
return ReturnDouble(eng, (double)v);
}
uint32_t Shim_aeabi_ul2d(GuestEngine& eng, uint32_t lo, uint32_t hi, uint32_t, uint32_t, uint32_t) {
return ReturnDouble(eng, (double)(((uint64_t)hi << 32) | lo));
}
// Static-destructor registration. Nothing in this engine ever tears the guest
// image down, so recording the destructor would be write-only state.
uint32_t Shim_aeabi_atexit(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) { return 0; }
// The C++ unwinding personality routine. Reaching it means a real exception is
// unwinding through guest frames, which this engine cannot do (see
// __cxa_bad_typeid's own comment) - so say so loudly rather than return a code
// the unwinder would act on.
uint32_t Shim_aeabi_unwind_cpp_pr0(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
static bool logged = false;
if (!logged) {
logged = true;
Log("libc_shims: __aeabi_unwind_cpp_pr0 called - a C++ exception is unwinding through guest "
"frames and this engine has no guest stack unwinder. Returning 'unwind failed'; expect "
"the guest to abort rather than to catch.");
}
return 9; // _URC_FAILURE
}
// libm. Plain forwarding - the host has the same IEEE doubles the guest wants.
#define FMOD_MATH_D1(name, expr) \
uint32_t Shim_##name(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, \
uint32_t sp) { \
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp); \
return ReturnDouble(eng, (expr)); \
}
FMOD_MATH_D1(acos, acos(x))
FMOD_MATH_D1(cos, cos(x))
FMOD_MATH_D1(sin, sin(x))
FMOD_MATH_D1(tan, tan(x))
FMOD_MATH_D1(exp, exp(x))
FMOD_MATH_D1(log, log(x))
FMOD_MATH_D1(log10, log10(x))
FMOD_MATH_D1(sqrt, sqrt(x))
FMOD_MATH_D1(rint, rint(x))
#undef FMOD_MATH_D1
uint32_t Shim_atan2(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, atan2(ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp),
ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp)));
}
uint32_t Shim_log10f(GuestEngine&, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
float r = log10f(v);
uint32_t out; memcpy(&out, &r, 4);
return out;
}
uint32_t Shim_lrintf(GuestEngine&, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
return (uint32_t)(int32_t)lrintf(v);
}
// frexp/ldexp take or return a pointer/int alongside the double.
uint32_t Shim_frexp(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp);
uint32_t expPtr = eng.ReadIncomingArg(2, r0, r1, r2, r3, sp);
int e = 0;
double m = frexp(x, &e);
if (expPtr) { int32_t v = e; memcpy(eng.G2H(expPtr), &v, 4); }
return ReturnDouble(eng, m);
}
uint32_t Shim_ldexp(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp);
int32_t e = (int32_t)eng.ReadIncomingArg(2, r0, r1, r2, r3, sp);
return ReturnDouble(eng, ldexp(x, e));
}
// Remaining libc gaps.
uint32_t Shim_usleep(GuestEngine&, uint32_t us, uint32_t, uint32_t, uint32_t, uint32_t) {
return (uint32_t)usleep(us);
}
uint32_t Shim_memmem(GuestEngine& eng, uint32_t hay, uint32_t hayLen, uint32_t needle, uint32_t needleLen, uint32_t) {
if (!hay || !needle) return 0;
void* found = memmem(eng.G2H(hay), hayLen, eng.G2H(needle), needleLen);
if (!found) return 0;
return hay + (uint32_t)((uint8_t*)found - (uint8_t*)eng.G2H(hay));
}
uint32_t Shim_inet_addr(GuestEngine& eng, uint32_t s, uint32_t, uint32_t, uint32_t, uint32_t) {
return s ? (uint32_t)inet_addr((const char*)eng.G2H(s)) : 0xFFFFFFFFu;
}
uint32_t Shim_chown(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
// An app sandbox cannot chown anything anyway; real Android would fail this
// too, so reporting success costs nothing and failing could stall a caller.
return 0;
}
uint32_t Shim_pthread_attr_setdetachstate(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
// pthread_shim always creates detached-equivalent std::threads it owns, so
// the guest's preference here is already satisfied.
return 0;
}
// FMOD only reaches select() on its network-streaming path, which local
// playback never uses. Reporting "nothing ready" is the honest answer for a
// backend we do not provide - and it is logged, so if audio ever does depend on
// it that shows up as a named gap instead of silence.
uint32_t Shim_select(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
static bool logged = false;
if (!logged) {
logged = true;
Log("libc_shims: select() is not implemented - reporting 'no descriptors ready'. Only "
"FMOD's network-streaming path calls this; if audio depends on it, implement it here.");
}
return 0;
}
uint32_t Shim_operator_delete(GuestEngine& eng, uint32_t p, uint32_t, uint32_t, uint32_t, uint32_t) {
if (p) eng.heap().Free(p);
return 0;
}
void RegisterLibcImportShims(GuestEngine& engine) {
// Task #67 - runtime support the game's own ARM32 FMOD needs. See the
// block of implementations above for why these 37 live here.
engine.RegisterImportShim("__aeabi_idiv", Shim_aeabi_idiv);
engine.RegisterImportShim("__aeabi_uidiv", Shim_aeabi_uidiv);
engine.RegisterImportShim("__aeabi_idivmod", Shim_aeabi_idivmod);
engine.RegisterImportShim("__aeabi_uidivmod", Shim_aeabi_uidivmod);
engine.RegisterImportShim("__aeabi_uldivmod", Shim_aeabi_uldivmod);
engine.RegisterImportShim("__aeabi_dadd", Shim_aeabi_dadd);
engine.RegisterImportShim("__aeabi_dmul", Shim_aeabi_dmul);
engine.RegisterImportShim("__aeabi_dcmplt", Shim_aeabi_dcmplt);
engine.RegisterImportShim("__aeabi_f2d", Shim_aeabi_f2d);
engine.RegisterImportShim("__aeabi_ui2d", Shim_aeabi_ui2d);
engine.RegisterImportShim("__aeabi_ul2d", Shim_aeabi_ul2d);
engine.RegisterImportShim("__aeabi_atexit", Shim_aeabi_atexit);
engine.RegisterImportShim("__aeabi_unwind_cpp_pr0", Shim_aeabi_unwind_cpp_pr0);
engine.RegisterImportShim("acos", Shim_acos);
engine.RegisterImportShim("cos", Shim_cos);
engine.RegisterImportShim("sin", Shim_sin);
engine.RegisterImportShim("tan", Shim_tan);
engine.RegisterImportShim("exp", Shim_exp);
engine.RegisterImportShim("log", Shim_log);
engine.RegisterImportShim("log10", Shim_log10);
engine.RegisterImportShim("sqrt", Shim_sqrt);
engine.RegisterImportShim("rint", Shim_rint);
engine.RegisterImportShim("atan2", Shim_atan2);
engine.RegisterImportShim("log10f", Shim_log10f);
engine.RegisterImportShim("lrintf", Shim_lrintf);
engine.RegisterImportShim("frexp", Shim_frexp);
engine.RegisterImportShim("ldexp", Shim_ldexp);
engine.RegisterImportShim("usleep", Shim_usleep);
engine.RegisterImportShim("memmem", Shim_memmem);
engine.RegisterImportShim("inet_addr", Shim_inet_addr);
engine.RegisterImportShim("chown", Shim_chown);
engine.RegisterImportShim("pthread_attr_setdetachstate", Shim_pthread_attr_setdetachstate);
engine.RegisterImportShim("select", Shim_select);
engine.RegisterImportShim("_ZdlPv", Shim_operator_delete);
// ---- ctype.h ----
engine.RegisterImportShim("isalnum", Shim_isalnum);
engine.RegisterImportShim("isalpha", Shim_isalpha);
mpcore/src/main/cpp/emu/pthread_shim.cpp
+22 -3
View File
@@ -57,9 +57,22 @@ uint32_t Shim_pthread_create(GuestEngine& eng, uint32_t threadOutPtr, uint32_t /
// itself (see ARM64_TRANSLATION_LAYER.md, "register/stack corruption"
// investigation). Reject loudly and immediately instead - one clear
// diagnostic beats six confusing downstream ones.
if (startRoutine >= eng.image_end()) {
Log("pthread_shim: REFUSING pthread_create - startRoutine=0x%x is not real image code (%s)",
startRoutine, eng.DescribeAddress(startRoutine).c_str());
//
// Corrected 2026-09-21 (task #67): this used to compare against
// image_end(), which was right only while exactly ONE image existed. Once
// the game's own libfmodex/libfmodevent load as sibling images - far above
// image_end() - that test rejected FMOD's OWN mixer thread as "not real
// image code". The refusal propagated all the way up as silence:
//
// pthread_create -> EINVAL -> sub_A9120 -> 33 -> System::init -> 33
// -> EventSystem::init -> 33 -> fmodGetInfo -> -1
// -> FMODAudioDevice never builds its AudioTrack
//
// IsGuestImageCode knows about every loaded image, so the guard keeps its
// original purpose without the single-image assumption.
if (!eng.IsGuestImageCode(startRoutine)) {
Log("pthread_shim: REFUSING pthread_create - startRoutine=0x%x is not inside any loaded "
"guest image (%s)", startRoutine, eng.DescribeAddress(startRoutine).c_str());
return 22; // EINVAL - matches pthread_create's own errno-style failure contract
}
@@ -79,6 +92,12 @@ uint32_t Shim_pthread_create(GuestEngine& eng, uint32_t threadOutPtr, uint32_t /
eng.EnsureThreadEngine();
uint32_t result = eng.CallGuestFunction(startRoutine, arg);
Log("pthread_shim: guest thread (handle %u) start_routine returned 0x%x", handle, result);
// Hand this thread's guest stack(s) and uc_engine back. Without this
// the thread-stack arena was one-way: the game creates threads across
// a session (one per race, among others) and after kMaxGuestThreads of
// them CarveThreadStack started returning 0, the next thread's start
// routine never ran, and the race came up as a black screen.
eng.ReleaseThreadEngine();
});
{
mpcore/src/main/cpp/emu/rtti_shims.cpp
+212 -12
View File
@@ -1,3 +1,4 @@
#include <set>
#include <chrono>
#include <atomic>
#include "rtti_shims.h"
@@ -505,6 +506,7 @@ uint32_t Shim_ctype_char_do_widen(GuestEngine&, uint32_t /*this*/, uint32_t c, u
GuestAddr g_ctypeCharId = 0, g_ctypeCharFacet = 0;
GuestAddr g_numPutCharId = 0, g_numPutCharFacet = 0;
GuestAddr g_numGetCharId = 0, g_numGetCharFacet = 0;
// ---- num_put<char> facet - same "real vtable, only confirmed slots
// implemented" approach as ctype<char> above. Confirmed via IDA
@@ -679,6 +681,148 @@ uint32_t Shim_num_put_double(GuestEngine& eng, uint32_t, uint32_t streambuf, uin
return streambuf;
}
// ---- num_get<char> facet - reading numbers back OUT of a stream ----
//
// Confirmed live, not anticipated: the post-prologue dialog crashed because
// use_facet<num_get<char>> returned NULL and the guest called through it
// (guest 0x3e5248, decompiled as basic_istream<char>::operator>>). The NULL
// facet's "vtable" read produced the wild pointer 0x464c459b - the same
// "\x7fELF"-as-a-pointer crash shape FacetSlotCtx's comment warns about.
//
// libc++ basic_streambuf get-area pointers. The layout anchor is
// WriteCharToStreambuf's already-working put area (pptr@24, epptr@28): the six
// pointers run eback@8, gptr@12, egptr@16, pbase@20, pptr@24, epptr@28 after
// the vtable and the embedded locale, so the put offsets pin the get offsets.
constexpr uint32_t kSbGptr = 12, kSbEgptr = 16;
// ios_base::iostate bits, taken from THIS NDK's own <ios> rather than from
// memory (they differ between standard libraries - libstdc++ orders them the
// other way round).
constexpr uint32_t kIosBadbit = 0x1, kIosEofbit = 0x2, kIosFailbit = 0x4;
int PeekStreambuf(GuestEngine& eng, GuestAddr sb) {
if (!sb) return -1;
uint32_t g = 0, e = 0;
memcpy(&g, eng.G2H(sb + kSbGptr), 4);
memcpy(&e, eng.G2H(sb + kSbEgptr), 4);
// An exhausted get area would need a virtual underflow() call to refill.
// This shim does not make one - for the istringstream case that actually
// occurs here the whole string is already in the get area. Reported as
// end-of-input rather than silently treated as a parse failure.
if (!g || g >= e) return -1;
uint8_t c = 0;
memcpy(&c, eng.G2H(g), 1);
return c;
}
void BumpStreambuf(GuestEngine& eng, GuestAddr sb) {
uint32_t g = 0;
memcpy(&g, eng.G2H(sb + kSbGptr), 4);
g += 1;
memcpy(eng.G2H(sb + kSbGptr), &g, 4);
}
enum class NumGetKind { Bool, Signed, Unsigned, Float };
// Shared body for every do_get overload. Lifts the next token out of the get
// area and hands it to the host's own strtoll/strtoull/strtod - the same
// "offload the hard part to real libc instead of reimplementing it" choice
// num_put makes for formatting. Like num_put, it deliberately ignores
// ios_base's formatting flags (base/boolalpha), which is a documented scope
// cut, not an oversight.
uint32_t NumGetCommon(GuestEngine& eng, uint32_t sb, uint32_t errPtr, uint32_t valPtr,
NumGetKind kind, int width) {
uint32_t err = 0;
auto storeErr = [&]() { if (errPtr) memcpy(eng.G2H(errPtr), &err, 4); };
if (!sb || !valPtr) {
err = kIosFailbit | kIosBadbit;
storeErr();
return sb;
}
int c = PeekStreambuf(eng, sb);
while (c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v') {
BumpStreambuf(eng, sb);
c = PeekStreambuf(eng, sb);
}
std::string tok;
if (c == '+' || c == '-') { tok.push_back((char)c); BumpStreambuf(eng, sb); c = PeekStreambuf(eng, sb); }
bool sawDigit = false;
while (c >= 0) {
char ch = (char)c;
bool part = (ch >= '0' && ch <= '9');
if (!part && kind == NumGetKind::Float &&
(ch == '.' || ch == 'e' || ch == 'E' || ch == '+' || ch == '-')) part = true;
if (!part) break;
if (ch >= '0' && ch <= '9') sawDigit = true;
tok.push_back(ch);
BumpStreambuf(eng, sb);
c = PeekStreambuf(eng, sb);
}
if (!sawDigit) {
// No number here at all. Real num_get sets failbit and leaves the
// destination untouched; matching that is what lets the caller's
// stream go into a failed state instead of reading a fabricated value.
err = kIosFailbit;
if (c < 0) err |= kIosEofbit;
storeErr();
Log("rtti_shims: num_get found no number to parse (streambuf=0x%x) - setting failbit, "
"leaving the destination untouched", sb);
return sb;
}
if (c < 0) err |= kIosEofbit; // consumed right up to the end of input
uint8_t b8; uint16_t b16; uint32_t b32; uint64_t b64;
switch (kind) {
case NumGetKind::Bool:
b8 = (uint8_t)(strtoll(tok.c_str(), nullptr, 10) != 0);
memcpy(eng.G2H(valPtr), &b8, 1);
break;
case NumGetKind::Signed: {
long long v = strtoll(tok.c_str(), nullptr, 10);
if (width == 8) { b64 = (uint64_t)v; memcpy(eng.G2H(valPtr), &b64, 8); }
else { b32 = (uint32_t)(int32_t)v; memcpy(eng.G2H(valPtr), &b32, 4); }
break;
}
case NumGetKind::Unsigned: {
unsigned long long v = strtoull(tok.c_str(), nullptr, 10);
if (width == 8) { b64 = (uint64_t)v; memcpy(eng.G2H(valPtr), &b64, 8); }
else if (width == 2) { b16 = (uint16_t)v; memcpy(eng.G2H(valPtr), &b16, 2); }
else { b32 = (uint32_t)v; memcpy(eng.G2H(valPtr), &b32, 4); }
break;
}
case NumGetKind::Float: {
double v = strtod(tok.c_str(), nullptr);
if (width == 4) { float f = (float)v; memcpy(eng.G2H(valPtr), &f, 4); }
else { memcpy(eng.G2H(valPtr), &v, 8); }
break;
}
}
storeErr();
return sb;
}
// r0=this(facet, unused), r1=begin iterator (an istreambuf_iterator is a bare
// streambuf*), r2=end iterator (unused - the get area's own end bounds us),
// r3=ios_base* (unused, same scope cut as num_put), stack slot 4 = iostate*,
// slot 5 = the destination. Returns the iterator, i.e. the streambuf - exactly
// how the working num_put overloads return theirs.
uint32_t NumGetEntry(GuestEngine& eng, uint32_t sb, uint32_t sp, NumGetKind kind, int width) {
uint32_t errPtr = eng.ReadIncomingArg(4, 0, 0, 0, 0, sp);
uint32_t valPtr = eng.ReadIncomingArg(5, 0, 0, 0, 0, sp);
return NumGetCommon(eng, sb, errPtr, valPtr, kind, width);
}
uint32_t Shim_num_get_bool(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Bool, 1); }
uint32_t Shim_num_get_s32(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Signed, 4); }
uint32_t Shim_num_get_s64(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Signed, 8); }
uint32_t Shim_num_get_u16(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 2); }
uint32_t Shim_num_get_u32(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 4); }
uint32_t Shim_num_get_u64(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 8); }
uint32_t Shim_num_get_float(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Float, 4); }
uint32_t Shim_num_get_double(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp){ return NumGetEntry(e, sb, sp, NumGetKind::Float, 8); }
// use_facet<Facet>(locale) looks up a facet by its static `id` member and
// THROWS std::bad_cast if not found - facet types with no evidence of ever
// being requested by this binary (num_get, ...) keep the original
@@ -688,17 +832,30 @@ uint32_t Shim_num_put_double(GuestEngine& eng, uint32_t, uint32_t streambuf, uin
// and num_put<char> are different - confirmed real, actually-called use
// (see their own comments above) - return the real facet objects built
// above.
uint32_t Shim_use_facet(GuestEngine&, uint32_t, uint32_t idAddr, uint32_t, uint32_t, uint32_t) {
uint32_t Shim_use_facet(GuestEngine& eng, uint32_t, uint32_t idAddr, uint32_t, uint32_t, uint32_t) {
if (idAddr == g_ctypeCharId) return g_ctypeCharFacet;
if (idAddr == g_numPutCharId) return g_numPutCharFacet;
static bool logged = false;
if (!logged) {
logged = true;
Log("rtti_shims: std::locale::use_facet() for an id other than ctype<char>/num_put<char> "
"(0x%x) - this engine's locale objects have no real facet table for it (see "
"rtti_shims.cpp) - returning NULL rather than a facet object a subsequent virtual call "
"would crash through; real libc++ would throw std::bad_cast here, which needs a guest "
"stack unwinder this engine doesn't have (see __cxa_bad_typeid's own comment)", idAddr);
if (idAddr == g_numGetCharId) return g_numGetCharFacet;
// Report once PER DISTINCT ID, not once overall. The previous single
// `static bool` meant the first unimplemented facet hid every other one
// behind it, so a crash caused by the second facet looked like it had no
// diagnostic at all. Naming the facet and the call site matters just as
// much: the id is an address our own AllocPermanent handed out, which says
// nothing on its own - resolving one used to cost a rebuild and a repro.
static std::set<uint32_t> reported;
if (reported.insert(idAddr).second) {
const char* name = eng.NameForDataSymbol(idAddr);
uint32_t callerLr = 0;
if (uc_engine* uc = eng.uc()) uc_reg_read(uc, UC_ARM_REG_LR, &callerLr);
Log("rtti_shims: std::locale::use_facet(%s) is NOT implemented - id=0x%x, called from guest "
"LR=0x%x. Returning NULL; the caller will dereference it and take a wild pointer through "
"whatever the NULL facet's vtable slot reads (this is the known "
"\"\\x7fELF\"-as-a-pointer crash shape). ctype<char> and num_put<char> are the only "
"facets this engine builds - implement this one next if the game depends on it. Real "
"libc++ would throw std::bad_cast, which needs a guest stack unwinder this engine does "
"not have (see __cxa_bad_typeid's own comment).",
name ? name : "<unregistered id>", idAddr, callerLr);
}
return 0;
}
@@ -827,9 +984,52 @@ void SetupRttiDataSymbols(GuestEngine& engine) {
g_numPutCharId = engine.AllocPermanent(4);
engine.RegisterDataSymbol("_ZNSt6__ndk17num_putIcNS_19ostreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", g_numPutCharId);
}
// ---- Other facet `id` statics - inert (see Shim_use_facet's own
// comment - no evidence anything requests these yet) ----
engine.RegisterDataSymbol("_ZNSt6__ndk17num_getIcNS_19istreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", engine.AllocPermanent(4));
// ---- num_get<char> facet - a real, callable vtable. This used to be an
// inert `id` with no facet behind it, on the stated grounds that nothing
// requested it; that turned out to be wrong and it cost a crash (see
// NumGetCommon's own comment).
//
// The slot offsets are DERIVED, not guessed. num_put above is confirmed
// working, and its known-good slots (16=long, 20=long long,
// 24=unsigned long, 32=double, 40=const void*) only line up with this
// NDK's declaration order if three slots precede the first virtual:
// the complete and deleting destructors plus
// __shared_count::__on_zero_shared. Applying that same prefix to
// num_get's declaration order in this NDK's own <locale> produces the
// table below - and offset 28, the one the game actually calls, lands on
// `unsigned int&`, exactly what basic_istream::operator>>(unsigned int&)
// would invoke. Two independent routes, same answer.
//
// Slots outside this table still route to the logging stub rather than to
// a guess, so the next gap names itself instead of crashing.
{
constexpr int kNumGetCharVtableSlots = 24;
GuestAddr vtable = engine.AllocPermanent(kNumGetCharVtableSlots * 4);
for (int i = 0; i < kNumGetCharVtableSlots; i++) {
int byteOffset = i * 4;
ImportShimFn impl = nullptr;
switch (byteOffset) {
case 12: impl = Shim_num_get_bool; break; // bool&
case 16: impl = Shim_num_get_s32; break; // long&
case 20: impl = Shim_num_get_s64; break; // long long&
case 24: impl = Shim_num_get_u16; break; // unsigned short&
case 28: impl = Shim_num_get_u32; break; // unsigned int& <- the crashing call
case 32: impl = Shim_num_get_u32; break; // unsigned long& (32-bit here)
case 36: impl = Shim_num_get_u64; break; // unsigned long long&
case 40: impl = Shim_num_get_float; break; // float&
case 44: impl = Shim_num_get_double; break; // double&
case 48: impl = Shim_num_get_double; break; // long double& (== double on ARM32)
default: break;
}
auto* ctx = new FacetSlotCtx{byteOffset, impl};
GuestAddr stub = engine.AllocCodeStub(FacetSlotDispatch, ctx);
if (vtable && stub) memcpy(engine.G2H(vtable + (uint32_t)byteOffset), &stub, 4);
}
g_numGetCharFacet = engine.AllocPermanent(4);
if (g_numGetCharFacet && vtable) memcpy(engine.G2H(g_numGetCharFacet), &vtable, 4);
g_numGetCharId = engine.AllocPermanent(4);
engine.RegisterDataSymbol("_ZNSt6__ndk17num_getIcNS_19istreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", g_numGetCharId);
}
// ---- The shared "classic locale" every locale::locale() call returns
// (see Shim_locale_ctor's own comment) ----
mpcore/src/main/cpp/game_lifecycle_stubs_extra2.cpp
+109 -5
View File
@@ -9,17 +9,121 @@
#include <jni.h>
#include <cstdint>
#include <cstring>
#include <set>
#include <string>
#include "emu/jni_shim.h"
#include "util/util.h"
#include "real_native_call.h"
#include "real_native_offsets.h"
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodGetInfo(JNIEnv*, jobject, jint) {
return 0;
// ---- FMOD audio bridge (task #67, 2026-09-21) ----
//
// Both of these used to `return 0`, silently, on the stated grounds that FMOD
// was not loaded because no arm64 build exists. That is still true of arm64 -
// but the game's real ARM32 libfmodex/libfmodevent now run inside the engine
// as secondary guest images, and both of these symbols are among their
// exports, so the calls can be forwarded to the real implementations.
//
// Why this pair is the whole audio path: FMOD on this Android build does not
// drive the output device from native code. org/fmod/FMODAudioDevice.java owns
// an AudioTrack and pulls PCM down through these two methods. Its thread reads
//
// int rate = fmodGetInfo(FMOD_INFO_SAMPLERATE);
// if (rate > 0) { ...create the AudioTrack, then loop on fmodProcess... }
//
// so a `return 0` from fmodGetInfo alone made that thread give up before the
// AudioTrack was ever constructed. Nothing downstream could have produced
// sound regardless of what FMOD itself did.
// Re-enabled 2026-09-21 once the JNI call epoch became per-thread.
//
// These two run on FMODAudioDevice's own AudioTrack thread, and the first
// attempt aborted the process inside GLThread with "JNI DETECTED ERROR IN
// APPLICATION: jfieldID was NULL". The first diagnosis - that CallRealNative's
// SetRealEnv() clobbered a shared JNIEnv - was WRONG: real_env_ was already
// thread_local, with a lazy AttachCurrentThread fallback.
//
// The actual culprit was one line further on. SetRealEnv also calls
// JniHandleTable::BumpCallEpoch(), and that epoch was a single process-wide
// counter whose own comment said it invalidates the previous call's local refs
// "whether or not it's the same thread". So every 100 ms this bridge was
// invalidating the references GLThread held mid-call. The epoch is per-thread
// now, which is the more accurate model anyway - a local ref's lifetime is
// scoped to a native call on its own thread.
static constexpr bool kEnableFmodAudioBridge = true;
static GuestAddr FmodGuestSymbol(const char* name) {
GuestAddr addr = GuestEngine::Instance().LookupSecondaryExport(name);
if (!addr) {
static std::set<std::string> reported;
if (reported.insert(name).second) {
Log("fmod_bridge: '%s' is not exported by any loaded guest image - audio stays silent. "
"Did libfmodex.so/libfmodevent.so fail to load? (see LoadSiblingLibraries)", name);
}
}
return addr;
}
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodProcess(JNIEnv*, jobject, jobject) {
return 0;
Java_org_fmod_FMODAudioDevice_fmodGetInfo(JNIEnv* env, jobject thiz, jint info) {
if (!kEnableFmodAudioBridge) return 0;
static GuestAddr fn = FmodGuestSymbol("Java_org_fmod_FMODAudioDevice_fmodGetInfo");
if (!fn) return 0;
// Deliberately NOT logged per call: FMODAudioDevice polls this from its
// audio thread continuously (1,210 calls in a 40-second run), so a log
// line here is a steady drip into a buffer we already lose history to.
return (jint)CallRealNative(env, thiz, fn, {(uint32_t)info});
}
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodProcess(JNIEnv* env, jobject thiz, jobject buffer) {
if (!kEnableFmodAudioBridge) return 0;
static GuestAddr fn = FmodGuestSymbol("Java_org_fmod_FMODAudioDevice_fmodProcess");
if (!fn || !buffer) return 0;
// The ByteBuffer came from Java's own allocateDirect, so it lives at a host
// address the guest cannot write to - GetDirectBufferAddress would hand
// guest FMOD a zero. Bounce through a guest-memory buffer of the same size
// instead, exactly as the font fix does for AndroidBitmap_lockPixels, and
// copy the rendered PCM back out afterwards.
jlong capacity = env->GetDirectBufferCapacity(buffer);
void* hostDest = env->GetDirectBufferAddress(buffer);
if (capacity <= 0 || !hostDest) {
static bool logged = false;
if (!logged) {
logged = true;
Log("fmod_bridge: fmodProcess got a ByteBuffer with no direct address (capacity=%lld) - "
"returning silence", (long long)capacity);
}
return 0;
}
// Allocated once and reused: this runs on the AudioTrack thread at the
// audio buffer rate, so per-call allocation would be both wasteful and a
// source of jitter. The capacity is fixed by FMOD's DSP buffer settings
// and does not change after the device starts, but it is re-checked rather
// than assumed.
static GuestAddr guestBuf = 0;
static uint32_t guestBufSize = 0;
static uint32_t guestBufHandle = 0;
if (guestBufSize != (uint32_t)capacity) {
guestBuf = GuestEngine::Instance().heap().Alloc((uint32_t)capacity);
if (!guestBuf) {
Log("fmod_bridge: could not allocate a %lld-byte guest audio buffer - silence",
(long long)capacity);
return 0;
}
guestBufSize = (uint32_t)capacity;
guestBufHandle = JniShim::Instance().NewGuestBackedDirectBuffer(
GuestEngine::Instance(), guestBuf, guestBufSize);
Log("fmod_bridge: audio bounce buffer ready - %u bytes at guest 0x%x (handle %u)",
guestBufSize, guestBuf, guestBufHandle);
}
if (!guestBufHandle) return 0;
memset(GuestEngine::Instance().G2H(guestBuf), 0, guestBufSize);
jint r = (jint)CallRealNative(env, thiz, fn, {guestBufHandle});
memcpy(hostDest, GuestEngine::Instance().G2H(guestBuf), guestBufSize);
return r;
}
#define NIMBLE_COMPONENT_STUB(name) \
mpcore/src/main/cpp/main.cpp
+16
View File
@@ -5,6 +5,7 @@
#include <android/log.h>
#include <jni.h>
#include "main.h"
#include "crash_handler.h"
#include "util/util.h"
#include <unistd.h>
#include <unwind.h>
@@ -497,6 +498,21 @@ static constexpr bool kEnableMapScreenCtorTraceHook = false; // TEMP: isolating
// comment), which JNI_OnLoad has no reliable way to obtain on its own
// (no Context/AssetManager access at this point). See
// Java_..._MultiplayerCore_loadEmulatedLibapp below for where that now happens.
// extern "C" is NOT optional here: without it the name is C++-mangled and the
// JVM cannot find it by its JNI name. The first version omitted it, and the
// only symptom was "No implementation found" at runtime - which looked exactly
// like a load-order problem and cost a wrong fix before the symbol table was
// actually read.
extern "C" JNIEXPORT void JNICALL
Java_com_ea_ironmonkey_GameActivityMain_nativeInstallCrashHandler(JNIEnv* env, jobject,
jstring dir, jstring buildStamp) {
const char* d = dir ? env->GetStringUTFChars(dir, nullptr) : nullptr;
const char* b = buildStamp ? env->GetStringUTFChars(buildStamp, nullptr) : nullptr;
InstallCrashHandler(d, b);
if (d) env->ReleaseStringUTFChars(dir, d);
if (b) env->ReleaseStringUTFChars(buildStamp, b);
}
JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) {
JNIEnv* env = nullptr;
if (vm->GetEnv((void**)&env, JNI_VERSION_1_6) == JNI_OK) {
mpcore/src/main/cpp/third_party/unicorn/include/uc_priv.h
Vendored
+28
View File
@@ -361,6 +361,34 @@ struct uc_struct {
// linked lists containing hooks per type
struct list hook[UC_HOOK_MAX];
struct list hooks_to_del;
// NFSMW arm64-poc, task #61: flat guest mapping.
//
// When non-zero, the whole guest address space is ONE contiguous host
// block starting here, and a guest address translates to a host address
// by plain addition - which is exactly what GuestEngine's G2H() already
// does. The aarch64 TCG backend can then emit a single
// `LDR Wd, [X28, Wn, UXTW]` per guest memory access instead of the
// nine-instruction softmmu TLB check (see tcg_out_qemu_ld/st).
//
// Measured motivation: those TLB sequences are 29.8% of ALL generated
// host code (16,202 sequences x 35.8 bytes against 1,943,672 bytes total,
// counted during a prologue load).
//
// Set by the engine AFTER mapping its spans and BEFORE any translation
// happens, and never changed afterwards - the generated code bakes this
// value into a register, so changing it later would silently corrupt
// every existing translated block.
uint64_t flat_map_base;
// Regenerates the TCG prologue so a freshly-set flat_map_base actually
// reaches the generated code. Needed because the prologue is built inside
// uc_open(), before the embedder can say anything - and it is the prologue
// that loads the base into X28. Set by tcg_exec_init, which unlike uc.c is
// compiled per-architecture and can name tcg_prologue_init/tcg_nb_tbs.
// Returns 0 on success, -1 if blocks were already translated (in which
// case the base is left alone rather than corrupting those blocks).
int (*uc_reinit_prologue)(struct uc_struct *uc);
// NFSMW arm64-poc: page -> MemoryRegion cache for notdirty_write, which
// otherwise calls memory_mapping() on EVERY guest write just to test one
// permission bit (see cputlb.c). Lives here rather than in a __thread
mpcore/src/main/cpp/third_party/unicorn/include/unicorn/unicorn.h
Vendored
+29
View File
@@ -1449,6 +1449,35 @@ size_t uc_context_size(uc_engine *uc);
UNICORN_EXPORT
uc_err uc_context_free(uc_context *context);
/*
NFSMW arm64-poc extension (task #61). Declare that the entire guest address
space is one contiguous host block starting at @base, so the JIT can reach
guest memory by plain addition instead of a software TLB lookup. On aarch64
hosts with a 32-bit guest this replaces a nine-instruction TLB check with a
single LDR/STR, which is why it exists.
Enabling this DISABLES, for every emulated memory access:
- UC_PROT_* enforcement (uc_mem_protect becomes advisory),
- UC_HOOK_MEM_READ / _WRITE / _INVALID callbacks,
- self-modifying-code detection, so guest stores to pages that already
hold translated code will NOT invalidate those translations.
Callers depending on any of the above must leave it off.
@uc: handle returned by uc_open()
@base: host address the guest's address 0 maps to. The caller must guarantee
that [base, base + 4GiB) is reserved host address space, since a 32-bit
guest address is zero-extended and used as an unchecked offset from it.
Pass 0 to keep the normal software-MMU path.
Must be called before any code is translated - generated code bakes @base
into a register, so a later change would silently corrupt existing blocks.
@return UC_ERR_OK on success, or other value on failure (refer to uc_err enum
for detailed error).
*/
UNICORN_EXPORT
uc_err uc_set_flat_map_base(uc_engine *uc, uint64_t base);
#ifdef __cplusplus
}
#endif
mpcore/src/main/cpp/third_party/unicorn/qemu/accel/tcg/translate-all.c
Vendored
+18
View File
@@ -1217,6 +1217,23 @@ static uc_err uc_gen_tb(struct uc_struct *uc, uint64_t addr, uc_tb *out_tb)
/* Must be called before using the QEMU cpus. 'tb_size' is the size
(in bytes) allocated to the translation buffer. Zero means default
size. */
/*
* NFSMW arm64-poc, task #61. Rebuild the prologue in place after
* uc_set_flat_map_base(). Safe only with zero translated blocks - the caller
* enforces that - because tcg_prologue_init resets code_ptr to the start of
* the code buffer and re-emits from there.
*/
static int uc_reinit_prologue(struct uc_struct *uc)
{
if (tcg_nb_tbs(uc->tcg_ctx) != 0) {
return -1;
}
tb_exec_unlock(uc);
tcg_prologue_init(uc->tcg_ctx);
tb_exec_lock(uc);
return 0;
}
void tcg_exec_init(struct uc_struct *uc, uint32_t tb_size)
{
/* remove tcg object. init here. */
@@ -1233,6 +1250,7 @@ void tcg_exec_init(struct uc_struct *uc, uint32_t tb_size)
tb_exec_unlock(uc);
tcg_prologue_init(uc->tcg_ctx);
tb_exec_lock(uc);
uc->uc_reinit_prologue = uc_reinit_prologue;
/* cpu_interrupt_handler is not used in uc1 */
uc->l1_map = g_malloc0(sizeof(void *) * V_L1_MAX_SIZE);
/* Invalidate / Cache TBs */
mpcore/src/main/cpp/third_party/unicorn/qemu/tcg/aarch64/tcg-target.inc.c
Vendored
+50 -1
View File
@@ -69,13 +69,39 @@ static const int tcg_target_call_oarg_regs[1] = {
#define TCG_REG_TMP TCG_REG_X30
#define TCG_VEC_TMP TCG_REG_V31
/* The register holding the base of the guest's address space. Defined
unconditionally because task #61's flat mapping below uses it under
CONFIG_SOFTMMU too, for exactly the same purpose as user-mode does. */
#define TCG_REG_GUEST_BASE TCG_REG_X28
/*
* NFSMW arm64-poc, task #61: flat guest mapping under softmmu.
*
* When the embedder has told us the whole guest address space is one
* contiguous host block (uc->flat_map_base), a guest access needs no TLB
* lookup at all - the host address is base + zero-extended guest address,
* which aarch64 does in the addressing mode itself. That turns the nine
* instruction tcg_out_tlb_read sequence plus its slow-path label into a
* SINGLE instruction: LDR Wd, [X28, Wn, UXTW].
*
* Only valid because TARGET_LONG_BITS == 32 here: a 32-bit guest address
* zero-extends to at most 4 GiB - 1, so it cannot escape the host
* reservation. The guard is spelled out rather than assumed.
*
* What this deliberately gives up (see guest_engine.cpp's SetFlatMapBase):
* UC_PROT_* enforcement, UC_HOOK_MEM_* callbacks, and self-modifying-code
* detection via notdirty_write. The embedder must not enable it while it
* depends on any of those.
*/
#define UC_FLAT_MAP(s) (TARGET_LONG_BITS == 32 && (s)->uc->flat_map_base)
#ifndef CONFIG_SOFTMMU
/* Note that XZR cannot be encoded in the address base register slot,
as that actaully encodes SP. So if we need to zero-extend the guest
address, via the address index register slot, we need to load even
a zero guest base into a register. */
#define USE_GUEST_BASE (guest_base != 0 || TARGET_LONG_BITS == 32)
#define TCG_REG_GUEST_BASE TCG_REG_X28
#endif
static inline bool reloc_pc26(tcg_insn_unit *code_ptr, tcg_insn_unit *target)
@@ -1823,6 +1849,12 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg,
unsigned mem_index = get_mmuidx(oi);
tcg_insn_unit *label_ptr;
if (UC_FLAT_MAP(s)) {
tcg_out_qemu_ld_direct(s, memop, ext, data_reg,
TCG_REG_GUEST_BASE, otype, addr_reg);
return;
}
tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1);
tcg_out_qemu_ld_direct(s, memop, ext, data_reg,
TCG_REG_X1, otype, addr_reg);
@@ -1848,6 +1880,12 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg,
unsigned mem_index = get_mmuidx(oi);
tcg_insn_unit *label_ptr;
if (UC_FLAT_MAP(s)) {
tcg_out_qemu_st_direct(s, memop, data_reg,
TCG_REG_GUEST_BASE, otype, addr_reg);
return;
}
tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0);
tcg_out_qemu_st_direct(s, memop, data_reg,
TCG_REG_X1, otype, addr_reg);
@@ -2850,6 +2888,17 @@ static void tcg_target_qemu_prologue(TCGContext *s)
tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, guest_base);
tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE);
}
#else
/*
* Task #61. Same idea as the user-mode path above, but the base comes
* from the embedder instead of QEMU's global guest_base. X28 was already
* pushed as x27's pair partner a few lines up, and the epilogue restores
* it, so clobbering it here is safe for our caller.
*/
if (UC_FLAT_MAP(s)) {
tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, s->uc->flat_map_base);
tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE);
}
#endif
tcg_out_mov(s, TCG_TYPE_PTR, TCG_AREG0, tcg_target_call_iarg_regs[0]);
mpcore/src/main/cpp/third_party/unicorn/uc.c
Vendored
+32
View File
@@ -2594,6 +2594,38 @@ uc_err uc_context_restore(uc_engine *uc, uc_context *context)
return UC_ERR_OK;
}
UNICORN_EXPORT
uc_err uc_set_flat_map_base(uc_engine *uc, uint64_t base)
{
uint64_t previous = uc->flat_map_base;
bool was_initialised = uc->init_done;
// Set the base BEFORE UC_INIT. Engine setup is lazy in Unicorn - uc_open()
// only allocates the handle, and the first API call using UC_INIT is what
// builds the CPU, the TCG context and the prologue. Assigning here means
// that, in the common case, the prologue is generated with the right base
// the first time and needs no rebuilding at all.
uc->flat_map_base = base;
UC_INIT(uc);
if (!was_initialised) {
return UC_ERR_OK;
}
// The engine was already up, so a prologue exists that loaded a different
// base into the register. Rebuild it - which also refuses, leaving the old
// base in place, if any block has already been translated against it.
if (!uc->uc_reinit_prologue) {
uc->flat_map_base = previous;
return UC_ERR_HANDLE;
}
if (uc->uc_reinit_prologue(uc) != 0) {
uc->flat_map_base = previous;
return UC_ERR_ARG;
}
return UC_ERR_OK;
}
UNICORN_EXPORT
uc_err uc_context_free(uc_context *context)
{
mpcore/src/main/java/nfs/mod/mpcore/MultiplayerCore.kt
+14
View File
@@ -22,6 +22,20 @@ object MultiplayerCore {
*/
external fun triggerTrueDirectCarSelectJump()
/**
* ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md): loads
* the original armeabi-v7a libapp.so through mpcore's embedded ARM32
* CPU-emulation core (Unicorn) instead of a real System.loadLibrary
* dlopen(), and installs whichever hooks are enabled in main.cpp.
* `path` must be a real file (e.g. extracted from assets to
* filesDir/libapp.so at first run - libapp.so is shipped as a raw
* asset, not under jniLibs, since this app declares only arm64-v8a and
* the packaging system would refuse/ignore an armeabi-v7a .so there).
* Returns false on any load failure (bad ELF, mmap failure, ...) -
* check logcat's "GuestEngine"/"mpcore_log" tag for why.
*/
external fun loadEmulatedLibapp(path: String): Boolean
fun loadCore() {
System.loadLibrary("mpcore")
installCarSelectLoadoutTestTrigger()