megboyzz/nfsmw-online
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: megboyzz/nfsmw-online:edfa360c0aa5cb1754555af1b3b3d4b996529a91
Branches Tags
megboyzz/nfsmw-online:master megboyzz/nfsmw-online:arm64-guestfn-hook-port megboyzz/nfsmw-online:mod-hooks-work megboyzz/nfsmw-online:lan-event-injection-poc
...
compare: megboyzz/nfsmw-online:e7c76fc2dd503f98a1b34edc36cf8f69cb862744
Branches Tags
megboyzz/nfsmw-online:arm64-guestfn-hook-port megboyzz/nfsmw-online:master megboyzz/nfsmw-online:mod-hooks-work megboyzz/nfsmw-online:lan-event-injection-poc
10 Commits
edfa360c0a ... e7c76fc2dd
Author SHA1 Message Date
megboyzzandClaude e7c76fc2dd docs: bring 1.8 MB of project documentation under version control 
These files had never been tracked anywhere - they lived in a plain directory
with no git at all, which is also where the whole reverse-engineering record
sat. Code already committed refers to them by name (opponent_substitution.h
cites "ANALYSIS.md section 6hh", DebugMenuOverlay.kt cites "DEBUG_MENU.md
section 3"), so until now a fresh clone carried references to documents it did
not contain.

  ANALYSIS.md                       the RE record, and the reason the rest works
  ARCHITECTURE.md                   how the mod's pieces fit together
  ARM64_TRANSLATION_LAYER.md        the translation layer's running log
  PROGRESS.md                       chronological progress across both chats
  BETA_TELEMETRY_PLAN.md            how crash/telemetry reporting is meant to work
  LOBBY_UI_DESIGN.md + .html        lobby design and its clickable prototype
  DEBUG_MENU.md                     debug panel design
  STATIC_RECOMPILATION_FALLBACK.md  the plan if translation had not panned out
  evidence/                         font atlas capture from the glyph-corruption bug
  save_backups/                     saves at known milestones, for reproducing state

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-22 23:29:12 +03:00
megboyzzandClaude 725ffbd8ed arm64: flat guest mapping, audio, crash reporting, self-extracting data 
The run of work that took the translated build from "boots" to "playable".

Engine:
- Flat guest mapping replaces the software MMU on aarch64 hosts. A 4 GiB
  PROT_NONE reservation lets a 32-bit guest address zero-extend safely, so
  tcg_out_qemu_ld/st short-circuit before tcg_out_tlb_read and the prologue
  materialises the base into X28. Measured 1.24x (51 vs 41 fps, interleaved
  A/B). Note the base must be set BEFORE UC_INIT - Unicorn inits lazily, and
  setting it after uc_open silently falls back to softmmu.
- num_get<char> facet implemented, which was the real cause of the crash after
  the prologue race; a full race is now playable end to end.
- Thread-stack free list + ReleaseThreadEngine, fixing the arena leak that
  showed up as a black screen when entering a race. kMaxGuestThreads 16 -> 64.
- Real ARM32 FMOD now runs in-engine via the Java FMODAudioDevice bridge, with
  a per-thread JNIEnv. Two of the three blockers were our own single-image-era
  guards.

Host/app:
- Native crash handler: async-signal-safe, decodes the host fault back to a
  guest address, writes a report file and nothing else. CrashReportActivity
  picks it up on the NEXT launch, zips it, and offers to share. No backend, no
  automatic upload.
- Game data ships inside the APK and self-extracts on first launch, so a tester
  installs one file and plays. Copy-to-.part-then-rename, with a free-space
  check up front.
- EGL context preserved across pause, fixing black textures on resume.
- Navigation bar hidden and re-hidden on focus gain; volume keys reported as
  system keys, checked before the loading-state gate.
- x86_64 added to abiFilters: the ARM32 guest runs under tcg/i386 with no
  houdini in the path. The flat mapping is aarch64-only, so that host falls
  back to the software MMU - commented at the abiFilters line.

Ignore rules added for app/translated/ (611 MB of signed release APK, which
also carries the bundled OBB) and ostream_repro/build/.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-22 23:17:47 +03:00
megboyzzandClaude 80768652ea Cache the notdirty_write region lookup: 3.1% faster loading 
notdirty_write called uc->memory_mapping() on every guest write, before
testing whether the region is executable, and used the result for that one
bit. A probe counted over a million such calls per 3 seconds with
exec_region=0 - the full find_memory_mapping -> address_space_translate ->
flatview_translate chain running and the answer then discarded.

The map is static once CreateConfiguredEngine has mapped its five spans, so
a 512-entry direct-mapped page -> MemoryRegion cache removes all but cold
misses.

    interleaved A/B, engine start to first OnCarLoaded
      cache on:  31.27 / 31.34 / 31.22 s   mean 31.28
      cache off: 32.27 / 32.12 / 32.44 s   mean 32.28

3.1%, ranges fully separate, spreads under 0.35s. The profile confirms the
mechanism rather than just the outcome: flatview_translate 4.46% -> 3.17%,
find_memory_mapping 3.02% -> 2.36%.

THE FIRST VERSION OF THIS CACHE WAS SLOWER, and the measurement is why it
was caught. It used `static __thread`, which on Android resolves through
emulated TLS: __emutls_get_address appeared at 8.11% and pthread_getspecific
at 3.07% - together more than the 7.5% being removed. Storage now lives in
uc_struct, which is already per-thread here (one uc_engine per host thread),
so no TLS is involved. Do not reintroduce __thread on this path.

INVARIANT: the cache assumes mappings and permissions are fixed after engine
setup. Anyone changing them at runtime must clear mr_cache_ready.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-20 13:26:43 +03:00
megboyzzandClaude 5e246fc8db Find why the softmmu lookup is hot: memory_mapping runs on every notdirty write 
Task #53 ruled out TLB capacity as the cause of flatview_translate 4.46% +
find_memory_mapping 3.02%. A probe on notdirty_write's own first line found
the real reason, and it is a miss in this project's own earlier fix.

notdirty_write calls uc->memory_mapping() UNCONDITIONALLY, before testing
whether the region is executable - and that result is used only to test one
bit. Measured on device, per 3-second window:

    hits=3478174 exec_region=1   (startup, all in the 0-1MB bucket)
    hits=1184522 exec_region=0
    hits=902501  exec_region=0   (10MB and 11MB buckets = .data/.bss)

Over a million calls per 3 seconds where the lookup ran in full and the
answer then did not matter. Task #54 removed the tb_invalidate work for
non-executable spans but left this lookup standing in front of it, so most of
the cost survived that fix.

The map is static once CreateConfiguredEngine has mapped its five spans, so
a small per-thread direct-mapped page -> MemoryRegion cache removes the
lookup for everything but a cold miss. That cache is implemented here but
DEFAULTS OFF (UC_MR_CACHE_ENABLED 0) because it could not be measured:

- the Pixel 6a stopped being measurable (mScreenState=OFF within seconds
  despite a 30-minute timeout and svc power stayon true, with
  NotificationShade holding focus - it needs physical attention)
- the Galaxy A9 cannot run this build at all: "failed to attach / start
  timeout", the engine's startup work exceeds Android's attach deadline on
  that hardware

Shipping it enabled would repeat exactly the mistake this project has been
correcting all session. The measurement that motivates it is solid; the fix
is not yet evidence.

INVARIANT if it is ever enabled: anyone changing mappings or permissions at
runtime must clear the cache, or stale permissions will be used. It is
per-thread, so no locking is involved.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-20 04:29:32 +03:00
megboyzzandClaude daa537bfe8 Verify the exit-check removal at 6.3%; record the TLB experiment as refuted 
Two results, one positive and one negative.

CONFIRMED - removing the exit-request check from the guest load/store path
(ea079b8) is worth 6.3%. That commit went in on a single A/B pair because
the device wedged mid-measurement; re-run on a freshly rebooted Pixel 6a,
three interleaved pairs, load to first OnCarLoaded:

    without the check: 33.42 / 33.10 / 33.32 s   mean 33.28
    with the check:    35.62 / 35.43 / 35.49 s   mean 35.51

Ranges do not overlap and each spread is under a third of a second.

REFUTED - raising CPU_TLB_DYN_DEFAULT_BITS from 8 to 12. The premise checked
out: a probe in tlb_mmu_resize_locked showed it runs only about twice per
second and current_entries stayed at 256 for a whole run, so the adaptive
TLB genuinely never grows for this workload and sits at 1 MB of coverage
against a working set of hundreds of MB.

Raising it took effect (current_entries=4096, verified live) and bought
nothing:

    12 bits: 32.36 / 32.26 s
     8 bits: 32.86 / 32.32 s     0.28s apart, ranges overlap

The profile moved the WRONG way - tlb_set_page_with_attrs 0.86% -> 2.92%,
since a larger TLB costs more to fill and flush. So the softmmu lookup cost
(flatview_translate + find_memory_mapping, ~7.5%) is not driven by TLB
capacity. Reverted to the upstream default, with the refutation recorded in
place so nobody retries it without a new theory.

Also removes the temporary TLB probe.

Worth noting for future measurements: after the device was rebooted, the
same build measured ~33s where it had measured ~41s before. The earlier
drift was device state, not code.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-20 03:59:47 +03:00
megboyzzandClaude ea079b8e9f Stop emitting the exit-request check on every guest load and store 
Unicorn calls check_exit_request() from tcg_gen_qemu_ld_i32/st_i32/ld_i64/
st_i64 - i.e. on EVERY GUEST MEMORY ACCESS, not per basic block. Upstream
QEMU does not do this. It exists so a uc_emu_stop() issued from a MEMORY hook
callback takes effect immediately instead of at the next block boundary.

Found by taking a profile contradiction seriously. In a clean in-game
profile, helper_check_exit_request_arm was the largest single symbol at
8.71%, against helper_uc_tracecode's 0.98% - yet both are emitted together
at the ARM translator's two hook sites and should have matched. Grepping
every emission site across the tree, rather than just target/arm, found the
six in tcg/tcg-op.c.

This engine never stops emulation from a memory callback: mem_fault_hook_cb
only logs and returns false, and the one accelerator that does call
uc_emu_stop (FnvHashAccelHookCb) is a UC_HOOK_CODE hook, where the check is
still emitted. So on the load/store path it is pure overhead.

    helper_check_exit_request_arm   8.71%  ->  4.07%

The remainder is the legitimate per-hook emission. Game still loads, reaches
OnCarLoaded, zero MEM FAULTs.

EVIDENCE LEVEL, stated plainly: one clean interleaved A/B pair, 33.13s
without the check vs 35.89s with it (7.7%), plus the profile share above.
That is weaker than this project's usual bar - the follow-up pairs were lost
when the Pixel's screen dozed off mid-run and the engine stopped rendering
(glClear=0, no faults), and later attempts to reset the save between runs
were blocked. The change should be re-A/B'd on a woken, freshly-booted
device before being treated as settled.

Kept as a named constant rather than deleted because this is a vendored
tree: anyone adding a memory hook that calls uc_emu_stop MUST set
kEmitExitCheckOnMemoryAccess back to 1, or that stop will be deferred to the
next block boundary.

Also turns off the task #56 instrumentation (kLogPerfMetrics,
kCountStubDispatches): both took a steady_clock reading per draw and per shim
crossing, which showed up as __kernel_clock_gettime at 3.67% of a profile -
the measurement was costing nearly as much as some of the things being
measured.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-20 03:10:12 +03:00
megboyzzandClaude 633e99c3da Guest-side __dynamic_cast fast path: works, removes 41% of crossings, changes NOTHING 
Task #59's idea was that __dynamic_cast - 49% of all shim crossings, called
3x more often by native than this engine can serve (task #58) - could run as
emulated ARM32 instead, since it only reads guest memory and needs nothing
from the host. Emulated-but-not-crossing would then beat
native-but-crossing.

It is implemented, correct, and does remove the crossings. It also makes no
measurable difference to load time.

Interleaved A/B, one run per build, alternating to cancel device drift:

    fast path ON   40.67 / 41.13 / 40.98   mean 40.93s
    fast path OFF  41.35 / 40.52 / 41.46   mean 41.11s

0.18s apart with fully overlapping spreads. Noise.

THE RESULT THAT MATTERS IS THE NEGATIVE ONE: shim-crossing COUNT is not what
paces loading. Three separate findings now say this and were previously read
as three unrelated disappointments - removing 17 probe hooks bought nothing
(#48), cheapening the crossing bought ~3% (9c4a455), and removing ~41% of all
crossings buys nothing here. That closes a whole line of optimisation.

Kept behind kEnableGuestFastPath, default OFF: it is real added complexity
(hand-assembled guest Thumb, a second registration name, an executable-memory
allocator) for no measured gain, and a wrong dynamic_cast corrupts state
silently rather than crashing. The measurement and the infrastructure are
worth keeping; the behaviour change is not worth defaulting on.

Two things found along the way that ARE keepers:

- AllocGuestCode(): AllocPermanent carves from the control arena, which is
  mapped read-write only since task #54, so code placed there faults
  immediately with FETCH_PROT at its own entry address. Making the control
  arena executable to accommodate it cost a measured ~4s. AllocGuestCode
  carves from the trampoline arena, which is already executable and is not
  write-hot. Anything generating guest code must use it.

- Device drift is large enough to invalidate cross-session comparisons: the
  identical build that measured 35.20s earlier today measured 41.11s a few
  hours later. Only interleaved A/B within one sitting is trustworthy. Three
  near-miss wrong conclusions today traced back to comparing against a stale
  baseline.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-19 21:21:28 +03:00
megboyzzandClaude 7af48860eb Survey dynamic_cast call shapes + native rate counter: 84% need no hierarchy walk 
Task #58 asked whether the engine induces the __dynamic_cast flood or the
game does. A native LD_PRELOAD interposer added to trace_agent answers it:
native ARM32 on the A9 calls it up to 824,200/sec, against this engine's
284,986/sec. The game is simply that RTTI-heavy, and native absorbs it
because a real call costs tens of nanoseconds. Our rate is not demand, it is
supply - the shim boundary is throttling the guest.

(Interposing __dynamic_cast collides with libc++abi.a, which the NDK links
statically into the agent; resolved with --allow-multiple-definition scoped
to that target, since our object precedes the archive and wins.)

That makes a guest-side implementation interesting: Shim_dynamic_cast is
already a pure guest-memory walk - it reads the object's vtable, vtable[-1]
(dynamic type) and vtable[-2] (offset-to-top), then compares type_info
records. Nothing comes from the host, so the same algorithm could run as
emulated ARM32 with NO boundary crossing at all.

Whether that is worth doing depends on how big it would have to be, so this
adds a temporary shape survey. Measured over a prologue load, 2.7M calls:

    exact (depth 0)  84.2%
    one base         0.4%
    deeper           0.0%     (max depth reached all run: 3)
    not found       15.4%

84% of calls need NO hierarchy walk - the object's dynamic type already IS
the target. A guest fast path of roughly six instructions (load vtable,
compare vtable[-1] to the target, return ptr + vtable[-2]) with a fallback
to this shim would eliminate 84% of dynamic_cast crossings, which is ~41% of
ALL shim crossings given dynamic_cast is 49% of them.

Also confirms there is no implementation to reuse inside libapp.so: not only
__dynamic_cast but the type_info vtables themselves (__class_type_info,
__si_class_type_info, __vmi_class_type_info, __pointer_type_info) are all
UND - this engine supplies them from rtti_shims.cpp.

Survey left in the tree behind kSurveyDynamicCast, default off.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-19 20:28:01 +03:00
megboyzzandClaude 9c4a455bb0 Cheapen the shim-call boundary: lock-free dispatch + batched register reads 
Every shim crossing took a std::mutex in MiscStubDispatch and five separate
uc_reg_read calls in import_stub_dispatch_cb. Measured at ~170,000 crossings
per second during gameplay, from several threads.

Dispatch table is now a fixed array plus an atomic published count. It is
append-only by construction (AllocCodeStub bump-allocates 4 bytes per stub
and refuses past the arena end), entries are never removed or rewritten, and
a writer fills its slot before publishing the count with release ordering -
so an acquiring reader always sees a complete entry. The mutex remains on
the append side, which runs a few hundred times at startup.

Register reads go through one uc_reg_read_batch instead of five API entries.

RESULT, and it is smaller than the reasoning predicted:

    time to first OnCarLoaded  36.35s -> 35.20s   (35.14 / 35.28 / 35.18)

About 3%, consistent across runs. The estimate that each crossing costs
~6us - derived by dividing a saturated core by the measured crossing rate -
implied far more headroom than this. Either the mutex and register reads
were a small part of the crossing cost, or the thread is not actually
saturated and the estimate is wrong. Recorded rather than explained away;
the change is correct and free either way, but it is not the lever the
arithmetic suggested.

Also adds temporary task #56 instrumentation, all opt-in:
- gles_shim PERF line: fps, draws/frame, shimCalls/frame in ONE line, so the
  correlation is readable live instead of joined across logs afterwards.
- guest_engine TOPSHIMS line: the busiest shims once a second, counted per
  ImportStubEntry so no shared map or mutex taxes the hot path.

Those two measurements corrected an earlier wrong conclusion. Draw calls
looked like the cause of the in-race slowdown the user demonstrated, but
with the crossing counter in place: draws/frame 178 vs 149 across a 4x fps
gap, while shimCalls/frame went 22,000 vs 5,600 - and fps falls out exactly
as crossings-per-second / crossings-per-frame (148350/4385 = 33.8,
167811/21665 = 7.7). Draws were a bystander.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-19 19:33:53 +03:00
megboyzzandClaude 1a74432c48 Add an opt-in glViewport probe; settles task #52 (0.8 render scale, not a bug) 
A Xiaomi 14 capture showed viewport 2136x960 against a 2670x1200 surface and
raised the question of whether the engine was feeding the game a wrong size -
the shape of task #39, where the UI projection used 2000x1000 instead of the
real 2400x1080.

Logging each DISTINCT viewport with its bound framebuffer and the real EGL
surface size answers it. Pixel 6a:

    VIEWPORT [0,0 2400x1080] fb=0  | EGL surface 2400x1080   <- screen, exact
    VIEWPORT [0,0 1920x864]  fb=3  | EGL surface 2400x1080   <- an FBO
    VIEWPORT [0,0 512x512]   fb=1  | EGL surface 2400x1080   <- another FBO

The default framebuffer matches the surface exactly; the smaller viewport is
an offscreen target. And the ratio is decisive:

    Pixel 6a   1920/2400 = 0.80   864/1080 = 0.80
    Xiaomi 14  2136/2670 = 0.80   960/1200 = 0.80

Exactly 0.8 on both devices and both axes - the game's own render scale,
drawing the 3D scene into a reduced FBO and upscaling while the HUD stays at
full resolution. Nothing is broken; the engine reproduces it faithfully.

The probe stays in the tree behind kLogViewportChanges (default off, per the
standing rule that diagnostics are opt-in), because "which framebuffer was
bound" is exactly the context whose absence made the original observation
ambiguous.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-09-19 17:34:23 +03:00
65 changed files with 11808 additions and 116 deletions
Expand all files Collapse all files
.gitignore
+25
View File
@@ -22,5 +22,30 @@ local.properties
*.til
# trace_agent is built by its own build.sh, outside Gradle
trace_agent/build/
# ostream_repro likewise - CMake/ninja output, the sources next to it are kept
ostream_repro/build/
# ad-hoc device capture logs
device_run_*.log
# Where `gradlew assembleTranslatedRelease` output was collected by hand for
# signing. 611 MB of signed APK - a build product, and one that carries the
# bundled OBB with it, so it must never reach history.
app/translated/
# Release signing material never belongs in the repo - see app/build.gradle.kts
*.jks
*.keystore
keystore.properties
*-release.properties
# Bundled game data (~595 MB). A build INPUT, not source: it is the game's own
# OBB, copied into assets so the APK self-extracts it on first launch (see
# GameDataInstaller.kt). Keeping it out of history on purpose - a 595 MB blob
# is rejected outright by most hosts, and once committed it can only be removed
# by rewriting history.
#
# To build: copy the OBB here yourself, under exactly this name.
# cp main.<versionCode>.<package>.obb app/src/main/assets/game_data.obb
# Without it the app still builds and runs; it just expects the OBB to already
# be on the device at getObbDir(), the way it was before this was bundled.
app/src/main/assets/game_data.obb
.idea/deploymentTargetSelector.xml
Generated
+16 -7
View File
@@ -2,15 +2,24 @@
<project version="4">
<component name="deploymentTargetSelector">
<selectionStates>
<SelectionState runConfigName="savemanager">
<option name="selectionMode" value="DROPDOWN" />
<DialogSelection />
</SelectionState>
<SelectionState runConfigName="app">
<option name="selectionMode" value="DROPDOWN" />
<DropdownSelection timestamp="2025-08-28T12:35:54.649730307Z">
<Target type="DEFAULT_BOOT">
<handle>
<DeviceId pluginId="PhysicalDevice" identifier="serial=2B101JEGR07875" />
</handle>
</Target>
</DropdownSelection>
<DialogSelection />
</SelectionState>
<SelectionState runConfigName="main">
<option name="selectionMode" value="DROPDOWN" />
<DialogSelection />
</SelectionState>
<SelectionState runConfigName="unitTest">
<option name="selectionMode" value="DROPDOWN" />
<DialogSelection />
</SelectionState>
<SelectionState runConfigName="androidTest">
<option name="selectionMode" value="DROPDOWN" />
<DialogSelection />
</SelectionState>
</selectionStates>
app/build.gradle.kts
+168 -5
View File
@@ -1,3 +1,8 @@
// Inside the android { } block the name `java` resolves to Gradle's own
// java extension and shadows the package, so these types have to be
// imported here rather than fully qualified at the use site.
import java.util.Properties
plugins {
alias(libs.plugins.android.application)
alias(libs.plugins.kotlin.android)
@@ -5,22 +10,78 @@ plugins {
}
android {
namespace = "com.ea.games.nfs13_mod"
// ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md): "_arm"
// instead of "_mod" so this build can be installed side by side with
// the real armeabi-v7a mod, not overwrite it.
namespace = "com.ea.games.nfs13_arm"
compileSdk = 35
defaultConfig {
applicationId = "com.ea.games.nfs13_mod"
applicationId = "com.ea.games.nfs13_arm"
minSdk = 27
targetSdk = 35
versionCode = 1003128
versionName = "1.3.128"
ndk.abiFilters.add("armeabi-v7a")
//ndk.abiFilters.add("x86")
// mpcore is now plain native ARM64 code driving an embedded ARM32
// CPU-emulation core - no armeabi-v7a native code is built/shipped
// for this app at all.
ndk.abiFilters.add("arm64-v8a")
// x86_64 (2026-09-22): lets the translated build install in WayDroid,
// whose Android image is x86_64. Unicorn picks its tcg/i386 backend
// there automatically, so the ARM32 guest is translated by US rather
// than by houdini - which is what made earlier WayDroid attempts with
// the native ARM build unreliable.
//
// NOTE: the flat guest mapping (task #61, worth 1.24x) is implemented
// only in tcg/aarch64, so an x86_64 host silently falls back to the
// software MMU and will be slower. Fine for a test target; port the
// same change into tcg/i386 if it ever becomes a shipping target.
ndk.abiFilters.add("x86_64")
// Default: false. Overridden to true by the "native32" flavor - see
// its own comment below.
buildConfigField("boolean", "NATIVE32", "false")
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
}
ndkVersion = "21.0.6113669"
ndkVersion = "27.0.12077973"
// Release signing. The keystore and its password live OUTSIDE this repo
// (~/keystores/), deliberately: this is a git working tree, and a signing
// identity that can be committed eventually is committed. Gradle reads
// them from a properties file if it is there, and falls back to the debug
// key if it is not - so a checkout without the key still builds, it just
// produces something that cannot be shipped.
//
// To create the key (once, and then back it up - losing it means never
// being able to update an installed build):
//
// keytool -genkeypair -v -keystore ~/keystores/nfsmw-release.jks \
// -alias nfsmw -keyalg RSA -keysize 4096 -validity 10950
//
// then write ~/keystores/nfsmw-release.properties with:
// storeFile=/home/<you>/keystores/nfsmw-release.jks
// storePassword=...
// keyAlias=nfsmw
// keyPassword=...
signingConfigs {
val releaseProps = File(System.getProperty("user.home"), "keystores/nfsmw-release.properties")
if (releaseProps.isFile) {
val props = Properties().apply { releaseProps.inputStream().use { load(it) } }
create("release") {
storeFile = file(props.getProperty("storeFile"))
storePassword = props.getProperty("storePassword")
keyAlias = props.getProperty("keyAlias")
keyPassword = props.getProperty("keyPassword")
// Both modern signature schemes: v2 covers the whole APK and
// is what Android 7+ verifies, v3 allows rotating the key
// later without invalidating existing installs.
enableV2Signing = true
enableV3Signing = true
}
}
}
buildTypes {
release {
@@ -29,6 +90,28 @@ android {
getDefaultProguardFile("proguard-android-optimize.txt"),
"proguard-rules.pro"
)
// Local-testing-only (2026-09-05, "why is this ~100x slower than
// native" investigation - see ARM64_TRANSLATION_LAYER.md): no
// real release signing config exists in this prototype, and
// there's no intent to ship this build anywhere - reusing the
// auto-generated debug keystore just makes `assembleTranslatedRelease`
// installable on a test device the same way translatedDebug
// already is. The actual point of testing this build type is the
// NATIVE side (mpcore's CMakeLists.txt now forces
// CMAKE_BUILD_TYPE=RelWithDebInfo unconditionally, so this
// isn't even required for that fix to take effect - it's here so
// this variant can be installed at all for a genuine debuggable=false
// comparison, e.g. CheckJNI's behavior).
// Real key when it exists, debug key otherwise. Announced at
// configuration time rather than discovered later from a
// mysteriously unshippable APK.
signingConfig = signingConfigs.findByName("release") ?: run {
logger.warn(
"RELEASE SIGNING: ~/keystores/nfsmw-release.properties not found - " +
"signing with the DEBUG key. This APK must NOT be distributed."
)
signingConfigs.getByName("debug")
}
}
}
compileOptions {
@@ -38,8 +121,88 @@ android {
kotlinOptions {
jvmTarget = "11"
}
androidResources {
// The bundled game data (assets/game_data.obb, ~595 MB) is an already
// compressed archive - deflating it again would cost a long build and
// a slower install for no size gain, and it must stay STORED so it can
// be streamed straight out of the APK.
noCompress += "obb"
}
buildFeatures {
compose = true
buildConfig = true
}
// "native32" flavor - diagnostic-only reference build for the native
// ARM32 tracing investigation (see ARM64_TRANSLATION_LAYER.md /
// sorted-popping-plum.md's "Native ARM32 tracing harness" plan). Real
// armeabi-v7a support and a BuildConfig.NATIVE32 flag GameActivityMain
// uses to load the real, unmodified native libraries directly
// (System.loadLibrary) instead of going through mpcore's emulation path
// - only meaningful on genuinely 32-bit-capable hardware (the Galaxy
// A9), never intended to ship. MUST be armeabi-v7a-ONLY (not just
// "armeabi-v7a added alongside arm64-v8a") - confirmed live on-device
// that shipping both ABIs makes Android launch the process via the
// 64-bit app_process64/Zygote (arm64-v8a is present, so it's preferred),
// which can never dlopen/LD_PRELOAD a 32-bit .so at all (a process's
// bitness is fixed for its whole lifetime, not per-library) - defeats
// the entire point of this flavor. ndk.abiFilters.clear() first,
// since AGP's per-flavor abiFilters otherwise ADD to defaultConfig's
// set rather than replace it.
flavorDimensions += "abi"
productFlavors {
create("translated") {
dimension = "abi"
// Default/existing behavior - the emulated arm64-v8a path,
// unchanged from before this flavor split existed.
}
create("native32") {
dimension = "abi"
ndk.abiFilters.clear()
ndk.abiFilters.add("armeabi-v7a")
buildConfigField("boolean", "NATIVE32", "true")
}
}
// 2026-09-16 (ARM64_TRANSLATION_LAYER.md - trace_agent wrap.sh deployment):
// manually injecting a bundled wrap.sh into the packaged native32 APK
// (AGP's own native-lib merge/strip pipeline only recognizes *.so files
// and silently drops anything else) failed to install at all with
// "Failed to extract native libraries, res=-2" under the default
// extractNativeLibs=false packaging - PackageManager validates every
// entry under lib/<abi>/ as a loadable library when it plans to mmap
// straight from the APK, and a plain shell script fails that check.
// Legacy (extracted-to-disk) packaging doesn't do that same strict
// validation. Only applied to native32 - the translated flavor doesn't
// need or want this (slightly larger install, marginally slower first
// native-lib load).
productFlavors.getByName("native32") {
packaging {
jniLibs {
useLegacyPackaging = true
}
}
}
}
// 2026-09-06 (native32-on-real-hardware launch investigation - see
// ARM64_TRANSLATION_LAYER.md): ndk.abiFilters above only restricts THIS
// module's own native code - the "native32" flavor still needs `mpcore` on
// the compile classpath (GameActivityMain.kt's shared, non-flavor-specific
// source calls mpcore functions in its non-NATIVE32 branch), and mpcore's
// own CMakeLists always builds arm64-v8a regardless of which app flavor
// pulls it in. That arm64-v8a .so was silently merged into the native32
// APK too, so a real device with both an arm64-v8a AND an armeabi-v7a lib
// present chose to launch the process as 64-bit (app_process64) - which can
// never load this flavor's 32-bit fmodex/fmodevent/Nimble/app libraries at
// all, confirmed live via `UnsatisfiedLinkError: couldn't find "libfmodex.so"`
// (nativeLibraryDirectories only listed arm64/arm64-v8a paths). Explicitly
// drop every arm64-v8a .so from the native32 variant's packaging so the
// APK is genuinely armeabi-v7a-only, restoring 32-bit process selection.
androidComponents {
onVariants(selector().withFlavor("abi" to "native32")) { variant ->
variant.packaging.jniLibs.excludes.add("lib/arm64-v8a/*.so")
}
}
app/src/androidTest/java/com/ea/games/nfs13_arm/ExampleInstrumentedTest.kt
+2 -2
View File
@@ -1,4 +1,4 @@
package com.ea.games.nfs13_mod
package com.ea.games.nfs13_arm
import androidx.test.platform.app.InstrumentationRegistry
import androidx.test.ext.junit.runners.AndroidJUnit4
@@ -19,6 +19,6 @@ class ExampleInstrumentedTest {
fun useAppContext() {
// Context of the app under test.
val appContext = InstrumentationRegistry.getInstrumentation().targetContext
assertEquals("com.ea.games.nfs13_mod", appContext.packageName)
assertEquals("com.ea.games.nfs13_arm", appContext.packageName)
}
}
app/src/main/AndroidManifest.xml
+29
View File
@@ -26,6 +26,7 @@
<uses-configuration android:reqTouchScreen="finger"/>
<application
android:name="nfs.mod.traceagent.TraceApplication"
android:theme="@style/Theme.AppCompat.NoActionBar"
android:label="@string/app_name"
android:icon="@mipmap/adaptive_icon"
@@ -48,6 +49,34 @@
android:value="bar"/>
</activity>
<!-- One-time unpack of the game data bundled in the APK. -->
<activity
android:name="com.ea.ironmonkey.GameDataUnpackActivity"
android:exported="false"
android:excludeFromRecents="true"
android:label="Подготовка данных" />
<!-- Shown on the launch after a crash, never during one - a signal
handler cannot start an Activity (see crash_handler.cpp). -->
<activity
android:name="com.ea.ironmonkey.CrashReportActivity"
android:exported="false"
android:excludeFromRecents="true"
android:label="Crash report" />
<!-- Lets the tester share the zip. Required: targetSdk 35 rejects
file:// URIs in ACTION_SEND. Grants access to the crash directory
only, and only for the duration of the send. -->
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.crashreports"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/crash_report_paths" />
</provider>
<activity
android:exported="true"
android:name="com.ea.ironmonkey.PermissionsActivity"
app/src/main/assets/native_probe/libapp.so
BIN
View File
Binary file not shown.
app/src/main/assets/native_probe/libfmodevent.so
BIN
View File
Binary file not shown.
app/src/main/assets/native_probe/libfmodex.so
BIN
View File
Binary file not shown.
app/src/main/java/com/ea/ironmonkey/CrashReportActivity.kt
+237
View File
@@ -0,0 +1,237 @@
package com.ea.ironmonkey
import android.content.Intent
import android.os.Build
import android.os.Bundle
import android.util.Log
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.verticalScroll
import androidx.compose.material3.Button
import androidx.compose.material3.Card
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.OutlinedButton
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.darkColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.unit.dp
import androidx.compose.ui.unit.sp
import androidx.core.content.FileProvider
import java.io.File
import java.text.SimpleDateFormat
import java.util.Date
import java.util.Locale
import java.util.zip.ZipEntry
import java.util.zip.ZipOutputStream
/**
* Shown on the launch AFTER a crash, never during one.
*
* The native handler (crash_handler.cpp) can only write a file - a signal
* handler runs on an already-broken process where JNI and Activities are not
* legal. So it drops `crash_pending.txt` and dies; this screen is what the
* tester sees next time they open the game.
*
* Everything it produces stays on the device unless the tester presses send.
* There is no backend and no automatic upload - see BETA_TELEMETRY_PLAN.md.
*
* Built with Compose/Material3 and its own dark colour scheme rather than the
* game's `Theme.AppCompat.NoActionBar`: this screen must render correctly no
* matter what the rest of the app's theming does, and it is the one screen a
* tester sees when everything else has already gone wrong.
*/
class CrashReportActivity : ComponentActivity() {
companion object {
private const val TAG = "CrashReport"
private const val PENDING_NAME = "crash_pending.txt"
/** Reports live here: Android/data/<pkg>/files/crashes - reachable over USB, no permission needed. */
fun crashDir(activity: android.app.Activity): File =
File(activity.getExternalFilesDir(null), "crashes")
/** The file the native handler writes. Null if there is nothing waiting. */
fun pendingReport(activity: android.app.Activity): File? =
File(crashDir(activity), PENDING_NAME).takeIf { it.isFile && it.length() > 0 }
}
private var bundle: File? = null
override fun onCreate(savedInstanceState: Bundle?) {
// Android 15 draws every app edge to edge whether it asks or not, so
// the choice is between handling insets or having text sit under the
// status bar. Declaring it explicitly and letting Scaffold apply the
// padding is the supported way; the first version did neither and the
// content ran under the system bars.
enableEdgeToEdge()
super.onCreate(savedInstanceState)
val pending = pendingReport(this)
if (pending == null) {
// Nothing to report - never block the tester on an empty screen.
finish()
return
}
// Rename out of the way FIRST, before anything that could fail. The
// native handler always writes the same fixed name (it cannot safely
// format a timestamp inside a signal handler), so leaving it in place
// would let the next crash overwrite a report not yet sent.
val stamp = SimpleDateFormat("yyyyMMdd-HHmmss", Locale.US).format(Date())
val kept = File(crashDir(this), "crash-$stamp.txt")
if (!pending.renameTo(kept)) {
Log.w(TAG, "could not rename $pending - reporting it in place")
}
val report = if (kept.isFile) kept else pending
val details = buildString {
append(deviceSummary())
append("\n\n")
append(runCatching { report.readText() }.getOrElse { "(could not read the report: $it)" })
}
bundle = runCatching { zipReport(report, details, stamp) }
.onFailure { Log.w(TAG, "could not build the zip", it) }
.getOrNull()
setContent {
MaterialTheme(colorScheme = darkColorScheme()) {
CrashReportScreen(
details = details,
bundlePath = bundle?.absolutePath,
canSend = bundle != null,
onSend = ::share,
onContinue = ::startGame,
)
}
}
}
private fun deviceSummary(): String = buildString {
append("device: ${Build.MANUFACTURER} ${Build.MODEL}\n")
append("android: ${Build.VERSION.RELEASE} (API ${Build.VERSION.SDK_INT})\n")
append("soc: ${Build.HARDWARE}\n")
append("abis: ${Build.SUPPORTED_ABIS.joinToString(", ")}\n")
append("app: ${appVersion()}")
}
private fun appVersion(): String = runCatching {
val p = packageManager.getPackageInfo(packageName, 0)
"${p.versionName} (${p.longVersionCode})"
}.getOrElse { "(unknown)" }
private fun zipReport(report: File, details: String, stamp: String): File {
val out = File(crashDir(this), "crash-$stamp.zip")
ZipOutputStream(out.outputStream().buffered()).use { zip ->
zip.putNextEntry(ZipEntry("crash.txt"))
zip.write(details.toByteArray())
zip.closeEntry()
if (report.isFile) {
zip.putNextEntry(ZipEntry(report.name))
report.inputStream().use { it.copyTo(zip) }
zip.closeEntry()
}
}
return out
}
private fun share() {
val file = bundle ?: return
val uri = runCatching {
FileProvider.getUriForFile(this, "$packageName.crashreports", file)
}.getOrElse {
Log.w(TAG, "FileProvider failed for $file", it)
return
}
val send = Intent(Intent.ACTION_SEND).apply {
type = "application/zip"
putExtra(Intent.EXTRA_STREAM, uri)
putExtra(Intent.EXTRA_SUBJECT, "NFSMW arm64 - отчёт о сбое")
addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION)
}
startActivity(Intent.createChooser(send, "Отправить отчёт"))
}
private fun startGame() {
startActivity(Intent(this, GameActivityMain::class.java))
finish()
}
}
@Composable
private fun CrashReportScreen(
details: String,
bundlePath: String?,
canSend: Boolean,
onSend: () -> Unit,
onContinue: () -> Unit,
) {
// Scaffold's innerPadding carries the system-bar insets, so nothing ends up
// under the status bar or the gesture handle.
Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(innerPadding)
.padding(16.dp)
) {
Text(
text = "Игра аварийно завершилась",
style = MaterialTheme.typography.headlineSmall,
)
Text(
// Say plainly what is in the file before offering to send it.
// The tester is the one sending it; they should know what it
// contains.
text = "Отчёт сохранён на устройстве. В нём модель телефона, версия Android, " +
"версия сборки и технические данные о сбое. Личных данных и игрового " +
"аккаунта в нём нет.",
style = MaterialTheme.typography.bodyMedium,
modifier = Modifier.padding(top = 8.dp),
)
if (bundlePath != null) {
Text(
text = bundlePath,
style = MaterialTheme.typography.bodySmall,
color = MaterialTheme.colorScheme.onSurfaceVariant,
modifier = Modifier.padding(top = 8.dp),
)
}
Card(modifier = Modifier
.fillMaxWidth()
.padding(top = 16.dp)
.weight(1f)
) {
Text(
text = details,
fontSize = 11.sp,
lineHeight = 15.sp,
modifier = Modifier
.verticalScroll(rememberScrollState())
.padding(12.dp),
)
}
Row(
modifier = Modifier
.fillMaxWidth()
.padding(top = 16.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp, Alignment.End),
) {
OutlinedButton(onClick = onContinue) { Text("Продолжить") }
Button(onClick = onSend, enabled = canSend) { Text("Отправить отчёт") }
}
}
}
}
app/src/main/java/com/ea/ironmonkey/GameActivityMain.kt
+186 -18
View File
@@ -40,6 +40,7 @@ import com.ea.nimble.ApplicationLifecycle
import com.ea.nimble.Global
import nfs.mod.mpcore.GameInput
import nfs.mod.mpcore.MultiplayerCore.loadCore
import nfs.mod.mpcore.MultiplayerCore.loadEmulatedLibapp
import nfs.mod.mpcore.SyntheticInputDispatcher
import org.fmod.FMODAudioDevice
import java.io.File
@@ -50,6 +51,9 @@ import java.util.concurrent.TimeUnit
import javax.microedition.khronos.egl.EGLConfig
import javax.microedition.khronos.opengles.GL10
import androidx.core.net.toUri
import androidx.core.view.WindowCompat
import androidx.core.view.WindowInsetsCompat
import androidx.core.view.WindowInsetsControllerCompat
import com.ea.ironmonkey.domain.FSNode
import org.apache.http.BuildConfig
import kotlin.system.exitProcess
@@ -76,9 +80,24 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
private var isSleepModeEnabled = true
private var mRotation = 0
// nativeRestoreContext() is a real native call that can block for a
// long time (on the current emulated engine, indefinitely - see
// ARM64_TRANSLATION_LAYER.md's render-stall root cause: it transitively
// reaches NimbleWrapper::InitNimble(), which gets stuck polling an
// empty directory). Real hardware keeps this kind of init work off the
// GLThread entirely (confirmed via a native trace comparison - the
// real device's own version-check activity and its GL calls run on two
// separate OS threads). Dispatched to its own background thread here so
// onDrawFrame - which MUST keep running every frame to keep rendering -
// never blocks on it, instead of calling it synchronously every frame
// like the original code did.
@Volatile private var restoreContextThreadStarted = false
@Volatile private var restoreContextDone = false
@Volatile private var restoreContextResult = false
// cont.44 DEBUG: fires MultiplayerCore.triggerCarSelectTest() on
// demand via `adb shell am broadcast -a nfs.mod.mpcore.TEST_OPEN_CARSELECT
// -p com.ea.games.nfs13_mod` - stands in for a future real lobby-overlay
// -p com.ea.games.nfs13_arm` - stands in for a future real lobby-overlay
// "select car" button, letting the on-demand car_select-opening call be
// exercised at an arbitrary moment (not just automatically at boot) for
// live testing. See PROGRESS.md cont.44.
@@ -90,7 +109,7 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
// cont.48 DEBUG: fires the EXPERIMENTAL true-direct-jump variant via
// `adb shell am broadcast -a nfs.mod.mpcore.TEST_TRUE_DIRECT_CARSELECT
// -p com.ea.games.nfs13_mod`. See PROGRESS.md cont.48.
// -p com.ea.games.nfs13_arm`. See PROGRESS.md cont.48.
private val trueDirectCarSelectTestReceiver = object : android.content.BroadcastReceiver() {
override fun onReceive(context: android.content.Context?, intent: Intent?) {
nfs.mod.mpcore.MultiplayerCore.triggerTrueDirectCarSelectJump()
@@ -100,7 +119,19 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
private fun updateRequestedOrientation(i: Int) {
}
fun IsSystemKey(i: Int) = false
// Keys the SYSTEM must handle, not the game.
//
// This used to be a flat `= false`, which combined with the callers'
// `return !IsSystemKey(keyCode)` meant the activity claimed EVERY key it
// ever saw - including volume. That is why changing the volume needed the
// notification shade: the keys were being swallowed before Android could
// act on them.
fun IsSystemKey(i: Int) = when (i) {
KeyEvent.KEYCODE_VOLUME_UP,
KeyEvent.KEYCODE_VOLUME_DOWN,
KeyEvent.KEYCODE_VOLUME_MUTE -> true
else -> false
}
companion object {
private const val DOWNLOAD_PROPERTIES = "downloadcontent/config.properties"
@@ -156,11 +187,36 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
}
fun useAssetsFileSystem(): Boolean {
return mAssetLocationType != AssetLocationType.EXTERNAL
val result = mAssetLocationType != AssetLocationType.EXTERNAL
d(TAG, "useAssetsFileSystem() called, mAssetLocationType=$mAssetLocationType, result=$result, thread=${Thread.currentThread().name}")
return result
}
external fun nativeOnCreate()
/**
* Hooks the fatal signals so a native crash leaves a report behind. Must be
* called AFTER loadCore(); see its call site.
*/
private fun installNativeCrashHandler() {
try {
val dir = java.io.File(getExternalFilesDir(null), "crashes")
val info = packageManager.getPackageInfo(packageName, 0)
nativeInstallCrashHandler(
dir.absolutePath,
"${info.versionName} (${info.longVersionCode}) ${Build.MANUFACTURER} ${Build.MODEL} " +
"Android ${Build.VERSION.RELEASE}"
)
} catch (e: Throwable) {
// Diagnostics must never stop the game from starting - but say so,
// otherwise "no crash reports" looks like "no crashes".
i(TAG, "installNativeCrashHandler FAILED - native crashes will go unreported: $e")
}
}
/** See crash_handler.cpp. Writes to `dir`; `buildStamp` goes at the top of every report. */
external fun nativeInstallCrashHandler(dir: String, buildStamp: String)
external fun nativeOnDestroy()
external fun nativeOnMusicPlayerStateChanged()
@@ -199,6 +255,13 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
i(TAG, "onCreate")
super.onCreate(bundle)
// Volume keys should move the MUSIC stream - the one the game actually
// plays through - rather than whatever Android would pick by default.
volumeControlStream = AudioManager.STREAM_MUSIC
applyImmersiveMode()
handler = Handler()
@@ -244,14 +307,18 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
i(TAG, "onCreate() isMusicActive = $isAnyMusicPlaying")
mFMODAudioDevice = FMODAudioDevice()
d(TAG, "obb.size check: mAssetLocationType before = $mAssetLocationType")
if (mAssetLocationType != AssetLocationType.ASSETS) {
try {
val open2 = getResources().assets.open("obb.size")
mAssetLocationType = AssetLocationType.OBB
open2.close()
d(TAG, "obb.size check: opened OK, mAssetLocationType now OBB")
} catch (e2: IOException) {
Log.e(TAG, e2.message!!)
Log.e(TAG, "obb.size check FAILED: " + e2.message)
}
} else {
d(TAG, "obb.size check: skipped, already ASSETS")
}
val sensorManager = getSystemService(SENSOR_SERVICE) as SensorManager
val defaultSensor = sensorManager.getDefaultSensor(1)
@@ -281,12 +348,39 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
mFrameLayout.addView(gameGLSurfaceView)
mFrameLayout.addView(buildCarSelectionOverlay())
setContentView(mFrameLayout)
System.loadLibrary("fmodex")
System.loadLibrary("fmodevent")
System.loadLibrary("c++_shared")
System.loadLibrary(Global.NIMBLE_ID)
System.loadLibrary("app")
loadCore()
// ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md):
// fmodex/fmodevent/Nimble/app are all armeabi-v7a-only builds (no
// arm64-v8a .so shipped for any of them, confirmed this session -
// see jniLibs/) - System.loadLibrary() for them would throw
// UnsatisfiedLinkError immediately on an arm64-v8a-only APK. libapp.so
// is loaded through mpcore's embedded ARM32 emulation core instead
// (see loadCore() below); FMOD/Nimble audio and the real game
// render/boot loop are NOT bridged yet - this build proves the
// emulation-core load+hook mechanism, not a running game.
//
// BuildConfig.NATIVE32 (the "native32" Gradle flavor - see
// app/build.gradle.kts and the native-arm32-trace-harness plan in
// ARM64_TRANSLATION_LAYER.md) restores the ORIGINAL, real loading
// path instead: on genuinely 32-bit-capable hardware (the Galaxy
// A9), load the real, unmodified armeabi-v7a libraries directly, no
// emulation involved - the ground-truth reference build for that
// investigation. Never true for the normal (translated) flavor.
if (com.ea.games.nfs13_arm.BuildConfig.NATIVE32) {
System.loadLibrary("fmodex")
System.loadLibrary("fmodevent")
System.loadLibrary("c++_shared")
System.loadLibrary(Global.NIMBLE_ID)
System.loadLibrary("app")
} else {
loadCore()
// Immediately after loadCore(), and not a line earlier: this is
// the first moment libmpcore.so is loaded, so it is the first
// moment the native symbol exists. Installing it up in onCreate's
// opening lines threw UnsatisfiedLinkError, silently leaving every
// crash unreported.
installNativeCrashHandler()
loadEmulatedLibappFromAssets()
}
// RECEIVER_EXPORTED: needs to be reachable from an external `adb
// shell am broadcast` sender (there's no in-app sender for this
// debug-only trigger), and ContextCompat handles the API 33+
@@ -307,6 +401,42 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
nativeOnCreate()
}
// ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md): copies
// the bundled armeabi-v7a libapp.so (assets/native_probe/libapp.so - a
// raw asset, not jniLibs, since this app declares only arm64-v8a) to a
// real file the first time, then hands mpcore's embedded ARM32
// emulation core the path. Idempotent - skips the copy if the file
// already exists with a plausible size.
private fun loadEmulatedLibappFromAssets() {
val outFile = File(filesDir, "libapp_armeabi_v7a.so")
try {
if (!outFile.exists() || outFile.length() < 1_000_000L) {
assets.open("native_probe/libapp.so").use { input ->
outFile.outputStream().use { output -> input.copyTo(output) }
}
d(TAG, "loadEmulatedLibappFromAssets: extracted asset to " + outFile.absolutePath)
}
// The game's own ARM32 FMOD (task #67). Shipped as raw assets for the
// same reason libapp.so is: this APK declares only arm64-v8a, so
// jniLibs/armeabi-v7a is never packaged. The engine loads them as
// secondary guest images so libapp's FMOD imports reach real code
// instead of fmod_shims.cpp's no-ops.
for (lib in arrayOf("libfmodex.so", "libfmodevent.so")) {
val f = File(filesDir, lib)
if (!f.exists() || f.length() < 1_000L) {
assets.open("native_probe/" + lib).use { input ->
f.outputStream().use { output -> input.copyTo(output) }
}
d(TAG, "loadEmulatedLibappFromAssets: extracted " + f.absolutePath)
}
}
val ok = loadEmulatedLibapp(outFile.absolutePath)
d(TAG, "loadEmulatedLibappFromAssets: loadEmulatedLibapp -> " + ok)
} catch (e: IOException) {
d(TAG, "loadEmulatedLibappFromAssets: failed to extract/load asset: " + e)
}
}
fun forEach(input: String?): Array<String?> {
return arrayOf<String?>(input)
}
@@ -503,6 +633,10 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
override fun onKeyDown(keyCode: Int, keyEvent: KeyEvent): Boolean {
super.onKeyDown(keyCode, keyEvent)
// Checked BEFORE the state gate on purpose: the gate returns true for
// every key whenever the game is not in STATE_GAME_START, which would
// otherwise keep swallowing volume through the whole load.
if (IsSystemKey(keyCode)) return false
if (state != 8) {
return true
}
@@ -518,6 +652,7 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
override fun onKeyUp(i: Int, keyEvent: KeyEvent): Boolean {
super.onKeyUp(i, keyEvent)
if (IsSystemKey(i)) return false
if (state != 8) {
return true
}
@@ -545,7 +680,28 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
}
}
/**
* Hides the status and navigation bars for the duration of the game.
*
* Nothing did this before, which is why the navigation bar sat on top of
* the game. TRANSIENT_BARS_BY_SWIPE keeps them reachable - a swipe from the
* edge brings them back temporarily - so this hides the bars without taking
* the system away from the player.
*
* Must be re-applied on every focus gain: Android restores the bars after a
* dialog, the shade, or a task switch, and a one-shot call in onCreate
* silently stops working the first time any of those happens.
*/
private fun applyImmersiveMode() {
WindowCompat.setDecorFitsSystemWindows(window, false)
WindowInsetsControllerCompat(window, window.decorView).apply {
hide(WindowInsetsCompat.Type.systemBars())
systemBarsBehavior = WindowInsetsControllerCompat.BEHAVIOR_SHOW_TRANSIENT_BARS_BY_SWIPE
}
}
override fun onWindowFocusChanged(z: Boolean) {
if (z) applyImmersiveMode()
super.onWindowFocusChanged(z)
i(TAG, "onWindowsFocusChanged($z) state=$state")
getGameGLSurfaceView().renderMode = if (z) 1 else 0
@@ -714,9 +870,9 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
override fun onDrawFrame(gl10: GL10?) {
var inputStream: InputStream?
var shouldCleanupSplash = false
Log.d("state::game", "onDrawFrame state=$state")
// Логирование изменения состояния
if (state != laststate) {
Log.d("state::game", "onDrawFrame state=$state")
Log.d(TAG, "onDrawFrame state=$state")
laststate = state
}
@@ -750,12 +906,24 @@ class GameActivityMain : AppCompatActivity(), DrawFrameListener {
}
STATE_GAME_START -> {
if (splashTimer < System.currentTimeMillis() && nativeRestoreContext()) {
isAssetsReady = true
nativeOnStart()
nativeOnResume()
gameRenderer.setDrawFrameListener(null)
shouldCleanupSplash = true
if (splashTimer < System.currentTimeMillis()) {
if (!restoreContextThreadStarted) {
restoreContextThreadStarted = true
d(TAG, "nativeRestoreContext: starting background thread from tid=" + Thread.currentThread().id)
Thread {
d(TAG, "nativeRestoreContext: background thread running, tid=" + Thread.currentThread().id)
restoreContextResult = nativeRestoreContext()
restoreContextDone = true
d(TAG, "nativeRestoreContext: background thread finished, result=$restoreContextResult")
}.apply { isDaemon = true }.start()
}
if (restoreContextDone && restoreContextResult) {
isAssetsReady = true
nativeOnStart()
nativeOnResume()
gameRenderer.setDrawFrameListener(null)
shouldCleanupSplash = true
}
}
}
}
app/src/main/java/com/ea/ironmonkey/GameDataInstaller.kt
+129
View File
@@ -0,0 +1,129 @@
package com.ea.ironmonkey
import android.content.Context
import android.util.Log
import java.io.File
/**
* Unpacks the game data bundled inside the APK to the place the game expects.
*
* The ~595 MB archive ships as `assets/game_data.obb`, stored uncompressed
* (see `noCompress` in build.gradle.kts - it is already a compressed archive,
* so deflating it again would only cost build and install time). On first
* launch it is copied to `getObbDir()/main.<versionCode>.<package>.obb`, which
* is exactly the path `GameActivityMain.obbFullPath` builds, so nothing else in
* the game needs to know this happened.
*
* The point is that a tester installs one APK and plays - no separate download,
* no file manager, no instructions about where to put an .obb.
*/
object GameDataInstaller {
private const val TAG = "GameDataInstaller"
private const val ASSET_NAME = "game_data.obb"
/** Progress callback: (bytes copied, total bytes). Called from a worker thread. */
fun interface Progress {
fun onProgress(copied: Long, total: Long)
}
sealed interface Result {
/** Already unpacked, or just unpacked successfully. */
object Ready : Result
/** Could not unpack - the message is safe to show a tester. */
data class Failed(val message: String) : Result
}
fun targetFile(context: Context): File {
val versionCode = runCatching {
context.packageManager.getPackageInfo(context.packageName, 0).versionCode
}.getOrDefault(0)
return File(context.obbDir, ObbHelper.getObbFileName(context, versionCode))
}
/** Size of the bundled asset, or -1 if it is not in this build. */
fun bundledSize(context: Context): Long = runCatching {
context.assets.openFd(ASSET_NAME).use { it.length }
}.getOrElse {
// openFd only works for an UNCOMPRESSED asset. If this ever starts
// failing, the noCompress rule has been lost and the asset is being
// deflated - worth knowing, because the copy below would still work
// but every install would be needlessly slower.
Log.w(TAG, "openFd($ASSET_NAME) failed - is noCompress still set? $it")
runCatching { context.assets.open(ASSET_NAME).use { s -> s.available().toLong() } }
.getOrDefault(-1L)
}
/** True when the data is already in place at its full expected size. */
fun isInstalled(context: Context): Boolean {
val expected = bundledSize(context)
if (expected <= 0) return false
val target = targetFile(context)
return target.isFile && target.length() == expected
}
/**
* Copies the bundled data into place. Blocking - call from a worker thread.
*
* Writes to a temporary file and renames only on success, so an interrupted
* copy can never leave a half-written archive that looks complete. A partial
* file that passed a mere "does it exist" check would send the game off to
* read truncated data, which fails far away from the real cause.
*/
fun install(context: Context, progress: Progress?): Result {
val expected = bundledSize(context)
if (expected <= 0) {
return Result.Failed("В этой сборке нет игровых данных (assets/$ASSET_NAME).")
}
val target = targetFile(context)
if (target.isFile && target.length() == expected) return Result.Ready
val dir = target.parentFile
if (dir != null && !dir.isDirectory && !dir.mkdirs()) {
return Result.Failed("Не удалось создать каталог ${dir.absolutePath}")
}
// Check free space before starting rather than failing 500 MB in.
val free = dir?.freeSpace ?: 0L
if (free in 1 until expected) {
return Result.Failed(
"Недостаточно места: нужно ${expected / 1_048_576} МБ, свободно ${free / 1_048_576} МБ."
)
}
val tmp = File(target.parentFile, target.name + ".part")
tmp.delete()
return try {
var copied = 0L
context.assets.open(ASSET_NAME).use { input ->
tmp.outputStream().buffered(1 shl 20).use { output ->
val buf = ByteArray(1 shl 20)
while (true) {
val n = input.read(buf)
if (n <= 0) break
output.write(buf, 0, n)
copied += n
progress?.onProgress(copied, expected)
}
output.flush()
}
}
if (copied != expected) {
tmp.delete()
return Result.Failed("Распаковка оборвалась: $copied из $expected байт.")
}
target.delete()
if (!tmp.renameTo(target)) {
tmp.delete()
return Result.Failed("Не удалось переименовать во ${target.absolutePath}")
}
Log.i(TAG, "unpacked $expected bytes to ${target.absolutePath}")
Result.Ready
} catch (e: Throwable) {
tmp.delete()
Log.w(TAG, "unpack failed", e)
Result.Failed("Ошибка распаковки: ${e.message}")
}
}
}
app/src/main/java/com/ea/ironmonkey/GameDataUnpackActivity.kt
+148
View File
@@ -0,0 +1,148 @@
package com.ea.ironmonkey
import android.content.Intent
import android.os.Bundle
import androidx.activity.ComponentActivity
import androidx.activity.compose.setContent
import androidx.activity.enableEdgeToEdge
import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.fillMaxSize
import androidx.compose.foundation.layout.fillMaxWidth
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.Button
import androidx.compose.material3.LinearProgressIndicator
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Scaffold
import androidx.compose.material3.Text
import androidx.compose.material3.darkColorScheme
import androidx.compose.runtime.Composable
import androidx.compose.runtime.getValue
import androidx.compose.runtime.mutableFloatStateOf
import androidx.compose.runtime.mutableStateOf
import androidx.compose.runtime.setValue
import androidx.compose.ui.Alignment
import androidx.compose.ui.Modifier
import androidx.compose.ui.text.style.TextAlign
import androidx.compose.ui.unit.dp
import kotlin.concurrent.thread
/**
* First-launch unpacking of the game data bundled in the APK.
*
* Shown only when [GameDataInstaller.isInstalled] is false, so it appears once
* after install and never again. It exists because copying ~595 MB takes long
* enough that a tester staring at a frozen launcher would reasonably assume the
* game had hung.
*/
class GameDataUnpackActivity : ComponentActivity() {
override fun onCreate(savedInstanceState: Bundle?) {
enableEdgeToEdge()
super.onCreate(savedInstanceState)
if (GameDataInstaller.isInstalled(this)) {
startGame()
return
}
var fraction by mutableFloatStateOf(0f)
var copiedMb by mutableStateOf(0L)
var totalMb by mutableStateOf(0L)
var error by mutableStateOf<String?>(null)
setContent {
MaterialTheme(colorScheme = darkColorScheme()) {
UnpackScreen(
fraction = fraction,
copiedMb = copiedMb,
totalMb = totalMb,
error = error,
onRetry = { recreate() },
)
}
}
thread(name = "game-data-unpack", isDaemon = true) {
val result = GameDataInstaller.install(this) { copied, total ->
// Updating Compose state is safe from any thread; the recomposition
// is scheduled onto the main thread by the snapshot system.
fraction = if (total > 0) copied.toFloat() / total else 0f
copiedMb = copied / 1_048_576
totalMb = total / 1_048_576
}
runOnUiThread {
when (result) {
is GameDataInstaller.Result.Ready -> startGame()
is GameDataInstaller.Result.Failed -> error = result.message
}
}
}
}
private fun startGame() {
startActivity(Intent(this, GameActivityMain::class.java))
finish()
}
}
@Composable
private fun UnpackScreen(
fraction: Float,
copiedMb: Long,
totalMb: Long,
error: String?,
onRetry: () -> Unit,
) {
Scaffold(modifier = Modifier.fillMaxSize()) { innerPadding ->
Column(
modifier = Modifier
.fillMaxSize()
.padding(innerPadding)
.padding(24.dp),
verticalArrangement = Arrangement.Center,
horizontalAlignment = Alignment.CenterHorizontally,
) {
if (error != null) {
Text(
text = "Не удалось подготовить игровые данные",
style = MaterialTheme.typography.headlineSmall,
textAlign = TextAlign.Center,
)
Text(
text = error,
style = MaterialTheme.typography.bodyMedium,
textAlign = TextAlign.Center,
modifier = Modifier.padding(top = 12.dp),
)
Button(onClick = onRetry, modifier = Modifier.padding(top = 24.dp)) {
Text("Повторить")
}
} else {
Text(
text = "Подготовка игровых данных",
style = MaterialTheme.typography.headlineSmall,
textAlign = TextAlign.Center,
)
Text(
text = "Выполняется один раз после установки.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onSurfaceVariant,
textAlign = TextAlign.Center,
modifier = Modifier.padding(top = 8.dp),
)
LinearProgressIndicator(
progress = { fraction },
modifier = Modifier
.fillMaxWidth()
.padding(top = 24.dp),
)
Text(
text = if (totalMb > 0) "$copiedMb из $totalMb МБ" else "…",
style = MaterialTheme.typography.bodySmall,
modifier = Modifier.padding(top = 12.dp),
)
}
}
}
}
app/src/main/java/com/ea/ironmonkey/GameGLSurfaceView.java
+39 -9
View File
@@ -11,6 +11,10 @@ import javax.microedition.khronos.egl.EGLDisplay;
public class GameGLSurfaceView extends GLSurfaceView {
private static final String TAG = "GameGLSurfaceView";
// See createContext/destroyContext below (task #62).
static int eglContextCreateCount = 0;
static int eglContextDestroyCount = 0;
private boolean enableHistoricalEvents;
private boolean kMotionEvent_GetSource;
private GameActivityMain mActivity;
@@ -35,15 +39,29 @@ public class GameGLSurfaceView extends GLSurfaceView {
setGLESVersion2();
setFocusable(true);
setFocusableInTouchMode(true);
if (Build.VERSION.SDK_INT >= 11) {
try {
Log.i(TAG, "setPreserveEGLContextOnPause");
getClass().getMethod("setPreserveEGLContextOnPause", Boolean.TYPE).invoke(this, false);
Log.e(TAG, "setPreserveEGLContextOnPause(false) success");
} catch (Exception unused2) {
Log.e(TAG, "setPreserveEGLContextOnPause failed");
}
}
// Task #62 - the black-textures-after-resume bug. The shipped game asked
// for the context NOT to be preserved (this call passed `false`) and took
// responsibility for rebuilding its GL objects afterwards. Under this
// engine that rebuild does not happen, which was confirmed live rather
// than assumed:
//
// EGL context CREATED (#1) launch
// EGL context DESTROYED (#1) on pause <- every texture gone here
// EGL context CREATED (#2) on resume
//
// and across that boundary the guest's own `Renderer::RestoreContext` ran
// to completion in ONE millisecond and re-uploaded a single texture, while
// 86 uploads had built the scene before it. Geometry survived only because
// vertex data is re-sent per frame; textures are uploaded once, so they
// came back black.
//
// Asking the platform to keep the context is the fix that does not depend
// on the guest restoring anything. It is a HINT, not a guarantee - the
// system may still drop the context under memory pressure - which is why
// the create/destroy logging added alongside it stays in permanently. A
// second CREATED line in a report means this fell back and the guest-side
// restore path is the next thing to fix.
setPreserveEGLContextOnPause(true);
}
public void setEnableHistoricalEvents(boolean z) {
@@ -56,11 +74,23 @@ public class GameGLSurfaceView extends GLSurfaceView {
@Override // android.opengl.GLSurfaceView.EGLContextFactory
public EGLContext createContext(EGL10 egl10, EGLDisplay eGLDisplay, EGLConfig eGLConfig) {
// Counted and logged (task #62). Whether the EGL context actually
// dies across a pause decides which bug we have: a SECOND create
// means every GL object the guest owns was destroyed and something
// must re-upload them, while a single create for the whole session
// means the black textures come from somewhere else entirely.
// Nothing logged this before, so the question was unanswerable.
eglContextCreateCount++;
android.util.Log.i("mpcore_log", "GameGLSurfaceView: EGL context CREATED (#"
+ eglContextCreateCount + ") - every GL object must be (re)uploaded after this");
return egl10.eglCreateContext(eGLDisplay, eGLConfig, EGL10.EGL_NO_CONTEXT, new int[]{EGL_CONTEXT_CLIENT_VERSION, 2, 12344});
}
@Override // android.opengl.GLSurfaceView.EGLContextFactory
public void destroyContext(EGL10 egl10, EGLDisplay eGLDisplay, EGLContext eGLContext) {
eglContextDestroyCount++;
android.util.Log.i("mpcore_log", "GameGLSurfaceView: EGL context DESTROYED (#"
+ eglContextDestroyCount + ") - all textures, buffers and shaders are now gone");
egl10.eglDestroyContext(eGLDisplay, eGLContext);
}
});
app/src/main/java/com/ea/ironmonkey/PermissionsActivity.kt
+20 -1
View File
@@ -175,7 +175,26 @@ class PermissionsActivity : AppCompatActivity() {
private fun initActivity() {
try {
startActivity(Intent(this, GameActivityMain::class.java))
// A crash report left by the previous run takes precedence over
// starting the game. The check lives HERE, not in GameActivityMain:
// this activity is the visible, foreground launcher entry, so the
// start is allowed. Doing it from GameActivityMain.onCreate - which
// starts the report and immediately finishes itself - was refused
// by the platform ("Activity start ONLY allowed by
// BAL_ALLOW_GRACE_PERIOD"), and the tester just landed back on the
// home screen with no report shown.
val next = when {
// A crash report from the previous run comes first - it is the
// thing the tester needs to see, and the next crash would
// overwrite it.
CrashReportActivity.pendingReport(this) != null -> CrashReportActivity::class.java
// Then the one-time unpack of the game data bundled in the APK.
// Checked on every launch but only does work once, so there is
// no cost after the first run.
!GameDataInstaller.isInstalled(this) -> GameDataUnpackActivity::class.java
else -> GameActivityMain::class.java
}
startActivity(Intent(this, next))
finish()
} catch (e: Exception) {
e.printStackTrace()
app/src/main/java/nfs/mod/traceagent/TraceAgentBridge.kt
+12
View File
@@ -0,0 +1,12 @@
package nfs.mod.traceagent
// Bridge to the standalone libtrace_agent.so (see trace_agent/ at the repo
// root and ARM64_TRANSLATION_LAYER.md's "Native ARM32 tracing harness"
// plan) - diagnostic-only, for the "native32" flavor running on the Galaxy
// A9. Not present in the APK's own jniLibs; loaded from wherever it was
// pushed on-device (see TraceApplication).
object TraceAgentBridge {
// outputPath: absolute path (app's own private files dir) the native
// side appends a full call trace to - see trace_agent/trace_log.h.
external fun install(outputPath: String)
}
app/src/main/java/nfs/mod/traceagent/TraceApplication.kt
+45
View File
@@ -0,0 +1,45 @@
package nfs.mod.traceagent
import android.app.Application
import android.content.Context
import android.util.Log
// Earliest available hook point for installing the JNI trace table patch
// (see TraceAgentBridge/trace_agent's own comments) - as early as possible,
// before any real game library gets a chance to grab its own JNIEnv
// reference. Entirely inert on the normal ("translated") flavor: gated
// behind BuildConfig.NATIVE32, and the .so path is checked for existence
// before attempting System.load, so a normal run that never pushed
// libtrace_agent.so to the device just logs and continues - no crash, no
// behavior change, matching the plan's "does not touch the emulated path"
// constraint.
class TraceApplication : Application() {
override fun attachBaseContext(base: Context) {
super.attachBaseContext(base)
if (!com.ea.games.nfs13_arm.BuildConfig.NATIVE32) return
// Must live under the app's own private data dir (app_data_file
// SELinux context), not /data/local/tmp (shell_data_file) - confirmed
// live on the Galaxy A9 that Enforcing SELinux denies mmap-exec of a
// shell_data_file-labeled .so from an app-domain process ("couldn't
// map ... segment 1: Permission denied"). The same path is what
// wrap.<packageName>'s LD_PRELOAD value must point at too (see the
// plan's Phase 4 deployment notes).
val soPath = base.filesDir.absolutePath + "/libtrace_agent.so"
if (!java.io.File(soPath).exists()) {
Log.d(TAG, "libtrace_agent.so not found at $soPath - JNI tracing not installed")
return
}
try {
System.load(soPath)
val outputPath = base.filesDir.absolutePath + "/trace_output.log"
TraceAgentBridge.install(outputPath)
Log.d(TAG, "trace agent JNI hooks installed, writing to $outputPath")
} catch (e: Throwable) {
Log.d(TAG, "failed to install trace agent JNI hooks: $e")
}
}
companion object {
private const val TAG = "TraceApplication"
}
}
app/src/main/jniLibs/armeabi-v7a/wrap.sh
Executable
+16
View File
@@ -0,0 +1,16 @@
#!/system/bin/sh
# APK-bundled wrap.sh (2026-09-16, ARM64_TRANSLATION_LAYER.md - trace_agent
# LD_PRELOAD deployment investigation). The loose-file mechanism
# (/data/local/tmp/wrap.<packageName>) was confirmed NOT invoked by zygote
# on this specific Galaxy A9/Samsung build - a canary marker file placed by
# that script never got created across multiple relaunches, root and
# non-root, SELinux Enforcing and an attempted Permissive switch. This is
# the OTHER officially documented NDK mechanism (developer.android.com/ndk/
# guides/wrap-script): a wrap.sh bundled alongside the app's own native
# libraries, picked up automatically for a debuggable app. Points LD_PRELOAD
# at the app's own private files dir (not this APK-bundled location) so the
# actual libtrace_agent.so can still be updated by just `run-as`-copying a
# freshly-built one in, without needing to reinstall the whole APK each
# iteration.
export LD_PRELOAD=/data/data/com.ea.games.nfs13_arm/files/libtrace_agent.so
exec "$@"
app/src/main/res/xml/crash_report_paths.xml
+6
View File
@@ -0,0 +1,6 @@
<?xml version="1.0" encoding="utf-8"?>
<!-- Shares ONLY the crash directory, nothing else. FileProvider is required
because targetSdk 35 rejects file:// URIs in an ACTION_SEND intent. -->
<paths>
<external-files-path name="crashes" path="crashes/" />
</paths>
app/src/test/java/com/ea/games/nfs13_arm/ExampleUnitTest.kt
+1 -1
View File
@@ -1,4 +1,4 @@
package com.ea.games.nfs13_mod
package com.ea.games.nfs13_arm
import org.junit.Test
docs/ANALYSIS.md
+971
View File
@@ -0,0 +1,971 @@
# ANALYSIS.md — libapp.so Reverse Engineering Notes
Target binary: `native_lib/libapp.so` (NFS Most Wanted 2012, Android v1.3.128, Firemonkeys/Iron Monkey engine, ARM EABI5 32-bit, Thumb/ARM mixed).
IDA database: `native_lib/libapp.so.i64` (opened via `ida-pro-mcp` idalib MCP server, session handle re-created per session — see "IDA session" below).
This document is a living record. Update it as findings are made; do not wait until the end of a session.
---
## 1. Provenance of prior findings (from `chat-export-1785094024785.json`)
A previous chat session (Open WebUI, model "deepseek-v4-flash uncensored", 2026-05 timeframe) analyzed a `libapp.so` **downloaded from a third-party file-sharing link** (`private-ai.tools/files/*.so`) inside a sandboxed Linux container, using `radare2` + hand-written Python byte-scanning — **not** IDA, and **not** this project's `native_lib/libapp.so`. The user re-uploaded the file at least 3 times under different hashes during that conversation, so it is unclear whether it was ever the same build as ours.
**Verification performed this session:** opening our actual `native_lib/libapp.so.i64` and searching for the same RTTI/mangled class names confirms this **is the same engine and the same general class taxonomy** (see §3), but at **completely different addresses** — the old chat's addresses top out around `0x00350000` while our `.text` segment alone spans `0xa1aa8–0xc28c44` and the image is `0xe52ebc` bytes (~14.9 MB). The old chat's specific offsets, vtable layouts, and struct field guesses **do not apply directly** and must be re-derived. They are recorded below only as **architectural hypotheses**, not verified facts.
The old chat's generated "C++ bindings header" was also never actually completed — the final message was a Python script emitting a header template whose format-string placeholders (`0x%08X`) were never substituted with real values (the export cuts off mid-generation). So no usable header exists from that session; it must be rebuilt from scratch against our binary.
### 1.1 What the prior session got right (reusable knowledge)
- **Game/engine identification**: EA "Iron Monkey" engine (`im::app::*`, `im::components::*` C++ namespaces), Need for Speed Most Wanted 2012 Mobile. JNI entry points under `com.ea.ironmonkey.*`.
- **SB/SBA resource format** (fully reconstructed from `NFSMW12MobileTools` Java source, not from the binary — this part is source-verified, not a guess):
- Header: `"SBIN"` (4 bytes) + version (1 byte, `0x03` for NFS MW12).
- Chunk stream, each chunk: `sig(4) + data_size(4, LE) + fnv1_32_hash(4, LE) + data[size] + pad-to-4`.
- Chunk types: `ENUM`, `STRU`, `FIEL`, `OHDR`, `DATA`, `CHDR`, `CDAT`, `BULK` (texture mip offsets), `BARG` (raw texture bytes).
- Field types (`SBinFieldType`): `INT8/16/32/64`, `FLOAT32`, `BOOLEAN`, `ENUM_ID_INT16/32`, `CDAT_STRING`, `SUB_STRUCT`, `GAME_UNIT`, plus opaque `HEX_*` types.
- Ordinary `.sb`: `SBIN→ENUM→STRU→FIEL→OHDR→DATA→CHDR→CDAT`. Save files swap `ENUM` after `STRU/FIEL`. Texture `.sba` appends `BULK→BARG`.
- **Save files are NOT parsed into an object model** by NFSMW12MobileTools — only HEX + string edits are supported for saves. Confirmed present in our `game_cache/` (see §5).
- This matches the directory layout we actually have: `game_cache/published/{prefabs,data,flow,stringdata,textures,models,sounds}/...` (see §5).
- **⚠️ Tool reliability caveat**: `NFSMW12MobileTools.jar unpack` does **not** reliably unpack every `.sb`/`.sba` file — it has real, reproducible parser bugs (confirmed example: `region3_colorado_track2.scene.sb` throws `NegativeArraySizeException` in `SBin.getCleanElementHex`/`parseDATABlock` on every attempt, while sibling files like `region3_colorado_track1.scene.sb` from the same directory unpack cleanly with identical invocation — see §6m). This is a pre-existing bug in the third-party tool, not something introduced by this project, and not worth "fixing" upstream for our purposes. **Don't treat a crash as "this file can't be analyzed" — try the workarounds first**: `-disableDATAObjectsUnpack` (skips per-object field parsing but still emits the top-level `CDAT_Strings` table, i.e. every unique string referenced anywhere in the file — often enough on its own to answer "does entity X exist / what's it named", exactly as used in §6m) or `-disableMipmapUnpack` (for `.sba` texture-pack issues). If a specific file still won't unpack under any flag, a same-directory sibling file is usually a decent structural proxy (same region/asset-authoring convention) when an exact answer isn't required. Command reference: `java -jar NFSMW12MobileTools/NFSMW12MobileTools.jar unpack <file.sb> [-disableMipmapUnpack] [-disableDATAObjectsUnpack]` — run from a directory containing `HCStructFileArray.json` (copy from the tool's own repo root) alongside the target file; output is `<file>.json` next to the input.
- **High-level race-loading architecture** (conceptually confirmed, see §3 for real anchors in our binary):
- Menu → Flow-machine (SB-scripted reactive state machine, `/published/flow/*.sb`) → `FlowAction "DoLoadRace"` → `RaceLoaderTask` (boost::shared_ptr-managed, polymorphic) → parses `RaceDefinition` from `/published/data/races/<id>.sb` → builds `OpponentCollection` (array of `Opponent` descriptors) → `RaceStartingGrid`/`StreetRaceStartingGrid` places cars using `TrackNavigator` (track-spline coordinate system: `distance_along_spline`, `lateral_offset`, `height_offset` ↔ world `Vector3`) → loads car/track prefabs → `InRaceState` (namespace `im::app::race::states`) runs the race.
- **Important safety conclusion (still believed valid conceptually)**: `RaceLoaderTask` cannot be safely constructed and invoked from an arbitrary moment — it's a `boost::shared_ptr`-owned polymorphic task queued through the Flow-machine, with refcount fields checked internally; calling it out of context risks a crash. The safe integration pattern is **hook, don't originate**: let the game's own Flow-machine create/drive `RaceLoaderTask` normally (player picks any existing event from the menu), and intercept it at well-defined points (opponent-list build, starting-grid placement, track-navigator position resolution) to substitute custom data — rather than trying to synthesize the whole load sequence from scratch.
- This maps directly onto our mod's needs: subtask 2 (arbitrary map + N players) and subtask 4 (coordinate interception) are naturally the same hook points.
- **Text rendering claim in the task brief needs correction** (see §4) — the previous chat never investigated this; it's a fresh finding this session.
### 1.2 What must be re-derived from scratch (do not trust old numbers)
- All function addresses (`RaceLoaderTask::*`, `OpponentCollection::Build`, `TrackNavigator::*`, `Health::*`, `NitroBehaviour::*`, etc.)
- All vtable layouts and indices
- All struct field offsets (the `Opponent` "0x50 bytes" struct, `TrackNavigator` fields, etc. — these were never disassembled against our binary, only guessed by pattern-matching a different file)
- The claimed GOT/PLT/vtable addresses in the old chat's "priority hook map" table
---
## 2. IDA database state (`libapp.so.i64`)
- Opened via idalib MCP (`idb_open`), auto-analysis + Hex-Rays already available (`hexrays_ready: true`).
- **50,772 total functions**, only **2,789 named** (~5.5%) — the rest are `sub_XXXXXX`. No demangled C++ names have been applied to *functions* yet (searching `list_funcs` for `*RaceLoaderTask*`, `*Opponent*`, `*TrackNavigator*`, `*Health*`, `*Nitro*` returns zero function matches), even though the **mangled RTTI name strings** for these classes exist in `.rodata`/data segments and are found via string/regex search. In other words: the `.i64`'s size (192 MB) comes from IDA's analysis caches (Hex-Rays microcode, xrefs, etc.), not from prior manual RE work — there is no pre-existing "someone already named all the classes" state to build on. We are starting from a clean, auto-analyzed base.
- Segments: `.text` (`0xa1aa8`–`0xc28c44`, ~11.7 MB), `.ARM.extab`/`.ARM.exidx` (unwind tables), `.rodata` (`0xc81010`–`0xd6fbc0`), `.data.rel.ro.local` (`0xd71558`–`0xd77ce4`), `.init_array`/`.fini_array`. Image base is `0x0` in the IDB (i.e., all addresses here are file-relative / load-relative offsets, matching what `mpcore`'s `APP_ADDR()` macro expects — see §6).
- 10 JNI entry points auto-detected as "entrypoints", all under `com.ea.ironmonkey.*` (`GameActivity`, `GameGLSurfaceView`, `MogaController` (game controller support), `RunLoop`, and `JNI_OnLoad` itself at `0x56a52c`).
- Imports include OpenGL ES 2.0 (`glCreateShader` etc.), FMOD (via `_ZN4FMOD12ChannelGroup...`), POSIX sockets (`socket/recv/send/connect` — used by the EA "Synergy" backend HTTP/HTTPS client, see §3.4, not by any game-level multiplayer protocol), pthreads, and EA Nimble bridge registration functions.
---
## 3. Confirmed class/subsystem inventory (RTTI-verified, this session)
Method: `find_regex` over decoded strings to locate Itanium-mangled RTTI type names (`N2im3app...E`) and `boost::shared_ptr` counted-impl wrappers (`N5boost6detail17sp_counted_impl_pI...E`), which reveal exactly which C++ classes exist even though functions aren't named. Addresses below are the **string** locations (evidence the class exists), not yet the vtables — vtable/function recovery is future work (see §7 plan).
### 3.1 Race / track / opponent subsystem (`im::app::race::*`, `im::app::track::*`)
| Class | Namespace | String address |
|---|---|---|
| `RaceLoaderTask` | `im::app::race` | `0xcfe924` (sp_counted_impl wrapper at `0xcf1420`) |
| `OpponentCollection` | `im::app::race::description` | `0xcfeaf8` |
| `Opponent` | `im::app::race::description` | referenced inside `boost::bind` signature at `0xcfe884` (`bind_t<..., mf1<..., RaceLoaderTask, shared_ptr<Opponent>>...>`) |
| `RaceStartingGrid` | `im::app::race::description` | `0xcfeb5c` |
| `StartingGrid` (base) | `im::app::race::description` | `0xcfeb8c` |
| `StreetRaceStartingGrid` | `im::app::race::description` | `0xcfebb8` |
| `InRaceState` | `im::app::race::states` | `0xcff010` (with `boost::bind` callbacks referencing `Checkpoint`, `Driver` component weak-ptrs) |
| `TrackNavigator` | `im::app::track` | `0xd00454` |
| `TrackNavigatorSubSystem` | `im::app::track` | `0xd004c0` (implements `ISceneComponentListener<TrackNavigator>`) |
### 3.2 Car subsystem (`im::app::car::*`)
| Class | String address | Notes |
|---|---|---|
| `NitroBehaviour` | `0xcf2f58` | |
| `AICarController` | `0xcf1c7c` | base AI controller |
| `CopAICarController` | `0xcf29f0` | police AI |
| `PlayerAICarController` | `0xcf2f98` | |
| `RaceAICarController` | `0xcf35d4` | |
| `PlayerCarController` | `0xcf31a0` | |
| `CarLoaderTask` | `0xcf2458` | |
Not yet re-located this session (present in old-chat hypothesis, still need string/RTTI confirmation in our binary): `Health`, `DamageDealtMultiplier`, `CarDamage`, `Nitro` (base), `SpikeStrip` (`im::app::bt::SpikeStrip`). These were found via `strings`/`r2` in the *other* binary; treat as "likely present, not yet confirmed here" until searched.
### 3.3 Text / UI rendering — corrected (this section was wrong in an earlier revision, see below)
**Earlier revision of this section incorrectly concluded text is rendered purely natively via GLES/EAMText with no Java bridge.** That was wrong — it only checked for `Java_com_ea_ironmonkey_*` *exported* JNI functions (native called *from* Java) and missed the reverse direction: native code calling *into* Java via cached `FindClass`/`GetMethodID`/`CallVoidMethod` upcalls, which don't show up as exported symbols at all. The user pointed to the actual mechanism, already reverse-engineered and sitting in the launcher project as `launcher/app/src/main/java/com/ea/ironmonkey/BitmapGraphics.kt` (comment in that file: *"Весь текст в игре отрисовывается от сюда"* — "all game text is drawn from here"). Verified this session against `libapp.so.i64` by decompiling the actual call sites (functions renamed in the IDB accordingly):
- **`BitmapGraphics_ctor_jni`** @ `0x5640ec` — `FindClass("com/ea/ironmonkey/BitmapGraphics")` + `GetMethodID("<init>", "(II)V")` + `NewObject(width, height)`. Matches `BitmapGraphics(width: Int, height: Int)`, which internally creates an `android.graphics.Bitmap` (`ARGB_8888`) and wraps it in a `Canvas`.
- **`BitmapGraphics_drawString_jni`** @ `0x5625a0` (thunk at `0x56274c`) — lazily resolves and calls `GetMethodID("drawString", "(Landroid/graphics/Paint;Ljava/lang/String;II)V")`. Matches `fun drawString(paint: Paint, text: String, x: Int, y: Int) = canvas.drawText(...)`.
- **`BitmapGraphics_createPaintFromFamilyName_jni`** @ `0x562a58` / **`BitmapGraphics_createPaintFromFile_jni`** @ `0x562b94` — call the companion-object `createPaintFromFamilyName(String, F): Paint` / `createPaintFromFile(path, F): Paint` statics.
- **`ResolveFontPaint_ttfOtfOrFamily`** @ `0x51145c` — checks for a `.ttf`/`.otf` file at the requested font path first, falls back to `createPaintFromFamilyName` (system font family) otherwise.
- **`BitmapGraphics_blitBitmapToAtlasTexture`** @ `0x563d30` — the missing link, found by tracing xrefs to the `"getBitmap"` string: calls `BitmapGraphics.getBitmap()`, then `AndroidBitmap_getInfo`/`AndroidBitmap_lockPixels`, then `memcpy`s the locked pixel buffer **row-by-row with a vertical flip** into what is, by the region/offset arithmetic (`v27,v28,v29` = atlas rect x/y/width, bounds-clipped against a requested sub-rect), a **native GL texture atlas** (consistent with the `EA::Graphics::OGLES20::Texture` class seen elsewhere in the binary). Flip direction makes sense: Android `Bitmap` rows are top-down, GL texture data is bottom-up.
**Corrected conclusion:** text rendering pipeline is **Kotlin `BitmapGraphics` (owns an offscreen `Bitmap`+`Canvas`) → `Canvas.drawText` via `Paint`/`Typeface` (handles proper font shaping for all 11 locales in `game_cache/published/stringdata/`, including CJK, which a bespoke native shaper would struggle with) → JNI readback via `AndroidBitmap_lockPixels` → row-flipped `memcpy` into a shared native GL font-atlas texture → drawn every frame by the game's normal GLES renderer as textured quads.** The `EAMText`/`GlyphMeshGLES`/FreeType strings noted in the previous revision are real and still present in the binary, but their exact role relative to this `BitmapGraphics` path (alternate/legacy renderer? metrics-only? a different UI layer?) is not yet determined — do not assume they're the primary path; the `BitmapGraphics` bridge above is the one with a live, traceable call chain from a known Kotlin source file.
**Why this matters for the mod**: since `BitmapGraphics.kt` is fully ours to extend (it's reverse-engineered launcher source, not opaque binary), we can add a new method — e.g. `drawMarker(x: Int, y: Int, color: Int)` using `canvas.drawCircle(...)` — and it will be blitted into the game's own font-atlas texture and rendered through the game's existing GLES pipeline exactly like any other UI text. This gives a low-risk, verified path for drawing simple lobby indicators (e.g. green/red event markers on the map) **without** needing to author new Flow/SB screens and without needing a separate Android overlay `View`. It does **not** by itself solve touch input or full interactive lobby screens (player list, ready buttons) — that's a separate concern, still likely served best by an overlay `View` for now (see `ARCHITECTURE.md`).
### 3.4 Networking / backend — no reusable multiplayer transport
Searched for `gamespy|eaonline|matchmak|lobby|leaderboard|raknet|socket|multiplayer|EA::Online|Synergy|CDMA|Freeway`. Findings:
- Extensive **EA "Synergy"** backend integration: `SERVER_SYNERGY_{DIRECTOR,MTU,PRODUCT,DRM,USER,TRACKING,CIPGL,S2S}`, hardcoded synergy endpoint URLs (`synergy-dev/int/stage.eamobile.com`), pinned TLS certs (`synergy-GeoTrustGlobalCA.crt` etc.), and world/friends leaderboard events (`SPEvent_GetWorldLeaderboardDone/Error`, `SPEvent_GetFriendsLeaderboardDone/Error`, `MostWantedLeaderboard`). This is EA's account/IAP/DRM/leaderboard HTTP(S) backend (likely long dead), **not** a peer race-sync protocol.
- Raw POSIX socket imports (`socket/recv/recvfrom/connect/send/sendto`) exist but are consistent with the above HTTP(S) client, not a custom UDP game protocol.
- **No RakNet, no GameSpy, no P2P/lobby-discovery code found.** Confirms the task brief's premise: there is nothing to reuse for multiplayer transport — RakNet must be integrated fresh, both client (embedded in the mod) and dedicated server (standalone), as already planned.
- `libnimble.so` (separately investigated, see §6) is EA's **Nimble SDK** — a generic native↔Java bridge/callback framework (`EA::Nimble::JavaClass`, `BridgeCallback`, `MTXNativeCallbackBridge` for in-app-purchase UI, `PushNotification`) — unrelated to multiplayer, but relevant as a *pattern* for how native↔Java callback bridging is done in this codebase if we need more JNI bridge surface later.
---
## 4. `libgnustl_shared.so`, `libfmodex.so`, `libfmodevent.so`, `libnimble.so` — quick disposition
- `libfmodex.so` / `libfmodevent.so`: FMOD audio engine runtime — not relevant to the multiplayer mod except that hooked code must not disturb their init order.
- `libgnustl_shared.so`: GNU libstdc++ runtime — no action needed.
- `libnimble.so`: EA Nimble SDK (see §3.4) — native↔Java bridge/callback plumbing for IAP (`MTX`), push notifications, identity. Not part of the race/multiplayer surface. Its `Java_com_ea_nimble_bridge_BaseNativeCallback_native{Callback,Finalize}` pattern is a useful reference for how this codebase wires native callbacks into Java if the lobby UI ends up needing a similar bridge.
---
## 5. `game_cache/` — confirms SB/SBA architecture in practice
Directory layout under `game_cache/published/`:
`prefabs/{cars,tracks,racefsms,racetypes,checkpoints,roadblocks,traffic,environments,garage,skydomes,props,lensflares,blacklisttech}`, `data/{races,cars,careers,achievements,sponsors,enginesounds,autolog,pursuit,tiers}`, `flow/{menus,race,postgame,frank,...}.sb`, `stringdata/<LANG>/`, `textures/...`, `models/...`, `sounds/...`, `fonts/`, `layouts/`, `tweaks/`, `particles/`, `replays/`. Plus `published.1x/2x/4x` texture-pack variants (resolution tiers).
This confirms: `prefabs/tracks` = track geometry, `data/races` = `RaceDefinition` SB files, `prefabs/racefsms` = likely the actual `InRaceState`/Flow FSM scripts driving a race (worth inspecting directly — plain SB, no binary RE needed), `flow/race/*.sb` = the Flow-machine scripts for the race-start UI sequence the user described (location → event list → info screen → car select → race). **These SB files are directly readable/editable with `NFSMW12MobileTools`** without touching the native binary at all for a large fraction of subtask 2/3 work (e.g., authoring a custom `RaceDefinition` SB with our own opponent list, if we go the "replace the file the game reads" route instead of the "hook the C++ struct after parse" route).
---
## 6. `launcher/mpcore/` — existing draft state, and a verified ground-truth anchor
Confirmed via direct file read: `mpcore` is exactly what the task brief said — an early draft, not production code. Current contents:
- `src/main/cpp/main.cpp`: `JNI_OnLoad` resolves `libapp.so` base via `FindLibrary()`, logs it, then `raise(SIGSTOP)` (presumably to attach a debugger before the process continues) and returns. A `pending_thread()` function exists but is unused (never spawned) — it sleeps 10s then reads/logs a value at `APP_ADDR(0x00E4B8EC)` and calls `unProtect(APP_ADDR(0x00E4A738))`. All hooking is currently commented out.
- `util/armhook.cpp`: a working **Thumb-mode (16-bit ISA) inline-hook toolkit** — `unProtect` (mprotect with EACCES fallback), `makeNOP`/`makeJMP`/`makeBLX` (Thumb branch encoding), `WriteHookProc`/`InstallHook`/`InstallJMPHook`/`InstallBLXHook`/`InstallMethodHook` (vtable-slot overwrite)/`CodeInject` (register-move injection into a small stub), all built around a fixed trampoline region `memlib_start..memlib_end = APP_ADDR(0x000A1B44)..+0x1A36` plus a separate `mmap`'d PROT_RWX scratch page for original-bytes backup. **This confirms hooking must target Thumb encoding, not ARM32** for at least this region of the binary (consistent with the 16-bit `MOVS`/`STR`/`BX LR` opcodes the old chat's radare2 dump was implicitly assuming when scanning for "0x2050" MOVS patterns — that detail was accidentally right even though the addresses were wrong).
- **Verified ground truth**: a commented-out line in `main.cpp` reads `WriteMemory(APP_ADDR(0xC8C9D8), (uintptr_t)"APPLICATION_NO", 14)` after logging the existing string at that address. Checked against our `.i64` via `get_string`: **`0xC8C9D8` does contain the string `"APPLICATION_OK"`** — confirming the mod author was working against *this exact binary* (not a different build), and that `APP_ADDR()` is a direct file-offset-to-runtime-address identity mapping (base address 0 in the IDB matches). **This is our first fully-verified address in this session** and a template for how to cross-check future finds: locate a string/constant in IDA, then confirm the same offset in the live/dumped binary.
- `NativeLib.kt` (Kotlin/Java side): stub class, `System.loadLibrary("mpcore")` commented out — not yet wired into the app's load sequence.
- Build system: Gradle module already produces `libmpcore.so` for `armeabi-v7a` (matches our target ABI) via CMake (`src/main/cpp/CMakeLists.txt`), for both Debug and RelWithDebInfo.
**Implication for integration plan**: the `armhook.cpp` toolkit is a solid, already-working foundation for ARM32 Thumb inline hooking — we should build on it rather than writing a new hooking library from scratch, once we've confirmed it handles our specific hook sites (it was seemingly exercised only against the `0xC8C9D8` string-patch and a couple of NOP/log experiments so far, not against a real virtual-function hook).
---
## 6a. `RaceLoaderTask` real vtable — located and partially mapped (subtask 1 groundwork)
Method used (repeatable for the other classes in §3.1/3.2): from the class's RTTI name string, find who points to it (`xrefs_to`) — that location is `type_info->name`, so `type_info` itself sits 4 bytes earlier. Then `find_bytes` for a little-endian pointer to *that* `type_info` address across the binary; the real class vtable starts 4 bytes after the match (the found slot is the `typeinfo_ptr` field of the vtable group, per Itanium ABI: `[offset_to_top][typeinfo_ptr][vfunc0][vfunc1]...`). Cross-check: the resulting array should be a long unbroken run of `.text` addresses immediately followed by the class's own `type_info` fields — exactly what was found, confirming the technique. (Two other `find_bytes` hits for the same typeinfo address were false leads: one was the Iron Monkey engine's own custom reflection/pointer-tagging table — same `{ptr, 0x17}` pattern seen earlier for `BitmapGraphics`, unrelated to C++ vtables — the other was a different class's `base_type_info` field, i.e. some other class derives from `RaceLoaderTask`, not yet identified.)
**`RaceLoaderTask` vtable**: `0xd86210` (19 virtual function slots), immediately followed by its own `type_info` at `0xd8625c` (`{vtable=0xdc12a0, name=0xcfe924 "N2im3app4race14RaceLoaderTaskE", base_typeinfo=0xd79b78}`).
| Slot | Address | Renamed to | Role / evidence |
|---|---|---|---|
| 0 | `0xd5b04` | — | Shared thunk (xrefs from multiple unrelated typeinfo structs `0xd79b60`, `0xd79b90`, `0xd7a300`, plus ours) — generic `LoaderTask` base-class helper, not `RaceLoaderTask`-specific. Not decompiled in depth. |
| 1 | `0x2d8e04` | `RaceLoaderTask_dtor_complete` | Sets vptr back to `0xd86210` (own vtable), decrefs several shared-ptr-like members at `this+100/112/124/136/148/156`, destroys an internal vector of ref-counted elements (12-byte stride). |
| 2 | `0x2d8f18` | `RaceLoaderTask_dtor_deleting` | Calls slot 1, then `operator delete`. Standard Itanium destructor pair. |
| 3 | `0xd5818` | — | Same shared-thunk region as slot 0. Not decompiled. |
| 4 | `0x2dbba4` | `RaceLoaderTask_ExecuteLoadSequence` | **Main load orchestrator.** Calls `SetLoadProgress(this, N)` (via `sub_42EE9C`) with progress fractions `0.1, 0.2, 0.3, 0.4, 0.45, 0.5, 0.6, 0.7, 0.8` interleaved with stage calls (`sub_2DB384`, `sub_2DA710`, `sub_2DAA7C`, `sub_2D994C`, `sub_2DAB50`, `sub_2D9AD0`, `sub_2DB534`, `sub_2D969C` — candidate individual load stages, addresses known but not yet decompiled/named). Loads `/published/texturepacks_ui/in_game.sba` (loading-screen texture) partway through. **This is the prime hook candidate for subtasks 1/2** — top-level entry to intercept before track/opponent data reaches the stage functions. |
| 5 | `0xd5820` | — | Same shared-thunk region as slot 0/3. Not decompiled. |
| 6 | `0x2d83cc` | — | `nullsub` (unused/pure-virtual slot). |
| 7 | `0x2d9ba4` | — | Takes `(this, a2)` where `a2` looks like a 3-word ref-counted handle; assigns it into `this+64/68/72`, decrefs the old value, calls `sub_D7C00`/`sub_2DDED4`. Likely a "Set&lt;SharedResource&gt;" setter (track ref? scene ref?) — role unconfirmed, but the field it writes (`this+64..72`) is read by slot 16. |
| 8 | `0x2db160` | `RaceLoaderTask_ResolveDriverPositionComponent` | Calls `dynamic_cast<DriverPosition*>` against RTTI `im::components::Component` → `im::app::race::data::DriverPosition`. **Direct link to the coordinate/position system** — relevant to subtask 4 as well as 1/2. |
| 9 | `0x2d9d10` | `RaceLoaderTask_SetupPlayerCar` | References string `"playerCar"`. Large function (~1.1KB), gated by a state-check helper (`sub_360CC0`, shared with slot 11). |
| 10 | `0x2d9088` | — | Same `sub_360CC0` state-gate pattern as slots 9/11, but no distinctive string found — purpose unconfirmed (candidate: a third car-category setup, e.g. traffic). |
| 11 | `0x2dae0c` | `RaceLoaderTask_SetupOpponentCar` | References string `"opponentCar"`. Same state-gate pattern as slot 9 — clear sibling function (player vs. opponent car setup). **Prime hook candidate for subtask 2.** |
| 12 | `0x2d8884` | — | Trivial one-line wrapper calling `sub_2DC82C(a1, a3)` — thin delegation, not investigated further. |
| 13 | `0x2d8a74` | — | Allocates small objects and links them via another vtable (`off_D86110`, not yet investigated) — looks like constructing an auxiliary completion-callback object. Not fully understood. |
| 14 | `0x2da2a0` | `RaceLoaderTask_ResetStartingLine` | References string `"ResetLine"`. Large function (~1.1KB). **Prime hook candidate for subtask 2's starting-grid placement.** |
| 15 | `0x2d83d0` | — | `nullsub`. |
| 16 | `0x2d8f34` | — | Reads the field slot 7 writes (`this+64`), iterates a vector at `this+76/80` (same vector shape as the destructor's), calling `sub_18B670` per element — looks like a listener-notification loop. Role unconfirmed but clearly paired with slot 7. |
| 17 | `0x2da880` | `RaceLoaderTask_HandleSpikeStrip` | Calls `dynamic_cast` from `im::app::bt::BlacklistTech` to `im::app::bt::SpikeStrip`, references string `"Spike Strip"`. Confirms spike strips are handled as a kind of trackside `BlacklistTech` prop during race load. |
| 18 | `0x2d8c1c` | — | Not yet decompiled. |
## 6b. `ExecuteLoadSequence`'s 8 stage functions — decoded (subtask 1 hook point found)
Decompiled all 8 stages called from `RaceLoaderTask_ExecuteLoadSequence` (`0x2dbba4`), in call order:
| Order | Address | Renamed to | Role / evidence |
|---|---|---|---|
| 1 | `0x2db384` | **`RaceLoaderTask_BuildTrackScenePath`** | **★ Subtask-1 hook point.** Builds `"published/prefabs/tracks/<name>.scene.sb"` from a string field at `this+8`-struct `+72` (the track name, presumably parsed from the `RaceDefinition` SB), optionally builds `"published/prefabs/environments/<env>.prefabs.sb"` too (only if two fields differ — an environment-override check), then calls `GenericLoadScene_trackEnvWrapper` with both paths. **To load an arbitrary track: patch the track-name string this function reads, or hook the function and rewrite the constructed path before it reaches the loader.** |
| — | `0x2dbf58` | `GenericLoadScene_trackEnvWrapper` | Thin wrapper forwarding to `sub_33C284(out, trackPath, envPath, -1, 0)` — likely the engine's shared generic scene/asset loader (not race-specific), not yet investigated further. |
| 2 | `0x2da710` | `RaceLoaderTask_SetupAIDifficultyProfiles` | Builds three literal strings `"ai_easy"`, `"ai_hard"`, `"default"` and passes them to `sub_121288` along with a value read from `this+44`-struct`+152`. AI behavior-tuning profile lookup/registration. |
| 3 | `0x2daa7c` | `RaceLoaderTask_TagPlayerEntity` | Calls two virtual functions on `this` (vtable+36, vtable+28) and writes the literal string `"Player"` into an offset (`+52`) of the object returned by the first virtual call — tags/names the player's entity in the scene graph. |
| 4 | `0x2d994c` | `RaceLoaderTask_RegisterTrafficFlow` | Scans a component vector via `dynamic_cast` against RTTI `im::app::traffic::TrafficFlow`, then registers whichever component matches. Wires up traffic/AI-pathing infrastructure from the now-loaded track scene. |
| 5 | `0x2dab50` | *(not renamed — uncertain)* | Copies three fields (words 38–40, i.e. `+152/156/160`) from the `this+8`-struct into a large object at `this+44`-struct`+304` (offsets `+3996..4004`), plus a byte at `+120→+4036`. Looks like copying `RaceDefinition` metadata (camera/weather/track-meta?) onto the loaded scene/race object — exact semantics unclear, left uncommented-name. |
| 6 | `0x2d9ad0` | *(not renamed — uncertain)* | Allocates a small object, passes `this+16` (the field group `RaceLoaderTask` slot 7 sets / slot 16 reads — see §6a) into `sub_2B6900`, wraps the result in a ref-counted adapter (different vtable, `off_D86150`, not `RaceLoaderTask`'s own), registers it into the scene at `this+44`-struct`+152`. Looks like constructing an anonymous listener/callback object tied to whatever slot 7/16 manage — not confidently named. |
| 7 | `0x2db534` | **`RaceLoaderTask_LoadRaceFSM`** | Loads `"/published/prefabs/racefsms/{0}.prefabs.sb"` (template-substituted, confirms the `game_cache/published/prefabs/racefsms/` directory noted in §5 is exactly the race rules/state-machine prefab). Searches the resulting scene for a component matching RTTI `im::app::race::Race` via `dynamic_cast` — **this is the runtime race-FSM/rules controller object**, stored at `this+52`. Then iterates the opponent/car vector (same vector shape as the destructor) and `dynamic_cast`s each entry to `im::app::car::Health` — **confirms `im::app::car::Health` exists in this binary** (RTTI-searchable, not yet done — see §7), tagging a flag byte at `Health+528` conditionally. |
| 8 | `0x2d969c` | **`RaceLoaderTask_DispatchInitialFSMEvents`** | Dispatches a sequence of numbered events (codes `17,18,22,2,20,19` at priorities `2..7`) via a generic `sub_2D9398(this, eventCode, priority, args)` — almost certainly `SendEvent(raceFSM, code, priority, args)` into the `Race` FSM object stage 7 just loaded — followed by a race-type-dependent final code (`switch` on a byte at `this+8`-struct`+112`: `0→11, 1→(12 or 0), 2→10, 5→13, default→46`), then event `9`. Event-code meanings not yet decoded, but this is clearly the "kick off the freshly-loaded race FSM" step. |
**Conclusion for subtask 1**: `RaceLoaderTask_BuildTrackScenePath` (`0x2db384`) is the concrete hook point — it's stage 1 of 8, runs before anything else touches track data, and constructs the track path from a single string field. Two viable approaches, to be decided once the field's origin (where `this+8`-struct`+72` gets populated from the `RaceDefinition` SB) is traced: (a) hook this function and substitute the track-name argument before path construction, or (b) find and patch the earlier SB-parsing step that fills that field in the first place. Either avoids touching `RaceLoaderTask_LoadRaceFSM`/`ExecuteLoadSequence`'s overall sequencing, keeping the change minimal.
## 6c. Is there a "SceneLoader"? — theory and findings
The user recalled a "SceneLoader" from memory. No C++ class with that literal name exists (searched RTTI strings exhaustively; also checked the reverse-engineered Java launcher sources — nothing named `SceneLoader` there either, unlike `BitmapGraphics` which did turn out to be real). However, the underlying *system* the user is likely remembering is real and now identified:
- **`sub_33C284` → `sub_33B038` is a generic, engine-wide scene-loading function, confirmed independent of `RaceLoaderTask`.** Traced via `xrefs_to`: it has **9 separate call sites** spread across four unrelated address ranges (`0x21e9f4`, `0x233f30`, `0x23bec8`, `0x23e8c4`, `0x242a28`, `0x245cd8`, `0x24a1f4`, `0x24d2f4`, plus our own `GenericLoadScene_trackEnvWrapper` at `0x2dbf58`). It builds an `im::app::NFSScene` object (confirmed via RTTI: `N2im3app8NFSSceneE`, and via string refs `NFSScene::PreUpdate`/`NFSScene::PostUpdate`) from a pair of paths. **This is functionally "the scene loader"** — shared infrastructure used by race loading, and evidently by several other subsystems (menu/garage/track-test — see below), even though it's not organized as a named class.
- **`im::app::NFSScene`** participates in a generic async task-pipeline abstraction: `im::general::pipeline::Stage<boost::shared_ptr<im::app::NFSScene>>` (RTTI confirmed at `0xd88004`). Located its vtable (`0xd88040`) but it turned out to be pure template scaffolding (destructor pair + a `pure_virtual_called` trap at slot 3) — the actual "load in progress"/"is complete"/"get result" logic must live in an undiscovered concrete subclass. Not pursued further this session.
- **`im::app::LoaderTask`** (RTTI confirmed `N2im3app10LoaderTaskE`, base of `RaceLoaderTask` per §6a) is the generic async "task" abstraction — sibling classes confirmed: `im::app::MenuLoaderTask`, `im::app::StartupLoaderTask`. So the engine's actual naming is `LoaderTask` (for the task/lifecycle side) + `NFSScene` (for the loaded-result side) + the shared `sub_33C284` free function (for the actual load work) — three different pieces, none literally called "SceneLoader", together doing what that name implies.
### Two new hot leads found while chasing this (not yet decompiled in full)
1. **`sub_233F30` (`0x233f30`)** — references **both** `/published/data/races/` (the `RaceDefinition` SB path) **and** `published/prefabs/tracks/` + `.scene.sb` (the track path) in the same function, then calls the generic scene loader directly. This may be the actual top-level "load race by ID" entry point that creates/drives `RaceLoaderTask` — which earlier investigation (see §1.1's provenance notes and the original "who calls DoLoadRace" question) never located. **Worth decompiling in full next.**
2. **`sub_24A1F4` (`0x24a1f4`)** — references strings `"TrackTestLayer"`, `"Car"`, `"start"`, `"finish"`. Strongly suggests an **existing track-testing/debug harness** that loads a track directly, likely bypassing the normal race-select Flow entirely. If confirmed, this could be the safest possible hook point for subtask 1 — reusing an already-exercised internal test path instead of a raw hook into `RaceLoaderTask`'s internals. **Worth decompiling in full next**, and worth checking whether it's reachable from the existing `devmenu` module in the launcher (the launcher already has a dev-menu concept — this could be a natural fit).
Both addresses are commented in the `.i64` as "HOT LEAD" for continuity.
## 6d. `TrackTestLayer` — a real, built-in EA QA track-testing tool (major subtask-1 lead)
Followed up on `0x24a1f4` per user request. **This is the constructor of a real class literally named `TrackTestLayer`** (confirmed: `sub_42BAFC(a1, "TrackTestLayer")` tags the object immediately before the vptr is set) — a genuine EA/Firemonkeys internal QA tool, fully present and functional in this shipped binary, not something we're inferring loosely.
**`TrackTestLayer_ctor` (`0x24a1f4`) signature and behavior:**
- Second constructor argument is a pointer to **the scene/track path string to load**.
- Immediately calls the generic scene loader (`sub_33C284`, see §6c) with that path — i.e. **this bypasses `RaceLoaderTask`/Flow entirely** and loads a scene directly.
- After loading, locates `"start"`/`"finish"` named nodes in the scene (`sub_695A10`) — the track's start/finish line markers.
- Spawns a **hardcoded list of 8 real test cars** (description IDs: `she_day_cou_65`, `ast_one_77_10`, `aud_r8_v10_10`, `bmw_m3_gts_10`, `lam_mur_sv_71`, `lot_evo_stk_10`, `mer_sls_amg_10`, `nis_240_zg_71` — Shelby Daytona Coupe, Aston Martin One-77, Audi R8 V10, BMW M3 GTS, Lamborghini Murciélago SV, Lotus Evora Stack?, Mercedes SLS AMG, Nissan 240SX), each with a `RaycastCar` (physics) component **and bound to the scene's `TrackNavigator` component** — i.e. cars actually drive on the track via the coordinate system we found in §6a/§6b.
- Writes telemetry to `/var/{0}.csv` per car (lap-time/QA logging, standard for this kind of tool).
**Callers / how it's triggered:**
- `TrackTestLayer_factory_wrapper` (`0x24aec4`) — a thin `TrackTestLayer(path)` factory, called from `0x21fd8c` inside `DebugTestHarness_DispatchByName`, and also referenced as a data pointer at `0x22092c` (a second, not-yet-identified call site).
- `TrackTestLayer_QABatchRunner_tick` (`0x243160`) — an **automatic QA batch-runner**: every N frames (default 30, at `this+40`), advances through up to 15 track-path entries in a runtime string table (`dword_DD0B54`, bounded by `dword_DD0C44`/`dword_DD0C48` — all-zero in the static image, so populated at runtime, not found statically; likely from a debug config file), constructing a fresh `TrackTestLayer` for each. I.e., there's a built-in "cycle through every track automatically" QA mode too.
- **`DebugTestHarness_DispatchByName` (`0x2217ec`)** — takes a single `const char* modeName` argument. Matches it against category prefixes: `"Track/"` (→ `TrackTestLayer`), `"Performance/"`, `"CarPreview/"` (with a confirmed concrete path `/published/prefabs/garage/car_preview.scene.sb`), `"CarThumbnailMaker/"`, `"CollisionTest/"`, `"RacingLine/"`, plus standalone modes `"MetaPerformanceLayer"`, `"MemoryLeakCheck/ClearMemory"`, `"MemoryLeakCheck/NoClear"`, and — critically — **`"MainMenu"`/`"MAIN_MENU"`, which is presumably the normal shipping-game default.**
- Only one caller found this session: `ResourceDirListeners_Init_maybeCallsDebugHarness` (`0xcf904`), itself called once at startup from `0xde1e4`. **Where the actual runtime `modeName` value comes from is not yet traced** — candidates: a debug config file, an environment variable, a hardcoded literal further up the call chain, or (less likely in a shipping build) a command-line/Intent-extra. This is the single most valuable next step: if `modeName` is externally overridable (e.g. read from a file we can write, or an env var `mpcore` can set before `libapp.so`'s init runs), **we could boot the game directly into `TrackTestLayer` with an arbitrary path — a fully legitimate, pre-built loading path that bypasses `RaceLoaderTask`, Flow, and the menu system entirely, and comes with working start/finish detection and reference cars already bound to `TrackNavigator`.**
**Practical implication for subtask 1 (and partially 2/4)**: this is now the strongest candidate approach, ahead of hooking `RaceLoaderTask_BuildTrackScenePath` (§6b) — construct `TrackTestLayer` directly. It gets us: scene loading, start/finish detection, and car-to-`TrackNavigator` binding all for free, using code EA already tested.
## 6e. Where the debug mode string comes from — traced to a dead end (use a direct call instead)
Traced `ResourceDirListeners_Init_maybeCallsDebugHarness` (`0xcf904`) in full: its last line is `return DebugTestHarness_DispatchByName(*(const char **)(a1 + 292));` — the mode-name string is read from offset `+292` of its argument object. That function is called exactly once, at boot, from the app bootstrap function `sub_DE180` (`0xde180`), as `ResourceDirListeners_Init_maybeCallsDebugHarness(v9)` where `v9` comes from a multi-step "resolve current instance" call chain (`sub_3C969C→sub_5821AC→sub_1911C8→sub_19073C→sub_D0AAC→...→sub_D2880`) — not `sub_DE180`'s own parameter, a separately-resolved object. Did not fully identify this object's class or trace who populates its `+292` field with a concrete value.
**A second, related mechanism was found a few lines later in the same `sub_DE180` bootstrap function**: it looks up a config value for the literal key `"flow"` (via `sub_3BF344`, backed by a global config/tweaks singleton at `dword_DD2E9C`) and compares it against the literal string `"STARTUP_RACE"` — if equal, a special code path runs instead of the normal Flow-init callback. This looked very promising (a named, deliberate "boot straight into a race" switch), so it was checked against every plausible shipped-asset source:
- **All 8 files in `game_cache/published/tweaks/`** (`debug_options.sb`, `tweaks.sb`, `race_tweaks.sb`, `track_performance_tweaks.sb`, `tweaks_ipad.sb`, `traffic_tweaks.sb`, `car_preview_tweaks.sb`, `lod_tweaks.sb`) were unpacked via `NFSMW12MobileTools` (no binary RE needed — see ANALYSIS.md §1.1 for the tool). **None contain a `"flow"` key or a `"STARTUP_RACE"` value.** (`debug_options.sb` did turn out to be a rich, directly-editable in-game debug menu tree — see the bonus finding below.)
- **The reverse-engineered launcher Java sources** (`launcher/app/src/main/java/...`) have no `STARTUP_RACE` string and no `getIntent()`/`getStringExtra()` handling that looks related (the only `getIntent`/`getStringExtra` usages found are in EA Nimble's push-notification/referrer-tracking code, unrelated).
**Conclusion: this is very likely an EA-internal, build-time-only debug switch** (probably set via their internal QA build tooling, e.g. a custom launcher argument or build flavor never shipped to us), **not reachable through any config file, Intent extra, or asset present in this project.** Chasing the "legitimate" source further is a dead end with the material available.
**This does not block subtask 1.** We already have everything needed to use `TrackTestLayer` without going through this dispatch mechanism at all: its constructor address (`0x24a1f4`) and calling convention (2nd arg = track/scene path pointer) are known (§6d). The practical plan is to **call `TrackTestLayer_ctor`/`TrackTestLayer_factory_wrapper` directly from a `mpcore` hook** once `libapp.so` is loaded and initialized, passing our own path — bypassing the mode-string plumbing entirely rather than trying to trigger it "the intended way."
## 6f. First live test: calling `TrackTestLayer_ctor` directly from `mpcore` (on-device, WayDroid)
Actually implemented and tested this, twice, on the running WayDroid setup (`com.ea.games.nfs13_na`, `no_devmenu` debug build). Both attempts crashed, but each crash pinpointed a concrete, understood cause — this is real progress, not a dead end.
**Design choice**: used a plain `std::thread` with a 15-second sleep in `mpcore`'s `JNI_OnLoad`, calling the constructor directly, rather than an inline hook via `armhook.cpp`'s `InstallHook`. Reasoning: that hook infrastructure is untested (see ANALYSIS.md §6 / PROGRESS.md), and mixing an untested hook mechanism with a first attempt at calling a new function would make any crash ambiguous (hook bug vs. constructor-call bug). A direct delayed call isolates the variable under test. `raise(SIGSTOP)` (existing in `main.cpp`, presumably a debugger-attach aid) was commented out for this experiment since it suspends the whole process, including the new thread.
**Attempt 1 — crash inside the string-builder helper itself.** Called `sub_CF5F8(destObj, cstr)` with what was assumed to be a 2-argument "assign from C-string" signature. Crashed instantly (SIGSEGV, `SEGV_MAPERR`) at `sub_CF5F8+0x68`, inside a `memmove` call. **Root cause, confirmed by decompiling `sub_CF5F8` properly**: its real signature is `sub_CF5F8(dest, rangeBegin, rangeEnd)` — a `[begin, end)` range constructor (length = `rangeEnd - rangeBegin`), not an implicit-strlen C-string assign. Every caller seen throughout the binary manually scans for the string's end pointer first (the odd-looking `do { ...*ptr++... } while(*ptr)` patterns noted in earlier decompiles were exactly this) before calling it with 3 arguments. Calling with only 2 left the 3rd ABI register (`end`) as garbage, producing a bogus `memmove` length. **Fixed** by computing `trackPath + strlen(trackPath)` as the third argument.
**Attempt 2 — crash inside the start/finish node lookup.** After the fix, the log confirmed the call reached `TrackTestLayer_ctor` (`0x24a1f4`) itself and got well past scene loading and car-catalog setup (no crash there — meaningful validation that scene loading via the generic loader works when called this way). Crashed later, inside `sub_FDA64` (called on the result of `sub_695A10("start"/"finish", ...)` — the named-node lookup from ANALYSIS.md §6d). **Root cause**: `sub_695A10` returns `{0,0}` when no top-level scene node with that exact name exists, and `TrackTestLayer_ctor` passes that result straight into `sub_FDA64` without a null check, which dereferences it (offset `+0x14`) and segfaults. Checked the actual test track used (`published/prefabs/tracks/region1_foothills_track3.scene.sb`, picked somewhat arbitrarily from the 48 available) by unpacking it via `NFSMW12MobileTools`: it does contain `"start"`/`"finish"` as **nested path segments** (e.g. `checkpoints/banner/en/mesh_start/mesh/vertices`) and several `"Name": "Start"` (capitalized) entries, but apparently not a **top-level node named exactly `"start"`** (lowercase) — either a case-sensitivity mismatch or this specific track genuinely isn't structured the way `TrackTestLayer` expects (it may not be the actual track used by any real race event — picked without checking that).
**What this establishes**: the calling convention, ABI, and approach are correct — we successfully call from a foreign thread into `libapp.so`'s C++ internals, construct its native string objects correctly (after the fix), and drive real engine subsystems (scene loading, car catalog) without crashing. The remaining blocker is track-specific (this constructor expects the target scene to already have "start"/"finish" markers at the top level) and not fundamental. `mpcore`'s current committed code **will crash on launch as-is** (still targets `region1_foothills_track3.scene.sb`) until either a track confirmed to have the right top-level markers is substituted, or the missing null-check is patched around.
**Follow-up (same session): tried a race-linked track — same crash, ruling out "wrong track" entirely.**
Cross-referenced `game_cache/published/data/races/event_01_race.prefabs.sb` (a real, shipped race event): its `TrackName` field points to `region4_chicago_track4`. Unpacked `region4_chicago_track4.scene.sb` via `NFSMW12MobileTools` and confirmed it contains `"actor"` structs named exactly `"start"` (id `1C610000`, prefab `/start.prefabs.xml/start`) and `"finish"` (id `AA090000`, prefab `/finish.prefabs.xml/finish`) — and, critically, **both IDs are listed in the scene's top-level `"actors"` `DataIdsMap` (id `04000000`)**, alongside 11 other sibling actor IDs — i.e. these are not deeply nested; they're direct entries in what is almost certainly the exact list `sub_695A10`'s lookup iterates.
Rebuilt `mpcore` targeting this track and re-ran the live test on WayDroid. **Identical crash, same PC (`sub_FDA64+0x18`, i.e. dereferencing a null lookup result).** This rules out "wrong/non-gameplay track" as the explanation — the data is present and structurally exactly where expected, yet the runtime lookup still fails.
**Revised theory**: the failure is not about the scene's authored content but about *runtime state at the moment of the lookup*. Two candidates, neither confirmed:
1. **Scene loading may be asynchronous** — `sub_33C284`/`sub_33B038` (§6c) could parse/instantiate the scene over multiple engine ticks rather than fully synchronously within the call, and `TrackTestLayer_ctor` (in its normal, EA-authored call context) may rely on being re-entered or polled across frames before doing the start/finish lookup, something a single direct constructor call from a foreign thread can't replicate.
2. **`TrackTestLayer` may depend on ambient global/singleton engine state** (e.g. a "current active world/scene" pointer) that's normally established by whatever code path constructs it in EA's own tooling — a path we never found (§6e) and therefore can't replicate — rather than being self-contained from just its two constructor arguments.
**Recommendation: stop iterating on `TrackTestLayer` blindly.** Three attempts (bad-track-guess, then a confirmed race-linked track, both crashing identically) is enough signal that this isn't a simple parameter-tuning problem, and further guessing without new information is unproductive. Two real forward paths, to decide with the user:
- **(a)** Properly instrument/trace the scene-load completion state (e.g. poll whatever ref-count or "loaded" flag `sub_33C284`'s output object exposes before attempting the lookup) — a scoped, legitimate next RE step if `TrackTestLayer` is still wanted.
- **(b)** Pivot subtask 1 to hooking `RaceLoaderTask_BuildTrackScenePath` (§6b) instead — the engine's own always-used, always-correctly-initialized code path for every normal race load, sidestepping whatever implicit dependency `TrackTestLayer` has. Given `TrackTestLayer` itself has an unguarded null-deref bug (§6d) that fires on a supposedly-correct track, treating it as a reliable, maintained tool in this specific shipped build (1.3.128) is now in question — it may be stale/unexercised in this build even though the code is present.
**User chose (b).** See §6g for the implementation and where live testing currently stands.
## 6g. Pivoted to hooking `RaceLoaderTask_BuildTrackScenePath` — ARM-mode inline hook, installed and stable; end-to-end test blocked by WayDroid networking
**Key discovery before implementing anything**: disassembled `RaceLoaderTask_BuildTrackScenePath` (`0x2db384`)'s prologue and found it is compiled in **ARM mode**, not Thumb — `E92D41F0` (`PUSH {r4-r8,lr}`), `E24DD040` (`SUB SP, SP, #0x40`), both classic 32-bit ARM encodings (condition-code nibble `E` = "always", a dead giveaway vs. Thumb's 16-bit encodings). This matters a great deal: `armhook.cpp`'s existing `InstallHook`/`makeJMP`/`HOOK_PROC` machinery is hardcoded for **Thumb** targets (its `HOOK_PROC` byte sequence starts `01 B4 01 B4 01 48...`, all 16-bit Thumb opcodes). Using it on this ARM-mode function would misinterpret/corrupt the target — the existing hook infrastructure **cannot be used here as-is**. This confirms the earlier caution (ANALYSIS.md/PROGRESS.md) about that code being untested was well-founded, and now there's a concrete reason it would specifically fail on this target.
**Traced the exact field being overridden**: `sub_F5154`'s real signature is `sub_F5154(dest, literalCStr, srcRangeObjPtr)` — the third argument is the **address** of a 3-word `{begin,end,capacity}` string object (same layout as `sub_CF5F8`'s objects, read via `a3[0]`/`a3[1]`), not a raw string. So in `RaceLoaderTask_BuildTrackScenePath`'s call `sub_F5154(&v16, "published/prefabs/tracks/", a1[8] + 72)`, the track name lives as an **inline 3-word string object at byte offset 72 within the `RaceDefinition`-like struct pointed to by `RaceLoaderTask.this[8]`** (word index 8 = byte offset 32). To override the track: read `raceDefPtr = this[8]`, then call `sub_CF5F8(raceDefPtr + 72, newName, newName + strlen(newName))` to overwrite that field in place before the path gets built — reusing the exact same string-range helper already understood from the `TrackTestLayer` work (§6f), just applied to an *existing* object's field instead of a fresh stack buffer.
**Implemented (`launcher/mpcore/src/main/cpp/main.cpp`)**: an ARM-mode-correct inline hook, separate from `armhook.cpp` (documented inline why):
- Overwrite the target's first 8 bytes (exactly 2 whole ARM instructions — safe, ARM instructions are always 4 bytes wide, so there's no mid-instruction split risk the way there is in mixed 16/32-bit Thumb-2) with `LDR PC, [PC, #-4]` (`0xE51FF004`) + the hook function's address.
- A trampoline (RWX `mmap`'d page) holds the 2 displaced original instructions (both confirmed position-independent — plain `PUSH`/`SUB`, no PC-relative addressing, so relocating them is safe) followed by the same `LDR PC, [PC,#-4]` pattern jumping back to `target+8` to resume the original function in ARM mode.
- Mode-switch correctness: the hook function's address (a normal Thumb-2-compiled C++ function, since `mpcore` builds as Thumb-2 by default) has bit 0 set automatically by the compiler/linker for ARM/Thumb interworking — loading it into `PC` via `LDR` triggers the CPU to switch to Thumb state, per standard ARMv5T+ interworking semantics. The trampoline's jump back to `target+8` uses a plain (bit-0-clear) address, staying in ARM mode as required.
- The hook itself: reads `this[8]` (the `RaceDefinition` pointer), overwrites the string object at `+72` with a hardcoded override track name (`region3_colorado_track2` — deliberately different from any track a real race would use, so a successful override is visually unmistakable), logs, then calls through to the original via the trampoline.
- Installed unconditionally and immediately in `JNI_OnLoad` (no delay needed, unlike the `TrackTestLayer` experiment — patching code bytes is safe at any time since nothing is being called yet; the patched function only executes later, whenever the player naturally starts a race through the normal menu).
- The old `TrackTestLayer` thread-spawn experiment (§6f) is left in place but commented out (superseded, not deleted, consistent with this file's existing style of preserving prior experiments as commented-out code).
**Live test status (WayDroid)**: hook installed cleanly — `mpcore_log` confirmed `"Installed RaceLoaderTask_BuildTrackScenePath hook, orig trampoline=0x..."` and the app continued running (and slowly growing in RSS, i.e. actively doing work, not frozen) for 2+ minutes afterward with **no crash** — reasonably strong indirect evidence the 8-byte ARM patch itself is correctly formed and didn't corrupt anything nearby. However, the screen stayed black the entire time and never reached the main menu, so the hook itself was never actually exercised (it only fires when `RaceLoaderTask_BuildTrackScenePath` is called, i.e. once a race load begins). Diagnosed the black screen: **not a bug in this session's changes** — `ip route` inside the WayDroid container shows only the local `192.168.240.0/24` subnet with **no default gateway**, confirmed by 100% ping packet loss to `8.8.8.8`; the boot log shows `NIM_ERROR: No network connection` during EA's Nimble/EASP init, and the app likely hangs on a network-dependent init step (or a long timeout) before ever reaching the menu. Fixing this needs root inside the WayDroid container (`waydroid shell` requires root; no passwordless `sudo` available in this session) — a genuine infrastructure gap, not something resolvable from here without the user's involvement.
**Bottom line**: the hook mechanism itself is implemented and shows no signs of corrupting the binary (survives 2+ minutes of live execution). What's *unverified* is whether it actually fires and successfully substitutes the track name when a real race is started, because the WayDroid container currently can't get far enough into the game's boot sequence to reach the menu. Next step once network is available: launch, reach the menu, start any race event, and confirm via `mpcore_log` that the hook fired and via screenshot that `region3_colorado_track2` (not the race's real track) loads.
**Correction (same day)**: the "no network" theory above was wrong. The user identified the real cause: the black screen is because this build (`applicationId com.ea.games.nfs13_na`) has no game_cache to find — WayDroid already has a cache, but only under the package name `com.ea.games.nfs13_mod` (confirmed: `com.ea.games.nfs13_mod` is installed there separately, `versionName 1.3.128`, and `/sdcard/Android/obb/com.ea.games.nfs13_mod/main.1003128.com.ea.games.nfs13_mod.obb`, a 623MB file dated 2020, exists on the device). The `NIM_ERROR: No network connection` log line is expected/harmless — a stub the user placed during reverse engineering, not a real blocker.
## 6h. Switching to `com.ea.games.nfs13_mod` — versionCode fix was necessary for install, but its OBB-lookup justification is UNVERIFIED
Changed `launcher/app/build.gradle.kts`: `applicationId` → `com.ea.games.nfs13_mod` (`namespace` left as `com.ea.games.nfs13_na` deliberately, since source files still `import com.ea.games.nfs13_na.BuildConfig` and changing `namespace` would require updating those). First install attempt failed with `INSTALL_FAILED_VERSION_DOWNGRADE` (our `versionCode=1` vs. the already-installed real game's `versionCode=1003128`) — this is a **confirmed, OS-level fact** (Android's package installer itself enforces version-code-based downgrade protection; `adb install -r -d` bypasses it, or bumping our own `versionCode` to match avoids it entirely). Set `versionCode = 1003128`, `versionName = "1.3.128"` to match, after which `adb install -r` succeeded without needing `-d`.
**The comment originally added alongside that change — that the game's own code looks up its OBB file via `main.<versionCode>.<applicationId>.obb`, so versionCode needed to match for asset loading — was an unverified assumption, not a checked fact, and the user correctly asked for evidence.** Checked properly:
- `grep` (case-insensitive) for `obb` across the **entire** `launcher` source tree (all `.java`/`.kt`/`.xml`) — **zero matches**.
- `find_regex` for `addAssetPath|mountObb|StorageManager|\.obb` in `libapp.so` — **zero matches** (this check was actually already done earlier in the session, but its implication for the versionCode comment wasn't connected until asked).
**Conclusion: there is no evidence in this codebase (Java or native) that the game constructs or checks an OBB filename against its own versionCode, or uses Android's OBB/StorageManager APIs at all.** The `main.1003128.....obb` file on the test device is most likely a leftover from the original Play Store install of the real game (Google's own expansion-file delivery mechanism placed it there historically), not something this reverse-engineered launcher's code actively looks for. The `versionCode` fix was corrected in `build.gradle.kts` to state only what's actually confirmed (avoids the installer-level downgrade block) and flags the OBB-lookup theory as unverified. It should not be treated as "the fix" for the black-screen/cache-loading problem — that mechanism is still unidentified.
**Still open**: how does this build actually locate `game_cache/published/...` at runtime? Three candidate mechanisms were floated as hypotheses, none confirmed: (a) Android's native `AAssetManager` reading straight from the APK's own bundled `assets/` folder (plausible: `EAIO.Startup()` does pass `activity.assets`, an `AssetManager`, into native code) — but this would require `game_cache` to be bundled inside the APK's `assets/`, not on external storage at all; (b) plain loose files read via `fopen`/NDK file I/O from some external-storage path (would need to find where that path is set) — for AllocationList and the actual root/prefix, not yet traced; (c) some other, not-yet-considered mechanism. Not pursued further this session — genuinely needs either tracing the native file-open call chain from `RaceLoaderTask_BuildTrackScenePath`'s eventual `sub_33B038`/`sub_33C284` down to the actual `fopen`/`AAssetManager_open` call, or the user's own recollection from prior reverse-engineering of this launcher.
**Update — resolved live, same day**: switching the launcher project (now at `/home/megboyzz/AndroidStudioProjects/NFSMostWanted128`, symlinked as `launcher/`) to its current/correct state and rebuilding actually answered this empirically rather than requiring more static RE: the boot log shows `Mounting SKU: texture_dxt3 to /published`, `Mounting SKU: 1x/2x/4x to /published`, followed by real asset loads (`FlowManager splash.sba sba`, `ResourceManager: Add texture: /published/texturepacks_ui/splash_1775.sba_0`) — confirmed on-screen too, the actual EA logo splash rendered (not a black screen) on WayDroid. So the OBB *is* mounted somewhere such that `/published/...` resolves — mechanism (a)/(b)/(c) above still isn't pinned down architecturally, but it demonstrably works end-to-end now with the corrected launcher+native_libs+OBB file combination. Not worth further static tracing unless it breaks again.
## 6i. CRITICAL: the previous session's `.i64` was stale — silently out of sync with the on-disk binary; rebuilt fresh
**What happened**: the user replaced `native_lib/libapp.so` (and the whole `native_lib/` set, and the `launcher/` project itself — see §6h) with corrected versions partway through the previous session. `native_lib/libapp.so.i64` was **not** rebuilt at that point, but continued to open "successfully" and even reported `survey_binary` metadata (`sha256`/`md5`) that exactly matched the *new* on-disk file — which looked like reassuring confirmation that the database was current. **It was not.** Proof, obtained by direct comparison:
- Raw bytes read straight from the current `native_lib/libapp.so` file at offset `0x2db384` (via plain Python `open().seek().read()`, bypassing IDA entirely): `74 00 8D E2 0C 00 8D E5 04 00 A0 E1 15 B6 06 EB` → decodes to `ADD R0,SP,#0x74` / `STR R0,[SP,#0xC]` / `MOV R0,R4` / `BL ...`.
- What the (at-the-time still-open, "hash-matching") `.i64` showed for that same address: `PUSH {R4-R8,LR}` / `SUB SP,SP,#0x40` / `LDR R2,[R0,#0x20]` / ... — the `RaceLoaderTask_BuildTrackScenePath` prologue found and hooked in §6g/§6h.
**These are different instructions at the same address — conclusive proof the open `.i64` was analyzing stale, cached content, not the file on disk it claimed to correspond to.** The likely mechanism: `survey_binary`'s reported `sha256`/`md5` is computed live by re-reading whatever is *currently* at the recorded `input_path` on disk, independent of whatever was actually analyzed and cached inside the `.i64` when it was first built — so a hash match there proves the *external file* is unchanged since some point, but says nothing about whether the loaded database's internal snapshot still matches it. **Takeaway for future sessions: never trust `survey_binary`'s hash fields alone as proof an `.i64` is current after a binary might have changed underneath it — cross-check actual disassembly/raw bytes at a known address directly, exactly as done here.**
**Fix applied**: moved the stale database aside (`native_lib/libapp.so.i64.stale_2026-07-31`, not deleted — kept for reference/comparison against the old, pre-relocation build if ever needed) and opened `native_lib/libapp.so` fresh via `idb_open` (pointing at the `.so`, not an existing `.i64`), forcing a full auto-analysis. Verified the fix the same way it was diagnosed: disassembly at `0x2db384` in the fresh session now matches the raw file bytes exactly (`ADD R0,SP,#0x74` / `STR` / `MOV R0,R4` / `BL`).
**New binary's basic stats** (`survey_binary`, fresh session): `image_size 0xb167d0` (~11.6MB, vs. the old build's `0xe52ebc`/~14.9MB), **34,726 total functions** (vs. 50,772), 2,425 named, 21,031 strings, 19 segments. JNI entry points are now `Java_com_ea_ironmonkey_GameActivityMain_*` (matching the renamed `GameActivityMain` Kotlin class from §6h — the *native* side was rebuilt to match too, not just Java). New debug-assertion strings not seen in the old build (`"GetComponent: called with a component type that allows multiple instances..."`, `"Dereferencing a NULL component pointer."`) and `libc++`'s `__ndk1` namespace suggest a different/newer NDK toolchain and possibly a less aggressively stripped build than before. **This is a genuinely different compiled build, not merely a renamed or re-packaged file.**
**Good news: the architecture holds.** Re-ran the RTTI string search (`RaceLoaderTask|TrackTestLayer|NFSScene|TrackNavigator|OpponentCollection|BitmapGraphics`) against the fresh binary — every one of these classes is still present, confirming this is the same engine/codebase lineage, just recompiled with different addresses/layout. One correction/refinement: `TrackTestLayer`'s full namespace is confirmed as **`im::app::layers::debug::TrackTestLayer`** (a `layers::debug` namespace — previously only knew the bare class name), which also confirms the earlier characterization of it as a debug/QA tool directly from its own mangled namespace, not just inferred behavior.
**Practical implication**: every specific address in §6a–§6h (RaceLoaderTask's vtable at `0xd86210`, `RaceLoaderTask_BuildTrackScenePath` at `0x2db384`, `TrackTestLayer_ctor` at `0x24a1f4`, `BitmapGraphics_*` functions, etc.) is **invalid for this binary** and needs to be re-derived from scratch using the same techniques (RTTI name → `find_bytes` walk to vtable, decompile candidate functions, confirm via distinctive strings/constants). The `mpcore` hook installed in §6g/§6h currently targets a now-meaningless offset in this binary and must not be re-enabled as-is. The current `mpcore/main.cpp` (per the user's own recent edit, visible in the system reminder) has already moved on from that hook to a fresh `dl_iterate_phdr`-based base-address lookup with no hooks installed yet — consistent with starting over.
### Bonus finding: `game_cache/published/tweaks/debug_options.sb` is a real, editable in-game debug menu
While searching for `"flow"`, unpacking `debug_options.sb` revealed it's a full debug-menu configuration tree (`PropertyNameCHDR`/`DebugMenuPath` entries), directly useful for the mod and editable with zero binary patching (`NFSMW12MobileTools` unpack → edit JSON → repack). Top-level categories: `AI`, `Black Market`, `Camera`, `Collision`, `Garage`, `HUD`, `RacingLinePreviewLayer`, `Shader`. Notable entries directly relevant to this project:
- `AI/Race/Max Num AI Opponents`, `AI/Race/Disable AI Opponents` — direct control over opponent count, relevant to subtask 2.
- `AI/Race/Disable Rubber Banding`, `AI/Race/Disable Nitro`, `AI/Race/Enable Player AI Nitro`, `AI/Race/Weaken Opponents` (top-level `Weaken Opponents` also exists) — race-tuning toggles.
- `Everything is Available`, `Everything is Free` — content-unlock flags.
- `Infinite Player Nitro`, `Disable Traffic`, `Disable HUD`, `Enable Soak Test` — misc QoL/testing toggles.
- `AI/Race/Draw AI Track Navigators`, `AI/Dynamic Raceline/*` debug-rendering toggles — could help visualize the `TrackNavigator` coordinate system from §6a while developing subtask 4.
Not yet investigated: how `debug_options.sb`'s values actually get read into the running game (same `dword_DD2E9C`-style config singleton as `"flow"`, presumably) and whether editing this file alone is sufficient or whether the game reads a *device-local* copy that would need pushing separately. Worth a quick practical test via WayDroid before relying on it.
## 6j. `RaceLoaderTask` rediscovered in the fresh binary — vtable, 6 slots confirmed, `BuildTrackScenePath` equivalent found
Per user instruction ("Начни с RaceLoaderTask"), redid the §6a RTTI-walk against the current, verified-fresh `native_lib/libapp.so.i64` (rebuilt in §6i). Same technique: RTTI name string → `xrefs_to` → `type_info` struct → `find_bytes` for a pointer to that `type_info` → real vtable starts 4 bytes after the match.
- RTTI name `N2im3app4race14RaceLoaderTaskE` now at `0xa2f570` (was `0xcfe924` in the old binary).
- `type_info` struct at `0xaa7938` = `{vtable=0xb15fb4, name=0xa2f570, base_typeinfo=0xa9b5a0}`.
- `find_bytes` for `0xaa7938` (LE) returned 3 hits: `0x620d0` (false lead — the engine's own custom reflection/type-registry table, pattern `{ptr, 0x1802}` this time instead of the old binary's `{ptr, 0x17}` — same false-lead shape as §6a, just a different tag constant), `0xaa78e8` (the real vtable — confirmed by the same "long run of `.text` addresses immediately followed by the class's own `type_info`" pattern used in §6a), `0xac6514` (unexamined — likely a different class's `base_typeinfo` field, same as the unresolved false lead in §6a).
**`RaceLoaderTask` vtable**: `0xaa78e8`, **18 virtual function slots** (`0xaa78ec`–`0xaa7930`) — one fewer than the old binary's 19 (§6a slot 0's "shared thunk" role may have been folded/removed; not confirmed, low priority).
| Slot | Address | Renamed to | Role / evidence |
|---|---|---|---|
| 0 | `0x7e320` | — | Very low `.text` address, shared across many unrelated vtables (same "generic base-class thunk" shape as old binary's slot 0/3/5) — not decompiled. |
| 1 | `0x2a7e58` | `RaceLoaderTask_dtor_complete` | Sets vptr back to `0xaa78ec` (own vtable), decrefs a member at `a1[41]`. Same shape as old slot 1. |
| 2 | `0x2a8130` | `RaceLoaderTask_dtor_deleting` | Calls slot 1 then `sub_3D0C04` (`operator delete`). Same shape as old slot 2. |
| 3 | `0x7e450` | — | Shared-thunk region, same as slot 0. Not decompiled. |
| 4 | `0x2a8144` | `RaceLoaderTask_ExecuteLoadSequence` | **Main load orchestrator**, direct equivalent of old slot 4 (`0x2dbba4`). Calls a progress-setter (`sub_408AB4`, floats `0.1..0.8`) interleaved with 8 stage sub-calls (`0x2a8424, 0x2a89a4, 0x2a8b4c, 0x2a8ce8, 0x2a8e2c, 0x2a9274, 0x2a9338, 0x2a9bcc`), references `/published/texturepacks_ui/in_game.sba`. |
| 5 | `0x7e458` | — | Shared-thunk region. Not decompiled. |
| 6 | `0x2ade58` | — | `nullsub_258` — matches old slot 6 (also a nullsub) in relative position. |
| 7 | `0x2aa934` | — | Setter: `a1[16..18] = a2[0..2]` with refcount release on the old value — matches old slot 7's "Set<SharedResource>" shape. Role unconfirmed. |
| 8 | `0x2aab34` | — | Takes `(this, a2)`; builds a key via `sub_66FFB0`/`sub_2AAE68`, then appends a 3-word entry `{a2[0],a2[1],a2[2]}` into a growable vector at `this+76/80` (realloc via `sub_D746C`), plus registers into `this[11]+160` via `sub_27DFF0`. Vector-append shape suggests building a collection (candidate: opponent-car list), but **no distinctive string found** — do not treat as confirmed "SetupOpponentCar" (old slot 11 had the literal string `"opponentCar"`; this one doesn't). Left unnamed. |
| 9 | `0x2aafac` | `RaceLoaderTask_SetupPlayerCar` | References string `"playerCar"` — same as old slot 9. |
| 10 | `0x2ab4c4` | — | Single-arg `(this)`; refcounted lookup/creation against a component at `this+40`/`this+32`, dispatches through a callback vtable at `off_AA7978`. No distinctive string. Not confidently named. |
| 11 | `0x2abbf0` | — | Trivial 1-line thunk: `return sub_2AFED0(a1, a3);` — pure forwarding, not investigated further. |
| 12 | `0x2abc04` | — | Factory: allocates either a 0x14-byte or 0xAC-byte object (branching on a flag byte at `*a3+76`), tagging it with mini-vtable `off_AA7990` or `off_AA5048`. These mini-vtables are refcount-only closure wrappers (`{funcptr, funcptr}`, no RTTI/typeinfo — confirmed by reading their first word, which is a plain code address, not a typeinfo pointer), so class identity can't be recovered via RTTI here. Not confidently named. |
| 13 | `0x2abe08` | `RaceLoaderTask_ResetStartingLine` | References string `"m_StartLine"` + `"Assertion failed ("` — direct equivalent of old slot 14 (`"ResetLine"`). |
| 14 | `0x2acfa8` | — | `nullsub_257`. |
| 15 | `0x2acfac` | — | Iterates the same vector shape as slot 8 (`this[19]/this[20]`, 3-word stride), calling `sub_152250(raceCtx, ..., key, ...)` per element plus once before the loop — looks like a per-entry registration/lookup pass over whatever collection slot 8 builds. Not confidently named. |
| 16 | `0x2ad4a8` | `RaceLoaderTask_HandleSpikeStrip` | `dynamic_cast<im::app::bt::BlacklistTech> → im::app::bt::SpikeStrip` — direct equivalent of old slot 17. |
| 17 | `0x2ada80` | — | Allocates 3 closure objects (mini-vtables `off_AA7A58/AA7A80/AA7AA8`, same no-RTTI shape as slot 12) and registers each via `sub_2848A0(this[40], ...)`, then loops calling a virtual at `*this+68` per element of a collection at `*(this+16)+68`. Looks like registering 3 event listeners against whatever `this[40]` is (a dispatcher/FSM?), then notifying per-collection-element. Not confidently named. |
**Renamed and saved this round** (6 of 18 slots, all backed by a distinctive string or exact structural match to the old binary — the rest are left as `sub_`/generic to avoid repeating the "unverified claim" mistake from §6h): `RaceLoaderTask_dtor_complete`, `RaceLoaderTask_dtor_deleting`, `RaceLoaderTask_ExecuteLoadSequence`, `RaceLoaderTask_SetupPlayerCar`, `RaceLoaderTask_ResetStartingLine`, `RaceLoaderTask_HandleSpikeStrip`.
**`BuildTrackScenePath` equivalent found — stage 1/8 of `ExecuteLoadSequence`, address `0x2a8424`.** Decompiling all 8 stage sub-calls, the first one (`0x2a8424`, called first, matching old slot's call order) references the exact strings `"published/prefabs/tracks/"` (`0x9d7498`) and `".scene.sb"` (`0x9d74b2`) — the same path-construction signature as old `RaceLoaderTask_BuildTrackScenePath` (`0x2db384`, now stale). Renamed to `RaceLoaderTask_BuildTrackScenePath`.
**ARM-mode confirmed** at the new address too: raw bytes at `0x2a8424` are `F0 4F 2D E9` = `0xE92D4FF0` = `PUSH {R4-R11,LR}` (cond nibble `E`, classic 32-bit ARM encoding) — same situation as §6g's old-binary finding, so the same custom ARM-mode inline-hook design (overwrite first 8 bytes / 2 whole ARM instructions with `LDR PC,[PC,#-4]` + hook address; trampoline relocates the displaced instructions) is directly reusable here, just against this new address. Not yet re-implemented in `mpcore` this round (the old hook code was removed from `main.cpp` by the user per §6i — see PROGRESS.md).
**Next step**: trace where `RaceLoaderTask_BuildTrackScenePath`'s track-name field (equivalent of old `this[8]+72`) gets populated, confirm the field offset in this binary (register/stack layout may differ from the old build), then re-implement the ARM-mode hook in `mpcore/src/main/cpp/main.cpp` targeting `0x2a8424`, matching the design already proven working in §6g (just against new addresses).
## 6k. ARM-mode hook implemented and live-tested on real hardware (Pixel 6a) — mechanism works, exposes a real data-consistency limit
Per user instruction, implemented the ARM-mode inline hook designed in §6j/§6g against the fresh binary's `RaceLoaderTask_BuildTrackScenePath` (`0x2a8424`), in `launcher/mpcore/src/main/cpp/main.cpp`, and tested live via adb on the Pixel 6a (GrapheneOS) device — see `reference-pixel6a-grapheneos-testing` memory. This is the first live test of the mod's actual hook mechanism on any device (WayDroid never got far enough; the previous ARM hook in §6g was tested against the now-stale old binary).
**Implementation** (`Hook_BuildTrackScenePath`): reads `raceDefPtr = a1[8]` (word offset 8 = byte 32, confirmed identical to the old binary), then repoints the `{begin,end}` pointer pair for the track-name field (`raceDefPtr+72/+76`) at a static literal `"region3_colorado_track2"` — and, in a follow-up fix, also the environment-name field (`raceDefPtr+100/+104`) at `"colorado"` (matches the `.prefabs.sb`'s actual region folder). Deliberately does **not** free/reallocate the original buffers (leaks them — one tiny allocation per race load, negligible) since `BuildTrackScenePath` only ever reads these fields, never frees them; this avoids the capacity-field-offset guessing risk flagged as a concern before implementation. Hook installed via 8-byte ARM-mode patch (`LDR PC,[PC,#-4]` + hook address) at `JNI_OnLoad`, exactly as designed in §6g, just retargeted to the new address; trampoline relocates the 2 displaced `PUSH`/`ADD` instructions (both confirmed position-independent) and jumps back to `target+8`.
**Live test 1 (track-name override only)**: installed cleanly (`mpcore_log`: `Installed RaceLoaderTask_BuildTrackScenePath hook at 0xd7eff424, trampoline=0xe69e3000` — matches `libapp_base + 0x2a8424` exactly), no crash through menu navigation. Started the "Петерсон стрит" event (a real, working event — confirmed crash-free with the unmodified build in the same session, see PROGRESS.md). Hook fired (`BuildTrackScenePath hook fired: overriding track name -> region3_colorado_track2`) and the engine genuinely started loading Colorado-region assets (`Add asset: /published/textures/collidables/texture_collidables_colorado.sba`, colorado skydome references) instead of the real event's track — **conclusive proof the field override reaches the engine's actual path-construction logic**. However, ~150ms later: `Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x00000038` in a background thread (`Thread-9`), `Cause: null pointer dereference`, register `r0=00000000`.
**Live test 2 (track-name + environment-name override)**: hypothesized the crash was caused by the environment field still pointing at the original (non-colorado) event's environment while the track name pointed at colorado — a plausible mismatch given `BuildTrackScenePath` builds `"published/prefabs/environments/" + envName + ".prefabs.sb"` from that same struct. Added the environment override and retested. **Identical crash** — same fault address `0x38`, same thread name `Thread-9`, and (confirmed by computing `pc - libapp_base` from both tombstones) the **exact same code offset both times** (`0x53a604`), ruling out the environment-mismatch theory.
**Root cause, confirmed by decompiling the crash site** (`sub_53A5FC` at `0x53a5fc`, file offset `0x53a604` is its second instruction):
```c
int sub_53A5FC(int *a1, int *a2) {
if (a2) { *a2 = a1[14]; a2[1]=a1[15]; a2[2]=a1[16]; a2[3]=a1[17]; }
return a1[13];
}
```
`a1[14]` = byte offset `14*4 = 56 = 0x38` — **exactly the crash fault address**, and `r0` (== `a1`, the first argument) was `00000000` at crash time, per the tombstone register dump. So some caller passed a **NULL object pointer** into this small getter (looks like a generic transform/bounds accessor — copies a 4-word block, e.g. a quaternion or bounding-sphere, plus a flags word). `xrefs_to` found **21 distinct call sites** across what looks like physics/rendering component code (`sub_496AE8`, `sub_4970D8`, `sub_53EDAC`, `sub_53F060`, `sub_5749A0`, `sub_57AB8C`, `sub_57B0C4`, etc.) — this is common shared utility code, not something track/environment-specific, so tracing the *exact* call chain back to a specific `RaceLoaderTask` stage was not pursued further this session (would need substantial additional tracing across 21 call sites).
**Working theory** (consistent with the evidence, not yet independently confirmed by tracing further): `RaceDefinition` carries more than just track/environment name — checkpoint list, opponent list, starting-grid data, etc. (see `RaceLoaderTask_ResetStartingLine`, `RaceLoaderTask_LoadRaceFSM`'s checkpoint/car iteration in §6j's stage table) — none of which our hook touches. Those still reference IDs/objects from the **original** "Петерсон стрит" event, which don't exist in the substituted Colorado scene. A background streaming/physics thread (`Thread-9`) walking one of these now-dangling references gets a NULL lookup result and calls the unguarded getter on it. In other words: **swapping only the track-name/environment-name fields performs a real, engine-level scene substitution, but is not sufficient for a fully arbitrary track swap** — the race-logic fields (checkpoints, opponents, starting grid) need to come from a source consistent with the new track, not the old event.
**Bottom line for subtask 1**: the hook *mechanism* itself (ARM-mode 8-byte patch, trampoline, field-pointer override, no-free-leak strategy) is proven correct and crash-free in isolation — the crash is a downstream data-consistency issue, not a hook bug. Confirmed no tombstones/crashes at all when the same event was played through normally (unmodified) earlier in this session (see PROGRESS.md), isolating the regression to the override itself. **Next step**: either (a) pick a substitute track from an event whose checkpoint/opponent data is actually compatible with the new scene (unlikely to generalize), or (b) extend the hook to also intercept/rewrite the checkpoint and opponent-placement stages (`RaceLoaderTask_ResetStartingLine`, `RaceLoaderTask_LoadRaceFSM`) so they resolve against the substituted track's own data instead of the original event's — the more general, correct fix for subtask 1's "arbitrary track" goal.
## 6l. Debugger attach attempted, blocked by environment; lightweight `Log()` diagnostics confirmed the §6k theory instead
Per user request, tried attaching `lldb` to the live game process on the Pixel 6a for a full memory-map view (the "harder path" alternative to more `Log()` calls), since the device is `userdebug` with `su` available.
**Debugger attempt — inconclusive, blocked by environment, not by anything RE-specific**: pushed a version-matched `lldb-server` (confirmed exact match: both client and server report `lldb version 18.0.1`, identical revision hash `d8003a456d14a3deb8054cdaa529ffbf02d9b262`, client sourced from `$ANDROID_SDK/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/`). `su 0 lldb-server platform --listen '*:1234' --server` started and bound the port cleanly (`netstat` confirmed `LISTEN`), `adb forward` and raw TCP connect both worked. But the gdb-remote handshake never got a reply (`error: failed to get reply to handshake packet`) even with the client-side timeout raised to 90s, and a raw Python socket probe sending a well-formed `$QStartNoAckMode#b0` packet got zero bytes back within 10s on an otherwise-healthy connection. Root cause eventually found: **the device's screen had locked (fingerprint lock screen) partway through the session**, and while locked, `adb shell`/`su` round-trips degraded unpredictably (34ms one moment, 80–300s+ hangs the next, tracked via `time adb shell echo`) — almost certainly Android's Doze/screen-off throttling affecting the `su`-elevated shell and/or `lldb-server`'s connection-handling thread. After the user physically unlocked the device, `adb shell` responsiveness returned to consistent sub-100ms — but by then the debugger session itself needed re-establishing from scratch, and rather than keep re-attempting, pivoted to the lighter-weight `Log()` approach (see below) at the user's direction, since it doesn't depend on a fragile interactive session. **Not marked as "doesn't work"** — the version-matching and forward/listen mechanics are confirmed correct; a retry after ensuring the device stays unlocked/awake (e.g. `adb shell svc power stayon true` or disabling screen timeout first) would very plausibly succeed. Worth revisiting if `Log()`-based diagnostics hit their limit.
**`Log()`-based diagnostic (the approach actually used this round)**: added temporary diagnostic logging to `Hook_BuildTrackScenePath` (`launcher/mpcore/src/main/cpp/main.cpp`) dumping, *before* any override is applied:
- The original track name (to confirm which real event/track was hit).
- `raceDef+40/44` — a second `{begin,end}` string field `sub_2A9338` (`RaceLoaderTask_LoadRaceFSM`, stage 7) reads and compares against itself (`!=` — turned out to just be an "is this non-empty" check, not an equality-against-another-field check as originally guessed in the stage-7 decompile).
- `*(raceDef+12)` — a **nested struct pointer** `sub_2A9338` dereferences, then reads *that* struct's `+48/+52` as the actual name substituted into `"/published/prefabs/racefsms/{0}.prefabs.sb"`.
Live result on the real "Петерсон стрит" event (before override, i.e. ground truth):
```
diag: original track name = 'region1_foothills_track4' (len=24)
diag: raceDef+40/44 field = '' (len=0, begin=0xd5670a0c)
diag: racefsm name = 'point_to_point_fsm_newintro' (len=27)
```
**This is a meaningful finding**: the race-FSM prefab (`point_to_point_fsm_newintro`) is a **generic, reusable race-type template** — "point to point" checkpoint-race rules, not a per-track asset — confirming the racefsms layer itself isn't what breaks when the *track* is swapped. This refines (without contradicting) the §6k working theory: the crash isn't from loading the *wrong FSM* (the FSM is track-agnostic by design), it's from the **generic FSM's checkpoint/actor lookups failing against the substituted scene** — `point_to_point_fsm_newintro` presumably walks checkpoint-tagged actors by name/count in whatever scene is currently loaded, and `region3_colorado_track2`'s actual checkpoint layout doesn't satisfy whatever this FSM variant (note the `_newintro` suffix — possibly implies an expected intro-camera/cutscene actor too) expects.
A quick `strings`-based comparison of `region1_foothills_track4.scene.sb` vs `region3_colorado_track2.scene.sb` for checkpoint-related tokens (`Checkpoint`, `start`, `finish`) found identical **type/prefab names** in both — but `strings` can't distinguish instance counts or IDs within the packed SB `DataIdsMap` structure, so this doesn't confirm or rule out a checkpoint-count/tag mismatch. A real answer needs either the full `NFSMW12MobileTools`-based SB unpack-and-diff (fast, no binary RE) or the debugger. Done immediately after, below — **root cause now confirmed directly**.
## 6m. Root cause confirmed: checkpoint-container name AND count both mismatch between the original event and the substituted track
Per user request, unpacked both `.scene.sb` files via `NFSMW12MobileTools.jar unpack` (jar at `NFSMW12MobileTools/NFSMW12MobileTools.jar`, requires `HCStructFileArray.json` copied into the working directory) and diffed their checkpoint actor structure directly — no debugger needed, this closes the investigation with hard evidence.
**Tool note**: `region3_colorado_track2.scene.sb` (the exact file we hook to) crashes the unpacker's full DATA-object parser (`NegativeArraySizeException` in `SBin.getCleanElementHex`/`parseDATABlock` — an unrelated pre-existing bug in the community tool, not something we introduced). Worked around two ways: (a) `-disableDATAObjectsUnpack` still parses the CDAT string table (`CDAT_Strings`) even though it skips structured per-object field parsing, which was sufficient here; (b) as a cross-check, `region3_colorado_track1.scene.sb` (a sibling colorado track, same region) unpacks fully with no flags and confirms the same pattern independently.
**`region1_foothills_track4.scene.sb`** (the *real* track behind the "Петерсон стрит" event we've been testing against, confirmed via the live `diag: original track name` log in §6l) — full unpack, walked the top-level `04000000` actors `DataIdsMap` (11 entries: `checkpoints_timetrial_event_2`, `end_of_track`, `environment_sound`, `event_02_finish`, `finish`, `foothills_export_group`, `roadblock_level_02`, `root`, `skydome`, `start`, `track_info`), then the checkpoint container's own `children` map:
- **Container name: `checkpoints_timetrial_event_2`** — note the `_event_2` suffix, tying it to a *specific numbered event instance*, not a generic per-track asset.
- **6 checkpoints**: `timetrialcheckpoint`, `timetrialcheckpoint_2` … `timetrialcheckpoint_6`.
**`region3_colorado_track2.scene.sb`** (our override target) — via the `CDAT_Strings` table (all unique strings referenced anywhere in the file, present even with `-disableDATAObjectsUnpack`):
- **Container name: `timetrial_checkpoints`** — generic, no event-number suffix, structurally different name than the foothills track's container.
- **8 checkpoints**: `timetrialcheckpoint`, `timetrialcheckpoint_2` … `timetrialcheckpoint_8`.
Cross-checked against `region3_colorado_track1.scene.sb` (full parse succeeded): same generic container name (`timetrial_checkpoints`, no event suffix) and a *different* checkpoint count again (7) — confirming this naming convention (generic container, no event-number tie-in) is consistent across the colorado region, and that per-track checkpoint counts vary freely, not fixed at 6.
**This is the confirmed root cause of the §6k SIGSEGV**: `RaceDefinition` (or whatever populates the checkpoint-lookup path reached via `RaceLoaderTask_LoadRaceFSM`'s generic `point_to_point_fsm_newintro` FSM) looks for a checkpoint container tied to the **original** event — by name (`checkpoints_timetrial_event_2`) and/or by an expected count of **6** — inside whatever scene is currently loaded. Our hook substitutes the scene with `region3_colorado_track2`, which has **no actor named `checkpoints_timetrial_event_2`at all** (its container is `timetrial_checkpoints`) and has **8**, not 6, checkpoints. The lookup fails, returns NULL, and a background thread walking the (non-existent) 6th/7th checkpoint or the missing named container dereferences it unchecked — matching the tombstone's `r0=NULL` / `fault addr 0x38` exactly.
**Practical implication for subtask 1 (arbitrary track loading)**: a clean, general "load any track" hook needs to do more than swap the track/environment name strings (§6k's approach). Two viable directions, neither implemented yet:
- **(a) Track-compatible substitution**: only offer tracks whose checkpoint container is named to match what the current event expects (i.e. rename/alias at the hook level: also patch whatever field the FSM uses to look up the container name, pointing it at the substituted track's *actual* container name — here `timetrial_checkpoints` instead of `checkpoints_timetrial_event_2` — this is a bounded, mechanical fix: one more string-field override, same pattern as the track/environment overrides already working).
- **(b) Generic N-checkpoint handling**: confirm whether the FSM's checkpoint walk is truly hardcoded to a specific expected count (would need to also override wherever that count is read from) or dynamically discovers however many checkpoints exist under whatever container name it's given (more likely, given `timetrialcheckpoint_N` numbering is open-ended per track) — if the latter, fixing (a) alone might be sufficient for a full solution.
**Recommended next step**: extend `Hook_BuildTrackScenePath` (or add a second hook nearer `RaceLoaderTask_LoadRaceFSM`, `0x2a9338`) to also override the checkpoint-container-name field with the substituted track's real container name (`timetrial_checkpoints` for any colorado track, but this will differ by region/track — needs to be read from the target `.scene.sb` or hardcoded per supported track). This is the same low-risk "repoint a `{begin,end}` string pair" pattern already proven working for track/environment names in §6j/§6k — just needs the right field offset identified (not yet located; `RaceLoaderTask_LoadRaceFSM`'s decompile in §6k shows the *racefsm name* field at `*(raceDef+12)+48/52`, but not yet which field feeds the checkpoint-container lookup specifically — likely a sibling field on that same nested struct, worth checking first).
## 6n. `region3`/colorado is cut/incomplete content — switched hook target to `region4_chicago_track4`; exposed the *real* bug (missing null check in `RaceLoaderTask_ResetStartingLine`)
Continued the §6m whack-a-mole (patching each newly-discovered null-deref with a defensive ARM-mode entry-hook/trampoline guard — `sub_53A5FC`, `sub_52A9B8`, `sub_52A620`) until it led somewhere conclusive rather than indefinitely: `sub_58E5E8`, a recursive spatial-index/BVH builder, crashing on a NULL array pointer at `a1+152` — traced its allocation (`sub_58E2E8`) back to a per-item count populated by iterating actor bounding-boxes (`sub_58B65C`), which only reaches zero/absent if the substituted scene's geometry never actually loaded.
**Checked directly**: `game_cache/published/models/environments/` contains folders for `chicago`, `desert`, `foothills`, `garage`, `newyork` — **no `colorado` folder at all**, even though `region3_colorado_track1/2/3.scene.sb` and a single un-numbered `colorado.prefabs.sb` exist under `prefabs/`. Every other region ships **numbered per-track** environment prefabs (`foothills1..6`, `desert1..6`, `chicago1..6`, `newyork1..6`); region3/colorado has only the one combined file, no `colorado1..6`. Unpacked `region3_colorado_track2.scene.sb` and confirmed it references `"published/models/environments/colorado/region3_colorado_track2.m3g"` as a loose external file — which was never shipped for the mobile release. **Conclusion: region3/colorado is unfinished/cut content** — its track-layout and prop data survived in the package, but its environment model was never exported as a loadable asset. No amount of downstream null-guarding can fix this; the geometry genuinely cannot load.
**Action taken**: reverted all 4 defensive null-guard patches (`sub_53A5FC`/`sub_52A9B8`/`sub_52A620`, the `sub_58E5E8` chain was never patched, investigation stopped there) and switched `Hook_BuildTrackScenePath`'s override target to `region4_chicago_track4` (confirmed shipped and playable — ties to `event_01_race.prefabs.sb`'s real `TrackName`, verified `start`/`finish` actors, per §6). First rebuild used `kOverrideEnvName = "chicago"` — wrong: hit `Could not open database at published/prefabs/environments/chicago.prefabs.sb` (immediate SIGSEGV), because — like all other regions — environment prefabs are the numbered per-track files, not a generic `<region>.prefabs.sb` (that pattern is unique to unfinished colorado). Fixed to `"chicago4"` (matching `region4_chicago_track4`) — geometry then loaded cleanly (no more "not found"/"could not open" warnings in logcat).
With real geometry loading, hit a **new, different, and much more informative** crash: `fault addr 0x14`, `r0=r1=r2=r3=0`, inside `sub_870E8` — decompiled cleanly as a textbook ECS `GetComponent<im::app::race::Checkpoint>(entity)` helper (iterate a `Component*` range on `entity`, `dynamic_cast` each to `Checkpoint`, return the first match). Crash is the very first field read (`entity[5]`, i.e. `entity+20 = 0x14`, matching the fault address exactly) — `entity` itself is NULL. Its only 3 callers are all inside the *already-named* `RaceLoaderTask_ResetStartingLine` (`0x2abe08`, named in an earlier session from its `"m_StartLine"` string reference) — which looks up an actor by name via `sub_672D64(&result, scene, raceDef+164/180/196/212)` for `m_StartLine`/`m_FinishLine`/`m_EndOfTrack`/a 4th field, then calls `GetComponent<Checkpoint>` on the result **with no null check**, unlike every other access in that same function (which all consistently assert "Dereferencing a NULL component pointer" first). This is a genuine, pre-existing engine bug that the original developers never had reason to hit, because every shipped event's named actors always existed in its own paired scene.
Added temporary diagnostics dumping `raceDef+164/180/196/212` as `{begin,end}` string pairs (same layout as the track/env name fields) and reproduced live on "Петерсон стрит" (still targeting `region4_chicago_track4`/`chicago4`):
```
raceDef+164 name = 'start'
raceDef+180 name = 'event_02_finish'
raceDef+196 name = 'end_of_track'
raceDef+212 name = 'checkpoints_timetrial_event_2'
```
**This is the true, final root cause, and it fully subsumes §6m's checkpoint-count theory**: `m_StartLine`/`m_EndOfTrack` are generic names, present in effectively any track's scene — those lookups succeed regardless of substitution. But `m_FinishLine` and the checkpoint-container field hold **per-event custom actor names** (`event_02_finish`, `checkpoints_timetrial_event_2` — the literal event-2/timetrial identifiers, tied to the *original* "Петерсон стрит" event's own data, not to any track/scene convention). No substituted scene will ever have an actor named `event_02_finish` unless it happens to be the exact original track. This is why §6k/§6m's whack-a-mole never converged: the underlying issue isn't a fixable data mismatch, it's that **time-trial/checkpoint-style events are inherently incompatible with simple track-name substitution** — their `RaceDefinition` hard-references scene-specific actor names that only the original track satisfies.
**Confirmed by direct test**: switched to a different event of the *regular point-to-point race* type — "Побудка" (Macklein street, class "Каждый день", original track `region5_newyork_track2`). Diagnostics on this event show only generic names:
```
raceDef+164 name = 'start'
raceDef+180 name = 'finish'
raceDef+196 name = 'end_of_track'
raceDef+212 name = '' (empty)
```
No custom suffixes anywhere. Rebuilt/reinstalled with the reverted (no defensive patches) build, played this event with `region4_chicago_track4`/`chicago4` substituted in — **the race loaded and ran successfully**: live gameplay on chicago4's geometry, correct HUD (position 6/6, timer), AI opponents present, no crash, sustained over multiple seconds. **The `BuildTrackScenePath` hook mechanism is fully validated end-to-end for regular races.**
**Practical implication for subtask 1**: the simple track/environment-name override (as implemented) is sufficient and correct for **regular point-to-point races**. Time-trial/checkpoint events need one of: (a) also overriding `raceDef+180`/`+212` to generic names when the target scene doesn't have the original's custom-named actors (risks silently changing the event's intended finish-line/checkpoint layout), or (b) restricting arbitrary-track substitution to regular-race event types only and leaving time-trial events pinned to their original track. Not decided/implemented yet — a scope decision, not a bug to patch.
## 6o. Scope decided (regular races only); cleaned up to the minimal working hook; visually confirmed with a baseline A/B comparison
Following §6n's finding that time-trial/checkpoint events are structurally incompatible with simple track-name substitution (their `RaceDefinition` hard-references per-event custom actor names), the user made the scope call directly: **the track-substitution hook only needs to support regular point-to-point races.** Time-trial and single-opponent/pursuit-style races are explicitly out of scope — their crashes are expected-unsupported, not bugs to chase. Recorded as a standing project-memory fact (`track-substitution-scope`) so future sessions don't re-litigate or re-chase this.
**Cleanup**: removed all temporary investigation code from `Hook_BuildTrackScenePath` in `launcher/mpcore/src/main/cpp/main.cpp` — the `raceDef+40/44`, `+12` nested-struct, and `+164/180/196/212` diagnostic `Log()` dumps (their job was done once §6n's findings were captured). The 4 defensive null-guard patches from the §6n whack-a-mole (`sub_53A5FC`/`sub_52A9B8`/`sub_52A620`, plus the abandoned `sub_58E5E8` investigation) were already reverted in §6n itself. **Current `main.cpp` state**: `JNI_OnLoad` installs exactly one hook, `Hook_BuildTrackScenePath`, which does nothing but repoint the track-name and environment-name `{begin,end}` string pairs on `raceDef` (`+72/76` and `+100/104`) to `kOverrideTrackName = "region4_chicago_track4"` / `kOverrideEnvName = "chicago4"`. No diagnostics, no defensive guards, no dead code.
**Visual A/B verification** (requested by the user — logcat text isn't proof the geometry itself changed, only that the string pointers were overwritten): built and ran two variants of the identical event ("Побудка", a regular race at Macklein):
- **Hook disabled** (temporarily commented out the `InstallBuildTrackScenePathHook()` call in `JNI_OnLoad`, rebuilt, reinstalled): loads the real, original `region5_newyork_track2` — a nighttime downtown street, a "HOTEL" building, road signs reading "Franklin Plaza / Rochelle Hall" and "Emerson Greenway", a distinct guardrail/road style.
- **Hook enabled** (re-reverted the comment-out, rebuilt, reinstalled): loads `region4_chicago_track4` — a daytime highway/overpass, road signs reading "South 92", "McClane", "Ripley's Point", visibly different geometry, lighting, and time-of-day.
Different time-of-day, different road geometry, different signage, different buildings — conclusive, non-coincidental visual proof the same event now genuinely renders different track geometry depending solely on whether the hook is installed. **Subtask 1 (arbitrary track loading) is now considered validated end-to-end for regular races** — both mechanically (hook installs, fires, no crash, sustained multi-minute gameplay) and visually (real geometry swap, not just a label change).
## 6p. Street/POI event lists are NOT simple SB data; found the engine's file-open chokepoint instead (validates a memory-only patching strategy)
Two separate investigations, prompted by the user's question about adding a virtual "LAN: <lobby>" entry to an existing street's event list, and the follow-up theory that SB files could be edited entirely in-memory from `mpcore` without touching `game_cache`/the OBB.
**1. Where street/POI groupings come from — not found in data yet.** Unpacked all 62 `game_cache/published/data/races/event_*.prefabs.sb` files (`NFSMW12MobileTools`, full struct parse, no workaround flags needed — small files, none hit the `NegativeArraySizeException` bug). Each contains one `RaceEvent` struct with a `Location` field — but across all 62 events, `Location` only takes **4 values**: `Chicago`, `Foothills`, `NewYork`, `Desert` (the coarse region, matching the environment). It is **not** the street-level label shown on the map (`МАККЛЕЙН`/`ПЕТЕРСОН СТРИТ`). `Name` is a localization key (`EVENT_NAME_2`, etc.), not a literal string either. Also unpacked `flow/menus/map_overworld.sb` (the map screen's Flow-machine script) — it contains only generic UI screen/transition wiring (`output`/`node`/`Transition` triples for buttons like `EVENT`, `GARAGE`, `STORE`) and a `layout: map_overworld` reference; no event-ID lists, no street names, no coordinates. **Conclusion: the street/POI clustering that groups nearby events under a named map pin is not stored as simple, easily-editable SB data anywhere checked so far** — it's most likely computed natively in C++ at runtime (e.g. proximity-clustering each `RaceEvent`'s track start-position against a small set of named zone boundaries), which would need further binary RE (not attempted yet) to locate precisely, not just an SB edit. This directly answers the original "street event" question: adding a synthetic entry isn't a quick data-only change; it needs a native hook once the clustering code is found.
**2. The memory-only-patching theory — confirmed correct, and the hook point is now identified.** Traced how `libapp.so` actually opens files on disk, starting from the `RaceEvent`-parsing code's generic string-keyed property accessors (`sub_4F9A80` etc., confirming SB files are parsed into an in-memory key-value property bag once, then read by name everywhere — not re-parsed per access) down to the real I/O layer:
```
(resource-loading code, not yet fully enumerated)
│
▼
sub_8604A0(a1, path, a3) "load whole file into buffer" helper
│ calls sub_8598A4(path, "rb")
▼
sub_8598A4(path, mode) -> FILE* only 4 callers total in the whole binary
│ calls j_fopen(path, mode)
▼
j_fopen (0x89d350) thin thunk to libc fopen(), only 4 callers
│
▼
libc fopen()
```
`sub_8598A4` (`0x8598a4`) is the practical hook point: a small, direct `(const char* path, const char* mode) -> FILE*` function with exactly 4 call sites in the entire binary (`sub_8604A0` "read whole file" helper, plus 3 others not yet inspected: `sub_8BBD94`, `sub_8D6D9C`, `sub_8FC64C`). This is far more general-purpose than the track/environment-name hook — it's the choke point for **every file the engine opens by path**, not just track scenes.
**What this enables**: hook `sub_8598A4`, check the incoming `path` against a list of virtual filenames we care about (e.g. `"published/data/races/event_02_timetrial.prefabs.sb"`, or an entirely new, game-never-shipped filename), and if it matches, return a `FILE*` from `fmemopen()`/`open_memstream()` backed by a buffer **we** control (a hand-edited copy of the original SB bytes, or a fully synthetic one) instead of calling through to the real `fopen`. For every other path, call through unmodified (pass-through, matching this project's "no-op when not relevant" hook principle). This achieves genuinely disk/cache-untouched patching — no `native_lib`/`game_cache` file is ever modified, the substitution happens purely in `mpcore`'s memory at load time, and it works for **any** SB file the engine reads (race definitions, flow scripts, checkpoints, etc.), not just the track path fields the current `BuildTrackScenePath` hook touches.
**Not yet done**: actually implementing/testing this hook (would need a `mmap`/trampoline install like the existing `BuildTrackScenePath` hook, since `sub_8598A4` is a plain ARM function — first-instruction check not yet done), confirming the mode-string comparison approach works for text vs. binary opens, and inspecting the 3 other callers (`sub_8BBD94`/`sub_8D6D9C`/`sub_8FC64C`) to rule out anything env-specific. This is a substantial, foundational new capability (general asset override, not just track substitution) — worth a deliberate go/no-go and its own test cycle before implementing, rather than folding into the existing hook.
## 6q. `sub_8598A4` hook implemented and tested — mechanism proven, but it's the wrong chokepoint for game assets (corrects §6p)
Implemented the hook proposed in §6p as a proof of concept: trampoline-hooked `sub_8598A4` (`libapp_base + 0x8598a4`, same PUSH/SUB-relocation pattern as `BuildTrackScenePath`), logging every `(path, mode)` pair and substituting an `fmemopen()`-backed buffer for any path containing `"event_02_timetrial"`. The substitute payload: `event_02_timetrial.prefabs.sb` unpacked via `NFSMW12MobileTools`, `CashReward.Gold/Silver/Bronze` edited `10500/8500/7000` → `99999/88888/77777`, repacked to a valid `.sb`, embedded as a C byte array (`mpcore/src/main/cpp/test_event_02_data.h`). Gated behind its own toggle flag (`kEnableFileOpenHook`, same pattern as §6o's `kEnableTrackSubstitutionHook`), on a new branch (`file-open-hook-poc`, based on `track-hook-toggle-flag` — `master` in this repo is a stale, unrelated baseline predating all of this work, confirmed this session, not touched).
**Mechanism confirmed working**: the hook installs cleanly and genuinely intercepts real engine `fopen()` calls — logcat shows `sub_8598A4 hook: fopen('/home/ogami/output-arm/openssl.cnf', 'rb')` firing during app startup (a build-machine-path leak from EA's OpenSSL config, harmless but proves real interception).
**Wrong function for game assets**: navigated through several screens that definitely read `.sb` data (map, event list, the `event_02_timetrial`-backed "Петерсон стрит" → "На время" event card, which still showed the original `10 500$/8 500$/7 000$` unchanged) — **`sub_8598A4` was never called again after the one OpenSSL open at startup.** Checked the other 2 named callers of the shared `j_fopen` thunk that §6p's trace was built on: `sub_859528` is a generic stream-command dispatcher (seek/tell/flush/close opcodes, not a general per-asset opener) and `sub_8AC1F4` is a **file-hashing utility** (reads in 1KB chunks through an accumulator function, `sub_8AC10C` — looks like a checksum/integrity-check pass over a whole file, not the resource loader). None of `j_fopen`'s 4 callers are the actual `published/data/races/*.sb`-reading code path.
**Revised theory**: individual game asset files are most likely **not opened via `fopen()`/`j_fopen()` at all**. The likelier design (common for mobile games, avoids per-asset syscall overhead): the whole OBB/asset bundle is opened/mapped **once** at startup (candidate: the `sub_3FB6E4`/"Mounting SKU" registration path from §6c's exploration, or a raw `open()`+`mmap()` pair — `open` does have 11 callers in this binary, not yet checked), and individual "files" like `event_02_timetrial.prefabs.sb` are served by looking up a name in an in-memory index/directory and returning a `pointer+length` slice **into that already-mapped memory** — no repeated per-file `fopen`. If true, this is actually a *better* hook target than a fake `FILE*`: intercepting after the name→pointer lookup would let a substitute just be a raw buffer swap, no `fmemopen` needed.
**Status**: hook mechanism (trampoline install, path matching, in-memory substitute payload, toggle flag) is proven and reusable — only the *target function* was wrong. Not reverted (harmless with `kEnableFileOpenHook` currently pointed at the wrong function, effectively a no-op for game data since `sub_8598A4` is never called with a matching path) — left in place on the `file-open-hook-poc` branch as a ready-to-repoint scaffold. **Not yet done**: trace the `sub_3FB6E4` mount-table path, or the 11 `open()` call sites, to find the real name→data resolution function.
## 6r. Found `VFS::OpenInputStream` (the real universal file-resolver) — then pivoted per user direction: runtime object injection, not file substitution
**Continuation of §6q's search for the real path→data resolver.** Traced up from `sub_208C88` (`RaceEvent`-loading function, confirmed via its `"/published/data/races/"` string reference and by producing the same `raceDef` struct the `BuildTrackScenePath`/`ResetStartingLine` hooks already use) through `sub_6753FC` → `sub_4F0138`, which calls a **virtual method at vtable offset+8** on a lazily-constructed singleton (`sub_40E8E8()`, whose class vtable is `off_AB2084`, explicitly confirmed as the engine's `VFS` class via the literal string `"VFS::AddVariant("` in its constructor `sub_40E9F4`). Read the vtable bytes directly (`get_bytes` at `0xAB2084`) and resolved slot+8 to **`sub_410808`**, which decompiles to exactly `VFS::OpenInputStream` (confirmed via its own literal strings: `"openInputStream \""`, `" .. returning variant \""`, `"\" mapping to fs path \""`) — and, decisively, `sub_4F0138` prints the exact literal `"Could not open database at "` when this returns null, **the identical error text observed live in §6n** when the `chicago` vs `chicago4` environment-name mistake was made. This conclusively identifies `sub_410808` as the true, universal, per-path file resolver used for every `published/...` asset in the game (not `fopen`-based at all - §6q's `sub_8598A4` hook was chasing a red herring; individual SB files are resolved via this VFS virtual-path→real-fs-path mapping, then handed to a per-backend "open real file" call, not raw libc `fopen`). First 2 instructions (`PUSH {R4-R11,LR}`; `ADD R11,SP,#0x1C`) are the same hookable, position-independent shape as every other hook this session.
**Not pursued further as a hook target**, per explicit user redirection: `sub_410808` returns a refcounted C++ stream *interface* object (`AddRef`/`Release`-style calls visible on it in `sub_4F0138`), not a raw buffer/`FILE*` - faking one correctly would need reverse-engineering its exact vtable contract, a nontrivial and crash-prone undertaking. More importantly, the user clarified the actual goal isn't "swap file content before the game reads it" at all - it's **runtime injection into already-loaded/parsed game objects** (e.g. the map loads normally, then a hook adds a synthetic "LAN: <lobby>" entry into an already-populated street's event list) - the same category of technique already proven working for `RaceDefinition` (`raceDef+72/76` field repointing in `BuildTrackScenePath`), just applied to a different, later point in the pipeline. `sub_410808`'s file-open-time hook doesn't serve that goal even if fully implemented.
**New lead for the actual goal**: searching RTTI for the map/street screen surfaced **`im::app::flow::nfs::MapScreen`** (the map screen's controller class - sibling of the already-known `im::app::flow::nfs::PostRaceMetagameScreen`/`GarageScreen`/`CarUnlockScreen`/`CongratsScreen` family) and, in one of its method signatures, **`boost::shared_ptr<im::app::ui::MapTrack>`** - strongly suggesting `MapTrack` is the per-event/per-marker runtime object the map screen holds one of per visible event. **Not yet done**: find where `MapScreen` builds/holds its `MapTrack` collection (constructor or an `UpdateXxx`/`Populate`-style method), and `MapTrack`'s field layout (display name, reward, target event-ID/track reference) - once both are known, the established "read/repoint fields on an already-loaded C++ object" technique (same as every hook this session) can inject a synthetic entry after the real collection is populated, exactly matching the user's actual request. Checked `career.prefabs.sb` (the one non-numbered file under `data/careers/`) as a possible data-driven source for street groupings first - it only holds progression tiers (`tier_1`..`tier_55`-style `TierItem`s) and unlockable car lists, **not** street/location data, ruling it out.
## 6s. Runtime injection into a street's event list — found the real classes and the setup function, not yet the exact "add card" call
Continuation of §6p, redirected by explicit user feedback: the user does **not** want a disk-touching approach (writing a substitute file to app-private storage, or faking a VFS stream) — they want the street's event list modified **in memory, at runtime, after the map has already loaded**, the same way a real multiplayer client would announce a discovered LAN lobby as an extra, synthetic event card under an existing street pin (e.g. `МАККЛЕЙН` → `LAN: <lobby name>`), without touching `game_cache`/the OBB/`native_lib` at all. This reframes the goal from "override what a file contains" to "hook the C++ code that turns loaded `RaceEvent`s into UI, and inject one extra fake entry into its output."
**Confirmed classes (RTTI-verified)**:
- `im::app::flow::nfs::MapScreen` — the world-map screen controller. Its main setup function is `sub_1781BC` (`0x1781bc`, **~8.7KB**, by far the largest function found in this project so far) — too large to fully decompile through the MCP tool in one call (response gets truncated); explored via targeted disassembly windows instead.
- `im::app::ui::MapTrack` — one visual pin/marker widget on the map (one instance per street shown, e.g. "МАККЛЕЙН", "ПЕТЕРСОН СТРИТ"). Has a `TrackId`/`TrackName`-keyed property read from the layout (confirmed via the literal error string `"Couldn't find TrackId property on MapTrack widget "` and a live `"TrackName"` property-name string used inside `sub_1781BC`).
- `im::app::ui::MapTrackEventList` — the list-of-event-cards widget. Looked up by a **fixed, singular widget name: `"event_list"`** (not one list per street — confirmed via `sub_17A4CC`, a generic `FindOrCreateLayoutEntity<T>(scene, name)` helper reused for several named widgets, called with the literal string `"event_list"` at `0x178308`-`0x178318`). A sibling `"map_scroll"` widget is looked up the same way immediately after. This means the event list is **one shared widget that gets repopulated each time the player selects a different street pin**, not N separate always-populated lists — matches the observed UI flow (map → tap pin → event cards appear).
- `sub_1781BC` contains exactly 2 `dynamic_cast<Node → MapTrack>` calls (`0x179034`, `0x179af4`), each inside a loop over the scene's child nodes filtering for `MapTrack` instances — i.e. two separate passes over "every pin on the map" (likely: one pass to wire up click handlers via the `boost::bind`-based `MapScreen::OnTrackClicked(shared_ptr<MapTrack> const&)` binding also found in this function's `.data.rel.ro` references, and a second pass for something else not yet identified - badge/lock-state refresh is a plausible guess, not confirmed).
**Not yet found**: the specific call, somewhere in the ~7KB of `sub_1781BC` not yet inspected, that iterates loaded `RaceEvent`s, matches each one's `TrackName` field against a `MapTrack` pin's `TrackId`, and adds a corresponding child card into the `"event_list"` widget. This is the actual hook point needed - either to call the *same* function ourselves with a synthetic/fake `RaceEvent`-shaped argument (reusing the engine's own card-construction logic, much lower-risk than hand-building a UI node), or to hook whatever lower-level "add child to `MapTrackEventList`" method it ultimately calls.
**Why static disassembly stalled here**: `sub_1781BC` is too large to decompile through the MCP tool in one shot (Hex-Rays output gets cut off around 62K characters before reaching the interesting part), and windowed raw-disassembly reading (as used successfully for smaller functions all session) doesn't scale well to a function this size - each 50-instruction window only covers a tiny fraction, and there's no shortcut like a distinctive string to `search_text` for near the exact call (tried `"TrackName"`/`"TrackId"`/`RaceEvent`/`dynamic_cast` - all found *something* relevant but not the precise add-card call yet).
**Recommended next step (not yet attempted at the time this section was first written)**: given the static-analysis approach is hitting diminishing returns on this specific function, live debugging on-device would likely be faster here. See §6t for the actual attempt.
## 6t. Live debugging: attach/registers/disassembly work via a cross-arch `lldb-server`, but breakpoints do not — every "negative result" below this point is unverified, not evidence
**What's genuinely confirmed**: the ARM32 `lldb-server` failures from earlier the same day (both `lldb` and Frida blocked, see `PROGRESS.md`) traced to a real bug in this NDK's **ARM32** `lldb-server` build - a control experiment (attaching it to a harmless system process, `systemui`) reproduced the identical `GetMaxU64 invalid byte_size!` assertion and unresolvable-PC symptom seen against the game, proving it's generic to that binary, not anti-debug in the game. The **AArch64** `lldb-server` (same NDK) attached cleanly to that same process, and - since Linux/Android's ptrace lets a 64-bit tracer debug a 32-bit compat-mode tracee - **also attaches cleanly to the 32-bit game process itself**: correct PC, correct ARM32 disassembly, correct thread names, no assertion. This part is solid and reusable: for passive inspection (attach, read registers/memory, disassemble at whatever point the process happens to be stopped), use the AArch64 `lldb-server` against this device regardless of the target's own bitness.
**What is NOT confirmed, and was wrongly reported as confirmed in an earlier version of this section**: that setup was used to breakpoint several functions (`sub_1781BC`'s post-widget-lookup point, `sub_17A4CC`, `sub_7D2E8`, `sub_208C88`, `sub_7CE58`) across multiple live sessions, tap a street pin each time, and observe zero hits - which was written up as a real negative result ("the click handler doesn't call X"). **This was premature.** Prompted by the user directly questioning the evidence, a sanity check was run: breakpointing `j_malloc_0` (a function guaranteed to be called continuously - dozens of times per second at minimum) through the exact same setup. **It also never fired, waiting 15 full seconds during active gameplay.** A hardware breakpoint was tried as a fallback and failed outright with an explicit error: `failed to set breakpoint site ... hardware breakpoint resources might be exhausted or unavailable`. Conclusion: **this cross-arch configuration cannot actually insert working breakpoints (software or hardware) at all** - `lldb` prints a success-looking `Breakpoint N: address = 0x...` message regardless, which is only confirming the *address resolves*, not that a trap was successfully placed and will fire. The passive capabilities (attach, register read, disassembly at an already-stopped point) are real; active tracing (breakpoints, and by extension anything depending on them) is not currently usable with this tool/device combination.
**Consequence**: every "X was never called" claim from the breakpoint experiments this session (previously written up as §6t/§6u findings about `sub_1781BC`, `sub_17A4CC`, and `sub_7D2E8`'s callers being load-time-only) is **retracted as unverified** - not necessarily wrong, just not actually tested. The `RaceEvent`-loaded-once-at-startup theory and the "always exactly 3 card slots" UI observation both still stand on their own (the former from static call-site counting in §6s, the latter from direct visual observation of screenshots), but the *live-debugging support* claimed for them does not hold.
**Follow-up: tried a genuine same-architecture `gdb`/`gdbserver` pair, ruled that out too.** Installed `gdb-multiarch` on the host (extracted from the `.deb` without root, via `apt-get download` + `dpkg-deb -x` - no `sudo` available in this environment). It could not talk to the AArch64 `lldb-server` at all (`Invalid hex digit 59` parsing register replies - a genuine wire-protocol incompatibility between LLDB's and GDB's remote-serial-protocol dialects, independent of the cross-arch issue above). Obtained a **real, native ARM32 `gdbserver`** (extracted only the one binary from the legacy `android-ndk-r16b` zip - Google's NDK dropped `gdbserver` after r17 - via a full 852MB download, since partial/range-request extraction against `dl.google.com` failed with SSL/range errors in this environment). Native-architecture attach worked meaningfully better than either `lldb` path: `interrupt` correctly stopped the process and produced a **real, correct backtrace** (`syscall() ← libart.so`, matching genuine ARM32 register/stack state) - clearly better than anything the cross-arch `lldb-server` could do. A software breakpoint was accepted (`Breakpoint 1 at 0xc389bbfc`, on `j_malloc_0`), but issuing `continue` **crashed `gdbserver` itself** (the game process survived unharmed; `gdbserver`'s process simply disappeared, and the GDB client reported `Remote connection closed`) - no breakpoint hit was ever observed. Most likely cause: a ~7-year version skew between this 2017-era `gdbserver` (bundled with a GDB 7.x/8.x-generation NDK) and the 2024 `gdb-multiarch` 15.1 client - `qSupported` handshake warnings (`unrecognized item "timeout"`, `Ignoring packet error`) were visible in earlier connection attempts, confirming real protocol drift between the two ends even where the connection nominally succeeded.
**Overall conclusion across all three attempts (cross-arch `lldb-server`, cross-arch `lldb-server` vs. `gdb-multiarch`, native `gdbserver` vs. `gdb-multiarch`)**: passive inspection (attach, interrupt, read registers/memory/backtrace at whatever point the process happens to be stopped) is achievable and was proven working more than once; **actively resuming execution with a breakpoint installed has not been achieved by any combination tried in this environment.**
**Root cause identified (user research, not further tooling changes needed to explain it)**: the Fairphone 5's SoC has **no native AArch32 (32-bit ARM) hardware support at all** — like an increasing number of recent Qualcomm chips, it runs 32-bit code (this project's `armeabi-v7a`-only `libapp.so` included) through a software translation/compat layer, conceptually similar to Intel's Houdini layer on x86 Chromebooks, rather than real 32-bit silicon. This retroactively explains *every* symptom hit this session in one stroke: there are no genuine AArch32 hardware debug registers underneath for a hardware breakpoint to program (matches the explicit "hardware breakpoint resources might be exhausted or unavailable" error); a software breakpoint's `PTRACE_POKETEXT` patches bytes in what may not be the actual code stream the translator is executing, so it can install without error yet never trigger; and register-context edge cases (the ARM32 `lldb-server`'s VFP/NEON assertion) are unsurprising against a translated/emulated register file. Passive operations (attach, `interrupt`, reading whatever the OS-level compat layer reports as current register/stack state) still worked because those go through the kernel's own ptrace compat translation, which is solid — only the "patch code and expect it to run" class of operation is affected.
**Practical implication**: this isn't a tooling bug to keep working around on this device - live breakpoint-based debugging needs a device whose SoC still has genuine AArch32 hardware. [[reference-pixel6a-grapheneos-testing]] (already available, already proven to run this project's `armeabi-v7a` libs with "no Houdini/binary-translation layer" per its own memory entry) is the most immediately-available candidate to retry this on. The user also plans to bring a Pixel 5a or a 2018 Galaxy A9 specifically for this. Until confirmed working on genuine 32-bit-capable hardware, treat this project's debugging capability as **read-only** (attach-and-inspect-current-state only) and rely on static IDA analysis for anything requiring "does function X get called when Y happens."
## 6u. Static analysis (post-debugging-pivot): decoded the actual click hit-test and per-slot lock refresh inside `sub_1781BC` — real evidence, no debugger needed
Per the user's direction after §6t ("continue with statics for now"), went back into `sub_1781BC` with `func_profile`/`disasm` windows instead of full decompile (which truncates around 62K characters for this function). Note: the plain `callees` tool returned an empty list for this function for unclear reasons (possibly a size-related edge case) — `func_profile` with `include_lists: true` worked correctly and returned all 46 real callees, so prefer that tool for large functions going forward.
**§6s's "exactly 2" call-site count stands, re-confirmed**: `xrefs_to` on the `MapTrack` RTTI typeinfo address (`0xac6434`) surfaced a third reference at `~0x17a02c` inside `sub_1781BC`, but checking it directly showed it's just a reused literal-pool constant (ARM32 `LDR =literal` pool word), not a distinct `dynamic_cast` call — that region (`0x179f54-0x17a118`) is unrelated first-run onboarding logic (checks "have I shown this before" flags and pushes `"TUTORIAL_MAP"`/`"CONGRATULATIONS_POPUP"` messages onto a queue at `MapScreen+0x230`). The two real call sites remain `~0x179034` and `~0x179af4` (analyzed below).
**Region A (`0x178afc`-`0x178e40`): a straight-line "populate selected-event summary" block, not a per-card loop.** Reads five named properties — `TrackName`, `Completion`, `EventName`, `class_restriction`, `event_type` — via the generic property-getter `sub_15F2DC` (four of them) and `sub_15E934` (for `event_type`), each call reading from a single object at `[MapScreen+0x120]` and caching the `{ptr,len}`-style result pair into fixed `MapScreen` fields (`0x1F8/0x1FC`, `0x200/0x204`, `0x208/0x20C`, `0x210/0x214`, `0x218/0x21C` respectively), with the old cached value released (refcounted `Release`-style vtable call) before each overwrite. This reads from one already-resolved object, once, straight-line — it's a details/header panel populated from "whatever is currently selected," not a loop building N cards.
**Region B (`0x179a2c`-`0x179cfc`): the actual touch/click hit-test.** Confirms (now via static evidence, not the retracted live-debugging claims from §6t) that clicking a street pin does **not** reload or re-extract any `RaceEvent` — it's a pure nearest-neighbor search over already-resident objects:
- Gets the current touch/click point from a global singleton (`sub_890EC()+0x9C/0xA0`), stored into `MapScreen+0x13C/0x140`.
- Iterates `MapScreen+0x1C8`'s child array (begin=`+0x5C`, end=`+0x60` — this is the `map_scroll` container's children), `dynamic_cast<MapTrack>` on each (typeinfo `_ZTIN2im3app2ui8MapTrackE` at `0xac6434`, confirmed to only be referenced from 3 places project-wide, see below).
- For each successfully-cast `MapTrack*`, gets its position via `sub_369EEC` and computes a blended squared-distance to the touch point (0.5 mix factor against the pin's extents, via VFP), tracking the minimum seen so far.
- **For the current closest match, copies two fields directly off the `MapTrack` object itself — offsets `+0xB8` and `+0xBC` — into `MapScreen+0x1C0`/`+0x1C4`.** There is no VFS call, no prefab load call, no `sub_7D2E8`/`sub_208C88` call anywhere in this path. This is decisive, statically-verified confirmation that each `MapTrack` pin already carries a pointer (almost certainly to its associated `RaceEvent`-derived data, or a thin wrapper around it) set once when the street/map loads, and a click is nothing more than "find nearest pin, copy its pre-existing pointer into the selection slot."
**Region C (`0x179d30`-`0x179dfc`): per-card-slot "locked" refresh — real evidence for the "fixed slot pool" theory.** Walks a linked list rooted at `MapScreen+0x18C` (sentinel = a stack local holding the list's `end()`); for each list node (one per card slot), if not already resolved (`node+0x10 == 0`), reads the `"locked"` property via `sub_406644` on the slot's associated object (`node+8`), then calls a vtable setter (`vtable+0x5C`) on that object with the locked value, followed by a refresh/invalidate call (`sub_4D364C`) on a sub-object at `node+8 → +0xC0`. This is a fixed-size (or at least pre-existing, non-dynamically-added) collection of slot entities whose lock state gets pushed per-refresh — consistent with, and now backed by real disassembly for, the "always exactly 3 card slots" UI observation from earlier sessions (still not confirmed as *exactly* 3 by count, but the mechanism — toggle pre-existing slots' locked flag rather than add/remove children — is now confirmed).
**Searched the whole binary for other `MapTrack`-related functions**: only 3 functions anywhere reference the `MapTrack` RTTI typeinfo (`0xac6434`) — `sub_1781BC` (this one), `sub_17C120`, and `sub_17FC6C`. Both of the other two are now fully decompiled and characterized:
- `sub_17FC6C` (~1.2KB) — a "scroll/pan the map camera to a named track" helper: resolves a target track name from a small candidate list, finds the matching `MapTrack` child, computes a tween/scroll command into `MapScreen+308..+376`. Almost certainly "auto-scroll to next unlocked event."
- `sub_17C120` (~4.4KB) — `MapScreen`'s **per-frame update/tick handler**, not a one-shot setup function: dispatches touch-down events by `dynamic_cast`-testing the tapped node against `UIButton` (e.g. the `"BLACKMARKET"` button) then `MapTrack`, caching the hit into `MapScreen+0x1A0/0x1A4`; drives an "unlock reveal" animation state machine (`MapScreen+0x134` state values incl. `3`/`1000`) that reads an `"unlocking"` property and calls a setter (`vtable+92`) on the newly-unlocked pin's icon object when a `"NEXT_EVENT"`-flagged pending-unlock list becomes empty; clamps camera scroll bounds; and periodically (every 4s of playtime) fires a QA-only "Soak Test" feature that auto-picks a random `"RACE"` track. None of this writes `MapTrack+0xB8`/`+0xBC` either — it only reads/reacts to already-attached pin state.
**Still not found**: the function that *writes* `MapTrack+0xB8`/`+0xBC` in the first place — i.e., where a `RaceEvent`'s data actually gets attached to a pin when a street/map scene loads. All 3 functions that `dynamic_cast` to `MapTrack` in the whole binary are now accounted for and none of them do this write, so the real population site must construct/populate `MapTrack` instances without ever needing to `dynamic_cast` to their own type (e.g. it already has a statically-typed `MapTrack*` from a factory/constructor call, no RTTI check needed) — the RTTI-xref search approach is exhausted. This is the real target for the user's "inject a synthetic LAN lobby event under an existing street" goal — either construct a fake pointer at that offset from a synthetic `RaceEvent`-shaped struct and write it directly into an already-loaded (but currently `locked`) `MapTrack`'s `+0xB8` field at runtime, or find and re-enter the same population function with synthetic data so the engine's own logic builds it correctly (much lower-risk, matches the general approach already favored in §6s). **Next static-analysis approach should pivot away from RTTI-xref searching** (dead end, confirmed) **toward finding `MapTrack`'s constructor directly** — e.g. via its vtable symbol (should sit near the RTTI typeinfo in `.data.rel.ro`, same pattern used successfully for `RaceLoaderTask` in §6a) and following xrefs to *that*.
## 6v. FOUND IT: `MapTrack::AddEvent` — the exact function that attaches a `RaceEvent` to a street pin, and the concrete injection point for the user's "LAN lobby" goal
Continuation of §6u, abandoning the RTTI-xref dead end in favor of the proven §6a technique: locate a class's real vtable via `entity_query` on `names` near its RTTI typeinfo address, then follow xrefs to *that* instead of to the typeinfo. This worked immediately.
**`MapTrack`'s real vtable**: `_ZTVN2im3app2ui8MapTrackE` at `0xaabfdc` (found in the same `.data.rel.ro` neighborhood as the typeinfo, via `entity_query`). Its usable function-pointer table starts at `off_AABFE4` (standard Itanium ABI: vtable symbol points at the offset-to-top slot; the RTTI pointer and actual vtable-proper follow).
**`MapTrack`'s real constructor**: `sub_368860` (found via `xrefs_to` on the vtable symbol). Sets the vtable pointer, then `memset(this+0xB8, 0, 0x23)` — confirms `+0xB8` (and everything through `+0xDA`) starts **zeroed**, i.e. every `MapTrack` pin is created with no event data attached; it must be populated by a separate step. (`sub_3688D4`, the other vtable-referencing function, is the destructor — releases held smart-pointer members then chains to the base class dtor.)
**`MapTrack`'s factory**: `sub_395B38` (the sole caller of the constructor) — a `make_shared<MapTrack>`-style allocator (raw object + a separate refcounted control block). Its sole caller is `sub_38DA44`, a ~27KB function that turned out to be a dead end: it's a generic class-factory **registration bootstrap** (hundreds of `{"ClassName" string, ctor function pointer}` pairs registered into a lookup table for the data-driven layout/prefab deserializer — the same string-keyed pattern seen for `"MostWantedLeaderboard"` and dozens of other unrelated classes). It doesn't populate anything itself; it just tells the generic layout loader how to construct a `MapTrack` when one is named in a `.sb`/layout resource.
**The real chain, found via the `"TrackId"` error string** (`"Couldn't find TrackId property on MapTrack widget "`, only ever emitted from one function):
1. **`sub_369040` = `MapTrack::RefreshEvents()`** (or equivalent). Reads its own `TrackId` layout property (via `sub_3684B4`); for a small internal collection of name/zone strings (`this+0x224..0x228`), looks each one up in a global registry (`sub_242778`) to get a collection of matching `RaceEvent` handles, and calls `sub_369AB0(this, &eventHandle)` for **every** match. It then also drives all of the pin's visual state — `"locked"`/`"available"`/`"new"`/`"blacklist"` property setters, a `"blacklist_icon"` visibility toggle, and (critically) a **completion-percentage badge**: `(this+0x276) / (this+0x272) * 100`, formatted and pushed via `sub_15F2DC(this+0x184, "completion")` — the same generic property-setter used for `EventName`/`TrackName` etc. in `sub_1781BC` (§6u Region A).
2. **`sub_369AB0` = `MapTrack::AddEvent(RaceEventHandle*)`** — has exactly one caller (`sub_369040`'s loop), confirming it's a dedicated, single-purpose method. For the given event handle it: `push_back`s it into a growable vector living directly on the `MapTrack` object (`this+0x240` begin / `+0x244` end / `+0x248` capacity — a real `std::vector`-style growth pattern, confirmed via the `sub_3DE038`-realloc-then-`memmove` sequence when full) — **so a street pin can and does hold *multiple* events, this is not a single-event field**; increments `this+0x272` by 3 (3 medals — Gold/Silver/Bronze — per event) and `this+0x276` by however many medals the player has actually earned for it (via `sub_4F0C1C`/`sub_4F9A80`, reading a `"medal"` field off some save/progress lookup) — feeding directly into `RefreshEvents`'s completion-percentage badge; and updates `this+0x216/0x217/0x218/0x220` lock/availability/"new" flags based on the same save-data lookup.
3. **`sub_368DFC` = `MapTrack::HandleEvent(eventTypeId, im::Event*)`** — `MapTrack`'s override of the engine's generic message-handler virtual method. On event type `1025`, `dynamic_cast`s the event to `im::app::events::FlowSetLayoutScreenEvent` ("the screen this pin lives in just finished its layout pass") and, if it matches, calls `RefreshEvents()` on itself. On event type `1048` (`UIButtonClickEvent`) matching its own embedded button component, it invokes a stored `boost::function` callback (`this+0x256`) — a **second, independent click-handling path** from the raw touch-coordinate hit-test found in `sub_1781BC` §6u Region B (this one is UI-focus/button-event-driven, not distance-based).
**This settles the multi-session architecture question**: population is **bottom-up and per-pin, not top-down**. There is no single "MapScreen iterates all RaceEvents and assigns them to pins" function to find, because that isn't how it works — each `MapTrack` pin, upon receiving its own `FlowSetLayoutScreenEvent`, independently looks up and self-registers whichever `RaceEvent`s match its own `TrackId` from a shared registry.
**Concrete implementation path for the user's actual goal** (inject a synthetic `"LAN: <lobby name>"` entry into an already-loaded street's card list, purely at runtime, no file/OBB/`game_cache` changes): hook `MapTrack::AddEvent` (`sub_369AB0`, `0x369AB0`) and, after the real `RefreshEvents` loop finishes populating a chosen pin (or by directly calling `AddEvent` again on an already-loaded `MapTrack*` at any later time, e.g. when a LAN lobby is discovered), call it a second time with a synthetic event handle. Because `AddEvent` already does all the real bookkeeping — vector growth, completion-percentage accounting, lock/availability flags — this reuses the exact same engine machinery real events go through, matching the general "call the engine's own function with fake data" strategy favored since §6s (far lower-risk than hand-building a UI node or a fake `MapTrackEventList` child directly).
**Not yet determined**: the exact shape/type of the `RaceEventHandle` argument `AddEvent` expects (`a2` in the decompile) — it's passed by pointer and resolved through what looks like this engine's entity-component-system indirection (`sub_7566C`/`sub_173350`-style "resolve a component from a handle" calls, matching the "Dereferencing a NULL component pointer" ECS error strings seen elsewhere in the binary, §6u), not a raw `RaceEvent*`. Determining the exact handle format (likely a small ID/generation-pair struct rather than a pointer) is the next concrete step before this can actually be called with synthetic data.
## 6w. `AddEvent`'s handle format decoded: a 32-bit FNV-1a hash keying a global resource cache — `AddEvent` doesn't need the TrackId registry at all
Continuation of §6v. Decompiled the two functions in the resolution chain to pin down exactly what `MapTrack::AddEvent`'s `RaceEventHandle` argument actually is.
**`sub_173350`** (called from `AddEvent`/`sub_369AB0` as `sub_173350(&out, context+320, &handle)`) is a **generic "resolve an ID into a cached, refcounted resource" lookup** against an intrusive hash map: buckets at `context+4`/count at `context+8` (relative to the `context` pointer it's given), bucket index computed via `sub_97D23C(hashKey, bucketCount)`, each node laid out as `[0]=key(int), ..., [3]=+12, [4]=+16, [5]=+20 refcounted-ptr, [6]=+24 next-in-chain`. On a hit it returns the `{+12, +16, +20}` triple (incrementing the refcount at `+20+8`) — this is **the same generic "prefab instance cache keyed by ID, refcounted" system already known from `sub_7CE58`** (§6s: "Failed to load prefab %s... already present in this database, ignoring"). On a miss it builds and logs `"Database of <name> prefabs has no entry with ID <N>"` — confirming the key really is just an **integer ID**, not a pointer or struct.
**`sub_242778`** (called from `RefreshEvents`/`sub_369040`, *not* from `AddEvent` itself) is a **different, string-keyed hash map**: FNV-1a-32 hashes (`offset basis -2128831035` / `prime 16777619` — the literal, unmistakable FNV-1a constants) the `TrackId`-derived string passed in, looks it up in a hash map rooted in some outer registry context (`a1+68`/`a1+72`), and on a hit returns a pointer to a `{begin,end}` pair at `foundNode+16` — exactly the `int*` array `RefreshEvents` iterates to get the list of matching hash IDs for that track. On a miss, returns a pointer to a static empty triple (`&dword_AD4710`) — i.e., an empty, safely-iterable result rather than null.
**So the full resolution is two independent hash maps chained together**: `TrackId string --[sub_242778, FNV-1a keyed]--> vector<uint32 hash>` (one entry per matching `RaceEvent`), then each `hash --[sub_173350, int keyed]--> refcounted RaceEvent-prefab-instance pointer`. Critically, **`AddEvent` (`sub_369AB0`) only ever touches the second map** (`sub_173350` directly) — it takes an already-resolved hash and looks it up in the prefab-instance cache; it never calls `sub_242778` or touches the `TrackId` registry itself. That lookup is entirely `RefreshEvents`'s job, upstream of `AddEvent`.
**This meaningfully simplifies the injection plan from §6v**: to add a synthetic `"LAN: <lobby>"` card to an already-loaded street pin, there is no need to touch the `TrackId`→events registry at all. It's enough to:
1. Fabricate one entry in the **prefab-instance cache** that `sub_173350` queries — pick an unused 32-bit ID, construct a fake `RaceEvent`-shaped object (using the already-known field layout from §6p: `TrackName`, `EventName` = `"LAN: <lobby name>"`, `Completion`, `class_restriction`, `event_type`, `CashReward` Gold/Silver/Bronze, etc.), wrap it in the expected `{+12, +16, +20-refcounted-ptr}` node shape, and insert it into that hash map's bucket chain under the chosen ID.
2. Call `MapTrack::AddEvent(existingPinPtr, &chosenId)` (`0x369AB0`) directly on an already-loaded, real `MapTrack*` (obtainable from the same hit-test/selection machinery already mapped in §6u).
`AddEvent` then does everything else itself using real engine code: grows the pin's own event vector, updates the completion-percentage badge, and sets lock/availability flags — exactly the "reuse the engine's own logic with fake data" strategy favored since §6s, and now with a concrete, minimal (single hash-map entry) fabrication requirement instead of needing to replicate the whole `TrackId` registry or a VFS/prefab load.
**Not yet determined**: the exact in-memory shape of a loaded `RaceEvent` prefab instance (i.e., what the refcounted pointer at cache-node `+20` actually points to — presumably the same object `sub_7D2E8` extracts a `RaceEvent` component from, per much earlier sessions) precisely enough to hand-construct a convincing fake one; and the exact hash-map node/bucket-array construction details (allocation sizes, `sub_97D23C`'s bucket-index formula) needed to splice a new node in safely at runtime without corrupting the real map.
## 6x. The cache's *insert* side (`sub_7CE58`) confirms the value triple's meaning — and reveals a much cheaper injection strategy: relabel a real `RaceEvent`, don't fabricate one
Continuation of §6w. Decompiled `sub_7CE58` — already known from §6s ("Failed to load prefab %s... already present in this database, ignoring") to be the generic prefab-loading/caching function — and confirmed it's the **write side of the exact same hash map** `sub_173350` reads (same bucket-array-at-`+4`/count-at-`+8` layout, same `sub_97D23C` bucket-index call, same "already present, ignoring" log path).
**What it does**: given a prefab resource path string, hashes it (via `sub_67223C` — a *different* hash function than the FNV-1a used for the `TrackId` registry in §6w, so the two maps use different hashing even though both are keyed by strings-derived-to-ints at the storage layer), checks the cache, and on a miss actually loads the prefab (`sub_671330`/`sub_6714E0`) and extracts its `RaceEvent` component via **`sub_7D2E8`** — the same function identified across many earlier sessions as "extracts the `RaceEvent` component from a loaded prefab via `dynamic_cast`." The two outputs of `sub_7D2E8` are then written into a newly-inserted cache node's `+12`/`+16` fields, and the loaded prefab's own refcounted instance pointer goes into `+20`.
**This confirms precisely what `AddEvent` receives once it resolves a handle**: the `+12` field is the actual `RaceEvent` component pointer (the same kind of pointer `sub_7D2E8` has always produced, that downstream code like `sub_208C88` reads `m_StartLine`/`m_FinishLine`/etc. from at fixed offsets, per much earlier sessions), `+16` is an accompanying tag/generation word, and `+20` is the refcounted owning `Actor`/prefab-instance pointer.
**This changes the practical injection recommendation**: rather than hand-fabricating a fake `RaceEvent`-shaped C++ object from scratch (risky — would need its exact vtable, RTTI, and full field layout, only partially known from §6p), a **much cheaper and lower-risk approach** is to reuse an already loaded, real, well-formed `RaceEvent` object: pick any currently-loaded event's component pointer (a guaranteed-valid, correctly-vtabled object), insert a *new* cache node under a fresh unused hash ID whose `+12`/`+16`/`+20` fields simply copy that real object's identity (bumping its refcount, mirroring the same refcount-increment pattern `sub_7CE58`/`sub_173350` already do), then use the existing property-setter machinery (`sub_15F2DC`/`sub_406644`, already reverse-engineered in §6u Region A and `sub_369040`) to overwrite just its display fields — `EventName` → `"LAN: <lobby name>"`, `TrackName`, `class_restriction`, `event_type`, etc. — in place on that shared object, or on a shallow copy of it if mutating the original is undesirable. This "clone an existing valid object and relabel it" strategy avoids ever needing to construct a `RaceEvent` instance's vtable/RTTI/full binary layout by hand, at the cost of needing a real donor event to already be loaded somewhere (which is true for every street with at least one authored event, per every street observed so far in this project).
**Two viable concrete strategies now on the table** (not yet chosen/implemented):
1. **Clone-and-relabel** (recommended, lower risk): reuse a real, already-loaded `RaceEvent*` as the cache entry's payload; overwrite only its display properties via the engine's own property setters before/after calling `AddEvent`.
2. **Fabricate-from-scratch** (higher risk, more complete control): hand-construct a new `RaceEvent`-shaped object with a real vtable pointer and correctly-laid-out fields, matching the full memory layout (not just the SB-file field layout already known from §6p) — would need further work to pin down `RaceEvent`'s actual C++ class layout (vtable location, exact field offsets in the live object, not just the on-disk SB representation).
## 6y. `RaceEvent`'s real vtable, constructor, and a byte-precise field map — the concrete basis for fabricating one from scratch (per user's choice of "Вариант Б")
Continuation of §6x. The user chose the fabricate-from-scratch strategy, so the next task was pinning down `RaceEvent`'s actual live C++ object layout (not just the on-disk SB field names already known from §6p).
**Vtable/RTTI**: found via the same `entity_query`-on-names technique as §6v — `_ZTVN2im3app4race9RaceEventE` at `0xaa78a8` (real vtable symbol), `_ZTIN2im3app4race9RaceEventE` typeinfo immediately after at `0xaa78d8`. `xrefs_to` on the vtable surfaced exactly 3 functions: `sub_2A4B58` (constructor), `sub_2A4C70` (presumably destructor, not yet decompiled), and `sub_2A7BF4` (not yet decompiled, 476 bytes — likely another virtual method or a clone/copy function).
**Constructor** (`sub_2A4B58`): `malloc(0xE4)` — **the live object is exactly 228 bytes**. Calls a base-class constructor (`sub_670454` — sets a temporary base vtable `off_ABB274`, a 4-byte field at `+4` to `0`, and a 2-byte field at `+8` to `256`/`0x100`; this looks like a generic ECS `Component` base: owner/actor pointer + a type-tag default), then overwrites the vtable pointer with the real one (`off_AA78B0` — the usable, RTTI-header-skipped vtable, same Itanium-ABI convention as `MapTrack` in §6v), then zero/default-initializes every field through offset `+220`.
**Field map** (cross-referenced against `sub_2A4D70`, the `RaceEvent` field deserializer from §6w/§6x, now fully decompiled — every write in it targets `a1+<offset>` with an explicit property-name string literal right next to it, giving a byte-precise, high-confidence mapping):
| Offset | Field (property name) | Type | Default |
|---|---|---|---|
| `+0` | vtable ptr | ptr | `off_AA78B0` |
| `+4` | (base class: owner/actor?) | int | `0` |
| `+8` | (base class: type tag) | word | `256` |
| `+12` | *(unnamed — not written by `sub_2A4D70`; set elsewhere)* | string (12B: begin/end/cap) | empty |
| `+24` | `RaceType` | string (12B) | empty |
| `+40` | `RaceFSMPrefabOverride` | string (12B) | empty |
| `+56` | `CarRestriction` | string (12B) | empty |
| `+72` | `TrackName` | string (12B) | empty |
| `+88` | `EventName` (via an indirect "Name" lookup on a sub-list, then interned via `sub_406644`) | interned string ptr (4B) | empty |
| `+92` | `Location` (same indirect-lookup + intern pattern) | interned string ptr (4B) | empty |
| `+96` | `Zone` — **computed**, not a raw property: `"ZONE_" + uppercase(Location)`, then interned | interned string ptr (4B) | empty |
| `+100` | `EnvironmentPrefab` | string (12B) | empty |
| `+116` | `TrafficCarCount` (indirect lookup) | int (4B) | `0`/unset |
| `+120` | `OpenWorldTrack` (indirect lookup) | bool (1B) | `0` |
| `+124` | `AutologID` (indirect lookup) | int (4B) | `-1` |
| `+128` | `BlacklistEvent` (indirect lookup) | bool (1B) | `0` |
| `+132` | `ClassRestriction` (indirect lookup) | int/enum (4B) | `0`/unset |
| `+136` | `PursuitType` | string (12B) | empty |
| `+152` | `StartLineNoSpawnZone` | float (4B) | `1000.0` (`0x447A0000`, confirmed via the exact literal `1148846080`) |
| `+156` | `FinishLineNoSpawnZone` | float (4B) | `1000.0` |
| `+160` | `SpawnDistance` | float (4B) | `1000.0` |
| `+164` | `StartLine` | string (12B) | empty |
| `+180` | `FinishLine` | string (12B) | empty |
| `+196` | `EndOfTrack` | string (12B) | empty |
| `+212` | `CheckpointCollection` | string (12B) | empty |
`StartLine`/`FinishLine`/`EndOfTrack`/`CheckpointCollection` at `+164/180/196/212` match, to the byte, the `raceDef+164/180/196/212` offsets already established across much earlier sessions (§6p and before) for `m_StartLine`/`m_FinishLine`/`m_EndOfTrack`/checkpoint-container — strong cross-session consistency check, both derivations agree exactly.
**Important correction**: earlier in this session (§6v/§6x speculation) the 3 consecutive floats at `+152/+156/+160` were guessed as a plausible match for `CashReward` Gold/Silver/Bronze. **That guess is now disproven** — they're confirmed to be `StartLineNoSpawnZone`/`FinishLineNoSpawnZone`/`SpawnDistance` (spawn-safety-radius tuning values), all defaulting to the same `1000.0`. `CashReward` and the UI-facing properties read reflectively elsewhere (`class_restriction`, `event_type`, `Completion` — read via `sub_15F2DC`/`sub_406644` in `sub_1781BC` §6u Region A and `sub_369040` §6v) are **not present anywhere in this 228-byte struct**. They must live on a separate component attached to the same `Actor`/prefab instance (this engine's ECS allows multiple components per entity, already established via the `component_weak_ptr<RaceEvent>` string constants in §6y's own vtable search) or be computed at read-time rather than stored — not yet confirmed which.
**Practical implication for fabrication**: a synthetic `RaceEvent` needs, at minimum: the real vtable pointer (`off_AA78B0`), a plausible base-class header (owner pointer + type tag), and populated `TrackName` (to match the target `MapTrack`'s own `TrackId` so `RefreshEvents`'s registry lookup — if that path is used — or manual placement resolves correctly) plus `EventName` (interned, e.g. `"LAN: <lobby name>"`) at minimum for a card to display meaningfully. `CashReward`/`class_restriction`/`event_type`/`Completion` — everything the actual UI card visibly shows beyond the name — are **not** part of this struct and remain the next concrete unknown: need to find the reflective property-descriptor table `sub_15F2DC` consults (likely a separate registered table mapping name strings to getter functions/offsets, not necessarily this struct at all) to know what a synthetic object must expose for those properties to resolve.
## 6z. Correction: `sub_15F2DC` is a named-widget lookup (`FindOrCreateLayoutEntity<Text>`), not a property read — re-interprets §6u Region A and narrows what's genuinely still missing from `RaceEvent`
Decompiled `sub_15F2DC` in full (previously only inferred as "a generic property getter" by analogy with its call pattern). It is **not** that — it's the exact same pattern as `sub_17A4CC` (§6s: `FindOrCreateLayoutEntity<T>`, searching a linked list of named layout entities at `scene[23]`/`scene[24]`, `strcmp` against the requested name, falling back to `"Unable to locate layout entity: "` — the identical error string), just instantiated for `im::scene2d_new::Text` instead of a generic `Node`. It looks up a **named child widget** in a scene graph and `dynamic_cast`s it to `Text` (a text-label widget), returning that widget pointer — it does not read or return a *value* at all.
**This re-interprets §6u Region A**: `sub_1781BC`'s block at `0x178afc-0x178e40` does not read `TrackName`/`Completion`/`EventName`/`class_restriction`/`event_type` as *properties off a RaceEvent-like object*. It looks up **named `Text` widgets** — child nodes of some scene rooted at `[MapScreen+0x120]` — by exactly those fixed names, and caches the resulting widget pointers into `MapScreen+0x1F8..+0x21C` for later use (almost certainly so a separate, not-yet-located binding/formatting step can set each widget's displayed text from the real data). `[MapScreen+0x120]` is therefore a **scene/`Node` pointer** (the event-detail-card template instance), not a `RaceEvent*` — consistent with `sub_15F2DC`'s second argument being treated as a scene object with an `[23]`/`[24]` named-entity list, exactly like `sub_17A4CC`'s.
**This narrows, rather than widens, the remaining gap from §6y**: `"class_restriction"` and `"event_type"` are very likely just the *names of the Text/icon widgets* that get filled in from `RaceEvent.ClassRestriction` (`+132`) and `RaceEvent.RaceType` (`+24`) respectively — both of which **are** already in the byte-precise field map from §6y. What's still genuinely unaccounted-for in the 228-byte `RaceEvent` struct is narrower than previously stated: only **`CashReward`** (Gold/Silver/Bronze) and **`Completion`** (already known, per §6v, to be *computed* by `MapTrack::AddEvent` from player-progress data, not read off `RaceEvent` at all — so not actually missing, just not stored on this object by design). `CashReward` remains the one open question — plausibly read via the same indirect "sub-list" lookup pattern (`sub_4F99F0`/`sub_4F9A80`) used for `EventName`/`Location`/`TrafficCarCount`/`AutologID`/`BlacklistEvent`/`ClassRestriction` in `sub_2A4D70`, just for a nested "Rewards" key not yet located, or extracted by an entirely separate deserializer function for a second component on the same prefab `Actor` (this engine's ECS allows several components per entity, as already established).
**Practical implication for the fabrication plan (§6y)**: the synthetic `RaceEvent` object's field requirements are essentially unchanged from §6y's table — `TrackName`, `EventName`, `RaceType`, `ClassRestriction` cover what the visible card UI needs (name, class icon, restriction), and `Completion` is handled automatically by `AddEvent`'s own bookkeeping (§6v) rather than needing to be set on the object at all. `CashReward` is the only remaining unknown, and is optional in the sense that a card lacking it would very likely just show a blank/default reward rather than fail outright (not yet confirmed, but low-risk either way, given every other field the deserializer sets defaults for missing prefab data rather than erroring).
## 6aa. `CashReward`'s real layout found — closes the last gap for fabricating a complete synthetic `RaceEvent`
Continuation of §6z. `CashReward` (`im::app::metagame::CashReward`) is confirmed to be a genuinely separate C++ class, not a `RaceEvent` field, part of a small class family also including `im::app::metagame::Reward` (base) and `im::app::metagame::RewardsCollection` (container) — found via the same `entity_query`-on-names / `xrefs_to`-on-vtable technique used for `MapTrack` (§6v) and `RaceEvent` (§6y).
**`CashReward`'s constructor** (`sub_23E2C4`): `malloc(0x1C)` — **28 bytes total**, the smallest object found this session. Calls `Reward`'s base constructor (`sub_25D4F0`), which itself calls the *same* generic `Component` base constructor already seen for `RaceEvent` (`sub_670454` — sets a temporary vtable, `+4`=owner ptr default `0`, `+8`=type-tag word default `256`) before setting its own vtable (`off_AA60AC`). `CashReward`'s own constructor then overwrites the vtable a third time with its own (`off_AA5B54`, the real usable vtable) and sets:
| Offset | Field | Default |
|---|---|---|
| `+0` | vtable ptr | `off_AA5B54` |
| `+4` | (Component base: owner ptr) | `0` |
| `+8` | (Component base: type tag) | `256` |
| `+12` | unknown (not a reward amount — separate field, possibly a type/flags int) | `0` |
| `+16` | Bronze cash amount | `10000` |
| `+20` | Silver cash amount | `30000` |
| `+24` | Gold cash amount | `50000` |
(Tier-to-offset mapping is inferred from ascending magnitude matching the conventional Bronze < Silver < Gold ordering — not confirmed via an explicit property-name deserializer the way `RaceEvent`'s fields were in §6y, since no `CashReward`-specific field-deserializer function was located this pass; still high-confidence given the exact 3-tier shape and clean ascending defaults.)
**This closes the fabrication gap identified in §6y/§6z.** A synthetic event needs two objects, both now fully specified: a 228-byte `RaceEvent` (§6y's table: `TrackName`, `EventName`, `RaceType`, `ClassRestriction` at minimum for a meaningful card) and a 28-byte `CashReward` (this section) for the reward amounts shown on the card. Both constructors are simple, malloc-and-fill patterns with no complex dependencies — straightforward to replicate for a fabricated pair of objects at runtime.
**Still open, not yet located**: exactly how a `RaceEvent` and its `CashReward` are associated on the same prefab `Actor` (i.e., which field/mechanism lets code go from "I have this `RaceEvent`" to "here is its `CashReward`") — this wasn't needed to build the two objects individually, but will matter if the actual card-rendering code fetches the reward via that association rather than via a hardcoded second cache lookup. **Resolved — see §6bb.**
## 6bb. `RaceEvent`↔`CashReward` link resolved: sibling components on the same `Actor`, each deserialized independently per-race from a shared prefab property table — explains why every race has its own distinct reward
The user specifically asked to pin down this relationship, and flagged the important constraint that different races have different rewards (i.e. the link cannot be to one shared/global `CashReward`).
**Found the accessor**: `sub_164540(out, actorPtr)` = **`Actor::GetComponent<CashReward>()`** — confirmed via the profiling string `"GetComponent"` plus the exact same generic pattern used everywhere else in this engine (iterate an `Actor`'s component array at `actor[5]`..`actor[6]`, `dynamic_cast` each `Component*` to the target type, return the first match as a weak/shared handle). Crucially, its second argument (`a2`) is an **`Actor*`** — the *same* kind of object `sub_7D2E8` (the long-established "extract `RaceEvent` from a loaded prefab" function, referenced since much earlier sessions) also operates on. **This settles the relationship**: `RaceEvent` and `CashReward` are not linked to each other directly at all — they are independent **sibling components attached to the same `Actor`** (the loaded `.prefabs.sb` instance). Getting from "this `RaceEvent`" to "its `CashReward`" means going through the owning `Actor`'s component list, not through any field on `RaceEvent` itself. (A second function, `sub_25D998` = `Actor::GetComponents<Reward>()` — plural, base-class `Reward` rather than `CashReward` specifically — confirms the same component-list mechanism generically collects *all* reward-type components on an `Actor`, in case a race ever has more than one.)
**Found `CashReward`'s actual field deserializer**: `sub_23E47C`, in the same code region as `CashReward`'s constructor (§6aa). It reads exactly three named properties from the shared per-prefab property table (parameter `a3` — the same kind of indexed property-list object `sub_2A4D70`/`RaceEvent`'s deserializer read its own indirect properties from, e.g. `Location`/`TrafficCarCount`/`ClassRestriction`) via the identical `sub_4F99F0`/`sub_4F9A80`/`sub_50CE64` helper chain: **`"Bronze"` → `+16`** (default `10000`), **`"Silver"` → `+20`** (default `30000`), **`"Gold"` → `+24`** (default `50000`) — confirming, with explicit property-name strings this time (not just inferred from default ordering as in §6aa), the exact tier-to-offset mapping.
**This closes the loop architecturally**: a single `.prefabs.sb` file's property table is evidently shared across *all* of that prefab's component deserializers at `Actor`-construction time — `RaceEvent`'s deserializer (`sub_2A4D70`) and `CashReward`'s deserializer (`sub_23E47C`) each independently pull their own named subset of properties from the same source. Since every race has its own separate `.prefabs.sb` file (confirmed since the earliest sessions of this project, e.g. `event_02_timetrial.prefabs.sb`), every race's `Actor` gets its own freshly-constructed `CashReward` instance with that specific race's own `Bronze`/`Silver`/`Gold` values baked in at load time — exactly matching the user's point that rewards differ per race, and explaining precisely *how* that variation is represented (per-`Actor` component instances, not a shared/global reward table).
**Practical implication for fabrication (final piece)**: a synthetic race's fake `Actor` needs a component list containing both the fake `RaceEvent*` and the fake `CashReward*` (with whatever `Bronze`/`Silver`/`Gold` values are desired for the synthetic "LAN: <lobby>" entry). `sub_7D2E8` and `sub_164540`/`GetComponent<CashReward>()` will then resolve correctly against it exactly as they do for any real, loaded race — no need to intercept or special-case either accessor.
## 6cc. Live breakpoint-based debugging CONFIRMED WORKING — Samsung Galaxy A9 (2018), genuine AArch32 hardware
Per the user's own research (§6t "Root cause identified"), the Fairphone 5's failure to sustain breakpoints was attributed to its SoC lacking native AArch32 hardware. The user then obtained a Samsung Galaxy A9 (2018), SM-A920F, Snapdragon 660 (Kryo 260 = Cortex-A73/A53-based — genuinely pre-dates Qualcomm's native-32-bit-hardware removal), Android 10, rooted via Magisk, and asked for a debugger to be attached as a direct test of that hypothesis.
**Setup**: reused the native ARM32 `gdbserver` already extracted from NDK r16b in an earlier session, but this time paired it with the **matching-generation GDB client** rather than a modern `gdb-multiarch` — extracted `gdb-orig` (GDB 7.11) from the same NDK r16b archive's `prebuilt/linux-x86_64/bin/`, working around its legacy `libncurses.so.5` dependency via a local `LD_LIBRARY_PATH` symlink to the system's `libncurses.so.6` (no matching system package exists for that old ABI, and this avoided any system-wide install). This eliminates the ~7-year GDB/gdbserver protocol-version skew that was the leading suspect for the earlier `continue`-crashes-`gdbserver` failure on Fairphone 5's own gdbserver test.
**Result — decisive and clean**: the exact same "sanity check" that disproved breakpoints on the Fairphone 5 (a software breakpoint on `malloc`, expected to fire within seconds of any activity) **fired correctly on the very first `continue`**, on a background `GLThread`, with a correct `pc` and a real caller return-address backtrace frame. `gdbserver` then detached cleanly ("Detaching from process ...", not a crash), and the game process remained alive and undisturbed afterward. This is a genuine, reproducible, working breakpoint — the first of this entire project.
**Two practical gotchas hit and solved along the way** (now also recorded in `[[reference-native-arm32-debugging-requirement]]`):
- `gdbserver --attach` pauses the process immediately via ptrace; attaching mid-loading-screen freezes the app there (looked like a hang until understood — detaching let it resume immediately).
- This device (Android 10) mmaps native libraries **directly from inside the APK's zip** (uncompressed, page-aligned storage) rather than extracting a standalone `libapp.so` — so `/proc/PID/maps` never shows a `libapp.so`-named mapping to grep for, and the runtime load base has to be computed manually: get `libapp.so`'s data offset within `base.apk` via Python's `zipfile` module, match that offset against a `/proc/PID/maps` line, and that line's start address is the load base. Verified byte-for-byte against the reference `libapp.so`'s own ELF header before trusting it. This load base changes on every process (re)launch (ASLR) and must be recomputed each time.
**Practical implication**: this project's live-debugging capability is no longer read-only. The Galaxy A9 is now the reference device for any future "does function X actually get called" investigation — a question that blocked several static-analysis threads earlier this session (e.g. confirming exactly where a `MapTrack`'s pin gets its `RaceEvent` handles attached, §6v-§6y, was done entirely via static analysis precisely because live debugging wasn't available at the time).
## 6dd. Live-verified, end-to-end: `MapTrack::HandleEvent → RefreshEvents → AddEvent` — the full §6v-§6w chain confirmed exactly as reverse-engineered, plus base-address-resolution sanity check
Continuation of §6cc, now that live breakpoint debugging works on the Galaxy A9. Two things were checked, both live, on-device.
**Base-address resolution — no divergence found, single algorithm is sufficient.** The user's concern: since this device (Android 10) mmaps native libraries directly from inside the APK zip rather than extracting a standalone `libapp.so` (§6cc), does the existing `get_libapp_base()` (`dl_iterate_phdr`-based, in `launcher/mpcore/src/main/cpp/main.cpp`) compute the same address as manual inspection would? Since the installed APK on this device already bundles `libmpcore.so` (not stripped, full debug info), this was tested directly rather than synthetically: located `libmpcore.so`'s own runtime load address (same zip-offset technique as for `libapp.so`, verified byte-for-byte against its ELF header), then read the live value of its `libapp_base` global variable (found via `readelf -sW`, offset `0x3b6c`) straight out of process memory. **Result: `0xb8798000`, exactly matching** the value independently computed by hand for the same process. `dl_iterate_phdr` already correctly abstracts over the "loaded from an extracted file" vs. "loaded directly from within an APK zip" distinction — this is precisely what it's designed to do, and Android's own linker keeps `dlpi_addr` correct either way. **No fallback/dual-algorithm logic is needed**; the existing single implementation in `main.cpp` is correct as-is on this device.
**Live chain verification.** Computed runtime addresses for `MapTrack::HandleEvent` (`sub_368DFC`), `RefreshEvents` (`sub_369040`), and `AddEvent` (`sub_369AB0`) using the confirmed load bias, attached `gdbserver` to a **freshly relaunched** game process early enough to catch the map screen's first-ever layout pass (revisiting the map screen after a Garage trip, tried first, turned out *not* to refire the event — `MapScreen` evidently persists underneath Garage rather than being torn down and recreated, so a fresh app launch was needed instead), and set:
- `break *HandleEvent if *(int*)($r1+4) == 1025` (only stop on the real `FlowSetLayoutScreenEvent` dispatch, filtering out the very frequent, unrelated event type `31` noise already seen and characterized in this pass)
- plain breakpoints on `RefreshEvents` and `AddEvent`
**Result — the entire chain fired exactly as predicted, for real, back to back, for multiple pins in sequence**:
1. `HandleEvent` hit with `evtype==1025` confirmed live (first time this specific condition was ever directly observed, not just inferred from static analysis).
2. Immediately followed by `RefreshEvents` hit with the **identical `this`** pointer — confirming `HandleEvent`'s `sub_369040(a1)` call live.
3. Immediately followed by **multiple consecutive `AddEvent` hits, all with that same `this`** — 6 calls for the first pin observed, 4 for the second, 3 for the third (a fourth pin's `HandleEvent` was still starting when the test's stop budget ran out) — confirming `RefreshEvents`'s per-match loop calling `AddEvent` once per resolved `RaceEvent` hash, live, and giving the first-ever **real observed count** of how many events a single street pin can carry (previously only "each street pin has its own event vector, supports more than one" was established structurally, §6v; now concretely: real streets in this save have pins with 3-6 events attached, not just 1).
This closes the loop on the multi-session `MapTrack`/`AddEvent` investigation with actual runtime evidence, not just static inference — every claim in §6v-§6y about this call chain is now independently confirmed.
## 6ee. Implementation: the injection hook works end-to-end (no crash) — two real Actor-layout bugs found and fixed live on-device
Implemented the plan from `/home/megboyzz/.claude/plans/dreamy-giggling-hearth.md` on a new branch `lan-event-injection-poc` (based on `track-hook-toggle-flag`, not bare `master` — that branch is missing the working hook-trampoline infrastructure `master` was assumed to have; `master`'s `main.cpp` turned out to be a much earlier, scratch-code-laden state that was never actually merged forward). New file: `launcher/mpcore/src/main/cpp/lan_event_injection.h`, wired into `main.cpp` behind `kEnableLanEventInjectionHook`.
**Iterative on-device debugging found two real bugs in the "fabricate a fake Actor" approach from §6y**, both only surfaced by actually running it (exactly the residual risk the plan called out in advance):
1. **`Actor+4` must be non-null.** `AddEvent` (`sub_369AB0`) checks `*(actorPtr+4)` and logs "Dereferencing a component pointer whose actor has been deleted." (this string is Android's tombstone "Abort message" field even though the actual signal is SIGSEGV, not SIGABRT — a genuine tombstone quirk, not indicative of an abort() call) when it's zero. Fixed by giving the fake `Actor` a self-pointer at that offset.
2. **`Actor+8` must start at exactly `1`, not `0`.** This is a refcount: `sub_173350` (the handle resolver) increments it by 1 on every successful resolve (a temporary borrowed reference), and `AddEvent` releases that same reference before returning — if the release brings it to exactly 0, it invokes a virtual "release" method through the object's own vtable (`(*(int*)(*(int*)actorPtr + 12))(actorPtr)`). With refcount starting at 0, this net-negative pattern hit 0 and crashed on the null vtable (fault addr `0xc` = `NULL+12`, confirmed via live `objdump` disassembly of the exact crash offset). Fixed by starting the refcount at 1, so the borrow-then-release cycle nets back to 1 and never triggers that call — safe for any future number of resolves, since each is always a matched borrow+release pair.
**A third, unrelated bug was found and fixed in the diagnostic/observation logging code** (not the core injection path): reading a `MapTrack`'s event vector (`this+0x240/0x244`) from within the hook occasionally caught a garbage `begin` pointer (`0x100`) paired with a small, plausible-looking element count — most likely a torn read racing a concurrent update from a different thread (the crash always occurred on a `GLThread`, and `AddEvent`'s own `push_back` isn't atomic: realloc + memmove + three separate pointer writes). Fixed with a defensive plausibility check (the pointer must look like a real heap address, and the count must be small) before trusting it enough to dereference.
**Result after both fixes**: injection completes cleanly for all ~12 `MapTrack` pins this hook currently reaches, with zero crashes across repeated fresh-launch tests — a real, meaningful validation that the whole `RaceEvent`/`CashReward`/fake-`Actor`/hash-insert/`AddEvent` chain from §6v-§6bb works as designed against the live game.
**Not yet achieved**: visual confirmation of the synthetic card actually rendering on a real, currently-visible street. The three streets visible on this save's map ("РЭЙНОЛЬДЗ ЛЭЙН" — a "Most Wanted #10" boss battle + a real event; "КЭМЕРОН ДРАЙВ"; "КРЮГЕР АВЕНЮ" — a real event) were each checked directly (tapped, event-list panel inspected) and show no injected card — meaning **none of them are among the ~12 `MapTrack` instances this hook's `evtype==1025` condition catches**. Since `HandleEvent`'s dispatch mechanism itself is confirmed correct (§6dd) and injection is confirmed to work once a pin is reached, the remaining gap is purely "which pin is which" / "why do these 3 special-content streets not go through this same broadcast path" — plausibly because Most-Wanted-boss and other hand-authored story content gets attached through a different, not-yet-identified mechanism than the generic `TrackId`-registry-driven path this hook intercepts. This is the natural next investigative thread if visible confirmation on a known street is wanted, but is a separate question from "does the injection mechanism itself work," which is now answered: yes.
## 6ff. Delayed-crash root cause narrowed to the "Blacklist" rival system — exact faulting instruction not yet pinned down
Continuation of §6ee's flagged delayed crash (fault addr `0xc0ffee00`, our first injected cache key, dereferenced directly as a pointer roughly 5 minutes after injection). Investigated via a mix of static analysis and, eventually, successful live breakpoint-catching once a gdb/gdbserver state-sync quirk was worked around.
**Static trail**: the crash backtrace's frame #01 static offset (`0x233b70`, from `libapp.so (offset 0x4d8000)`) traces into `sub_233684` — a function that checks a fixed table of hardcoded event names (`"event_60_blacklist_1"` down to `"event_04_blacklist_10"`, all in a literal pool at `0x233b10`) to determine an event's **Blacklist rival rank** (NFS Most Wanted's rival-racer ranking system — matches the "Рэйнольдз Лэйн" street's observed "Most Wanted #10" boss card from §6ee). This function then walks a `boost::function`-keyed map/tree structure, invoking a stored callback per entry via a generic invoker thunk (`sub_234F38`, confirmed via decompile to be boost::function's standard dispatch pattern, not itself buggy).
**Live-caught**: after working around a recurring gdb/gdbserver synchronization bug (`continue` intermittently failing with "Cannot execute this command while the target is running" on already-running, multi-threaded processes — reliably avoided by attaching **immediately** after process spawn, before the game's worker threads fully start, rather than to an already-running instance), set a breakpoint at `0x233b70` and caught it firing **28+ times in a row with completely stable, valid register values** (`r0=0xb8c4f0f4`, a genuine stack address). Disassembling the actual bytes there (not a defined IDA function — literal pool followed by unrecognized code) revealed `0x233b70` is itself a `BL` instruction into a **fourth, previously unknown function at `0x233b4c`** (only reachable via an address taken and stored elsewhere, e.g. as a registered callback — never called directly by name anywhere), which itself calls into `0x40602C`.
**Conclusion so far**: this whole call chain (`sub_233684` → per-entry callback → `sub_233b4c` → `sub_40602C`) runs **frequently and safely** under normal conditions — it's a routine, periodic Blacklist-rival scan over some always-present collection, not something our injection specifically triggers. The crash is a **rare condition within this hot path**: at least once, an entry in whatever collection this scan walks contained our raw injected cache key (`0xc0ffee00`) instead of a properly-resolved value, and something downstream dereferenced it directly as a pointer. The **exact single faulting instruction is not yet pinned down** — live-catching it requires either a much longer soak (the crash took ~5 minutes to occur naturally in the two observed cases) or a smarter conditional breakpoint (e.g., break only when a register looks like one of our `0xC0FFEE0X`-range keys), neither attempted yet due to time already invested in this thread.
**Practical implication / mitigation direction (untested)**: real cache keys are hash values (computed via `sub_67223C`, not human-chosen constants), and this Blacklist-scan code appears to expect either a resolvable-through-`sub_173350` handle or a real hash-shaped value at some point in its processing — using an easily-recognizable sentinel range (`0xC0FFEE00`+) for synthetic keys, while convenient for debugging, may itself be more likely to look "plausible enough to use directly" to whatever misbehaving check exists here than a value that looks like a real hash would. Worth testing empirically: switch the injection's key-generation scheme to something structurally closer to a real `sub_67223C`-style hash (e.g., actually hash a fake resource-path string through that same function) and re-run the ~5-minute soak test to see if the crash still reproduces — this doesn't require finding the exact faulting instruction to potentially resolve the crash, though it's a mitigation-by-avoidance rather than a confirmed fix.
## 6gg. Delayed crash: EXACT root cause found — it's the QA-only "Soak Test" auto-race feature, not the Blacklist system; clean, cheap mitigation identified
Correction/completion of §6ff: that section traced the wrong backtrace frame (frame #01, a caller/return-address site inside the Blacklist rank-lookup code, which turned out to be an unrelated red herring that just happens to execute very frequently and safely). Re-examining the *original* crash report's **frame #00** (the actual faulting instruction, previously not converted to a static offset) gives the real answer.
**The faulting instruction**: static offset `0x406cd0`, inside `sub_406CAC(int **a1, const void **a2)` — a generic "construct an `eastl`-style string object from a C-string" helper (matches its call sites elsewhere in the binary, e.g. `sub_17C120`'s `sub_406CAC(&v150, v101)`). Its very first operation is `v2 = *a2;` (treat the input as a pointer to a C-string pointer) followed by a classic `strlen`-style scan: `v4 = v2 - 1; while (*++v4);` — this is the exact line at `0x406cd0`. If `*a2` isn't a real string pointer, this immediately faults trying to read the "string"'s first byte — and the original crash's fault address was **exactly `0xc0ffee00`**, our first injected cache key, confirming `*a2` held our raw integer key value directly, not a string pointer.
**Where this gets called with bad data — the QA "Soak Test" auto-race feature** (already identified and named in §6v/§6bb, `sub_17C120`): once every ~4 seconds of active gameplay (`flt_AD417C > 4.0`), it picks an entry by **numeric index** (a persistent, decrementing global counter `dword_AD4180`, wrapping around based on an array's element count) from an array `v156` obtained via `sub_242904(&v156, someContext + 320, 0)` — the same `+320`-offset context pattern established (§6w) as this project's prefab-instance cache — and calls `sub_406CAC(&v150, v101)` on that entry to build a debug log line ("Soak Test Run No: N - `<name>`"). This is **QA/debug-only instrumentation**, not player-facing functionality: it auto-picks and logs (likely also auto-launches) random cached races purely for automated soak testing.
**Root cause of the crash, precisely**: `sub_242904` appears to return a *separate, parallel* list (of resource-path-string pointers, one per real cached prefab) rather than reading the same hash-table `sub_7D638`/`sub_173350` operate on directly. Our injection only inserts into the **primary hash-table cache** (via `sub_7D638`) — it never adds a corresponding entry to this *other* parallel list. This desyncs the two structures' sizes/contents, so `dword_AD4180`'s index-based lookup into `v156` eventually reads memory that doesn't correspond to a real entry — landing, in the observed crash, on memory holding our raw injected key rather than a valid string pointer.
**Practical mitigation — cheap and clean, doesn't require touching the parallel list at all**: this whole code path is a **QA-only feature with no player-facing purpose** (auto-launches random races for soak testing — not something the mod needs to preserve, and almost certainly disabled in normal retail play already via whatever build/debug flag gates `sub_15811C(*(a1+8))`, one of the two conditions guarding it). The simplest fix is to **prevent this branch from ever running** rather than trying to keep a second, not-yet-reverse-engineered list in sync with every injected entry — e.g., hook `sub_17C120` (or more surgically, force `flt_AD417C` to never exceed `4.0`, or short-circuit right before the `sub_406CAC` call) so the Soak Test logic never fires. This fully eliminates the crash's trigger condition without needing to understand or replicate whatever `sub_242904`'s parallel list actually is.
## 7. Open questions / next steps (see plan presented to user for full detail)
1. Locate the real vtables (not just RTTI name strings) for §3.1/§3.2 classes in **this** binary, starting from the confirmed string addresses (walk backwards from `name_ptr` to the `type_info`/vtable structure, same technique demonstrated working this session for `RaceLoaderTask`'s `sp_counted_impl_p` wrapper at `0xd7a2c8`).
2. Decompile (Hex-Rays) the actual `RaceLoaderTask`/`OpponentCollection`/`TrackNavigator`/`RaceStartingGrid` virtual functions once addresses are known, to get real field offsets and signatures (replacing the old chat's guesses).
3. Confirm whether `Health`, `Nitro`, `DamageDealtMultiplier`, `CarDamage`, `SpikeStrip` exist under the same names in this binary (not yet searched this session — lower priority, not core to the multiplayer subtasks).
4. Read `game_cache/published/prefabs/racefsms/*.sb` and `game_cache/published/flow/race/*.sb` directly (via `NFSMW12MobileTools`) to get the *authoritative*, binary-RE-free picture of the race-start Flow sequence and FSM structure — likely faster and more reliable than reversing `InRaceState` from disassembly alone.
5. Confirm car-selection screen invocation path (how the "native car-selection menu" is invoked/returns a result) — needed for the "Choose Car" lobby button requirement.
**Not yet confirmed**: whether the "3 slots" cap is a real per-street constant (worth checking a street with 2 or 3 *unlocked* real events, if one exists in this save, to see whether it ever shows 4+ cards) or coincidental to the two streets tested so far (both had only 1 real event authored). If a street with 3 unlocked events still shows only 3 slots and a street with fewer shows fewer non-locked ones, that would strongly confirm the fixed-pool theory.
**Resolved — see §6v.** `MapTrack::AddEvent` (`sub_369AB0`, `0x369AB0`) is the exact function that attaches a `RaceEvent` to a pin (into a growable per-pin vector at `MapTrack+0x240..0x248`, not the `+0xB8` field originally suspected in §6u — that turned out to be a red herring from a different, secondary "closest track under touch" cache read by `sub_1781BC`, unrelated to the pin's actual owned event list). Next step: determine the exact `RaceEventHandle` argument shape `AddEvent` expects, so it can be called with synthetic data.
## 6hh. Subtask 2 groundwork — `RaceStartingGrid`/`StreetRaceStartingGrid` fully mapped (grid is procedural, not per-track data), `Opponent`/`OpponentCollection` live layout found, `TrackNavigator`'s spline→world resolver identified (reusable for subtask 4), cop-spawn scheduler located
Session 2026-08-26 (overnight, autonomous per explicit instruction — "работай до исхода лимита токенов, в конце расскажешь"). Scoped in `ARCHITECTURE.md` §3b the prior session; this entry answers all four of that section's open questions, all via static SB-data inspection + IDA decompilation, no live device needed.
### Q1 (is the street-race grid layout per-track or shared) — ANSWERED: shared, procedural, not per-track data
First checked the DATA side: unpacked and diffed 15+ different `event_*_race.prefabs.sb` files (`NFSMW12MobileTools`) spanning every region — **every single one has exactly 5 `Opponent` entries** (confirmed field-by-field, see Q3 below), regardless of track. Then unpacked two full track scene files (`region4_chicago_track1.scene.sb`, `region1_foothills_track1.scene.sb`, ~75k `DATA_Elements` each) and searched every `actor`'s `name` field for grid/spawn-related strings: **found exactly one actor literally named `"start"` per track** (plus many `"mesh_start"` visual props, not gameplay locators) — no numbered spawn-point actors (`start_1`, `grid_pos_2`, etc.) anywhere in either track.
This ruled out "per-track authored grid" and pointed at "single locator + code-computed offsets," confirmed by decompiling `RaceStartingGrid`/`StreetRaceStartingGrid` (RTTI: `im::app::race::description::{RaceStartingGrid,StreetRaceStartingGrid}`, vtables `_ZTVN2im3app4race11description{16RaceStartingGrid,22StreetRaceStartingGrid}E` at `0xaa7d28`/`0xaa7da8` — the old `0xcfeb5c`/`0xcfeb8c`/`0xcfebb8` addresses recorded in §3.1 are from a stale `.i64`, do not use them). `StreetRaceStartingGrid`'s constructor (`sub_2B884C`) hardcodes 5 float defaults directly in the C++ constructor:
| Offset | Field name (confirmed via deserializer `sub_2B8A60`) | Default | Notes |
|---|---|---|---|
| `+12` | `MinDistanceBetweenRacers` | `10.0` | |
| `+16` | `MaxDistanceBetweenRacers` | `15.0` | |
| `+20` | `MaxTrackWidthFraction` | `0.8` | **fraction**, not an absolute value — proportional to whatever track's actual width |
| `+24` | `PlayerStartingSpeed` | `27.778` (= 100 km/h, `100/3.6`) | read as km/h from data, converted `*0.27778` at load time; ctor default is pre-converted |
| `+28` | `OpponentsStartingSpeed` | `36.111` (= 130 km/h) | opponents start faster than the player by default |
The deserializer (`sub_2B8A60`) reads these same 5 property names reflectively (`sub_4F99F0`/`sub_4F9A80`, the same generic property-getter used throughout this codebase) from a property table, falling back to the ctor defaults if absent. **None of the 62 unpacked event files' schemas contain any struct/field with these names** — confirmed via a full struct-name enumeration across every `event_*.sb.json` in the repo. Conclusion: **every regular street race uses the identical 5 hardcoded grid parameters; the only per-track variation is the natural, unavoidable difference in where each track's own single `"start"` locator sits in world space** — which the existing code already resolves correctly for any track via `TrackNavigator` (see Q4). No per-track special-casing is needed for subtask 2's grid work.
`StreetRaceStartingGrid` is itself instantiated through a generic by-name component factory (`sub_2B8654`, registers the string `"StreetRaceStartingGrid"` → constructor `sub_2B87C4` via `sub_670758`, the same reflective-registration idiom already seen for other `Component`-derived classes) — i.e. it's a component on some Actor's shared race-FSM prefab (matching every checked event's `RaceFSMPrefabOverride` field, e.g. `"point_to_point_fsm_newintro"`), not something authored per-track or per-event.
### Q2 (is "player always last" an index or an insertion-order effect) — ANSWERED: insertion-order effect, in the racer-placement algorithm itself
`StreetRaceStartingGrid` overrides 4 of `RaceStartingGrid`'s 13 vtable slots (compared both vtables word-for-word: `off_AA7D28`/`off_AA7DA8`, slots 1/2/5/10 differ, slots 0/3/4/6/7/8/9/11 are inherited unchanged). Slot 5 is the deserializer above (Q1); **slot 10 (`sub_2B88BC`) is the actual placement algorithm**:
```
sub_2B88BC(this, raceContext, playerHandle, opponentsVector):
distance = 0.0
for (index, opponentEntry) in enumerate(opponentsVector): // 12-byte elements
PlaceCar(raceContext, opponentEntry, distance, LateralOffset(trackWidthFraction, index), OpponentsStartingSpeed)
distance += MinDistanceBetweenRacers + random(0,1) * (MaxDistanceBetweenRacers - MinDistanceBetweenRacers)
PlaceCar(raceContext, playerHandle, distance, 0 /* lateral: dead-center */, PlayerStartingSpeed)
```
The player is placed **once, after the loop, using whatever `distance` the loop accumulated** — not a fixed index and not an authored "last" flag. This is a structural consequence of the function's own two-phase design (place every opponent first, accumulating randomized spacing; place the player last, at the tail). **Practical implication for subtask 2**: real multiplayer players do not need to fight or bypass this rule at all — inserting them as entries in `opponentsVector` (the same vector `Opponent`/`OpponentCollection` builds, see Q3) gives them completely normal, correctly-spaced, correctly-jittered grid slots via the exact same code path a real AI opponent would get. Only the *local device's own* player stays in the dedicated last slot via the separate `playerHandle` call — which is fine, since each device's own player is already a distinct, privileged local entity (input/camera) regardless.
Lateral placement for opponents (the `LateralOffset(...)` call, inlined in `sub_2B88BC`) is a 3-lane zigzag: `((index+1) % 3) * 0.5 * trackWidthFraction + (1 - trackWidthFraction) * 0.5`, scaled by the track's actual width (resolved by `PlaceCar`/`TrackNavigator`, not baked into this formula) — so lane assignment naturally cycles through 3 lateral positions as the opponent index increases, using whatever `MaxTrackWidthFraction` (0.8 default) allows.
### Q3 (opponent car model + color write path) — ANSWERED: byte-precise live `Opponent` layout found, plus its owning `OpponentCollection`
Data-side confirmed first: every `event_*_race.prefabs.sb`'s `Opponent` struct schema is `DriverName` (string), `CarDescriptionName` (string, e.g. `"ford_mustang_boss_302_2012_desc"` — same format as the already-known `GetCurrentCarId()` result), `ColourIndex` (int32 — same field this project already reads for the *player's own* car via `LookupCarRecord`, cont.63), plus AI-tuning floats (`RacingLineScale`/`SpeedFactor`/`CorneringFactor`/`RubberBandingDifficulty`/`RubberBandingTweaksToUse`/`RubberBandingTargetDifficulty`/`PathfindingSkill`), `Stationary` (bool), `MaxHealth` (float). Every regular race event has exactly 5 `Opponent` entries (see Q1).
Found `Opponent`'s deserializer (`sub_2B5DD0`, located via the `"CarDescriptionName"` string xref) and its only caller, `OpponentCollection::PopulateFromProperties` (`sub_2B649C`, located via `"Opponents"` string xref) — both fully decompiled, giving the real live C++ layout (80-byte `malloc`, matches `Opponent`'s highest field offset `+76`+4):
| Offset | Field | Type |
|---|---|---|
| `+0` | vtable ptr | `off_AA7C78` |
| `+4` / `+8` / `+12` | `DriverName` | `{begin,end,capacity}` eastl string (same 12-byte string idiom as `RaceEvent`'s `TrackName`) |
| `+20` / `+24` / `+28` | `CarDescriptionName` | same string idiom — **the field to overwrite for opponent substitution** |
| `+36` | `ColourIndex` | plain `int` — **the field to overwrite for opponent color** |
| `+40..+75` | AI-tuning sub-object | built by `sub_2B4B08`/`sub_2B4B44`, not yet mapped field-by-field (not needed for subtask 2 — a substituted real-player slot doesn't need AI tuning) |
| `+76` | `Stationary` | `bool` |
`OpponentCollection::PopulateFromProperties` (`sub_2B649C`) reads the `"Opponents"` property (a map), and for each entry: `malloc(0x50)` a new `Opponent`, deserializes it via `sub_2B5DD0`, wraps it in a 16-byte refcounted handle (`vtable=off_AA7CCC, refcount=1, +12=Opponent*`), and `push_back`s that wrapper into a vector living on the `OpponentCollection` object itself at `this+12`/`+16`/`+20` (begin/end/capacity — standard vector, 8-byte/2-word elements).
**This is the exact vector `sub_2B88BC` (Q2) iterates to place opponents on the grid** — not yet cross-confirmed by directly tracing the argument at the call site (a remaining, low-risk verification step), but the shapes match exactly (`OpponentCollection`'s built vector of wrapped `Opponent*` vs. the grid placement loop's 12-byte-strided vector — the 12-byte stride matches the wrapper's own `{vtable-tag, Opponent*}`-plus-something shape closely enough to be the same data, needs one direct trace to fully nail down before implementing).
**Practical implication for subtask 2.1**: hook `OpponentCollection::PopulateFromProperties` (`sub_2B649C`), let it build the normal AI opponent list unmodified (matches this whole project's established "hook after, tweak fields" pattern — never originate, always let the real engine construct first), then for up to N real lobby players, overwrite the corresponding `Opponent` entries' `CarDescriptionName` (via `sub_7B524`, the same string-append/set helper already used for `RaceEvent.TrackName` in the track-substitution hook) and `ColourIndex` (plain int write) with that player's own already-captured `GetCurrentCarId()`/color-index values (`car_selection.h`). No grid-specific code needed — Q2 already established substituted entries get correct placement for free.
### Q4 (bonus — found while chasing Q2/Q3, directly relevant to subtask 4) — `TrackNavigator`'s real spline→world resolver
`PlaceCar` (`sub_2914DC`, own assert strings confirm the name: `"m_Navigator"`/`"PlaceCar"`/`"foundHeight"`, `"Navigator must be created before placing cars"`) is the single, universal placement primitive both the opponent loop and the player call use — signature `PlaceCar(gridThis, racerHandle, distanceAlongSpline, lateralOffset, startingSpeed)`. Internally:
1. `sub_3261B0(navigator, outBuffer, distanceAlongSpline, lateralOffset)` — **this is the real `TrackNavigator::Resolve` implementation**: reads a vector of 12-byte spline-segment records at `navigator+308`/`+312`, binary-searches for the segment containing `distanceAlongSpline` (`sub_327824`), then resolves world position + tangent direction (`sub_32273C`) into `outBuffer`. Exactly the "`distance_along_spline`, `lateral_offset` ↔ world `Vector3`" transform `ARCHITECTURE.md` §5 predicted subtask 4 would need — found here, as a side effect of subtask 2 work, not yet independently verified live but the decompile is unambiguous.
2. Resolves ground height via `sub_85660` (a raycast/height-query, own assert `"foundHeight"`).
3. Builds a heading quaternion from the resolved tangent direction (`atan2f`/`sinf`/`cosf`).
4. Writes the car's **initial velocity** as `direction * startingSpeed` directly into a physics component at a fixed sub-offset (`+320/+324/+328` off a resolved pointer chain) — confirms starting speed isn't just cosmetic, it's a real initial-velocity write.
5. Writes the resolved transform (position + heading quaternion) to **two** separate components (`sub_10B09C` and `sub_D5138` — likely physics/simulation vs. render/visual transform, not yet distinguished which is which).
**Not yet found**: the inverse resolver (world position → spline distance/lateral offset), which subtask 4 will need for reading a moving car's *current* position to broadcast over the network (this entry only found the forward direction, used for placement at race start). Worth checking `TrackNavigator`'s other vtable slots when subtask 4 starts.
### Task 4 (traffic/cop removal for multiplayer) — cops: exact scheduler found; civilian traffic: data-level path clear, runtime spawner not yet traced
**Cops**: found `SpawnCopCar` itself (`sub_F85B8`, own assert string confirms the name) — large (0x11a0 bytes), spawns+places+configures a cop actor, not fully mapped field-by-field (not needed). Found both its callers: `sub_F7E9C` is a **spawn scheduler/gate** — checks a cooldown timer (`a1+4020/+4028/+4032`) and an active-cop-count-vs-max check (`a1+4188+16`), and only calls `SpawnCopCar` if both pass. This is the natural hook point: skip `sub_F7E9C`'s body entirely (return early, no call to `SpawnCopCar` ever happens) when a multiplayer session is active — structurally identical to the already-proven `CopSoundsTick` skip hook (`crash_workarounds.h`), same low-risk pattern, just a different target function. (`sub_F8078`, the second caller, not yet decompiled — likely a second trigger context, e.g. scripted/pursuit-specific vs. ambient.)
Confirmed via RTTI that cops are a substantial, separate subsystem (`im::app::car::CopAICarController`, `CopAttackBehaviour`, `CopBustBehaviour`, `im::app::race::description::CopDescription` — its own `PrefabDatabase<CopDescription>`, mirroring `CarDescription`'s own pattern) — full understanding of cop AI behavior is out of scope for "just stop them from spawning," which the scheduler-skip hook achieves without touching any of that.
**Civilian traffic**: `RaceEvent.TrafficCarCount` (int, confirmed in every event's SB data, e.g. `1` for `event_05_race`) and a sibling `TrafficFlow` component (`MaxIncomingCarsOverride`/`MaxOutgoingCarsOverride`/congestion-distance tuning/`TrafficCarSpawnDescs`) fully describe ambient traffic density per-event, at the data level — same shape as every other per-race tunable already reverse-engineered in this project (`CashReward`, `MedalPosition`, etc.). The runtime spawner that actually consumes these fields was not traced this session (found `TrafficFlow`'s own deserializer, `sub_33A108`, but not its downstream spawn-trigger function) — lower priority than cops since the data-level override (set `TrafficCarCount` to `0` post-deserialization, same "hook after, tweak field" pattern as everywhere else) is very likely sufficient on its own and doesn't require finding the runtime spawner at all. Worth a quick live A/B test (does `TrafficCarCount=0` actually produce an empty road?) before investing further RE time here.
### Snapshot / Outcome
Pure research this session — **no code written, no hooks installed, nothing live-tested on device** (all findings are static SB-data inspection + IDA decompilation). All four of `ARCHITECTURE.md` §3b's open questions now have concrete, address-level answers; `ARCHITECTURE.md` §3b itself needs a rewrite to reflect this (queued as immediate next step). Remaining gaps before subtask 2 can be implemented: (1) directly confirm `sub_2B88BC`'s opponent vector argument really is `OpponentCollection`'s own vector (high confidence, not yet byte-traced), (2) decompile `sub_F8078` (cop spawn scheduler's second caller), (3) live-test whether `TrafficCarCount=0` actually suppresses civilian traffic, (4) map `Opponent`'s AI-tuning sub-object if a substituted real-player slot ever needs to suppress AI behavior explicitly (may not be necessary — a real player's own input should simply override AI control, but this hasn't been confirmed).
## 6ii. Subtask 2.1 — opponent car substitution CONFIRMED LIVE end-to-end, despite the two vectors being genuinely different objects
Direct follow-up to §6hh, same session (2026-08-26, continued autonomously per explicit instruction to keep working through open items without waiting). Implemented `opponent_substitution.h`: two diagnostic hooks on `OpponentCollection::PopulateFromProperties` (`sub_2B649C`) and `StreetRaceStartingGrid`'s placement method (`sub_2B88BC`), logging every pointer both touch.
**Confirmed live: the two vectors are genuinely different objects**, not the same data viewed two ways. `StreetRaceGrid::Place`'s own 12-byte-element vector held addresses (`0xb03xxxxx`/`0x7fcxxxxx` range) that never appeared anywhere in `OpponentCollection`'s own 8-byte-element vector (`Opponent*` values all in a `0xbc2xxxxx`/`0xd7dxxxxx`-range heap arena) across two separate live captures. There is a real intermediate step - likely a "spawn the actual racer Actor" stage - between `OpponentCollection` building its data-only `Opponent` list (confirmed to happen once, early, at **map load** for every nearby event, not per actual race start - all ~20-45 `OpponentCollection`s for surrounding events populate within about 1 second of the map screen appearing) and the grid actually placing physical, rendered cars at race start (confirmed to happen once, ~minutes later, exactly when the player actually starts a race).
**Despite that gap, overwriting `Opponent.CarDescriptionName`/`ColourIndex` at `Populate` time was confirmed, live, to reach the actual rendered car at the starting grid.** Test: hooked `Populate` to unconditionally force opponent-slot-0 of every collection to `"ford_focus_rs500_2010_desc"`/`ColourIndex=0`, rebuilt, replayed a real (non-synthetic) event ("Перед вами FAIRHAVEN", `event_05_race`-shaped, 5 opponents) end-to-end via `adb` UI taps through to the starting-grid cinematic. **Screenshot comparison**: same race, same car, before the hook showed a white sedan + red car at the front of the pack; after the hook, a silver/gray Ford Focus RS500 is unmistakably the lead car. Logcat confirms the in-memory write held (`CarDescriptionName="ford_focus_rs500_2010_desc" ColourIndex=0`) across every collection populated that map-load pass.
**Practical conclusion for subtask 2.1**: whatever the intermediate spawn step is, it reads car/color off the same `Opponent` objects `OpponentCollection` builds (or a copy taken strictly after `Populate` has already run) - it does not use some earlier-fixed snapshot from before `Populate`. This means the hook point identified in §6hh (write immediately after `OpponentCollection::PopulateFromProperties` returns) is sufficient on its own; no need to trace or hook the intermediate spawn step at all. The only remaining gap before this becomes a real (not blanket-test) feature is a data source: a lobby/session layer that can tell the hook *which* `OpponentCollection` belongs to the race the player is actually about to start, and *which* real player's car/color to write into which slot - neither exists yet (no lobby UI/data model built). The intermediate spawn step's own identity/location remains unknown and is not needed for this subtask.
**Snapshot**: `opponent_substitution.h`'s substitution write is gated behind `g_enableBlanketOpponentSubstitutionTest` (default `false`) - the live-tested build had it hardcoded on for this one test, reverted to gated-off before ending the session so normal play isn't affected. Diagnostic logging (both hooks, generous budget) stays on by default - low-risk, budget-capped, matches this project's established diagnostic-hook pattern.
## 6jj. Subtask 2.4 — cop-spawn scheduler hook implemented and installs cleanly; civilian traffic hook implemented and CONFIRMED LIVE (corrects this section's own earlier mistake)
- **Cops**: implemented `cop_traffic_disable.h`, hooking `sub_F7E9C` (the cop-spawn scheduler identified in §6hh) to skip its body entirely when a test flag is on. Confirmed live: installs cleanly (`Installed Cop spawn scheduler skip hook`), process stays stable through map/menu navigation, no crash. **Not yet confirmed**: the scheduler never actually fired during map/menu browsing in this test session (0 log lines from the hook while idle on the map) - it's very likely gated on active gameplay/a pursuit context, not a continuous background process, so a real visual "no cops spawned" confirmation needs an active-driving or pursuit-triggering test longer than this session's `adb`-tap-only navigation could practically set up. Structurally sound and ready; visual confirmation is the remaining step.
- **Civilian traffic — this section's own first pass was wrong, corrected by re-reading the same decompile more carefully**: this entry originally claimed `TrafficCarCount` needed a keyed reflective-lookup hook because `sub_2A4D70` (`RaceEvent`'s deserializer) reads it via `sub_4F99F0`/`sub_4F9A80`. On closer reading, that reflective read happens exactly **once**, during deserialization - identical in shape to every other field `sub_2A4D70` reads (`RaceType`, `Location`, `ClassRestriction`, etc.) - and the resolved value is then stored as a **plain int at a fixed offset**, `RaceEvent+116`, on the live object (`*(a1+116) = resolvedValue`, confirmed directly in the decompile, right next to the already-known `ClassRestriction`-at-`+132` pattern). The original §6hh plan (hook after deserialization, overwrite the fixed offset) was correct all along.
- **Implemented and confirmed live**: `cop_traffic_disable.h`'s second hook, on `sub_2A4D70` itself, overwrites `*(a1+116)` to `0` after the real deserializer runs. Live-tested: fired once per event at map-load time (same timing as `OpponentCollection::Populate`/§6ii), correctly zeroed real, non-trivial values (`1`, `2`, even `10` for one event) to `0`, no crash, process stayed stable. **Not independently visually confirmed** (no ambient traffic was visible in either the before- or after-hook starting-grid screenshot, but that's expected either way - traffic wouldn't render at a stationary pre-race moment regardless of the hook - a real visual check needs actual driving, not just reaching the grid).
- **Snapshot**: both `cop_traffic_disable.h` hooks gated off by default (`g_enableCopSpawnSkipTest`/`g_enableTrafficCarCountZeroTest`, both `false`), same pattern as `opponent_substitution.h`.
## 6kk. Subtask 2 — full multi-car/color roster AND random player grid position, both CONFIRMED LIVE simultaneously
Direct follow-up to §6ii/§6jj, same overnight session, per explicit request to emulate a fuller mock session (varied opponents + a non-last player slot).
- **Varied roster**: extended `opponent_substitution.h`'s blanket test from one fixed car to a 5-entry roster (`ford_focus_rs500_2010_desc`, `bmw_m3_e92_2009_desc`, `dodge_challenger_srt8_392_2011_desc`, `ford_mustang_boss_302_2012_desc`, `lancia_delta_hf_integrale_evoluzione_1993_desc`, distinct colour indices), cycling by opponent slot index in `Hook_OpponentCollectionPopulate`. Live-tested on the same real event replay: screenshot shows three visibly distinct cars (white BMW M3, white/blue Dodge Challenger, dark Ford Focus RS500) on the grid simultaneously, logcat confirms all 5 slots got their distinct roster entry, in order, every time.
- **Random player grid position — required reimplementing the placement algorithm, not just data tweaks**: §6hh Q2 already established "player always last" is a call-order artifact of `sub_2B88BC` (loop places every opponent first, accumulating randomized spacing, THEN places the player once at the tail in a separate call) - there's no field to flip for this, since the ordering itself is the behavior. Implemented a full reimplementation in `Hook_StreetRaceGridPlace` (gated behind `g_enableRandomPlayerGridPositionTest`, falls through to the untouched original otherwise): calls the same two real primitives orig uses - `sub_291BA4` (`PlaceOpponent`, keeps its own lateral-zigzag math) for opponents, `sub_2914DC` (`PlaceCar`) directly for the player (lateral=0, matching orig's own player call) - in a loop of `count+1` slots, with one slot chosen by `rand() % (count+1)` for the player and the rest going to opponents in original order. Distance accumulation mirrors orig's own shape (place, then advance by a random offset in `[MinDistanceBetweenRacers, MaxDistanceBetweenRacers]`) but uses plain `rand()` instead of replicating `sub_75680`/`sub_61C9F8`'s own RNG chain (seeded from the just-placed car's return value in a way not fully understood - not worth the risk for a test hook).
- **Live-tested successfully, simultaneously with the varied roster**: logcat showed `RANDOM GRID TEST: 5 opponents, player placed at slot 0/5` (i.e. the player was placed FIRST this run, not last) - no crash, process stayed stable through the same real-event replay used throughout this session. Both features work together: the same test run that produced the 3-distinct-cars screenshot also had the player at a randomized (non-last) slot.
- **Snapshot**: both extensions live in `opponent_substitution.h`, gated off by default (`g_enableBlanketOpponentSubstitutionTest`/`g_enableRandomPlayerGridPositionTest`, both `false`) before ending the test - device left in a safe, normal-play state.
- **Outcome**: this is now a fairly complete emulation of what a real multi-player race grid would look like (N distinct cars/colors, player at an unpredictable position) - purely as a local, single-device demonstration (all "opponents" are still AI, not real network peers). The only remaining gap to a real feature is, as before, the lobby/session data layer to source real per-player car/color/slot-order choices instead of this hardcoded test roster and `rand()`.
## 6ll. Subtask 2.4 — cop-spawn scheduler's real two-path shape found and CONFIRMED LIVE via a before/after roadblock; ambient-traffic spawner found, reducible but not fully chokeable
User caught a police car live, mid-race, in the exact same race that produced the random-grid-position spawn-collision bug documented in `PROGRESS.md` cont.73 - directly contradicting §6jj's "never fires" observation. That contradiction was the trigger to dig one level deeper on both the cop scheduler and (independently, on a hunch that the same "shallow hook" mistake might be repeated) the traffic hook.
**Cops — the missing second path.** §6hh/§6jj only decompiled `sub_F7E9C`. Decompiling its neighbor `sub_F8078` (previously unexamined) and both functions' own callers found the real shape:
```
sub_F5BB4 (CopManager::Update, per-tick, also runs unrelated bust-timer/etc. bookkeeping)
-> sub_F5EA4 (dispatcher, gated on `sub_33FF1C(a1)+88` byte + `a1+3736` byte + `sub_31A8A4(a1+3952)`)
branches on `*(byte*)(a1+4036)`:
true -> sub_F7E9C (cooldown-timer, single-candidate scheduler)
false -> sub_F8078 (distance-sorted candidate LIST, multiple checks/spawns per tick)
both leaves call sub_F85B8 (SpawnCopCar, own assert string confirms name)
```
`xrefs_to(0xF85B8)` confirms exactly 2 callers - `sub_F7E9C` (`0xf8018`) and `sub_F8078` (`0xf822c`) - so these two are the *complete* set of spawn paths, nothing else calls `SpawnCopCar`. `xrefs_to(0xF7E9C)` and `xrefs_to(0xF8078)` both resolve to the same single caller, `sub_F5EA4` (`0xf5f4c`/`0xf5f58` respectively) - confirming the dispatcher is the correct, minimal hook point that covers both leaves without touching `sub_F5BB4`'s other per-tick logic (bust timers etc., via `sub_F9D34`/`FA4F8`/`F6640`/`F67A8`/`F6BCC` - explicitly NOT wanted to be disturbed).
Fix: moved the hook from `sub_F7E9C` to `sub_F5EA4` (renamed `Hook_CopSpawnScheduler`→`Hook_CopSpawnDispatcher` in `cop_traffic_disable.h`), skipping the dispatcher's body entirely (return 0, discarded by the caller anyway) when `g_enableCopSpawnSkipTest` is on.
**Live A/B proof** (same event, "Битва на шоссе"/Reynolds Lane, replayed via "Начать заново" between runs):
- **Flag off** (old, single-leaf-hooked build, or the new build with the flag toggled off): drove through and hit a full, unambiguous police roadblock mid-race - multiple marked "POLICE"-liveried cars with light bars, cops standing in the road, a stop sign, on a completely plain street race with no Most-Wanted branding whatsoever. Screenshotted.
- **Flag on** (new dispatcher hook): replayed the identical route. Logcat showed `DIAG CopSpawnDispatcher: skipped` fire 7 times in a tight ~230ms burst right around the same point in the route (matching `sub_F8078`'s "check several distance-sorted candidates per tick" shape). The road was completely clear at the equivalent location - no roadblock, no cops, clean pass-through, screenshotted.
This is the strongest live confirmation in the project so far - an actual, reproducible visual difference on the identical content, not just an absence-of-crash or a logcat line.
**Ambient traffic — the same "shallow hook" mistake, found and only partially fixed.** §6jj's `TrafficCarCount=0` hook was re-tested by actually driving the race (not just checking logcat, which is what "confirmed" it before) - civilian traffic (a red pickup truck, a blue sedan) was still clearly visible with the hook active. Traced the real spawner:
- `sub_33D734` — `TrafficCarSpawner`'s populate function (own assert string: `"TrafficCarSpawner"`). Signature `(int a1/*this*/, int a2/*road context*/, char a3/*direction: 0 or 1*/, int a4/*ptr to a 4-float struct incl. spacing*/, int a5/*max candidate count*/)`. Builds a per-direction candidate list: for each road "node" in a waypoint/spline list (count `v66`, read from a road-context sub-object's own `+108/+112` list bounds - track-authored, nothing to do with `a5`), a `do { ...push candidate...; } while (++n < ceil(a5/(v66+something)))` loop runs. Because it's a `do-while`, **the body executes at least once per node regardless of `a5`** - forcing `a5=0` lowers the *additional* iterations to zero but can't prevent the guaranteed-first one.
- `sub_33C020` calls `sub_33D734` exactly twice - `(v7, a1, 0, v8, *(a1[3]+16))` and `(v17, a1, 1, v19, *(a1[3]+20))` - once per traffic direction, with each call's max-count read from offsets `+16`/`+20` on a *different* object (`a1[3]`) than `RaceEvent` (where `TrafficCarCount` lives at `+116`, per §6jj) - confirming `TrafficCarCount` was never the right field to begin with.
- `sub_2A8CE8` calls `sub_33C020` exactly once, at race setup (`xrefs_to(0x33c020)` → single caller) - meaning the two candidate lists `sub_33D734` builds are very likely the *entire* traffic roster for the race, built once, not a queue that's topped up per-tick the way cops are. This reframes the residual cars as this hook's structural floor, not a sign of an unrelated live spawner still running.
Fix implemented: `Hook_TrafficCarSpawnerPopulate` on `sub_33D734`, forcing `a5` to `0` when `g_enableTrafficSpawnerZeroTest` is on. **Live-tested**: logcat confirmed `maxCount=1 -> 0` and `maxCount=2 -> 0` for the two direction calls on the same event - and a single red pickup was still visible on-road, consistent with the `do-while` floor (1 node in one direction group, 2 in the other, before any hook involvement - already a small roster) rather than the hook silently failing.
**Left open, honestly scoped**: full elimination needs either (a) the track-waypoint-count source itself (harder - track-authored scene data, not a simple parameter), or (b) whatever consumes `sub_33C020`'s two candidate-list handles (stored at `a1[28]`/`a1[30]` on its own `this`) to actually instantiate world actors - not yet located. Worth checking that owning object's other methods for a read of those two slots before assuming a new decompile pass is needed from scratch.
**Snapshot**: `cop_traffic_disable.h` - dispatcher hook renamed and retargeted (`Hook_CopSpawnDispatcher`/`sub_F5EA4`); new `Hook_TrafficCarSpawnerPopulate`/`InstallTrafficCarSpawnerZeroHook` on `sub_33D734`, gated by new `g_enableTrafficSpawnerZeroTest`. All test flags reverted to `false` and a clean final build reinstalled before ending the session.
## 6mm. Subtask 2.4 — civilian traffic fully eliminated, CONFIRMED LIVE over a 2+ minute drive (resolves §6ll's open item)
Direct continuation of §6ll on explicit user instruction: keep digging into traffic, the goal is literally zero cars on the road, not just fewer. §6ll's `sub_33D734` do-while fix reduces the candidate list to a structural minimum but can never reach zero (the loop is `do { ...push candidate... } while`, so it always runs at least once per track-authored waypoint regardless of the max-count parameter) - this section traces one level further, past candidate-list *population* to actual world *placement*.
**Found `sub_C26A0` — the real placement primitive, a third confirmed user of the `PlaceCar`/`SpawnCopCar` write pair.** Own assert strings confirm the function's identity: `"foundTrackInfo"` and `"Reset"` (i.e. `CarReset`). Signature `(int a1, uint32_t* a2, char a3, int a4)`. Resolves a spline distance + lateral offset to a world position (via `sub_2B2D18`/`sub_85660`, the same `TrackNavigator`-family shape used throughout this project) and writes the result via `sub_10B09C`/`sub_D5138` - the exact same transform-write pair `PlaceCar` (`sub_2914DC`, §6hh) uses for the grid and `SpawnCopCar` (`sub_F85B8`, §6ll) uses for cops. `xrefs_to(0xC26A0)` returns exactly two callers:
- `sub_2A0470` — registered via `sub_31AAEC` (the same event-subscription idiom `SpawnCopCar` uses for its own `"EndOfTrack"` event, per §6hh) as the handler for a `"ResetLine"` event, one registration per traffic car (built in `sub_2BF4C4`, itself invoked - not traced further - once per traffic car object).
- `sub_C201C` — the traffic car's own **per-tick controller**. Second parameter is a delta-time-carrying struct (`*a2 * -0.001` appears twice, converting an integer millisecond tick into seconds). Calls `sub_C26A0` in two places: once when an idle/wander timer (a float field, counted down by delta-time each tick) reaches `<= 0.0` (computing a fresh target position via `sub_690A94`), and once, gated by a separate flag byte, to retry a *previously deferred* reset whose parameters were stashed in fields at `a1+112..128` - the exact same fields `sub_C26A0` itself writes on its own internal "couldn't resolve a position yet, save the request" fallback branch. This field layout (offsets 40/44/52/58/60/68/72/80/84/88/92/93/96/112-128) doesn't match `Opponent`, `PlaceCar`'s context object, or anything else touched by this project so far - a dedicated, traffic-specific per-car component.
**First hook attempt, live-tested and disproven**: hooked `sub_2A0470` on the reasonable-looking assumption that a `"ResetLine"` handler would cover both a car's initial placement and its ongoing recycling (analogous to how cops' single dispatcher covered both of *their* two paths in §6ll). Built with an **unconditional** diagnostic log (not gated by the test flag, specifically so its absence would be meaningful) and drove an actual race rather than just checking whether it installed. Result: the log never fired once during a ~1 minute race replay, yet a screenshot taken mid-race clearly showed the same red pickup truck from §6ll's testing, moving normally. This directly disproves the hypothesis for normal-length races - `sub_2A0470` isn't reached on whatever path keeps that traffic car active. (It *did* fire once, later, during the longer successful test below - so it's a real, occasionally-used path, just not the primary one.) Left installed as a harmless secondary hook, explicitly not relied upon.
**Real fix**: hooked `sub_C201C` itself, skipping its entire body (`g_enableTrafficControllerSkipTest`, returns early without calling orig or touching any of its fields). Since this is the traffic car's dedicated per-tick controller and the *only* other path into the actual position-write primitive, skipping it should leave that car's actor permanently un-positioned - never rendered moving on the road.
**Live-confirmed, unambiguous, over an extended drive**: replayed "Битва на шоссе"/Reynolds Lane - the same event that, across every prior test this session (§6ll and cont.73-74), reliably produced a red pickup truck in the tunnel section (~14s in) and a blue sedan later on a straight. With both traffic hooks active: screenshotted the tunnel section - completely clear, no pickup. Continued driving past 2 minutes total (well past where the blue sedan always appeared before) - still completely clear, no crash. The `sub_C201C` hook's own diagnostic log confirmed it firing continuously and rapidly (hit its 50-entry budget cap almost immediately after the race started, consistent with a genuine per-tick function), and the `sub_2A0470` hook fired exactly once late in the drive - both hooks coexisting without conflict.
**Snapshot**: `cop_traffic_disable.h` gains `Hook_TrafficCarControllerTick`/`InstallTrafficCarControllerSkipHook` on `sub_C201C` (the real fix, `g_enableTrafficControllerSkipTest`) and `Hook_TrafficResetLineHandler`/`InstallTrafficResetLineSkipHook` on `sub_2A0470` (secondary/harmless, `g_enableTrafficResetLineSkipTest`). `main.cpp` installs both unconditionally in `JNI_OnLoad` (runtime-gated by the flags, same pattern as every other test hook in this project). All flags reverted to `false`, final build reinstalled, before ending the session.
**Outcome**: subtask 2.4 is now fully solved on both halves - cops (§6ll) and traffic (this section) - each confirmed live via an actual before/after drive, not just a logcat line or an absence of crashes. The pattern that solved both: find the genuine per-tick controller/dispatcher for the subsystem and skip it wholesale, rather than trying to starve a data-driven candidate list or count field that turns out to have its own independent structural floor. Subtask 2 as a whole is now essentially complete pending only the lobby/session data layer; subtask 4 (coordinate/position sync) is the natural next major branch.
docs/ARCHITECTURE.md
+328
View File
@@ -0,0 +1,328 @@
# ARCHITECTURE.md — Multiplayer Mod for NFS Most Wanted (2012) Android
Living document. Revisit and edit whenever a design decision changes — do not let this drift out of sync with reality.
---
## 1. Goal recap
Layer local + dedicated-server multiplayer onto the existing singleplayer game, without breaking singleplayer. Lobby-based flow (map marker → enter lobby → player list → native car-select → ready-up → countdown → race), LAN discovery (SAMP-style broadcast) + internet servers, RakNet-based network layer with a separate headless C++ server (not a repurposed game runtime).
---
## 2. Integration layers (top to bottom)
```
┌───────────────────────────────────────────────────────────────────┐
│ launcher (Android APK wrapper) │
│ - ships the original game_cache unmodified │
│ - ships modified native libraries (or hooks them at runtime) │
│ - hosts the lobby UI (approach TBD-per-screen, see §4) │
└───────────────────────────────────────────────────────────────────┘
│ loads
▼
┌───────────────────────────────────────────────────────────────────┐
│ mpcore (native module, JNI_OnLoad entry point) │
│ - loaded AFTER libapp.so and friends (per task brief) │
│ - resolves libapp.so base via FindLibrary() │
│ - installs inline hooks (Thumb ARM32, armhook.cpp toolkit) │
│ - owns the RakNet CLIENT instance (peer, connects to a server) │
│ - bridges native <-> Java for lobby UI + car-select result │
└───────────────────────────────────────────────────────────────────┘
│ hooks into
▼
┌───────────────────────────────────────────────────────────────────┐
│ libapp.so (game engine, Iron Monkey, C++) │
│ - RaceLoaderTask / OpponentCollection / RaceStartingGrid / │
│ TrackNavigator / InRaceState (race lifecycle — see ANALYSIS.md) │
│ - Flow-machine (SB-scripted UI/state flow) │
│ - EAMText/GLES native text+UI renderer │
│ - PlayerCarController / AICarController family (car physics/AI) │
└───────────────────────────────────────────────────────────────────┘
│ (future, separate process/machine)
▼
┌───────────────────────────────────────────────────────────────────┐
│ Dedicated server (standalone, NOT libapp.so) │
│ - headless C++, links RakNet │
│ - authoritative-ish relay: lobby state, ready-check, position sync│
│ - local discovery: UDP broadcast responder (SAMP-style) │
│ - internet: direct connect / master-list registration (TBD) │
└───────────────────────────────────────────────────────────────────┘
```
Key architectural principle carried over from `ANALYSIS.md` §1.1: **we never originate `RaceLoaderTask`/race-start ourselves.** The player always starts a race through the game's own menu → Flow-machine → RaceLoaderTask path (any existing event works — see subtask plan). Our hooks intercept that *already-in-context* flow at specific points (opponent list build, starting grid, track navigator) to substitute multiplayer data. This keeps singleplayer's own code paths completely intact when no lobby session is active — hooks should be written to no-op/pass-through cleanly when there's no active multiplayer session.
---
## 3. Where hooks live and how they're installed
- **Injection point**: `mpcore`'s `JNI_OnLoad`, invoked after `libapp.so`/`libfmodex.so`/`libfmodevent.so`/`libnimble.so` are already loaded (confirmed as the intended design in the task brief and matches the existing `mpcore` draft's `FindLibrary("libapp.so")` call).
- **Hook mechanism**: reuse and extend `launcher/mpcore/src/main/cpp/util/armhook.cpp`. It already implements Thumb-mode (16-bit) inline hooking (`InstallHook`/`InstallJMPHook`/`InstallBLXHook`), vtable-slot overwrite (`InstallMethodHook`), and a register-copy code-injection stub (`CodeInject`), backed by a fixed trampoline region inside `libapp.so`'s own address space (`APP_ADDR(0x000A1B44)` sized `0x1A36`) plus an `mmap`'d PROT_RWX scratch page for original-bytes preservation. This is a working foundation, not a rewrite target — see ANALYSIS.md §6 for what's already verified.
- **Addressing**: `APP_ADDR(offset)` = `libapp_base + offset`, and the IDA database's addresses are already base-`0x0`-relative (i.e., IDA addresses == file offsets == `APP_ADDR` argument, no translation needed). Verified this session via the `0xC8C9D8` string anchor (see ANALYSIS.md §6).
- **Two hook strategies available depending on the subsystem** (decide per hook site once its real vtable/function is located):
1. **Inline/trampoline hook** on a free-function or a specific vtable-slot's target — for functions we want to wrap (call original, then modify results), e.g. `OpponentCollection::Build`.
2. **Direct vtable-slot replacement** (`InstallMethodHook`) — simpler when we want full control and don't need the original implementation, e.g. potentially `TrackNavigator`'s position resolver if we decide to fully own opponent positioning instead of post-processing it.
---
## 3a. Working hook reference: `RaceLoaderTask_BuildTrackScenePath` (arbitrary track substitution)
The first fully working, validated hook (mechanically and visually confirmed — see `ANALYSIS.md` §6g–§6o and `PROGRESS.md`'s 2026-08-05 entries). Lives entirely in `launcher/mpcore/src/main/cpp/main.cpp`, ~95 lines, no changes anywhere else. Use this as the template for future hooks (opponent list, starting grid, coordinate sync) — same trampoline pattern, same "read the struct, repoint a couple of pointers, call through to the original" shape.
**What it does**: whichever race event the player starts through the normal menu flow, this hook makes it load a *different*, hardcoded track's geometry instead of the event's real one — while everything else about the event (rewards, opponent count, car-class restriction, HUD) stays as designed. Confirmed with a live A/B test: same event, same car, same opponents — different city, different road layout, different time of day, purely by flipping the hook on/off.
**Target function**: `RaceLoaderTask_BuildTrackScenePath`, `libapp_base + 0x2a8424` (`BUILDTRACKSCENEPATH_OFFSET`). ARM-mode (`PUSH {R4-R11,LR}; ADD R11,SP,#0x1C`), both displaced instructions are position-independent, so it uses a **trampoline hook**: an `mmap`'d RWX page holds the two displaced instructions followed by a jump back to `target+8`, and the function's first 8 bytes are overwritten with `LDR PC,[PC,#-4]` + the hook's address. `Hook_BuildTrackScenePath` runs first, then always calls through to `orig_BuildTrackScenePath` — this is a wrap, not a replace.
**`RaceDefinition` struct offsets used** (confirmed live, not guessed — see `ANALYSIS.md` §6j/§6n for how each was found):
| Offset from `raceDef` | Field | Notes |
|---|---|---|
| (a1+32 → deref) | `raceDef` pointer itself | `a1` is the `RaceLoaderTask`; `raceDef = *(void**)(a1+32)` |
| `+72` / `+76` | track name `{begin,end}` char pointers | e.g. `"region1_foothills_track4"` — this hook overwrites both |
| `+100` / `+104` | environment name `{begin,end}` char pointers | e.g. `"foothills"`-style — **not** the same string as the track name; see naming convention below |
| `+164` / `+168` | `m_StartLine` actor name `{begin,end}` | generic across tracks (`"start"`) — safe |
| `+180` / `+184` | `m_FinishLine` actor name `{begin,end}` | **generic for regular races** (`"finish"`), but **per-event custom** for time-trial events (e.g. `"event_02_finish"`) — this is why time-trial events are out of scope, see below |
| `+196` / `+200` | `m_EndOfTrack` actor name `{begin,end}` | generic (`"end_of_track"`) |
| `+212` / `+216` | checkpoint-container actor name `{begin,end}` | empty for regular races; per-event custom for time-trial (e.g. `"checkpoints_timetrial_event_2"`) |
Only `+72/+76` and `+100/+104` are overridden by this hook. The original string buffers are deliberately leaked (never freed) — `BuildTrackScenePath` only reads them, and avoiding a free sidesteps needing to know an unconfirmed capacity/allocator-owner field.
**Environment-name gotcha**: environment prefabs are **per-track-numbered** files (`chicago1.prefabs.sb` … `chicago6.prefabs.sb`, matching `region4_chicago_track1`…`track6`), not one generic `<region>.prefabs.sb` per city. Setting the env name to just `"chicago"` fails to open the prefab database and crashes immediately. The env-name string must match the specific track number, not just the region (`"chicago4"` for `region4_chicago_track4`). The one region that doesn't follow this convention (`region3`/colorado, shipped as a single un-numbered `colorado.prefabs.sb`) turned out to be cut/incomplete content with no loadable geometry at all — see `ANALYSIS.md` §6n before picking a new target track.
**Picking a different target track**: edit `kOverrideTrackName`/`kOverrideEnvName` near the top of `main.cpp` to any `region{N}_{city}_track{M}` / `{city}{M}` pair that's confirmed to exist under `game_cache/published/prefabs/tracks/` and `game_cache/published/prefabs/environments/` respectively (and has a `game_cache/published/models/environments/{city}/` folder — the colorado lesson).
**Toggle flag**: `kEnableTrackSubstitutionHook` (`constexpr bool`, just above `JNI_OnLoad`) — `true` installs the hook, `false` runs the game completely unmodified. Flip, rebuild (`./gradlew :app:assembleDebug`), reinstall — no other code changes needed. Exists specifically so before/after comparisons don't require repeated edits to `JNI_OnLoad` itself.
**Known limitation — regular races only** (explicit project-scope decision, not a bug): time-trial and single-opponent/pursuit-style events reference their finish-line and checkpoint-group actors by the per-event custom names shown in the table above, which only exist in that event's *original* scene. Substituting the track for those event types either does nothing useful or crashes on a missing-null-check in `RaceLoaderTask_ResetStartingLine` (`GetComponent<Checkpoint>()` on a NULL actor lookup — a genuine pre-existing engine bug, not ours). This hook is only intended to run for regular point-to-point races. See the `track-substitution-scope` memory and `ANALYSIS.md` §6n for the full trace.
**Standing gap, not yet closed**: this hook currently fires for *every* race load, gated only by the build-time `kEnableTrackSubstitutionHook` flag (default `false` since `PROGRESS.md` cont.66's fresh-save prologue crash) — it has no runtime check for "is a multiplayer session actually active." Before this can run as part of normal multiplayer play, it needs to be re-scoped to only substitute the track when a real lobby session is active, not left as an always-on-or-always-off toggle.
---
## 3b. Subtask 2 — opponent substitution, starting grid, and traffic removal
Scoped 2026-08-25; all four open questions answered via RE 2026-08-26 (`ANALYSIS.md` §6hh — full addresses/offsets/decompiles there, this section is the implementation-facing summary). **[RE complete, no code written/tested yet]** — this is the actual "make N real players race together" mechanism; everything proven so far (§3a's track substitution, the car/color/class/upgrade capture in `car_selection.h`/`mod_slot_tracking.h`) is supporting infrastructure around a single local player's own car-select flow, not this.
**Mode scope**: street race only, consistent with §3a's "regular races only" decision.
### 2.1 — Opponent substitution **[mechanism CONFIRMED LIVE — cont.70]**
`OpponentCollection::PopulateFromProperties` (`sub_2B649C`) builds a vector of `Opponent` objects (80 bytes each) from `RaceDefinition` SB data, one per AI opponent (always 5 for a regular race — confirmed across 15+ events, every track). Live `Opponent` layout: `+20/+24/+28` = `CarDescriptionName` (eastl string, same idiom as `RaceEvent.TrackName`), `+36` = `ColourIndex` (plain int). Implemented and live-tested (`opponent_substitution.h`): hooking `sub_2B649C`, letting it build the normal AI list unmodified, then overwriting an entry's `CarDescriptionName`/`ColourIndex` **is confirmed to reach the actual spawned, rendered car at the starting grid** — screenshotted a forced Ford Focus RS500 replacing a real opponent's original car on a real (non-synthetic) event replay. This holds despite `StreetRaceStartingGrid`'s own placement vector (2.2) being confirmed, live, to hold **different heap objects** than `OpponentCollection`'s vector — there's an unidentified intermediate spawn step between them, but it reads car/color off these same `Opponent` objects (or a copy taken after `Populate` already ran), so hooking here is sufficient; the intermediate step itself doesn't need to be found. See `ANALYSIS.md` §6ii for the full live-test writeup. **What's left**: this is currently a blanket test (forces opponent-slot-0 of every race to the same fixed test car, gated off by default via `g_enableBlanketOpponentSubstitutionTest`) — turning it into the real feature needs a lobby/session data source (which race is this, which real player's car/color goes in which slot), which doesn't exist yet since no lobby UI/data model is built.
### 2.2 — Starting grid **[RE complete — turned out simpler than expected]**
Both open questions resolved, and favorably: the grid is **not per-track data** — `StreetRaceStartingGrid`'s 5 spacing/speed parameters (`MinDistanceBetweenRacers`/`MaxDistanceBetweenRacers`/`MaxTrackWidthFraction`/`PlayerStartingSpeed`/`OpponentsStartingSpeed`) are hardcoded C++ constants, identical for every regular street race regardless of track (confirmed: no event's SB data ever overrides them). Each track only supplies a single `"start"` locator actor (confirmed via two full track scenes, no per-slot/numbered spawn actors exist) — `TrackNavigator` resolves that plus a computed `(distance, lateralOffset)` into a world position (see 2.5 below), so the same code naturally adapts to any track's geometry with zero special-casing needed.
"Player always last" is **not a fixed index** — it's a structural side effect of the placement algorithm (`sub_2B88BC`): it loops over the opponents vector first (accumulating randomized spacing between each), *then* places the player once, at the tail, in a separate call. **Practical upshot: multiplayer doesn't need to fight this rule at all.** Real players substituted into the opponents vector (2.1) get completely normal, correctly-spaced slots via the same code path as any AI opponent — only each device's own local player keeps the dedicated "always last" slot, which is fine since the local player is already a distinct, privileged entity (input/camera) regardless of multiplayer.
**Cont.73 pitfall, worth remembering**: a test reimplementation that inserted the player mid-sequence (instead of at the tail) reproduced a real spawn-inside-another-car collision live. Root cause: `TrackNavigator::Resolve`'s lane bounds are symmetric around 0, so the player's hardcoded lateral offset (0, dead center) exactly coincides with an opponent's own lane-cycle position whenever that opponent's index is `≡ 0 (mod 3)`. Vanilla is safe only because the player is always maximally far (past the whole accumulated distance) from every opponent — never adjacent. Any future code that changes *where* the player is inserted into this sequence must widen the longitudinal gap around that insertion point, not just reuse the normal opponent-to-opponent spacing (sized for adjacent-different-lane cars, not same-lane).
### 2.3 — Spawn order policy **[unchanged — still a design decision, not an RE question]**
Since 2.1/2.2 both resolve to "just insert real players as opponent-vector entries," the policy question becomes simpler: which N of the lobby's players get inserted, and their relative order within the vector (which determines lane-zigzag assignment and grid-row spacing, not "who's on the grid at all" — everyone inserted gets a normal slot). Default for the first version: alphabetical by display name, implemented as an isolated `ResolveGridOrder(lobbyPlayers) -> orderedList` function, swappable later without touching the hook itself.
### 2.4 — Remove police and civilian traffic **[CONFIRMED LIVE, both cops and traffic]**
**Cops — CONFIRMED LIVE** (cont.74): the original `sub_F7E9C` hook missed a second spawn path. Real shape: `sub_F5EA4` is a per-tick dispatcher (called only from `sub_F5BB4`, CopManager's broader Update - too broad to skip wholesale since it also runs unrelated bust-timer bookkeeping) that branches on a live flag to either `sub_F7E9C` (the originally-hooked leaf) or `sub_F8078` (a second, previously-unhooked leaf - a distance-sorted-candidate-list scheduler). Both leaves call the same `SpawnCopCar`/`sub_F85B8` (confirmed exactly 2 callers via `xrefs_to`). Hooking the dispatcher (`sub_F5EA4`) itself covers both leaves with one hook, without touching `sub_F5BB4`'s other per-tick maintenance. **Live A/B proof**: same event replayed 3x - flag off hit a full marked police roadblock mid-race (screenshotted: light bars, "POLICE" livery, cops standing in the road) on a plain street race with no Most-Wanted branding; flag on (dispatcher hook), same route, dispatcher fired 7x in a tight burst at the same point and the road was completely clear. Gated off by default (`g_enableCopSpawnSkipTest`).
**Civilian traffic — CONFIRMED LIVE, zero traffic achieved** (cont.75, resolving cont.74's open item): the `sub_33D734` candidate-list hook (below) can only reduce traffic to a structural minimum, never zero, so the real fix needed the actor's own placement path, not its candidate-list population. Traced `sub_C26A0` (`CarReset` - own assert strings confirm "foundTrackInfo"/"Reset") - resolves a spline distance + lateral offset to a world position and writes it via the same `sub_10B09C`/`sub_D5138` transform-write pair used by `PlaceCar` (2.2) and `SpawnCopCar` (2.4/cops) - i.e. the actual moment a traffic car's position becomes real, analogous to those two. It has exactly two callers: `sub_2A0470` (a `"ResetLine"` event handler, registered per traffic car via `sub_31AAEC` - same idiom `SpawnCopCar` uses for its own `"EndOfTrack"` event) and `sub_C201C`, the traffic car's own **per-tick controller** (second parameter carries a delta-time value; calls `sub_C26A0` once when an idle/wander timer expires, once to retry a previously-deferred reset stored in fields `sub_C26A0` itself writes on its "can't resolve yet" fallback path - i.e. genuinely traffic-car-specific, not a generic car utility shared with the player/opponents). First hooked `sub_2A0470` on the assumption it also handled initial placement - **live-tested and disproven**: its own unconditional diagnostic log never fired once across a full race replay, yet traffic was clearly visible and moving the whole time, so it isn't on the path that matters for a normal-length race (left in, harmless, but not the fix). Hooking `sub_C201C` instead (skipping its body wholesale, `g_enableTrafficControllerSkipTest`) **is** the fix: **live-confirmed zero traffic across a full 2+ minute drive** on the same route that previously showed a persistent red pickup truck and a blue sedan - clean road throughout, no crash, hook fired continuously (~50+ times, log budget exhausted) as expected for a per-tick controller.
**`sub_33D734` do-while structural-minimum hook** (superseded as the primary fix by the above, but harmless to keep): `TrafficCarSpawner`'s populate function (own assert string confirms the class name), called twice per race from `sub_33C020` (once per traffic direction), max-count read from a different object (`*(a1[3]+16)`/`*(a1[3]+20)`) than `RaceEvent.TrafficCarCount`. Forcing that max-count to 0 (`g_enableTrafficSpawnerZeroTest`) can't reach zero alone because the populate loop is a `do { ... } while`, unconditionally queuing at least one candidate per track waypoint regardless of the max-count parameter - this is what motivated tracing one level further to `sub_C26A0`/`sub_C201C` above.
### 2.5 — Bonus finding: `TrackNavigator`'s real spline→world resolver (relevant to subtask 4)
Found while tracing grid placement: `PlaceCar` (`sub_2914DC`) calls `sub_3261B0(navigator, outBuffer, distanceAlongSpline, lateralOffset)` — the real, working `TrackNavigator::Resolve` implementation §5 already predicted subtask 4 would need. Only the forward direction (spline→world) is confirmed; the inverse (world position → spline distance, needed to read a moving car's *current* position for network sync) hasn't been located — worth checking `TrackNavigator`'s other vtable slots when subtask 4 starts. See `ANALYSIS.md` §6hh Q4.
### Remaining before implementation
1. Directly confirm `sub_2B88BC`'s opponent-vector argument is really `OpponentCollection`'s own built vector (high confidence from matching shapes, not yet byte-traced).
2. Only if a substituted real-player slot needs it: map `Opponent`'s AI-tuning sub-object (`+40..+75`) to explicitly suppress AI behavior — may not be necessary if a real player's own input naturally overrides it.
---
## 4. Lobby UI & flow — see LOBBY_UI_DESIGN.md
Everything about the lobby overlay's screens, flow, data schema, player identity/profile, IP:port addressing, rewards validation, and ready-state UX now lives in a dedicated file — **[LOBBY_UI_DESIGN.md](LOBBY_UI_DESIGN.md)** — kept separate so this document stays focused on native/engine integration. Clickable prototype: **[LOBBY_PROTOTYPE.html](LOBBY_PROTOTYPE.html)**.
One decision from that doc is load-bearing here too, since it's really a native-engine question: the wait for slow-loading peers happens **at the engine level**, not on a custom Compose screen — a client that's ready sits in the game's own native pre-race starting-grid scene (looping, not our UI) with only a thin overlay status strip on top, and the real 3-2-1-GO countdown is held until everyone's in (or the 45s hard timeout). See LOBBY_UI_DESIGN.md §7 for the full reasoning and the still-open hook work this needs.
<details>
<summary>Archived: original §4-4c content (superseded by LOBBY_UI_DESIGN.md, kept here for history)</summary>
### 4. Lobby UI decision (original)
**Revised recommendation (updated after correcting ANALYSIS.md §3.3): use a hybrid.** Full interactive lobby screens (player list, ready-up, countdown) still go through option (a) — an overlay Android `View` on top of the game's `GLSurfaceView` — because that pipeline has no native concept of touch-driven widgets. But simple in-world/in-menu indicators (the green/red map markers for active lobbies) should go through the **`BitmapGraphics` native text/atlas bridge** instead of a separate overlay, since it's verified, already trusted by the game's own renderer, and requires no new Flow/SB authoring.
Corrected reasoning (this session confirmed, via decompiling the actual JNI call sites, that `com.ea.ironmonkey.BitmapGraphics` — already reverse-engineered as `launcher/app/src/main/java/com/ea/ironmonkey/BitmapGraphics.kt` — is a real, live native→Java upcall bridge: `Canvas.drawText` into an offscreen `Bitmap`, read back via `AndroidBitmap_lockPixels`, and blitted into a native GLES font-atlas texture every time UI text is drawn. See ANALYSIS.md §3.3 for the full call chain and renamed IDA functions):
| Concern | Map markers → `BitmapGraphics` extension | Full lobby screens → Android overlay `View` |
|---|---|---|
| Mechanism | Add a method (e.g. `drawMarker(x, y, color)` via `canvas.drawCircle`) to `BitmapGraphics.kt`; call it from a hook alongside the existing `drawString` JNI call site (`0x5625a0`) | Standard Android `View`/Compose overlay added to the activity's view hierarchy above/around the `GLSurfaceView` |
| Implementation cost | Low — one new Kotlin method, one new hook call from `mpcore` at the point we know a lobby is active | Low — standard Android UI, fast iteration |
| Visual integration | Genuinely native — rendered through the exact same atlas-blit path as the game's own UI text, in the game's own coordinate space | Good but technically a separate layer composited over/under the GL surface |
| Touch/interaction support | None — this pipeline only produces pixels in a texture; no input handling | Full Android input handling, exactly what's needed for player list + ready buttons |
| Risk to singleplayer | Low — purely additive draw call, no-ops when no lobby is active | None — overlay is purely additive, toggled only when a multiplayer session exists |
| Car/upgrade display integration | N/A | We don't render the car ourselves either way — invoke the game's existing native car-select screen (same one singleplayer uses) as a sub-flow, our overlay only wraps around it |
Both are additive and don't touch Flow/SB authoring, so neither destabilizes the shared menu system. This split (markers via the verified `BitmapGraphics` path, interactive screens via overlay `View`) is still a **preliminary** call — revisit once the "Choose Car" native hand-off and a first marker hook are actually implemented. Record any reversal here with reasoning.
Still open: whether `EAMText`/`GlyphMeshGLES`/FreeType (also present in the binary, see ANALYSIS.md §3.3) plays any role that would interfere with or duplicate a `BitmapGraphics`-based marker — not yet determined, low priority until the marker hook is actually attempted.
---
## 4a. Lobby flow decision and data schema
**Flow decision:** Scenario 1 — Compose/View overlay for lobby creation and management, native car-select (filtered by the lobby's class setting) as a sub-flow, ready-checkmark to confirm. The "lobbies as cards on real street pins" idea (Scenario 2) is shelved: it depends on hooking into visible-street `MapTrack` event population, which the [[visible-streets-investigation]] found no viable path for after exhausting `AddEvent`/`Career`/`Progression`/property-registry. Scenario 1 needs no further RE — arbitrary track loading (§3a) and the native car-select screen already work or are known-reachable.
**Data schema** (pure data shape, no UI/transport concerns):
```
Lobby {
id: string
hostPlayerId: string
trackId: string // arbitrary track, see §3a BuildTrackScenePath hook
classRestriction: int | null // car class filter applied to native car-select; null = any
rewards: { // reuses the CashReward shape found on RaceEvent, ANALYSIS.md §6aa
bronze: int
silver: int
gold: int
}
players: [LobbyPlayer]
}
LobbyPlayer {
playerId: string
displayName: string // from the local PlayerProfile, see §4c — sent to peers on join
avatarId: string // preset id ("color+icon"), see §4c — not a photo/upload
isLeader: bool // explicit flag, not inferred from hostPlayerId match —
// keeps leader-ness a first-class, UI-visible fact instead of
// something every screen has to re-derive by comparison
ready: ReadyState
carId: string
colourIndex: int
mods: [int | null, int | null] // exactly 2 upgrade slots; each is a CarMod ModType id
// (or null = "НЕТ"/empty). Purely for display — other players
// resolve name/icon/description client-side from the car's own
// CarDescription.Mods catalog, already loaded for the garage
// screen. No numeric value/balance computation server-side.
}
enum ReadyState {
SetsUpLobby // leader only — track/class/rewards configuration
ChoosingCar
ChoosingRace
Loading // race-start fired, client is loading its own RaceLoaderTask
NotReady
Ready
}
```
Confirmed against the live in-game upgrade screen (2 slots, `ВЫБОР УЛУЧШЕНИЯ` list) and the `CarMod` struct in `alfaromeo_4c_2012_desc.prefabs.sb` — 7/7 checked entries matched by price (`ModType` 1/4/5/6/7/8 confirmed, `TYRES_REINFLATING`/`CHASSIS_REINFORCED`/`BODY_IMPACT`/`POWERTRAIN_POWERPACK`/`NITROUS_BURN`/`NITROUS_EARN`). `ModType` is a unique per-option id, not a shared category grouping — an earlier hypothesis to that effect is superseded. The separate `im::app::car::CarPart`/`UpgradeParts` catalog (with `skipCost`/`orderTimeMinutes` timer-gating fields) has no live instance data in any checked file and appears unused in this build — not part of this schema.
**Resolved (2026-08-18, see §4b for reasoning):**
- Host-leaves-lobby: auto-promote the longest-connected remaining `LobbyPlayer` to leader; dissolve only when the lobby becomes empty. Keeps a session alive across a flaky host connection instead of punishing everyone else.
- `classRestriction`: **enforced explicitly, at the overlay layer, not the native carousel** (revised 2026-08-19) — car_select has no known native class-filter hook (`RACEEVENT_CATEGORYTAG_OFFSET` was tried and is a confirmed false lead, PROGRESS.md cont.58), and rather than leave that as an excuse to only show the restriction as a label, the client-side flow explicitly checks the chosen `carId`'s class against `classRestriction` right after `onCarSelected` fires: a mismatch **blocks** `ready` (visible error state, forces the player back into car_select) instead of just being advisory. Revisit as a true hard filter — blocking the native carousel itself before a mismatched pick is even possible — if the real class-filter mechanism is ever found (cont.59/60's open leads).
**Still open, needs a product decision:** the straggler/timeout policy during the loading phase — see §4b step 5.
---
## 4b. Full lobby flow (designed 2026-08-18)
Every step below is tagged **[proven]** (already implemented and live-tested on device per PROGRESS.md), **[wired, needs new UI]** (the native hook/JNI plumbing exists, only the lobby overlay code calling it is missing), or **[open]** (real engineering work not started). This distinction matters more than usual here because two of this project's biggest proven results — the track-substitution hook (§3a) and the car/color/upgrade capture (§3a of PROGRESS.md cont.58-63b) — were each built against a *different* entry point (a real, normal race event) than the one the synthetic "LAN: `<lobby>`" event currently reaches, and those two paths are not yet joined. See step 5 for why that matters.
### Phase 0 — Entry point **[open, trivial]**
A small persistent button (e.g. bottom-corner Compose overlay, always present once `onMapLoaded` has fired at least once) opens the lobby overlay full-screen on top of `GameGLSurfaceView`. Same `FrameLayout`-stacking pattern already used for `buildCarSelectionOverlay()` (`GameActivityMain.kt`) — the GL surface keeps rendering underneath (map stays visible/animated), our Compose content just takes over touch input while open.
### Phase 1 — Create or join **[open — no networking yet]**
- **Create**: leader picks lobby name, track (any `region{N}_{city}_track{M}` confirmed to exist per §3a's own caveats — reuse that same validated list, do not hand-author a new one), optional `classRestriction`, reward tier (`CashReward`, ANALYSIS.md §6aa — real engine struct, not invented). Leader becomes `LobbyPlayer{isLeader=true, ready=SetsUpLobby}`.
- **Join**: LAN broadcast discovery (ARCHITECTURE §5, SAMP-style UDP) or direct IP:port. Neither transport exists yet — RakNet integration hasn't started (§5). The prototype (see chat) stands this phase up with **local mock state only** (no real second device) until RakNet lands.
### Phase 2 — Player list / waiting room **[open UI, wired data underneath]**
Real-time roster: name, ready-state icon, chosen car icon + colour swatch (same visual as the existing `CarSelectionBadge`, just one per row instead of a single corner badge), leader crown. Leader additionally sees "Начать гонку", disabled until every `LobbyPlayer.ready == Ready`.
### Phase 3 — Native car-select hand-off **[wired, needs new UI]**
This is the one phase where the hard native work is already done:
- Tapping "Выбрать машину" calls `MultiplayerCore.triggerTrueDirectCarSelectJump()` — **[proven]**, jumps straight into the real native car_select screen for our synthetic event, no map navigation needed (PROGRESS.md cont.48/57). Fallback to `MultiplayerCore.triggerCarSelectTest()` (opens EventDetails first, then auto-skips — also **[proven]**) if the true-jump's own warm-up precondition (`g_realEventDetailsVisitHappened`, i.e. at least one real event_detail→car_select transition already happened this session) hasn't been satisfied yet — the "cold session" gap noted in cont.55/57, still open. The lobby overlay should shrink to a small corner badge (not fully dismiss) while this runs, matching how `CarSelectionBadge` already coexists with the game's own UI.
- Player interacts with the fully native, unmodified car_select + loadout screens (real rendering, real touch — by design, ARCHITECTURE §4 already decided this stays native).
- `onCarSelected(carId, colorName, r,g,b,a)` — **[proven, wired to Kotlin]** — fires the moment car_select's own CONTINUE is tapped (cont.61/63b).
- `onUpgradesAccepted(slotIds, carMods)` — **[proven, wired to Kotlin]** — fires when the loadout screen's controlled BACK×3 exit lands back on the map (cont.39-41).
- The lobby overlay listens for `onUpgradesAccepted` (or, if the player has no mods to set and the loadout screen is skipped, the underlying "back on map" signal that same hook chain already detects) as the "player is done, back to lobby" trigger, restores itself to full view, and fills that player's roster row from the already-captured `CarSelectionState`. Player then taps a "Готов" toggle → `ready = Ready`.
### Phase 4 — Ready-check & start **[open — needs RakNet]**
Leader's "Начать гонку" broadcasts the final race definition (track id, per-player car/colour/mods) once every player is `Ready`. No transport exists yet to carry this — it's a data-shape problem only right now (the `Lobby`/`LobbyPlayer` schema above already covers what needs to go over the wire).
### Phase 5 — Loading, and the "someone's still loading" problem **[mostly open — the real gap in this whole flow]**
Two unresolved issues sit here, and they're different in kind:
1. **Which race event actually carries the load.** The synthetic `RaceEvent` used to reach car_select on demand (step 3) has its loadout-confirm **deliberately redirected back to the map today**, specifically because it has no real track/scene reference — letting it proceed into `RaceLoaderTask` crashes on a NULL start-line lookup (PROGRESS.md cont.36, confirmed root-caused via cont.41's A/B: a *real* event's loadout-confirm sails through to an actual playable race with zero crashes, using the exact same code path). Giving the synthetic event a real track/scene reference is flagged in cont.37/41 as "a materially bigger investigation than anything solved so far." **Recommended path, not yet implemented**: don't fight that — have the lobby's "start" instead re-target a *real*, existing race event as the vessel (any regular point-to-point event works, since §3a's track-substitution hook already proves the geometry can be swapped after the fact), and let that real event's own loadout-confirm proceed normally instead of being redirected. This sidesteps the missing-track-scene crash entirely instead of solving it. Needs its own short investigation (mostly wiring, since both halves — real-event loadout-confirm passthrough, and track substitution — already independently work) before it can be marked proven.
2. **Holding a fast-loading client at the start line.** Nothing hooks the moment between "this client's `RaceLoaderTask` finished" (candidate signal: `InRaceState` construction, RTTI-confirmed §3.1, or `RaceLoaderTask_DispatchInitialFSMEvents`'s final event, §6b stage 8 — either is a real, already-located anchor, just not yet hooked for this purpose) and "gameplay actually starts ticking." **This needs new hook work**: freeze input/sim on that client once loaded, show a small "Ожидание игроков (2/4)" overlay (same Compose layer, minimized), and only release once the host broadcasts "everyone's in." `RaceLoaderTask_ExecuteLoadSequence`'s own `SetLoadProgress(this, N)` calls (§6a slot 4, fractions 0.1–0.8) are a good source for a real, granular per-client progress percentage to report to peers, rather than inventing a synthetic one.
3. **Straggler/timeout policy — resolved 2026-08-18**: hard timeout, auto-start. Default 45s (configurable) counted from when the first client reports fully loaded; whoever hasn't reported loaded by then is left behind and the race starts without them. Simplest to implement/reason about, no host-facing decision UI needed — accepted tradeoff is occasionally starting a race short a player on a slow device. Still needs the actual per-client "loaded" signal (point 2 above) and a decision on how a left-behind player is represented (spectator vs. rubber-band-in-when-ready) — lower priority than the timeout mechanism itself.
---
## 4c. Player identity (name + avatar) — added 2026-08-18
Raised as a gap while prototyping the lobby screens: every player row needs *something* to display before any of them have picked a car, and it can't come from an EA account — ARCHITECTURE §8 already rules out the Synergy backend for anything multiplayer-related, and this project has no login system of its own.
**`PlayerProfile`**, stored **locally on-device only** (Android `SharedPreferences`/DataStore — no server round-trip, no RE work, plain new Kotlin), edited once on first run and editable any time after from a chip in the lobby browser header:
```
PlayerProfile {
displayName: string // free text, local default e.g. "ROOKIE_01"
avatarId: string // one of a small fixed set of {color, glyph} presets
}
```
**Avatar is a preset, not a photo** — a small fixed palette of colour+glyph combinations (e.g. 🏎️/⚡/🔥/🏁/★/🌙/⚙/💀 over a themed accent colour), picked from a grid. Deliberately not a camera/gallery photo upload: no permissions prompt, no image storage/transport concerns once RakNet is carrying lobby state, and it matches the HUD-badge visual language already established by `CarSelectionBadge`/the roster rows. Revisit only if the game ever needs to show a real player photo somewhere — nothing in the current design requires it.
`PlayerProfile` is copied into `LobbyPlayer.displayName`/`avatarId` (schema above) at join/create time and travels with the rest of the lobby state once RakNet exists — no separate sync mechanism needed.
</details>
---
## 4d. Debug menu (Compose) — see DEBUG_MENU.md
An on-map, developer-only debug overlay, added 2026-08-19 — full detail (activation mechanism, enable flag, panel inventory) now lives in a dedicated file, **[DEBUG_MENU.md](DEBUG_MENU.md)**, kept separate for the same reason as the lobby UI split (§4). One decision worth surfacing here: the enable/disable switch (`DebugFeatures.debugMenuEnabled`) deliberately lives in `mpcore`, not `app` — a direct instruction, so debug-tooling gating stays in the shared module rather than duplicated per-Activity.
---
## 5. Network layer (RakNet) — high-level plan (later stages, laid out now)
- **Client**: embedded inside `mpcore` (same process as the game). Owns a RakNet `RakPeerInterface`, connects to either a LAN-discovered server or a manually-entered/internet server address.
- **Server**: standalone headless C++ binary/service, links RakNet directly — explicitly **not** a copy of `libapp.so`'s game runtime. Responsibilities: lobby membership, ready-check aggregation, countdown trigger, relaying position/state updates between clients (topology TBD: pure relay vs. light authority — default to relay first, revisit if cheating/desync becomes a concern).
- **LAN discovery**: SAMP-style UDP broadcast query/response on a fixed port, server replies with lobby metadata (name, player count, track/event). Reference implementation pattern: `GTASA_examples/` (any version — wrappers differ, core discovery/RakNet integration pattern is what matters).
- **Internet servers**: direct-connect by IP:port initially; a master-server/list mechanism is a later refinement (not required for first working version).
- **Position sync**: builds directly on subtask 4 (coordinate read/write hook on the car/track-navigator layer) — the RakNet layer's job is just to move `(distance_along_spline, lateral_offset, height, or raw Vector3+rotation)` tuples between peers at some tick rate; the actual "make the opponent car appear at X" mechanism is the native hook, not the network code.
No RakNet integration work has started yet — this section exists so later native-layer decisions (e.g., what a hooked position-update struct looks like) are made with the eventual network payload shape in mind.
---
## 6. Build & deploy pipeline for native patches
Confirmed this session:
- `native_lib/` (top-level) is **not under any version control** (`git rev-parse` fails there — the repo root itself has no `.git`). It is the sole safety net for "what the game originally shipped" and the file the IDA `.i64` was built from. **Rule: `native_lib/libapp.so` (and siblings) must never be overwritten in place.** Any patched binary is a *new* file.
- `launcher/` is its own separate git repository, and — checked via `git ls-files` — `launcher/app/src/main/jniLibs/armeabi-v7a/*.so` **is tracked in git** (not gitignored), including `libapp.so`. Currently byte-identical to `native_lib/libapp.so` (same SHA-256). This means the copy that actually feeds the APK build has its own independent safety net (`git diff`/`git log`/`git checkout` inside `launcher/`), on top of our manual changelog.
- **Workflow for every native patch, going forward**:
1. Produce the patched bytes into a new file under `native_lib/patched/` (created this session, empty for now — no patches exist yet), never touching `native_lib/libapp.so` itself.
2. Log the change in `PROGRESS.md` (offset, old/new bytes or hook description, why, which patched-file snapshot it corresponds to).
3. Copy the patched file to `launcher/app/src/main/jniLibs/armeabi-v7a/libapp.so` (overwriting the build input — safe, since that copy is git-tracked and diffable).
4. Build the APK (Gradle, already configured per the task brief) — user installs/tests on their own device or WayDroid (see §7).
- `mpcore` itself is the other half of "the magic": it does no static patching of `libapp.so` — instead, at `JNI_OnLoad` time, it resolves `libapp.so`'s **runtime base address** and installs hooks against `base + offset` (the `APP_ADDR()` macro, already in `armhook.cpp`). So there are two independent things to track in `PROGRESS.md`, and they should be logged distinctly: (a) actual byte patches to the shipped `.so` file itself (rare — e.g. removing an anti-tamper check), and (b) runtime hooks installed by `mpcore` (the common case for all the race/opponent/coordinate hooks in the subtask plan) — these don't modify the `.so` on disk at all, only memory at runtime, but should still be documented (what address, what it does, why) since they're just as capable of breaking singleplayer if wrong.
## 7. On-device testing capability (WayDroid)
Checked this session: `adb` and `waydroid` CLI are both installed on this machine and reachable from this session's shell. Attempted `waydroid session start` — **it failed**: `Wayland socket '/run/user/1000/wayland-0' doesn't exist; are you running a Wayland compositor?`. Confirmed cause: the active desktop session here is GNOME-on-**Xorg** (`DISPLAY=:1`, real `Xorg`/`gnome-shell` processes, `WAYLAND_DISPLAY` unset) — WayDroid requires a genuine Wayland compositor and cannot render under plain X11.
**Fix (user action needed)**: log out and pick a Wayland session at the GDM login screen (e.g. "GNOME" without the "(X11)" suffix), then `waydroid session start` should succeed.
**Once a Wayland session is active, this session can plausibly**: `waydroid app install <apk>`, `adb connect`/`adb shell` for logcat and file access, `adb shell screencap` piped to a file and viewed via the image-reading tool for visual verification, and even blind UI interaction via `adb shell input tap/swipe` driven off those screenshots (same pattern as the browser-automation tool, just for Android). This would close most of the test loop without needing the user present for every check — still recommend the user do final verification on real hardware, since WayDroid's GPU/driver stack can behave differently than a real device for this kind of engine.
## 8. Non-goals / explicit constraints carried from the task brief
- Singleplayer must keep working with the mod installed and multiplayer inactive — every hook must be conditional/pass-through when no session is active.
- The dedicated server is a separate C++ service, not a headless instance of the Android game runtime.
- No use of any legacy EA "Synergy" backend endpoints (leaderboards/IAP/DRM) for multiplayer — confirmed in ANALYSIS.md §3.4 this is unrelated infrastructure and likely defunct server-side anyway.
---
## 9. ARM64-only device support (Android 14+ AArch32-less hardware) — see ARM64_TRANSLATION_LAYER.md
Everything about running `libapp.so` (ARM32) on newer devices whose CPU cores dropped AArch32 execution entirely now lives in a dedicated file — **[ARM64_TRANSLATION_LAYER.md](ARM64_TRANSLATION_LAYER.md)** — kept separate so this document stays focused on the mod's own integration architecture. Pure theory as of 2026-08-19, nothing implemented: the short version is an in-process, dynamic CPU-level translation layer (Unicorn/QEMU-TCG-class embeddable ARM32 core, Hangover/Box64-style API-boundary breakout for libc/JNI/GLES/audio) rather than a full-VM (VMOS-style) or a static ARM32→ARM64 recompile — the latter would invalidate every offset in `ANALYSIS.md`/§3a's RE work, the former discards all of this project's native-process integration for no reason.
docs/ARM64_TRANSLATION_LAYER.md
+3637
View File
File diff suppressed because it is too large Load Diff
docs/BETA_TELEMETRY_PLAN.md
+299
View File
@@ -0,0 +1,299 @@
# Closed beta: diagnostics, crash reports and tester workflow
Plan for shipping a preview build to a small group of testers and getting back
reports we can actually act on. Written 2026-09-21, the day the first playable
build appeared.
Decisions already taken (owner's call):
- **Delivery: a "send report" button using the system share sheet.** No backend,
no background upload. The tester sees the file and sends it. This keeps us out
of collecting data from other people's devices, which would otherwise need
consent handling, storage and a retention policy.
- **Scope: crashes and bug reports, performance counters, unhonoured shim
contracts, device model and OS version.**
## Why this needs building at all
Everything the engine currently reports goes to `__android_log_write` and
nowhere else (`util.cpp`'s `Log()`). That fails for beta in three separate ways,
each of which already bit us during development:
1. **logcat wraps.** Twice on 2026-09-21 a measurement was lost to it - once the
whole startup sequence, once the frame-counter history, which made a
frame-rate figure look far better than it was.
2. **Testers cannot retrieve logcat.** It needs a cable and a developer setup.
3. **A hang produces no log at all.** The black-screen bug that day left the
process alive and silent; the only evidence was three lines that had already
scrolled past.
## Part 1 - what the build must produce
### 1.1 In-memory log ring (the foundation)
`Log()` keeps writing to logcat, and additionally appends into a fixed-size
in-memory ring buffer (suggest 2 MB, tunable). **Nothing is written to disk
during normal play** - the log rate is high (per-frame GLES sampling alone
produces thousands of lines) and per-line file I/O would show up in the frame
time we just spent the day reducing.
The ring is allocated once at startup. It must be usable from a signal handler,
so: a plain pre-allocated byte array, no `malloc`, no locks that a crashed
thread might hold. A per-record sequence number plus a short spinlock-free
write is enough; a torn last record in a crash dump is acceptable.
### 1.2 Native crash handler
A handler for `SIGSEGV`, `SIGBUS`, `SIGABRT`, `SIGILL`, `SIGFPE` that writes a
report file and then chains to the previous handler so the normal tombstone
still happens.
**Async-signal-safety is not optional here and we have a recorded incident:**
calling `mmap()` inside a `SIGSEGV` handler has deadlocked on bionic in this
project before. The handler must use only a pre-allocated buffer, a
pre-opened-or-`open()`ed fd, and `write()`. No `malloc`, no `stdio`, no `Log()`,
no C++ allocation, no locks.
What to capture - this is the part specific to a translation layer, and it is
what made today's crash solvable in one reproduction:
- **The decoded guest address.** With the flat mapping, a host fault address is
`flat_map_base + guest_address`. On 2026-09-21 the fault at `0x6f464c459b`
decoded to guest `0x464c459b` by subtracting `x28`. The handler should do that
subtraction itself and print the guest address, since nobody reading a report
will do it by hand.
- **Which arena the guest address belongs to.** `guest_engine.cpp` already has
the classifier (`"thread-stacks arena"` and friends) - reuse it. "Fault in the
heap arena" and "fault 1.2 GB past the end of everything" are completely
different bugs and the report should say which.
- Full guest register set, and the host registers from `ucontext`.
- The last N KB of the log ring.
- Build stamp and device identity (below).
### 1.3 Hang detector
The black-screen bug was a **hang, not a crash** - no signal, no tombstone,
process alive at 2% CPU. A crash handler would have caught nothing.
A watchdog thread checks the `onDrawFrame` counter. If it has not advanced for
~10 seconds while the activity is resumed, it writes the same report the crash
handler would, tagged `HANG`, including a snapshot of every thread's state and
stack pointer. It should fire **once** per hang, not repeatedly.
### 1.4 Unhonoured-contract registry
Today's crash was found because a shim logged that it could not honour a
request, one line above the fault. That should be a first-class, structured
record rather than a log line we grep for.
A small registry: `ReportUnhonouredContract(area, detail)`, deduplicated by
string, counting occurrences. `rtti_shims.cpp`'s `use_facet`,
`jni_shim.cpp`'s silent zero-returns (task #51) and every other
"returning NULL because we do not implement this" path calls it. The report
carries the full deduplicated list.
This turns beta into a **gap-discovery mechanism**: the union of these lists
across testers is a prioritised work queue for what the game actually needs,
discovered from real play instead of guessed.
### 1.5 Session counters
Sampled once a second into a compact rolling summary, not one line per sample:
- frames per second - min, median, 10th percentile, and where the low ones
happened (which is what the median alone hides, as it did today)
- time from launch to first frame, and each level-load duration
- CPU time consumed by the process, and by the GLThread specifically
- peak CPU temperature
- guest heap: live, peak, arena exhaustion events
- thread-stack arena: peak in use, exhaustion events (the black-screen cause -
this must never again be discovered by reading a log tail)
### 1.6 Build and device identity
Every report starts with: build stamp (version plus a short git hash or build
timestamp baked in at compile time), device model, SoC, Android version, ABI,
available RAM, and screen size.
The build stamp matters more than it sounds. Twice on 2026-09-21 the wrong APK
was nearly measured - once a three-week-old release build installed by mistake.
With testers there is no chance to check by hand; the report must say which
build produced it.
### 1.7 The share button
A screen reachable from the pause menu: **"Report a problem"**. It bundles the
newest reports plus the current log ring into a single zip in the app's own
files directory and hands it to `ACTION_SEND`.
Before sharing it shows the tester a short plain-language summary of what is in
the file - log lines, device model, no personal data, no game account details.
They are sending it themselves; they should know what it is.
If the previous session ended in a crash or hang, offer to send that report on
the next launch, since the tester will not go looking for it.
## Part 2 - the tester-facing report form
Free-text bug reports from testers are usually unusable not because testers are
careless but because nobody told them which three facts matter. Keep it short -
a long form gets skipped.
**In-app, attached automatically:** build stamp, device, the log bundle. The
tester never types any of this.
**What we ask the tester for, in this order:**
1. **What were you doing?** One line. "Entered a race from the city map."
2. **What happened?** One line. "Black screen, music kept playing."
3. **What did you expect?** Only when it is not obvious.
4. **Can you make it happen again?** Every time / sometimes / happened once.
This single question decides whether we can chase it at all.
5. **Did you play for a while before it happened?** Yes/no. Specifically
included because the whole class of resource-exhaustion bugs - the
thread-stack arena, the guest heap - only shows up after a long session, and
testers do not think to mention it.
**Severity, defined by consequence rather than by feeling**, so it is not
argued about:
- **Blocker** - cannot continue playing; progress lost.
- **Major** - a feature does not work, but the session survives.
- **Minor** - visual or audio defect, gameplay unaffected.
**Ask them explicitly to send the report even when the game recovers.** A hang
that resolved itself still wrote a `HANG` report, and that is often the easier
one to diagnose.
## Part 3 - what we do with reports
Triage order, informed by what has actually been expensive to find:
1. **Unhonoured-contract list first, before reading the crash.** Today the
answer was in that list. It is cheap to check and frequently decisive.
2. **Decoded guest address and its arena.** Distinguishes a wild pointer from
arena exhaustion from a real logic bug, without any further work.
3. **Exhaustion counters.** If a thread-stack or heap arena hit its ceiling, the
crash is a symptom and the ceiling is the bug.
4. **Only then the register dump and the log tail.**
Group reports by build stamp before comparing anything. Mixing builds is how a
fixed bug looks like it is still present.
## Suggested order of work
Each step is independently useful, so the beta does not wait on the whole set.
1. Log ring + report file + share button, with device and build identity.
**Minimum shippable** - a tester can send something useful.
2. Crash handler with guest-address decoding.
3. Unhonoured-contract registry, with `use_facet` and the JNI zero-returns as
the first callers.
4. Hang detector.
5. Session counters.
## Deliberately out of scope
- **No backend, no automatic upload.** Chosen above. Revisit only if the manual
path proves too lossy in practice.
- **No unique device or user identifier.** Grouping by build stamp and device
model is enough at this scale and avoids tracking individuals.
- **No gameplay telemetry** - what cars, which races, how long played. It is not
needed to fix defects, and collecting it would change what this file is.
---
## Status 2026-09-21: the crash handler is built (plan section 1.2 + 1.7 partial)
Implemented and verified end to end on the Pixel 6a.
**Native** (`mpcore/src/main/cpp/crash_handler.cpp`). Hooks SIGSEGV, SIGBUS,
SIGABRT, SIGILL, SIGFPE with `SA_SIGINFO | SA_ONSTACK`, on a pre-allocated
alternate stack so a stack-overflow crash is still reportable. Everything the
handler needs - the output path, the build stamp - is built at install time;
inside the handler only `open`/`write`/`close` and hand-written integer
formatters run. No malloc, no snprintf, no JNI. It chains to the previous
handler afterwards, so Android still writes its own tombstone.
The report decodes the fault address: a host address inside the guest window is
also printed as the **guest** address, and flagged when it is past the end of
the mapped region ("a wild pointer, not a real guest object"). That is the
number worth reading, and nobody will subtract the base by hand from a tester's
report.
**Java**. `CrashReportActivity` renames the pending report (the handler writes a
fixed name, since it cannot safely format a timestamp), zips it with device and
build details, shows it, and offers ACTION_SEND through a FileProvider scoped to
the crash directory only. Reports live in
`Android/data/<pkg>/files/crashes` - reachable over USB with no permission.
**Verified**: handler installs; `kill -11` produces a report with the right
signal, registers and a correct "outside the guest window" verdict; the next
launch detects it, renames it, builds the zip, and `CrashReportActivity` becomes
the resumed activity. Files land where intended.
**Not verified**: what the screen actually looks like. The test device locked
itself, so every screenshot was of a sleeping or locked display - which is also
why an early "black screen" reading was wrong and led to one unnecessary fix
(explicit colours, harmless and kept). The layout needs a human to unlock the
phone and look.
### Three failures on the way, all worth keeping
1. **Installed too early.** The call sat at the top of `onCreate`, but
`libmpcore.so` is only loaded later by `loadCore()` -
`UnsatisfiedLinkError`, caught and logged. Moved to immediately after
`loadCore()`.
2. **Missing `extern "C"`.** The JNI function was C++-mangled
(`_Z65Java_...`), so the JVM could not find it. The symptom was identical
to the load-order bug above, which cost a wrong fix before the symbol table
was actually read.
3. **Blocked activity start.** Checking for a pending report in
`GameActivityMain.onCreate` - which starts the report screen and finishes
itself - was refused by the platform (`BAL_ALLOW_GRACE_PERIOD`) and dumped
the tester on the home screen. The check belongs in `PermissionsActivity`,
the visible launcher entry.
---
## Status 2026-09-22: game data ships inside the APK
A tester now installs one file and plays. No separate .obb download, no file
manager, no instructions about where to put anything.
**How.** The ~595 MB archive ships as `assets/game_data.obb`, and
`androidResources { noCompress += "obb" }` keeps it **stored** rather than
deflated - it is already compressed, so re-compressing would cost build and
install time for nothing. On first launch `GameDataUnpackActivity` copies it to
`getObbDir()/main.<versionCode>.<package>.obb`, which is exactly the path
`GameActivityMain.obbFullPath` already builds, so no other code knows this
happened.
The copy writes to a `.part` file and renames only on success. A half-written
archive that merely *exists* would pass a naive check and send the game off to
read truncated data - failing far from the cause, which is the failure mode this
project keeps paying for. Free space is checked before starting rather than
500 MB in.
**Measured on the Pixel 6a**, with the real OBB renamed aside to simulate a
clean device:
| | |
|---|---|
| APK size | 615 MB (was 22 MB) |
| build time | 16 s - aapt2 handles the stored asset without trouble |
| `adb install` | 31 s |
| unpack | under 8 s - it finished before the first progress poll |
| result | **md5 identical** to the original OBB |
| game afterwards | 2,893 frames, 0 faults, `mAssetLocationType=OBB` |
**Costs worth stating.** The device needs the APK plus the unpacked copy at
once: about 1.2 GB free at install time, ~600 MB after. And the data exists
twice on disk permanently, since Android keeps the APK.
**The alternative not taken.** Because the asset is stored uncompressed, its
bytes sit contiguously in the APK, so the engine's own `Shim_open`/`Shim_read`
could serve the OBB path straight out of the APK at an offset - no copy, no
duplication. That is a real option if the 600 MB ever matters, but it adds a new
failure surface in file I/O right before a beta, and the ask here was explicitly
for self-extraction.
docs/DEBUG_MENU.md
+31
View File
@@ -0,0 +1,31 @@
# DEBUG_MENU.md — On-map debug menu (Compose)
Living document for the in-app debug menu — a developer-only overlay for poking at mod state while testing, starting with a money editor. Split out as its own file (matching the `LOBBY_UI_DESIGN.md` precedent) so `ARCHITECTURE.md` doesn't accumulate UI-only debug-tooling detail. Update this file whenever the debug menu gains a new panel or its wiring changes.
---
## 1. Purpose and scope
Requested as a debugging aid, not a player-facing feature: a way to poke at mod/game state live on-device without rebuilding. First panel is a money editor. More debug panels are expected to be added here over time (same menu, more entries) rather than each getting its own ad-hoc overlay.
**Explicit enable/disable mechanism**: `DebugFeatures.debugMenuEnabled` (`mpcore/src/main/java/nfs/mod/mpcore/DebugFeatures.kt`) — a single `var` flag, default `true` during active development. `GameActivityMain.onCreate` only adds the debug overlay's `ComposeView` to `mFrameLayout` when this flag is `true`, so flipping it to `false` removes the debug menu from the view hierarchy entirely (not just hidden) ahead of any release build. Chosen to live in `mpcore` (not `app`) per direct instruction, so the one gate for all debug tooling stays in the shared module rather than scattered across Activities.
## 2. Activation — button on the map
Reuses the exact mechanism `CarSelectionBadge` already established (`ARCHITECTURE.md` §4 / `LOBBY_UI_DESIGN.md` §1): a `ComposeView` overlay added to `mFrameLayout` above the game's `GLSurfaceView`, visibility driven by `GameEvents.onMapLoaded()` (fires via the existing, already-proven `MapTrack::HandleEvent` native hook — see `PROGRESS.md`'s `dispatchMapLoaded` history). No new native hook needed: the map-loaded signal already exists and already fires reliably.
- `DebugMenuVisibility` (new, app module) — a `GameEventListener` object exposing `mapLoaded` as Compose state, same shape as `CarSelectionState` in `CarSelectionOverlay.kt`.
- A small floating button, bottom-end corner (deliberately opposite `CarSelectionBadge`'s top-start corner so the two never overlap), appears once `mapLoaded` is true.
- Tapping it opens the debug panel as a dialog over whatever screen is currently showing.
## 3. First panel — money editor (stub, not wired to real game state)
**Explicit scope decision (2026-08-19, direct user answer)**: for now this is a **stub** — a plain Compose text field + "Apply" button that only updates local Compose state (`DebugMoneyState.amount`), logged but **not** written into the game's actual memory/save state. Reason: no RE work has located the player's real cash balance (getter/setter/address) yet — `ANALYSIS.md`/`PROGRESS.md` only cover `CashReward` (a *race event's* bronze/silver/gold reward definition, §6aa/6z of `ANALYSIS.md`), not the player's own wallet/balance. Wiring this panel to the real balance is tracked as a **separate, later RE task** (find where `Profile`/`CurrentState` — both named in the `ISaveable` family, `PROGRESS.md` cont. "2026-08-06" save/profile entry — actually stores the spendable cash total, then add a native setter hook), not part of this UI work.
## 4. Screen inventory
| Panel | State | What it does |
|---|---|---|
| Money editor | **Stub** | Text field + Apply button; writes to local Compose state only, no game effect yet |
More rows added here as panels are added.
docs/LOBBY_PROTOTYPE.html
+1029
View File
File diff suppressed because one or more lines are too long
docs/LOBBY_UI_DESIGN.md
+200
View File
@@ -0,0 +1,200 @@
# LOBBY_UI_DESIGN.md — Multiplayer Lobby UI & Flow
Living document for the lobby overlay's UI/UX — screens, flow, data shape, and the product decisions behind them. Split out of `ARCHITECTURE.md` on 2026-08-18 (that file now only links here) so the native/engine-integration doc doesn't keep growing with UI-only decisions. Update this file whenever a screen or flow decision changes; don't let it drift from the prototype.
**Clickable prototype:** [`LOBBY_PROTOTYPE.html`](LOBBY_PROTOTYPE.html) — self-contained, open directly in any browser, no server needed. Landscape orientation (matches the game). Every screen is tagged OVERLAY (our Compose UI) / NATIVE (real, unmodified game screen) / TRANSITION, with inline notes on what's already proven on-device vs. still open.
---
## 1. Lobby UI mechanism — overlay `View`, markers via `BitmapGraphics`
**Recommendation: hybrid.** Full interactive lobby screens (player list, ready-up, countdown) go through an Android overlay `View`/Compose layer on top of the game's `GLSurfaceView` — the native UI pipeline has no concept of touch-driven widgets, so it can't host these itself. Simple in-world/in-menu indicators (e.g. a future green/red map marker for an active lobby) should instead go through the **`BitmapGraphics` native text/atlas bridge** (`launcher/app/src/main/java/com/ea/ironmonkey/BitmapGraphics.kt`, ANALYSIS.md §3.3) — verified, already trusted by the game's own renderer, no new Flow/SB authoring needed.
| Concern | Map markers → `BitmapGraphics` extension | Full lobby screens → Android overlay `View` |
|---|---|---|
| Mechanism | Add a method (e.g. `drawMarker(x, y, color)` via `canvas.drawCircle`) to `BitmapGraphics.kt`; call it from a hook alongside the existing `drawString` JNI call site (`0x5625a0`) | Standard Android `View`/Compose overlay added to the activity's view hierarchy above/around the `GLSurfaceView` |
| Implementation cost | Low — one new Kotlin method, one new hook call from `mpcore` at the point we know a lobby is active | Low — standard Android UI, fast iteration |
| Visual integration | Genuinely native — rendered through the exact same atlas-blit path as the game's own UI text, in the game's own coordinate space | Good but technically a separate layer composited over/under the GL surface |
| Touch/interaction support | None — this pipeline only produces pixels in a texture; no input handling | Full Android input handling, exactly what's needed for player list + ready buttons |
| Risk to singleplayer | Low — purely additive draw call, no-ops when no lobby is active | None — overlay is purely additive, toggled only when a multiplayer session exists |
| Car/upgrade display integration | N/A | We don't render the car ourselves either way — invoke the game's existing native car-select screen (same one singleplayer uses) as a sub-flow, our overlay only wraps around it |
Both are additive and don't touch Flow/SB authoring, so neither destabilizes the shared menu system. **Preliminary** — revisit once a first marker hook is actually implemented.
Still open: whether `EAMText`/`GlyphMeshGLES`/FreeType (also present in the binary, ANALYSIS.md §3.3) plays any role that would interfere with a `BitmapGraphics`-based marker — low priority until that hook is attempted.
---
## 2. Flow decision & data schema
**Flow:** Compose/View overlay for lobby creation and management, native car-select (filtered by the lobby's class setting) as a sub-flow, ready-checkmark to confirm. "Lobbies as cards on real street pins" is shelved — depends on hooking visible-street `MapTrack` event population, which [[visible-streets-investigation]] found no viable path for. The overlay approach needs no further RE — arbitrary track loading (ARCHITECTURE §3a) and the native car-select screen already work.
```
Lobby {
id: string
hostPlayerId: string
address: { ip: string, port: int } // see §4 — shown for both LAN and WAN join
trackId: string // arbitrary track, see ARCHITECTURE §3a BuildTrackScenePath hook
classRestriction: int | null // car class filter applied to native car-select; null = any
maxPlayers: int // real cap not decided yet (see §6a) — UI works for whatever it ends up being
rewards: { // reuses the CashReward shape found on RaceEvent, ANALYSIS.md §6aa
bronze: int
silver: int
gold: int
} // see §5 — explicit fields, validated at creation, not free-floating pills
players: [LobbyPlayer]
}
LobbyPlayer {
playerId: string
displayName: string // from the local PlayerProfile, see §3 — sent to peers on join
avatarId: string // preset id ("color+icon"), see §3 — not a photo/upload
isLeader: bool // explicit flag, not inferred from hostPlayerId match —
// keeps leader-ness a first-class, UI-visible fact instead of
// something every screen has to re-derive by comparison
ready: ReadyState
carId: string
colourIndex: int
mods: [int | null, int | null] // exactly 2 upgrade slots; each is a CarMod ModType id
// (or null = "НЕТ"/empty). Purely for display — other players
// resolve name/icon/description client-side from the car's own
// CarDescription.Mods catalog, already loaded for the garage
// screen. No numeric value/balance computation server-side.
}
enum ReadyState {
SetsUpLobby // leader only — track/class/rewards configuration
ChoosingCar
ChoosingRace
Loading // race-start fired, client is loading its own RaceLoaderTask
NotReady
Ready
}
```
Confirmed against the live in-game upgrade screen (2 slots, `ВЫБОР УЛУЧШЕНИЯ` list) and the `CarMod` struct in `alfaromeo_4c_2012_desc.prefabs.sb` — 7/7 checked entries matched by price (`ModType` 1/4/5/6/7/8 confirmed, `TYRES_REINFLATING`/`CHASSIS_REINFORCED`/`BODY_IMPACT`/`POWERTRAIN_POWERPACK`/`NITROUS_BURN`/`NITROUS_EARN`). `ModType` is a unique per-option id, not a shared category grouping. The separate `im::app::car::CarPart`/`UpgradeParts` catalog (`skipCost`/`orderTimeMinutes` timer-gating fields) has no live instance data anywhere checked — not part of this schema.
**Resolved decisions:**
- **Host-leaves-lobby**: auto-promote the longest-connected remaining `LobbyPlayer` to leader; dissolve only when the lobby becomes empty.
- **`classRestriction`**: **enforced explicitly by our own overlay** (revised 2026-08-19, was "soft/advisory only") — car_select has no known native class-filter hook (`RACEEVENT_CATEGORYTAG_OFFSET` is a confirmed false lead, PROGRESS.md cont.58; 5 independent dead ends, cont.58-60), so enforcement can't happen inside the native carousel — but that's not a reason to skip enforcing it at all. The overlay checks the chosen `carId`'s class against `classRestriction` the moment `onCarSelected` fires; a mismatch **blocks** readying up (a visible warning replaces the ready button, "Сменить" is the only way forward) rather than silently letting a mismatched car through. Still a client-side check, not a true hard filter on the carousel itself — revisit if the real class-filter mechanism is ever found.
- **Straggler/timeout policy**: hard timeout, auto-start, default 45s from when the first client finishes loading — see §6.
---
## 3. Player identity — profile chip, not a form
Every player row needs *something* to display before anyone's picked a car, and it can't come from an EA account — ARCHITECTURE §8 already rules out the Synergy backend for multiplayer, and this project has no login system.
**Where the button lives**: a small persistent chip (avatar + name) pinned in a fixed corner, present on **every OVERLAY screen** (map's entry point, lobby browser, create, waiting room) — not buried inside one specific screen's header. It's global identity, so it shouldn't only be reachable from one place. It deliberately does **not** appear on the two screens where our footprint is intentionally minimal (the native car_select hand-off, the start countdown) — same principle as the minimized lobby badge there.
Tapping it opens a **popover, not a full-screen navigation** — name field + an 8-preset avatar grid, right there over whatever screen you were on. Editing your name mid-lobby-browsing shouldn't cost you your place in the flow.
**First run — not a forced form.** Auto-generate a usable default (`ROOKIE_####`, random 4-digit suffix, random avatar preset) the moment the profile is first read, so multiplayer works immediately with zero typing. Edit anytime via the chip. This mirrors how most lobby-based games (Fortnite, Fall Guys, etc.) handle it — a name you can live with immediately, not a gate.
```
PlayerProfile {
displayName: string // local default "ROOKIE_####" (random), freely editable
avatarId: string // one of a small fixed set of {color, glyph} presets
}
```
Stored **locally on-device only** (Android `SharedPreferences`/DataStore — no server round-trip, no RE work, plain new Kotlin). **Avatar is a preset, not a photo** — a small fixed palette of colour+glyph combinations (🏎️/⚡/🔥/🏁/★/🌙/⚙/💀 over a themed accent colour), picked from a grid. Deliberately not a camera/gallery upload: no permissions prompt, no image transport concerns once RakNet carries lobby state, and it matches the HUD-badge visual language already established by `CarSelectionBadge`. Revisit only if the game ever needs a real player photo — nothing today requires it.
`PlayerProfile` is copied into `LobbyPlayer.displayName`/`avatarId` at join/create time and travels with the rest of the lobby state once RakNet exists.
---
## 4. Addressing — IP and port, LAN and WAN alike
Every lobby reference carries an explicit `address: { ip, port }` (schema above), surfaced consistently rather than hidden behind "LAN auto-discovers, WAN doesn't need it":
- **LAN browser cards** show the host's LAN address (`192.168.1.42:7777`) alongside name/track/players — not just a bare "● LAN" ping indicator.
- **Manual "по IP" join tab** gets real address + port input fields (not a stub), validated before the connect action enables: IPv4 dotted-quad or hostname pattern, port an integer in `1–65535`.
- **Waiting room** shows the host's own address in the side panel ("Адрес: `192.168.1.42:7777`", with a copy affordance — a plain icon button, not a text label, to stay out of the way of the address itself) — for sharing with WAN friends who need to type it in manually, since there's no master-server/matchmaking list yet (ARCHITECTURE §5 — direct-connect only for the first working version).
Default placeholder port used throughout mocks/design: `7777` — not a final decision, just a consistent stand-in until RakNet setup actually picks one.
---
## 5. Rewards — explicit fields, validated
Bronze/Silver/Gold are real numeric inputs in Create Lobby, not fixed display pills. Validation, checked live as the leader types:
- Each value is a positive integer.
- Non-decreasing: `bronze ≤ silver ≤ gold`.
- "Создать" stays disabled with an inline error message until all three pass.
No other reward shape changes — still the same `CashReward`-derived structure (ANALYSIS.md §6aa) on the `Lobby` schema above.
**The waiting room shows all three tiers too**, not just gold — three small cells (🥇/🥈/🥉 + amount), not one pill. This mirrors the real game's own EventDetails screen (see §9 — its "1-Й/2-Й/3-Й НА ФИНИШЕ" reward list is exactly this shape), so the convention isn't invented, it's matched to what the real UI already does one screen over.
---
## 6. Waiting room — track hero, ready as a button
Revised 2026-08-18 after direct feedback that the first landscape pass was really just a portrait layout squeezed sideways, not a real redesign:
- **Player rows** show the car name and paint colour next to the player name — no car-shaped icon. An earlier pass added a small CSS-drawn car silhouette per row; cut it after direct feedback that it read as visual noise ("некрасиво") rather than useful information. The colour swatch (below) already carries the "this player picked a car" signal on its own.
- **Track preview**: the leader's chosen track shows as a placeholder hero block (a wide gradient/road-motif rectangle standing in for a real screenshot, which doesn't exist yet) with the track's friendly name and internal id caption — more prominent than a small text pill, since "what track is this" is the single most important piece of context in the room.
- **Readiness is a button, not a toggle switch** — deliberately different from the small iOS-style switch most lobby UIs default to. A full-width button reading "Я ГОТОВ" that becomes a solid, distinctly-coloured "✓ ГОТОВ" on press (press again to un-ready) is a clearer tap target and a clearer at-a-glance state, especially at the smaller touch scale of a landscape phone overlay.
- **Car choice stays changeable after readying up.** A persistent "Машина: `<car>` · Сменить" row is always tappable — even when `ready == Ready` — and always reopens native car_select. Changing car always resets `ready → NotReady` automatically, so no peer ever sees a stale "ready" attached to a car that just changed; the player has to explicitly re-press ready afterward.
- **Paint colour is its own swatch**, a small bordered round dot next to the car name (matching the visual weight of the game's own PAINT1–PAINT6 respray swatches) — never baked into an icon shape. An intermediate version tinted the (since-removed) car icon with the colour directly; that read poorly at small size, especially light/white paints against the dark panel. Same underlying `colorHex` data (from `onCarSelected`, proven), just its own unambiguous element.
- **Upgrades are shown too**, using the exact `mods: [int|null, int|null]` shape already in the `Lobby`/`LobbyPlayer` schema (§2) and already flowing live off `onUpgradesAccepted` (proven, cont.39-41): the compact player list shows a 2-dot filled/empty indicator per row (room space is tight — a name is more useful there than two upgrade names), while your own car card in the side panel shows the full resolved names for both slots (or "НЕТ" for an empty slot, matching the schema comment's own convention). Slot→name resolution (`ModType` id → display string) is a placeholder mapping in the prototype — the real display strings live in the game's own localized upgrade-screen text, not extracted here.
- **Fixed 2026-08-19**: the demo car assigned when confirming through the normal flow (`Chevrolet Corvette ZR1`) now matches the lobby's default class (`СПОРТКАР`), so the ready button is visible on the main click-through path instead of being silently replaced by the class-mismatch warning every time. The mismatch state (§2, still a real, useful thing to demo) is now reached deliberately, via a small "[демо] показать несовпадение класса" toggle link next to the ready button — not something the primary flow trips over by accident.
---
## 6a. Max players per lobby — real number TBD, UI works either way
Added 2026-08-19: the actual cap isn't decided yet, so the UI is built to not assume one.
- **Create Lobby** gets a `−`/`+` stepper next to the track selector (range 2–12, arbitrary bounds — just enough to make the control meaningfully bounded, not a real design decision).
- **Waiting room** shows a live `👤 N/max` pill next to the class tag, turning the same warning colour used elsewhere (`--danger`) once the room is full.
- **Server browser** cards show `current/max` instead of a static player-count string, and a full server (`current == max`) is visually locked — dimmed, tagged "ЗАПОЛНЕНО", not tappable — rather than silently allowing a join that wouldn't actually work.
None of this enforces anything server-side (there's no server yet) — it's UI that reflects whatever `Lobby.maxPlayers` ends up being once that number is actually decided.
---
## 7. Loading & waiting for stragglers — engine-level, not a custom screen
**Revised 2026-08-18 per direct correction** — the earlier draft had this as our own Compose "Загрузка заезда" screen with a list of progress bars. That's the wrong layer for it to live on. The actual requirement: one phone loads fast, another loads slow, and whoever's ready first needs to visibly still be *waiting on people*, not staring at our UI — the intro/starting-line scene should keep playing, on loop, until everyone's in.
- Once a client's own race load reaches the point where the game would normally show its native pre-race starting-grid scene (cars lined up, camera settled, about to cut to the 3-2-1-GO countdown), **that's what stays on screen** — not a custom loading UI panel. Consistent with this project's standing principle (ARCHITECTURE §2): hook the game's own flow, don't replace it with ours.
- Our overlay's only contribution here is a **thin status strip**, not a full panel — something like "Ожидаем игроков (2/4) · SLOWPOKE_88 загружается", pinned to one edge, with a subtle looping pulse so it visibly reads as "still working," not "frozen."
- The native 3-2-1-GO countdown itself is **held** until the host signals everyone's in, then plays exactly as it would in a normal single-player race start — no substitute countdown of our own.
- **Engineering reality check**: the hold/release mechanism doesn't exist yet. Candidate hook anchor is `InRaceState` construction (RTTI-confirmed, ANALYSIS.md §3.1) or the final event in `RaceLoaderTask_DispatchInitialFSMEvents` (ANALYSIS.md §6b stage 8) — either is a real, already-located point, just not yet wired for this purpose. `RaceLoaderTask_ExecuteLoadSequence`'s own `SetLoadProgress(this, N)` calls (ANALYSIS.md §6a slot 4, fractions 0.1–0.8) are the natural source for the real per-client progress number to report to peers, instead of inventing a synthetic one.
- **Straggler timeout still applies on top of this**: hard 45s timeout (resolved 2026-08-18) from when the first client is ready — if it expires before everyone's in, the held countdown releases anyway and whoever isn't in yet is left behind. Left-behind representation (spectator vs. rubber-band-in-when-ready) is still an open, lower-priority call.
- Also still open, and *upstream* of all of this (see ARCHITECTURE §4b step 5.1): which race event actually carries the load. The synthetic "LAN" event currently has its loadout-confirm deliberately redirected back to the map because it has no real track/scene reference (crashes otherwise, PROGRESS.md cont.36) — the recommended fix is having the lobby's "start" re-target a real, existing race event as the vessel and let *its* loadout-confirm proceed normally (real events already sail through with zero crashes, cont.41), rather than trying to give the synthetic event a scene reference of its own.
---
## 8. Screen inventory (prototype)
Matches `LOBBY_PROTOTYPE.html`'s screen rail. Profile is **not** in this numbered flow — it's the global chip/popover from §3, demonstrable from any screen.
| # | Screen | Tag | What it is |
|---|---|---|---|
| 0 | Карта — точка входа | OVERLAY | Entry button on the map, same `FrameLayout` pattern as `CarSelectionBadge` |
| 1 | Обзор лобби | OVERLAY | LAN list (with address) + "по IP" tab (validated) + create |
| 2 | Создание лобби | OVERLAY | Name, track (with placeholder preview), class, validated rewards |
| 3 | Комната ожидания | OVERLAY | Player rows (ready-button, reward trio, mod dots), track hero, address |
| 4 | event_details (перед car_select) | NATIVE | Real on-device screenshot — any ordinary event, not ours; see §9 |
| 5 | car_select (после event_details) | NATIVE | Real on-device screenshot — our overlay shrinks to a corner badge |
| 6 | Снова в лобби: авто выбрано | OVERLAY | Car/colour captured; "Сменить" stays available; ready button |
| 7 | Все готовы | OVERLAY | Host's "Начать гонку" enabled once everyone's `Ready` |
| 8 | Ожидание в гонке | NATIVE + thin overlay | Native starting-grid scene loops; thin status strip shows stragglers |
| 9 | Старт заезда | TRANSITION | Native 3-2-1-GO, released once everyone's in (or timeout) |
---
## 9. event_details before car_select — real screenshots, real reason
Added 2026-08-19, direct request: show the actual native hand-off as two steps, not one, using real captures instead of a drawn placeholder.
**Why event_details comes first, not just car_select alone**: this isn't a UX preference, it's the same technical constraint already on record in ARCHITECTURE §4b step 5.2 / PROGRESS.md cont.55/57 — `TriggerTrueDirectCarSelectJump()` needs at least one *real* `event_detail → car_select` transition to have happened this session (`g_realEventDetailsVisitHappened`) before a cold direct jump is reliable. Showing event_details first in the prototype makes that real engine requirement visible in the flow itself, instead of leaving it as an invisible precondition documented only in prose.
**The screenshots are real**, captured live via `adb screencap` on the project's own Galaxy A9 test device (`com.ea.games.nfs13_mod`), not drawn: an ordinary street (МАККЛЕЙН) → an ordinary event (ПОБУДКА, everyday street race) → its EventDetails screen → continue → car_select showing whatever car happened to be equipped (Subaru Cosworth Impreza STI CS400). Deliberately **not** the project's own synthetic "LAN" event — per the request, any real street/event/car works, and using a genuinely ordinary one keeps the mockup honest about what the native screens actually look like today, unmodified.
Both screenshots are embedded directly in `LOBBY_PROTOTYPE.html` (inline `data:` JPEGs, cropped to drop the on-screen nav bar) — the prototype has zero external image dependencies.
**On-device viewing**: `LOBBY_PROTOTYPE.html?kiosk=1` hides the review-tool chrome (masthead, screen rail, notes panel) and lets the phone-frame content fill the real viewport — this is the mode to use when checking the mockup on an actual phone via `adb`, not the full review page.
docs/PROGRESS.md
+1318
View File
File diff suppressed because it is too large Load Diff
docs/STATIC_RECOMPILATION_FALLBACK.md
+257
View File
@@ -0,0 +1,257 @@
# Статическая перекомпиляция ARM32 → LLVM IR → ARM64
**Статус: запасной путь.** Не начинать, пока не исчерпан текущий подход (JIT-трансляция через Unicorn).
Документ создан 2026-09-21 по запросу как запись направления, к которому осмысленно вернуться, если
скорость движка упрётся в потолок.
---
## 1. Когда переключаться на этот путь
Критерий один и он числовой. Текущий подход даёт **отставание примерно в 2,6 раза** от реального времени.
Измеренный потолок его оптимизации:
| шаг | ожидаемый результат |
|---|---|
| обход softmmu (задача #61) | ~1,4x → остаётся ~1,8x отставания |
| дальнейшие микрооптимизации | в лучшем случае 1,3–1,5x, оптимистично |
| **натив** | **недостижим в JIT-подходе** |
**Переключаться, если:** после обхода softmmu игра всё ещё не держит 30 кадров в гонке, и дальнейшие
замеры не показывают крупных резервов.
**Не переключаться, если:** 30 кадров достигнуты. Цель — играбельность, а не бенчмарк.
### Цена переключения, которую надо знать заранее
**Этот путь обнуляет наработки по мультиплееру.** Вся работа по внедрению сетевого кода делалась в
расчёте на перехват функций живого ARM32-бинарника через хуки Unicorn. После перекомпиляции бинарника
не будет — будет свой нативный код, и точки внедрения придётся искать заново, уже в другом виде
(зато, вероятно, удобнее: в статически слинкованном коде можно просто подменить символ).
Это самая серьёзная цена, и она не техническая, а проектная. Учитывать при решении.
---
## 2. Что это за подход и чем он отличается от «отреверсить игру»
**Это НЕ декомпиляция.** Никто не читает код, не восстанавливает классы, не пишет C++ заново.
Машинные инструкции ARM32 **механически** переводятся в промежуточное представление LLVM, а затем
компилируются в нативный ARM64. Инструмент не понимает, что делает код — он сохраняет его поведение
команда за командой.
Известные работающие примеры этого класса: **N64Recomp** (использован для портов игр с Nintendo 64 на PC),
аналогичные проекты для PS2 и GameCube.
### Почему это даёт скорость, которой не даст JIT
Ключ не в том, что трансляция происходит заранее. Ключ в том, что после лифтинга код попадает в
**настоящий оптимизирующий компилятор**:
| | JIT (сейчас) | статическая перекомпиляция |
|---|---|---|
| Флаги процессора ARM32 | пересчитываются после каждой операции, даже если не нужны | LLVM выбросит мёртвые вычисления |
| Регистры | 16 гостевых мапятся на 31 хостовый, лишние простаивают | распределение регистров с нуля, все 31 |
| Область оптимизации | внутри одного блока трансляции | межпроцедурная, всё приложение |
| Инлайнинг, векторизация | нет | стандартные проходы LLVM |
Именно поэтому это **единственный путь** к «неотличимо от натива». Для ориентира: Rosetta 2 от Apple —
заранее скомпилированная трансляция плюс аппаратная поддержка в процессоре — даёт 70–80% нативной
скорости.
---
## 3. Почему именно наш случай необычно удобен
Обычные блокеры статической трансляции у нас частично или полностью сняты, и это **измеренные факты**,
а не предположения.
### Границы функций известны
В бинарнике есть секция `.ARM.exidx` — таблица раскрутки стека для исключений C++:
```
.ARM.exidx 0x97e150 – 0x9aad18 (0x2cbc8 байт)
```
По 8 байт на запись это **≈22 900 записей**, каждая указывает на начало функции. Главная проблема
статической трансляции — «где начинается код» — решена самим бинарником. IDA независимо нашла
**34 726 функций**, что согласуется.
### Самомодифицирующегося кода нет
Проверено счётчиком на уровне Unicorn: за полный прогон загрузки пролога — **ноль записей гостя в
`.text`**. Это значит, что переведённый код не нужно инвалидировать и перетранслировать.
### Код и данные разделены
`.text` (9,5 МБ) отделён от `.rodata`, `.data`, `.bss`. Не надо угадывать, где инструкции, а где таблицы.
### Релокации дают карту указателей
`.rel.dyn` (387 КБ, ~48 000 записей `R_ARM_RELATIVE`) перечисляет все места, где лежат адреса.
Это карта того, что является указателем, а что числом.
### Чужой код, который действительно не надо переводить — 19,3%, а не треть
**Исходная оценка «треть бинарника чужая» оказалась завышенной примерно вдвое.** Подсчёт по адресным
диапазонам, подтверждённый тремя независимыми методами (кластеризация ссылок на строки, минимальный
разрез графа вызовов, тип записей `.ARM.exidx`):
| библиотека | диапазон | функций | байт | vtable внутри |
|---|---|---|---|---|
| zlib 1.2.11 | `0x660400–0x66a000` | 61 | 39 612 | 0 |
| libjpeg | `0x774000–0x790900` | 285 | 114 836 | 0 |
| libpng 1.5.10 | `0x790900–0x7ad844` | 394 | 113 808 | 0 |
| curl 7.56.0 | `0x7c72c4–0x80e9b0` | 748 | 281 300 | 0 |
| OpenSSL 1.1.0f | `0x80e9b0–0x963120` | 5 014 | 1 156 034 | 0 |
| **итого** | | **6 502** | **1 705 590** | **0** |
Границы подтверждены independently: одна запись `.ARM.exidx` с признаком CANTUNWIND покрывает
1 687 132 байта одним куском — curl и OpenSSL собраны с `-fno-unwind-tables`, больше ничто в образе так
не собрано. Её концы совпадают с локальными минимумами разреза графа вызовов. И **ни одной C++ vtable**
внутри этих диапазонов — игровой код туда не затёк.
### Три ошибки первой редакции этого документа
| было записано | на самом деле | как проверено |
|---|---|---|
| FMOD влинкован статически | **Нет.** `DT_NEEDED: libfmodex.so, libfmodevent.so` — отдельные библиотеки | `readelf -d` |
| libc++ влинкован статически | **Нет.** `DT_NEEDED: libc++_shared.so`; в образе только заголовочные шаблоны | `readelf -d` |
| 2 432 «именованные» функции | Из них 1 170 — автогенерация IDA (`nullsub_*`), почти всё остальное — PLT-заглушки. **Восстановленных внутренних символов практически ноль** | гистограмма префиксов |
Пропущены были **libjpeg** (опознан по таблице сообщений `jerror.c`) и **Bullet Physics** (210 имён
классов `bt*`).
### Почему boost, EASTL и libc++ заменить НЕЛЬЗЯ
Первая редакция утверждала, что шаблоны стандартной библиотеки «пересобираются из заголовков, а не
переводятся». **В механическом лифтере это не работает.** У лифтера нет исходников. Чтобы не переводить
тело `std::vector<Foo>::push_back`, нужно опознать инстанцирование, восстановить точную раскладку `Foo` и
доказать совместимость с хостовой версией — а это ручной реверс, ровно то, ради отказа от чего и выбран
этот путь.
То же касается boost (610 имён классов, 792 vtable, 1 247 виртуальных целей) и EASTL: они **размазаны по
всему `.text`** и не отделяются по адресам. Переводить как обычный код.
### Что действительно облегчает задачу
**Образ целиком в режиме ARM, без Thumb.** 2 249 305 инструкций в 8 989 480 байтах — ровно 4,0 байта на
инструкцию. В `.data.rel.ro` 27 107 указателей на ARM-код против 123 с Thumb-битом. Ноль `tbb`/`tbh`,
ноль `ldr pc,`, ноль `mov pc,`. Проверено независимо: все 34 экспортируемые функции имеют чётные адреса.
**Нет переключения режимов, нет IT-блоков, нет Thumb-таблиц переходов** — заметно более простая цель, чем
предполагалось.
> Побочно: комментарий в `guest_engine.cpp:3901` называет Thumb «единственным реальным режимом этого
> движка». Это **неверно** и может ввести в заблуждение. Работе движка не мешает (режим берётся из CPSR),
> но как ориентир — ошибка.
**`.ARM.exidx` покрывает `.text` на 100%** — 22 905 записей размечают 9 526 532 из 9 527 632 байт.
**Дубликаты.** 4 918 функций побайтово идентичны и сводятся к 739 различным телам: 1 148 × `bx lr`
(пустой виртуальный метод), 678 × `mov r0,#0; bx lr`, 322 × переходник `boost::function`. Лифтер,
хеширующий тела, выдаёт 739 вместо 4 918 — **4 179 функций бесплатно**.
**Длинный хвост мелочи.** 10 646 функций (31,6%) короче 32 байт и занимают всего 1,4% кода. При этом
1 613 функций (4,8%) длиннее килобайта и занимают 39,6%.
## 4. Главная нерешённая трудность: косвенные переходы
`blx <reg>` — вызов по адресу из регистра. Статически неизвестно, куда он ведёт.
**Замерено по всем 2 249 305 инструкциям образа:**
| | количество |
|---|---|
| `blx <reg>` — косвенные вызовы | **36 232** |
| `bl` — прямые вызовы | 136 075 |
| доля косвенных среди всех вызовов | **21%** |
| `bx <reg>` (в основном `bx lr`, возвраты) | 9 364 + 1 833 условных |
| `pop`/`ldm` с `pc` (возвраты) | 27 011 |
| `ldr pc,` / `tbb` / `tbh` / `mov pc,` | **0** |
### Сколько целей удаётся собрать статически
| | количество |
|---|---|
| vtable, привязанных к typeinfo | **4 001** |
| слотов в них | 25 631 (из них 441 чисто виртуальных) |
| **различных виртуальных целей** | **10 624** |
| все указатели на код в данных (`.data.rel.ro`, `.got`, `.data`, `.init_array`) | 28 003 → **12 409 различных целей** |
| **покрытие функций `.text`** | **36,7%** |
### И вот здесь главная оговорка, которой не было в первой редакции
Первая редакция утверждала, что «большинство виртуальных целей можно собрать статически». Для **vtable**
это верно. Для **колбэков — нет.**
В этой сборке с позиционно-независимым кодом взятие адреса функции в регистр выглядит как
`ldr rX,[pc,#N]; add rX,pc` — литерал хранит **смещение относительно PC** и **не требует релокации**
(проверено на дизассемблере по адресу `0x7c758`). Значит таблица релокаций такие цели **не видит**.
Мера того, насколько она их не видит: **8 818 функций (1 405 236 байт) не имеют ни одного входящего
прямого вызова, ни одного указателя из данных.** Часть — мёртвый код, оставленный компоновщиком.
Остальное — колбэки, достижимые только анализом литеральных пулов.
**Отсюда следует порядок работ:** первым делом нужен не транслятор, а **сканер литеральных пулов**.
Если покрытие косвенных целей не удастся поднять существенно выше 37%, то запасной путь через
хеш-таблицу «адрес → функция» съест ровно тот выигрыш в скорости, ради которого всё затевается.
## 5. Что ещё придётся решить
| задача | сложность | комментарий |
|---|---|---|
| **Исключения C++** | высокая | Есть `.ARM.exidx`/`.ARM.extab`. Раскрутка стека ARM32 не переносится на ARM64 напрямую — нужна либо своя реализация, либо отображение на нативные исключения |
| **Модель памяти** | средняя | ARM32 и ARM64 имеют разные гарантии упорядочивания. При многопоточности возможны тонкие гонки, которых не было на оригинале |
| **Точность флагов** | средняя | Где флаги реально читаются — надо сохранить. LLVM выбросит лишнее только если правильно разметить |
| **JNI-граница** | низкая | Уже решена в текущем движке, переносится почти как есть |
| **Системные вызовы и libc** | низкая | Уже есть полный набор шимов, линкуется напрямую |
---
## 6. Первый шаг, если решим начать
**Не писать транслятор.** Порядок такой:
1. **Сканер литеральных пулов** (см. раздел 4). Это главный риск всего направления, и он проверяется
раньше всего. Цель — поднять покрытие косвенных целей существенно выше 37%. Если не выходит —
направление не окупается, и лучше узнать это на первом шаге.
2. **Покрытие кода.** Включить блочный профилировщик (`EnableProfiling()`, уже есть в движке) и записать
исполненные адреса за полный сеанс: загрузка, меню, гонка, финиш. Это покажет, сколько из 23 000 тел
реально работает, а сколько — мёртвый код.
3. **Прототип на одной функции.** Перевести одну чистую вычислительную функцию через LLVM IR,
подставить в работающий движок вместо эмулируемой и **замерить**. Это даст реальный коэффициент
ускорения — единственную цифру в этом документе, которая будет фактом, а не оценкой.
---
## 7. Объём работ — итоговая таблица
| категория | функций | доля | что делать |
|---|---|---|---|
| zlib, libjpeg, libpng, curl, OpenSSL | 6 502 | 19,3% | **заменить линковкой** — непрерывные диапазоны, ноль vtable |
| Bullet Physics | 1 828 | 5,4% | переводить (замена — отдельное исследование) |
| фреймворки EA, однозначные | 3 784 | 11,2% | переводить; обёртка GLES — единственный кандидат на замену |
| смешанные области EA и игры | 3 798 | 11,3% | переводить, по адресам не разделяются |
| **игра и движок** | **17 783** | **52,8%** | **переводить — неустранимое ядро** |
Всего реальных функций в `.text`: **33 695** (прежние 34 726 включали 516 заглушек PLT и 515
плейсхолдеров импорта, которые кодом не являются).
**Требуют механического перевода: 27 193.** После дедупликации по содержимому — около **23 000 различных
тел, ~1,82 млн инструкций ARM**.
---
## 8. Честный вывод
Направление **выполнимо** в том смысле, что 23 000 тел — работа для машины, а не для человека. Условия
лучше, чем казалось: режим только ARM без переключений, `.ARM.exidx` покрывает код на 100%, **все 2 310
имён классов RTTI сохранились целиком**, 4 001 vtable дают 10 624 разрешённых виртуальных цели, а треть
функций короче 32 байт.
Но **главный риск не в объёме, а в 36 232 косвенных вызовах**, чьи цели собираются лишь частично.
Начинать надо с проверки именно этого, а не с транслятора.
И цена из раздела 1 остаётся в силе: **этот путь обнуляет наработки по мультиплееру.**
docs/evidence/font_atlas_was_guest_addr_0.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 90 KiB

docs/save_backups/nfstr_save_2026-09-19_A9-native-67k.sb
BIN
View File
Binary file not shown.
docs/save_backups/nfstr_save_2026-09-19_prologue-done.sb
BIN
View File
Binary file not shown.
docs/save_backups/nfstr_save_2026-09-19_worldmap.sb
BIN
View File
Binary file not shown.
docs/save_backups/nfstr_save_2026-09-19_xiaomi.sb
BIN
View File
Binary file not shown.
mpcore/build.gradle.kts
+5 -1
View File
@@ -11,7 +11,11 @@ android {
minSdk = 27
ndk {
abiFilters += listOf("armeabi-v7a")
// ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md):
// mpcore itself is now plain native ARM64 code that drives an
// embedded ARM32 CPU-emulation core (Unicorn) instead of being
// ARM32 code injected into a natively-loaded libapp.so.
abiFilters += listOf("arm64-v8a", "x86_64")
}
testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
mpcore/scripts/test_on_device.sh
Executable
+103
View File
@@ -0,0 +1,103 @@
#!/usr/bin/env bash
# Repeatable on-device smoke test for the ARM32-in-ARM64 prototype (see
# /ARM64_TRANSLATION_LAYER.md). Replaces the manual adb-shell archaeology
# this project's live debugging sessions have needed up to now: install,
# launch, watch logcat for a bounded window, then report the same signals
# those sessions kept hand-checking - onCreate timing, MEM FAULT count (see
# guest_engine.cpp's mem_fault_hook_cb), and process-death cause (ANR / OOM /
# MIUI SwipeUpClean / other) - as one command, so every future interface-
# layer change gets checked against the same baseline instead of a fresh
# investigation each time.
#
# Usage: mpcore/scripts/test_on_device.sh [device-serial] [watch-seconds]
# device-serial defaults to the first device `adb devices` lists.
# watch-seconds defaults to 180 (most observed onCreate runs finish in
# 100-120s; this leaves headroom to also see the post-onCreate lifecycle
# calls that previously kept faulting).
set -uo pipefail
# Script lives at mpcore/scripts/ - project root (where app/ and mpcore/ are
# siblings) is two levels up.
cd "$(dirname "$0")/../.."
PKG="com.ea.games.nfs13_arm"
ACTIVITY="com.ea.ironmonkey.GameActivityMain"
# "translated" (the emulated arm64-v8a path) - see app/build.gradle.kts's
# product flavors, added for the native ARM32 tracing investigation
# (ARM64_TRANSLATION_LAYER.md). The pre-flavor-split path
# (app/build/outputs/apk/debug/app-debug.apk) is now a STALE, never-updated
# file left over from before that split - silently reinstalling it here
# instead of the real current build wasted real debugging time confirming
# this fix, so this path is now the single source of truth.
APK="app/build/outputs/apk/translated/debug/app-translated-debug.apk"
SERIAL="${1:-$(adb devices | awk 'NR==2{print $1}')}"
WATCH_SECONDS="${2:-180}"
if [[ -z "$SERIAL" ]]; then
echo "No adb device found. Connect the device and retry." >&2
exit 1
fi
ADB="adb -s $SERIAL"
if [[ ! -f "$APK" ]]; then
echo "APK not found at $APK - build it first (./gradlew :app:assembleDebug)." >&2
exit 1
fi
echo "== Installing $APK on $SERIAL =="
$ADB install -r "$APK" || { echo "install failed" >&2; exit 1; }
echo "== Launching =="
$ADB shell am force-stop "$PKG"
$ADB logcat -c
$ADB shell monkey -p "$PKG" -c android.intent.category.LAUNCHER 1 >/dev/null
sleep 2
PID=$($ADB shell pidof "$PKG" | tr -d '\r')
if [[ -z "$PID" ]]; then
echo "Process did not start." >&2
exit 1
fi
echo "pid=$PID"
echo "== Watching logcat for ${WATCH_SECONDS}s =="
LOG="$(mktemp)"
timeout "${WATCH_SECONDS}s" $ADB logcat --pid="$PID" > "$LOG" 2>/dev/null || true
echo ""
echo "===================== RESULTS ====================="
ONCREATE_LINE=$(grep "GameActivityMain onCreate took" "$LOG" | tail -1)
if [[ -n "$ONCREATE_LINE" ]]; then
echo "onCreate: $(echo "$ONCREATE_LINE" | grep -oE '[0-9]+ms')"
else
echo "onCreate: did not complete within ${WATCH_SECONDS}s"
fi
FAULT_COUNT=$(grep -c "MEM FAULT" "$LOG" || true)
echo "MEM FAULT lines: $FAULT_COUNT"
if [[ "$FAULT_COUNT" -gt 0 ]]; then
echo " first: $(grep "MEM FAULT" "$LOG" | head -1)"
echo " last: $(grep "MEM FAULT" "$LOG" | tail -1)"
fi
CRASHED_LINE=$(grep -c "refusing to run - engine already crashed" "$LOG" || true)
echo "engine crashed (fail-fast tripped): $([[ "$CRASHED_LINE" -gt 0 ]] && echo yes || echo no)"
STILL_ALIVE=$($ADB shell "pidof $PKG" | tr -d '\r')
if [[ -n "$STILL_ALIVE" ]]; then
echo "process status: alive (pid=$STILL_ALIVE) at end of watch window"
else
echo "process status: DEAD"
DEATH_LINE=$($ADB logcat -d 2>/dev/null | grep -E "Killing $PID:|$PID.*died->background" | tail -3)
if [[ -n "$DEATH_LINE" ]]; then
echo " cause:"
echo "$DEATH_LINE" | sed 's/^/ /'
else
echo " cause: unknown (no ActivityManager kill/death line found - check for a FATAL crash instead)"
$ADB logcat -d --pid="$PID" 2>/dev/null | grep -iE "FATAL|AndroidRuntime" | sed 's/^/ /'
fi
fi
echo "====================================================="
echo "Full log: $LOG"
mpcore/src/main/cpp/CMakeLists.txt
+2
View File
@@ -34,6 +34,7 @@ add_subdirectory(third_party/unicorn)
add_library(${CMAKE_PROJECT_NAME} SHARED
main.cpp
crash_handler.cpp
game_lifecycle_stubs.cpp
game_lifecycle_stubs_extra.cpp
game_lifecycle_stubs_extra2.cpp
@@ -45,6 +46,7 @@ add_library(${CMAKE_PROJECT_NAME} SHARED
emu/pthread_shim.cpp
emu/jni_shim.cpp
emu/gles_shim.cpp
emu/dyncast_fastpath.cpp
emu/libc_shims.cpp
emu/rtti_shims.cpp
emu/fmod_shims.cpp
mpcore/src/main/cpp/crash_handler.cpp
+283
View File
@@ -0,0 +1,283 @@
#include "crash_handler.h"
#include "util/util.h"
#include <cerrno>
#include <csignal>
#include <cstring>
#include <fcntl.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/ucontext.h>
#include <unistd.h>
// ---- Everything below the handler boundary must be async-signal-safe ----
//
// The rule is narrow and unforgiving: inside the handler, only functions
// POSIX lists as async-signal-safe may be called. write(), open(), close(),
// _exit(), sigaction(), raise() are; snprintf(), malloc(), any C++ container,
// anything that takes a lock, and every JNI call are NOT.
//
// This project already paid for ignoring that once - see
// ARM64_TRANSLATION_LAYER.md's note that mmap() inside a SIGSEGV handler
// deadlocked on bionic. So: every string is built HERE, at install time, and
// the handler only appends bytes to a pre-opened path and formats integers
// with the hand-written helpers below.
namespace {
constexpr size_t kPathMax = 512;
constexpr size_t kStampMax = 256;
// Pre-built at install time. `g_installed` also guards against a second
// install and against the handler running before setup finished.
char g_reportPath[kPathMax];
char g_buildStamp[kStampMax];
bool g_installed = false;
// Guest memory window, for decoding a host fault address back to the guest
// address that produced it. Plain integers written once from the engine
// thread and only read afterwards.
uint64_t g_guestFlatBase = 0;
uint32_t g_guestRegionSize = 0;
// The alternate stack. A stack-overflow SIGSEGV cannot be handled on the
// stack that just overflowed, so the handler gets its own - allocated here,
// at install time, never inside the handler.
constexpr size_t kAltStackSize = 64 * 1024;
char g_altStack[kAltStackSize];
// Previous dispositions, so the default handler still runs afterwards and
// Android still produces its own tombstone. Our report is extra evidence, not
// a replacement for the system's.
struct sigaction g_prevActions[NSIG];
// ---- async-signal-safe output helpers ----
void SafeWrite(int fd, const char* s, size_t len) {
while (len > 0) {
ssize_t n = write(fd, s, len);
if (n <= 0) {
if (n < 0 && errno == EINTR) continue;
return; // nothing useful left to do inside a handler
}
s += n;
len -= (size_t)n;
}
}
void SafeStr(int fd, const char* s) {
if (!s) return;
size_t len = 0;
while (s[len]) len++;
SafeWrite(fd, s, len);
}
void SafeHex(int fd, uint64_t v) {
static const char kDigits[] = "0123456789abcdef";
char buf[19];
buf[0] = '0';
buf[1] = 'x';
int pos = 18;
buf[pos] = '\0';
if (v == 0) {
buf[--pos] = '0';
} else {
while (v && pos > 2) {
buf[--pos] = kDigits[v & 0xF];
v >>= 4;
}
}
SafeWrite(fd, "0x", 2);
SafeStr(fd, buf + pos);
}
void SafeDec(int fd, long v) {
char buf[24];
int pos = 23;
buf[pos] = '\0';
bool neg = v < 0;
unsigned long u = neg ? (unsigned long)(-v) : (unsigned long)v;
if (u == 0) buf[--pos] = '0';
while (u && pos > 0) {
buf[--pos] = (char)('0' + (u % 10));
u /= 10;
}
if (neg && pos > 0) buf[--pos] = '-';
SafeStr(fd, buf + pos);
}
const char* SignalName(int sig) {
switch (sig) {
case SIGSEGV: return "SIGSEGV (bad memory access)";
case SIGBUS: return "SIGBUS (misaligned or unmapped access)";
case SIGABRT: return "SIGABRT (abort - often a failed assertion or a JNI error)";
case SIGILL: return "SIGILL (illegal instruction)";
case SIGFPE: return "SIGFPE (arithmetic fault)";
default: return "unknown signal";
}
}
void WriteRegisters(int fd, void* ucontextRaw) {
if (!ucontextRaw) return;
auto* uc = static_cast<ucontext_t*>(ucontextRaw);
#if defined(__aarch64__)
const mcontext_t& mc = uc->uc_mcontext;
SafeStr(fd, "registers (host arm64):\n");
for (int i = 0; i < 31; i++) {
SafeStr(fd, " x");
SafeDec(fd, i);
SafeStr(fd, i < 10 ? " = " : " = ");
SafeHex(fd, mc.regs[i]);
SafeStr(fd, (i % 2) ? "\n" : " ");
}
SafeStr(fd, "\n sp = ");
SafeHex(fd, mc.sp);
SafeStr(fd, " pc = ");
SafeHex(fd, mc.pc);
SafeStr(fd, "\n");
// x28 is where the flat guest mapping keeps its base (task #61). Printing
// it makes the guest-address arithmetic below checkable by hand from the
// report alone.
SafeStr(fd, " x28 (guest base register) = ");
SafeHex(fd, mc.regs[28]);
SafeStr(fd, "\n");
#else
(void)uc;
SafeStr(fd, "registers: not captured on this architecture\n");
#endif
}
void CrashSignalHandler(int sig, siginfo_t* info, void* ucontextRaw) {
if (g_installed) {
// O_TRUNC, not O_APPEND: one pending report at a time. If a second
// crash happens before the first is collected, the newer one is the
// one worth having - it is the one the tester just saw.
int fd = open(g_reportPath, O_WRONLY | O_CREAT | O_TRUNC, 0644);
if (fd >= 0) {
SafeStr(fd, "=== NFSMW arm64 engine - native crash report ===\n\n");
SafeStr(fd, "build: ");
SafeStr(fd, g_buildStamp);
SafeStr(fd, "\n\nsignal: ");
SafeStr(fd, SignalName(sig));
SafeStr(fd, " (");
SafeDec(fd, sig);
SafeStr(fd, ")\n");
const uint64_t faultAddr = info ? (uint64_t)(uintptr_t)info->si_addr : 0;
SafeStr(fd, "fault address (host): ");
SafeHex(fd, faultAddr);
SafeStr(fd, "\n");
// The number that actually helps. A host fault address inside the
// guest window is really a guest pointer; reported raw it is
// meaningless, and nobody reading a tester's report will subtract
// the base by hand.
if (g_guestFlatBase && faultAddr >= g_guestFlatBase &&
faultAddr < g_guestFlatBase + 0x100000000ull) {
uint64_t guestAddr = faultAddr - g_guestFlatBase;
SafeStr(fd, "fault address (GUEST): ");
SafeHex(fd, guestAddr);
if (g_guestRegionSize && guestAddr >= g_guestRegionSize) {
SafeStr(fd, " <- BEYOND the mapped guest region (");
SafeHex(fd, g_guestRegionSize);
SafeStr(fd, ") - a wild pointer, not a real guest object");
}
SafeStr(fd, "\n");
} else if (g_guestFlatBase) {
SafeStr(fd, "fault address is OUTSIDE the guest window - this is a fault in the "
"engine's own native code, not in emulated guest code\n");
}
SafeStr(fd, "thread id: ");
SafeDec(fd, (long)gettid());
SafeStr(fd, "\n\n");
WriteRegisters(fd, ucontextRaw);
SafeStr(fd,
"\nnote: Android's own tombstone for this crash has more detail "
"(/data/tombstones). This file exists because a tester cannot reach that.\n");
close(fd);
}
}
// Chain to whatever was installed before us so the platform still writes
// its tombstone and the process dies the way it would have. Restoring the
// previous action and re-raising is the portable way to do that.
if (sig >= 0 && sig < NSIG) {
sigaction(sig, &g_prevActions[sig], nullptr);
}
raise(sig);
}
} // namespace
void SetCrashHandlerGuestBase(uint64_t flatBase, uint32_t regionSize) {
g_guestFlatBase = flatBase;
g_guestRegionSize = regionSize;
}
void InstallCrashHandler(const char* crashDir, const char* buildStamp) {
if (g_installed) return;
if (!crashDir || !*crashDir) {
Log("crash_handler: no crash directory given - native crashes will NOT be reported");
return;
}
// mkdir here, at install time. Doing it lazily from the handler would mean
// a filesystem call on a broken process, which is exactly what this file
// exists to avoid.
if (mkdir(crashDir, 0755) != 0 && errno != EEXIST) {
Log("crash_handler: could not create %s (%s) - native crashes will NOT be reported",
crashDir, strerror(errno));
return;
}
// Fixed filename. The handler cannot safely format a timestamp, so Java
// renames this to something unique when it collects it on the next launch.
static const char kFileName[] = "/crash_pending.txt";
size_t dirLen = strlen(crashDir);
if (dirLen + sizeof(kFileName) >= kPathMax) {
Log("crash_handler: crash directory path is too long (%zu) - not installing", dirLen);
return;
}
memcpy(g_reportPath, crashDir, dirLen);
memcpy(g_reportPath + dirLen, kFileName, sizeof(kFileName));
if (buildStamp) {
size_t n = strlen(buildStamp);
if (n >= kStampMax) n = kStampMax - 1;
memcpy(g_buildStamp, buildStamp, n);
g_buildStamp[n] = '\0';
} else {
memcpy(g_buildStamp, "(unknown)", sizeof("(unknown)"));
}
stack_t ss{};
ss.ss_sp = g_altStack;
ss.ss_size = sizeof(g_altStack);
ss.ss_flags = 0;
if (sigaltstack(&ss, nullptr) != 0) {
// Not fatal: without it, a stack-overflow crash goes unreported, but
// every other kind still works. Worth saying out loud rather than
// discovering the gap from a missing report later.
Log("crash_handler: sigaltstack failed (%s) - stack-overflow crashes will not be "
"reported, other crashes still will", strerror(errno));
}
struct sigaction sa{};
sa.sa_sigaction = CrashSignalHandler;
sa.sa_flags = SA_SIGINFO | SA_ONSTACK | SA_RESTART;
sigemptyset(&sa.sa_mask);
static const int kSignals[] = {SIGSEGV, SIGBUS, SIGABRT, SIGILL, SIGFPE};
for (int sig : kSignals) {
if (sigaction(sig, &sa, &g_prevActions[sig]) != 0) {
Log("crash_handler: could not hook signal %d (%s)", sig, strerror(errno));
}
}
g_installed = true;
Log("crash_handler: installed - native crashes will be written to %s", g_reportPath);
}
mpcore/src/main/cpp/crash_handler.h
+33
View File
@@ -0,0 +1,33 @@
#pragma once
// Native crash reporting (task #66 / BETA_TELEMETRY_PLAN.md).
//
// Most crashes in this project are NATIVE - a SIGSEGV inside JIT-generated
// code - so Java's uncaught-exception handler never sees them and the tester
// sees only "the game closed". This writes a report the moment it happens.
//
// What it does NOT do, deliberately: show any UI. A signal handler runs on a
// process that has already gone wrong, where only async-signal-safe calls are
// legal - no malloc, no JNI, no Activity. It writes one file with write() and
// then lets the process die. The report screen is shown by Java on the NEXT
// launch, which is how every serious crash reporter handles native crashes.
//
// The report lands on external storage (Android/data/<pkg>/files/crashes), so
// a tester can reach it over USB or a file manager without any permission.
#include <cstdint>
// Call once, early, with the directory reports should be written to (the app's
// external files dir). Creates the directory if needed, pre-builds every string
// the handler will need, installs an alternate signal stack so a stack-overflow
// crash can still be reported, and hooks the fatal signals.
//
// Safe to call more than once; only the first call installs anything.
void InstallCrashHandler(const char* crashDir, const char* buildStamp);
// Tells the handler where guest memory starts, so a host fault address can be
// reported as the GUEST address that caused it. Without this a report says
// "fault at 0x6f464c459b", which means nothing to anyone; with it the report
// also says "guest 0x464c459b", which is the number worth reading. Called by
// GuestEngine once its region is mapped.
void SetCrashHandlerGuestBase(uint64_t flatBase, uint32_t regionSize);
mpcore/src/main/cpp/emu/dyncast_fastpath.cpp
+131
View File
@@ -0,0 +1,131 @@
// Guest-side __dynamic_cast fast path (task #59, 2026-09-19).
//
// WHY THIS EXISTS, and why it inverts this project's usual instinct.
//
// Everything accelerated so far - crc32, inflate, the FNV hash - was moved
// TO THE HOST, because host code is faster than emulated code. __dynamic_cast
// is the opposite case, and the measurements say so plainly:
//
// - It is 49% of ALL shim crossings during a prologue load (284,986/sec),
// found by the per-shim TOPSHIMS counter.
// - Native ARM32 on the A9 calls it up to 824,200/sec - roughly THREE TIMES
// more (trace_agent's DYNCAST interposer, task #58). So our rate is not
// what the guest wants, it is all we can supply: the boundary is
// throttling it.
// - Cheapening the boundary generally (lock-free dispatch, batched register
// reads - commit 9c4a455) bought only ~3%.
//
// So the win is not in making the call faster, it is in NOT CROSSING. The
// algorithm reads only guest memory (an object's vtable, vtable[-1] = dynamic
// type, vtable[-2] = offset-to-top) and needs nothing from the host, so it can
// run as ordinary emulated ARM32 with zero crossings. Emulated-but-not-
// crossing beats native-but-crossing here.
//
// HOW SMALL IT CAN BE, measured rather than assumed. Instrumenting
// Shim_dynamic_cast over one prologue load (2,745,185 calls):
//
// exact match at depth 0 ... 84.2% <- dynamic type already IS the target
// one base (depth 1) ....... 0.4%
// deeper ................... 0.0% (max depth all run: 3)
// not found ................ 15.4%
//
// 84% need no hierarchy walk at all. This handles exactly that case and tail-
// calls the existing host shim for everything else, so correctness is
// unchanged - the fallback is the same code that served every call before.
// Removing 84% of a 49% share removes ~41% of the total crossing load.
#include "dyncast_fastpath.h"
#include "guest_engine.h"
#include "../util/util.h"
#include <cstring>
namespace {
// Assembled with the NDK's clang for armv7a Thumb-2 (not hand-encoded - see
// this file's .S source kept below verbatim, and objdump output that was
// checked against it). Clobbers ONLY r12, deliberately: r1 (src type_info)
// and r3 (src2dst hint) must reach the fallback untouched, which is why the
// hit path re-loads the vtable instead of keeping it in a second register.
//
// dyncast_fast: @ r0=sub r1=src r2=dst r3=hint
// cmp r0, #0
// beq .Lret0
// ldr r12, [r0] @ vtable pointer
// cmp r12, #0
// beq .Lret0
// ldr r12, [r12, #-4] @ vtable[-1] = dynamic type_info
// cmp r12, r2
// bne .Lslow @ not an exact match -> host shim
// ldr r12, [r0]
// ldr r12, [r12, #-8] @ vtable[-2] = offset-to-top
// add r0, r0, r12
// bx lr
// .Lret0:
// movs r0, #0
// bx lr
// .Lslow:
// ldr r12, .Lslowaddr
// bx r12
// .align 2
// .Lslowaddr:
// .word 0xDEADBEEF @ patched with the slow-path stub
const uint8_t kFastPathCode[] = {
0x00, 0x28, 0x0e, 0xd0, 0xd0, 0xf8, 0x00, 0xc0,
0xbc, 0xf1, 0x00, 0x0f, 0x09, 0xd0, 0x5c, 0xf8,
0x04, 0xcc, 0x94, 0x45, 0x07, 0xd1, 0xd0, 0xf8,
0x00, 0xc0, 0x5c, 0xf8, 0x08, 0xcc, 0x60, 0x44,
0x70, 0x47, 0x00, 0x20, 0x70, 0x47, 0xdf, 0xf8,
0x04, 0xc0, 0x60, 0x47, 0xef, 0xbe, 0xad, 0xde,
};
constexpr uint32_t kSlowAddrPatchOffset = 44; // the 0xDEADBEEF word
} // namespace
// A/B switch (temporary): false leaves __dynamic_cast entirely on the host
// shim, so the control-arena permission change can be measured WITHOUT the
// fast path confounding it - the two landed together and must be separated
// before either is judged.
static constexpr bool kEnableGuestFastPath = false;
void RegisterDynamicCastFastPath(GuestEngine& engine) {
if (!kEnableGuestFastPath) {
Log("dyncast_fastpath: guest fast path DISABLED (kEnableGuestFastPath=false) - "
"__dynamic_cast stays on the host shim");
return;
}
engine.RegisterDataSymbolSetup([](GuestEngine& eng) {
// The fallback target. Registered under its own name so that
// ResolveOrCreateImportStub still builds a real callable stub for it -
// the data symbol installed below would otherwise win for the name
// "__dynamic_cast" and no stub would ever exist to fall back to.
GuestAddr slowStub = eng.ResolveOrCreateImportStub("__dynamic_cast_slowpath");
if (!slowStub) {
Log("dyncast_fastpath: could not create the slow-path stub - leaving __dynamic_cast "
"on the host shim entirely (no fast path installed)");
return;
}
// AllocGuestCode, NOT AllocPermanent: the latter carves from the
// control arena, which is mapped read-write only. Placing code there
// produced an immediate FETCH_PROT at this function's own entry, and
// making that arena executable to accommodate it cost a measured ~4s
// of load time (more than this fast path saves). The trampoline arena
// is already executable and is not a write-hot region.
GuestAddr code = eng.AllocGuestCode(sizeof(kFastPathCode));
if (!code) {
Log("dyncast_fastpath: AllocGuestCode(%zu) failed - leaving __dynamic_cast on the "
"host shim entirely (no fast path installed)", sizeof(kFastPathCode));
return;
}
memcpy(eng.G2H(code), kFastPathCode, sizeof(kFastPathCode));
memcpy(eng.G2H(code + kSlowAddrPatchOffset), &slowStub, 4);
// Thumb bit - this engine's only real mode (see MapSegments). Without
// it the guest would branch here in ARM mode and misdecode every byte.
GuestAddr entry = code | 1u;
eng.RegisterDataSymbol("__dynamic_cast", entry);
Log("dyncast_fastpath: __dynamic_cast now resolves to guest code at 0x%x (slow path stub "
"0x%x) - the 84%% exact-match case no longer crosses the shim boundary",
entry, slowStub);
});
}
mpcore/src/main/cpp/emu/dyncast_fastpath.h
+13
View File
@@ -0,0 +1,13 @@
#pragma once
// Installs a guest-side ARM32 fast path for __dynamic_cast so the common case
// never crosses the shim boundary. See dyncast_fastpath.cpp's own comment for
// the measurements that motivated it (it is 49% of all crossings, native calls
// it 3x more often than we can serve, and 84% of calls need no hierarchy walk).
//
// Must be called during shim registration, BEFORE the image is loaded: it
// registers a data-symbol setup function, which the loader runs after
// MapSegments (AllocPermanent is live by then) and before ProcessRelocations
// (so the resolved address reaches the GOT).
class GuestEngine;
void RegisterDynamicCastFastPath(GuestEngine& engine);
mpcore/src/main/cpp/emu/gles_shim.cpp
+112 -3
View File
@@ -4,6 +4,7 @@
#include <cstring>
#include <vector>
#include <set>
#include <tuple>
#include <unordered_map>
#include <atomic>
#include <thread>
@@ -13,6 +14,12 @@
#include <EGL/egl.h>
#include <android/bitmap.h>
// Defined in guest_engine.cpp at global scope - counts EVERY shim crossing
// (libc, JNI, GLES), which is what the PERF line needs to answer whether our
// shim boundary is what paces the frame. Declared here rather than in a
// header because it is temporary task #56 instrumentation.
extern std::atomic<uint64_t> g_stubDispatchCount;
// 2026-09-18 (task #39): needs real external linkage, unlike everything else
// in this file's anonymous namespace below, because guest_engine.cpp's
// TextClipDispatchEntryProbeHookCb reads it via `extern` from a different
@@ -63,6 +70,10 @@ uint32_t GuestCallerLR(GuestEngine& eng) {
// eglSwapBuffers - see Shim_glClear's own comment for why presenting a
// back buffer that received none of them is what made the screen cycle
// between stale, black and real frames.
// Task #52 probe switch - see Shim_glViewport. Opt-in per this
// project's standing rule that diagnostics default to off.
constexpr bool kLogViewportChanges = false;
std::atomic<int> g_drawsSinceSwap{0};
// Shadowed GL state (2026-09-19). These exist so the per-draw code paths
@@ -86,6 +97,61 @@ void NoteDrawForSwapGate() {
}
}
// Task #56 live metrics (2026-09-19, temporary - flip off when the session
// ends). The user's observation is the whole reason this exists: after
// crashing into a wall with nothing ahead the game runs fast, and the moment
// objects or particles (motion blur, nitro, collision sparks) appear the
// speed drops at once. That points at per-DRAW cost on our side rather than
// at physics, race logic, or the GPU - the in-race profile had the Mali
// driver at only 1.3%.
//
// So: report draw calls and frames once a second, together, in ONE line, so
// the correlation can be read live while driving instead of joining two logs
// by timestamp afterwards. Frames are counted as synthesized-frame
// boundaries would be - here, default-framebuffer draws reset per swap - so
// drawsPerFrame is the number that matters.
constexpr bool kLogPerfMetrics = false;
std::atomic<uint64_t> g_totalDraws{0};
std::atomic<uint64_t> g_totalIndices{0};
std::atomic<uint64_t> g_framesSeen{0};
void NotePerfDraw(GLsizei count) {
if (!kLogPerfMetrics) return;
g_totalDraws.fetch_add(1, std::memory_order_relaxed);
g_totalIndices.fetch_add((uint64_t)(count > 0 ? count : 0), std::memory_order_relaxed);
static std::atomic<uint64_t> lastNs{0};
uint64_t now = (uint64_t)std::chrono::duration_cast<std::chrono::nanoseconds>(
std::chrono::steady_clock::now().time_since_epoch()).count();
uint64_t last = lastNs.load(std::memory_order_relaxed);
if (now - last < 1000000000ull) return;
if (!lastNs.compare_exchange_strong(last, now, std::memory_order_relaxed)) return;
if (last == 0) return;
static uint64_t prevDraws = 0, prevIdx = 0, prevFrames = 0, prevGl = 0;
double dt = (now - last) / 1e9;
uint64_t d = g_totalDraws.load(std::memory_order_relaxed);
uint64_t ix = g_totalIndices.load(std::memory_order_relaxed);
uint64_t f = g_framesSeen.load(std::memory_order_relaxed);
double frames = (double)(f - prevFrames) / dt;
double draws = (double)(d - prevDraws) / dt;
// glCalls distinguishes the two candidate explanations for the drop the
// user demonstrated (task #56). If GLES shim calls per FRAME rise along
// with draws, the cost is on our side of the boundary - more state
// changes, each one a shim crossing. If calls per frame stay flat while
// fps collapses, our boundary is not what is pacing the frame, and the
// next suspect is the GPU: particles are blended overdraw, and a CPU
// profile cannot see a thread that is BLOCKED rather than busy (the
// lesson task #49 already charged us for once).
uint64_t gl = g_stubDispatchCount.load(std::memory_order_relaxed);
Log("gles_shim: PERF fps=%.1f draws/s=%.0f draws/frame=%.0f indices/draw=%.0f "
"shimCalls/s=%.0f shimCalls/frame=%.0f shimCalls/draw=%.1f",
frames, draws, frames > 0 ? draws / frames : 0.0,
(d - prevDraws) ? (double)(ix - prevIdx) / (double)(d - prevDraws) : 0.0,
(double)(gl - prevGl) / dt, frames > 0 ? (double)(gl - prevGl) / dt / frames : 0.0,
(d - prevDraws) ? (double)(gl - prevGl) / (double)(d - prevDraws) : 0.0);
prevGl = gl;
prevDraws = d; prevIdx = ix; prevFrames = f;
}
std::atomic<int> g_textUploadTraceWindow{0};
bool LooksLikeTextOverlayUpload(GLsizei width, GLsizei height) {
// 738x302 specifically, plus a little slack for other similarly-shaped
@@ -533,11 +599,23 @@ uint32_t Shim_glClear(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, u
// is found - that symptom was diagnosed under the false premise above and
// its actual explanation is therefore still open.
constexpr bool kSynthesizeSwap = false;
const bool frameBoundary =
kSynthesizeSwap &&
// The boundary CONDITION is still worth evaluating even though we no
// longer present on it (task #47): it is the only per-frame signal this
// shim has natively, and task #56 needs draws-per-FRAME, not draws per
// second. Counting here also restores the g_drawsSinceSwap reset that the
// disabled swap block used to perform - without it "drew into screen"
// would latch true after the first draw and every later FBO clear would
// be counted as a frame.
const bool rawFrameBoundary =
((boundFb != 0 && drewIntoScreen) ||
(boundFb == 0 && clearsColor && drewIntoScreen &&
!sawFboClear.load(std::memory_order_relaxed)));
if (rawFrameBoundary && kLogPerfMetrics && !kSynthesizeSwap) {
if (g_drawsSinceSwap.exchange(0, std::memory_order_relaxed) > 0) {
g_framesSeen.fetch_add(1, std::memory_order_relaxed);
}
}
const bool frameBoundary = kSynthesizeSwap && rawFrameBoundary;
if (frameBoundary) {
static std::atomic<bool> hasClearedBefore{false};
// 2026-09-19: also require that SOMETHING was actually drawn into the
@@ -809,6 +887,7 @@ uint32_t Shim_glDrawArrays(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t
glDrawArrays((GLenum)eng.ReadIncomingArg(0, r0, r1, r2, r3, sp), (GLint)eng.ReadIncomingArg(1, r0, r1, r2, r3, sp), count);
g_drawArraysCalls.fetch_add(1, std::memory_order_relaxed);
g_drawArraysVerts.fetch_add((uint64_t)(count > 0 ? count : 0), std::memory_order_relaxed);
NotePerfDraw(count);
NoteDrawForSwapGate();
return 0;
}
@@ -981,6 +1060,7 @@ uint32_t Shim_glDrawElements(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_
: (a3_g ? eng.G2H(a3_g) : nullptr);
glDrawElements((GLenum)eng.ReadIncomingArg(0, r0, r1, r2, r3, sp), count, (GLenum)eng.ReadIncomingArg(2, r0, r1, r2, r3, sp), indicesArg);
g_drawElementsCalls.fetch_add(1, std::memory_order_relaxed);
NotePerfDraw(count);
NoteDrawForSwapGate();
g_drawElementsIndices.fetch_add((uint64_t)(count > 0 ? count : 0), std::memory_order_relaxed);
{
@@ -1722,7 +1802,36 @@ uint32_t Shim_glVertexAttribPointer(GuestEngine& eng, uint32_t r0, uint32_t r1,
}
uint32_t Shim_glViewport(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
glViewport((GLint)eng.ReadIncomingArg(0, r0, r1, r2, r3, sp), (GLint)eng.ReadIncomingArg(1, r0, r1, r2, r3, sp), (GLsizei)eng.ReadIncomingArg(2, r0, r1, r2, r3, sp), (GLsizei)eng.ReadIncomingArg(3, r0, r1, r2, r3, sp));
GLint vx = (GLint)eng.ReadIncomingArg(0, r0, r1, r2, r3, sp);
GLint vy = (GLint)eng.ReadIncomingArg(1, r0, r1, r2, r3, sp);
GLsizei vw = (GLsizei)eng.ReadIncomingArg(2, r0, r1, r2, r3, sp);
GLsizei vh = (GLsizei)eng.ReadIncomingArg(3, r0, r1, r2, r3, sp);
// Task #52 probe (2026-09-19, temporary). A Xiaomi 14 capture showed the
// game rendering into viewport=[0,0,2136,960] while the surface was
// 2670x1200 - i.e. a reduced internal resolution being upscaled. Whether
// that is the game's own choice or a wrong size we fed it is unknown, and
// the project already has form here: task #39 was exactly this shape (the
// UI projection used 2000x1000 instead of the real 2400x1080 because a
// JNI float argument was arriving as zero). Logging each DISTINCT
// viewport against the real EGL surface size, so the comparison is a fact
// rather than an inference.
if (kLogViewportChanges) {
static std::mutex mu;
static std::set<std::tuple<GLint, GLint, GLsizei, GLsizei>> seen;
std::lock_guard<std::mutex> lock(mu);
if (seen.insert({vx, vy, vw, vh}).second && seen.size() <= 24) {
EGLint sw = -1, sh = -1;
EGLDisplay dpy = eglGetCurrentDisplay();
EGLSurface surf = eglGetCurrentSurface(EGL_DRAW);
eglQuerySurface(dpy, surf, EGL_WIDTH, &sw);
eglQuerySurface(dpy, surf, EGL_HEIGHT, &sh);
GLint fb = g_shadowBoundFramebuffer.load(std::memory_order_relaxed);
Log("gles_shim: VIEWPORT [%d,%d %dx%d] fb=%d | EGL surface %dx%d%s",
vx, vy, vw, vh, fb, sw, sh,
(fb == 0 && (vw != sw || vh != sh)) ? " <<< SMALLER THAN THE SCREEN >>>" : "");
}
}
glViewport(vx, vy, vw, vh);
return 0;
}
mpcore/src/main/cpp/emu/guest_engine.cpp
+462 -28
View File
@@ -4,11 +4,14 @@
#include "zlib_accel.h"
#include "name_lookup_accel.h"
#include "../util/util.h"
#include "../crash_handler.h"
#include <cstdio>
#include <cstring>
#include <cerrno>
#include <cstdlib>
#include <sys/mman.h>
#include <dlfcn.h>
#include <set>
#include <array>
#include <string>
#include <thread>
@@ -133,6 +136,13 @@ constexpr uint32_t kGuardPageSize = kPageSize;
// TCG to re-translate (tb_invalidate_phys_page_fast_arm, 3.15%). If this
// reports zero, the invalidation comes from somewhere else and that lead
// is dead.
// Task #61: bypass the software MMU and let generated code reach guest memory
// as `[X28, Wn, UXTW]`. Kept as an explicit switch because it trades away real
// safety nets (UC_PROT_* enforcement, UC_HOOK_MEM_* callbacks, self-modifying
// code detection) - when chasing a memory-corruption bug, turn it OFF to get
// MEM FAULT diagnostics back, then turn it on again.
constexpr bool kUseFlatGuestMapping = true;
constexpr bool kCountStubDispatches = false;
constexpr bool kCountTextWrites = false;
// kCountArenaWrites - task #54. tb_invalidate_phys_page_fast_arm is 5.56% of
@@ -169,7 +179,13 @@ constexpr uint32_t kControlArenaSize = 256 * 1024;
// engine is not expected to spawn anywhere near this many worker threads -
// generous headroom, not a measured requirement. CarveThreadStack() fails
// (logged, not fatal) if this is ever exceeded.
constexpr uint32_t kMaxGuestThreads = 16;
// Raised 16 -> 64 (2026-09-21) alongside ReleaseThreadEngine's free list.
// The free list is the actual fix for the exhaustion that showed up as a black
// screen entering a race; this is headroom for the genuinely-concurrent peak,
// since a single thread can hold one primary stack plus up to
// kMaxNestedEngines nested ones at the same time. The arena is lazily-mapped
// address space (see MapSegments), so unused entries cost no resident memory.
constexpr uint32_t kMaxGuestThreads = 64;
constexpr uint32_t kThreadStacksArenaSize = kMaxGuestThreads * kStackSize;
// AllocMmap's arena - backs real anonymous mmap() (see libc_shims.cpp's
@@ -186,9 +202,46 @@ constexpr uint32_t kArmLdrPcPcMinus4 = 0xE51FF004u;
struct ImportStubEntry {
std::string name;
ImportShimFn shim; // nullptr if not implemented - see guest_engine's own unresolved-import log-once behavior
// Task #67. Non-zero means "this symbol IS implemented, by real guest code
// in a secondary image, and we want to watch the call". The dispatcher
// logs the arguments, forwards to that address, and logs what comes back.
// Used to find where the game's FMOD initialisation chain stops: the three
// entry points libapp.so imports all return an FMOD_RESULT, so the first
// non-zero one names the failure.
//
// Deliberately on OUR side of the boundary. The same thing attempted from
// outside, via an LD_PRELOAD interposer on the A9, could not resolve the
// real symbols and ended up standing in for them - silencing audio on the
// reference device. Here the real address is known, so forwarding cannot
// silently degrade into replacing.
GuestAddr tracedGuestTarget = 0;
// How many arguments the traced function really takes. AAPCS32 puts the
// first four in r0-r3 and the REST ON THE STACK, so forwarding only the
// registers silently truncates the call. The first version of this probe
// did exactly that and produced error codes that were its own doing -
// FMOD_Memory_Initialize takes six arguments, EventSystem::init five.
int tracedArgCount = 4;
bool loggedUnresolved = false;
// Task #56 instrumentation (2026-09-19, temporary). Live measurement on
// the Xiaomi 14 showed shim crossings/sec pinned near a ceiling (~170k)
// regardless of scene, with the frame rate falling out as
// ceiling / crossings-per-frame - 148350/4385 = 33.8fps fast,
// 167811/21665 = 7.7fps slow, both exact. Draw calls turned out NOT to
// be the driver (178 vs 149 per frame across a 4x fps gap). So the
// question is WHICH shims make up that flood. Counted per entry rather
// than in a shared map because a mutex on every crossing would tax the
// very thing being measured.
std::atomic<uint64_t> calls{0};
uint64_t prevCalls = 0;
};
// All entries ever created, for the per-second top-N report. Appended under
// the mutex at creation time (rare); read without one by the reporter, which
// is safe because entries are never destroyed and the vector only grows
// during startup.
std::mutex g_importStubEntriesMutex;
std::vector<ImportStubEntry*> g_importStubEntries;
// Diagnostic only (see ARM64_TRANSLATION_LAYER.md's "why does the process
// die" investigation) - fires on every UC_ERR_READ/WRITE/FETCH_UNMAPPED
// fault AND every _PROT violation (registered as UC_HOOK_MEM_INVALID, not
@@ -2215,17 +2268,85 @@ void RenderCrashProbeHookCb(uc_engine* uc, uint64_t, uint32_t, void*) {
// executes the real `BX LR` immediately after, still inside the *same*
// uc_emu_start() call, using LR exactly as it already stands (untouched by
// this function) for correct ARM/Thumb interworking.
// Prints the busiest shims once a second. Time-gated with a CAS so only one
// thread does the work; everything else just returns after its increment.
void ReportTopShims() {
static std::atomic<uint64_t> lastNs{0};
uint64_t now = (uint64_t)std::chrono::duration_cast<std::chrono::nanoseconds>(
std::chrono::steady_clock::now().time_since_epoch()).count();
uint64_t last = lastNs.load(std::memory_order_relaxed);
if (now - last < 1000000000ull) return;
if (!lastNs.compare_exchange_strong(last, now, std::memory_order_relaxed)) return;
if (last == 0) return;
double dt = (now - last) / 1e9;
std::vector<std::pair<uint64_t, ImportStubEntry*>> hot;
uint64_t total = 0;
{
std::lock_guard<std::mutex> lock(g_importStubEntriesMutex);
for (auto* e : g_importStubEntries) {
uint64_t c = e->calls.load(std::memory_order_relaxed);
uint64_t delta = c - e->prevCalls;
e->prevCalls = c;
total += delta;
if (delta) hot.emplace_back(delta, e);
}
}
if (hot.empty()) return;
std::sort(hot.begin(), hot.end(), [](auto& a, auto& b) { return a.first > b.first; });
std::string s;
for (size_t i = 0; i < hot.size() && i < 8; i++) {
char buf[128];
snprintf(buf, sizeof(buf), " %s=%.0f/s(%.0f%%)", hot[i].second->name.c_str(),
(double)hot[i].first / dt, 100.0 * (double)hot[i].first / (double)total);
s += buf;
}
Log("GuestEngine: TOPSHIMS total=%.0f/s over %zu distinct |%s", (double)total / dt, hot.size(),
s.c_str());
}
void import_stub_dispatch_cb(uc_engine* uc, uint64_t address, uint32_t /*size*/, void* user_data) {
auto* entry = static_cast<ImportStubEntry*>(user_data);
if (kCountStubDispatches) {
entry->calls.fetch_add(1, std::memory_order_relaxed);
ReportTopShims();
}
// One batched read instead of five separate uc_reg_read calls (task #56).
// Each uc_reg_read is a full Unicorn API entry - argument checks plus a
// per-arch switch - and this runs on EVERY shim crossing, measured at
// ~170,000/sec during gameplay. uc_reg_read_batch does the same work
// behind a single entry.
uint32_t r0 = 0, r1 = 0, r2 = 0, r3 = 0, sp = 0;
uc_reg_read(uc, UC_ARM_REG_R0, &r0);
uc_reg_read(uc, UC_ARM_REG_R1, &r1);
uc_reg_read(uc, UC_ARM_REG_R2, &r2);
uc_reg_read(uc, UC_ARM_REG_R3, &r3);
uc_reg_read(uc, UC_ARM_REG_SP, &sp);
{
static const int kRegs[5] = {UC_ARM_REG_R0, UC_ARM_REG_R1, UC_ARM_REG_R2,
UC_ARM_REG_R3, UC_ARM_REG_SP};
void* vals[5] = {&r0, &r1, &r2, &r3, &sp};
uc_reg_read_batch(uc, kRegs, vals, 5);
}
uint32_t result = 0;
if (entry->shim) {
if (entry->tracedGuestTarget) {
GuestEngine& eng = GuestEngine::Instance();
// Rebuild the WHOLE argument list, stack arguments included, before
// forwarding - see tracedArgCount's own comment for what truncating it
// cost the first time.
uint32_t args[8] = {r0, r1, r2, r3, 0, 0, 0, 0};
int n = entry->tracedArgCount;
if (n < 1) n = 1;
if (n > 8) n = 8;
for (int i = 4; i < n; i++) {
args[i] = eng.ReadIncomingArg(i, r0, r1, r2, r3, sp);
}
std::string argText;
for (int i = 0; i < n; i++) {
char buf[24];
snprintf(buf, sizeof(buf), "%s0x%x", i ? ", " : "", args[i]);
argText += buf;
}
Log("GUESTCALL %s(%s) - entering real guest code at 0x%x",
entry->name.c_str(), argText.c_str(), entry->tracedGuestTarget);
result = eng.CallGuestFunction(entry->tracedGuestTarget, args, n);
Log("GUESTCALL %s -> 0x%x", entry->name.c_str(), result);
} else if (entry->shim) {
result = entry->shim(GuestEngine::Instance(), r0, r1, r2, r3, sp);
} else {
if (!entry->loggedUnresolved) {
@@ -2360,6 +2481,19 @@ bool GuestEngine::LoadImage(const char* path) {
// addresses by the time any GOT slot references them.
for (auto fn : data_symbol_setup_fns_) fn(*this);
EnsureThreadEngine();
// Sibling guest libraries, loaded HERE - after the engine exists (they
// need one to build their own import stubs) but BEFORE this image's
// relocations are processed. The ordering is the whole point: a GOT
// slot resolved against a shim cannot be un-resolved later, so anything
// that should reach real code has to be exported before the first
// relocation is applied (task #67).
//
// Failure is not fatal. A missing library leaves fmod_shims.cpp's
// no-ops in charge, which is exactly the behaviour that shipped before
// this - the game runs, silently, rather than not at all.
if (t_state_.uc) LoadSiblingLibraries(path);
ok = t_state_.uc != nullptr && ProcessRelocations(fileData, fileSize);
}
free(fileData);
@@ -2445,13 +2579,42 @@ bool GuestEngine::MapSegments(const uint8_t* fileData, size_t fileSize) {
mmap_end_ = mmap_cursor_ + kMmapArenaSize;
region_size_ = AlignUp(mmap_end_ + kPageSize, kPageSize);
void* backing = mmap(nullptr, region_size_, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
// Reserve the FULL 4 GiB a 32-bit guest address can name, then commit the
// part we actually use at its start (task #61).
//
// Why reserve the whole span rather than just region_size_: with the flat
// mapping enabled (uc_set_flat_map_base), generated code reaches guest
// memory as `[X28, Wn, UXTW]` - the guest address is zero-extended and
// added with no bounds check whatsoever. A wild guest pointer beyond
// region_size_ (we have hit real ones, e.g. 0x3d3d3d3d in task #33) would
// otherwise land on whatever unrelated host mapping happens to sit there
// and be read or WRITTEN silently. Holding the whole 4 GiB as PROT_NONE
// turns every such access into an immediate, honest SIGSEGV instead.
//
// This costs address space, not memory: PROT_NONE|MAP_NORESERVE commits no
// pages, and the host is 64-bit so 4 GiB of VA is unremarkable.
constexpr size_t kGuestAddressSpace = 4ull * 1024 * 1024 * 1024;
void* reservation = mmap(nullptr, kGuestAddressSpace, PROT_NONE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE, -1, 0);
if (reservation == MAP_FAILED) {
Log("GuestEngine: reserving the 4GiB guest address space failed (%s) - cannot continue",
strerror(errno));
return false;
}
void* backing = mmap(reservation, region_size_, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0);
if (backing == MAP_FAILED) {
Log("GuestEngine: mmap(0x%x) for guest region failed", region_size_);
Log("GuestEngine: mmap(0x%x) for guest region failed (%s)", region_size_, strerror(errno));
munmap(reservation, kGuestAddressSpace);
return false;
}
host_region_ = static_cast<uint8_t*>(backing);
// The crash handler needs this to turn a host fault address back into the
// guest address that caused it - see crash_handler.h. Set as soon as the
// region exists, so a crash during the rest of loading is already decodable.
SetCrashHandlerGuestBase((uint64_t)(uintptr_t)host_region_, region_size_);
// No uc_engine created here anymore - each host thread gets its own,
// mapped onto this same host_region_, the first time it needs one (see
// EnsureThreadEngine). This call (LoadImage's caller) will get its
@@ -2622,6 +2785,36 @@ bool GuestEngine::ProcessRelocations(const uint8_t* fileData, size_t fileSize) {
// of a clean crash - a small, self-contained, independently-readable
// function is safer here than a shared helper parameterized just enough to
// serve both call sites' different assumptions.
void GuestEngine::LoadSiblingLibraries(const char* primaryImagePath) {
// libfmodex FIRST: libfmodevent lists it in its own DT_NEEDED, so its
// relocations only resolve to real code if libfmodex's exports are already
// recorded. Load order here IS dependency order.
static const char* kSiblings[] = { "libfmodex.so", "libfmodevent.so" };
std::string dir(primaryImagePath ? primaryImagePath : "");
size_t slash = dir.find_last_of('/');
dir = (slash == std::string::npos) ? std::string(".") : dir.substr(0, slash);
// Every instruction in libfmodex that loads FMOD_RESULT 33 into a register
// (found by scanning the disassembly for the constant - 39 sites, far too
// many to reason about by hand). Watching them all at once turns "which
// one fires" into an observation instead of an argument. Offsets are
// file-relative and biased by the load address below.
//
// Temporary, for the audio bring-up (task #67) - delete the list to remove
// the hooks entirely.
for (const char* lib : kSiblings) {
std::string full = dir + "/" + lib;
if (LoadSecondaryImage(full.c_str(), nullptr)) continue;
// Say which one and what it costs. A silent miss here would present
// later as "the game runs but makes no sound", with nothing pointing
// back to the real cause.
Log("GuestEngine::LoadSiblingLibraries: could not load %s - its entry points stay on the "
"no-op shims in fmod_shims.cpp, so expect no audio from it", full.c_str());
}
}
GuestAddr GuestEngine::LoadSecondaryImage(const char* path, const char* entrySymbol) {
if (!host_region_) {
Log("GuestEngine::LoadSecondaryImage(%s): no primary image loaded yet (host_region_ null) - "
@@ -2826,24 +3019,67 @@ GuestAddr GuestEngine::LoadSecondaryImage(const char* path, const char* entrySym
for (size_t s = 0; s < symCount; s++) {
if (syms[s].st_shndx == SHN_UNDEF) continue;
const char* name = strs + syms[s].st_name;
if (strcmp(name, entrySymbol) == 0) {
entryAddr = base + syms[s].st_value; // st_value already carries the Thumb bit, same convention as everywhere else
break;
if (!name || !*name) continue;
// st_value already carries the Thumb bit, same convention as everywhere else.
GuestAddr addr = base + syms[s].st_value;
// Record the export so a later import of this name reaches the
// real code. First definition wins, matching how a real dynamic
// linker resolves against load order - and a collision is worth
// saying out loud rather than silently preferring one.
auto existing = secondary_image_exports_.find(name);
if (existing == secondary_image_exports_.end()) {
secondary_image_exports_[name] = addr;
} else if (existing->second != addr) {
Log("GuestEngine::LoadSecondaryImage(%s): symbol '%s' already exported at 0x%x by an "
"earlier image - keeping the first, ignoring 0x%x", path, name, existing->second, addr);
}
if (entrySymbol && !entryAddr && strcmp(name, entrySymbol) == 0) {
entryAddr = addr;
}
}
}
}
free(fileData);
if (!entryAddr) {
if (entrySymbol && !entryAddr) {
Log("GuestEngine::LoadSecondaryImage: entry symbol '%s' not found in %s's .dynsym - "
"was it declared extern \"C\" with default visibility?", entrySymbol, path);
return 0;
}
Log("GuestEngine::LoadSecondaryImage: loaded %s at base=0x%x span=0x%x - entry '%s'=0x%x",
path, base, span, entrySymbol, entryAddr);
return entryAddr;
secondary_image_ranges_.push_back({base, span});
Log("GuestEngine::LoadSecondaryImage: loaded %s at base=0x%x span=0x%x - entry '%s'=0x%x, "
"%zu exported symbol(s) now available to later imports",
path, base, span, entrySymbol ? entrySymbol : "(none requested)", entryAddr,
secondary_image_exports_.size());
return entrySymbol ? entryAddr : 1;
}
// Symbols whose calls are logged and forwarded rather than branched to
// directly (task #67, temporary). Empty this list to remove the overhead -
// each entry costs a host crossing per call, so only initialisation-time
// functions belong here, never anything on a per-frame path.
// Argument counts come from FMOD's public headers, not from guesswork:
// FMOD_Memory_Initialize(poolmem, poollen, useralloc, userrealloc,
// userfree, memtypeflags) - 6
// FMOD_EventSystem_Create(eventsystem) - 1
// EventSystem::init(this, maxchannels, flags, extradriverdata,
// eventflags) - 5
static int TracedGuestSymbolArgCount(const std::string& name) {
if (name == "FMOD_Memory_Initialize") return 6;
if (name == "FMOD_EventSystem_Create") return 1;
if (name == "_ZN4FMOD11EventSystem4initEijPvj") return 5;
// System::init(this, maxchannels, flags, extradriverdata) - libfmodevent
// calls this INTO libfmodex, so it crosses an image boundary and goes
// through import resolution like any other. Watched to separate "the event
// layer refused" from "the core system refused".
if (name == "_ZN4FMOD6System4initEijPv") return 4;
return 0; // not traced
}
static bool IsTracedGuestSymbol(const std::string& name) {
return TracedGuestSymbolArgCount(name) > 0;
}
GuestAddr GuestEngine::ResolveOrCreateImportStub(const std::string& symbolName) {
@@ -2861,8 +3097,50 @@ GuestAddr GuestEngine::ResolveOrCreateImportStub(const std::string& symbolName)
return dataIt->second;
}
// A symbol DEFINED by a secondary guest image is real code - branch
// straight to it, no stub and no host crossing. Checked before the shim
// table on purpose: once the game's own FMOD is loaded, its entry points
// must win over fmod_shims.cpp's no-op stand-ins.
auto exportIt = secondary_image_exports_.find(symbolName);
if (exportIt != secondary_image_exports_.end()) {
if (registered_shims_.count(symbolName)) {
Log("GuestEngine: '%s' resolves to real guest code at 0x%x from a secondary image - "
"the registered shim for it is now unused", symbolName.c_str(), exportIt->second);
}
// Watched symbols get a stub that forwards; everything else branches
// straight to the real code with no stub and no host crossing.
if (!IsTracedGuestSymbol(symbolName)) {
import_stub_by_symbol_[symbolName] = exportIt->second;
return exportIt->second;
}
auto* traced = new ImportStubEntry();
traced->name = symbolName;
traced->shim = nullptr;
traced->tracedGuestTarget = exportIt->second;
traced->tracedArgCount = TracedGuestSymbolArgCount(symbolName);
{
std::lock_guard<std::mutex> lock(g_importStubEntriesMutex);
g_importStubEntries.push_back(traced);
}
GuestAddr tracedStub = AllocCodeStub(import_stub_dispatch_cb, traced);
if (!tracedStub) {
Log("GuestEngine: could not allocate a tracing stub for '%s' - calling it directly, "
"untraced", symbolName.c_str());
import_stub_by_symbol_[symbolName] = exportIt->second;
return exportIt->second;
}
Log("GuestEngine: watching '%s' - calls will be logged and forwarded to 0x%x",
symbolName.c_str(), exportIt->second);
import_stub_by_symbol_[symbolName] = tracedStub;
return tracedStub;
}
auto* entry = new ImportStubEntry();
entry->name = symbolName;
{
std::lock_guard<std::mutex> lock(g_importStubEntriesMutex);
g_importStubEntries.push_back(entry);
}
auto shimIt = registered_shims_.find(symbolName);
entry->shim = (shimIt != registered_shims_.end()) ? shimIt->second : nullptr;
@@ -2892,6 +3170,30 @@ uc_engine* GuestEngine::CreateConfiguredEngine() {
Log("GuestEngine::CreateConfiguredEngine: uc_open failed: %d", (int)err);
return nullptr;
}
// Task #61: tell the JIT it can reach guest memory by plain addition.
//
// Must happen here, before anything is translated on this engine. Our
// guest address space is already exactly what the flat path requires - one
// contiguous host block where G2H(a) == host_region_ + a - so the software
// TLB is pure overhead: a nine-instruction check per access, measured at
// 29.8% of all generated host code.
//
// The price, paid knowingly: UC_PROT_* is no longer enforced on data
// accesses, UC_HOOK_MEM_* stops firing (so MEM FAULT diagnostics go
// quiet), and guest stores no longer invalidate translated code. The
// 4 GiB PROT_NONE reservation in MapSegments is what keeps a wild guest
// pointer from silently touching unrelated host memory.
if (kUseFlatGuestMapping) {
uc_err flatErr = uc_set_flat_map_base(newUc, (uint64_t)(uintptr_t)host_region_);
if (flatErr != UC_ERR_OK) {
// Not fatal - the software MMU still works - but it means this
// engine silently runs several times slower than its siblings,
// which is exactly the kind of thing that must not pass in silence.
Log("GuestEngine::CreateConfiguredEngine: uc_set_flat_map_base failed: %d - this "
"engine falls back to the software MMU and will be much slower", (int)flatErr);
}
}
// uc_mem_map_ptr shares OUR host buffer directly as the guest's backing
// store, instead of Unicorn allocating its own - this is what makes
// G2H/H2G plain pointer arithmetic (see guest_engine.h's class comment).
@@ -2957,6 +3259,15 @@ uc_engine* GuestEngine::CreateConfiguredEngine() {
{0, image_end_, UC_PROT_ALL, "image (RWX)"},
{image_end_, heap_end_ - image_end_, UC_PROT_READ | UC_PROT_WRITE, "heap (RW)"},
{heap_end_, misc_stub_end_ - heap_end_, UC_PROT_ALL, "trampoline+stubs (RWX)"},
// Control arena keeps EXEC: AllocPermanent hands out memory from it,
// and generated guest code lives there - the __dynamic_cast fast path
// (dyncast_fastpath.cpp) is the first such case. Found the direct way:
// marking this span non-executable produced an immediate, precisely
// located FETCH_PROT at the fast path's own entry address. It is also
// not the span that motivated the split - the STACKS are what absorb
// the write traffic, and they stay RW below, so task #54's win is
// untouched (control arena is kControlArenaSize, a rounding error next
// to the 768MB heap).
{misc_stub_end_, thread_stacks_end_ - misc_stub_end_, UC_PROT_READ | UC_PROT_WRITE, "control+stacks (RW)"},
{thread_stacks_end_, region_size_ - thread_stacks_end_, UC_PROT_ALL, "mmap arena (RWX)"},
};
@@ -3342,6 +3653,15 @@ void GuestEngine::ReplayHooksOnEngine(uc_engine* newUc) {
GuestAddr GuestEngine::CarveThreadStack() {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
// Reuse a returned stack before growing the arena. The guest does not read
// a fresh stack before writing it, so no scrubbing is needed here - and
// leaving the previous thread's bytes in place has caught real
// uninitialized-read bugs before.
if (!thread_stack_free_list_.empty()) {
GuestAddr top = thread_stack_free_list_.back();
thread_stack_free_list_.pop_back();
return top;
}
if (thread_stacks_cursor_ + kStackSize > thread_stacks_end_) {
return 0;
}
@@ -3350,7 +3670,46 @@ GuestAddr GuestEngine::CarveThreadStack() {
return base + kStackSize; // top = highest usable address, same convention the old stack_top_ used
}
void GuestEngine::ReleaseThreadEngine() {
// Nested-call engines first: each one holds its OWN dedicated stack
// (GetOrCreateNestedEngine), and those leaked exactly like the primary
// one did - a single guest thread could consume several stacks out of the
// arena and give none of them back.
for (uint32_t i = 0; i < kMaxNestedEngines; i++) {
if (t_state_.nestedEngines[i]) {
uc_close(t_state_.nestedEngines[i]);
t_state_.nestedEngines[i] = nullptr;
}
if (t_state_.nestedStackTop[i]) {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
thread_stack_free_list_.push_back(t_state_.nestedStackTop[i]);
t_state_.nestedStackTop[i] = 0;
}
}
if (!t_state_.uc) return;
uc_close(t_state_.uc);
t_state_.uc = nullptr;
if (t_state_.stackTop) {
std::lock_guard<std::mutex> lock(thread_stacks_mutex_);
thread_stack_free_list_.push_back(t_state_.stackTop);
}
t_state_.stackBase = 0;
t_state_.stackTop = 0;
t_state_.callDepth = 0;
t_state_.lastCallHiWord = 0;
}
// Task #56: every shim crossing (libc, JNI and GLES alike) passes through
// here, so one relaxed increment gives a total that gles_shim's PERF line can
// print alongside fps and draws in a SINGLE log line - which is the whole
// point, since the question is whether shim calls per FRAME rise when the
// frame rate collapses. At ~50-90k dispatches/sec a relaxed add is noise.
std::atomic<uint64_t> g_stubDispatchCount{0};
void GuestEngine::MiscStubDispatch(uc_engine* uc, uint64_t address, uint32_t size, void* /*userData*/) {
g_stubDispatchCount.fetch_add(1, std::memory_order_relaxed);
auto& eng = GuestEngine::Instance();
// Task #48 follow-up measurement (2026-09-19, temporary). The in-race
// profile puts 41.66% of CPU in helper_uc_tracecode but only 0.04% in
@@ -3380,16 +3739,16 @@ void GuestEngine::MiscStubDispatch(uc_engine* uc, uint64_t address, uint32_t siz
}
if (address < eng.misc_stub_arena_start_) return; // shouldn't happen - registered range starts here
size_t idx = (address - eng.misc_stub_arena_start_) / 4;
GuestEngine::MiscStubEntry entry;
{
std::lock_guard<std::mutex> lock(eng.misc_stub_dispatch_table_mutex_);
if (idx >= eng.misc_stub_dispatch_table_.size()) {
Log("GuestEngine::MiscStubDispatch: address 0x%llx -> index %zu out of range (table size %zu)",
(unsigned long long)address, idx, eng.misc_stub_dispatch_table_.size());
return;
}
entry = eng.misc_stub_dispatch_table_[idx];
// No lock here - see misc_stub_dispatch_table_'s own comment. Acquiring
// the published count pairs with the release store in AllocCodeStub, so
// any entry within `count` is fully written.
size_t count = eng.misc_stub_dispatch_count_.load(std::memory_order_acquire);
if (idx >= count) {
Log("GuestEngine::MiscStubDispatch: address 0x%llx -> index %zu out of range (table size %zu)",
(unsigned long long)address, idx, count);
return;
}
const GuestEngine::MiscStubEntry& entry = eng.misc_stub_dispatch_table_[idx];
entry.callback(uc, address, size, entry.userData);
}
@@ -3435,7 +3794,15 @@ GuestAddr GuestEngine::AllocCodeStub(uc_cb_hookcode_t callback, void* userData)
// since this is a pure sequential 4-byte bump allocator with no frees.
{
std::lock_guard<std::mutex> lock(misc_stub_dispatch_table_mutex_);
misc_stub_dispatch_table_.push_back({callback, userData});
size_t n = misc_stub_dispatch_count_.load(std::memory_order_relaxed);
if (n >= kMaxMiscStubs) {
Log("GuestEngine::AllocCodeStub: dispatch table full (%zu entries) - stub arena and "
"table are sized together, so this means kMiscStubArenaSize grew without "
"kMaxMiscStubs following it", n);
return 0;
}
misc_stub_dispatch_table_[n] = {callback, userData};
misc_stub_dispatch_count_.store(n + 1, std::memory_order_release);
}
// Callers store this as a guest-visible, indirectly-callable code
// pointer (GOT/PLT relocation targets, guest JNIEnv function-table
@@ -3550,7 +3917,7 @@ uint32_t GuestEngine::CallGuestFunction(GuestAddr target, const uint32_t* args,
if (plainTarget >= misc_stub_arena_start_ && plainTarget < misc_stub_end_) {
size_t idx = (plainTarget - misc_stub_arena_start_) / 4;
std::lock_guard<std::mutex> lk(misc_stub_dispatch_table_mutex_);
if (idx < misc_stub_dispatch_table_.size()) {
if (idx < misc_stub_dispatch_count_.load(std::memory_order_acquire)) {
const MiscStubEntry& e = misc_stub_dispatch_table_[idx];
if (e.callback == import_stub_dispatch_cb) {
stubKind = "import stub";
@@ -3962,6 +4329,59 @@ void GuestEngine::RegisterDataSymbol(const char* symbolName, GuestAddr address)
registered_data_symbols_[symbolName] = address;
}
namespace {
void watch_address_cb(uc_engine*, uint64_t address, uint32_t, void* user_data) {
auto* label = static_cast<const char*>(user_data);
// Once per address: these sit on error paths, and an error path that runs
// in a loop would otherwise bury everything else in the log.
static std::mutex seenMutex;
static std::set<uint64_t> seen;
{
std::lock_guard<std::mutex> lock(seenMutex);
if (!seen.insert(address).second) return;
}
Log("WATCH reached guest 0x%llx - %s", (unsigned long long)address, label ? label : "?");
}
} // namespace
void GuestEngine::WatchGuestAddress(GuestAddr addr, const char* label) {
if (!addr) return;
char* owned = strdup(label ? label : "");
{
std::lock_guard<std::mutex> lock(hook_registrations_mutex_);
hook_registrations_.push_back({watch_address_cb, owned, addr});
}
if (t_state_.uc) {
uc_hook h;
uc_err err = uc_hook_add(t_state_.uc, &h, UC_HOOK_CODE, (void*)watch_address_cb, owned,
addr, addr);
if (err != UC_ERR_OK) {
Log("GuestEngine::WatchGuestAddress(0x%x): uc_hook_add failed: %d", addr, (int)err);
}
}
}
bool GuestEngine::IsGuestImageCode(GuestAddr addr) const {
if (addr && addr < image_end_) return true;
for (const auto& r : secondary_image_ranges_) {
if (addr >= r.base && addr < r.base + r.span) return true;
}
return false;
}
GuestAddr GuestEngine::LookupSecondaryExport(const char* symbolName) const {
if (!symbolName) return 0;
auto it = secondary_image_exports_.find(symbolName);
return it == secondary_image_exports_.end() ? 0 : it->second;
}
const char* GuestEngine::NameForDataSymbol(GuestAddr address) const {
for (const auto& entry : registered_data_symbols_) {
if (entry.second == address) return entry.first.c_str();
}
return nullptr;
}
std::string GuestEngine::DescribeAddress(GuestAddr addr) const {
char buf[128];
if (addr >= region_size_) {
@@ -3990,6 +4410,20 @@ std::string GuestEngine::DescribeAddress(GuestAddr addr) const {
return buf;
}
GuestAddr GuestEngine::AllocGuestCode(uint32_t size) {
std::lock_guard<std::mutex> lock(control_mutex_);
uint32_t aligned = AlignUp(size, 8);
if (trampoline_cursor_ + aligned > trampoline_end_) {
Log("GuestEngine::AllocGuestCode: trampoline arena exhausted (requested %u bytes, %u "
"remaining)", size, trampoline_end_ - trampoline_cursor_);
return 0;
}
GuestAddr addr = trampoline_cursor_;
trampoline_cursor_ += aligned;
memset(G2H(addr), 0, aligned);
return addr;
}
GuestAddr GuestEngine::AllocPermanent(uint32_t size) {
std::lock_guard<std::mutex> lock(control_mutex_);
uint32_t aligned = AlignUp(size, 8);
mpcore/src/main/cpp/emu/guest_engine.h
+94 -3
View File
@@ -126,8 +126,24 @@ public:
// other resolved address in this engine) ready to hand straight to
// CallGuestFunction, or 0 on any failure (bad ELF, arena exhaustion,
// symbol not found - all logged).
// Loads an additional guest ELF alongside the primary image and records
// everything it EXPORTS, so a later import of one of those names resolves
// to the real code instead of a shim (see ResolveOrCreateImportStub).
//
// Pass entrySymbol = nullptr when the caller only wants the library loaded
// and does not need one specific address back - which is the case for the
// game's own FMOD libraries, whose entry points are reached through
// libapp.so's ordinary imports rather than by name from the host.
// Returns the entry symbol's address, or 1 for "loaded, no entry requested",
// or 0 on failure.
GuestAddr LoadSecondaryImage(const char* path, const char* entrySymbol);
// Loads the guest libraries that sit alongside the primary image in the
// same directory - currently the game's own FMOD. Called from LoadImage at
// the one point where it is both possible and still useful; see the call
// site's own comment for why the ordering is not negotiable.
void LoadSiblingLibraries(const char* primaryImagePath);
bool loaded() const { return host_region_ != nullptr; }
// ---- Address translation ----
@@ -289,6 +305,37 @@ public:
// ResolveOrCreateImportStub BEFORE falling back to a code stub.
void RegisterDataSymbol(const char* symbolName, GuestAddr address);
// Reverse of RegisterDataSymbol: which symbol, if any, was given this
// address. Exists so a shim that cannot honour a request can NAME what it
// was asked for instead of printing a bare arena address - the facet
// addresses handed out by AllocPermanent mean nothing on their own, and
// resolving one otherwise costs a rebuild and a reproduction (see
// Shim_use_facet). Linear scan: this is an error path only.
// Returns nullptr when the address is not a registered data symbol.
const char* NameForDataSymbol(GuestAddr address) const;
// Address of a symbol defined by a secondary guest image (see
// LoadSecondaryImage), or 0 if no loaded image exports that name. Lets
// host-side JNI entry points forward into real guest code - see
// game_lifecycle_stubs_extra2.cpp's FMOD audio bridge.
GuestAddr LookupSecondaryExport(const char* symbolName) const;
// Logs once when execution first reaches `addr`, without altering control
// flow - a pure observation point, unlike InstallTrampolineHookRaw which
// displaces instructions. Used to answer "which of these N places produced
// the error code" by watching every candidate at once instead of reasoning
// about which one is reachable.
//
// `label` is copied and used verbatim in the log line.
void WatchGuestAddress(GuestAddr addr, const char* label);
// True if `addr` lies inside ANY loaded guest image - the primary one or
// a sibling loaded by LoadSecondaryImage. Callers that need to tell "real
// guest code" from "one of this engine's own arenas" must ask this rather
// than comparing against image_end(), which only ever described the
// primary image and silently rejected every sibling.
bool IsGuestImageCode(GuestAddr addr) const;
// Registers a callback GuestEngine invokes exactly once per LoadImage
// call, right after MapSegments succeeds (so host_region_/
// AllocPermanent are usable) but strictly BEFORE ProcessRelocations
@@ -309,6 +356,14 @@ public:
// at a stub address, it only ever exists to be intercepted).
GuestAddr AllocCodeStub(uc_cb_hookcode_t callback, void* userData);
// Allocates `size` bytes of guest memory that may be EXECUTED, out of the
// trampoline arena. AllocPermanent cannot be used for code: it carves from
// the control arena, which CreateConfiguredEngine maps read-write only
// (task #54 - an executable mapping makes every write to it pay QEMU's
// notdirty_write path, and making the control arena executable again cost
// a measured ~4s of load time). Returns 0 if the arena is exhausted.
GuestAddr AllocGuestCode(uint32_t size);
// Resolves `symbolName` to a real, guest-callable address, creating a
// fresh import stub (via AllocCodeStub) the first time it's asked for
// and caching it thereafter - the same path a real ELF PLT import goes
@@ -346,6 +401,13 @@ public:
// before that thread can run any guest code at all.
void EnsureThreadEngine();
// Tears down everything EnsureThreadEngine (and any nested-call engine)
// set up for the CALLING host thread, returning its guest stacks to the
// arena for reuse. Must be called by any thread that created an engine
// and is about to exit - pthread_shim's thread body does. Safe to call on
// a thread that never had one.
void ReleaseThreadEngine();
// Reserved, never-fetched guest PC used as the "return to caller"
// target for every CallGuestFunction invocation (see .cpp for why a
// fixed constant is safe to reuse across nested calls).
@@ -507,7 +569,15 @@ private:
GuestAddr thread_stacks_cursor_ = 0, thread_stacks_end_ = 0; // CarveThreadStack arena
GuestAddr mmap_cursor_ = 0, mmap_end_ = 0; // AllocMmap arena - see its own comment
std::mutex thread_stacks_mutex_; // guards thread_stacks_cursor_ (concurrent pthread_create calls)
std::mutex thread_stacks_mutex_; // guards thread_stacks_cursor_ and the free list below
// Stacks handed back by ReleaseThreadEngine, ready to be reused.
//
// Without this the arena was a one-way bump allocator: every guest thread
// that finished kept its stack forever, and so did every nested-call
// engine, so a long session simply ran out. Confirmed live - the game
// created its race thread, CarveThreadStack returned 0, the thread's
// start routine never ran, and the race rendered as a black screen.
std::vector<GuestAddr> thread_stack_free_list_;
std::mutex control_mutex_; // guards control_cursor_
std::mutex mmap_mutex_; // guards mmap_cursor_
@@ -515,6 +585,14 @@ private:
std::unordered_map<std::string, ImportShimFn> registered_shims_;
std::unordered_map<std::string, GuestAddr> registered_data_symbols_; // see RegisterDataSymbol
// Symbols DEFINED by secondary guest images (see LoadSecondaryImage).
// Consulted by ResolveOrCreateImportStub ahead of the shim table, so that
// once the game's real ARM32 FMOD is loaded its own entry points win over
// the no-op stand-ins fmod_shims.cpp registers.
std::unordered_map<std::string, GuestAddr> secondary_image_exports_;
// [base, base+span) for each image LoadSecondaryImage has loaded.
struct LoadedImageRange { GuestAddr base; uint32_t span; };
std::vector<LoadedImageRange> secondary_image_ranges_;
std::vector<DataSymbolSetupFn> data_symbol_setup_fns_; // see RegisterDataSymbolSetup
std::unordered_map<std::string, GuestAddr> import_stub_by_symbol_; // dedupe: one stub per symbol name
std::unordered_map<GuestAddr, ImportShimFn> shim_by_stub_addr_;
@@ -551,8 +629,21 @@ private:
uc_cb_hookcode_t callback;
void* userData;
};
std::vector<MiscStubEntry> misc_stub_dispatch_table_;
std::mutex misc_stub_dispatch_table_mutex_;
// Lock-free on the read side (2026-09-19, task #56). This table is hit on
// EVERY shim crossing - measured at ~170,000/sec during gameplay on the
// Xiaomi 14, from several threads - and taking a std::mutex there taxed
// the exact thing being measured. A fixed array plus an atomic published
// count removes it safely: the arena is only kMiscStubArenaSize/4 stubs
// wide by construction (AllocCodeStub bump-allocates 4 bytes each and
// refuses past the end), entries are appended and never removed or
// rewritten, and a writer fills its slot BEFORE publishing the new count
// with release ordering. A reader that acquires the count therefore sees
// a fully-written entry. The mutex stays for the append side, which runs
// a few hundred times at startup.
static constexpr size_t kMaxMiscStubs = 16 * 1024 / 4;
MiscStubEntry misc_stub_dispatch_table_[kMaxMiscStubs] = {};
std::atomic<size_t> misc_stub_dispatch_count_{0};
std::mutex misc_stub_dispatch_table_mutex_; // append side only
// See crashed()'s own comment. Global (not per-thread) on purpose - a
// fault on ANY thread means the ONE shared host_region_ every thread's
mpcore/src/main/cpp/emu/jni_shim.cpp
+25 -5
View File
@@ -317,13 +317,12 @@ std::string GuestCStr(GuestEngine& eng, uint32_t guestPtr) {
} // namespace
std::atomic<uint32_t> JniHandleTable::g_callEpoch{0};
thread_local uint32_t JniHandleTable::t_callEpoch = 0;
uint32_t JniHandleTable::Alloc(void* real, bool isGlobal) {
if (!real) return 0;
std::lock_guard<std::mutex> lock(mutex_);
table_.push_back(Entry{real, std::this_thread::get_id(), isGlobal,
g_callEpoch.load(std::memory_order_relaxed)});
table_.push_back(Entry{real, std::this_thread::get_id(), isGlobal, t_callEpoch});
return (uint32_t)(table_.size() - 1);
}
@@ -344,8 +343,10 @@ bool JniHandleTable::IsSafeToUseFromCurrentThread(uint32_t handle) const {
// nativeOnRunLoopTick calls SetRealEnv/BumpCallEpoch on every single
// entry, so a handle from an earlier tick on the SAME thread is just
// as stale as one from a different thread).
return e.owner == std::this_thread::get_id() &&
e.epoch == g_callEpoch.load(std::memory_order_relaxed);
// Both halves are per-thread now: the ref must have been created by THIS
// thread, during the native call this thread is currently inside. Another
// thread entering or leaving a native call cannot affect either test.
return e.owner == std::this_thread::get_id() && e.epoch == t_callEpoch;
}
thread_local JNIEnv* JniShim::real_env_ = nullptr;
@@ -1734,6 +1735,11 @@ uint32_t Impl_NewDirectByteBuffer(GuestEngine& eng, uint32_t, uint32_t r1, uint3
g_directBufferGuestAddrs[h] = r1;
return h;
}
void RegisterDirectBufferGuestAddr(uint32_t handle, GuestAddr guestAddr);
void RegisterDirectBufferGuestAddr(uint32_t handle, GuestAddr guestAddr) {
g_directBufferGuestAddrs[handle] = guestAddr;
}
uint32_t Impl_GetDirectBufferAddress(GuestEngine&, uint32_t, uint32_t r1, uint32_t, uint32_t, uint32_t) {
auto it = g_directBufferGuestAddrs.find(r1);
if (it == g_directBufferGuestAddrs.end()) {
@@ -2143,6 +2149,20 @@ uint32_t Impl_RegisterNatives(GuestEngine& eng, uint32_t, uint32_t r1, uint32_t
} // namespace
uint32_t JniShim::NewGuestBackedDirectBuffer(GuestEngine& engine, GuestAddr guestAddr,
uint32_t capacity) {
if (!guestAddr || !capacity || !RealEnv()) return 0;
jobject buf = RealEnv()->NewDirectByteBuffer(engine.G2H(guestAddr), (jlong)capacity);
if (!buf) {
Log("jni_shim: NewGuestBackedDirectBuffer(0x%x, %u) - NewDirectByteBuffer failed",
guestAddr, capacity);
return 0;
}
uint32_t handle = handles().Alloc(buf);
RegisterDirectBufferGuestAddr(handle, guestAddr);
return handle;
}
GuestAddr JniShim::BuildGuestJNIEnv(GuestEngine& engine) {
if (guest_env_) return guest_env_;
mpcore/src/main/cpp/emu/jni_shim.h
+29 -3
View File
@@ -81,15 +81,30 @@ public:
// IsSafeToUseFromCurrentThread now checks BOTH the owning thread AND
// the owning epoch.
bool IsSafeToUseFromCurrentThread(uint32_t handle) const;
static void BumpCallEpoch() { g_callEpoch.fetch_add(1, std::memory_order_relaxed); }
//
// Corrected 2026-09-21 (task #67): the epoch used to be ONE process-wide
// counter. That was indistinguishable from correct while only one host
// thread ever crossed into JNI - but a local ref's lifetime is scoped to
// a native call ON ITS OWN THREAD, and a bump from a different thread has
// no business invalidating it.
//
// It stopped being academic the moment the FMOD audio bridge began
// calling in from FMODAudioDevice's AudioTrack thread every 100 ms: each
// of those bumps invalidated the refs GLThread was holding mid-call, and
// the process aborted with "JNI DETECTED ERROR IN APPLICATION: jfieldID
// was NULL" inside GLThread. Per-thread is both the fix and the more
// accurate model - no atomics needed either, since a thread's own epoch
// is only ever read and written by that thread.
static void BumpCallEpoch() { ++t_callEpoch; }
static uint32_t CurrentCallEpoch() { return t_callEpoch; }
private:
static std::atomic<uint32_t> g_callEpoch;
static thread_local uint32_t t_callEpoch;
struct Entry {
void* real = nullptr;
std::thread::id owner;
bool isGlobal = false;
uint32_t epoch = 0; // g_callEpoch's value at Alloc() time - see IsSafeToUseFromCurrentThread
uint32_t epoch = 0; // the OWNING THREAD's epoch at Alloc() time - see IsSafeToUseFromCurrentThread
};
mutable std::mutex mutex_;
std::vector<Entry> table_{Entry{}}; // index 0 reserved for guest NULL
@@ -104,6 +119,17 @@ public:
// function that expects a JNIEnv*.
GuestAddr BuildGuestJNIEnv(GuestEngine& engine);
// Wraps a range of GUEST memory in a real Java direct ByteBuffer and
// returns the guest handle for it, registered so that the guest's own
// GetDirectBufferAddress resolves it back to `guestAddr`.
//
// Needed because GetDirectBufferAddress can only answer for buffers this
// shim created (see its own comment): a ByteBuffer that Java allocated
// lives at a host address the guest cannot reach. Anything handing guest
// code a Java-allocated buffer therefore has to bounce through one of
// these - the same shape as the AndroidBitmap_lockPixels fix.
uint32_t NewGuestBackedDirectBuffer(GuestEngine& engine, GuestAddr guestAddr, uint32_t capacity);
// Builds a minimal guest JavaVM (8-slot JNIInvokeInterface) - only
// GetEnv and AttachCurrentThread are real (both just return the same
// guest JNIEnv from BuildGuestJNIEnv - this shim only ever has one
mpcore/src/main/cpp/emu/libc_shims.cpp
+269
View File
@@ -17,6 +17,7 @@
#include <functional>
#include <algorithm>
#include <unistd.h>
#include <sys/syscall.h>
#include <fcntl.h>
#include <dirent.h>
#include <dlfcn.h>
@@ -667,8 +668,63 @@ uint32_t Shim_vsnprintf(GuestEngine& eng, uint32_t dst, uint32_t bufSize, uint32
}
// ==================== unistd.h / fcntl.h (POSIX file I/O) ====================
// /proc/cpuinfo, rewritten for a 32-bit reader (task #67).
//
// This is the root cause of "no audio", traced all the way down. FMOD detects
// CPU capabilities by reading /proc/cpuinfo and string-matching the Features
// line for "vfp", "vfpv3" and "neon" (libfmodex sub_BFFD8). If it finds none
// of them it leaves its capability mask at zero, and sub_A9AF8 then returns
// FMOD_RESULT 48, which propagates out through System::init and
// EventSystem::init - measured live as exactly that chain.
//
// On an arm64 kernel the same silicon is described with the AArch64 names:
//
// Pixel 6a: Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 ...
//
// "fp" IS VFP and "asimd" IS NEON - the hardware has both, the 64-bit format
// simply spells them differently, and a 32-bit library from 2012 cannot know
// that. So this is not a lie told to the guest: it is the same CPU, described
// in the vocabulary the guest was built to read. Everything reported here is
// genuinely present on any ARMv8 core running this engine.
//
// Kept minimal on purpose - only the fields FMOD's parser actually looks at
// ("CPU architecture", "Processor", "Features"), plus enough shape to look
// like a real file.
static const char kGuestCpuInfo[] =
"Processor\t: ARMv7 Processor rev 1 (v7l)\n"
"processor\t: 0\n"
"BogoMIPS\t: 38.40\n"
"Features\t: swp half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt\n"
"CPU implementer\t: 0x41\n"
"CPU architecture: 7\n"
"CPU variant\t: 0x0\n"
"CPU part\t: 0xd03\n"
"CPU revision\t: 1\n"
"\n"
"Hardware\t: ARMv7 Processor\n"
"Revision\t: 0000\n"
"Serial\t\t: 0000000000000000\n";
uint32_t Shim_open(GuestEngine& eng, uint32_t path, uint32_t flags, uint32_t mode, uint32_t, uint32_t) {
const char* p = (const char*)eng.G2H(path);
if (p && strcmp(p, "/proc/cpuinfo") == 0) {
// memfd_create keeps this a perfectly ordinary fd, so read/lseek/close
// need no special cases and the guest cannot tell the difference.
int memfd = syscall(__NR_memfd_create, "guest_cpuinfo", 0);
if (memfd >= 0) {
size_t len = sizeof(kGuestCpuInfo) - 1;
if (write(memfd, kGuestCpuInfo, len) == (ssize_t)len && lseek(memfd, 0, SEEK_SET) == 0) {
Log("libc_shims: open(\"/proc/cpuinfo\") -> %d, serving an ARMv7-format copy "
"(the host's arm64 kernel calls VFP \"fp\" and NEON \"asimd\", which no 32-bit "
"library recognises - see kGuestCpuInfo)", memfd);
return (uint32_t)memfd;
}
close(memfd);
}
Log("libc_shims: could not build the ARMv7 /proc/cpuinfo replacement (%s) - falling through "
"to the host's own, which will make FMOD report no VFP/NEON and fail to initialise",
strerror(errno));
}
int fd = open(p, (int)flags, (mode_t)mode);
Log("libc_shims: open(\"%s\", flags=0x%x, mode=0%o) -> %d%s", p, flags, mode, fd,
fd < 0 ? " [FAILED]" : "");
@@ -1692,7 +1748,220 @@ uint32_t Shim_glGetBufferPointervOES(GuestEngine& eng, uint32_t target, uint32_t
} // namespace
// ---- Runtime support the game's own ARM32 FMOD needs (task #67) ----
//
// libfmodex.so is a real, shipped ARM32 shared library, and the plan is to run
// it under this engine rather than reimplement 63 FMOD entry points. Comparing
// its 117 undefined symbols against everything already registered left exactly
// 37 gaps, and every one of them is compiler-runtime or libm - no reverse
// engineering, just forwarding. They live here rather than in fmod_shims.cpp
// because none of them is FMOD-specific; libapp.so may well want them too.
//
// The __aeabi_* helpers are the ARM EABI's own arithmetic routines. They use a
// softfp register convention (a double is an r0:r1 pair), which is exactly what
// ReadDoubleArg/ReturnDouble above already handle.
// Integer division. ARM has no integer divide instruction in this profile, so
// the compiler emits calls to these instead.
uint32_t Shim_aeabi_idiv(GuestEngine&, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
int32_t n = (int32_t)a, d = (int32_t)b;
return d ? (uint32_t)(n / d) : 0; // divide-by-zero: EABI leaves it undefined, 0 is as good as anything
}
uint32_t Shim_aeabi_uidiv(GuestEngine&, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
return b ? (a / b) : 0;
}
// ...mod variants return quotient in r0 AND remainder in r1 - the same r0:r1
// pair convention ReturnU64 writes.
uint32_t Shim_aeabi_idivmod(GuestEngine& eng, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
int32_t n = (int32_t)a, d = (int32_t)b;
int32_t q = d ? n / d : 0, r = d ? n % d : 0;
return ReturnU64(eng, ((uint64_t)(uint32_t)r << 32) | (uint32_t)q);
}
uint32_t Shim_aeabi_uidivmod(GuestEngine& eng, uint32_t a, uint32_t b, uint32_t, uint32_t, uint32_t) {
uint32_t q = b ? a / b : 0, r = b ? a % b : 0;
return ReturnU64(eng, ((uint64_t)r << 32) | q);
}
// 64-bit unsigned divide: quotient in r0:r1, remainder in r2:r3. The remainder
// half needs registers the shared dispatch contract does not cover, so it is
// written directly, the same way ReturnU64 writes r1.
uint32_t Shim_aeabi_uldivmod(GuestEngine& eng, uint32_t nlo, uint32_t nhi, uint32_t dlo, uint32_t dhi, uint32_t) {
uint64_t n = ((uint64_t)nhi << 32) | nlo;
uint64_t d = ((uint64_t)dhi << 32) | dlo;
uint64_t q = d ? n / d : 0, r = d ? n % d : 0;
if (uc_engine* uc = eng.uc()) {
uint32_t rlo = (uint32_t)r, rhi = (uint32_t)(r >> 32);
uc_reg_write(uc, UC_ARM_REG_R2, &rlo);
uc_reg_write(uc, UC_ARM_REG_R3, &rhi);
}
return ReturnU64(eng, q);
}
// Double arithmetic and conversions.
uint32_t Shim_aeabi_dadd(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) + ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp));
}
uint32_t Shim_aeabi_dmul(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) * ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp));
}
uint32_t Shim_aeabi_dcmplt(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp) < ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp) ? 1u : 0u;
}
uint32_t Shim_aeabi_f2d(GuestEngine& eng, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
return ReturnDouble(eng, (double)v);
}
uint32_t Shim_aeabi_ui2d(GuestEngine& eng, uint32_t v, uint32_t, uint32_t, uint32_t, uint32_t) {
return ReturnDouble(eng, (double)v);
}
uint32_t Shim_aeabi_ul2d(GuestEngine& eng, uint32_t lo, uint32_t hi, uint32_t, uint32_t, uint32_t) {
return ReturnDouble(eng, (double)(((uint64_t)hi << 32) | lo));
}
// Static-destructor registration. Nothing in this engine ever tears the guest
// image down, so recording the destructor would be write-only state.
uint32_t Shim_aeabi_atexit(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) { return 0; }
// The C++ unwinding personality routine. Reaching it means a real exception is
// unwinding through guest frames, which this engine cannot do (see
// __cxa_bad_typeid's own comment) - so say so loudly rather than return a code
// the unwinder would act on.
uint32_t Shim_aeabi_unwind_cpp_pr0(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
static bool logged = false;
if (!logged) {
logged = true;
Log("libc_shims: __aeabi_unwind_cpp_pr0 called - a C++ exception is unwinding through guest "
"frames and this engine has no guest stack unwinder. Returning 'unwind failed'; expect "
"the guest to abort rather than to catch.");
}
return 9; // _URC_FAILURE
}
// libm. Plain forwarding - the host has the same IEEE doubles the guest wants.
#define FMOD_MATH_D1(name, expr) \
uint32_t Shim_##name(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, \
uint32_t sp) { \
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp); \
return ReturnDouble(eng, (expr)); \
}
FMOD_MATH_D1(acos, acos(x))
FMOD_MATH_D1(cos, cos(x))
FMOD_MATH_D1(sin, sin(x))
FMOD_MATH_D1(tan, tan(x))
FMOD_MATH_D1(exp, exp(x))
FMOD_MATH_D1(log, log(x))
FMOD_MATH_D1(log10, log10(x))
FMOD_MATH_D1(sqrt, sqrt(x))
FMOD_MATH_D1(rint, rint(x))
#undef FMOD_MATH_D1
uint32_t Shim_atan2(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
return ReturnDouble(eng, atan2(ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp),
ReadDoubleArg(eng, 2, r0, r1, r2, r3, sp)));
}
uint32_t Shim_log10f(GuestEngine&, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
float r = log10f(v);
uint32_t out; memcpy(&out, &r, 4);
return out;
}
uint32_t Shim_lrintf(GuestEngine&, uint32_t f, uint32_t, uint32_t, uint32_t, uint32_t) {
float v; memcpy(&v, &f, 4);
return (uint32_t)(int32_t)lrintf(v);
}
// frexp/ldexp take or return a pointer/int alongside the double.
uint32_t Shim_frexp(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp);
uint32_t expPtr = eng.ReadIncomingArg(2, r0, r1, r2, r3, sp);
int e = 0;
double m = frexp(x, &e);
if (expPtr) { int32_t v = e; memcpy(eng.G2H(expPtr), &v, 4); }
return ReturnDouble(eng, m);
}
uint32_t Shim_ldexp(GuestEngine& eng, uint32_t r0, uint32_t r1, uint32_t r2, uint32_t r3, uint32_t sp) {
double x = ReadDoubleArg(eng, 0, r0, r1, r2, r3, sp);
int32_t e = (int32_t)eng.ReadIncomingArg(2, r0, r1, r2, r3, sp);
return ReturnDouble(eng, ldexp(x, e));
}
// Remaining libc gaps.
uint32_t Shim_usleep(GuestEngine&, uint32_t us, uint32_t, uint32_t, uint32_t, uint32_t) {
return (uint32_t)usleep(us);
}
uint32_t Shim_memmem(GuestEngine& eng, uint32_t hay, uint32_t hayLen, uint32_t needle, uint32_t needleLen, uint32_t) {
if (!hay || !needle) return 0;
void* found = memmem(eng.G2H(hay), hayLen, eng.G2H(needle), needleLen);
if (!found) return 0;
return hay + (uint32_t)((uint8_t*)found - (uint8_t*)eng.G2H(hay));
}
uint32_t Shim_inet_addr(GuestEngine& eng, uint32_t s, uint32_t, uint32_t, uint32_t, uint32_t) {
return s ? (uint32_t)inet_addr((const char*)eng.G2H(s)) : 0xFFFFFFFFu;
}
uint32_t Shim_chown(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
// An app sandbox cannot chown anything anyway; real Android would fail this
// too, so reporting success costs nothing and failing could stall a caller.
return 0;
}
uint32_t Shim_pthread_attr_setdetachstate(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
// pthread_shim always creates detached-equivalent std::threads it owns, so
// the guest's preference here is already satisfied.
return 0;
}
// FMOD only reaches select() on its network-streaming path, which local
// playback never uses. Reporting "nothing ready" is the honest answer for a
// backend we do not provide - and it is logged, so if audio ever does depend on
// it that shows up as a named gap instead of silence.
uint32_t Shim_select(GuestEngine&, uint32_t, uint32_t, uint32_t, uint32_t, uint32_t) {
static bool logged = false;
if (!logged) {
logged = true;
Log("libc_shims: select() is not implemented - reporting 'no descriptors ready'. Only "
"FMOD's network-streaming path calls this; if audio depends on it, implement it here.");
}
return 0;
}
uint32_t Shim_operator_delete(GuestEngine& eng, uint32_t p, uint32_t, uint32_t, uint32_t, uint32_t) {
if (p) eng.heap().Free(p);
return 0;
}
void RegisterLibcImportShims(GuestEngine& engine) {
// Task #67 - runtime support the game's own ARM32 FMOD needs. See the
// block of implementations above for why these 37 live here.
engine.RegisterImportShim("__aeabi_idiv", Shim_aeabi_idiv);
engine.RegisterImportShim("__aeabi_uidiv", Shim_aeabi_uidiv);
engine.RegisterImportShim("__aeabi_idivmod", Shim_aeabi_idivmod);
engine.RegisterImportShim("__aeabi_uidivmod", Shim_aeabi_uidivmod);
engine.RegisterImportShim("__aeabi_uldivmod", Shim_aeabi_uldivmod);
engine.RegisterImportShim("__aeabi_dadd", Shim_aeabi_dadd);
engine.RegisterImportShim("__aeabi_dmul", Shim_aeabi_dmul);
engine.RegisterImportShim("__aeabi_dcmplt", Shim_aeabi_dcmplt);
engine.RegisterImportShim("__aeabi_f2d", Shim_aeabi_f2d);
engine.RegisterImportShim("__aeabi_ui2d", Shim_aeabi_ui2d);
engine.RegisterImportShim("__aeabi_ul2d", Shim_aeabi_ul2d);
engine.RegisterImportShim("__aeabi_atexit", Shim_aeabi_atexit);
engine.RegisterImportShim("__aeabi_unwind_cpp_pr0", Shim_aeabi_unwind_cpp_pr0);
engine.RegisterImportShim("acos", Shim_acos);
engine.RegisterImportShim("cos", Shim_cos);
engine.RegisterImportShim("sin", Shim_sin);
engine.RegisterImportShim("tan", Shim_tan);
engine.RegisterImportShim("exp", Shim_exp);
engine.RegisterImportShim("log", Shim_log);
engine.RegisterImportShim("log10", Shim_log10);
engine.RegisterImportShim("sqrt", Shim_sqrt);
engine.RegisterImportShim("rint", Shim_rint);
engine.RegisterImportShim("atan2", Shim_atan2);
engine.RegisterImportShim("log10f", Shim_log10f);
engine.RegisterImportShim("lrintf", Shim_lrintf);
engine.RegisterImportShim("frexp", Shim_frexp);
engine.RegisterImportShim("ldexp", Shim_ldexp);
engine.RegisterImportShim("usleep", Shim_usleep);
engine.RegisterImportShim("memmem", Shim_memmem);
engine.RegisterImportShim("inet_addr", Shim_inet_addr);
engine.RegisterImportShim("chown", Shim_chown);
engine.RegisterImportShim("pthread_attr_setdetachstate", Shim_pthread_attr_setdetachstate);
engine.RegisterImportShim("select", Shim_select);
engine.RegisterImportShim("_ZdlPv", Shim_operator_delete);
// ---- ctype.h ----
engine.RegisterImportShim("isalnum", Shim_isalnum);
engine.RegisterImportShim("isalpha", Shim_isalpha);
mpcore/src/main/cpp/emu/pthread_shim.cpp
+22 -3
View File
@@ -57,9 +57,22 @@ uint32_t Shim_pthread_create(GuestEngine& eng, uint32_t threadOutPtr, uint32_t /
// itself (see ARM64_TRANSLATION_LAYER.md, "register/stack corruption"
// investigation). Reject loudly and immediately instead - one clear
// diagnostic beats six confusing downstream ones.
if (startRoutine >= eng.image_end()) {
Log("pthread_shim: REFUSING pthread_create - startRoutine=0x%x is not real image code (%s)",
startRoutine, eng.DescribeAddress(startRoutine).c_str());
//
// Corrected 2026-09-21 (task #67): this used to compare against
// image_end(), which was right only while exactly ONE image existed. Once
// the game's own libfmodex/libfmodevent load as sibling images - far above
// image_end() - that test rejected FMOD's OWN mixer thread as "not real
// image code". The refusal propagated all the way up as silence:
//
// pthread_create -> EINVAL -> sub_A9120 -> 33 -> System::init -> 33
// -> EventSystem::init -> 33 -> fmodGetInfo -> -1
// -> FMODAudioDevice never builds its AudioTrack
//
// IsGuestImageCode knows about every loaded image, so the guard keeps its
// original purpose without the single-image assumption.
if (!eng.IsGuestImageCode(startRoutine)) {
Log("pthread_shim: REFUSING pthread_create - startRoutine=0x%x is not inside any loaded "
"guest image (%s)", startRoutine, eng.DescribeAddress(startRoutine).c_str());
return 22; // EINVAL - matches pthread_create's own errno-style failure contract
}
@@ -79,6 +92,12 @@ uint32_t Shim_pthread_create(GuestEngine& eng, uint32_t threadOutPtr, uint32_t /
eng.EnsureThreadEngine();
uint32_t result = eng.CallGuestFunction(startRoutine, arg);
Log("pthread_shim: guest thread (handle %u) start_routine returned 0x%x", handle, result);
// Hand this thread's guest stack(s) and uc_engine back. Without this
// the thread-stack arena was one-way: the game creates threads across
// a session (one per race, among others) and after kMaxGuestThreads of
// them CarveThreadStack started returning 0, the next thread's start
// routine never ran, and the race came up as a black screen.
eng.ReleaseThreadEngine();
});
{
mpcore/src/main/cpp/emu/rtti_shims.cpp
+273 -14
View File
@@ -1,3 +1,6 @@
#include <set>
#include <chrono>
#include <atomic>
#include "rtti_shims.h"
#include "../util/util.h"
#include <cstring>
@@ -64,9 +67,12 @@ struct GuestVmiClassTypeInfoHeader {
// __vmi_class_type_info (multiple bases, recurse each, fail on ambiguity).
// Virtual-base offsets (need the vbase table, not just a static offset)
// are a documented gap - skipped, not silently wrong.
thread_local int g_searchDepthReached = 0;
bool SearchBase(GuestEngine& eng, GuestAddr typeInfoAddr, GuestAddr dstTypeInfoAddr,
int32_t curOffset, int32_t* outOffset, int depth) {
if (!typeInfoAddr || depth > 32) return false;
if (depth > g_searchDepthReached) g_searchDepthReached = depth;
if (typeInfoAddr == dstTypeInfoAddr) {
*outOffset = curOffset;
return true;
@@ -111,9 +117,44 @@ bool SearchBase(GuestEngine& eng, GuestAddr typeInfoAddr, GuestAddr dstTypeInfoA
return false;
}
// Task #59 shape survey (2026-09-19, temporary). Native calls __dynamic_cast
// up to 824,000 times/sec and this engine can only serve ~285,000 - the
// boundary, not the algorithm, is the bottleneck (task #58). A guest-side
// ARM32 implementation would avoid the crossing entirely, but only pays off
// if it can be SMALL. So: which cases actually occur? If the overwhelming
// majority are "the object already is the target type" or one single-
// inheritance step, a short guest fast path with a fallback to this shim
// captures nearly everything. Counted, not assumed.
namespace {
constexpr bool kSurveyDynamicCast = false;
std::atomic<uint64_t> g_dcTotal{0}, g_dcNullArg{0}, g_dcExact{0}, g_dcDepth1{0},
g_dcDeeper{0}, g_dcMiss{0}, g_dcMaxDepth{0};
void SurveyReport() {
static std::atomic<uint64_t> lastNs{0};
uint64_t now = (uint64_t)std::chrono::duration_cast<std::chrono::nanoseconds>(
std::chrono::steady_clock::now().time_since_epoch()).count();
uint64_t last = lastNs.load(std::memory_order_relaxed);
if (now - last < 2000000000ull) return;
if (!lastNs.compare_exchange_strong(last, now, std::memory_order_relaxed) || last == 0) return;
uint64_t t = g_dcTotal.load(std::memory_order_relaxed);
if (!t) return;
auto pct = [t](std::atomic<uint64_t>& c) {
return 100.0 * (double)c.load(std::memory_order_relaxed) / (double)t;
};
Log("rtti_shims: DCSHAPE total=%llu | exact(depth0)=%.1f%% oneBase(depth1)=%.1f%% "
"deeper=%.1f%% notFound=%.1f%% nullArg=%.1f%% | maxDepth=%llu",
(unsigned long long)t, pct(g_dcExact), pct(g_dcDepth1), pct(g_dcDeeper), pct(g_dcMiss),
pct(g_dcNullArg), (unsigned long long)g_dcMaxDepth.load(std::memory_order_relaxed));
}
} // namespace
uint32_t Shim_dynamic_cast(GuestEngine& eng, uint32_t srcPtr, uint32_t /*srcTypeInfo*/,
uint32_t dstTypeInfo, uint32_t /*src2dstHint*/, uint32_t) {
if (!srcPtr || !dstTypeInfo) return 0;
if (kSurveyDynamicCast) { g_dcTotal.fetch_add(1, std::memory_order_relaxed); SurveyReport(); }
if (!srcPtr || !dstTypeInfo) {
if (kSurveyDynamicCast) g_dcNullArg.fetch_add(1, std::memory_order_relaxed);
return 0;
}
uint32_t objVtablePtr = 0;
memcpy(&objVtablePtr, eng.G2H(srcPtr), 4);
if (!objVtablePtr) return 0;
@@ -130,7 +171,20 @@ uint32_t Shim_dynamic_cast(GuestEngine& eng, uint32_t srcPtr, uint32_t /*srcType
GuestAddr mostDerivedPtr = (GuestAddr)((int32_t)srcPtr + (int32_t)offsetToTopRaw);
int32_t foundOffset = 0;
if (!SearchBase(eng, dynTypeInfo, dstTypeInfo, 0, &foundOffset, 0)) return 0;
if (kSurveyDynamicCast && dynTypeInfo == dstTypeInfo) {
g_dcExact.fetch_add(1, std::memory_order_relaxed);
return (uint32_t)mostDerivedPtr;
}
g_searchDepthReached = 0;
if (!SearchBase(eng, dynTypeInfo, dstTypeInfo, 0, &foundOffset, 0)) {
if (kSurveyDynamicCast) g_dcMiss.fetch_add(1, std::memory_order_relaxed);
return 0;
}
if (kSurveyDynamicCast) {
(g_searchDepthReached <= 1 ? g_dcDepth1 : g_dcDeeper).fetch_add(1, std::memory_order_relaxed);
uint64_t d = (uint64_t)g_searchDepthReached, prev = g_dcMaxDepth.load(std::memory_order_relaxed);
while (d > prev && !g_dcMaxDepth.compare_exchange_weak(prev, d, std::memory_order_relaxed)) {}
}
return (uint32_t)((int32_t)mostDerivedPtr + foundOffset);
}
@@ -452,6 +506,7 @@ uint32_t Shim_ctype_char_do_widen(GuestEngine&, uint32_t /*this*/, uint32_t c, u
GuestAddr g_ctypeCharId = 0, g_ctypeCharFacet = 0;
GuestAddr g_numPutCharId = 0, g_numPutCharFacet = 0;
GuestAddr g_numGetCharId = 0, g_numGetCharFacet = 0;
// ---- num_put<char> facet - same "real vtable, only confirmed slots
// implemented" approach as ctype<char> above. Confirmed via IDA
@@ -626,6 +681,148 @@ uint32_t Shim_num_put_double(GuestEngine& eng, uint32_t, uint32_t streambuf, uin
return streambuf;
}
// ---- num_get<char> facet - reading numbers back OUT of a stream ----
//
// Confirmed live, not anticipated: the post-prologue dialog crashed because
// use_facet<num_get<char>> returned NULL and the guest called through it
// (guest 0x3e5248, decompiled as basic_istream<char>::operator>>). The NULL
// facet's "vtable" read produced the wild pointer 0x464c459b - the same
// "\x7fELF"-as-a-pointer crash shape FacetSlotCtx's comment warns about.
//
// libc++ basic_streambuf get-area pointers. The layout anchor is
// WriteCharToStreambuf's already-working put area (pptr@24, epptr@28): the six
// pointers run eback@8, gptr@12, egptr@16, pbase@20, pptr@24, epptr@28 after
// the vtable and the embedded locale, so the put offsets pin the get offsets.
constexpr uint32_t kSbGptr = 12, kSbEgptr = 16;
// ios_base::iostate bits, taken from THIS NDK's own <ios> rather than from
// memory (they differ between standard libraries - libstdc++ orders them the
// other way round).
constexpr uint32_t kIosBadbit = 0x1, kIosEofbit = 0x2, kIosFailbit = 0x4;
int PeekStreambuf(GuestEngine& eng, GuestAddr sb) {
if (!sb) return -1;
uint32_t g = 0, e = 0;
memcpy(&g, eng.G2H(sb + kSbGptr), 4);
memcpy(&e, eng.G2H(sb + kSbEgptr), 4);
// An exhausted get area would need a virtual underflow() call to refill.
// This shim does not make one - for the istringstream case that actually
// occurs here the whole string is already in the get area. Reported as
// end-of-input rather than silently treated as a parse failure.
if (!g || g >= e) return -1;
uint8_t c = 0;
memcpy(&c, eng.G2H(g), 1);
return c;
}
void BumpStreambuf(GuestEngine& eng, GuestAddr sb) {
uint32_t g = 0;
memcpy(&g, eng.G2H(sb + kSbGptr), 4);
g += 1;
memcpy(eng.G2H(sb + kSbGptr), &g, 4);
}
enum class NumGetKind { Bool, Signed, Unsigned, Float };
// Shared body for every do_get overload. Lifts the next token out of the get
// area and hands it to the host's own strtoll/strtoull/strtod - the same
// "offload the hard part to real libc instead of reimplementing it" choice
// num_put makes for formatting. Like num_put, it deliberately ignores
// ios_base's formatting flags (base/boolalpha), which is a documented scope
// cut, not an oversight.
uint32_t NumGetCommon(GuestEngine& eng, uint32_t sb, uint32_t errPtr, uint32_t valPtr,
NumGetKind kind, int width) {
uint32_t err = 0;
auto storeErr = [&]() { if (errPtr) memcpy(eng.G2H(errPtr), &err, 4); };
if (!sb || !valPtr) {
err = kIosFailbit | kIosBadbit;
storeErr();
return sb;
}
int c = PeekStreambuf(eng, sb);
while (c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '\f' || c == '\v') {
BumpStreambuf(eng, sb);
c = PeekStreambuf(eng, sb);
}
std::string tok;
if (c == '+' || c == '-') { tok.push_back((char)c); BumpStreambuf(eng, sb); c = PeekStreambuf(eng, sb); }
bool sawDigit = false;
while (c >= 0) {
char ch = (char)c;
bool part = (ch >= '0' && ch <= '9');
if (!part && kind == NumGetKind::Float &&
(ch == '.' || ch == 'e' || ch == 'E' || ch == '+' || ch == '-')) part = true;
if (!part) break;
if (ch >= '0' && ch <= '9') sawDigit = true;
tok.push_back(ch);
BumpStreambuf(eng, sb);
c = PeekStreambuf(eng, sb);
}
if (!sawDigit) {
// No number here at all. Real num_get sets failbit and leaves the
// destination untouched; matching that is what lets the caller's
// stream go into a failed state instead of reading a fabricated value.
err = kIosFailbit;
if (c < 0) err |= kIosEofbit;
storeErr();
Log("rtti_shims: num_get found no number to parse (streambuf=0x%x) - setting failbit, "
"leaving the destination untouched", sb);
return sb;
}
if (c < 0) err |= kIosEofbit; // consumed right up to the end of input
uint8_t b8; uint16_t b16; uint32_t b32; uint64_t b64;
switch (kind) {
case NumGetKind::Bool:
b8 = (uint8_t)(strtoll(tok.c_str(), nullptr, 10) != 0);
memcpy(eng.G2H(valPtr), &b8, 1);
break;
case NumGetKind::Signed: {
long long v = strtoll(tok.c_str(), nullptr, 10);
if (width == 8) { b64 = (uint64_t)v; memcpy(eng.G2H(valPtr), &b64, 8); }
else { b32 = (uint32_t)(int32_t)v; memcpy(eng.G2H(valPtr), &b32, 4); }
break;
}
case NumGetKind::Unsigned: {
unsigned long long v = strtoull(tok.c_str(), nullptr, 10);
if (width == 8) { b64 = (uint64_t)v; memcpy(eng.G2H(valPtr), &b64, 8); }
else if (width == 2) { b16 = (uint16_t)v; memcpy(eng.G2H(valPtr), &b16, 2); }
else { b32 = (uint32_t)v; memcpy(eng.G2H(valPtr), &b32, 4); }
break;
}
case NumGetKind::Float: {
double v = strtod(tok.c_str(), nullptr);
if (width == 4) { float f = (float)v; memcpy(eng.G2H(valPtr), &f, 4); }
else { memcpy(eng.G2H(valPtr), &v, 8); }
break;
}
}
storeErr();
return sb;
}
// r0=this(facet, unused), r1=begin iterator (an istreambuf_iterator is a bare
// streambuf*), r2=end iterator (unused - the get area's own end bounds us),
// r3=ios_base* (unused, same scope cut as num_put), stack slot 4 = iostate*,
// slot 5 = the destination. Returns the iterator, i.e. the streambuf - exactly
// how the working num_put overloads return theirs.
uint32_t NumGetEntry(GuestEngine& eng, uint32_t sb, uint32_t sp, NumGetKind kind, int width) {
uint32_t errPtr = eng.ReadIncomingArg(4, 0, 0, 0, 0, sp);
uint32_t valPtr = eng.ReadIncomingArg(5, 0, 0, 0, 0, sp);
return NumGetCommon(eng, sb, errPtr, valPtr, kind, width);
}
uint32_t Shim_num_get_bool(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Bool, 1); }
uint32_t Shim_num_get_s32(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Signed, 4); }
uint32_t Shim_num_get_s64(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Signed, 8); }
uint32_t Shim_num_get_u16(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 2); }
uint32_t Shim_num_get_u32(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 4); }
uint32_t Shim_num_get_u64(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Unsigned, 8); }
uint32_t Shim_num_get_float(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp) { return NumGetEntry(e, sb, sp, NumGetKind::Float, 4); }
uint32_t Shim_num_get_double(GuestEngine& e, uint32_t, uint32_t sb, uint32_t, uint32_t, uint32_t sp){ return NumGetEntry(e, sb, sp, NumGetKind::Float, 8); }
// use_facet<Facet>(locale) looks up a facet by its static `id` member and
// THROWS std::bad_cast if not found - facet types with no evidence of ever
// being requested by this binary (num_get, ...) keep the original
@@ -635,17 +832,30 @@ uint32_t Shim_num_put_double(GuestEngine& eng, uint32_t, uint32_t streambuf, uin
// and num_put<char> are different - confirmed real, actually-called use
// (see their own comments above) - return the real facet objects built
// above.
uint32_t Shim_use_facet(GuestEngine&, uint32_t, uint32_t idAddr, uint32_t, uint32_t, uint32_t) {
uint32_t Shim_use_facet(GuestEngine& eng, uint32_t, uint32_t idAddr, uint32_t, uint32_t, uint32_t) {
if (idAddr == g_ctypeCharId) return g_ctypeCharFacet;
if (idAddr == g_numPutCharId) return g_numPutCharFacet;
static bool logged = false;
if (!logged) {
logged = true;
Log("rtti_shims: std::locale::use_facet() for an id other than ctype<char>/num_put<char> "
"(0x%x) - this engine's locale objects have no real facet table for it (see "
"rtti_shims.cpp) - returning NULL rather than a facet object a subsequent virtual call "
"would crash through; real libc++ would throw std::bad_cast here, which needs a guest "
"stack unwinder this engine doesn't have (see __cxa_bad_typeid's own comment)", idAddr);
if (idAddr == g_numGetCharId) return g_numGetCharFacet;
// Report once PER DISTINCT ID, not once overall. The previous single
// `static bool` meant the first unimplemented facet hid every other one
// behind it, so a crash caused by the second facet looked like it had no
// diagnostic at all. Naming the facet and the call site matters just as
// much: the id is an address our own AllocPermanent handed out, which says
// nothing on its own - resolving one used to cost a rebuild and a repro.
static std::set<uint32_t> reported;
if (reported.insert(idAddr).second) {
const char* name = eng.NameForDataSymbol(idAddr);
uint32_t callerLr = 0;
if (uc_engine* uc = eng.uc()) uc_reg_read(uc, UC_ARM_REG_LR, &callerLr);
Log("rtti_shims: std::locale::use_facet(%s) is NOT implemented - id=0x%x, called from guest "
"LR=0x%x. Returning NULL; the caller will dereference it and take a wild pointer through "
"whatever the NULL facet's vtable slot reads (this is the known "
"\"\\x7fELF\"-as-a-pointer crash shape). ctype<char> and num_put<char> are the only "
"facets this engine builds - implement this one next if the game depends on it. Real "
"libc++ would throw std::bad_cast, which needs a guest stack unwinder this engine does "
"not have (see __cxa_bad_typeid's own comment).",
name ? name : "<unregistered id>", idAddr, callerLr);
}
return 0;
}
@@ -774,9 +984,52 @@ void SetupRttiDataSymbols(GuestEngine& engine) {
g_numPutCharId = engine.AllocPermanent(4);
engine.RegisterDataSymbol("_ZNSt6__ndk17num_putIcNS_19ostreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", g_numPutCharId);
}
// ---- Other facet `id` statics - inert (see Shim_use_facet's own
// comment - no evidence anything requests these yet) ----
engine.RegisterDataSymbol("_ZNSt6__ndk17num_getIcNS_19istreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", engine.AllocPermanent(4));
// ---- num_get<char> facet - a real, callable vtable. This used to be an
// inert `id` with no facet behind it, on the stated grounds that nothing
// requested it; that turned out to be wrong and it cost a crash (see
// NumGetCommon's own comment).
//
// The slot offsets are DERIVED, not guessed. num_put above is confirmed
// working, and its known-good slots (16=long, 20=long long,
// 24=unsigned long, 32=double, 40=const void*) only line up with this
// NDK's declaration order if three slots precede the first virtual:
// the complete and deleting destructors plus
// __shared_count::__on_zero_shared. Applying that same prefix to
// num_get's declaration order in this NDK's own <locale> produces the
// table below - and offset 28, the one the game actually calls, lands on
// `unsigned int&`, exactly what basic_istream::operator>>(unsigned int&)
// would invoke. Two independent routes, same answer.
//
// Slots outside this table still route to the logging stub rather than to
// a guess, so the next gap names itself instead of crashing.
{
constexpr int kNumGetCharVtableSlots = 24;
GuestAddr vtable = engine.AllocPermanent(kNumGetCharVtableSlots * 4);
for (int i = 0; i < kNumGetCharVtableSlots; i++) {
int byteOffset = i * 4;
ImportShimFn impl = nullptr;
switch (byteOffset) {
case 12: impl = Shim_num_get_bool; break; // bool&
case 16: impl = Shim_num_get_s32; break; // long&
case 20: impl = Shim_num_get_s64; break; // long long&
case 24: impl = Shim_num_get_u16; break; // unsigned short&
case 28: impl = Shim_num_get_u32; break; // unsigned int& <- the crashing call
case 32: impl = Shim_num_get_u32; break; // unsigned long& (32-bit here)
case 36: impl = Shim_num_get_u64; break; // unsigned long long&
case 40: impl = Shim_num_get_float; break; // float&
case 44: impl = Shim_num_get_double; break; // double&
case 48: impl = Shim_num_get_double; break; // long double& (== double on ARM32)
default: break;
}
auto* ctx = new FacetSlotCtx{byteOffset, impl};
GuestAddr stub = engine.AllocCodeStub(FacetSlotDispatch, ctx);
if (vtable && stub) memcpy(engine.G2H(vtable + (uint32_t)byteOffset), &stub, 4);
}
g_numGetCharFacet = engine.AllocPermanent(4);
if (g_numGetCharFacet && vtable) memcpy(engine.G2H(g_numGetCharFacet), &vtable, 4);
g_numGetCharId = engine.AllocPermanent(4);
engine.RegisterDataSymbol("_ZNSt6__ndk17num_getIcNS_19istreambuf_iteratorIcNS_11char_traitsIcEEEEE2idE", g_numGetCharId);
}
// ---- The shared "classic locale" every locale::locale() call returns
// (see Shim_locale_ctor's own comment) ----
@@ -785,6 +1038,12 @@ void SetupRttiDataSymbols(GuestEngine& engine) {
void RegisterRttiImportShims(GuestEngine& engine) {
engine.RegisterImportShim("__dynamic_cast", Shim_dynamic_cast);
// Same shim under a second name, which is what the guest-side fast path
// tail-calls for the ~16% of cases it does not handle (see
// dyncast_fastpath.cpp). Registered here rather than there so the two
// names cannot drift apart: whatever __dynamic_cast resolves to, the
// fallback resolves to the same function.
engine.RegisterImportShim("__dynamic_cast_slowpath", Shim_dynamic_cast);
engine.RegisterImportShim("__cxa_bad_typeid", Shim_cxa_bad_typeid);
engine.RegisterImportShim("_ZNSt13runtime_errorC2EPKc", Shim_runtime_error_ctor);
mpcore/src/main/cpp/game_lifecycle_stubs_extra2.cpp
+109 -5
View File
@@ -9,17 +9,121 @@
#include <jni.h>
#include <cstdint>
#include <cstring>
#include <set>
#include <string>
#include "emu/jni_shim.h"
#include "util/util.h"
#include "real_native_call.h"
#include "real_native_offsets.h"
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodGetInfo(JNIEnv*, jobject, jint) {
return 0;
// ---- FMOD audio bridge (task #67, 2026-09-21) ----
//
// Both of these used to `return 0`, silently, on the stated grounds that FMOD
// was not loaded because no arm64 build exists. That is still true of arm64 -
// but the game's real ARM32 libfmodex/libfmodevent now run inside the engine
// as secondary guest images, and both of these symbols are among their
// exports, so the calls can be forwarded to the real implementations.
//
// Why this pair is the whole audio path: FMOD on this Android build does not
// drive the output device from native code. org/fmod/FMODAudioDevice.java owns
// an AudioTrack and pulls PCM down through these two methods. Its thread reads
//
// int rate = fmodGetInfo(FMOD_INFO_SAMPLERATE);
// if (rate > 0) { ...create the AudioTrack, then loop on fmodProcess... }
//
// so a `return 0` from fmodGetInfo alone made that thread give up before the
// AudioTrack was ever constructed. Nothing downstream could have produced
// sound regardless of what FMOD itself did.
// Re-enabled 2026-09-21 once the JNI call epoch became per-thread.
//
// These two run on FMODAudioDevice's own AudioTrack thread, and the first
// attempt aborted the process inside GLThread with "JNI DETECTED ERROR IN
// APPLICATION: jfieldID was NULL". The first diagnosis - that CallRealNative's
// SetRealEnv() clobbered a shared JNIEnv - was WRONG: real_env_ was already
// thread_local, with a lazy AttachCurrentThread fallback.
//
// The actual culprit was one line further on. SetRealEnv also calls
// JniHandleTable::BumpCallEpoch(), and that epoch was a single process-wide
// counter whose own comment said it invalidates the previous call's local refs
// "whether or not it's the same thread". So every 100 ms this bridge was
// invalidating the references GLThread held mid-call. The epoch is per-thread
// now, which is the more accurate model anyway - a local ref's lifetime is
// scoped to a native call on its own thread.
static constexpr bool kEnableFmodAudioBridge = true;
static GuestAddr FmodGuestSymbol(const char* name) {
GuestAddr addr = GuestEngine::Instance().LookupSecondaryExport(name);
if (!addr) {
static std::set<std::string> reported;
if (reported.insert(name).second) {
Log("fmod_bridge: '%s' is not exported by any loaded guest image - audio stays silent. "
"Did libfmodex.so/libfmodevent.so fail to load? (see LoadSiblingLibraries)", name);
}
}
return addr;
}
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodProcess(JNIEnv*, jobject, jobject) {
return 0;
Java_org_fmod_FMODAudioDevice_fmodGetInfo(JNIEnv* env, jobject thiz, jint info) {
if (!kEnableFmodAudioBridge) return 0;
static GuestAddr fn = FmodGuestSymbol("Java_org_fmod_FMODAudioDevice_fmodGetInfo");
if (!fn) return 0;
// Deliberately NOT logged per call: FMODAudioDevice polls this from its
// audio thread continuously (1,210 calls in a 40-second run), so a log
// line here is a steady drip into a buffer we already lose history to.
return (jint)CallRealNative(env, thiz, fn, {(uint32_t)info});
}
extern "C" JNIEXPORT jint JNICALL
Java_org_fmod_FMODAudioDevice_fmodProcess(JNIEnv* env, jobject thiz, jobject buffer) {
if (!kEnableFmodAudioBridge) return 0;
static GuestAddr fn = FmodGuestSymbol("Java_org_fmod_FMODAudioDevice_fmodProcess");
if (!fn || !buffer) return 0;
// The ByteBuffer came from Java's own allocateDirect, so it lives at a host
// address the guest cannot write to - GetDirectBufferAddress would hand
// guest FMOD a zero. Bounce through a guest-memory buffer of the same size
// instead, exactly as the font fix does for AndroidBitmap_lockPixels, and
// copy the rendered PCM back out afterwards.
jlong capacity = env->GetDirectBufferCapacity(buffer);
void* hostDest = env->GetDirectBufferAddress(buffer);
if (capacity <= 0 || !hostDest) {
static bool logged = false;
if (!logged) {
logged = true;
Log("fmod_bridge: fmodProcess got a ByteBuffer with no direct address (capacity=%lld) - "
"returning silence", (long long)capacity);
}
return 0;
}
// Allocated once and reused: this runs on the AudioTrack thread at the
// audio buffer rate, so per-call allocation would be both wasteful and a
// source of jitter. The capacity is fixed by FMOD's DSP buffer settings
// and does not change after the device starts, but it is re-checked rather
// than assumed.
static GuestAddr guestBuf = 0;
static uint32_t guestBufSize = 0;
static uint32_t guestBufHandle = 0;
if (guestBufSize != (uint32_t)capacity) {
guestBuf = GuestEngine::Instance().heap().Alloc((uint32_t)capacity);
if (!guestBuf) {
Log("fmod_bridge: could not allocate a %lld-byte guest audio buffer - silence",
(long long)capacity);
return 0;
}
guestBufSize = (uint32_t)capacity;
guestBufHandle = JniShim::Instance().NewGuestBackedDirectBuffer(
GuestEngine::Instance(), guestBuf, guestBufSize);
Log("fmod_bridge: audio bounce buffer ready - %u bytes at guest 0x%x (handle %u)",
guestBufSize, guestBuf, guestBufHandle);
}
if (!guestBufHandle) return 0;
memset(GuestEngine::Instance().G2H(guestBuf), 0, guestBufSize);
jint r = (jint)CallRealNative(env, thiz, fn, {guestBufHandle});
memcpy(hostDest, GuestEngine::Instance().G2H(guestBuf), guestBufSize);
return r;
}
#define NIMBLE_COMPONENT_STUB(name) \
mpcore/src/main/cpp/main.cpp
+20
View File
@@ -5,6 +5,7 @@
#include <android/log.h>
#include <jni.h>
#include "main.h"
#include "crash_handler.h"
#include "util/util.h"
#include <unistd.h>
#include <unwind.h>
@@ -28,6 +29,7 @@
#include "emu/gles_shim.h"
#include "emu/libc_shims.h"
#include "emu/rtti_shims.h"
#include "emu/dyncast_fastpath.h"
#include "emu/fmod_shims.h"
#include "emu/profiler.h"
#include "real_native_offsets.h"
@@ -133,6 +135,9 @@ bool LoadEmulatedLibapp(const char* path, JNIEnv* env, jobject thiz) {
RegisterPthreadImportShims(engine);
RegisterGlesImportShims(engine);
RegisterRttiImportShims(engine);
// Must follow RegisterRttiImportShims: the fast path's fallback resolves
// "__dynamic_cast_slowpath", which that call registers.
RegisterDynamicCastFastPath(engine);
RegisterFmodImportShims(engine);
// Must be registered BEFORE LoadImage() - SetupRttiDataSymbols needs to
// run after MapSegments (for AllocPermanent) but strictly before
@@ -493,6 +498,21 @@ static constexpr bool kEnableMapScreenCtorTraceHook = false; // TEMP: isolating
// comment), which JNI_OnLoad has no reliable way to obtain on its own
// (no Context/AssetManager access at this point). See
// Java_..._MultiplayerCore_loadEmulatedLibapp below for where that now happens.
// extern "C" is NOT optional here: without it the name is C++-mangled and the
// JVM cannot find it by its JNI name. The first version omitted it, and the
// only symptom was "No implementation found" at runtime - which looked exactly
// like a load-order problem and cost a wrong fix before the symbol table was
// actually read.
extern "C" JNIEXPORT void JNICALL
Java_com_ea_ironmonkey_GameActivityMain_nativeInstallCrashHandler(JNIEnv* env, jobject,
jstring dir, jstring buildStamp) {
const char* d = dir ? env->GetStringUTFChars(dir, nullptr) : nullptr;
const char* b = buildStamp ? env->GetStringUTFChars(buildStamp, nullptr) : nullptr;
InstallCrashHandler(d, b);
if (d) env->ReleaseStringUTFChars(dir, d);
if (b) env->ReleaseStringUTFChars(buildStamp, b);
}
JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) {
JNIEnv* env = nullptr;
if (vm->GetEnv((void**)&env, JNI_VERSION_1_6) == JNI_OK) {
mpcore/src/main/cpp/third_party/unicorn/include/uc_priv.h
Vendored
+39
View File
@@ -361,6 +361,45 @@ struct uc_struct {
// linked lists containing hooks per type
struct list hook[UC_HOOK_MAX];
struct list hooks_to_del;
// NFSMW arm64-poc, task #61: flat guest mapping.
//
// When non-zero, the whole guest address space is ONE contiguous host
// block starting here, and a guest address translates to a host address
// by plain addition - which is exactly what GuestEngine's G2H() already
// does. The aarch64 TCG backend can then emit a single
// `LDR Wd, [X28, Wn, UXTW]` per guest memory access instead of the
// nine-instruction softmmu TLB check (see tcg_out_qemu_ld/st).
//
// Measured motivation: those TLB sequences are 29.8% of ALL generated
// host code (16,202 sequences x 35.8 bytes against 1,943,672 bytes total,
// counted during a prologue load).
//
// Set by the engine AFTER mapping its spans and BEFORE any translation
// happens, and never changed afterwards - the generated code bakes this
// value into a register, so changing it later would silently corrupt
// every existing translated block.
uint64_t flat_map_base;
// Regenerates the TCG prologue so a freshly-set flat_map_base actually
// reaches the generated code. Needed because the prologue is built inside
// uc_open(), before the embedder can say anything - and it is the prologue
// that loads the base into X28. Set by tcg_exec_init, which unlike uc.c is
// compiled per-architecture and can name tcg_prologue_init/tcg_nb_tbs.
// Returns 0 on success, -1 if blocks were already translated (in which
// case the base is left alone rather than corrupting those blocks).
int (*uc_reinit_prologue)(struct uc_struct *uc);
// NFSMW arm64-poc: page -> MemoryRegion cache for notdirty_write, which
// otherwise calls memory_mapping() on EVERY guest write just to test one
// permission bit (see cputlb.c). Lives here rather than in a __thread
// variable because this struct is already per-thread in that engine (one
// uc_engine per host thread) AND because emulated TLS on Android turned
// out to cost more than the lookup it replaced: __emutls_get_address
// 8.11% + pthread_getspecific 3.07%, measured, against the 7.5% being
// removed.
uint64_t mr_cache_tag[512];
void *mr_cache_val[512];
int mr_cache_ready;
int hooks_count[UC_HOOK_MAX];
// hook to count number of instructions for uc_emu_start()
mpcore/src/main/cpp/third_party/unicorn/include/unicorn/unicorn.h
Vendored
+29
View File
@@ -1449,6 +1449,35 @@ size_t uc_context_size(uc_engine *uc);
UNICORN_EXPORT
uc_err uc_context_free(uc_context *context);
/*
NFSMW arm64-poc extension (task #61). Declare that the entire guest address
space is one contiguous host block starting at @base, so the JIT can reach
guest memory by plain addition instead of a software TLB lookup. On aarch64
hosts with a 32-bit guest this replaces a nine-instruction TLB check with a
single LDR/STR, which is why it exists.
Enabling this DISABLES, for every emulated memory access:
- UC_PROT_* enforcement (uc_mem_protect becomes advisory),
- UC_HOOK_MEM_READ / _WRITE / _INVALID callbacks,
- self-modifying-code detection, so guest stores to pages that already
hold translated code will NOT invalidate those translations.
Callers depending on any of the above must leave it off.
@uc: handle returned by uc_open()
@base: host address the guest's address 0 maps to. The caller must guarantee
that [base, base + 4GiB) is reserved host address space, since a 32-bit
guest address is zero-extended and used as an unchecked offset from it.
Pass 0 to keep the normal software-MMU path.
Must be called before any code is translated - generated code bakes @base
into a register, so a later change would silently corrupt existing blocks.
@return UC_ERR_OK on success, or other value on failure (refer to uc_err enum
for detailed error).
*/
UNICORN_EXPORT
uc_err uc_set_flat_map_base(uc_engine *uc, uint64_t base);
#ifdef __cplusplus
}
#endif
mpcore/src/main/cpp/third_party/unicorn/qemu/accel/tcg/cputlb.c
Vendored
+62 -1
View File
@@ -18,6 +18,8 @@
*/
#include "qemu/osdep.h"
#define UC_MR_CACHE_ENTRIES 512
#define UC_MR_CACHE_ENABLED 1
#include "cpu.h"
#include "exec/exec-all.h"
#include "exec/memory.h"
@@ -1194,7 +1196,66 @@ static void notdirty_write(CPUState *cpu, vaddr mem_vaddr, unsigned size,
struct uc_struct *uc = cpu->uc;
#endif
ram_addr_t ram_addr = mem_vaddr + iotlbentry->addr;
MemoryRegion *mr = cpu->uc->memory_mapping(cpu->uc, tlbe->paddr | (mem_vaddr & ~TARGET_PAGE_MASK));
// PROJECT PATCH (2026-09-20, NFSMW arm64-poc). memory_mapping() -
// find_memory_mapping -> address_space_translate -> flatview_translate -
// was called here on EVERY notdirty write, unconditionally, even though
// its result is used only to test one bit (is this region executable).
// Measured: flatview_translate 4.46% + find_memory_mapping 3.02% of a
// clean in-game profile, and a probe on this exact line counted over a
// million calls per 3 seconds with exec_region=0 - i.e. the lookup ran in
// full and then the answer turned out not to matter.
//
// Task #54 removed the tb_invalidate work for non-executable spans but
// left this lookup in front of it, so most of the cost survived. TLB
// capacity was separately ruled out as the cause (task #53).
//
// The map is static here: GuestEngine maps its five spans once in
// CreateConfiguredEngine and never calls uc_mem_map/protect again once
// guest code is running (guard pages and RELRO are set during that same
// setup). So the page -> MemoryRegion answer cannot change, and a small
// direct-mapped cache removes the lookup for everything but a cold miss.
//
// INVARIANT THIS RELIES ON: anyone who starts changing mappings or
// permissions at RUNTIME must clear this cache, or stale permissions will
// be used. It is per-thread, so no locking is needed.
//
// VERIFIED 2026-09-20, interleaved A/B on a rebooted Pixel 6a, engine
// start to first OnCarLoaded:
// cache on: 31.27 / 31.34 / 31.22 s mean 31.28
// cache off: 32.27 / 32.12 / 32.44 s mean 32.28
// 3.1% faster, ranges fully separate. Profile confirms the mechanism:
// flatview_translate 4.46% -> 3.17%, find_memory_mapping 3.02% -> 2.36%.
//
// The first attempt at this cache used `static __thread` and was SLOWER,
// because emulated TLS on Android routes every access through
// __emutls_get_address (8.11%) + pthread_getspecific (3.07%) - more than
// the 7.5% it was removing. Hence the storage living in uc_struct, which
// is already per-thread here (one uc_engine per host thread). Do not
// reintroduce __thread in this path.
MemoryRegion *mr;
if (!UC_MR_CACHE_ENABLED) {
mr = cpu->uc->memory_mapping(cpu->uc, tlbe->paddr | (mem_vaddr & ~TARGET_PAGE_MASK));
} else {
struct uc_struct *ucc = cpu->uc;
target_ulong page = tlbe->paddr | (mem_vaddr & ~TARGET_PAGE_MASK);
uint64_t tag = (uint64_t)(page >> TARGET_PAGE_BITS);
unsigned idx = (unsigned)(tag & (UC_MR_CACHE_ENTRIES - 1));
if (!ucc->mr_cache_ready) {
for (unsigned i = 0; i < UC_MR_CACHE_ENTRIES; i++) {
ucc->mr_cache_tag[i] = (uint64_t)-1;
}
ucc->mr_cache_ready = 1;
}
if (ucc->mr_cache_tag[idx] == tag) {
mr = (MemoryRegion *)ucc->mr_cache_val[idx];
} else {
mr = ucc->memory_mapping(ucc, page);
ucc->mr_cache_tag[idx] = tag;
ucc->mr_cache_val[idx] = mr;
}
}
if (mr && (mr->perms & UC_PROT_EXEC) != 0) {
struct page_collection *pages
mpcore/src/main/cpp/third_party/unicorn/qemu/accel/tcg/translate-all.c
Vendored
+18
View File
@@ -1217,6 +1217,23 @@ static uc_err uc_gen_tb(struct uc_struct *uc, uint64_t addr, uc_tb *out_tb)
/* Must be called before using the QEMU cpus. 'tb_size' is the size
(in bytes) allocated to the translation buffer. Zero means default
size. */
/*
* NFSMW arm64-poc, task #61. Rebuild the prologue in place after
* uc_set_flat_map_base(). Safe only with zero translated blocks - the caller
* enforces that - because tcg_prologue_init resets code_ptr to the start of
* the code buffer and re-emits from there.
*/
static int uc_reinit_prologue(struct uc_struct *uc)
{
if (tcg_nb_tbs(uc->tcg_ctx) != 0) {
return -1;
}
tb_exec_unlock(uc);
tcg_prologue_init(uc->tcg_ctx);
tb_exec_lock(uc);
return 0;
}
void tcg_exec_init(struct uc_struct *uc, uint32_t tb_size)
{
/* remove tcg object. init here. */
@@ -1233,6 +1250,7 @@ void tcg_exec_init(struct uc_struct *uc, uint32_t tb_size)
tb_exec_unlock(uc);
tcg_prologue_init(uc->tcg_ctx);
tb_exec_lock(uc);
uc->uc_reinit_prologue = uc_reinit_prologue;
/* cpu_interrupt_handler is not used in uc1 */
uc->l1_map = g_malloc0(sizeof(void *) * V_L1_MAX_SIZE);
/* Invalidate / Cache TBs */
mpcore/src/main/cpp/third_party/unicorn/qemu/include/exec/cpu-defs.h
Vendored
+20
View File
@@ -83,6 +83,26 @@ typedef uint64_t target_ulong;
#endif
#define CPU_TLB_DYN_MIN_BITS 6
// NFSMW arm64-poc, 2026-09-19: raising this to 12 was TRIED AND REFUTED.
//
// The reasoning looked sound: 8 bits = 256 entries x 4KB = 1 MB of guest
// coverage against a working set of hundreds of MB, and
// tlb_mmu_resize_locked resizes ONLY on a TLB flush. Measured on device,
// that function runs about twice per second and current_entries stayed at
// 256 for a whole run - so adaptive growth genuinely never helps here.
//
// Setting 12 (4096 entries, 16 MB covered) did take effect - verified live,
// current_entries=4096 - and bought nothing:
//
// interleaved A/B, load to first OnCarLoaded
// 12 bits: 32.36 / 32.26 s
// 8 bits: 32.86 / 32.32 s (0.28s apart, ranges overlap)
//
// The profile actually moved the wrong way: tlb_set_page_with_attrs went
// 0.86% -> 2.92%, because a larger TLB costs more to fill and flush. So the
// softmmu lookup cost (flatview_translate + find_memory_mapping, ~7.5%) is
// NOT driven by TLB capacity. Left at the upstream default; do not retry
// without a different theory for where those misses come from.
#define CPU_TLB_DYN_DEFAULT_BITS 8
# if HOST_LONG_BITS == 32
mpcore/src/main/cpp/third_party/unicorn/qemu/tcg/aarch64/tcg-target.inc.c
Vendored
+50 -1
View File
@@ -69,13 +69,39 @@ static const int tcg_target_call_oarg_regs[1] = {
#define TCG_REG_TMP TCG_REG_X30
#define TCG_VEC_TMP TCG_REG_V31
/* The register holding the base of the guest's address space. Defined
unconditionally because task #61's flat mapping below uses it under
CONFIG_SOFTMMU too, for exactly the same purpose as user-mode does. */
#define TCG_REG_GUEST_BASE TCG_REG_X28
/*
* NFSMW arm64-poc, task #61: flat guest mapping under softmmu.
*
* When the embedder has told us the whole guest address space is one
* contiguous host block (uc->flat_map_base), a guest access needs no TLB
* lookup at all - the host address is base + zero-extended guest address,
* which aarch64 does in the addressing mode itself. That turns the nine
* instruction tcg_out_tlb_read sequence plus its slow-path label into a
* SINGLE instruction: LDR Wd, [X28, Wn, UXTW].
*
* Only valid because TARGET_LONG_BITS == 32 here: a 32-bit guest address
* zero-extends to at most 4 GiB - 1, so it cannot escape the host
* reservation. The guard is spelled out rather than assumed.
*
* What this deliberately gives up (see guest_engine.cpp's SetFlatMapBase):
* UC_PROT_* enforcement, UC_HOOK_MEM_* callbacks, and self-modifying-code
* detection via notdirty_write. The embedder must not enable it while it
* depends on any of those.
*/
#define UC_FLAT_MAP(s) (TARGET_LONG_BITS == 32 && (s)->uc->flat_map_base)
#ifndef CONFIG_SOFTMMU
/* Note that XZR cannot be encoded in the address base register slot,
as that actaully encodes SP. So if we need to zero-extend the guest
address, via the address index register slot, we need to load even
a zero guest base into a register. */
#define USE_GUEST_BASE (guest_base != 0 || TARGET_LONG_BITS == 32)
#define TCG_REG_GUEST_BASE TCG_REG_X28
#endif
static inline bool reloc_pc26(tcg_insn_unit *code_ptr, tcg_insn_unit *target)
@@ -1823,6 +1849,12 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg,
unsigned mem_index = get_mmuidx(oi);
tcg_insn_unit *label_ptr;
if (UC_FLAT_MAP(s)) {
tcg_out_qemu_ld_direct(s, memop, ext, data_reg,
TCG_REG_GUEST_BASE, otype, addr_reg);
return;
}
tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1);
tcg_out_qemu_ld_direct(s, memop, ext, data_reg,
TCG_REG_X1, otype, addr_reg);
@@ -1848,6 +1880,12 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg,
unsigned mem_index = get_mmuidx(oi);
tcg_insn_unit *label_ptr;
if (UC_FLAT_MAP(s)) {
tcg_out_qemu_st_direct(s, memop, data_reg,
TCG_REG_GUEST_BASE, otype, addr_reg);
return;
}
tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0);
tcg_out_qemu_st_direct(s, memop, data_reg,
TCG_REG_X1, otype, addr_reg);
@@ -2850,6 +2888,17 @@ static void tcg_target_qemu_prologue(TCGContext *s)
tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, guest_base);
tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE);
}
#else
/*
* Task #61. Same idea as the user-mode path above, but the base comes
* from the embedder instead of QEMU's global guest_base. X28 was already
* pushed as x27's pair partner a few lines up, and the epilogue restores
* it, so clobbering it here is safe for our caller.
*/
if (UC_FLAT_MAP(s)) {
tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_GUEST_BASE, s->uc->flat_map_base);
tcg_regset_set_reg(s->reserved_regs, TCG_REG_GUEST_BASE);
}
#endif
tcg_out_mov(s, TCG_TYPE_PTR, TCG_AREG0, tcg_target_call_iarg_regs[0]);
mpcore/src/main/cpp/third_party/unicorn/qemu/tcg/tcg-op.c
Vendored
+36 -6
View File
@@ -2850,6 +2850,36 @@ static void gen_ldst_i64(TCGContext *tcg_ctx, TCGOpcode opc, TCGv_i64 val, TCGv
// Unicorn engine
// check if the last memory access was invalid
// if so, we jump to the block epilogue to quit immediately.
// PROJECT PATCH (2026-09-19, NFSMW arm64-poc). Unicorn emits an exit-request
// check inside tcg_gen_qemu_ld/st - i.e. on EVERY GUEST LOAD AND STORE, not
// per basic block. Upstream QEMU does not do this; it exists so that a
// uc_emu_stop() issued from a MEMORY hook callback takes effect immediately
// rather than at the next block boundary.
//
// Measured cost in this project: helper_check_exit_request_arm was the single
// largest symbol in a clean in-game profile at 8.71%, against 0.98% for
// helper_uc_tracecode - a ratio that made no sense until these call sites
// were found, because the two are emitted together at the ARM translator's
// hook sites and should have matched.
//
// This engine never stops emulation from a memory callback: mem_fault_hook_cb
// returns false (it only logs), and the one accelerator that does call
// uc_emu_stop (FnvHashAccelHookCb) is a UC_HOOK_CODE hook, where the check is
// still emitted. So the per-access check is pure overhead here.
//
// VERIFIED 2026-09-19 on a freshly rebooted device, three interleaved pairs,
// load to first OnCarLoaded:
// without the check: 33.42 / 33.10 / 33.32 s mean 33.28
// with the check: 35.62 / 35.43 / 35.49 s mean 35.51
// 6.3% faster, ranges completely separate. (The first attempt at this
// measurement produced one usable pair before the device's screen dozed and
// the engine stopped rendering entirely - see the session log.)
//
// Kept as a named constant rather than deleted, because this is a vendored
// tree: anyone adding a memory hook that calls uc_emu_stop MUST flip this
// back, or that stop will be deferred to the next block boundary.
static const int kEmitExitCheckOnMemoryAccess = 0;
void check_exit_request(TCGContext *tcg_ctx)
{
// Unicorn:
@@ -2917,7 +2947,7 @@ void tcg_gen_qemu_ld_i32(TCGContext *tcg_ctx, TCGv_i32 val, TCGv addr, TCGArg id
}
}
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
}
void tcg_gen_qemu_st_i32(TCGContext *tcg_ctx, TCGv_i32 val, TCGv addr, TCGArg idx, MemOp memop)
@@ -2952,7 +2982,7 @@ void tcg_gen_qemu_st_i32(TCGContext *tcg_ctx, TCGv_i32 val, TCGv addr, TCGArg id
tcg_temp_free_i32(tcg_ctx, swap);
}
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
}
void tcg_gen_qemu_ld_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop)
@@ -2967,7 +2997,7 @@ void tcg_gen_qemu_ld_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg id
} else {
tcg_gen_movi_i32(tcg_ctx, TCGV_HIGH(tcg_ctx, val), 0);
}
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
return;
}
#endif
@@ -3009,7 +3039,7 @@ void tcg_gen_qemu_ld_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg id
g_assert_not_reached();
}
}
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
}
void tcg_gen_qemu_st_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg idx, MemOp memop)
@@ -3019,7 +3049,7 @@ void tcg_gen_qemu_st_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg id
#if TCG_TARGET_REG_BITS == 32
if ((memop & MO_SIZE) < MO_64) {
tcg_gen_qemu_st_i32(tcg_ctx, TCGV_LOW(tcg_ctx, val), addr, idx, memop);
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
return;
}
#endif
@@ -3055,7 +3085,7 @@ void tcg_gen_qemu_st_i64(TCGContext *tcg_ctx, TCGv_i64 val, TCGv addr, TCGArg id
if (swap) {
tcg_temp_free_i64(tcg_ctx, swap);
}
check_exit_request(tcg_ctx);
if (kEmitExitCheckOnMemoryAccess) check_exit_request(tcg_ctx);
}
static void tcg_gen_ext_i32(TCGContext *tcg_ctx, TCGv_i32 ret, TCGv_i32 val, MemOp opc)
mpcore/src/main/cpp/third_party/unicorn/uc.c
Vendored
+32
View File
@@ -2594,6 +2594,38 @@ uc_err uc_context_restore(uc_engine *uc, uc_context *context)
return UC_ERR_OK;
}
UNICORN_EXPORT
uc_err uc_set_flat_map_base(uc_engine *uc, uint64_t base)
{
uint64_t previous = uc->flat_map_base;
bool was_initialised = uc->init_done;
// Set the base BEFORE UC_INIT. Engine setup is lazy in Unicorn - uc_open()
// only allocates the handle, and the first API call using UC_INIT is what
// builds the CPU, the TCG context and the prologue. Assigning here means
// that, in the common case, the prologue is generated with the right base
// the first time and needs no rebuilding at all.
uc->flat_map_base = base;
UC_INIT(uc);
if (!was_initialised) {
return UC_ERR_OK;
}
// The engine was already up, so a prologue exists that loaded a different
// base into the register. Rebuild it - which also refuses, leaving the old
// base in place, if any block has already been translated against it.
if (!uc->uc_reinit_prologue) {
uc->flat_map_base = previous;
return UC_ERR_HANDLE;
}
if (uc->uc_reinit_prologue(uc) != 0) {
uc->flat_map_base = previous;
return UC_ERR_ARG;
}
return UC_ERR_OK;
}
UNICORN_EXPORT
uc_err uc_context_free(uc_context *context)
{
mpcore/src/main/java/nfs/mod/mpcore/MultiplayerCore.kt
+14
View File
@@ -22,6 +22,20 @@ object MultiplayerCore {
*/
external fun triggerTrueDirectCarSelectJump()
/**
* ARM64-only-device prototype (see /ARM64_TRANSLATION_LAYER.md): loads
* the original armeabi-v7a libapp.so through mpcore's embedded ARM32
* CPU-emulation core (Unicorn) instead of a real System.loadLibrary
* dlopen(), and installs whichever hooks are enabled in main.cpp.
* `path` must be a real file (e.g. extracted from assets to
* filesDir/libapp.so at first run - libapp.so is shipped as a raw
* asset, not under jniLibs, since this app declares only arm64-v8a and
* the packaging system would refuse/ignore an armeabi-v7a .so there).
* Returns false on any load failure (bad ELF, mmap failure, ...) -
* check logcat's "GuestEngine"/"mpcore_log" tag for why.
*/
external fun loadEmulatedLibapp(path: String): Boolean
fun loadCore() {
System.loadLibrary("mpcore")
installCarSelectLoadoutTestTrigger()
ostream_repro/CMakeLists.txt
+26
View File
@@ -0,0 +1,26 @@
# Standalone armeabi-v7a artifact for the 2026-09-16 "isolated
# std::ostringstream repro" test (see ARM64_TRANSLATION_LAYER.md and
# ostream_repro.cpp's own top comment). Same pattern as ../trace_agent's own
# CMakeLists.txt - deliberately NOT wired into the main Gradle build
# (settings.gradle.kts), built and pushed to the device independently via
# build.sh. Default STL linkage (c++_shared, the NDK CMake toolchain's own
# default - NOT overridden to c++_static here) is a deliberate choice, not
# an oversight - see ostream_repro.cpp's top comment for why matching
# libapp.so's own dynamic-libc++ linkage matters for this specific test.
cmake_minimum_required(VERSION 3.22.1)
project(ostream_repro)
add_library(ostream_repro SHARED
ostream_repro.cpp
)
target_compile_options(ostream_repro PRIVATE -Wall -Wno-unused-parameter)
# Standalone native executable (2026-09-17, ARM64_TRANSLATION_LAYER.md - "does
# real hardware ever see runaway basic_stringbuf::overflow() growth" check).
# See ostream_stress.cpp's own top comment. Independent target, does not
# affect the ostream_repro library above.
add_executable(ostream_stress
ostream_stress.cpp
)
target_compile_options(ostream_stress PRIVATE -Wall -Wno-unused-parameter)
ostream_repro/build.sh
Executable
+45
View File
@@ -0,0 +1,45 @@
#!/usr/bin/env bash
# Builds libostream_repro.so for armeabi-v7a via the NDK's CMake toolchain
# file - standalone, deliberately outside the Gradle build, same pattern as
# ../trace_agent/build.sh. Output: build/libostream_repro.so, ready to push
# to a device for GuestEngine::LoadSecondaryImage to load (see
# mpcore/src/main/cpp/emu/ostream_repro_test.cpp).
set -euo pipefail
cd "$(dirname "$0")"
NDK="${ANDROID_NDK_HOME:-/home/megboyzz/Android/Sdk/ndk/27.0.12077973}"
TOOLCHAIN="$NDK/build/cmake/android.toolchain.cmake"
if [ ! -f "$TOOLCHAIN" ]; then
echo "NDK toolchain file not found at $TOOLCHAIN - set ANDROID_NDK_HOME" >&2
exit 1
fi
# ANDROID_STL=c++_static explicitly - see ostream_repro.cpp's own top
# comment for the full reasoning (tried c++_shared FIRST, actually built
# and inspected both outcomes with llvm-readelf before deciding, not
# guessed): the modern NDK r27 toolchain's libc++ headers extern-template-
# declare basic_stringbuf<char>/basic_ostream<char>/basic_ios<char>/
# basic_ostringstream<char> (their vtables AND ctors/dtors become UNDEFINED
# imports resolved against libc++_shared.so, confirmed via
# `llvm-readelf --dyn-syms` on a real c++_shared build of this exact file),
# which GuestEngine has zero shims for - that combination would make this
# test fail for an uninteresting, unrelated reason (unimplemented vtable
# stub returning 0) instead of actually exercising real, compiled
# std::basic_stringbuf<char>::str()/write() logic. c++_static compiles that
# logic directly into THIS .so's own .text - both a working build AND a
# closer functional match to what libapp.so's own (much older) toolchain
# evidently did for sub_79CD4/sub_27160C (confirmed real, compiled,
# fixed-address code inside libapp.so itself, never external imports -
# see ARM64_TRANSLATION_LAYER.md's 2026-09-16 entries).
cmake -B build -G Ninja \
-DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN" \
-DANDROID_ABI=armeabi-v7a \
-DANDROID_PLATFORM=android-27 \
-DANDROID_STL=c++_static \
-DCMAKE_BUILD_TYPE=Debug \
.
cmake --build build
echo "Built: $(pwd)/build/libostream_repro.so"
ostream_repro/ostream_repro.cpp
+132
View File
@@ -0,0 +1,132 @@
// Standalone, minimal armeabi-v7a artifact for the 2026-09-16
// ARM64_TRANSLATION_LAYER.md "isolated std::ostringstream repro" test - see
// that doc's entry of the same date for the full methodology and the
// GuestEngine-side driver (mpcore/src/main/cpp/emu/ostream_repro_test.cpp)
// that loads and calls this.
//
// Why this exists: the real game's own libapp.so, deep inside its shader-
// variant builder (sub_4702D8, per ARM64_TRANSLATION_LAYER.md's 2026-09-16
// entries), writes several string literals to a real, compiled
// std::ostringstream-shaped object via operator<< and then extracts the
// accumulated text via what's effectively basic_stringbuf<char>::str() -
// and the extraction always comes back empty under GuestEngine, even though
// register/memory probes confirmed real content was genuinely written
// first. Rather than keep chasing that one binary's own hardcoded
// addresses (explicitly the wrong approach per the user's own direction -
// "ты сейчас пытаешься подогнать эмуляцию к одному единственному бинарнику,
// это не правильный подход" - that only tells us about THIS game, not
// whether GuestEngine itself has a real, general ostringstream-extraction
// bug), this reproduces the EXACT SAME write-then-extract shape in total
// isolation from every other line of game code, compiled fresh by the same
// NDK toolchain and loaded as its own tiny ELF image
// (GuestEngine::LoadSecondaryImage) alongside (not instead of) libapp.so.
//
// Deliberately built the SAME way libapp.so itself is (default NDK
// ANDROID_STL=c++_shared, not a static-libstdc++/c++_static build) rather
// than statically linking libc++ into this .so: rtti_shims.cpp's own
// RegisterRttiImportShims already shows GuestEngine hand-reimplements a
// SPECIFIC set of libc++-internal symbols (locale/ios_base/ctype<char>/
// num_put<char>/__shared_weak_count/std::mutex - all undefined imports in
// libapp.so's own .dynsym, meaning libapp.so links these dynamically
// against libc++_shared.so too, while template-heavy code like
// basic_stringbuf<char>'s own methods gets compiled directly into
// libapp.so's .text, same as here). A c++_static build of this artifact
// would sidestep ALL of those hand-shims entirely and test a completely
// different code path - less isolated from "does the real game's runtime
// dependency surface work," not more. Matching libapp.so's own linkage
// means this test exercises the EXACT SAME shim functions the real crash
// investigation already spent most of 2026-09-16 on (Shim_locale_ctor,
// Shim_use_facet, Shim_ctype_char_do_widen, Shim_ios_base_init) - if the
// bug lives in one of those, this test should reproduce it; if it doesn't,
// that's real evidence the bug is specific to something in libapp.so's own
// state/control flow instead.
//
// GuestEngine::LoadSecondaryImage resolves every undefined symbol through
// the SAME ResolveOrCreateImportStub/RegisterImportShim table libapp.so's
// own imports already use (see that function's own comment,
// guest_engine.cpp) - so this artifact needs no special-casing on the
// engine side beyond what already exists, with one confirmed exception:
// operator new/delete (_Znwj/_ZdlPv et al.) had NO shim registered anywhere
// in this codebase before this test - grepped the whole emu/ tree and came
// up empty. Not a pre-existing bug: libapp.so's own build apparently
// defines ITS OWN operator new/delete locally (a common AAA-engine pattern,
// pooled/custom allocators overriding the global operator) - a LOCALLY
// DEFINED symbol never touches ResolveOrCreateImportStub at all, so this
// engine never had to shim it before. A plain artifact like this one that
// does NOT override global operator new/delete needs the real ones, added
// to rtti_shims.cpp (Shim_operator_new/Shim_operator_delete) specifically
// to make this test possible - see that file's own comment.
#include <sstream>
#include <cstring>
#include <cstdint>
extern "C" __attribute__((visibility("default")))
int TestOstreamAssembly(char* outBuf, int outBufSize) {
// Matches the real game's own pattern (sub_4702D8): several SEPARATE
// writes via operator<< before one .str() extraction, not a single
// combined write - see this session's investigation notes on why that
// distinction might matter (a single write could mask a bug that only
// shows up across multiple overflow()/sputn() calls into the same
// streambuf). Content itself doesn't matter - it's deliberately
// boilerplate-shaped (comment lines + a function skeleton) to loosely
// mirror the real shader source text without claiming to BE a shader.
std::ostringstream oss;
oss << "//FRAGMENT SHADER\n";
oss << "//===========\n\n";
oss << "void main()\n{\n";
oss << "}\n";
std::string result = oss.str();
int32_t len = (int32_t)result.size();
if (outBuf && outBufSize >= 4) {
memcpy(outBuf, &len, sizeof(len));
int avail = outBufSize - 4;
int copyLen = (int)result.size();
if (copyLen > avail) copyLen = avail;
if (copyLen > 0) memcpy(outBuf + 4, result.data(), (size_t)copyLen);
if (copyLen < avail) outBuf[4 + copyLen] = 0; // NUL-terminate for easy logging, if room
}
return len;
}
// 2026-09-17, ARM64_TRANSLATION_LAYER.md - the 0x3d3d3d3d heap-overflow
// chase. TestOstreamAssembly's own pattern (above) uses 4 short, similarly-
// sized writes with no intervening function call - none of them alone force
// an IMMEDIATE SSO->heap transition, and there's no real CALL boundary
// between writes into the SAME stream. The real crash's own sequence
// (sub_46FD58) is different in both respects: its FIRST write is
// "//VERTEX SHADER\n//=============\n\n" (33 bytes - by itself already past
// libc++'s ~22-byte SSO capacity, forcing an immediate heap transition on
// the very first write), immediately followed by a call into a SEPARATE
// function (sub_4711C8) that writes MORE content ("//Attributes\n
// //==========\n", 26 bytes) into the SAME shared stream object, passed by
// pointer across that real call boundary. Reproducing that exact shape here
// - noinline to guarantee a genuine function-call boundary, not something
// the compiler could inline away - to test whether THIS specific pattern
// (not just "several small writes in one function") is what's needed to
// expose the bug under GuestEngine.
extern "C" __attribute__((noinline))
void WriteAttributesSectionNested(std::ostringstream& oss) {
oss << "//Attributes\n//==========\n";
}
extern "C" __attribute__((visibility("default")))
int TestOstreamAssemblyNested(char* outBuf, int outBufSize) {
std::ostringstream oss;
oss << "//VERTEX SHADER\n//=============\n\n";
WriteAttributesSectionNested(oss);
std::string result = oss.str();
int32_t len = (int32_t)result.size();
if (outBuf && outBufSize >= 4) {
memcpy(outBuf, &len, sizeof(len));
int avail = outBufSize - 4;
int copyLen = (int)result.size();
if (copyLen > avail) copyLen = avail;
if (copyLen > 0) memcpy(outBuf + 4, result.data(), (size_t)copyLen);
if (copyLen < avail) outBuf[4 + copyLen] = 0;
}
return len;
}
ostream_repro/ostream_stress.cpp
+79
View File
@@ -0,0 +1,79 @@
// Standalone native ARM32 EXECUTABLE (not a .so, unlike ostream_repro.cpp's
// own SHARED library target - see this directory's CMakeLists.txt) for the
// 2026-09-17 "does real hardware ever see runaway basic_stringbuf::overflow()
// growth" check (ARM64_TRANSLATION_LAYER.md). Built c++_static (same as
// ostream_repro.cpp's own build.sh switched to - real, compiled
// basic_string<char>/basic_stringbuf<char> logic baked directly into this
// binary's own .text, not external imports), meant to run DIRECTLY on the
// Galaxy A9 via `adb shell` - no Unicorn, no GuestEngine, no shims of any
// kind involved. Pure ground truth: does the SAME growth pattern observed
// under GuestEngine (push_back-driven capacity doubling via
// basic_stringbuf<char>::overflow(), real disasm at sub_2700E4/sub_27036C/
// sub_27003C) ever get stuck, or does it complete normally no matter how far
// it's pushed?
#include <cstdio>
#include <cstdint>
#include <sstream>
#include <string>
#include <chrono>
using Clock = std::chrono::steady_clock;
int main(int argc, char** argv) {
// Default: push well past the ~8MB (0x800000) point where GuestEngine's
// own trace showed the crash - if real hardware sails through this same
// magnitude with no trouble, that's strong evidence the growth mechanism
// itself is fine and the bug is specific to GuestEngine (its own malloc
// shim, or something else in its translation of these particular
// instructions), not a genuine bug in the shipped game/libc++ pairing.
long targetChars = (argc > 1) ? atol(argv[1]) : 20L * 1024 * 1024; // 20M
printf("ostream_stress: appending %ld chars one at a time via push_back\n", targetChars);
fflush(stdout);
auto t0 = Clock::now();
std::string s;
for (long i = 0; i < targetChars; i++) {
s.push_back((char)('a' + (i % 26)));
// Progress heartbeat every 1M chars, and explicitly flag whenever
// capacity crosses the same doubling milestones GuestEngine's own
// register trace captured (0xfffff, 0x1fffff, 0x3fffff, 0x7fffff) -
// lets a hang be diagnosed by "last milestone reached" even if the
// process needs to be killed rather than exiting cleanly.
if (i != 0 && (i % (1L * 1024 * 1024)) == 0) {
auto elapsedMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t0).count();
printf(" [%ldms] i=%ld cap=%zu size=%zu\n", (long)elapsedMs, i, s.capacity(), s.size());
fflush(stdout);
}
}
auto totalMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t0).count();
printf("ostream_stress: push_back loop done in %ldms - final size=%zu cap=%zu\n",
(long)totalMs, s.size(), s.capacity());
fflush(stdout);
// Second phase: the ACTUAL crashing pattern is via basic_stringbuf's own
// xsputn/overflow chain (operator<<), not raw std::string::push_back -
// exercise that path too, via repeated small ostringstream writes,
// mirroring sub_4702D8's own "several separate writes before one
// extraction" shape but looped enough times to force the same magnitude
// of reallocation.
printf("ostream_stress: now stress-testing via ostringstream operator<<\n");
fflush(stdout);
auto t1 = Clock::now();
std::ostringstream oss;
long chunkCount = targetChars / 16; // ~16 chars per write, same order of magnitude
for (long i = 0; i < chunkCount; i++) {
oss << "0123456789ABCDEF";
if (i != 0 && (i % (65536)) == 0) {
auto elapsedMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t1).count();
std::string cur = oss.str();
printf(" [oss %ldms] i=%ld size-so-far=%zu\n", (long)elapsedMs, i, cur.size());
fflush(stdout);
}
}
std::string result = oss.str();
auto ossMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t1).count();
printf("ostream_stress: ostringstream loop done in %ldms - final size=%zu\n",
(long)ossMs, result.size());
fflush(stdout);
return 0;
}
trace_agent/CMakeLists.txt
+8
View File
@@ -31,3 +31,11 @@ target_link_libraries(trace_agent
)
target_compile_options(trace_agent PRIVATE -Wall -Wno-unused-parameter)
# __dynamic_cast is also defined in libc++abi.a, which the NDK links
# statically into this agent. Interposing it (task #58 - counting the real
# call rate on native hardware) therefore produces a duplicate-symbol link
# error. Our definition comes from this target's own objects, which the
# linker sees before the archive, so it wins; the archive copy is simply
# left unused. Scoped to this target rather than set globally.
target_link_options(trace_agent PRIVATE "-Wl,--allow-multiple-definition")
trace_agent/libc_gles_trace.cpp
+84
View File
@@ -13,6 +13,7 @@
#include <fcntl.h>
#include <atomic>
#include <cstdarg>
#include <cstddef>
#include <cstdint>
#include <cstdio>
#include <cstring>
@@ -564,4 +565,87 @@ void glLinkProgram(GLuint program) {
}
}
// ---- FMOD bring-up trace (2026-09-21, task #67) ----
//
// The emulated engine now loads the game's REAL libfmodex/libfmodevent and
// runs them, but FMOD never reaches output initialisation - Shim_dlopen logs
// every call and there is not one, so `libOpenSLES.so` is never opened. The
// question that cannot be answered by staring at our side: what does this
// sequence look like on real ARM32 hardware, where sound works?
//
// The user's observation that audio starts right after the EA logo says the
// chain runs early, so these four points should all appear near the start of a
// native trace. Each logs its FMOD_RESULT (0 == FMOD_OK), which names the
// failure directly if one of them is where the two runs diverge.
//
// Per-call logging is fine here, unlike __dynamic_cast below: these are
// one-shot initialisation calls, not a quarter million per second.
// NOTE: do NOT interpose dlopen here. The first attempt did, and it killed the
// process before the game even started: the Android runtime dlopen()s
// libart.so during startup, our wrapper could not resolve the real symbol that
// early (RealSym goes through dlsym, which is not usable from a dlopen
// interposer at that point), so it returned NULL and the runtime died on the
// null handle - "Failed to dlopen libart.so", then SIGSEGV at address 0,
// "wrap.sh terminated by signal 11". Exactly the hazard this file already
// documents for pthread_once.
//
// It is also unnecessary: our own side already logs every dlopen through
// Shim_dlopen. What the native run has to answer is where FMOD's INIT chain
// goes, and the three interposers below do that without touching the loader.
// FMOD interposition was tried here on 2026-09-21 and REMOVED. Two failures,
// both worth keeping as a warning:
//
// 1. Interposing dlopen killed the process at startup - the runtime's own
// dlopen("libart.so") got our wrapper before it could resolve the real
// symbol, returned NULL, and the app died on a null handle.
// 2. libfmodex/libfmodevent live in the APP's linker namespace, which an
// LD_PRELOAD'd agent cannot reach - neither dlsym(RTLD_NEXT) nor
// dlopen(RTLD_NOLOAD) found a single FMOD symbol. The wrapper therefore
// always took its fallback path, and that fallback REPLACED FMOD's
// initialisation with a stub - silencing audio on the very device that
// was supposed to serve as the working reference.
//
// The lesson is the measurement one: an instrument that cannot do the real
// work must not stand in for it. Whether the native game reaches OpenSL is
// answerable from OUTSIDE the process entirely, by looking for libOpenSLES.so
// in /proc/<pid>/maps - no injection, no interference, no way to break what is
// being measured.
// ---- __dynamic_cast rate (2026-09-19, task #58) ----
// The emulated engine's own per-shim counter found __dynamic_cast making up
// 49% of ALL shim crossings during a prologue load - 284,986 calls/sec. The
// obvious question that number does NOT answer: is that the game's own
// behaviour, or something this engine induces? libapp.so imports
// __dynamic_cast as an undefined symbol, so LD_PRELOAD can count the real
// thing on native ARM32 hardware and settle it.
//
// Reports a rate once a second rather than logging each call: at a quarter
// million per second, per-call logging would dominate the measurement (and
// this agent writes to a file, see InitFileLog). Deliberately mirrors the
// engine's TOPSHIMS accounting so the two numbers are directly comparable.
void* __dynamic_cast(const void* sub, const void* src, const void* dst, ptrdiff_t src2dst) {
static auto real = RealSym<void* (*)(const void*, const void*, const void*, ptrdiff_t)>(
"__dynamic_cast");
static std::atomic<uint64_t> calls{0};
static std::atomic<uint64_t> lastNs{0};
static uint64_t prevCalls = 0;
uint64_t n = calls.fetch_add(1, std::memory_order_relaxed) + 1;
struct timespec ts;
clock_gettime(CLOCK_MONOTONIC, &ts);
uint64_t now = (uint64_t)ts.tv_sec * 1000000000ull + (uint64_t)ts.tv_nsec;
uint64_t last = lastNs.load(std::memory_order_relaxed);
if (now - last >= 1000000000ull &&
lastNs.compare_exchange_strong(last, now, std::memory_order_relaxed) && last != 0) {
double dt = (now - last) / 1e9;
LOGI("DYNCAST native rate=%.0f/s (total %llu)", (double)(n - prevCalls) / dt,
(unsigned long long)n);
prevCalls = n;
}
return real(sub, src, dst, src2dst);
}
} // extern "C"