arm64: flat guest mapping, audio, crash reporting, self-extracting data
The run of work that took the translated build from "boots" to "playable". Engine: - Flat guest mapping replaces the software MMU on aarch64 hosts. A 4 GiB PROT_NONE reservation lets a 32-bit guest address zero-extend safely, so tcg_out_qemu_ld/st short-circuit before tcg_out_tlb_read and the prologue materialises the base into X28. Measured 1.24x (51 vs 41 fps, interleaved A/B). Note the base must be set BEFORE UC_INIT - Unicorn inits lazily, and setting it after uc_open silently falls back to softmmu. - num_get<char> facet implemented, which was the real cause of the crash after the prologue race; a full race is now playable end to end. - Thread-stack free list + ReleaseThreadEngine, fixing the arena leak that showed up as a black screen when entering a race. kMaxGuestThreads 16 -> 64. - Real ARM32 FMOD now runs in-engine via the Java FMODAudioDevice bridge, with a per-thread JNIEnv. Two of the three blockers were our own single-image-era guards. Host/app: - Native crash handler: async-signal-safe, decodes the host fault back to a guest address, writes a report file and nothing else. CrashReportActivity picks it up on the NEXT launch, zips it, and offers to share. No backend, no automatic upload. - Game data ships inside the APK and self-extracts on first launch, so a tester installs one file and plays. Copy-to-.part-then-rename, with a free-space check up front. - EGL context preserved across pause, fixing black textures on resume. - Navigation bar hidden and re-hidden on focus gain; volume keys reported as system keys, checked before the loading-state gate. - x86_64 added to abiFilters: the ARM32 guest runs under tcg/i386 with no houdini in the path. The flat mapping is aarch64-only, so that host falls back to the software MMU - commented at the abiFilters line. Ignore rules added for app/translated/ (611 MB of signed release APK, which also carries the bundled OBB) and ostream_repro/build/. Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
Claude
@@ -0,0 +1,26 @@
|
||||
# Standalone armeabi-v7a artifact for the 2026-09-16 "isolated
|
||||
# std::ostringstream repro" test (see ARM64_TRANSLATION_LAYER.md and
|
||||
# ostream_repro.cpp's own top comment). Same pattern as ../trace_agent's own
|
||||
# CMakeLists.txt - deliberately NOT wired into the main Gradle build
|
||||
# (settings.gradle.kts), built and pushed to the device independently via
|
||||
# build.sh. Default STL linkage (c++_shared, the NDK CMake toolchain's own
|
||||
# default - NOT overridden to c++_static here) is a deliberate choice, not
|
||||
# an oversight - see ostream_repro.cpp's top comment for why matching
|
||||
# libapp.so's own dynamic-libc++ linkage matters for this specific test.
|
||||
cmake_minimum_required(VERSION 3.22.1)
|
||||
project(ostream_repro)
|
||||
|
||||
add_library(ostream_repro SHARED
|
||||
ostream_repro.cpp
|
||||
)
|
||||
|
||||
target_compile_options(ostream_repro PRIVATE -Wall -Wno-unused-parameter)
|
||||
|
||||
# Standalone native executable (2026-09-17, ARM64_TRANSLATION_LAYER.md - "does
|
||||
# real hardware ever see runaway basic_stringbuf::overflow() growth" check).
|
||||
# See ostream_stress.cpp's own top comment. Independent target, does not
|
||||
# affect the ostream_repro library above.
|
||||
add_executable(ostream_stress
|
||||
ostream_stress.cpp
|
||||
)
|
||||
target_compile_options(ostream_stress PRIVATE -Wall -Wno-unused-parameter)
|
||||
Executable
+45
@@ -0,0 +1,45 @@
|
||||
#!/usr/bin/env bash
|
||||
# Builds libostream_repro.so for armeabi-v7a via the NDK's CMake toolchain
|
||||
# file - standalone, deliberately outside the Gradle build, same pattern as
|
||||
# ../trace_agent/build.sh. Output: build/libostream_repro.so, ready to push
|
||||
# to a device for GuestEngine::LoadSecondaryImage to load (see
|
||||
# mpcore/src/main/cpp/emu/ostream_repro_test.cpp).
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")"
|
||||
|
||||
NDK="${ANDROID_NDK_HOME:-/home/megboyzz/Android/Sdk/ndk/27.0.12077973}"
|
||||
TOOLCHAIN="$NDK/build/cmake/android.toolchain.cmake"
|
||||
|
||||
if [ ! -f "$TOOLCHAIN" ]; then
|
||||
echo "NDK toolchain file not found at $TOOLCHAIN - set ANDROID_NDK_HOME" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ANDROID_STL=c++_static explicitly - see ostream_repro.cpp's own top
|
||||
# comment for the full reasoning (tried c++_shared FIRST, actually built
|
||||
# and inspected both outcomes with llvm-readelf before deciding, not
|
||||
# guessed): the modern NDK r27 toolchain's libc++ headers extern-template-
|
||||
# declare basic_stringbuf<char>/basic_ostream<char>/basic_ios<char>/
|
||||
# basic_ostringstream<char> (their vtables AND ctors/dtors become UNDEFINED
|
||||
# imports resolved against libc++_shared.so, confirmed via
|
||||
# `llvm-readelf --dyn-syms` on a real c++_shared build of this exact file),
|
||||
# which GuestEngine has zero shims for - that combination would make this
|
||||
# test fail for an uninteresting, unrelated reason (unimplemented vtable
|
||||
# stub returning 0) instead of actually exercising real, compiled
|
||||
# std::basic_stringbuf<char>::str()/write() logic. c++_static compiles that
|
||||
# logic directly into THIS .so's own .text - both a working build AND a
|
||||
# closer functional match to what libapp.so's own (much older) toolchain
|
||||
# evidently did for sub_79CD4/sub_27160C (confirmed real, compiled,
|
||||
# fixed-address code inside libapp.so itself, never external imports -
|
||||
# see ARM64_TRANSLATION_LAYER.md's 2026-09-16 entries).
|
||||
cmake -B build -G Ninja \
|
||||
-DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN" \
|
||||
-DANDROID_ABI=armeabi-v7a \
|
||||
-DANDROID_PLATFORM=android-27 \
|
||||
-DANDROID_STL=c++_static \
|
||||
-DCMAKE_BUILD_TYPE=Debug \
|
||||
.
|
||||
|
||||
cmake --build build
|
||||
|
||||
echo "Built: $(pwd)/build/libostream_repro.so"
|
||||
@@ -0,0 +1,132 @@
|
||||
// Standalone, minimal armeabi-v7a artifact for the 2026-09-16
|
||||
// ARM64_TRANSLATION_LAYER.md "isolated std::ostringstream repro" test - see
|
||||
// that doc's entry of the same date for the full methodology and the
|
||||
// GuestEngine-side driver (mpcore/src/main/cpp/emu/ostream_repro_test.cpp)
|
||||
// that loads and calls this.
|
||||
//
|
||||
// Why this exists: the real game's own libapp.so, deep inside its shader-
|
||||
// variant builder (sub_4702D8, per ARM64_TRANSLATION_LAYER.md's 2026-09-16
|
||||
// entries), writes several string literals to a real, compiled
|
||||
// std::ostringstream-shaped object via operator<< and then extracts the
|
||||
// accumulated text via what's effectively basic_stringbuf<char>::str() -
|
||||
// and the extraction always comes back empty under GuestEngine, even though
|
||||
// register/memory probes confirmed real content was genuinely written
|
||||
// first. Rather than keep chasing that one binary's own hardcoded
|
||||
// addresses (explicitly the wrong approach per the user's own direction -
|
||||
// "ты сейчас пытаешься подогнать эмуляцию к одному единственному бинарнику,
|
||||
// это не правильный подход" - that only tells us about THIS game, not
|
||||
// whether GuestEngine itself has a real, general ostringstream-extraction
|
||||
// bug), this reproduces the EXACT SAME write-then-extract shape in total
|
||||
// isolation from every other line of game code, compiled fresh by the same
|
||||
// NDK toolchain and loaded as its own tiny ELF image
|
||||
// (GuestEngine::LoadSecondaryImage) alongside (not instead of) libapp.so.
|
||||
//
|
||||
// Deliberately built the SAME way libapp.so itself is (default NDK
|
||||
// ANDROID_STL=c++_shared, not a static-libstdc++/c++_static build) rather
|
||||
// than statically linking libc++ into this .so: rtti_shims.cpp's own
|
||||
// RegisterRttiImportShims already shows GuestEngine hand-reimplements a
|
||||
// SPECIFIC set of libc++-internal symbols (locale/ios_base/ctype<char>/
|
||||
// num_put<char>/__shared_weak_count/std::mutex - all undefined imports in
|
||||
// libapp.so's own .dynsym, meaning libapp.so links these dynamically
|
||||
// against libc++_shared.so too, while template-heavy code like
|
||||
// basic_stringbuf<char>'s own methods gets compiled directly into
|
||||
// libapp.so's .text, same as here). A c++_static build of this artifact
|
||||
// would sidestep ALL of those hand-shims entirely and test a completely
|
||||
// different code path - less isolated from "does the real game's runtime
|
||||
// dependency surface work," not more. Matching libapp.so's own linkage
|
||||
// means this test exercises the EXACT SAME shim functions the real crash
|
||||
// investigation already spent most of 2026-09-16 on (Shim_locale_ctor,
|
||||
// Shim_use_facet, Shim_ctype_char_do_widen, Shim_ios_base_init) - if the
|
||||
// bug lives in one of those, this test should reproduce it; if it doesn't,
|
||||
// that's real evidence the bug is specific to something in libapp.so's own
|
||||
// state/control flow instead.
|
||||
//
|
||||
// GuestEngine::LoadSecondaryImage resolves every undefined symbol through
|
||||
// the SAME ResolveOrCreateImportStub/RegisterImportShim table libapp.so's
|
||||
// own imports already use (see that function's own comment,
|
||||
// guest_engine.cpp) - so this artifact needs no special-casing on the
|
||||
// engine side beyond what already exists, with one confirmed exception:
|
||||
// operator new/delete (_Znwj/_ZdlPv et al.) had NO shim registered anywhere
|
||||
// in this codebase before this test - grepped the whole emu/ tree and came
|
||||
// up empty. Not a pre-existing bug: libapp.so's own build apparently
|
||||
// defines ITS OWN operator new/delete locally (a common AAA-engine pattern,
|
||||
// pooled/custom allocators overriding the global operator) - a LOCALLY
|
||||
// DEFINED symbol never touches ResolveOrCreateImportStub at all, so this
|
||||
// engine never had to shim it before. A plain artifact like this one that
|
||||
// does NOT override global operator new/delete needs the real ones, added
|
||||
// to rtti_shims.cpp (Shim_operator_new/Shim_operator_delete) specifically
|
||||
// to make this test possible - see that file's own comment.
|
||||
#include <sstream>
|
||||
#include <cstring>
|
||||
#include <cstdint>
|
||||
|
||||
extern "C" __attribute__((visibility("default")))
|
||||
int TestOstreamAssembly(char* outBuf, int outBufSize) {
|
||||
// Matches the real game's own pattern (sub_4702D8): several SEPARATE
|
||||
// writes via operator<< before one .str() extraction, not a single
|
||||
// combined write - see this session's investigation notes on why that
|
||||
// distinction might matter (a single write could mask a bug that only
|
||||
// shows up across multiple overflow()/sputn() calls into the same
|
||||
// streambuf). Content itself doesn't matter - it's deliberately
|
||||
// boilerplate-shaped (comment lines + a function skeleton) to loosely
|
||||
// mirror the real shader source text without claiming to BE a shader.
|
||||
std::ostringstream oss;
|
||||
oss << "//FRAGMENT SHADER\n";
|
||||
oss << "//===========\n\n";
|
||||
oss << "void main()\n{\n";
|
||||
oss << "}\n";
|
||||
|
||||
std::string result = oss.str();
|
||||
|
||||
int32_t len = (int32_t)result.size();
|
||||
if (outBuf && outBufSize >= 4) {
|
||||
memcpy(outBuf, &len, sizeof(len));
|
||||
int avail = outBufSize - 4;
|
||||
int copyLen = (int)result.size();
|
||||
if (copyLen > avail) copyLen = avail;
|
||||
if (copyLen > 0) memcpy(outBuf + 4, result.data(), (size_t)copyLen);
|
||||
if (copyLen < avail) outBuf[4 + copyLen] = 0; // NUL-terminate for easy logging, if room
|
||||
}
|
||||
return len;
|
||||
}
|
||||
|
||||
// 2026-09-17, ARM64_TRANSLATION_LAYER.md - the 0x3d3d3d3d heap-overflow
|
||||
// chase. TestOstreamAssembly's own pattern (above) uses 4 short, similarly-
|
||||
// sized writes with no intervening function call - none of them alone force
|
||||
// an IMMEDIATE SSO->heap transition, and there's no real CALL boundary
|
||||
// between writes into the SAME stream. The real crash's own sequence
|
||||
// (sub_46FD58) is different in both respects: its FIRST write is
|
||||
// "//VERTEX SHADER\n//=============\n\n" (33 bytes - by itself already past
|
||||
// libc++'s ~22-byte SSO capacity, forcing an immediate heap transition on
|
||||
// the very first write), immediately followed by a call into a SEPARATE
|
||||
// function (sub_4711C8) that writes MORE content ("//Attributes\n
|
||||
// //==========\n", 26 bytes) into the SAME shared stream object, passed by
|
||||
// pointer across that real call boundary. Reproducing that exact shape here
|
||||
// - noinline to guarantee a genuine function-call boundary, not something
|
||||
// the compiler could inline away - to test whether THIS specific pattern
|
||||
// (not just "several small writes in one function") is what's needed to
|
||||
// expose the bug under GuestEngine.
|
||||
extern "C" __attribute__((noinline))
|
||||
void WriteAttributesSectionNested(std::ostringstream& oss) {
|
||||
oss << "//Attributes\n//==========\n";
|
||||
}
|
||||
|
||||
extern "C" __attribute__((visibility("default")))
|
||||
int TestOstreamAssemblyNested(char* outBuf, int outBufSize) {
|
||||
std::ostringstream oss;
|
||||
oss << "//VERTEX SHADER\n//=============\n\n";
|
||||
WriteAttributesSectionNested(oss);
|
||||
|
||||
std::string result = oss.str();
|
||||
|
||||
int32_t len = (int32_t)result.size();
|
||||
if (outBuf && outBufSize >= 4) {
|
||||
memcpy(outBuf, &len, sizeof(len));
|
||||
int avail = outBufSize - 4;
|
||||
int copyLen = (int)result.size();
|
||||
if (copyLen > avail) copyLen = avail;
|
||||
if (copyLen > 0) memcpy(outBuf + 4, result.data(), (size_t)copyLen);
|
||||
if (copyLen < avail) outBuf[4 + copyLen] = 0;
|
||||
}
|
||||
return len;
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
// Standalone native ARM32 EXECUTABLE (not a .so, unlike ostream_repro.cpp's
|
||||
// own SHARED library target - see this directory's CMakeLists.txt) for the
|
||||
// 2026-09-17 "does real hardware ever see runaway basic_stringbuf::overflow()
|
||||
// growth" check (ARM64_TRANSLATION_LAYER.md). Built c++_static (same as
|
||||
// ostream_repro.cpp's own build.sh switched to - real, compiled
|
||||
// basic_string<char>/basic_stringbuf<char> logic baked directly into this
|
||||
// binary's own .text, not external imports), meant to run DIRECTLY on the
|
||||
// Galaxy A9 via `adb shell` - no Unicorn, no GuestEngine, no shims of any
|
||||
// kind involved. Pure ground truth: does the SAME growth pattern observed
|
||||
// under GuestEngine (push_back-driven capacity doubling via
|
||||
// basic_stringbuf<char>::overflow(), real disasm at sub_2700E4/sub_27036C/
|
||||
// sub_27003C) ever get stuck, or does it complete normally no matter how far
|
||||
// it's pushed?
|
||||
#include <cstdio>
|
||||
#include <cstdint>
|
||||
#include <sstream>
|
||||
#include <string>
|
||||
#include <chrono>
|
||||
|
||||
using Clock = std::chrono::steady_clock;
|
||||
|
||||
int main(int argc, char** argv) {
|
||||
// Default: push well past the ~8MB (0x800000) point where GuestEngine's
|
||||
// own trace showed the crash - if real hardware sails through this same
|
||||
// magnitude with no trouble, that's strong evidence the growth mechanism
|
||||
// itself is fine and the bug is specific to GuestEngine (its own malloc
|
||||
// shim, or something else in its translation of these particular
|
||||
// instructions), not a genuine bug in the shipped game/libc++ pairing.
|
||||
long targetChars = (argc > 1) ? atol(argv[1]) : 20L * 1024 * 1024; // 20M
|
||||
printf("ostream_stress: appending %ld chars one at a time via push_back\n", targetChars);
|
||||
fflush(stdout);
|
||||
|
||||
auto t0 = Clock::now();
|
||||
std::string s;
|
||||
for (long i = 0; i < targetChars; i++) {
|
||||
s.push_back((char)('a' + (i % 26)));
|
||||
// Progress heartbeat every 1M chars, and explicitly flag whenever
|
||||
// capacity crosses the same doubling milestones GuestEngine's own
|
||||
// register trace captured (0xfffff, 0x1fffff, 0x3fffff, 0x7fffff) -
|
||||
// lets a hang be diagnosed by "last milestone reached" even if the
|
||||
// process needs to be killed rather than exiting cleanly.
|
||||
if (i != 0 && (i % (1L * 1024 * 1024)) == 0) {
|
||||
auto elapsedMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t0).count();
|
||||
printf(" [%ldms] i=%ld cap=%zu size=%zu\n", (long)elapsedMs, i, s.capacity(), s.size());
|
||||
fflush(stdout);
|
||||
}
|
||||
}
|
||||
auto totalMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t0).count();
|
||||
printf("ostream_stress: push_back loop done in %ldms - final size=%zu cap=%zu\n",
|
||||
(long)totalMs, s.size(), s.capacity());
|
||||
fflush(stdout);
|
||||
|
||||
// Second phase: the ACTUAL crashing pattern is via basic_stringbuf's own
|
||||
// xsputn/overflow chain (operator<<), not raw std::string::push_back -
|
||||
// exercise that path too, via repeated small ostringstream writes,
|
||||
// mirroring sub_4702D8's own "several separate writes before one
|
||||
// extraction" shape but looped enough times to force the same magnitude
|
||||
// of reallocation.
|
||||
printf("ostream_stress: now stress-testing via ostringstream operator<<\n");
|
||||
fflush(stdout);
|
||||
auto t1 = Clock::now();
|
||||
std::ostringstream oss;
|
||||
long chunkCount = targetChars / 16; // ~16 chars per write, same order of magnitude
|
||||
for (long i = 0; i < chunkCount; i++) {
|
||||
oss << "0123456789ABCDEF";
|
||||
if (i != 0 && (i % (65536)) == 0) {
|
||||
auto elapsedMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t1).count();
|
||||
std::string cur = oss.str();
|
||||
printf(" [oss %ldms] i=%ld size-so-far=%zu\n", (long)elapsedMs, i, cur.size());
|
||||
fflush(stdout);
|
||||
}
|
||||
}
|
||||
std::string result = oss.str();
|
||||
auto ossMs = std::chrono::duration_cast<std::chrono::milliseconds>(Clock::now() - t1).count();
|
||||
printf("ostream_stress: ostringstream loop done in %ldms - final size=%zu\n",
|
||||
(long)ossMs, result.size());
|
||||
fflush(stdout);
|
||||
return 0;
|
||||
}
|
||||
Reference in New Issue
Block a user