megboyzz/nfsmw-online
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Map the guest region by permission, not all-RWX: loading 8.3% faster (tasks #53/#54)

Browse Source 
CreateConfiguredEngine mapped the whole guest region with one
uc_mem_map_ptr(UC_PROT_ALL). UC_PROT_ALL includes EXEC, and QEMU's
notdirty_write (qemu/accel/tcg/cputlb.c:1199) does this on every write:

    mr = uc->memory_mapping(uc, ...);              // region lookup
    if (mr && (mr->perms & UC_PROT_EXEC) != 0) {
        page_collection_lock(...);
        tb_invalidate_phys_page_fast(...);
    }

So marking memory executable makes EVERY WRITE to it pay a region lookup
plus a translated-block invalidation check. With the whole region
executable, ordinary stack and data writes all took that path.

Found by elimination rather than guesswork. Two counters were written and
both answered zero: writes into .text (across two sessions) and writes into
the trampoline/stub arenas. Since no guest code and no engine-written stub
bytes were dirtying code pages, the writes driving tb_invalidate had to be
ordinary data writes that merely lived in an executable mapping.

The first split (heap RW, everything above heap_end_ RWX) moved the three
symbols by 0.5 points - i.e. nothing - because it left THREAD STACKS
executable. The heap is 768MB, but stores go overwhelmingly to the stack,
once per call frame. "Biggest region" and "most written region" were not the
same region.

Final layout: image RWX; heap RW; trampoline+stub arenas RWX; control and
thread stacks RW; mmap arena RWX (LoadSecondaryImage maps real code there).

    tb_invalidate_phys_page_fast_arm   4.52%  ->  0.04%
    time to first OnCarLoaded, 3 runs each:
        single RWX  40.95 / 39.06 / 38.96 s   (mean 39.66)
        split       36.31 / 35.87 / 36.88 s   (mean 36.35)

Ranges do not overlap: loading is 8.3% faster. Verified end-to-end with an
A/B switch rather than inferred from the profile share, because task #48
established that a share drop is not a speedup. Zero FETCH_PROT/WRITE_PROT
faults, so nothing executes from the now-non-executable spans.

flatview_translate_arm (5.25%) and find_memory_mapping_arm (2.99%) barely
moved, so they have callers beyond notdirty_write - task #53 stays open.

Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
megboyzz
2026-09-19 17:25:01 +03:00
co-authored by Claude
parent 7b15d85f61
commit edfa360c0a
1 changed files with 91 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
mpcore/src/main/cpp/emu/guest_engine.cpp
+91 -5
View File
@@ -135,6 +135,15 @@ constexpr uint32_t kGuardPageSize = kPageSize;
// is dead.
constexpr bool kCountStubDispatches = false;
constexpr bool kCountTextWrites = false;
// kCountArenaWrites - task #54. tb_invalidate_phys_page_fast_arm is 5.56% of
// the in-race profile and its cause is open: the .text write counter above
// read a FLAT ZERO across two sessions, so guest self-modifying code is
// ruled out. QEMU invalidates on a write to a page that holds translated
// code, so the remaining candidates are the arenas that DO contain executed
// code - the trampoline pool and the two stub arenas, all of which get real
// instruction bytes written into them (AllocCodeStub writes a Thumb BX LR,
// InstallTrampolineHookRaw writes its own). Counting before concluding.
constexpr bool kCountArenaWrites = false;
// libapp.so .text: offset 0x68000, size 0x916150 (readelf -S).
constexpr uint64_t kTextStart = 0x68000u;
constexpr uint64_t kTextEnd = 0x68000u + 0x916150u;
@@ -2888,11 +2897,78 @@ uc_engine* GuestEngine::CreateConfiguredEngine() {
// G2H/H2G plain pointer arithmetic (see guest_engine.h's class comment).
// All engines (one per host thread) map the SAME host_region_, exactly
// mirroring how real OS threads share one process's memory.
err = uc_mem_map_ptr(newUc, 0, region_size_, UC_PROT_ALL, host_region_);
if (err != UC_ERR_OK) {
Log("GuestEngine::CreateConfiguredEngine: uc_mem_map_ptr(0, 0x%x) failed: %d", region_size_, (int)err);
uc_close(newUc);
return nullptr;
//
// FIX (2026-09-19, task #54/#53). This used to be ONE uc_mem_map_ptr over
// the whole region with UC_PROT_ALL - which includes EXEC. QEMU's
// notdirty_write (qemu/accel/tcg/cputlb.c:1199) does this on a write:
// mr = uc->memory_mapping(uc, ...); // region lookup
// if (mr && (mr->perms & UC_PROT_EXEC) != 0) {
// page_collection_lock(...);
// tb_invalidate_phys_page_fast(...);
// }
// So marking a page executable means EVERY WRITE to it pays a region
// lookup plus a translated-block invalidation check. With the whole
// region mapped executable, ordinary heap, stack and .data writes - the
// overwhelming majority of all guest stores - were all taking that path.
// It accounts for three of the top symbols in the in-race profile at
// once: tb_invalidate_phys_page_fast_arm 5.56% + flatview_translate_arm
// 5.00% + find_memory_mapping_arm 2.90% = 13.5%.
//
// Found by elimination, not guesswork: a write counter over .text read a
// flat zero across two sessions, and a second counter over the trampoline
// and stub arenas also read zero - so no guest code and no engine-written
// stub bytes were dirtying code pages. The writes triggering this were
// ordinary data writes that merely happened to live in an
// executable-mapped region.
//
// Now mapped in three spans, EXEC only where instructions genuinely
// execute:
// [0, image_end_) RWX - real .text/.rodata/.data
// [image_end_, heap_end_) RW - guest heap, never executed
// [heap_end_, misc_stub_end_) RWX - trampoline + import-stub +
// misc-stub arenas, which do
// run real instructions
// [misc_stub_end_, thread_stacks_end_) RW - control arena + THREAD
// STACKS
// [thread_stacks_end_, region_size_) RWX - mmap arena; must stay
// executable because
// LoadSecondaryImage maps
// real code there via
// AllocMmap
//
// A first attempt split only at heap_end_, leaving stacks executable, and
// moved the three symbols by 0.5 points (13.5% -> 13.0%) - i.e. nothing.
// That is the measurement that produced this layout: the heap is 768MB
// but stores go overwhelmingly to the STACK, once per call frame, so
// leaving the stacks executable kept the entire cost. Recorded because
// "the biggest region" and "the most written region" turned out to be
// different regions.
//
// A wrong guess here fails LOUDLY (FETCH_PROT via mem_fault_hook_cb)
// rather than silently, which is what makes this safe to iterate on.
// A/B measured before keeping this, since a profile-share drop is not a
// speedup (task #48 taught that the hard way). Time from engine start to
// the first OnCarLoaded, three runs each, Pixel 6a:
// single RWX mapping : 40.95 / 39.06 / 38.96 s (mean 39.66)
// split by permission: 36.31 / 35.87 / 36.88 s (mean 36.35)
// Ranges do not overlap - loading is 8.3% faster.
struct SpanDef { GuestAddr base; uint32_t size; uint32_t perms; const char* what; };
SpanDef spans[] = {
{0, image_end_, UC_PROT_ALL, "image (RWX)"},
{image_end_, heap_end_ - image_end_, UC_PROT_READ | UC_PROT_WRITE, "heap (RW)"},
{heap_end_, misc_stub_end_ - heap_end_, UC_PROT_ALL, "trampoline+stubs (RWX)"},
{misc_stub_end_, thread_stacks_end_ - misc_stub_end_, UC_PROT_READ | UC_PROT_WRITE, "control+stacks (RW)"},
{thread_stacks_end_, region_size_ - thread_stacks_end_, UC_PROT_ALL, "mmap arena (RWX)"},
};
for (auto& s : spans) {
if (s.size == 0) continue;
err = uc_mem_map_ptr(newUc, s.base, s.size, s.perms, host_region_ + s.base);
if (err != UC_ERR_OK) {
Log("GuestEngine::CreateConfiguredEngine: uc_mem_map_ptr(%s @0x%x, 0x%x) failed: %d",
s.what, s.base, s.size, (int)err);
uc_close(newUc);
return nullptr;
}
}
// Guard pages (see kGuardPageSize's own comment and MapSegments' arena
@@ -2991,6 +3067,16 @@ uc_engine* GuestEngine::CreateConfiguredEngine() {
kTextStart, kTextEnd);
}
// See kCountArenaWrites. Covers trampoline + import-stub + misc-stub
// arenas in one range (they sit contiguously above heap_end_, separated
// only by guard pages), which is every region this engine writes real
// instruction bytes into.
if (kCountArenaWrites) {
uc_hook arenaWriteHook;
uc_hook_add(newUc, &arenaWriteHook, UC_HOOK_MEM_WRITE, (void*)TextWriteProbeHookCb, nullptr,
heap_end_, misc_stub_end_);
}
// Throwaway diagnostic (see profiler.h) - no-op unless EnableProfiling()
// was called. Covers the loaded image's own real code range only
// (arenas beyond image_end_ are stub addresses that redirect out via