Skip to content

feat: add read_process_memory_into for zero-copy buffer reuse (#71)#72

Merged
JeanExtreme002 merged 1 commit into
mainfrom
jeanextreme002/read-process-memory-into
Jun 15, 2026
Merged

feat: add read_process_memory_into for zero-copy buffer reuse (#71)#72
JeanExtreme002 merged 1 commit into
mainfrom
jeanextreme002/read-process-memory-into

Conversation

@JeanExtreme002

@JeanExtreme002 JeanExtreme002 commented Jun 15, 2026

Copy link
Copy Markdown
Owner

Summary

Closes #71.

read_process_memory allocates a fresh Python object on every call (a ctypes
buffer plus the returned bytes). In a long-running loop that polls the
same-sized region over and over — a recorder, a live overlay, a poller — those
throwaway objects create constant GC churn instead of constant memory use.

This adds read_process_memory_into(address, buffer) -> int, the zero-copy
counterpart of read_process_memory: it reads straight into a caller-owned,
reusable buffer with no intermediate allocation, and returns the number of
bytes read.

buffer = bytearray(4096)              # allocate once

with OpenProcess(name="game.exe") as process:
    while recording:
        process.read_process_memory_into(addr, buffer)
        handle(buffer)                # same buffer refilled in place — flat memory

How it works / why it fits the architecture

This is orthogonal to the existing optimizations, not a duplicate of them:

Mechanism What it saves
snapshot_memory_regions / memory_regions= region-map enumeration
search_by_addresses number of syscalls (reads each page once)
read_process_memory_into (this PR) per-read object allocation
  • Accepts any writable, contiguous buffer-protocol object — bytearray,
    ctypes array, writable memoryview, numpy array (sized in bytes, so a
    4-element int32 array reads 16 bytes). Bytes land verbatim (no decoding).
  • Buffer validation/wrapping lives in one shared helper,
    util.convert.as_writable_c_buffer, so there's no per-backend duplication.
  • Each backend's thin read_process_memory_into reuses its existing low-level
    read primitive (ReadProcessMemory / process_vm_readv /
    mach_vm_read_overwrite), including the same partial-read → OSError guard
    the regular read already enforces.

Errors

  • TypeError — buffer not writable (e.g. immutable bytes; use bytearray).
  • ValueError — buffer empty or not contiguous.
  • OSError — read failed or returned fewer bytes than requested.

Tests

New tests/memory/test_read_into.py (11 cases): bytearray / ctypes / memoryview
/ numpy targets, buffer reuse across reads, byte-length sizing, parity with
read_process_memory, and the three error paths.

  • Full suite green on macOS: 469 passed, 14 skipped.
  • The Windows and Linux backends mirror the macOS path exactly and compile
    cleanly, but were not executed locally — worth confirming via CI on those
    platforms.

Docs

  • New "Reusing a buffer (zero-copy reads)" section in docs/guide/read-write.md.
  • Method entry added to the API reference (docs/api/openprocess.md).

@github-actions github-actions Bot added docs Documentation changes (docs/) linux Linux backend changes (PyMemoryEditor/linux/) win32 Windows backend changes (PyMemoryEditor/win32/) macOS macOS backend changes (PyMemoryEditor/macos/) lib Library changes (PyMemoryEditor/) tests Test changes (tests/) labels Jun 15, 2026
read_process_memory allocates a fresh object per call, which creates
constant GC churn in long-running loops that poll the same-sized region
over and over. Add read_process_memory_into(address, buffer) to read
straight into a caller-owned, reusable buffer with no intermediate
allocation, keeping memory use constant.

Accepts any writable, contiguous buffer-protocol object (bytearray,
ctypes array, writable memoryview, numpy array - sized in bytes) and
returns the number of bytes read. Buffer prep/validation lives in a
shared util.convert.as_writable_c_buffer helper; each backend reuses its
existing low-level read primitive (ReadProcessMemory / process_vm_readv /
mach_vm_read_overwrite), including the same partial-read OSError guard.
@JeanExtreme002 JeanExtreme002 force-pushed the jeanextreme002/read-process-memory-into branch from e1eb3d1 to 6f155a9 Compare June 15, 2026 02:06
@JeanExtreme002 JeanExtreme002 merged commit 4aef4c2 into main Jun 15, 2026
18 checks passed
@github-actions github-actions Bot deleted the jeanextreme002/read-process-memory-into branch June 15, 2026 02:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Documentation changes (docs/) lib Library changes (PyMemoryEditor/) linux Linux backend changes (PyMemoryEditor/linux/) macOS macOS backend changes (PyMemoryEditor/macos/) tests Test changes (tests/) win32 Windows backend changes (PyMemoryEditor/win32/)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: read_process_memory optional argument to supply a buffer to read into

1 participant