feat: add read_process_memory_into for zero-copy buffer reuse (#71)#72
Merged
Merged
Conversation
read_process_memory allocates a fresh object per call, which creates constant GC churn in long-running loops that poll the same-sized region over and over. Add read_process_memory_into(address, buffer) to read straight into a caller-owned, reusable buffer with no intermediate allocation, keeping memory use constant. Accepts any writable, contiguous buffer-protocol object (bytearray, ctypes array, writable memoryview, numpy array - sized in bytes) and returns the number of bytes read. Buffer prep/validation lives in a shared util.convert.as_writable_c_buffer helper; each backend reuses its existing low-level read primitive (ReadProcessMemory / process_vm_readv / mach_vm_read_overwrite), including the same partial-read OSError guard.
e1eb3d1 to
6f155a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #71.
read_process_memoryallocates a fresh Python object on every call (a ctypesbuffer plus the returned
bytes). In a long-running loop that polls thesame-sized region over and over — a recorder, a live overlay, a poller — those
throwaway objects create constant GC churn instead of constant memory use.
This adds
read_process_memory_into(address, buffer) -> int, the zero-copycounterpart of
read_process_memory: it reads straight into a caller-owned,reusable buffer with no intermediate allocation, and returns the number of
bytes read.
How it works / why it fits the architecture
This is orthogonal to the existing optimizations, not a duplicate of them:
snapshot_memory_regions/memory_regions=search_by_addressesread_process_memory_into(this PR)bytearray,ctypesarray, writablememoryview,numpyarray (sized in bytes, so a4-element
int32array reads 16 bytes). Bytes land verbatim (no decoding).util.convert.as_writable_c_buffer, so there's no per-backend duplication.read_process_memory_intoreuses its existing low-levelread primitive (
ReadProcessMemory/process_vm_readv/mach_vm_read_overwrite), including the same partial-read →OSErrorguardthe regular read already enforces.
Errors
TypeError— buffer not writable (e.g. immutablebytes; usebytearray).ValueError— buffer empty or not contiguous.OSError— read failed or returned fewer bytes than requested.Tests
New
tests/memory/test_read_into.py(11 cases): bytearray / ctypes / memoryview/ numpy targets, buffer reuse across reads, byte-length sizing, parity with
read_process_memory, and the three error paths.cleanly, but were not executed locally — worth confirming via CI on those
platforms.
Docs
docs/guide/read-write.md.docs/api/openprocess.md).