Heap & Resource Management

Key heap-related types and traits for memory management

Heap & Resource Management

Monty uses a custom heap implementation for managing Python objects with reference counting and garbage collection. This page documents the key types and traits related to heap management.

Heap

The heap is the central memory arena for allocating and managing heap-allocated Python values.

Key Responsibilities

• Allocation: Allocates HeapData objects and returns HeapId references
• Reference Counting: Tracks reference counts for all heap objects
• Garbage Collection: Periodically reclaims unreachable objects
• Resource Tracking: Uses a ResourceTracker to enforce memory and allocation limits

Methods

allocate()

Allocates a new heap object and returns its HeapId.

data · HeapData · required

The data to allocate on the heap

Result<HeapId, ResourceError>

Returns the heap ID or a resource error if limits are exceeded

use monty::heap::{Heap, HeapData};
use monty::types::Str;
use monty::resource::NoLimitTracker;

let mut heap = Heap::new(1024, NoLimitTracker);
let str_id = heap.allocate(HeapData::Str(Str::new("hello".to_owned()))).unwrap();

get()

Retrieves a reference to a heap object by its HeapId.

id · HeapId · required

The heap ID of the object to retrieve

&HeapData

An immutable reference to the heap data

use monty::heap::{Heap, HeapData};

let heap = /* ... */;
let id = /* ... */;
let data = heap.get(id);

get_mut()

Retrieves a mutable reference to a heap object by its HeapId.

id · HeapId · required

The heap ID of the object to retrieve

&mut HeapData

A mutable reference to the heap data

clone_with_heap()

Clones a value while properly incrementing reference counts for any heap-allocated components.

value · &Value · required

The value to clone

Value

The cloned value with incremented reference counts

use monty::value::Value;
use monty::heap::Heap;

let heap = /* ... */;
let value = /* ... */;
let cloned = value.clone_with_heap(&heap);

drop_with_heap()

Drops a value while properly decrementing reference counts for any heap-allocated components.

This is the correct way to drop a Value that may contain heap references.

value · Value · required

The value to drop (consumes ownership)

heap · &mut Heap · required

Mutable reference to the heap for decrementing reference counts

All types that implement DropWithHeap hold heap references and must be cleaned up correctly on every code path. Use defer_drop! macro or HeapGuard to ensure cleanup on all paths (normal returns, early returns via ?, continue, etc.).

use monty::value::Value;
use monty::heap::Heap;

let mut heap = /* ... */;
let value = /* ... */;
value.drop_with_heap(&mut heap);

HeapId

An opaque identifier for a heap-allocated object.

Heap IDs are used to reference objects in the heap without directly holding pointers. This enables safe serialization, garbage collection, and reference counting.

HeapData

The actual data stored in the heap. This is an enum containing all heap-allocated Python types:

Str(Str) · HeapData

Heap-allocated string

Bytes(Bytes) · HeapData

Heap-allocated bytes object

List(List) · HeapData

Heap-allocated list

Tuple(Tuple) · HeapData

Heap-allocated tuple

Dict(Dict) · HeapData

Heap-allocated dictionary

Set(Set) · HeapData

Heap-allocated set

FrozenSet(FrozenSet) · HeapData

Heap-allocated frozenset

LongInt(LongInt) · HeapData

Heap-allocated arbitrary-precision integer

NamedTuple(NamedTuple) · HeapData

Heap-allocated named tuple

Dataclass(Dataclass) · HeapData

Heap-allocated dataclass instance

Exception(SimpleException) · HeapData

Heap-allocated exception object

Path(Path) · HeapData

Heap-allocated path object

Cell(Cell) · HeapData

Heap-allocated cell (for closures)

Closure(Closure) · HeapData

Heap-allocated closure

Coroutine(Coroutine) · HeapData

Heap-allocated coroutine

Range(Range) · HeapData

Heap-allocated range object

Iter(Iter) · HeapData

Heap-allocated iterator

Module(Module) · HeapData

Heap-allocated module

ResourceTracker

A trait for tracking and enforcing resource limits during execution.

Methods

track_allocation()

Called before each heap allocation to check if limits allow the allocation.

size · usize · required

The size of the allocation in bytes

Result<(), ResourceError>

Returns Ok(()) if the allocation is allowed, or a ResourceError if limits are exceeded

check_time()

Called periodically to check if execution time limits have been exceeded.

Result<(), ResourceError>

Returns Ok(()) if execution should continue, or a ResourceError if time limits are exceeded

Implementations

NoLimitTracker

A no-op resource tracker that allows unlimited allocations and execution time.

use monty::resource::NoLimitTracker;
use monty::{MontyRun, MontyObject, PrintWriter};

let runner = MontyRun::new(
    "x + 1".to_owned(),
    "test.py",
    vec!["x".to_owned()]
).unwrap();

let result = runner.run(
    vec![MontyObject::Int(41)],
    NoLimitTracker,
    &mut PrintWriter::Stdout
).unwrap();

Custom Trackers

You can implement your own ResourceTracker to enforce custom limits:

use monty::resource::{ResourceTracker, ResourceError};

struct MyTracker {
    max_allocations: usize,
    allocations: usize,
}

impl ResourceTracker for MyTracker {
    fn track_allocation(&mut self, _size: usize) -> Result<(), ResourceError> {
        self.allocations += 1;
        if self.allocations > self.max_allocations {
            Err(ResourceError::TooManyAllocations)
        } else {
            Ok(())
        }
    }

    fn check_time(&mut self) -> Result<(), ResourceError> {
        Ok(())
    }
}

PyTrait

The core trait implemented by all Python types (Value, HeapData).

Methods

py_type()

Returns the Python type of this value.

Type

The type enum variant (e.g., Type::Int, Type::Str)

py_len()

Returns the length of this value if it’s a sequence or collection.

Option<usize>

The length, or None if the type doesn’t support len()

py_bool()

Returns the truthiness of this value according to Python’s truth testing rules.

bool

True if the value is truthy

py_repr()

Returns the repr() string for this value.

Cow<'static, str>

The repr string (may be owned or borrowed)

py_str()

Returns the str() string for this value.

Cow<'static, str>

The str string (may be owned or borrowed)

Reference Counting Safety

All types that implement DropWithHeap hold heap references and must be cleaned up correctly on every code path. There are three mechanisms for ensuring this:

1. defer_drop! macro (preferred)

The simplest and safest approach. Use defer_drop! to bind a value into a guard that automatically drops it when scope exits.

let value = self.pop();
defer_drop!(value, heap);          // value is now &Value, heap is now &mut Heap
let result = value.py_repr(heap)?; // guard handles cleanup on all paths

Use defer_drop_mut! when you need a mutable reference:

let iter = vm.heap.get_iter(iter_ref);
defer_drop_mut!(iter, vm);
while let Some(item) = iter.for_next(vm)? { /* ... */ }

2. HeapGuard (when you need control)

Use HeapGuard directly when you need to conditionally extract the value instead of dropping it.

let mut lhs_guard = HeapGuard::new(self.pop(), self);
let (lhs, this) = lhs_guard.as_parts_mut();

if lhs.py_iadd(rhs, this.heap)? {
    let (lhs, this) = lhs_guard.into_parts(); // reclaim lhs, don't drop
    this.push(lhs);
    return Ok(());
}
// otherwise lhs_guard drops lhs automatically at scope exit

3. Manual drop_with_heap (simple cases only)

For very simple cases with a single linear code path:

let iter = self.pop();
iter.drop_with_heap(&mut self.heap); // single path, no branching

Avoid manual drop_with_heap whenever there are multiple code paths (branching, ?, continue, early returns) between acquiring and releasing the value.