Chapter 11 - Memory Layout and Low-Level Control

“Know your memory. Know your adversary.”

Systems programming requires precise control over memory layout: network protocols define wire formats, hardware registers have fixed offsets, and kernel structures must match ABI conventions. Rust provides tools for controlling memory layout while maintaining safety at the boundaries.

11.1 Representations: `repr`, Alignment, and Padding

11.1.1 Default Rust Layout

By default, Rust’s compiler is free to reorder struct fields and add padding for alignment:

#![allow(unused)]
fn main() {
struct NetworkHeader {
    version: u8,     // 1 byte
    flags: u8,       // 1 byte
    length: u16,     // 2 bytes (might be padded)
    seq: u32,        // 4 bytes
    ack: u32,        // 4 bytes
}
// Size: might be 12, 14, or 16 bytes depending on compiler decisions
}

This is fine for internal use but dangerous for parsing external data.

11.1.2 `#[repr(C)]` - C-Compatible Layout

#![allow(unused)]
fn main() {
#[repr(C)]
struct CNetworkHeader {
    version: u8,    // offset 0
    flags: u8,      // offset 1
    length: u16,    // offset 2
    seq: u32,       // offset 4
    ack: u32,       // offset 8
}
// Size: exactly 12 bytes on common ABIs, with no interior padding in this layout
}

#[repr(C)] guarantees:

Fields are laid out in declaration order.
Padding follows C ABI rules for the target platform.
The struct can be safely passed across FFI boundaries.

🔒 Security pattern: Always use #[repr(C)] for structs that:

Map to hardware registers
Define network wire formats
Are shared with C code
Are cast from raw byte arrays

11.1.3 `#[repr(C, packed)]` - No Padding

#![allow(unused)]
fn main() {
#[repr(C, packed)]
struct PackedHeader {
    version: u8,
    length: u16,  // No padding, but may be misaligned!
    seq: u32,
}
}

⚠️ Danger: Packed structs can cause unaligned memory access, which is:

Undefined behavior on some architectures (ARM, SPARC)
Slower on x86
Potentially a security issue (different behavior on different platforms)

🔒 Rule: Only use #[repr(packed)] for network packet parsing where the wire format has no padding. Always use read_unaligned and write_unaligned:

#![allow(unused)]
fn main() {
use std::{mem::size_of, ptr};

#[repr(C, packed)]
struct PackedHeader {
    version: u8,
    length: u16,
    seq: u32,
}

fn read_packed_header(data: &[u8]) -> Option<PackedHeader> {
    if data.len() < size_of::<PackedHeader>() {
        return None;
    }
    let ptr = data.as_ptr() as *const PackedHeader;
    Some(unsafe { ptr::read_unaligned(ptr) })
}
}

11.1.4 `#[repr(u8)]`, `#[repr(i32)]` - Enum Size Control

#![allow(unused)]
fn main() {
#[repr(u8)]
enum PacketType {
    Syn = 0x01,
    Ack = 0x02,
    Data = 0x03,
    Fin = 0x04,
}
// Guaranteed: sizeof(PacketType) == 1
}

11.1.5 `#[repr(transparent)]` - Single-Field Wrapper

#![allow(unused)]
fn main() {
#[repr(transparent)]
struct WrappedU64(u64);

// WrappedU64 has the exact same layout as u64
// Useful for newtypes that need FFI compatibility
}

11.2 Safe Parsing of Binary Data

11.2.1 The `zerocopy` Crate

Parsing binary data without copying is both a performance and security concern. Once you have a #[repr(C)] header type that derives the required zerocopy traits, parsing is straightforward:

[dependencies]
zerocopy = "0.8"

#![allow(unused)]
fn main() {
extern crate rust_secure_systems_book;
use rust_secure_systems_book::deps::zerocopy::TryFromBytes;
use rust_secure_systems_book::zerocopy_examples::TcpHeader;

fn parse_tcp_header(data: &[u8]) -> Option<&TcpHeader> {
    TcpHeader::try_ref_from_bytes(data).ok()
}
}

🔒 Security advantage: zerocopy verifies that:

The data is large enough for the type
Alignment requirements are met
No uninitialized memory is read

11.2.2 Manual Byte Parsing with `from_be_bytes`

For simpler cases, use the standard library’s byte conversion:

#![allow(unused)]
fn main() {
fn parse_u16_be(data: &[u8], offset: usize) -> Option<u16> {
    let end = offset.checked_add(2)?;
    let bytes: [u8; 2] = data.get(offset..end)?.try_into().ok()?;
    Some(u16::from_be_bytes(bytes))
}

fn parse_u32_be(data: &[u8], offset: usize) -> Option<u32> {
    let end = offset.checked_add(4)?;
    let bytes: [u8; 4] = data.get(offset..end)?.try_into().ok()?;
    Some(u32::from_be_bytes(bytes))
}
}

🔒 Security practice: Always use explicit endianness (from_be_bytes, from_le_bytes) rather than platform-dependent casts. Network protocols are big-endian; x86 is little-endian, while Wasm is specified as little-endian. Mixing them up is a subtle and dangerous bug.

11.3 Alignment and the `align` Representation

11.3.1 Controlling Alignment

#![allow(unused)]
fn main() {
#[repr(C, align(16))]  // 16-byte aligned (useful for SIMD, DMA buffers)
struct AlignedBuffer {
    data: [u8; 4096],
}
}

🔒 Security relevance: Some low-level interfaces benefit from aligned buffers:

SIMD and DMA interfaces may require or strongly prefer specific alignment
Alignment can reduce performance variance and avoid unaligned-access traps on some targets
Do not assume a crypto primitive requires a specific alignment unless its API or hardware manual says so

You cannot combine packed and align on the same struct. The compiler rejects that combination because packed lowers alignment while align(N) raises it. If you need a packed wire-format view and an aligned working buffer, keep them as separate types and copy or parse between them explicitly.

11.3.2 Cache-Line Alignment

#![allow(unused)]
fn main() {
#[repr(C, align(64))]  // Common cache-line size on modern x86-64/ARM64; verify on your target
struct CacheLineAligned {
    // Align the counter to reduce false sharing between adjacent instances.
    counter: std::sync::atomic::AtomicU64,
}
}

🔒 Security relevance: Reduces false sharing between threads, which can otherwise increase timing noise and contention. Treat this as a performance and isolation aid, not as a standalone side-channel defense.

11.3.3 `Pin<T>` and Address Stability

Most Rust values may move when ownership changes. That is fine for ordinary data, but self-referential types, intrusive data structures, and many async state machines rely on a stable address once initialized. Pin<&mut T> and Pin<Box<T>> are the tools that express “this value must not move again in safe code.”

Security relevance: if a type stores internal raw pointers or hands its address to foreign code, accidental movement can invalidate those pointers and turn a logic bug into memory unsafety in the surrounding unsafe code. Pinning does not make a type safe by itself, but it is a core part of making address-sensitive abstractions sound.

11.4 Working with Raw Memory

11.4.1 The Global Allocator

Rust uses the system allocator by default. For security-sensitive applications, you can use a custom allocator:

#![allow(unused)]
fn main() {
use std::alloc::{GlobalAlloc, Layout, System};
use std::sync::atomic::{compiler_fence, Ordering};

/// A simple allocator that zeroizes memory on free
struct SecureAllocator;

unsafe impl GlobalAlloc for SecureAllocator {
    unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
        unsafe { System.alloc(layout) }
    }
    
    unsafe fn dealloc(&self, ptr: *mut u8, layout: Layout) {
        // `GlobalAlloc::dealloc` is only called with a valid pointer that came
        // from the paired allocator, so zeroize first and then free it.
        for i in 0..layout.size() {
            unsafe { ptr.add(i).write_volatile(0); }
        }
        compiler_fence(Ordering::SeqCst);
        unsafe { System.dealloc(ptr, layout) };
    }
}

#[global_allocator]
static GLOBAL: SecureAllocator = SecureAllocator;
}

Because the zeroing loop already uses write_volatile, the compiler_fence here is a compiler barrier, not a hardware synchronization primitive. It stops the optimizer from moving or eliding the wipe without implying any cross-core memory-ordering requirement. Placing the fence after the loop is enough: it prevents the compiler from sinking the wipe past the eventual deallocation, so you do not need a fence between each volatile write.

⚠️ Note: This is a simplified example. A production secure allocator should also:

Lock pages containing keys (prevent swapping to disk)
Use mlock/VirtualLock to prevent paging
Use a zeroization primitive that is guaranteed not to be optimized away
Guard against heap metadata corruption

11.4.2 The `zeroize` Crate - Practical Memory Wiping

The zeroize crate gives you a practical way to wipe specific buffers before release, unlike naive manual loops that the compiler may optimize away:

[dependencies]
zeroize = { version = "1", features = ["derive"] }

#![allow(unused)]
fn main() {
extern crate rust_secure_systems_book;
extern crate zeroize;
use zeroize::{Zeroize, ZeroizeOnDrop};

struct CryptoKey {
    material: [u8; 32],
}

impl Drop for CryptoKey {
    fn drop(&mut self) {
        self.material.zeroize();
    }
}

// Or use the derive macro for automatic zeroization:
#[derive(Zeroize, ZeroizeOnDrop)]
struct SessionKey {
    key: [u8; 32],
    iv: [u8; 12],
}
// On normal drop paths, both `key` and `iv` are zeroized before release.

// Also works with Vec and other heap-allocated types:
#[derive(Zeroize, ZeroizeOnDrop)]
struct SecureBuffer {
    data: Vec<u8>,
}
}

🔒 Security practice: Use zeroize (with the derive feature) instead of manual zeroing loops. The crate is designed so the wipe operation itself is not optimized away, but it only affects the buffer you zeroize and only on code paths where zeroization runs. It does not erase copies you already made, and it cannot help if the process aborts or exits before Drop (see Chapter 2 §2.4 on panic = "abort").

Core dumps are another separate leak path: a crash can snapshot process memory before later cleanup would have happened, including live secrets that would normally be wiped on Drop. Chapter 19 section 19.6 shows how to disable core dumps for production services that handle sensitive material.

11.4.3 Safe Pointer Access with `&raw` (and `addr_of!`)

When working with structs that contain fields you cannot safely create (e.g., a MaybeUninit field), Edition 2024 prefers &raw const expr and &raw mut expr. They are the direct syntax behind the older addr_of! and addr_of_mut! macros and let you obtain raw pointers without creating intermediate references:

#![allow(unused)]
fn main() {
use std::mem::MaybeUninit;

#[repr(C)]
struct PacketBuffer {
    header: [u8; 4],
    payload: MaybeUninit<[u8; 1024]>,
}

impl PacketBuffer {
    fn new() -> Self {
        PacketBuffer {
            header: [0; 4],
            payload: MaybeUninit::uninit(),
        }
    }
    
    fn write_payload(&mut self, data: &[u8]) {
        let payload_ptr = &raw mut self.payload;
        unsafe {
            let raw = (*payload_ptr).as_mut_ptr() as *mut u8;
            raw.copy_from_nonoverlapping(data.as_ptr(), data.len().min(1024));
        }
    }
    
    fn read_header(&self) -> &[u8; 4] {
        // `header` is always initialized, so a normal shared reference is fine.
        &self.header
    }
}
}

The older macros still work and are useful in older code or macro-heavy contexts, but &raw const / &raw mut is the modern direct form.

In this example, &raw mut self.payload is the important operation because payload is the maybe-uninitialized field. You do not need &raw for every other initialized field in the same struct.

🔒 Security practice: Prefer &raw const / &raw mut (or addr_of! / addr_of_mut! in older code) over &self.field or &mut self.field when the struct may contain uninitialized data. Creating a reference to uninitialized memory is instant UB, even if you never read through it.

11.4.4 Memory Locking (Prevent Swapping)

#![allow(unused)]
fn main() {
extern crate rust_secure_systems_book;
#[cfg(unix)]
use rust_secure_systems_book::deps::libc as libc;
use rust_secure_systems_book::deps::windows_sys as windows_sys;
#[cfg(unix)]
fn lock_memory(ptr: *const u8, len: usize) -> Result<(), std::io::Error> {
    let result = unsafe { libc::mlock(ptr as *const libc::c_void, len) };
    if result != 0 {
        return Err(std::io::Error::last_os_error());
    }
    Ok(())
}

#[cfg(windows)]
fn lock_memory(ptr: *const u8, len: usize) -> Result<(), std::io::Error> {
    use windows_sys::Win32::System::Memory::VirtualLock;
    let result = unsafe { VirtualLock(ptr.cast(), len) };
    if result == 0 {
        return Err(std::io::Error::last_os_error());
    }
    Ok(())
}
}

⚠️ Operational caveat: Treat memory locking as a deployment requirement, not a best-effort hint. On Linux, a successful mlock means the whole covered range is locked; failures such as EAGAIN, ENOMEM, or EPERM should be handled as hard failures. In practice, unprivileged processes are constrained by RLIMIT_MEMLOCK and sometimes CAP_IPC_LOCK. On Windows, VirtualLock is limited by the process working-set size.

⚠️ System caveat: Locking pages prevents ordinary swapping, not every persistence path. Suspend-to-disk and hibernation can still write RAM contents to disk, so disk encryption and platform sleep policy still matter for high-value secrets.

🔒 Security impact: Prevents sensitive data (keys, passwords) from being written to the swap file, where they could persist after the process exits (CWE-316: Cleartext Storage of Sensitive Information).

11.5 Stack and Heap Security

11.5.1 Stack Observability and Protection

Rust already enables stack probing/stack-clash protection on mainstream targets, but stable Rust does not enable stack canaries for Rust code by default. That is less alarming than it sounds for safe Rust because the compiler rules out the classic out-of-bounds stack writes that canaries were designed to catch. The concern returns once you add unsafe code, inline assembly, or C/C++ objects to the same binary. Keep frame pointers for profiling and post-mortem analysis, and treat C/C++ stack canaries as a separate hardening step for any non-Rust objects you compile:

# .cargo/config.toml
[build]
rustflags = ["-C", "force-frame-pointers=yes"]

If you build C/C++ code via the cc crate, apply -fstack-protector-strong to those objects separately. For RELRO, NX, and PIE settings, use the linker hardening flags from Chapter 19. Frame pointers improve observability; they are not a canary mechanism.

11.5.2 Guard Pages

Rust’s default allocator does not place guard pages between heap allocations, neither in debug nor release mode. Guard pages exist at stack boundaries (enforced by the OS), not between individual heap allocations. For heap-level guard page protection, use a hardened allocator like GWP-ASan or run under AddressSanitizer (ASan):

# Run with ASan to detect heap buffer overflows
RUSTFLAGS="-Zsanitizer=address" cargo +nightly run

If you do not need allocator-specific behavior, omit #[global_allocator] entirely. This note applies to declarations such as static GLOBAL: std::alloc::System = std::alloc::System;, which simply rebind the default allocator to itself. The earlier SecureAllocator example is not a no-op: it installs a real wrapper that changes deallocation behavior.

11.6 Summary

Use #[repr(C)] for FFI-compatible and wire-format structures.
Use #[repr(u8)] / #[repr(i32)] to control enum size.
Use #[repr(C, align(N))] for alignment-critical data.
Prefer zerocopy crate for safe binary parsing.
Always use explicit endianness for network data.
Zero sensitive memory before freeing (use zeroize crate).
Lock memory pages containing secrets (prevent swapping).
Verify release hardening (RELRO/NX/PIE) and apply stack-protector flags to any C/C++ objects you compile.

In the next chapter, we cover secure network programming: building network services that resist common attacks.

11.7 Exercises

Wire-Format Parser: Define a #[repr(C)] struct representing a simplified IPv4 header (version, IHL, total length, TTL, protocol, source IP, dest IP). Use the zerocopy crate to parse a raw byte slice into the struct. Write tests with valid packets, truncated packets, and misaligned data. Verify that all invalid inputs return an error.
Endianness Trap: Write two versions of a u32 parser: one using from_be_bytes and one using from_le_bytes. Feed both the same byte sequence [0x00, 0x01, 0x02, 0x03]. Verify the results differ. Then write a function that parses a TCP header from bytes using correct big-endian for all multi-byte fields.
Aligned Buffer: Create a 16-byte-aligned buffer type using #[repr(C, align(16))]. Implement methods to safely write and read data. Verify alignment at runtime using pointer arithmetic. Discuss why aligned buffers can matter for SIMD or DMA paths, and why AES-NI itself does not require 16-byte alignment for correctness.

Keyboard shortcuts

Rust for Secure Systems Programming