An address is a process view
3 min readYour completed shell creates children and replaces their programs. You now need to understand the memory those programs use. Imagine a sensor service that reserves a large buffer. Its reservation succeeds, but the board later becomes slow when samples arrive. Reserving addresses and supplying their contents are different operations.
A A virtual address is a memory location expressed in the address space used by a process. is the kind of address stored in an ordinary C pointer. An address space is the collection of addresses that a process can use. A A physical address identifies a location in the machine's physical memory address map. belongs to the hardware view. That map includes RAM and can also include device registers. An ordinary user pointer does not directly identify a physical RAM location.
A A page is a fixed-size unit used to map and manage virtual memory. divides the address space into manageable pieces. A physical page frame is a page-sized RAM unit. Adjacent virtual pages can point to distant physical frames. Two processes can also map the same frame.
The The memory management unit is hardware that translates virtual addresses and checks memory access permissions. uses tables prepared by Linux. These page tables associate virtual page numbers with physical frames and access flags. The address within a page stays unchanged during translation.
For a system with 4096-byte pages, virtual byte 8195 lies in virtual page 2 at offset 3. Translation changes the page number, then adds that offset. This arithmetic is an example, not a promise about your board. The common 4 KB page means 4096 bytes here. AArch64 kernels can use other base page sizes, including 16 KB and 64 KB.
Ask your running Linux system for its page size before doing page arithmetic. This command prints bytes per base page:
getconf PAGESIZE
In C, use sysconf(_SC_PAGESIZE) and check for failure. Do not bake the host's answer into a cross-compiled board application. Cross-compilation means building on one machine for another machine. Runtime queries describe the target running the program.
Virtual memory provides isolation, flexible placement, and sharing. It does not mean that every page lives on disk. A board without swap still uses virtual memory. Swap is storage used to hold evicted anonymous memory contents. Linux configurations for processors without an MMU have different constraints; this lesson assumes MMU-enabled Linux.
Read the regions in a process map
3 min readA small C program already needs several kinds of memory. Its compiled instructions live in text. A global int rate = 5; normally belongs to initialized writable data. A global int samples; starts at zero and normally belongs to BSS. BSS is the conventional name for zero-initialized static storage. Its zero contents need not occupy equivalent bytes in the executable file.
The heap holds dynamically allocated objects, such as buffers obtained through malloc. The stack holds function-call state and many automatic local variables. Optimization can keep a variable in a register or remove it entirely. A diagram of regions is not a guarantee about every C variable.
The mmap region contains mappings for libraries, files, anonymous buffers, and many thread stacks. It is not one mandatory contiguous segment. Read-only constants often have a separate read-only mapping. The executable and each library can therefore occupy several map lines with different permissions.
A A virtual memory area is a contiguous virtual address range with a common set of mapping attributes. records a region's policy in the kernel. Its address range has an included start and an excluded end. A mapping can split into several VMAs when you change permissions in its middle. Compatible neighboring regions can merge.
Inspect a process that stays alive long enough to examine. These commands start sleep, remember its process ID, and read its mappings:
sleep 120 &
target_pid=$!
cat "/proc/$target_pid/maps"
pmap -x "$target_pid"
wait "$target_pid"
maps gives address range, permissions, file offset, device, inode number, and an optional pathname. An inode number identifies a file within its filesystem. Read r, w, and x as permission to read, write, and execute. The final p means private mapping; s means shared. A private mapping may still share physical pages before a write.
pmap -x, from procps, adds size and residency information. BusyBox builds may omit pmap or support fewer options. Read /proc directly when that tool is unavailable. Access to another process's map can be restricted by security policy.
cat /proc/self/maps shows the mappings of cat, because self identifies the process opening the file. It does not show your interactive shell. For the shell, inspect /proc/$$/maps. For your program, open /proc/self/maps inside that program.
On common x86-64 and AArch64 layouts, the main stack grows toward lower addresses. Raising the traditional heap boundary grows it upward. mmap placement is often top-down, but it depends on architecture, limits, and placement policy. An individual mapping does not automatically grow downward. Address randomization also changes locations between runs. Treat the kernel's map format as evidence; treat the diagram as a schematic.
How brk and malloc obtain storage
3 min readSuppose you allocate three buffers and free the middle one. The allocator can reuse its space. However, it cannot lower the heap boundary past a still-live buffer above it. This is a central limitation of one contiguous heap.
The The program break is the boundary at the end of the process's traditional data-and-heap region. moves through brk or sbrk. The C library's brk(addr) requests a new boundary and returns zero on success, minus one on failure. sbrk(increment) changes the boundary by a signed byte count. On success it returns the previous break, not the new one. Its failure value is (void *)-1.
sbrk(0) queries the current break. These are legacy interfaces, absent from modern POSIX. They describe a byte boundary, although Linux manages the underlying memory in pages. Moving the break by a few bytes need not allocate another physical page.
mmap became a more flexible source of allocator storage because it supports independent ranges, different backing objects, and separate permissions. It did not universally replace brk. On Linux, glibc's allocator can still extend its main arena through the break. An arena is a group of allocator state and storage used to manage allocations. Additional arenas can use mappings.
A free list tracks blocks available for reuse. glibc has several size-based collections, called bins, and a per-thread cache called tcache. It can split a larger block or combine adjacent free blocks. Some caches delay combining. Therefore, a textbook single free list explains the idea without fully describing glibc.
M_MMAP_THRESHOLD, configured with mallopt, influences when glibc considers a separate mapping for a sufficiently large request. It is not an unconditional size switch. Existing free space, allocator limits, and allocation history affect the path. glibc can adjust the threshold dynamically when it has not been explicitly fixed. See mallopt.
Freeing an independently mapped large allocation can unmap it. Freeing an ordinary heap block usually makes it reusable inside the allocator. Resident memory need not fall immediately. Fragmentation means wasted space caused by block sizing or by free pieces separated by live objects. Neither retained space nor fragmentation alone proves a leak.
Trace your own compiled program to see which kernel operations occur. The loader also creates mappings, so examine when calls happen:
strace -e trace=brk,mmap,munmap,mremap ./your_program
On some 32-bit targets, the trace can show mmap2 instead of mmap. That is a system-call interface detail handled by the C library. Do not apply glibc's bin layout or threshold rules to musl, another C library common on small Linux systems.
Going deeperThe library return value is not always the raw syscall result
Linux's raw brk system call returns the resulting break, including the unchanged break after failure. glibc checks this result and exposes the zero-or-minus-one API. sbrk is library bookkeeping built around that mechanism. See brk(2). Changing the break behind glibc's allocator can corrupt its assumptions; use a separate anonymous mapping for an experimental allocator.
Map a file with explicit boundaries
2 min readA A mapping connects a virtual address range to backing storage and access rules. can be file-backed or anonymous. File-backed means its initial contents come from a file or similar object. Anonymous means there is no ordinary file supplying those initial contents; new anonymous memory reads as zero.
mmap(addr, length, prot, flags, fd, offset) creates a mapping. Pass NULL as addr to let Linux choose a suitable address. Check the return against MAP_FAILED, not NULL. A mapping that failed never gives you usable storage.
1Choose access and sharing separately
PROT_READ, PROT_WRITE, and PROT_EXEC request reads, writes, and instruction execution. Combine permissions with bitwise OR. PROT_NONE denies all access. Hardware and policy can restrict combinations; a write permission can imply reading on some architectures. Ordinary data buffers should not request execution.
Choose MAP_PRIVATE for private modifications or MAP_SHARED for shared updates. Add MAP_ANONYMOUS for zero-initialized storage without a file; use descriptor -1 and offset zero. A writable shared regular-file mapping needs a descriptor opened for reading and writing. A writable private mapping can use a read-only descriptor because its writes do not update the file.
2Make the file large enough
This standalone example creates a disposable file, extends it, and changes its first byte through a shared mapping. Save it as mapped-file.c. ftruncate establishes the file size before the memory access.
#define _POSIX_C_SOURCE 200809L
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>
int main(void)
{
long page = sysconf(_SC_PAGESIZE);
if (page <= 0) return EXIT_FAILURE;
size_t len = (size_t)page;
int fd = open("mapping-demo.bin", O_RDWR | O_CREAT | O_EXCL, 0600);
if (fd == -1) { perror("open"); return EXIT_FAILURE; }
if (ftruncate(fd, (off_t)page) == -1) {
perror("ftruncate"); close(fd); return EXIT_FAILURE;
}
unsigned char *p = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED, fd, 0);
if (p == MAP_FAILED) {
perror("mmap"); close(fd); return EXIT_FAILURE;
}
p[0] = 'Z';
int failed = 0;
if (msync(p, len, MS_SYNC) == -1) { perror("msync"); failed = 1; }
if (munmap(p, len) == -1) { perror("munmap"); failed = 1; }
if (close(fd) == -1) { perror("close"); failed = 1; }
return failed ? EXIT_FAILURE : EXIT_SUCCESS;
}
Build and run it in an empty exercise directory. The exclusive-create flag deliberately refuses to overwrite an existing file. Then inspect the first byte:
cc -std=c11 -Wall -Wextra -O0 -g mapped-file.c -o mapped-file
./mapped-file
od -An -tc -N1 mapping-demo.bin
The inspected byte should be the character written by your program. MS_SYNC requests synchronous writeback and reports errors. Writeback means carrying modified cached contents to their backing file. It does not make multiple stores an atomic transaction. Crash-safe updates still need a suitable filesystem and storage design.
3Respect page and file boundaries
For ordinary mappings, length must be positive but need not be page-aligned. Linux maps whole pages covering it. The file offset must be a multiple of the runtime page size. munmap requires a page-aligned start and removes every page intersecting its positive length.
To expose file byte offset wanted, compute base = wanted - wanted % page_size. Then compute delta = wanted - base, map from base, and access mapping + delta. Request enough bytes for delta + desired_length. Check arithmetic overflow and file bounds first. Preserve the original pointer and length for cleanup.
Closing the descriptor after a successful mapping does not invalidate the mapping. Truncating the file can invalidate access to mapped pages beyond its new end, producing SIGBUS. A partial final page has special zero-filled tail behavior; never use bytes beyond the actual file size as storage. See mmap(2).
Sharing memory is different from sharing updates
1 min readMap a file privately and change one byte. Your process sees the change, but the file does not receive it. Copy-on-write shares existing contents until a write requires a private version. avoids copying every mapped byte at creation. Modified private file pages become anonymous memory from the process's perspective.
A private file mapping is not a guaranteed frozen snapshot of later external file changes. Coordinate with file writers if you require stable contents. Private means your writes stay private; it does not establish a file-version protocol.
For related processes, Linux supports MAP_SHARED | MAP_ANONYMOUS. Create the mapping before fork, so both processes inherit access to the same storage. Creating two separate anonymous mappings does not connect them, even if their addresses match.
This single-threaded example gives the child sole write access while the parent waits for its termination. Save it as shared-byte.c. waitpid also prevents the parent from checking before the child finishes.
#define _GNU_SOURCE
#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <sys/wait.h>
#include <unistd.h>
int main(void)
{
long page = sysconf(_SC_PAGESIZE);
if (page <= 0) return EXIT_FAILURE;
size_t len = (size_t)page;
unsigned char *p = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_SHARED | MAP_ANONYMOUS, -1, 0);
if (p == MAP_FAILED) { perror("mmap"); return EXIT_FAILURE; }
pid_t child = fork();
if (child == -1) {
perror("fork"); munmap(p, len); return EXIT_FAILURE;
}
if (child == 0) { p[0] = 42; _exit(0); }
int status;
pid_t done;
do { done = waitpid(child, &status, 0); }
while (done == -1 && errno == EINTR);
int ok = done == child && WIFEXITED(status)
&& WEXITSTATUS(status) == 0 && p[0] == 42;
if (done == -1) perror("waitpid");
printf("shared update: %s\n", ok ? "observed" : "failed");
if (munmap(p, len) == -1) { perror("munmap"); ok = 0; }
return ok ? EXIT_SUCCESS : EXIT_FAILURE;
}
Build this with the same compiler options as the file example. Changing MAP_SHARED to MAP_PRIVATE makes the parent's byte remain zero after the child changes its private copy. This controlled comparison isolates sharing from scheduling.
Shared memory does not provide locks, message boundaries, or atomic multi-byte updates. Later lessons introduce process-shared synchronization. A mutex is a lock that gives one participant exclusive access at a time. Shared-memory mutexes need an explicit process-shared setup; placing an ordinary private mutex in shared memory is insufficient.
msync handles mapped-file writeback. It is not a publication barrier for a shared-memory queue. Adding volatile does not solve concurrent access either. This example avoids concurrent access to the byte; a continuously running producer and consumer need a stronger design.
On an embedded board, bounded shared buffers can avoid copying samples through pipes. They also require an ownership rule for every slot and a recovery plan if a process dies. Lower copying cost does not remove those responsibilities.
What happens when a page faults
4 min readA A page fault is a processor exception caused when a memory access cannot complete using the current translation or permissions. is often normal. Demand paging means supplying a page only when it is first needed. A newly mapped anonymous range may have no private physical frames yet.
A first anonymous read can use a shared read-only zero page. A first write needs writable private storage. Accessing a resident file-cache page may only require installing a page-table entry. The page cache is RAM in which Linux keeps file contents. An absent file page may require storage input.
Minor faults resolve without fetching the needed page from storage. Major faults require such input, for example from a mapped file or swap. A minor fault can still be expensive: it can allocate memory, zero it, or trigger copying. Major does not mean invalid, and minor does not mean harmless to latency.
The The translation lookaside buffer is a processor cache of recent address translations. saves repeated table walks. A TLB miss can be resolved by walking valid page tables without entering the kernel. It is not automatically a page fault.
Access: The processor tries to read, write, or execute a virtual address.
Exception: Missing translation or disallowed access transfers control to a kernel fault handler.
Validate: Linux finds the VMA and checks whether this access is permitted.
Supply: Linux allocates, copies, or fetches contents when the access can be satisfied.
Install: Linux updates page tables and performs required translation-cache maintenance.
Resume: The instruction retries, or Linux delivers a signal if the access cannot succeed.
1Separate reservation from touching
Save this bounded probe as vm-touch.c. It maps a modest number of base pages, then writes once per page. volatile here keeps the demonstration stores observable to the compiler; it provides no interprocess synchronization.
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>
int main(void)
{
long page = sysconf(_SC_PAGESIZE);
if (page <= 0 || page > 65536) return EXIT_FAILURE;
size_t step = (size_t)page, len = step * 128;
void *base = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (base == MAP_FAILED) { perror("mmap"); return EXIT_FAILURE; }
printf("pid=%ld mapping=%p bytes=%zu; press Enter to touch\n",
(long)getpid(), base, len);
fflush(stdout);
(void)getchar();
volatile unsigned char *p = base;
for (size_t i = 0; i < len; i += step) p[i] = 1;
puts("Touched; press Enter to unmap");
(void)getchar();
if (munmap(base, len) == -1) { perror("munmap"); return EXIT_FAILURE; }
return EXIT_SUCCESS;
}
Inspect its printed PID with pmap -x at both pauses. Compare residency rather than assuming one visible map line per mmap call. Neighboring anonymous regions may merge.
2Count faults without inventing a fixed result
Build the probe, then count software fault events. The two newline inputs remove its interactive pauses for this run:
cc -std=c11 -Wall -Wextra -O0 -g vm-touch.c -o vm-touch
printf '\n\n' | perf stat -e page-faults,minor-faults,major-faults ./vm-touch
Counts also include startup and library activity. Huge-page policy, cache state, and prior accesses affect results. One store per base page does not guarantee exactly one fault per store. If perf is unavailable or denied, use getrusage(RUSAGE_SELF) before and after the loop. Its ru_minflt and ru_majflt fields provide process-level counters. Do not weaken system-wide permissions merely to obtain this measurement.
3Follow fork's private pages
After ordinary fork, parent and child have separate address-space metadata and page tables. Their entries can reference the same physical data pages. Writable private pages are protected so a later write enters copy-on-write handling. A writer normally receives its own copied page; Linux can avoid copying when it can safely reuse an exclusive page.
This is data-page sharing through page tables, not ordinary sharing of one writable page-table tree. fork still pays for task creation and memory metadata. Large or heavily populated address spaces can make that cost significant before any user write. A child that immediately calls exec often avoids copying most private data pages.
Going deeperWhere to read the kernel mechanism
In Linux source, start at mm/memory.c for handle_mm_fault and copy-on-write handling such as do_wp_page. Architecture entry code lives elsewhere, including arch/arm64/mm/fault.c. struct mm_struct describes an address space; struct vm_area_struct describes a region. Modern kernels index VMAs with a Maple Tree, a range-indexing data structure, rather than the older red-black-tree account found in older books. VMA locks protect mapping metadata; page-table locks protect different structures. See process addresses.
Protect pages and understand stack limits
3 min readmprotect(addr, length, prot) changes permissions on existing mapped pages. Linux requires a page-aligned starting address. A positive length covers all intersecting pages. It returns zero on success and minus one with errno on failure. Protecting part of an allocator block can also protect unrelated objects sharing that page.
A guard page is deliberately inaccessible memory placed beside usable storage. Crossing into it produces a protection fault. This catches crossings into that page, not every possible out-of-bounds access. A small overrun that remains inside the usable page can escape detection.
The following demonstration catches SIGSEGV and exits. Save it as guard-page.c. It uses a signal handler because earlier lessons already introduced signals. The handler uses write and _exit, which are safe in signal context; it does not call printf or return to repeat the faulting access.
#define _GNU_SOURCE
#include <signal.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <unistd.h>
static void guard_fault(int signo)
{
(void)signo;
static const char message[] = "guard page reached\n";
(void)write(STDERR_FILENO, message, sizeof message - 1);
_exit(77);
}
int main(void)
{
long page = sysconf(_SC_PAGESIZE);
if (page <= 0 || page > 65536) return EXIT_FAILURE;
size_t len = (size_t)page * 2;
void *base = mmap(NULL, len, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
if (base == MAP_FAILED) { perror("mmap"); return EXIT_FAILURE; }
struct sigaction sa = {0};
sa.sa_handler = guard_fault;
sigemptyset(&sa.sa_mask);
if (sigaction(SIGSEGV, &sa, NULL) == -1) {
perror("sigaction"); munmap(base, len); return EXIT_FAILURE;
}
unsigned char *guard = (unsigned char *)base + (size_t)page;
if (mprotect(guard, (size_t)page, PROT_NONE) == -1) {
perror("mprotect"); munmap(base, len); return EXIT_FAILURE;
}
*(volatile unsigned char *)guard = 1;
munmap(base, len);
return EXIT_FAILURE;
}
Build with the same warning and debug options as the other examples. Run it, then inspect $? immediately. Status 77 is this program's chosen handled-fault status. It is not Linux's universal segmentation-fault exit code. Kernel cleanup removes the mapping when the handler exits.
The main stack can expand on suitable faults, subject to limits and neighboring mappings. In Bash, ulimit -s displays the soft stack limit in KiB. /proc/$$/limits reports it with units. That limit is not a promise that all stack pages are already resident. Dynamically allocated thread stacks generally use separately sized mappings instead of automatic main-stack growth.
Linux maintains separation around growable stacks, but a schematic guard gap is not a universal single mapped PROT_NONE page. Large stack jumps can bypass a narrow guard. Compiler stack-clash protection probes large growth incrementally on supported targets. Avoid large automatic arrays and unbounded recursion in board services.
To diagnose an actual stack overflow, a handler may need an alternate signal stack installed with sigaltstack and SA_ONSTACK. The exhausted stack may be unable to run an ordinary handler. The demo above intentionally faults in a separate mapping, so it does not demonstrate recovery from stack exhaustion.
MAP_FIXED is another way to alter an address range, but it can discard overlapping mappings. Never choose an address from an old map listing and assume it remains free. MAP_FIXED_NOREPLACE requests an exact range without replacing existing mappings on supporting kernels. Code supporting older kernels must verify the returned address. Ordinary application buffers should let Linux choose their location.
Control residency without confusing the controls
3 min readSuppose a motor-control task first touches a buffer during its critical loop. A valid pointer can still fault at that moment. Prepare bounded memory before entering the loop, and measure the complete path. A reservation alone does not remove first-use work.
mlock(addr, length) makes the covered pages resident and prevents swapping while locked. munlock releases that lock. mlockall(MCL_CURRENT) covers current mappings; MCL_FUTURE covers future mappings. With MCL_ONFAULT, locking can defer population until access. Check every return value. See the locking API.
Unprivileged processes can lock memory within their RLIMIT_MEMLOCK allowance. They do not always require root. Bash's ulimit -l reports that allowance in KiB. The CAP_IPC_LOCK privilege changes the limit rules. A service manager can set a different allowance from your interactive shell.
Locking is not a guarantee of fault-free execution under every later operation. fork can introduce copy-on-write faults. New stack use can need preparation. With future locking enabled, later allocations or stack expansion can fail when limits are reached. Locking also does not provide a stable device address for direct memory access. DMA lets a device transfer memory without ordinary CPU loads; its pinning and cache rules belong to kernel and driver interfaces.
MAP_POPULATE asks mmap to prepare page tables and read ahead where applicable. It shifts some work toward mapping time. A successful mmap does not guarantee that every requested population succeeded. It also does not lock those pages against later eviction.
mincore(addr, length, vec) queries residency. Allocate one result byte per covered base page, use a page-aligned address, and test vec[i] & 1. Other bits are not yours to interpret. The answer can become stale immediately. Resident file-cache data can still need a minor mapping fault. Residency therefore does not prove that the next access takes no fault. See mincore(2).
madvise(addr, length, advice) supplies page-range advice. On Linux, align addr to a page boundary; the length covers whole intersecting pages. MADV_SEQUENTIAL tells Linux to expect forward scanning, allowing more read-ahead and earlier reclamation. MADV_RANDOM discourages read-ahead. These are useful workload hints, not required access orders.
MADV_DONTNEED has stronger consequences. On private anonymous memory, discarded contents are replaced by zero-filled contents on later access. On suitable file-backed memory, later access can reload the backing file. Do not use it as a harmless hint on live objects. It differs from munmap: addresses remain mapped even when contents are discarded. Consult madvise(2) for mapping-specific restrictions.
On a small ARM board, locking a large buffer deprives other services of reclaimable RAM. Use a measured working set: the pages actually needed during a phase. Record fault counts before and after warming that set, then repeat under realistic memory pressure. Cache warming, page residency, and scheduler delay remain separate sources of latency.
Diagnose memory growth and allocation failures
3 min readVSZ, or virtual size, measures mapped virtual space. RSS, or resident set size, measures resident mapped pages, including shared pages. USS, or unique set size, estimates resident memory unique to one process. PSS, or proportional set size, divides shared-page costs among their mappings. A large virtual reservation can have a small RSS; several processes can each report the same shared pages in RSS.
For ordinary mappings, adding Private_Clean and Private_Dirty from smaps gives a practical USS estimate. Clean pages have no uncommitted changes to their backing storage; dirty pages have modified contents. Explicit huge-page mappings have separate accounting fields and require special care. USS is not a universal single field or a perfect prediction of memory freed at exit.
Read detailed per-region accounting from /proc/PID/smaps, or totals from smaps_rollup when available. Substitute your probe's printed PID for 12345 before running:
target_pid=12345
cat "/proc/$target_pid/smaps_rollup"
grep -E '^(Size|Rss|Pss|Private_Clean|Private_Dirty|Anonymous|Swap):' "/proc/$target_pid/smaps"
cat /proc/sys/vm/overcommit_memory
grep -E '^(MemAvailable|SwapFree|CommitLimit|Committed_AS):' /proc/meminfo
The figures describe changing snapshots. Detailed scanning costs work, so avoid high-frequency smaps polling on a busy target.
Linux normally permits memory overcommit: it can accept reservations beyond immediately available backing capacity. Mode 0 uses heuristics and is the kernel default. Mode 1 relaxes commit checks. Mode 2 checks committed demand against a configured limit. Mode 1 does not disable address-space or other resource limits. Strict mode does not eliminate every possible OOM condition. See overcommit accounting.
In strict mode, the limit is based on swap plus a configured portion of RAM. overcommit_ratio sets a percentage; overcommit_kbytes supplies an absolute alternative. Inspect CommitLimit and Committed_AS rather than equating a virtual reservation with used RAM.
An ENOMEM failure can also mean an address-space limit, insufficient contiguous virtual space, or too many VMAs. Check /proc/PID/limits for RLIMIT_AS and RLIMIT_DATA, which limit address space and data-related allocations. Check /proc/sys/vm/max_map_count for the mapping-count ceiling. Large sparse reservations are especially difficult in a 32-bit address space.
When reclaim cannot satisfy demand, the out-of-memory killer can terminate a selected process with SIGKILL. Selection considers eligible processes, their memory use, and score adjustments. It does not always select the process with the largest RSS. /proc/PID/oom_score_adj ranges from -1000 to 1000; increasing it makes selection more likely. Lowering it can require privilege, and -1000 excludes the process from normal victim selection. See oom_score_adj.
A cgroup is a kernel mechanism for grouping processes and controlling resources. A cgroup v2 memory.max limit can cause local OOM even when the whole board has free RAM. Inspect the service's actual cgroup and its memory.events. Kernel OOM messages, available through sudo dmesg or the system journal where supported, help distinguish this from an application crash.
Common mistakes: symptom, cause, diagnosis, fix
| Symptom | Likely cause | Diagnose | Fix |
|---|---|---|---|
| VSZ jumps, RSS barely moves | Untouched reservation | Compare maps with smaps | Budget touched pages as well as reservations |
| RSS stays high after free | Allocator retention or fragmentation | Compare live allocation records with private residency | Reuse bounded pools; investigate live growth before tuning |
| mmap fails with EINVAL | Bad offset, flags, or zero length | Log checked arguments and runtime page size | Align the offset and preserve the interior delta |
| SIGBUS after a file update | Access beyond a truncated file | Inspect file size and coordinate with writer | Stabilize file lifetime and bounds |
| Service disappears without its handler | SIGKILL, possibly OOM | Check wait status, cgroup events, kernel logs | Bound demand and correct the measured limit or leak |
Evaluate advanced mapping mechanisms
4 min readStart with relocation. mremap(old, old_size, new_size, MREMAP_MAYMOVE) can resize a mapping and move it if necessary. Check for MAP_FAILED before replacing your saved pointer. After a successful move, old interior pointers are invalid. Store offsets within movable data structures and derive addresses from the new base. Moving page-table entries can avoid copying every byte, but resizing still has metadata and translation costs. See mremap(2).
Huge pages map more memory per translation than base pages. They can reduce TLB pressure and page-table storage for large dense buffers. Explicit MAP_HUGETLB mappings use configured HugeTLB resources. They can fail when the required pool or permissions are unavailable. Their offset, address, and unmapping alignment rules use the huge-page size; ordinary base-page rules are insufficient. See HugeTLB pages.
Transparent huge pages, or THP, let Linux use larger pages for eligible mappings under kernel policy. They differ from explicit HugeTLB reservations. Promotion, splitting, and compaction can introduce latency. Compaction rearranges movable physical allocations to create larger contiguous free areas. Sparse use can also waste RAM when a large page backs few useful bytes. Check AnonHugePages, KernelPageSize, and related smaps fields alongside the target's THP configuration. The THP documentation describes policy and available sizes.
NUMA means non-uniform memory access: reaching some memory nodes costs more than reaching others. A node groups memory and nearby processors. Linux's default local allocation often makes the thread that first writes a page important. Placement policy and permitted nodes can change that result. Many small boards have one node, so this optimization may offer no benefit there.
set_mempolicy changes the calling thread's policy for suitable future allocations. mbind applies policy to a range. Installing policy does not automatically relocate every existing page; migration requires appropriate flags and permissions. Both APIs must respect allowed nodes. Compare /proc/PID/numa_maps with CPU placement before tuning. See NUMA memory policy.
A cache line is the unit moved and tracked by CPU caches. Two cores writing separate counters in one line can repeatedly transfer ownership. This false sharing can occur inside perfectly valid shared mappings. Page alignment alone does not separate fields inside a page. Group data by ownership and use the target's measured cache-line requirements; padding every field wastes precious board memory.
Finally, userfaultfd lets a separate handler manage supported faults in registered ranges. It is useful for controlled paging and virtual-machine migration, where memory can arrive on demand from another host. It is a negotiated kernel interface, not a replacement for checking every pointer.
Create: Obtain a userfaultfd with suitable permissions and nonblocking or close-on-exec flags as needed.
Negotiate: Use UFFDIO_API to discover supported features and operations.
Register: Register an eligible range for a supported fault mode with UFFDIO_REGISTER.
Receive: A handler reads the fault event while the accessing thread waits.
Resolve: Supply contents with UFFDIO_COPY or another supported resolution operation.
Continue: The waiting thread resumes after resolution makes its access possible.
The handler must not fault on memory whose resolution depends on itself. Reserve its stacks and buffers outside the managed range. Kernel configuration, security filters, and permissions can restrict creation. UFFD_USER_MODE_ONLY narrows the syscall interface to user-mode faults on supporting kernels. /dev/userfaultfd is another permission-controlled creation path where available. See userfaultfd before choosing supported mapping types and fault modes.
Going deeperRecognize a legacy mapping interface
remap_file_pages rearranged file-page offsets inside a virtual range. Linux deprecated it and replaced its original special implementation with emulation. It is a legacy interface, not a recommended optimization for new code. Use ordinary supported mappings and explicit ownership instead of adopting old book examples uncritically. See remap_file_pages(2).
Try it
1 min readOpen Map, share, and protect Linux memory and annotate your probe's maps before and after touching its pages. Compare private and shared mappings, record fault counts, and explain the guard-page termination. Use the practice pack to reason about alignment, copy-on-write, and memory-pressure evidence. Carry your observations into Write your own malloc; valgrind and ASan.
Key takeaways
- Virtual addresses describe a process view; physical pages supply its resident contents.
- A mapping can exist before any private physical page has been allocated.
- glibc can obtain allocator storage through both brk and mmap.
- MAP_PRIVATE isolates writes; MAP_SHARED exposes updates without providing synchronization.
- A normal page fault may allocate memory, install a cached page, or resolve copy-on-write.
- Permissions, locking, and residency answer different questions about the same pages.
- RSS counts shared pages repeatedly; USS estimates pages unique to one process.
- Mapping success does not guarantee that later memory pressure cannot kill the process.
Reflect
- Explain to a colleague, in two sentences: identify text, data, BSS, heap, stack, and mapped regions in a process map.
- Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
- What surprised you when you ran the commands yourself?
On a real board
Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.
References and further reading
- Official docsLinux kernel: process addresses and VMAs
- Official docsLinux kernel: the proc filesystem
- Official docsLinux kernel: overcommit accounting
- Official docsLinux kernel: userfaultfd
- Official docsLinux kernel: transparent huge pages
- Man pagemmap(2) and munmap(2)
- Man pagemlock(2) and mlockall(2)
- Man pagemallopt(3): glibc allocation controls
Official documentation first. Links checked when the lesson was published.