From power-on to a prompt: the boot chain
4 min readWhen you press the power button on any computer, nothing useful is configured. The memory is not organised, there is no stack for C code, and no process exists. The code that runs first must create, step by step, everything that the rest of the system takes for granted. This lesson follows that process for xv6 on QEMU, from the first instruction to the $ prompt, so that the kernel stops being a black box.
The previous lesson gave you the vocabulary: harts, privilege modes, CSRs, mret and ecall. Now you use it. The whole journey is only a few hundred lines of code, and every line has a reason. The chain looks like this:
- QEMU copies the kernel file into simulated RAM and starts every hart at address 0x80000000.
_entryinkernel/entry.Ssets a stack and callsstart().start()inkernel/start.cconfigures machine mode and "returns" into supervisor mode atmain().main()inkernel/main.cinitialises the kernel, one subsystem at a time.userinit()creates process 1 with a tiny hand-written program,initcode.scheduler()picks that process, and it runs in user mode for the first time.initcodemakes anexecsystem call that replaces it with/init, and/initstartssh.
Real products add stages before this one. On a typical RISC-V board, a boot ROM in the chip loads firmware, firmware loads a boot loader such as U-Boot, and the boot loader loads Linux. On an ARM board such as a Raspberry Pi the first stages are different code but the same idea. xv6 removes every stage except the kernel itself, because QEMU is started with -bios none and -kernel kernel/kernel. That is a teaching simplification, and a later section explains exactly what it removes.
The software is also small enough to read in one sitting. xv6 began at MIT as a re-implementation of the sixth edition of Unix, written for the operating systems course in the mid-2000s, and its RISC-V version, which this course uses, arrived around 2019. The code you read is the real start-up path of a real, small kernel, not a pseudo-code diagram of one.
One more idea before the details. The typical output of a boot looks like this (yours will differ in the order of the hart lines, because the harts run in parallel):
xv6 kernel is booting
hart 2 starting
hart 1 starting
init: starting sh
$
Every line comes from a printf call somewhere in main.c or user/init.c. By the end of the lesson you can name the function that prints each one, and you can stop the machine just before it does.
Where everything lives: the memory map and the linker script
4 min readTwo files fix the geometry of the machine for xv6. kernel/memlayout.h lists the physical addresses of the devices and the RAM on QEMU's virt machine. kernel/kernel.ld is the linker script that tells the linker where to place the kernel's code and data. They must agree, and understanding both removes most of the mystery about "where things are".
Start with the physical map, which is partly a repeat of the last lesson. Low addresses hold devices that the kernel reads and writes through ordinary loads and stores, which is called memory-mapped input and output. RAM starts at 0x80000000, which xv6 names KERNBASE. The reason the kernel is not at address zero is simply that the range below 0x80000000 on the virt machine belongs to devices and a small reset-code area, not to RAM. PHYSTOP is the address where xv6 stops using memory. xv6 does not ask the hardware how much RAM it has. It assumes 128 MiB, so PHYSTOP is KERNBASE plus 128 MiB, which is 0x88000000, and the QEMU command line passes -m 128M to match.
Try the map below. Type an address and see which region owns it, then check the same address against memlayout.h in your tree.
The linker script
A linker script is A linker script is a file that tells the linker which input sections to combine into which output sections, and at which addresses they are placed. When the compiler turns each C and assembly file into an object file, every address inside it is relative. The linker assigns final addresses. xv6's script starts by naming the entry symbol _entry and setting the location counter to 0x80000000, then lists the output sections in order: .text (code), .rodata (constants), .data (initialised variables) and .bss (variables that start at zero).
Three details in the script matter for boot. First, entry.o is the first object file on the linker command line in the Makefile, so the very first bytes at 0x80000000 are the instructions of _entry. If another file came first, the hart would execute that file's first instruction instead, and boot would fail without a message. Second, inside .text the script aligns a page boundary, starts a section called trampsec there, and asserts that the trampoline code fits in exactly one 4096-byte page; that page is used later to cross between user and kernel mode. Third, the script defines two symbols that C code uses: etext, the end of the code, and end, the first address after all of the kernel's data. The free memory allocator starts at end.
You can check all of this on your own build without trusting any description. Run these in the xv6 directory after make:
riscv64-linux-gnu-readelf -h kernel/kernel | grep -i entry
riscv64-linux-gnu-nm kernel/kernel | grep -E ' (_entry|etext|end|stack0)$'
The first command prints the ELF entry point, which should be 0x80000000. The second prints the addresses of the symbols named above. Your cross-tool prefix may differ if you installed riscv64-unknown-elf- tools. The addresses of etext and end depend on your build, so write down yours rather than expecting a fixed number.
The ELF file records where each segment must be loaded, and QEMU's -kernel option reads those load addresses. That is why linking at 0x80000000 and loading at 0x80000000 agree without any extra boot loader.
Machine mode first: entry.S and start()
5 min readAll harts begin at _entry at the same moment. Each one has its own registers but they share memory, so any shared setup must be careful. The assembly file is very short, and you read it in the previous lesson as an example of a CSR in use. Here is what it accomplishes in order.
1a stack for every hart
C needs a stack, and at reset sp holds nothing useful. kernel/start.c declares a global array named stack0, large enough for 4096 bytes for each hart. _entry loads the address of stack0, reads this hart's number from the mhartid CSR, and sets sp = stack0 + (hartid + 1) * 4096. The stack grows down, so each hart begins at the top of its own 4096-byte slice. Hart 0 uses the first slice, hart 1 the second, and so on. After that, _entry calls start. If start ever returned, a short loop spins forever, which makes a bug visible as a hang rather than as random execution.
2prepare the fake return
start() runs in machine mode, the only mode in which it is allowed to touch the CSRs that control privilege. Its job is to set up the machine and then execute mret, the "return from machine-mode trap" instruction. No trap has happened. The trick from the previous lesson is to write the values that mret expects: the mode in the MPP field of mstatus and the address in mepc. Then mret behaves as though it were returning from a machine-mode interrupt into supervisor mode.
Choose the next mode: it clears the MPP field of mstatus and sets it to supervisor, so mret will enter supervisor mode.
Choose the next address: it writes the address of main into mepc, so mret will jump there.
Turn paging off: it writes 0 to satp, because the kernel's page table does not exist yet and addresses stay physical.
Delegate traps: it writes medeleg and mideleg so that exceptions and interrupts are handled in supervisor mode, not machine mode.
Open memory: it programs the physical memory protection registers so that supervisor mode may access all of physical memory.
Arm the timer: it enables the supervisor timer through the sstc extension and sets the first stimecmp value.
Remember the hart: it copies mhartid into the tp register, so supervisor code can learn which hart it is on.
Execute mret: the hart switches to supervisor mode and starts running main.
The MPP field is two bits wide. The value 1 means supervisor mode, 3 means machine mode, and 0 means user mode. Toggle the bits below to see how a single field decides what mret does. The widget holds a mstatus value with only that field set to 1:
The delegation step needs one sentence of explanation. By default, every trap goes to machine mode. xv6 wants its kernel, running in supervisor mode, to handle page faults, system calls and device interrupts directly. With medeleg and mideleg set, the hardware sends those traps to the supervisor trap vector. The physical memory protection (PMP) step exists because machine mode is allowed to restrict what lower modes can touch, and with no PMP entries a lower mode may be denied all access. One PMP entry that covers all memory gives the kernel everything.
The timer detail is the one that changed over the history of xv6. Older versions programmed the CLINT's machine-mode timer registers and ran a machine-mode interrupt handler. The version this course uses relies on the sstc extension, so supervisor code writes its own compare value into the stimecmp CSR and receives its own timer interrupts, and start() only has to enable the feature. You will see how a tick is handled in a later lesson. Read your tree's start.c: it is a short file, and a handful of CSR writes there are the entire machine-mode part of xv6.
Going deeperWhy the stack slices are 4096 bytes, and what happens if one overflows
The 4096 bytes in stack0 are only the boot stack and the stack each CPU's scheduler loop uses later. They are not the stacks of processes, which are allocated separately. Because the slices sit next to each other with no guard between them, a deep call chain on hart 0 before paging is on would silently overwrite the stack of hart 1. Nothing detects it. That is acceptable for the early boot path, where the call depth is tiny, and it is the reason xv6 later gives process kernel stacks a guard page in the page table, which the page table section covers.
main() and the initialisation order
4 min readWhen mret lands in main(), the hart is in supervisor mode, paging is still off, and tp holds the hart number. Every hart runs main, but only one of them should do the shared setup. The code checks cpuid(), which reads tp, and lets hart 0 initialise the machine while the others wait. The sequence below is the order of calls on hart 0. Open kernel/main.c in your tree and compare. The name of the print initialiser is printfinit in the revision this course builds, and some branches name it differently, so trust your own file.
| Call | What it sets up | Why it must come here |
|---|---|---|
consoleinit |
The UART and the console device | Nothing can print before it |
printfinit |
The lock that serialises printing | Safe output from several harts |
kinit |
The free list of physical pages | Almost everything after it allocates memory |
kvminit |
Builds the kernel page table | Needs the allocator |
kvminithart |
Loads satp on this hart and enables paging | Needs the finished table |
procinit |
The table of process structures and their kernel stacks | Needs paging data and the allocator |
trapinit |
Trap-related locks and global state | Before any trap handling |
trapinithart |
Points this hart's stvec at the kernel trap handler | Without it, a trap goes to an undefined address |
plicinit |
Sets interrupt priorities for the UART and disk | Before enabling them |
plicinithart |
Enables those interrupts for this hart | Per-hart, so it runs on every hart |
binit |
The buffer cache for disk blocks | Before any file system use |
iinit |
The cache of in-memory inodes | Before any file lookup |
fileinit |
The table of open files | Before any open |
virtio_disk_init |
Finds and sets up the virtio disk | Before any disk read |
userinit |
Creates the first process | Needs memory, paging and the process table |
scheduler |
Never returns: run processes | Last, because it takes over the hart |
After the banner prints, main sets a flag called started once hart 0 finishes, after a memory barrier so that the other harts see all of the earlier writes first. Each other hart spins until it sees the flag, prints its own "hart N starting" line, and then calls only the three per-hart functions: kvminithart, trapinithart and plicinithart. Everything else is global state that hart 0 already built. Then every hart calls scheduler.
kinit and kfree: giving the kernel memory
The allocator keeps free physical pages on a free list. A free list is a linked list of unused blocks, where each block stores the pointer to the next block inside its own unused bytes. No separate bookkeeping array is needed, because a free page has nothing else to hold.
kinit initialises a lock and then calls freerange from the address end, the first byte after the kernel image, up to PHYSTOP. freerange rounds the start up to a page boundary and calls kfree on every 4096-byte page in the range. kfree first fills the page with a junk byte value, so any code that keeps using a freed page reads garbage and fails quickly, then pushes the page on the front of the list. kalloc pops the front page. Both take the allocator's spinlock, because several harts allocate at once.
The idea in a few lines of my own code, without xv6's locks or checks:
// sketch of a free list (not xv6's code)
struct run { struct run *next; };
static struct run *freelist;
void page_free(void *page) {
struct run *r = (struct run *)page;
r->next = freelist; // the page itself stores the link
freelist = r;
}
void *page_alloc(void) {
struct run *r = freelist;
if (r)
freelist = r->next;
return r; // NULL means out of memory
}
With 128 MiB of RAM and 4096-byte pages, there are 32768 pages in total. The kernel image takes the first few dozen, so roughly 32,000 are free after kinit. That is also the number a boot-time counter would print in the lab.
Building the kernel page table: kvminit and kvminithart
5 min readUntil now every address has been physical. The kernel needs paging for three reasons that matter at boot: to give each kernel stack a guard page, to map the trampoline page at a fixed high address, and to prepare a model that user processes will share. xv6 uses the Sv39 scheme: 39-bit virtual addresses, three levels of page table, and 4096-byte pages. The previous lesson showed that satp selects the mode and the root table.
1allocate and fill the table
kvminit calls a helper that asks kalloc for one page, zeroes it, and then calls a mapping function once for each range the kernel needs. The mapping function walks the three levels, allocating intermediate table pages with kalloc as it goes. The ranges are these:
- The UART and the virtio disk, each mapped to the same virtual and physical address, readable and writable.
- The PLIC registers, mapped the same way.
- The kernel's code, from
KERNBASEtoetext, readable and executable but not writable. - The rest of RAM, from
etexttoPHYSTOP, readable and writable. This covers kernel data and all free pages. - The trampoline page, mapped a second time at the top of the virtual address space, readable and executable.
- One kernel stack for each process slot, each with an unmapped guard page next to it.
The first four mappings use direct mapping: Direct mapping, also called identity mapping, makes a virtual address equal to the physical address it translates to. This is the design choice that makes enabling paging safe. When satp is written, the program counter still holds a physical address such as 0x80001234. Under the new table, the virtual address 0x80001234 translates to the physical address 0x80001234, so the next instruction is fetched from the same place and the hart does not notice. It also lets the kernel read any physical page by using its physical address as a pointer, which is what kalloc returns.
2guard pages and kernel stacks
Every process needs its own kernel stack, the stack used when it runs inside the kernel. xv6 maps these stacks high in the address space, each followed by a page that is deliberately left unmapped. A guard page is A guard page is an unmapped or inaccessible page placed next to a stack so that an overflow causes an immediate page fault instead of silent corruption. If kernel code overflows its stack by a page, the next store hits an invalid page table entry and the hart takes a fault, which becomes a panic. A crash is better than corrupting a neighbour. The same physical stack page is also reachable through the direct map, but the high mapping is the one the kernel uses, because only it has a guard. The exact virtual address of each stack comes from a macro in memlayout.h; compute one yourself in the lab instead of memorising a formula.
3switch it on
Building the table does not use it yet. kvminithart runs on every hart, because satp belongs to the hart. It does three things in a fixed order: an sfence.vma instruction first, to make sure the earlier writes to the table memory have finished; then a write to satp with mode 8 (Sv39) and the page number of the root table; then another sfence.vma. The second fence clears the TLB, The translation lookaside buffer is a small cache inside the hart that remembers recent virtual-to-physical translations. so no old translation survives. After this, every address the hart uses is virtual.
In the hand-off between harts, hart 0 builds the table once, and each other hart only writes its own satp with the same root. That is why kvminithart is a separate call from kvminit.
The first process: userinit, initcode and /init
4 min readThe kernel has memory, paging and a process table, but no process. The first one cannot be made by fork, because there is nothing to copy. userinit creates it by hand.
1allocate a process
userinit calls allocproc, which finds an unused slot in the process table, gives it the next process ID (the first is 1), and marks it in use. Then it allocates a physical page for the trapframe, the structure where user registers are saved when a trap enters the kernel, and creates the process's page table. That page table starts nearly empty, but already contains two mappings the kernel always adds: the trampoline page and the trapframe page, both near the top. Finally, allocproc prepares the process's saved kernel context so that the first time the scheduler switches to it, swtch will "return" into a function called forkret, on the new kernel stack. This is the same swtch you read in the previous lesson, which restores ra and sp.
2load initcode
initcode is a program of a few instructions, written in assembly in user/initcode.S. The build turns it into raw bytes and links it into the kernel image as an array. userinit allocates one page, maps it at virtual address 0 in the new process's page table with user permissions, and copies the bytes there. It then sets the size to one page, sets the saved user program counter in the trapframe to 0, and sets the saved user stack pointer to the end of that page. Finally it names the process initcode, sets its current directory to the root, and marks it runnable.
What does initcode do? Three things. It loads the string /init and an argument array into registers, loads the number of the exec system call into a7, and executes ecall. If exec ever returns, which means it failed, it calls exit, with a loop as a last resort. The whole program is about a dozen instructions.
3from initcode to init
When exec runs on behalf of initcode, the kernel reads the file /init from the disk, checks that it is an ELF executable, builds a new page table, copies the program's segments into it, builds a user stack with the arguments, and frees the old page table. The process keeps its process ID and its place in the process table, and its program is now /init, built from user/init.c.
init does a short sequence. It tries to open the file console, and if that file does not exist it creates the device node and opens it again. It duplicates that descriptor twice, so file descriptors 0, 1 and 2 (standard input, output and error) all point to the console. Then it loops forever: it forks, the child executes sh, and the parent waits. If the shell exits, init starts another one. It also reaps any orphaned process whose parent died. The line init: starting sh comes from here. That is why the shell is not process 1: it is a child of process 1, restarted whenever it ends.
Going deeperNewer upstream revisions may remove initcode
The description above matches the xv6 book and the tree this course builds. The upstream xv6-riscv repository continues to change, and some recent revisions have renamed system call implementations with a k prefix and may start /init from the kernel without a hand-written initcode. If user/initcode.S is missing from your tree, or userinit looks different, that is the reason. Read userinit and forkret in kernel/proc.c in your own copy before relying on any detail in this section, and record any difference in your lab notes. The concept, a first process created by hand that becomes /init, stays the same.
The scheduler and the first switch to user mode
4 min readscheduler never returns. It runs on each hart's boot stack, the slice of stack0 from the first section, and it holds a per-hart structure called a CPU record, with a context field that swtch can save into. Its loop, in outline:
// the shape of the loop (not xv6's exact code)
for (;;) {
intr_on(); // let device interrupts in
for (each process p in the table) {
acquire(&p->lock);
if (p->state == RUNNABLE) {
p->state = RUNNING;
cpu->proc = p;
swtch(&cpu->context, &p->context); // run p until it gives up the hart
cpu->proc = 0;
}
release(&p->lock);
}
}
Interrupts have been off since boot, because main never enabled them. scheduler calls intr_on so that device interrupts can arrive. Without that, a loop that finds nothing to run would never be interrupted by the timer or the disk. All the harts run this loop and compete for the process table, which is why each process has its own lock.
Find process 1: scheduler sees initcode's slot marked runnable, takes its lock and marks it running.
swtch: it saves the scheduler's callee-saved registers and loads the saved context made by allocproc, so ra points to forkret and sp to the new kernel stack.
forkret: the hart returns into it, releases the process lock, and, only on the very first call, initialises the file system.
usertrapret: forkret calls it, and it sets stvec to the user trap entry, fills the trapframe's kernel fields, and sets sstatus and sepc.
userret: it is called through the trampoline, switches satp to the process's page table, and restores the registers from the trapframe.
sret: the hart drops to user mode and starts executing at the saved user pc, which is 0, the first instruction of initcode.
Why does forkret do the file system setup? Because initialising the file system reads the disk, and a disk read needs to sleep while the device works. Only a process can sleep. At boot, main is not a process, so it cannot call that code. The first process runs it as its first act, in kernel mode, before it ever reaches user mode. This is a good example of the order a kernel must find: it cannot start the file system until a process exists, and it cannot start the first process without memory.
The states of that first process are the same states every xv6 process passes through. Use the machine below to follow process 1 through the kernel's states. Start from unused, then send events:
Tracing the boot with GDB
You do not need to take any of this on trust. Open two terminals in the xv6 directory. In the first, run make qemu-gdb; QEMU starts and waits, frozen before its first instruction. In the second, run gdb-multiarch kernel/kernel. The Makefile generates a .gdbinit that connects to QEMU, and GDB may need to be told to trust that file the first time; it prints a message if so. Then set one breakpoint for each milestone:
(gdb) break _entry
(gdb) break start
(gdb) break main
(gdb) break userinit
(gdb) break scheduler
(gdb) break sys_fork
(gdb) continue
Each continue stops at the next milestone. The first stop is at 0x80000000, in _entry. Use info registers pc sp to see that sp has not been set up yet, and info threads to see that QEMU shows one thread for each hart. Do not be surprised if a breakpoint hits on a different hart than the one you were stepping; use thread 2 or thread 3 to move between them. At main you can print $satp and see zero, then step past kvminithart and print it again. The breakpoint at sys_fork stops the first time init forks the shell. Because the machine is deterministic apart from the ordering of the harts, the sequence you record is the real boot path.
Under the hood: the trampoline, the first ecall and xv6 without OpenSBI
5 min readWhy the trampoline must sit at the same address in both tables
When a user process traps, the hart arrives at the address in stvec with the user page table still in satp. The kernel needs to switch to the kernel page table, but the instruction that does csrw satp is itself being fetched through whichever table is active. After the write, the next instruction fetch uses the new table. If the code page were not mapped at that address in the new table, the hart would fault on its very next instruction.
xv6's answer is the trampoline: The trampoline is one page of code, containing uservec and userret, mapped at the same top virtual address in the kernel page table and in every user page table. The kernel maps it at TRAMPOLINE, one page below the highest virtual address, MAXVA. Each user page table maps the same physical page there too, without the user-accessible flag, so user code cannot read it. The trapframe sits just below it, mapped in user tables only. After uservec writes satp, the program counter is still on the trampoline page, and the same virtual address now translates through the kernel table to the same physical instructions. The same trick runs in reverse inside userret.
The first ecall, with no program loaded yet
At the end of the first-switch steps, initcode executes ecall and the hart traps. How does that work when no real user program has ever been loaded? Because every ingredient was prepared by hand. usertrapret had already set stvec to the address of uservec on the trampoline page, which is mapped in the new process's page table. ecall therefore finds a valid trap vector. uservec saves all registers into the trapframe at its fixed virtual address, switches to the kernel stack and page table recorded there by usertrapret, and calls usertrap. The kernel sees cause 8, an ecall from user mode, advances the saved pc by 4, and looks up the number in a7, which is the exec call. The file system is already initialised because forkret did it, so /init can be read. Nothing about the trap path depends on a user program existing, only on the process, its page table, the trampoline and the trapframe. These are all made by allocproc and userinit.
Why xv6 has its own machine-mode code, and what OpenSBI would replace
On most RISC-V systems, machine mode is occupied by firmware. The standard interface between that firmware and a supervisor-mode kernel is the Supervisor Binary Interface, SBI, and the common open implementation is OpenSBI. A Linux kernel on RISC-V never writes mstatus or medeleg; it calls the firmware with ecall for services such as setting a timer or starting another hart. OpenSBI does, in a fuller and more careful way, what entry.S and the machine-mode half of start.c do for xv6: set up a stack for each hart, configure delegation and PMP, set up timers, and jump into supervisor mode with the hart number in a0 and a pointer to the hardware description in a1.
xv6 on QEMU -bios none |
Typical Linux on RISC-V | |
|---|---|---|
| Code in machine mode | entry.S and start() (a few dozen lines) |
OpenSBI or other firmware, resident after boot |
| Kernel load address | 0x80000000 |
Usually after the firmware, commonly at an offset such as 0x80200000 |
| Learns the hardware from | Constants in memlayout.h |
A device tree passed in a1 |
| Starting the other harts | All start at once | Firmware starts one; the kernel asks for the rest through SBI calls |
| Timer | Kernel writes stimecmp after start() enabled sstc |
SBI timer call, or stimecmp if the platform has sstc |
This explains the linker script. With OpenSBI present, the firmware occupies the start of RAM, so a supervisor kernel cannot sit at 0x80000000; it has to be linked and placed higher. xv6 has no firmware to make room for, so it can claim the very first RAM address. If you ever want to run xv6 under OpenSBI, you would change the load address, drop the machine-mode code, replace direct timer setup with SBI calls and read the device tree instead of memlayout.h. That is a good exercise in understanding what the firmware stage is for.
Going deeperHow Linux on RISC-V picks one boot hart
Under OpenSBI on a multi-hart machine, all harts come out of reset together. The firmware makes the harts compete for a single "lottery" flag. The winner continues to boot the kernel. The losers wait in firmware for a request. Linux, entered in supervisor mode on the winner, later asks firmware, through the SBI hart state management calls, to start each of the others and tells it where to begin. xv6 skips the lottery: every hart enters main, and a software test on cpuid() == 0 plus the started flag provide the same ordering with ordinary memory. Check the Linux documentation on RISC-V boot requirements, listed in the references, for the register conventions at kernel entry.
Common boot mistakes and how to diagnose them
3 min readThe hardest property of boot bugs is silence. Before consoleinit runs there is no output, and before trapinithart runs there is no trap handler, so a fault has nowhere to go. These are the failures learners meet most often when they modify the boot path.
Blank screen and no output at all. Symptom: QEMU starts, prints nothing and never exits. Cause: the first instruction at 0x80000000 is not _entry, usually because the linker script, the Makefile object order or a changed KERNBASE no longer agree. Diagnose: run readelf -h kernel/kernel and compare the entry point with 0x80000000, then in GDB stop at the start and run x/4i $pc. Fix: keep kernel.ld, memlayout.h and the QEMU command line consistent, and keep entry.o first.
A hang with the program counter near zero. Symptom: boot stops partway, and in GDB $pc is a tiny address. Cause: a trap occurred before trapinithart set stvec, so the hart jumped to whatever stvec held at reset (the RISC-V spec leaves it unspecified; under QEMU it is zero), where there is no code. A typical trigger is a pointer bug or a wrong PHYSTOP that makes a store fall outside real RAM. Diagnose: print $scause, $sepc and $stval in GDB, which hold the real fault even though the hart is lost. Fix: correct the access, and remember that an early-boot fault is reported only by these registers.
Garbled or missing messages from code you added. Symptom: your new milestone line prints nothing, hangs, or appears mixed with other output. Cause: you called printf before consoleinit and printfinit, so the UART or the printing lock was not ready, or several harts printed at the same time. Diagnose: move the call after printfinit and gate it with a hart check. Fix: add milestone prints after those two calls, and print only from the hart doing the shared setup.
Allocator failure early. Symptom: a panic or a crash the first time something calls kalloc. Cause: you used kalloc before kinit, so the free list was empty and the function returned a null pointer. Diagnose: break on kalloc and print the list head in kmem. Fix: reorder so the allocator is initialised before anything that allocates; kinit comes first for exactly this reason.
Reading a trap fingerprint. For any unexplained stop, write down three values: scause says which kind of fault it was (for example 5 is a load access fault and 7 a store access fault), sepc says which instruction, and stval says which address. Then use x/i $sepc and the kernel.asm listing to find the C line.
Try it
1 min readStart the lab: boot xv6 under GDB, set breakpoints at _entry, start, main, userinit and the first fork, and record the real order in which they fire. Then add one message at a boot milestone, rebuild, and prove the new line appears in the right place. Finish by drawing a timeline of the boot with the registers each stage writes: satp, stvec and sstatus. Open the practice pack when you want questions that check what you saw on your own machine.
Key takeaways
- QEMU loads the kernel ELF at 0x80000000 and every hart starts in machine mode at _entry, with paging off and no stack.
- entry.S gives each hart a 4096-byte slice of stack0; start() then prepares an mret that lands in supervisor mode at main().
- main() runs the shared setup on hart 0 only, in a fixed order, while the other harts wait for a flag and then do their per-hart setup.
- kinit puts every 4096-byte page between the end of the kernel image and PHYSTOP on a free list threaded through the pages themselves.
- kvminit builds a direct-mapped kernel page table plus the trampoline and guard-paged kernel stacks; kvminithart loads satp and flushes the TLB.
- userinit builds process 1 with a hand-written initcode that makes the exec system call; exec loads /init, which starts sh.
- The scheduler's first swtch lands in forkret, which finishes file system setup and returns to user mode through the trampoline.
- kernel.ld, memlayout.h and GDB together answer where anything lives, and a breakpoint on each stage proves the order.
Reflect
- Explain to a colleague, in two sentences: recall the order of the xv6 boot chain from QEMU's load address 0x80000000 to the first shell prompt.
- Which objective are you least sure about? Re-read that section, then do its exercise in the practice pack.
- What surprised you when you ran the commands yourself?
On a real board
Repeat the key commands on your Raspberry Pi or lab board over SSH or the serial console. Note anything that behaves differently from your PC, such as missing tools, a BusyBox shell or a read-only filesystem.
References and further reading
- Official docsxv6-riscv source repository (MIT PDOS)
- Official docsMIT 6.1810 Operating System Engineering: course materials and xv6 book
- SpecificationThe RISC-V Instruction Set Manual, Volume II: Privileged Architecture
- SpecificationRISC-V Supervisor Binary Interface (SBI) specification
- Official docsOpenSBI: RISC-V open source supervisor execution environment
- Official docsQEMU documentation: RISC-V virt board
- Official docsLinux kernel documentation: RISC-V boot requirements
- Official docsGDB manual: debugging with GDB
Official documentation first. Links checked when the lesson was published.