1 Structure

xv6 Structure

The xv6 codebase is organised into four distinct parts:

Makefile
  ↓
kernel/   builds the OS kernel
  ↓
user/     builds xv6 user programs
  ↓
mkfs/     builds fs.img using those user programs
  ↓
QEMU      boots kernel + fs.img

Kernel Reading Order

Order	File	Generated object	Kind	Purpose
1	kernel/types.h	—	Header	Basic integer/type aliases.
2	kernel/param.h	—	Header	Kernel-wide size limits.
3	kernel/memlayout.h	—	Header	Physical/virtual memory map.
4	kernel/riscv.h	—	Header	RISC-V registers, paging, interrupt helpers.
5	kernel/defs.h	—	Header	Cross-file kernel declarations.
6	kernel/kernel.ld	—	Linker script	Kernel memory layout at link time.
7	kernel/entry.S	kernel/entry.o	Assembly	First code after QEMU jump.
8	kernel/start.c	kernel/start.o	C	Early CPU setup before main.
9	kernel/main.c	kernel/main.o	C	Kernel initialization order.
10	kernel/spinlock.h	—	Header	Spinlock structure.
11	kernel/spinlock.c	kernel/spinlock.o	C	Short critical-section locking.
12	kernel/printf.c	kernel/printf.o	C	Kernel printing and panic.
13	kernel/string.c	kernel/string.o	C	Basic memory/string helpers.
14	kernel/kalloc.c	kernel/kalloc.o	C	Physical page allocator.
15	kernel/vm.h	—	Header	sbrk allocation mode constants.
16	kernel/vm.c	kernel/vm.o	C	Page tables and virtual memory.
17	kernel/proc.h	—	Header	Process, CPU, trapframe structures.
18	kernel/proc.c	kernel/proc.o	C	Processes, scheduling, sleep/wakeup.
19	kernel/swtch.S	kernel/swtch.o	Assembly	Low-level context switch.
20	kernel/trampoline.S	kernel/trampoline.o	Assembly	User/kernel trap transition code.
21	kernel/kernelvec.S	kernel/kernelvec.o	Assembly	Kernel-mode trap vector.
22	kernel/trap.c	kernel/trap.o	C	Trap, syscall, interrupt handling.
23	kernel/syscall.h	—	Header	Syscall number definitions.
24	kernel/syscall.c	kernel/syscall.o	C	Syscall dispatch and argument fetching.
25	kernel/sysproc.c	kernel/sysproc.o	C	Process-related syscalls.
26	kernel/uart.c	kernel/uart.o	C	Low-level serial device driver.
27	kernel/console.c	kernel/console.o	C	Console input/output layer.
28	kernel/plic.c	kernel/plic.o	C	External interrupt controller.
29	kernel/virtio.h	—	Header	Virtio disk protocol definitions.
30	kernel/virtio_disk.c	kernel/virtio_disk.o	C	Virtual disk driver.
31	kernel/buf.h	—	Header	Disk buffer structure.
32	kernel/bio.c	kernel/bio.o	C	Buffer cache and LRU block reuse.
33	kernel/fs.h	—	Header	On-disk filesystem format.
34	kernel/sleeplock.h	—	Header	Sleeping lock structure.
35	kernel/sleeplock.c	kernel/sleeplock.o	C	Locks that sleep while waiting.
36	kernel/file.h	—	Header	In-memory file/inode/device structs.
37	kernel/fs.c	kernel/fs.o	C	Inodes, directories, path lookup.
38	kernel/log.c	kernel/log.o	C	Filesystem transaction log.
39	kernel/file.c	kernel/file.o	C	Open-file table and file operations.
40	kernel/fcntl.h	—	Header	File open flags.
41	kernel/stat.h	—	Header	File metadata structure.
42	kernel/sysfile.c	kernel/sysfile.o	C	File-related syscalls.
43	kernel/pipe.c	kernel/pipe.o	C	Pipes for process communication.
44	kernel/elf.h	—	Header	ELF executable file format.
45	kernel/exec.c	kernel/exec.o	C	Load and run user programs.

User-Space Reading Order

Order	File	Kind	Purpose
1	user/user.h	Header	User-visible syscall and library declarations.
2	user/usys.pl	Generator	Generates user syscall stubs.
3	user/usys.S	Generated assembly	User-side syscall wrappers using ecall.
4	user/ulib.c	User library	Basic user-space helper functions.
5	user/printf.c	User library	User-space formatted printing.
6	user/umalloc.c	User library	Simple user-space memory allocator.

User Programs

Order	File	Kind	Purpose
1	user/init.c	User program	First user process.
2	user/sh.c	User program	xv6 shell.
3	user/ls.c	User program	List directory contents.
4	user/cat.c	User program	Print file contents.
5	user/echo.c	User program	Print arguments.
6	user/grep.c	User program	Search text.
7	user/wc.c	User program	Count lines, words, bytes.
8	user/mkdir.c	User program	Create directories.
9	user/rm.c	User program	Remove files.
10	user/ln.c	User program	Create hard links.
11	user/kill.c	User program	Kill a process.
12	user/stressfs.c	Test program	Stress filesystem behavior.
13	user/forktest.c	Test program	Stress process creation.
14	user/grind.c	Test program	Stress processes/filesystem/concurrency.
15	user/usertests.c	Test program	Broad xv6 test suite.

mkfs and Filesystem Image

mkfs is a host-side tool that runs on the build machine before xv6 boots. It packs compiled user binaries into fs.img, the virtual disk QEMU presents to xv6.

Order	File / Artifact	Kind	Purpose
1	kernel/fs.h	Shared format header	Defines xv6 on-disk filesystem layout.
2	mkfs/mkfs.c	Host tool	Creates fs.img using xv6 filesystem format.
3	user/_init etc.	RISC-V binaries	Compiled user programs inserted into fs.img.
4	fs.img	Disk image	Virtual disk passed to xv6 by QEMU.

Full Build-to-Boot Pipeline

Order	Step	Runs where?	Purpose
1	Makefile	Host	Coordinates the build.
2	Build kernel files	Host cross-compiler	Produces kernel/kernel.
3	Build user support files	Host cross-compiler	Produces user runtime objects.
4	Build user programs	Host cross-compiler	Produces user/_init, user/_sh, etc.
5	Build mkfs/mkfs.c	Host compiler	Produces host executable mkfs/mkfs.
6	Run mkfs/mkfs	Host	Packs user binaries into fs.img.
7	Start QEMU	Host	Creates virtual RISC-V machine.
8	Run kernel/kernel	QEMU/RISC-V	Boots xv6 kernel.
9	Run /init	xv6 user mode	Starts first user process.
10	Run /sh	xv6 user mode	Starts shell.

Runtime flow:

QEMU
  ↓
kernel/entry.S
  ↓
kernel/start.c
  ↓
kernel/main.c
  ↓
disk + filesystem init
  ↓
kernel/exec.c loads /init
  ↓
user/init.c starts /sh
  ↓
user/sh.c runs commands

Makefile

The Makefile coordinates three separate builds and then launches QEMU.

Build Pipeline

Step	Input	Linker Script	Output
Kernel build	kernel/*.c + kernel/*.S	kernel/kernel.ld	kernel/kernel
User build	user/*.c + usys.S	user/user.ld	user/_init etc.
mkfs (Host)	mkfs/mkfs.c	—	mkfs/mkfs
Filesystem image	mkfs/mkfs + user/_init etc.	—	fs.img
Boot	kernel/kernel + fs.img	—	QEMU launches xv6

Linking all kernel object files with kernel.ld produces three files:

File	Content
kernel/kernel	Linked kernel binary loaded by QEMU
kernel/kernel.asm	Mixed source/disassembly for inspection
kernel/kernel.sym	Address-to-symbol map for debugging

Every user program links against a small runtime library:

Object	Source	Role
ulib.o	ulib.c	String helpers and syscall wrappers
usys.o	generated usys.S	Syscall stubs (ecall wrappers)
printf.o	printf.c	User-space printf
umalloc.o	umalloc.c	User-space malloc/free

Notes:

• usys.S is generated by running usys.pl through Perl where each stub loads the syscall number into a7 and executes ecall.
• User programs are named with a leading underscore (user/_init) to avoid clashing with host tools. mkfs strips it when packing into fs.img.
• forktest omits printf.o and umalloc.o to stay small enough to max out the process table.

These are the compiled user programs packed into fs.img by mkfs:

Program	Purpose
_init	First user process started by the kernel
_sh	xv6 shell
_ls	List directory contents
_cat	Print file contents
_echo	Print arguments
_grep	Search text
_wc	Count lines, words, bytes
_mkdir	Create directories
_rm	Remove files
_ln	Create hard links
_kill	Kill a process
_zombie	Demonstrate zombie process behavior
_forktest	Stress process creation
_stressfs	Stress filesystem writes
_usertests	Broad xv6 test suite
_grind	Stress processes, filesystem, and concurrency
_logstress	Stress filesystem logging
_forphan / _dorphan	Test orphaned process behavior

Toolchain

Tool	Role
$(CC)	Compiles C and preprocessed .S assembly
$(LD)	Links object files into binaries
$(OBJDUMP)	Generates .asm and .sym files for inspection
gcc (host)	Compiles mkfs/mkfs.c — runs on build machine, not RISC-V

Compiler Flags

Flag	Purpose
-Wall -Werror	Warnings as errors
-O	Basic optimization
-ggdb	GDB-friendly debug info
-gdwarf-2	DWARF v2 debug format
-fno-omit-frame-pointer	Keep frame pointers for stack traces
-march=rv64gc	Target 64-bit RISC-V with standard extensions
-mcmodel=medany	Addressing for code linked at 0x80000000, not near zero
-MD	Emit .d dependency files for incremental builds
-ffreestanding	No hosted C environment assumptions
-nostdlib	Do not link standard library or startup files
-fno-common	Catch duplicate global definitions at link time
-fno-builtin-*	Prevent GCC substituting xv6’s own memcpy, printf etc. with libc versions
-fno-stack-protector	No stack canary — kernel has no runtime support for it
-fno-pie -no-pie	Fixed-address binaries; xv6 does not use position-independent code
-I.	Include headers relative to project root

Linker Flags

Flag	Purpose
-z max-page-size=4096	Align ELF segments to 4 KiB, matching xv6’s page size.

QEMU Launch

Spec	Value
Machine	virt (generic RISC-V virtual board)
CPU	riscv64
Cores	3
RAM	128M
Kernel	kernel/kernel
Disk	fs.img via virtio-blk
Display	None (-nographic, serial console only)

The Linker Script

The kernel/kernel.ld tells the linker how to lay out the final kernel binary in memory. It defines where code goes, in what order, and what symbols the rest of the kernel can use to find section boundaries.

Memory Layout

The linker arranges the kernel into sections in this order, starting at 0x80000000:

Section	Contents	Notes
.text	Kernel code	_entry placed first, then all other code
.rodata	Read-only data	Constants, string literals, aligned to 16 bytes
.data	Initialized globals	Non-zero globals, stored in the binary
.bss	Zero-initialized globals	Not stored in binary; kernel zeroes at startup

.text:

• Executable code only, read and execute permissions.
• _entry lands at exactly 0x80000000.
• Trampoline is carved out at the end, aligned to a 4 KiB page boundary
• user-kernel transitions and must be mapped at the same virtual address in every page table.
• trampoline.S declares .section trampsec, which the linker places here.

.rodata:

• Read-only.
• Aligned to 16 bytes.
• Contains string literals and constant arrays that must not be modified at runtime.
• Separate from .data so the OS can enforce read-only page permissions.

.data:

• Read-write.
• Aligned to 16 bytes.
• Holds initialized globals with non-zero starting values, embedded in the binary and copied into RAM at load time.

.bss:

• Read-write.
• Aligned to 16 bytes.
• Holds zero-initialized globals.
• Not stored in the binary: the linker records only the size and the kernel zeroes the region at startup.

Note:

The 16-byte alignment across data sections ensures efficient memory access. On a 64-bit RISCV system, 16 bytes covers two 64-bit registers, which compilers exploit for multi-word loads and stores. Unaligned access can cause hardware exceptions or significant slowdowns on some architectures.

Symbols

The linker script exports two symbols the kernel uses at runtime:

Symbol	Meaning
etext	Address of the end of the text section
end	Address of the end of the entire kernel image
_trampoline	Start address of the trampoline page

The .rodata

Size: 2,080 bytes.

Contents:

• Lock debug names: passed to initlock() and initsleeplock() at startup.
• Panic and error messages: invariant violation and error detection strings across the kernel.
• Boot messages: printed during initialization and secondary hart startup.
• Format strings: used in trap handling and process dumps.
• Path strings: "/" and "/init" used during first process setup.
• Compiler-promoted immutable tables: digit lookup, process state names, syscall dispatch table.

Note: No const globals exist in xv6 everything here was placed by the compiler.

Permissions: xv6 maps this range as PTE_R | PTE_W in the kernel page table, so read-only is by convention only.

The .data

Size: 24 bytes almost nothing.

Contents:

• Thenextpid = 1 in proc.c is the only meaningful non-zero global.
• A static local first in kalloc.c to detect the first kfree call.

Why so small: most kernel state is zero-initialized and lives in .bss; string data lives in .rodata.

The .bss

Size: 103,224 bytes the bulk of xv6’s kernel state.

Contents: all global structs declared without explicit initializers. Not stored in the binary; the kernel zeroes this region at startup.

Symbol	Size	First used by	Purpose
stack0	32 KiB	entry.S / start.c	Boot stack for all CPUs before main
cons	168 B	consoleinit()	Console input buffer and lock
tx_lock	24 B	consoleinit()	UART transmit spinlock
tx_chan	4 B	consoleinit()	UART transmit sleep channel
tx_busy	4 B	consoleinit()	UART transmit busy flag
pr	24 B	printfinit()	printf serialization lock
panicked	4 B	printfinit()	Flag set when kernel has panicked
panicking	4 B	printfinit()	Flag set while panic is in progress
kmem	32 B	kinit()	Physical page free list and its lock
kernel_pagetable	8 B	kvminit()	Pointer to the kernel page table
proc	23 KiB	procinit()	Process table 64 process slots
cpus	1 KiB	procinit()	Per-CPU state (current process, scheduler context, lock depth)
wait_lock	24 B	procinit()	Condition lock for parent/child wait coordination
pid_lock	24 B	procinit()	Protects nextpid counter
tickslock	24 B	trapinit()	Serializes access to the ticks counter
ticks	4 B	trapinit()	Wall-clock tick counter incremented by timer interrupts
bcache	34 KiB	binit()	Buffer cache 30 LRU disk block buffers
sb	32 B	iinit()	Superblock filesystem geometry and metadata
itable	6.7 KiB	iinit()	In-memory inode table
log	168 B	iinit()	Filesystem transaction log state
ftable	4 KiB	fileinit()	Open file table shared across all processes
devsw	160 B	fileinit()	Device switch table mapping major numbers to read/write handlers
disk	320 B	virtio_disk_init()	Virtio disk driver state and descriptor ring
initproc	8 B	userinit()	Pointer to the init process
started	4 B	After CPU 0 init	Signals secondary CPUs that CPU 0 has finished initialization

The Free Memory

Starts at end (linker symbol at the end of .bss) and extends to PHYSTOP (0x88000000).

• Not part of the kernel binary — raw physical RAM managed at runtime.
• kalloc.c tracks it as a free list of 4 KiB pages.
• kinit() walks from end to PHYSTOP and adds each page to the free list.
• kalloc() pops a page from the free list; kfree() pushes one back.

Allocation	kalloc() site	kfree() site	Pages	What gets a page
Kernel page table	kvmmake()	Never	102	1 root + 3 L1 + 98 L0 nodes for the Sv39 tree
Kernel stacks	proc_mapstacks()	Never	64	One per process slot, called inside kvmmake()
Virtio rings	virtio_disk_init()	Never	3	Descriptor ring, available ring, used ring for disk DMA
Trapframe	allocproc()	freeproc()	1 per process	Per-process trap register save area
User page table	proc_pagetable()	proc_freepagetable()	1 per process	Root page table, called inside allocproc()
User memory	uvmalloc()	uvmdealloc() / uvmfree()	varies	User process address space pages during exec and sbrk
Intermediate page table nodes	walk() with alloc=1	freewalk()	varies	L1 and L0 pages as user page tables are built
Fork copy	uvmcopy()	uvmfree() on child exit	1 per mapped page	Physical page copies for child during fork
Page fault	vmfault()	uvmfree() on process exit	1 per fault	On-demand page on fault
Pipe buffer	pipealloc()	pipeclose()	1 per pipe	Pipe kernel buffer
exec argv	sys_exec()	After copy / on error	varies	Temporary argument string pages during exec