01 Introduction

Introduction and Essential Concepts

System Programming Foundation

• System programming is the practice of writing system software that interfaces directly with the kernel and core system libraries, distinguishing it from application programming which relies on high-level library abstractions.
• System software encompasses environments such as shells, text editors, compilers, debuggers, system daemons, and network servers.
• Modern Linux differentiates itself from traditional Unix systems by supporting additional system calls, distinct behaviors, and unique features, though the core API remains Unix-based.
• Three cornerstones form the foundation of Linux system programming: system calls, the C library, and the C compiler.
• System Calls (syscalls): Function invocations made from user space into the kernel to request operating system services or resources.
  • User-space applications cannot execute kernel code directly; they must trap into the kernel via a well-defined mechanism, such as a software interrupt (e.g., executing the int instruction with a value of 0x80 on i386).
  • The application specifies the target system call by placing its number (starting at $0$) into a designated machine register (e.g., eax) before issuing the interrupt.
  • System call parameters are passed through subsequent registers (e.g., ebx, ecx, edx, esi, and edi), or via a single register pointing to a user-space buffer if there are more than five parameters.
• The C Library (libc): Specifically GNU libc (glibc) on Linux, it provides system call wrappers, threading support, and basic application facilities.
• The C Compiler (gcc): The GNU Compiler Collection provides the standard C and C++ compilers (gcc and g++), which automatically handle the architecture’s system call invocation mechanisms and calling conventions.

To ensure these compiled programs execute reliably across environments, they must adhere to strict interface contracts.

Interfaces and Standards

• Application Programming Interface (API): Defines the source-level interfaces (typically functions) by which software communicates.
  • An API acts as an abstraction layer, guaranteeing source compatibility so that software compiles successfully against the API implementation.
• Application Binary Interface (ABI): Defines the binary-level interface between software components, the kernel, and libraries.
  • An ABI guarantees binary compatibility, allowing object code to function without recompilation on any system sharing the same ABI.
  • The ABI encompasses calling conventions, byte ordering, register use, system call invocation, and binary object formats.
  • ABIs are intimately tied to the specific machine architecture and are enforced by the toolchain (compiler and linker).
• Standards: Linux aims for compliance with POSIX (Portable Operating System Interface) and SUS (Single UNIX Specification), which codify the C API for Unix-like operating system interfaces.
• Linux Standard Base (LSB): A standard extending POSIX and SUS that attempts to provide a unified binary standard across Linux distributions.

The most fundamental abstraction dictated by these standards and the Unix philosophy is the file.

Files and Filesystems

• Linux follows an “everything-is-a-file” philosophy, meaning most interaction occurs via reading and writing files referenced by unique integer file descriptors (fd).
• Regular Files: A linear array of bytes (a byte stream) with no structural formatting enforced by the operating system.
  • File Position (Offset): An essential metadata value marking the current location within the file, starting at $0$.
  • Writing bytes to a position beyond the end of the file expands the file and fills the intervening space with zeros (creating a hole), but file positions cannot be negative.
  • Files can be opened concurrently by multiple processes; concurrent operations require explicit synchronization by user-space programs.
• Inodes: Physical and conceptual data structures storing file metadata (modification timestamp, owner, type, length, and data location).
  • Inodes are referenced by a filesystem-unique integer (the inode number), but importantly, they do not contain the file’s name.
• Directories and Links: Directories are files that map human-readable names to inode numbers; each name-to-inode pair is a link.
  • Directory resolution (pathname lookup) involves walking directory entries (dentry) to find the next inode, utilizing a dentry cache to optimize performance via temporal locality.
  • Absolute pathnames start at the root directory (/), whereas relative pathnames are resolved against the current working directory.
  • Hard Links: Multiple names resolving to the exact same inode. The inode contains a link count; the file’s data is only destroyed when this count reaches $0$.
  • Symbolic Links (Symlinks): Separate files containing the target pathname. They incur higher overhead, can point to nonexistent targets (broken links), and uniquely have the ability to span across different filesystems.
• Special Files: Represent kernel objects as files within the filesystem.
  • Block device files: Arrays of bytes mapped over a seekable device (e.g., hard disks).
  • Character device files: Linear queues of bytes accessed sequentially (e.g., keyboards).
  • Named pipes (FIFOs): A first-in, first-out IPC mechanism accessed via a file descriptor.
  • Unix domain sockets: Advanced IPC channels allowing communication between distinct processes on the local machine.
• Filesystems: Collections of files and directories in a formal hierarchy, mounted to the global namespace at a mount point.
  • Sectors vs. Blocks: Sectors are the smallest physical addressable unit on a storage device, whereas blocks are the logical addressable unit of the filesystem.
  • The relationship is constrained mathematically as: $\text{Sector Size}\le \text{Block Size}\le \text{Page Size}$.
  • Linux supports per-process namespaces, allowing processes to maintain a unique view of the system’s directory hierarchy.

While files encapsulate data and hardware, processes serve as the active abstraction executing the operations on these files.

Processes and Threads

• Processes: Object code in execution, consisting of data, resources, state, and a virtualized computer.
  • Binaries utilize the Executable and Linkable Format (ELF).
  • Text section: Executable code and read-only data (marked read-only and executable).
  • Data section: Initialized variables (marked readable and writable).
  • Bss section: Uninitialized global data. Optimizes disk space by simply listing variables and mapping the section to a zero page (a page of all zeros) in memory.
• Virtualization: The kernel provides every process with a virtualized processor (via seamless preemption) and a single linear virtual memory address space (via virtual memory and paging).
• Threads: The core unit of activity and execution within a process.
  • Each thread maintains its own processor state, instruction pointer, and stack.
  • Threads share the virtual memory abstraction (address space) and other process resources.
  • The Linux kernel views threads as standard processes that share specific resources, implementing POSIX threads via the Native POSIX Threading Library (NPTL).
• Process Hierarchy: Each process is identified by a unique process ID (pid).
  • Processes form a strict tree rooted at the init process ($pid=1$).
  • New processes are spawned via the fork() system call, creating a parent-child relationship.
  • Zombies: When a child terminates, it remains resident in memory as a zombie process until the parent executes a wait operation to inquire about its status, after which the child is fully destroyed.

To restrict these processes from accessing unauthorized files or terminating arbitrary peers, the system implements a strict security model.

Security and Access Control

• Users and Groups: Authorization is governed by unique positive integers: the user ID (uid) and group ID (gid).
  • The root user ($uid=0$) possesses special privileges capable of bypassing standard restrictions.
• Process IDs: A process maintains four distinct UIDs: real, effective, saved, and filesystem.
  • Real uid: Identifies the user who started the process.
  • Effective uid: The ID under which the process currently executes and validates rights.
  • Saved uid: Stores the original effective UID for switching back and forth.
  • Filesystem uid: Used specifically for verifying filesystem access.
• Capabilities: Linux has replaced simple binary root checks with a fine-grained capabilities system, allowing processes to receive distinct, specialized privileges.
• Permissions: Stored in the inode, utilizing 9 bits to define read, write, and execute permissions across three classes: owning user, owning group, and everybody else.
  • Extended access controls are provided via Access Control Lists (ACLs).

When a process violates these permissions or encounters hardware exceptions, the kernel must notify the execution thread directly.

Signals and IPC

• Signals: Mechanisms for one-way asynchronous notifications sent from the kernel to a process, or between processes.
  • Signals interrupt the executing process, forcing it to immediately perform a predetermined action.
  • Default actions include process termination, process termination with a core dump, stopping the process, or doing nothing.
  • Signal Handlers: A process can elect to catch a signal by providing a user-supplied function; because signals are asynchronous, handlers must exclusively execute async-safe (signal-safe) functions.
  • Certain signals, specifically SIGKILL and SIGSTOP, cannot be caught, ignored, or manipulated.
• Interprocess Communication (IPC): Mechanisms allowing independent processes to exchange information. Beyond the previously introduced named pipes and sockets, Linux supports semaphores, message queues, shared memory, and futexes.

When a process invokes IPC mechanisms or system calls that fail, the kernel communicates the failure cause synchronously.

Error Handling

• System calls and library functions signify failure by returning a special value, which is generally $-1$.
• errno: A special variable used to map a failure to a specific textual description of the error (e.g., EACCES for permission denied).
  • errno is a modifiable lvalue, valid only immediately after an errno-setting function indicates an error.
  • Any library or system call can modify errno; therefore, developers must save its value into a local variable immediately after a failure if it needs to be preserved across subsequent invocations.
  • In multithreaded programs (where the previously introduced thread memory sharing applies), errno is stored safely on a per-thread basis.
• Error Functions: The C library provides perror() to print the string representation of errno to standard error, and strerror()/strerror_r() to return a pointer to the string description.

This error handling mechanism ensures that the continuous execution of system programs, threads, and file operations remains robust and predictable across the Linux environment.