The foundational software layer managing physical hardware resources and enforcing process isolation. Master curriculum spanning Linux kernel subsystems, preemptive CPU scheduling, multi-threaded concurrency primitives, virtual memory multi-level paging, offensive binary exploitation (ROP, heap spray, format strings), modern kernel self-protection (KASLR, SMEP, SMAP, KPTI), and container/hypervisor sandboxing.
Definition: The fundamental architectural structural design of an OS kernel, determining whether device drivers, file systems, and network stacks execute inside privileged kernel space (Monolithic: Linux) or isolated user-space servers (Microkernel: seL4, QNX).
Security & Hardware Application: Monolithic kernels offer maximum raw throughput but expand the Ring 0 attack surface; a single vulnerability in any third-party device driver compromises the entire system.
Definition: Hardware-enforced processor privilege states (x86 Rings 0–3, ARM Exception Levels EL0–EL3) that restrict user applications (Ring 3 / EL0) from executing privileged CPU control instructions reserved exclusively for the kernel (Ring 0 / EL1).
Security & Hardware Application: The primary objective of Local Privilege Escalation (LPE) exploits is to cross this hardware privilege boundary to achieve unconstrained Ring 0 code execution.
Definition: The formal programmatic API and Application Binary Interface (ABI) used by user-space applications to request privileged OS operations via hardware instructions (syscall, sysenter, svc), optimized via the Virtual Dynamic Shared Object (vDSO) to avoid context switch overhead.
Security & Hardware Application: Tracing syscall parameters (via strace) maps process behavior; rootkits hook system call dispatch tables (sys_call_table) to hide malicious processes and files from monitoring tools.
Definition: Hardware and software signals that preempt CPU execution, saving register states and dispatching execution to Interrupt Service Routines (ISRs) via the Interrupt Descriptor Table (IDT), split into hardirqs (top halves) and deferred work (bottom halves / tasklets / workqueues).
Security & Hardware Application: Asynchronous race conditions in interrupt handlers can induce kernel panics or corrupt kernel state; IDT hooking allows rootkits to intercept low-level hardware events.
Definition: Dynamically loadable object binaries (.ko) that link directly into the running kernel address space at runtime, registering character, block, or network device driver callbacks into core subsystems.
Security & Hardware Application: Over 70% of all kernel vulnerabilities occur in third-party device drivers; kernel rootkits load unsigned LKMs to patch kernel symbols in memory (retrieved via /proc/kallsyms).
Definition: A universal in-kernel virtual machine that compiles sandboxed bytecode into native machine code, verified statically by the eBPF verifier to guarantee memory safety and loop termination before execution at kernel hook points.
Security & Hardware Application: Real-time zero-overhead kernel observability, high-speed DDoS mitigation (XDP), runtime intrusion detection (Cilium, Falco), and stealthy in-kernel eBPF rootkit implants.
task_struct & Context SwitchingDefinition: Kernel data structures (struct task_struct in Linux) maintaining an execution unit's complete lifecycle state (PID, CPU registers, memory descriptors, file descriptor tables, credentials), swapped during hardware context switches.
Security & Hardware Application: Overwriting the struct cred pointer in a target task_struct with init_cred is the classic milestone in kernel exploitation to instantly obtain root privileges (UID 0).
Definition: Algorithmic schedulers allocating CPU slices across runnable threads using virtual runtime tracking (CFS with Red-Black trees / EEVDF in modern Linux) or deterministic priority queues (SCHED_FIFO, SCHED_RR).
Security & Hardware Application: Attackers manipulate process nice levels or exploit priority inversion to starve security monitoring threads, causing Denial of Service in deterministic real-time systems.
Definition: Kernel-mediated data exchange mechanisms between isolated processes via message streaming (anonymous pipes, FIFOs, UNIX domain sockets) or mapped physical memory regions (POSIX shared memory shm_open).
Security & Hardware Application: Insecure file permissions on shared memory segments or UNIX domain sockets allow unprivileged local attackers to inject arbitrary commands or tamper with daemon state.
Definition: Low-level mutual exclusion and synchronization primitives, ranging from fast user-space futexes (sys_futex) and kernel spinlocks to lockless Read-Copy-Update (RCU) algorithms for high-concurrency read operations.
Security & Hardware Application: Complex race conditions in futex implementations (e.g., CVE-2014-3153 Towelroot) have historically yielded reliable arbitrary kernel write and root execution across millions of devices.
Definition: Timing and execution sequencing flaws where asynchronous operations interfere with shared state, including Time-of-Check to Time-of-Use (TOCTOU) file access windows and circular lock deadlocks satisfying Coffman conditions.
Security & Hardware Application: Exploiting TOCTOU race windows allows an attacker to swap a validated target path with a symlink to /etc/shadow in the microsecond interval before a privileged process executes a write.
Definition: Asynchronous notification mechanisms (SIGSEGV, SIGKILL, SIGALRM) where the kernel pauses thread execution, pushes a complete execution context frame (sigcontext) onto the user stack, and jumps to a signal handler.
Security & Hardware Application: The synthetic construction of forged signal frames on the stack allows binary exploit authors to perform Sigreturn-Oriented Programming (SROP), setting all CPU registers via a single sigreturn syscall.
Definition: Memory virtualization mapping 48-bit (PML4) or 57-bit (PML5) virtual addresses to fragmented physical RAM frames using hierarchical 4-level or 5-level page table tree structures with 4KB standard or 2MB/1GB Huge Pages.
Security & Hardware Application: Enforces process memory isolation; kernel exploits manipulate Page Table Entry (PTE) permission bits (clearing the User/Supervisor bit or setting Writable) to access arbitrary physical memory.
Definition: Hardware translation state machines executing multi-level page table walks upon virtual address references, caching recent virtual-to-physical translations inside the Translation Lookaside Buffer (TLB) tagged with ASID/PCID identifiers.
Security & Hardware Application: TLB shootdowns incur heavy inter-core interrupt penalties; side-channel attacks exploit TLB invalidation latencies to track cryptographic memory access across processes.
Definition: Hardware exceptions triggered on unmapped memory access (Major vs. Minor faults), handling demand paging from disk, lazy address space replication via Copy-on-Write (CoW) during fork(), and OOM killer termination under memory exhaustion.
Security & Hardware Application: Race conditions in kernel CoW handling (e.g., Dirty COW CVE-2016-5195) allow unprivileged processes to write directly to read-only memory mappings, overwriting root binaries on disk.
Definition: Hierarchical kernel dynamic memory managers that allocate contiguous power-of-two physical page frames (Buddy Allocator) and pre-allocate fixed-size object caches for kernel structs (SLUB allocator).
Security & Hardware Application: The primary battleground for Linux kernel exploitation (SLUB spraying); attackers groom kernel heap caches to overwrite adjacent target structures (e.g., msg_msg, pipe_buffer) following a Use-After-Free.
Definition: The kernel abstraction layer unifying diverse physical filesystems (ext4, XFS, Btrfs) into a common object model (inode, dentry, file, superblock), backed by the Page Cache for asynchronous dirty page writeback.
Security & Hardware Application: VFS path traversal vulnerabilities, insecure file descriptor passing across UNIX sockets, and page cache flush race conditions can be exploited to achieve root write primitives.
Definition: Hardware controllers allowing high-speed peripherals (NICs, NVMe drives) to read and write physical memory directly without CPU intervention, constrained by Input-Output Memory Management Units (IOMMUs).
Security & Hardware Application: Malicious Thunderbolt or PCIe peripherals conduct DMA attacks to dump encryption keys from physical RAM; IOMMUs enforce hardware-level page tables to restrict peripheral access.
Definition: The formal memory layout of an executable process (Text, Data, BSS, Heap, Stack, Memory Mappings) governed by calling conventions that dictate register usage (RDI, RSI, RDX, RCX, R8, R9) and stack alignment.
Security & Hardware Application: Exploit authors must construct memory payloads that strictly conform to target ABI calling conventions to properly pass function parameters into libc functions or syscall gates.
Definition: Writing data beyond allocated bounds of stack buffers to overwrite adjacent stack frame memory, specifically targeting the Saved Base Pointer (RBP) and Saved Return Address (RIP).
Security & Hardware Application: The classical control-flow hijacking vector; overwriting the saved return pointer redirects instruction execution flow to attacker-controlled shellcode or arbitrary memory addresses.
Definition: Advanced exploitation techniques that chain existing machine-code sequences ending in ret (ROP gadgets), indirect jumps (JOP), or forged signal frames (SROP) across executable binary pages to construct Turing-complete payloads.
Security & Hardware Application: The universal exploitation methodology used to bypass Data Execution Prevention (DEP / NX) without injecting any new executable instructions.
Definition: Vulnerabilities arising from bugs in dynamic memory allocators (glibc ptmalloc), where dangling pointers access freed chunks (Use-After-Free), chunks are freed twice, or singly-linked tcache/fastbin forward pointers are overwritten.
Security & Hardware Application: Exploited in browsers, hypervisors, and PDF readers to achieve arbitrary write-what-where primitives by forcing the allocator to return an arbitrary memory address on subsequent malloc() calls.
Definition: Vulnerabilities occurring when unvalidated user input is passed directly as the format string argument to printf()-family functions, evaluated by format specifiers (%p, %s, %n).
Security & Hardware Application: Provides both arbitrary memory disclosure (reading pointers to leak ASLR base addresses) and arbitrary memory writing (using %n to write calculated byte counts to pointers).
struct cred, SLUB Spraying, KROP)Definition: Kernel-space exploitation methodologies where kernel heap corruption (SLUB UAF) is turned into arbitrary kernel memory read/write, executing Kernel ROP (KROP) chains to invoke commit_creds(prepare_kernel_cred(0)).
Security & Hardware Application: Elevates an unprivileged local user into root (UID 0) and disables kernel security controls (SELinux enforcing mode, seccomp filters) directly in Ring 0 memory.
Definition: Operating system defense that randomizes the base virtual addresses of the Stack, Heap, Shared Libraries (libc), and Position-Independent Executable (PIE) binaries on every process execution.
Security & Hardware Application: Neutralizes hardcoded address exploits; forces attackers to discover and chain an initial memory information disclosure leak before constructing ROP chains.
Definition: Hardware-enforced memory page protection (via the No-Execute NX bit in Page Table Entries) enforcing the Write-XOR-Execute (W ⊕ X) policy, ensuring memory pages are never simultaneously writable and executable.
Security & Hardware Application: Prevents direct code execution of injected shellcode on the stack or heap, forcing modern exploits to rely on code reuse techniques (ROP/JOP).
Definition: Compiler-inserted random guard integers (__stack_chk_guard) placed on stack frames immediately before the saved return pointer, validated upon function exit to detect buffer overflows.
Security & Hardware Application: Instantly invokes __stack_chk_fail to terminate the process if a buffer overflow corrupts the canary, preventing control flow from jumping to forged return addresses.
Definition: Built-in hardware and kernel defenses: Kernel ASLR, Supervisor Mode Execution Prevention (SMEP), Supervisor Mode Access Prevention (SMAP), and Kernel Page Table Isolation (KPTI).
Security & Hardware Application: Blocks kernel-mode code from executing user-space instructions (SMEP) or reading user-space data (SMAP), neutralizing classic ret2usr kernel exploits.
Definition: Compiler-generated forward-edge validation tables that check function pointer signatures before indirect call jumps, ensuring execution only branches to legitimate, compiler-verified target functions.
Security & Hardware Application: Defeats Jump-Oriented Programming (JOP) and vtable hijacking attacks in C/C++ binaries and the Linux kernel without requiring specialized hardware support.
Definition: Kernel security frameworks that enforce explicit security policies on processes regardless of user identity (SELinux Type Enforcement) and partition monolithic root privileges into 40+ granular Linux Capabilities (CAP_NET_ADMIN, CAP_SYS_ADMIN).
Security & Hardware Application: Confines compromised daemons (e.g., an HTTP server running as root) so they cannot read unauthorized filesystem paths or modify kernel parameters.
Definition: Linux kernel isolation primitives that partition global system resources (Process IDs, Mount points, Network interfaces, User mappings) so processes within a namespace see an isolated instance of the operating system.
Security & Hardware Application: The core foundation of container engines (Docker, containerd); misconfigured User Namespaces or privileged container capabilities enable container escape vulnerabilities.
Definition: Hierarchical kernel resource management subsystems that monitor, allocate, and enforce hard limits on CPU shares, memory consumption, block I/O bandwidth, and process count (pids controller).
Security & Hardware Application: Prevents multi-tenant noisy-neighbor starvation and fork bomb Denial-of-Service attacks by strictly capping max PID allocation and physical memory ceilings per cgroup slice.
Definition: Secure Computing Mode with Berkeley Packet Filter (seccomp-bpf) extensions that attach programmable BPF filters to system call dispatch, evaluating syscall numbers and pointer arguments to allow, deny, or trap requests.
Security & Hardware Application: Drastically reduces accessible kernel attack surface for sandboxed applications (Chrome Renderer, systemd services), blocking dangerous syscalls (ptrace, mount, bpf).
Definition: Type-1 and Type-2 hypervisors utilizing hardware CPU virtualization extensions (Intel VT-x / AMD-V) to execute guest OS kernels inside VMX Non-Root mode, using Extended Page Tables (EPT/NPT) and VM-Exit hardware traps.
Security & Hardware Application: Enforces strong hypervisor security boundaries in public clouds (AWS, GCP); VM breakout exploits weaponize virtual device emulation bugs (QEMU/KVM) to escape guests into the host kernel.