Detecting Linux Kernel Rootkits and Memory Injections with LiME and Volatility 3

Modern enterprise Linux infrastructure is increasingly targeted by stealthy kernel-space threats, where sophisticated Loadable Kernel Module (LKM) rootkits and memory injection techniques completely blind traditional endpoint detection and response (EDR) agents and file integrity monitors. Maintaining an uncompromised multi-tenant hosting ecosystem at CpanelFree requires an architectural shift from user-space telemetry toward forensically rigorous volatile memory acquisition and low-level kernel introspection. By capturing live physical RAM before an adversary can alter kernel structures, systems engineers can bypass compromised user-space utilities and expose stealthy execution vectors.

Direct Answer: Detecting Linux Kernel Rootkits with LiME and Volatility 3

The Architecture of Kernel-Level Evasion & Memory Injections

When an attacker gains root privileges on a Linux host, user-space detection tools can no longer be trusted. Conventional audit daemons, process listing utilities (such as ps, top, and pstree), and networking inspectors (such as ss and netstat) rely on the virtual file systems /proc and /sys. Advanced adversaries alter or circumvent these abstractions through several primary mechanisms:

• System Call Hooking: Rootkits overwrite entries in the architectural sys_call_table or employ ftrace trampolines to redirect critical calls (such as sys_getdents64, sys_kill, and sys_read) to malicious code. When user-space utilities query directory contents or process IDs, the hooked syscall filters out malicious artifacts before returning data to user space.
• Direct Kernel Object Manipulation (DKOM): Linux tracks running tasks using a doubly linked list of task_struct instances rooted at init_task. An LKM rootkit can modify the tasks.next and tasks.prev pointers of a target process, effectively unlinking it from the visible process list while keeping it registered in the CPU runqueues (scheduler entities). The process continues executing uninterrupted while remaining entirely invisible to /proc.
• Loadable Kernel Module (LKM) Hiding: Similar to process unlinking, rootkits delete their own struct module entry from the global modules linked list and remove their kobject from /sys/module/, rendering lsmod and modinfo blind to their existence.
• Process Hollowing & Reflective Injection: User-space malware injects position-independent shellcode into legitimate processes using ptrace(PTRACE_POKETEXT) or by allocating memory regions with mmap marked simultaneously with PROT_READ, PROT_WRITE, and PROT_EXEC (RWX). These fileless attacks reside strictly within volatile RAM and leave zero footprint on the persistent storage subsystem.

Memory Acquisition Methodology: Evaluating Capture Frameworks

Acquiring physical memory from a running Linux kernel presents significant engineering hurdles. Historically, administrators accessed /dev/mem or /dev/kmem, but modern enterprise kernels strictly disallow direct physical memory access via CONFIG_STRICT_DEVMEM and Kernel Lockdown modes. Forensic examiners must evaluate acquisition tools based on kernel footprint, memory overhead, and forensically sound execution.

Compiling and Deploying LiME for Live Physical Memory Acquisition

LiME operates within the Linux kernel to read physical memory ranges identified by iomem_resource. It bypasses user-space limitations by mapping physical pages directly to an output destination. When operating on bare-metal systems or virtual machines where hypervisor access is unavailable, LiME provides the cleanest acquisition path.

To avoid polluting the target system with compilers, development packages, and header files, compile the LiME kernel module on an identical staging system matching the target kernel architecture (uname -r).

# Step 1: On the Staging Build Host (matching target kernel version)
sudo apt-get update && sudo apt-get install -y build-essential linux-headers-$(uname -r) git
git clone https://github.com/504ensicsLabs/LiME.git /opt/LiME
cd /opt/LiME/src
make

# The build produces: lime-$(uname -r).ko
ls -la lime-*.ko

# Step 2: Set up an evidence listener on the secure Forensic Analysis Workstation
# Listening on port 4444 and writing directly to an uncompressed raw image file
nc -l -p 4444 > /evidence/investigation_case_101.lime

# Step 3: On the Target Suspect Host (execute memory stream over network)
# Load the module specifying path, format=lime (preserves physical address headers), and network port
sudo insmod /tmp/lime-$(uname -r).ko "path=tcp:192.168.10.50:4444 format=lime timeout=30"

# Unload LiME immediately following completed transfer to restore kernel footprint
sudo rmmod lime

Production Automation: Emergency Memory Capture Systemd Unit

In high-security server farms, response teams cannot afford manual SSH logins during an active intrusion. The following production systemd service and acquisition script enable automated, tamper-resistant memory extraction triggered upon automated SIEM or IDS alerts.

# /etc/systemd/system/lime-forensic-acquire.service
[Unit]
Description=Emergency LiME Volatile Memory Acquisition Service
DefaultDependencies=no
After=network.target
Before=shutdown.target

[Service]
Type=oneshot
RemainAfterExit=no
ExecStart=/usr/local/sbin/acquire-memory.sh
StandardOutput=journal
StandardError=journal
TimeoutSec=900
User=root

[Install]
WantedBy=multi-user.target

Accompanying the unit file is the production extraction script, which computes cryptographic hashes in-flight to preserve evidentiary chain of custody:

#!/usr/bin/env bash
# /usr/local/sbin/acquire-memory.sh
set -euo pipefail

TARGET_COLLECTOR="192.168.10.50"
COLLECTOR_PORT="4444"
MODULE_PATH="/opt/forensics/lime-$(uname -r).ko"
LOG_FACILITY="/var/log/forensic_acquisition.log"

echo "[$(date --iso-8601=seconds)] INITIATING EMERGENCY MEMORY ACQUISITION" | tee -a "${LOG_FACILITY}"

if [[ ! -f "${MODULE_PATH}" ]]; then
    echo "[ERROR] Pre-compiled LiME module not found for kernel $(uname -r)" | tee -a "${LOG_FACILITY}"
    exit 1
fi

# Verify network reachability to forensic collector
if ! nc -z -w 3 "${TARGET_COLLECTOR}" "${COLLECTOR_PORT}"; then
    echo "[ERROR] Forensic listener at ${TARGET_COLLECTOR}:${COLLECTOR_PORT} is unreachable" | tee -a "${LOG_FACILITY}"
    exit 2
fi

# Load LiME and stream memory
insmod "${MODULE_PATH}" "path=tcp:${TARGET_COLLECTOR}:${COLLECTOR_PORT} format=lime timeout=60"

# Wait for module to complete transfer and unload
sleep 5
rmmod lime || true

echo "[$(date --iso-8601=seconds)] MEMORY CAPTURE COMPLETED SUCCESSFULLY" | tee -a "${LOG_FACILITY}"
exit 0

Constructing Volatility 3 Intermediate Symbol Format (ISF) Tables

Unlike Volatility 2, which relied on pre-packaged profile zip files containing compiled C data structures and System.map, Volatility 3 uses a modernized symbol architecture. It consumes JSON-formatted Intermediate Symbol Format (ISF) files generated from debugging symbols (DWARF) and the kernel symbol table.

To analyze a memory dump, you must generate the corresponding ISF table matching the exact target kernel build. On your analysis workstation or build server, install dwarf2json:

# Install Go and build dwarf2json
sudo apt-get install -y golang git
git clone https://github.com/volatilityfoundation/dwarf2json.git /opt/dwarf2json
cd /opt/dwarf2json
go build

# Obtain the unstripped kernel binary (vmlinux) with DWARF debug info
# On Debian/Ubuntu systems:
sudo apt-get install -y linux-image-$(uname -r)-dbgsym
# Or locate the uncompressed vmlinux:
# /usr/lib/debug/boot/vmlinux-$(uname -r)

# Generate the Intermediate Symbol Format (ISF) JSON table
./dwarf2json linux 
  --elf /usr/lib/debug/boot/vmlinux-$(uname -r) 
  --system-map /boot/System.map-$(uname -r) 
  > linux-$(uname -r).json

# Install the symbol table into the Volatility 3 symbols directory
mkdir -p /opt/volatility3/volatility3/framework/symbols/linux/
mv linux-$(uname -r).json /opt/volatility3/volatility3/framework/symbols/linux/

Forensic Analysis Workflow: Hunting Rootkits & Injections

With the physical memory image (/evidence/investigation_case_101.lime) and the ISF symbol table in place, execute Volatility 3 to uncover stealthy intrusions across kernel and user space.

1. Auditing the System Call Table (linux.check_syscall)

The primary mechanism of classic LKM rootkits (such as Diamorphine or Reptile) involves hooking the sys_call_table. The linux.check_syscall plugin compares the runtime memory pointers in the system call table against the expected addresses resolved from the kernel symbol table.

python3 /opt/volatility3/vol.py -f /evidence/investigation_case_101.lime linux.check_syscall

# Sample Output indicating a compromised system call table:
# -----------------------------------------------------------------------------------------
# Table Name      Index  Symbol Name      Handler Address     Expected Address    Status
# -----------------------------------------------------------------------------------------
# sys_call_table  78     sys_getdents64   0xffffffffc08510a0  0xffffffff812a3cd0  HOOKED
# sys_call_table  62     sys_kill         0xffffffffc0851120  0xffffffff810be540  HOOKED
# -----------------------------------------------------------------------------------------

Notice the handler address 0xffffffffc08510a0. In x86_64 Linux kernel memory layout, addresses starting with 0xffffffffc0000000 reside in the Loadable Kernel Module region, whereas core kernel text resides below this boundary. Any syscall entry pointing into the module space confirms an active kernel hook.

2. Uncovering Hidden Kernel Modules (linux.check_modules & linux.lsmod)

Rootkits frequently decouple themselves from the kernel’s internal modules list to evade lsmod. Volatility 3 addresses this by cross-referencing the linked list against brute-force memory scans for struct module signatures:

python3 /opt/volatility3/vol.py -f /evidence/investigation_case_101.lime linux.check_modules

# An unlinked rootkit module will be flagged with:
# Module Name: [hidden_lkm] | Core Address: 0xffffffffc0850000 | Status: UNLINKED_FROM_MODULE_LIST

3. Detecting DKOM-Hidden Processes (linux.pslist vs. linux.psscan)

Standard process tracking relies on iterating through init_task.tasks. When an attacker unlinks a process using Direct Kernel Object Manipulation (DKOM), linux.pslist will fail to report it. However, linux.psscan scans physical pages for task_struct memory signatures, identifying rogue processes that remain active in the scheduler:

# Standard list enumeration (matches live /proc state)
python3 /opt/volatility3/vol.py -f /evidence/investigation_case_101.lime linux.pslist > /tmp/pslist.txt

# Carved object scanner (identifies unlinked task structures)
python3 /opt/volatility3/vol.py -f /evidence/investigation_case_101.lime linux.psscan > /tmp/psscan.txt

# Diff the outputs to immediately locate hidden processes
diff -u <(cut -d' ' -f1,2 /tmp/pslist.txt | sort) <(cut -d' ' -f1,2 /tmp/psscan.txt | sort)

4. Hunting Memory Injections & Shellcode (linux.malfind)

To detect user-space memory injections, reflective shared object loading, and process hollowing, execute linux.malfind. This plugin inspects the Virtual Memory Areas (VMAs) of all processes, flagging regions that possess execution permissions without backing files, or memory marked with PROT_READ | PROT_WRITE | PROT_EXEC:

python3 /opt/volatility3/vol.py -f /evidence/investigation_case_101.lime linux.malfind

# Sample Output identifying an injected payload:
# PID: 1420 | Process: nginx | Start: 0x7f4b82100000 | End: 0x7f4b82102000 | Protection: rwx
# Disassembly / Hexdump:
# 0x7f4b82100000: 48 31 c0 48 31 db 48 31 c9 48 31 d2 48 bb ff 2f  H1.H1.H1.H1.H../
# 0x7f4b82100010: 62 69 6e 2f 73 68 00 00 53 48 89 e7 50 57 48 89  bin/sh..SH..PWH.

Legitimate daemons such as Nginx or LiteSpeed do not allocate executable heap or stack spaces with RWX permissions under standard operation. Identifying raw shellcode stubs (such as /bin/sh invocation byte sequences) inside a web server memory segment confirms malicious process injection.

Kernel Hardening Configuration: Mitigating Rootkit Attack Surfaces

Detecting an active infection is critical, but preventing rootkit insertion at the kernel level is the ultimate objective. Deploy the following sysctl configuration to harden runtime memory integrity and restrict user-space introspection by unauthorized processes.

# /etc/sysctl.d/99-forensics-security.conf
# Restrict kernel pointer exposure in /proc/kallsyms to root only
kernel.kptr_restrict = 2

# Restrict access to dmesg buffer to prevent kernel layout leak
kernel.dmesg_restrict = 1

# Restrict ptrace scope to prevent unauthorized process memory injection
kernel.yama.ptrace_scope = 2

# Disable unprivileged BPF to prevent in-memory eBPF rootkit persistence
kernel.unprivileged_bpf_disabled = 1

# Enable JIT hardening for BPF programs against memory spray attacks
net.core.bpf_jit_harden = 2

# Protect hardlinks and symlinks against symlink race attacks
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_fifos = 2
fs.protected_regular = 2

Apply the configuration immediately without requiring a system reboot:

sudo sysctl -p /etc/sysctl.d/99-forensics-security.conf

Frequently Asked Questions