Linux Kernel Hardening: 10 Critical sysctl.conf Tweaks to Prevent SYN Floods & Spoofing

Why Default Linux Kernel Configurations Are Vulnerable

Out of the box, Linux distributions prioritize broad network compatibility over defense-in-depth security. Default kernel settings frequently allow routers to redirect your traffic via ICMP, accept packets with forged source IP addresses, and permit unprivileged users to create symlink pointers to sensitive system files.

When deploying a public-facing cloud VPS, relying solely on firewall rules (like UFW or iptables) is insufficient. If an attacker floods your web server with spoofed TCP SYN packets or exploits a buffer overflow in an application daemon, kernel-level hardening provides an immutable lower-level defensive barrier that intercepts attacks before they consume server memory.

The 10 Critical sysctl.conf Security Tweaks

Create a dedicated security override file in /etc/sysctl.d/. Using modular files in this directory ensures your customizations will not be overwritten by OS distribution updates:

sudo nano /etc/sysctl.d/99-security-hardening.conf

Insert the following hardened security profile:

# 1. Defend Against TCP SYN Flood Denial-of-Service Attacks
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.tcp_synack_retries = 2

# 2. Block IP Spoofing via Strict Reverse Path Filtering
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1

# 3. Reject ICMP Redirects (Prevent Man-in-the-Middle Routing Attacks)
net.ipv4.conf.all.accept_redirects = 0
net.ipv4.conf.default.accept_redirects = 0
net.ipv6.conf.all.accept_redirects = 0
net.ipv6.conf.default.accept_redirects = 0

# 4. Do Not Send ICMP Redirects (Server is Not a Router)
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0

# 5. Disable Source Routing (Prevent Attackers Dictating Packet Paths)
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv6.conf.all.accept_source_route = 0

# 6. Ignore Broadcast ICMP Echo Requests (Mitigate Smurf Amplification)
net.ipv4.icmp_echo_ignore_broadcasts = 1

# 7. Log Martians (Log Impossible / Spoofed Packet Addresses)
net.ipv4.conf.all.log_martians = 1
net.ipv4.conf.default.log_martians = 1

# 8. Ignore Bogus ICMP Error Responses
net.ipv4.icmp_ignore_bogus_error_responses = 1

# 9. Restrict Kernel Pointer Exposure & dmesg to Root
kernel.dmesg_restrict = 1
kernel.kptr_restrict = 2

# 10. Memory Protection: Maximize Address Space Layout Randomization (ASLR)
kernel.randomize_va_space = 2
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_fifos = 2
fs.protected_regular = 2

Step 2: Activating & Testing the Hardened Profile

Load all sysctl profiles from disk and verify they took effect:

# Reload all system sysctl rules
sudo sysctl --system

# Confirm SYN cookies are enabled
sysctl net.ipv4.tcp_syncookies

# Confirm ASLR memory randomization is maximized (returns 2)
sysctl kernel.randomize_va_space

Detailed Security Threat Mitigation Matrix

Attack Vector	Target Directive	Defensive Mechanism
TCP SYN Flood	tcp_syncookies = 1	Encodes connection state into initial sequence numbers when backlog fills, preventing memory starvation.
IP Address Spoofing	rp_filter = 1	Verifies that incoming packets arrive on the network interface the kernel would use to reply. Discards spoofed packets.
MITM Traffic Hijacking	accept_redirects = 0	Prevents malicious rogue routers or compromised neighbors on the LAN from altering server routing tables.
Buffer Overflow Exploits	randomize_va_space = 2	Randomizes the memory locations of program stacks, data structures, and libraries, making Return-Oriented Programming (ROP) fail.

Frequently Asked Questions (FAQ)

File Descriptor Limits & Socket Ephemeral Port Exhaustion Tuning

Beyond defensive network parameters, hardening your server’s resilience involves preventing resource starvation when high-traffic web applications open thousands of concurrent socket connections:

# Raise system-wide maximum file descriptor allocation
fs.file-max = 2097152

# Expand ephemeral port range for high-concurrency reverse proxies
net.ipv4.ip_local_port_range = 10240 65535

# Allow reuse of TIME_WAIT sockets for outgoing client connections
net.ipv4.tcp_tw_reuse = 1

# Reduce TIME_WAIT timeout from 60s to 30s to cycle closed sockets faster
net.ipv4.tcp_fin_timeout = 30

Setting tcp_tw_reuse = 1 is critical when Nginx reverse proxies traffic to local backends (like Node.js, PHP-FPM, or Python Gunicorn), preventing the operating system from running out of free TCP socket pairs under heavy load spikes.

Kernel Memory & Privilege Hardening: ptrace & BPF JIT

Modern privilege escalation exploits frequently target process tracing (ptrace) and the in-kernel eBPF Just-In-Time (JIT) compiler. Add these advanced defensive directives to your /etc/sysctl.d/99-security-hardening.conf:

# Restrict ptrace so non-root users cannot spy on or attach to other processes
kernel.yama.ptrace_scope = 2

# Harden eBPF JIT compiler against kernel spray attacks
net.core.bpf_jit_harden = 2

# Restrict unprivileged user namespaces to prevent container breakout exploits
kernel.unprivileged_userns_clone = 0

Applying kernel.yama.ptrace_scope = 2 ensures that only administrative processes with the CAP_SYS_PTRACE capability can debug running binaries, stopping attackers who gain low-privileged shell access from dumping database passwords out of process memory.