A Linux routing daemon providing high availability via VRRP (Virtual Router Redundancy Protocol) and health checks for load-balanced server pools.
Table of Contents#
- Overview
- How VRRP Works
- Installation
- Basic Configuration
- VRRP Priorities and Preemption
- Health Check Scripts
- IPv6 and VRRPv3
- Notification Scripts
- Debugging and Diagnostics
- Troubleshooting
- See Also
- Sources
1. Overview#
Keepalived is a Linux-based routing daemon that provides high availability through VRRP and health checking. It enables multiple servers to share a virtual IP address (VIP), providing automatic failover when the active server becomes unreachable or a monitored service fails.
Key features:
- VRRP (v2 and v3) - automatic VIP failover between servers
- Health checks - monitor service availability (HTTP, TCP, scripts, process tracking)
- LVS integration - direct integration with Linux Virtual Server for load balancing
- IPv4 and IPv6 - dual-stack support via VRRPv3
- Unicast and multicast - flexible VRRP advertisement transport
Common Use Cases#
| Use Case | Description |
|---|---|
| Database failover | VIP follows the primary PostgreSQL/MySQL node |
| Load balancer HA | Active/standby HAProxy or Nginx pairs |
| Gateway redundancy | Default gateway failover for network segments |
| Service VIP | Any service that needs a floating IP for client access |
2. How VRRP Works#
VRRP defines a protocol where multiple routers (or servers) participate in a virtual router group. One is elected MASTER and owns the VIP; the others remain BACKUP.
Election Process#
- Each node has a priority (1-254; higher wins)
- The node with the highest effective priority becomes MASTER
- The MASTER sends periodic advertisements (default: every 1 second)
- If a BACKUP stops receiving advertisements for
(3 * advert_int) + skew_time, it transitions to MASTER
State Transitions#
+----------+
+----->| BACKUP |<-----+
| +----+-----+ |
| | |
| (higher priority | (MASTER seen with
| or timeout) | higher priority)
| v |
| +----+-----+ |
+------| MASTER |------+
+----------+3. Installation#
Debian/Ubuntu#
sudo apt-get update
sudo apt-get install keepalivedRHEL/Rocky/Alma#
sudo dnf install keepalivedArch Linux#
sudo pacman -S keepalivedFirewall Configuration#
VRRP uses IP protocol 112 (not a TCP/UDP port). Allow it through the firewall:
# firewalld
sudo firewall-cmd --add-rich-rule='rule protocol value="vrrp" accept' --permanent
sudo firewall-cmd --reload
# iptables
sudo iptables -A INPUT -p vrrp -j ACCEPT
# nftables
sudo nft add rule inet filter input ip protocol vrrp acceptFor unicast mode, also allow the VRRP advertisement port (default UDP 112 between peers).
4. Basic Configuration#
The configuration file is /etc/keepalived/keepalived.conf.
MASTER Node#
global_defs {
enable_script_security
script_user root
router_id <MASTER_HOSTNAME>
}
vrrp_track_process chk_service {
process nginx
weight 2
}
vrrp_instance VI_1 {
interface <interface>
virtual_router_id 27
state MASTER
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass <secret>
}
virtual_ipaddress {
<vip_address>/32
}
unicast_src_ip <master_ip>
unicast_peer {
<backup_ip>
}
track_process {
chk_service
}
}BACKUP Node#
global_defs {
enable_script_security
script_user root
router_id <BACKUP_HOSTNAME>
}
vrrp_track_process chk_service {
process nginx
weight 2
}
vrrp_instance VI_1 {
interface <interface>
virtual_router_id 27
state BACKUP
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass <secret>
}
virtual_ipaddress {
<vip_address>/32
}
unicast_src_ip <backup_ip>
unicast_peer {
<master_ip>
}
track_process {
chk_service
}
}Service Management#
sudo systemctl enable --now keepalived
sudo systemctl status keepalived5. VRRP Priorities and Preemption#
Priority Mechanics#
| Priority | Role |
|---|---|
| 255 | Reserved (IP address owner) |
| 101-254 | Typical MASTER range |
| 1-100 | Typical BACKUP range |
| 0 | Reserved (signals MASTER is stopping) |
The effective priority is the configured priority adjusted by weight values from tracked scripts/processes/interfaces:
effective_priority = base_priority + weight (if check passes)
effective_priority = base_priority - abs(weight) (if check fails and weight < 0)Preemption Modes#
| Mode | Behavior |
|---|---|
| Preempt (default) | A recovered higher-priority node reclaims MASTER |
| Nopreempt | Once a BACKUP becomes MASTER, it stays MASTER until it fails |
| Preempt delay | Higher-priority node waits N seconds before preempting |
vrrp_instance VI_1 {
state BACKUP
priority 101
nopreempt # Do not preempt; stay BACKUP if another node is MASTER
# Or use a delay before preempting:
# preempt_delay 300 # Wait 5 minutes before taking over
}When using nopreempt, both nodes should be configured with state BACKUP so the first to start becomes MASTER, and a recovered node does not immediately take over.
Multiple VRRP Instances#
For active/active setups, run two VRRP instances with opposite priorities:
# Node A
vrrp_instance VI_WEB {
priority 101
virtual_ipaddress { 192.0.2.100/32 }
}
vrrp_instance VI_DB {
priority 100
virtual_ipaddress { 192.0.2.101/32 }
}
# Node B
vrrp_instance VI_WEB {
priority 100
virtual_ipaddress { 192.0.2.100/32 }
}
vrrp_instance VI_DB {
priority 101
virtual_ipaddress { 192.0.2.101/32 }
}6. Health Check Scripts#
vrrp_track_process#
Track a running process by name:
vrrp_track_process chk_nginx {
process nginx
weight 2 # Add 2 to priority when nginx is running
quorum 1 # Minimum number of matching processes
delay 2 # Wait 2 seconds before reacting to changes
}vrrp_script#
Execute a custom script for health checking:
vrrp_script chk_haproxy {
script "/usr/local/bin/check_haproxy.sh"
interval 2 # Check every 2 seconds
weight -20 # Subtract 20 from priority on failure
fall 3 # Require 3 consecutive failures to mark as failed
rise 2 # Require 2 consecutive successes to mark as recovered
timeout 5 # Script timeout in seconds
}
vrrp_instance VI_1 {
track_script {
chk_haproxy
}
}Example Health Check Scripts#
HAProxy Check#
#!/usr/bin/env bash
# /usr/local/bin/check_haproxy.sh
if pgrep -x haproxy > /dev/null; then
exit 0
fi
exit 1HTTP Endpoint Check#
#!/usr/bin/env bash
# /usr/local/bin/check_http.sh
HTTP_CODE=$(curl -s -o /dev/null -w '%{http_code}' http://localhost:8080/health)
if [ "${HTTP_CODE}" = "200" ]; then
exit 0
fi
exit 1PostgreSQL Primary Check#
#!/usr/bin/env bash
# /usr/local/bin/check_pgsql_primary.sh
RESULT=$(su - postgres -c "psql -tAc 'SELECT pg_is_in_recovery();'" 2>/dev/null)
if [ "${RESULT}" = "f" ]; then
exit 0 # Is primary
fi
exit 1 # Is standby or unreachableTCP Port Check#
#!/usr/bin/env bash
# /usr/local/bin/check_port.sh
if /usr/bin/nc -z localhost 5432 2>/dev/null; then
exit 0
fi
exit 1Track Interface#
Monitor a network interface (failover if the interface goes down):
vrrp_instance VI_1 {
track_interface {
eth0 weight -50
eth1 weight -50
}
}7. IPv6 and VRRPv3#
VRRPv3 (RFC 5798) supports both IPv4 and IPv6 and removes the authentication mechanism (authentication is handled at the network layer instead).
IPv6 VIP Configuration#
vrrp_instance VI_6 {
interface eth0
virtual_router_id 61
state MASTER
priority 101
advert_int 1
use_vmac # Recommended for IPv6
vmac_xmit_base # Transmit on the base interface
virtual_ipaddress {
2001:db8::1/64
}
# No authentication block (VRRPv3 does not support it)
}Dual-Stack Configuration#
Run separate VRRP instances for IPv4 and IPv6:
vrrp_instance VI_4 {
interface eth0
virtual_router_id 51
state MASTER
priority 101
advert_int 1
authentication {
auth_type PASS
auth_pass <secret>
}
virtual_ipaddress {
192.0.2.100/24
}
}
vrrp_instance VI_6 {
interface eth0
virtual_router_id 61
state MASTER
priority 101
advert_int 1
use_vmac
vmac_xmit_base
virtual_ipaddress {
2001:db8::100/64
}
}Forcing VRRPv3#
global_defs {
vrrp_version 3
}8. Notification Scripts#
Keepalived can execute scripts on state transitions:
vrrp_instance VI_1 {
notify_master "/usr/local/bin/keepalived-notify.sh MASTER"
notify_backup "/usr/local/bin/keepalived-notify.sh BACKUP"
notify_fault "/usr/local/bin/keepalived-notify.sh FAULT"
notify_stop "/usr/local/bin/keepalived-notify.sh STOP"
# Or use a single notify script that receives state as argument:
# notify "/usr/local/bin/keepalived-notify.sh"
}Example notification script:
#!/usr/bin/env bash
# /usr/local/bin/keepalived-notify.sh
STATE=$1
TIMESTAMP=$(date '+%Y-%m-%d %H:%M:%S')
echo "${TIMESTAMP} - Keepalived transitioned to ${STATE}" >> /var/log/keepalived-notify.log
case "${STATE}" in
MASTER)
# Actions when becoming MASTER
systemctl start haproxy 2>/dev/null
;;
BACKUP|FAULT|STOP)
# Actions when losing MASTER
systemctl stop haproxy 2>/dev/null
;;
esac9. Debugging and Diagnostics#
Log Inspection#
# Keepalived logs to syslog by default
journalctl -u keepalived -f
# Or in syslog
tail -f /var/log/syslog | grep -i keepalivedDump VRRP State#
# Send SIGUSR1 to dump current data to /tmp/keepalived.data
sudo kill -USR1 $(cat /run/keepalived.pid)
cat /tmp/keepalived.dataDump Statistics#
# Send SIGUSR2 to dump stats to /tmp/keepalived.stats
sudo kill -USR2 $(cat /run/keepalived.pid)
cat /tmp/keepalived.statsVerify VIP Assignment#
# Check if VIP is assigned
ip addr show dev <interface> | grep <vip_address>
# Watch VRRP advertisements (multicast mode)
sudo tcpdump -i <interface> -nn proto 112
# Watch VRRP advertisements (unicast mode)
sudo tcpdump -i <interface> -nn host <peer_ip> and proto 112Configuration Validation#
# Check config syntax
keepalived --config-test -f /etc/keepalived/keepalived.confVerbose Logging#
global_defs {
vrrp_garp_master_delay 1
vrrp_garp_master_repeat 3
log_unknown_vrids
}Or start keepalived with increased verbosity:
keepalived -D -l -f /etc/keepalived/keepalived.confTroubleshooting#
| Issue | Cause | Solution |
|---|---|---|
| VIP not assigned to any node | VRRP traffic blocked by firewall | Allow protocol 112 (VRRP) in firewall; for unicast, also allow peer IPs |
| Both nodes claim MASTER (split brain) | Network partition or virtual_router_id mismatch | Ensure both nodes share the same virtual_router_id; verify network connectivity between peers |
| VIP flapping between nodes | Health check script flapping or too aggressive timings | Increase fall/rise counts in vrrp_script; check script reliability |
| BACKUP never takes over | Firewall blocking, wrong priorities, or nopreempt misconfigured | Verify BACKUP receives advertisements; check priorities; test by stopping keepalived on MASTER |
| Gratuitous ARP not reaching clients | Switches filtering GARP or GARP count too low | Increase vrrp_garp_master_repeat; check switch ARP table |
| Health check script not running | enable_script_security set but script not owned by root | Ensure script is owned by root and not writable by group/other; or adjust script_user |
| VIP assigned but clients cannot connect | Service not running on the new MASTER | Add notify_master script to start the service; or use track_script/track_process |
| "Received lower priority advert" in logs | Normal VRRP behavior when a lower-priority node starts | Informational only; no action needed unless it leads to flapping |