Fast, versatile file synchronization and transfer utility that copies only differences between source and destination.
Addresses below are RFC 5737 documentation ranges or placeholders - swap in your own.
Table of Contents#
- Overview
- Installation
- How Rsync Works
- Basic Usage and Syntax
- Trailing Slash Behavior
- Common Options Reference
- Remote Transfers Over SSH
- Bandwidth Throttling
- Excluding Files and Directories
- Checksum Mode
- Daemon Mode (rsyncd)
- Automating Backups with Cron and Systemd
- Real-Time Synchronization with inotify
- Troubleshooting
- See Also
- Sources
1. Overview#
Rsync (Remote Sync) is a utility for efficiently transferring and synchronizing files between local and remote locations. It uses a delta-transfer algorithm that sends only the differences between source and destination files, minimizing network bandwidth and time. Rsync is the foundation of many backup strategies on Linux systems.
Key features:
- Delta transfer - only changed bytes are sent, not entire files
- Preservation - maintains permissions, ownership, timestamps, symlinks, and hardlinks
- Compression - optional in-transit compression reduces bandwidth usage
- Remote support - works over SSH, native rsync protocol, or local filesystem
- Daemon mode - acts as a server for centralized backup collection
- Flexible filtering - include/exclude patterns for fine-grained control
2. Installation#
Rsync is pre-installed on most Linux distributions. If not:
# Debian/Ubuntu
sudo apt install rsync
# Fedora/RHEL
sudo dnf install rsync
# Arch Linux
sudo pacman -S rsync3. How Rsync Works#
The Delta-Transfer Algorithm#
Rsync divides files into fixed-size blocks and computes rolling checksums and strong checksums (MD5 or xxhash) for each block. The sender compares these checksums against its local copy and transmits only the blocks that differ. This makes rsync extremely efficient for updating files that have changed slightly.
Modes of Operation#
| Mode | Description | Transport |
|---|---|---|
| Local | Copy between paths on the same machine | Filesystem |
| Remote shell | Transfer via SSH (or rsh) | SSH (encrypted) |
| Daemon | Connect to an rsync daemon (rsyncd) | Native rsync protocol (port 873) |
4. Basic Usage and Syntax#
rsync [options] <source> <destination>Local Copy#
rsync -av /path/to/source/ /path/to/destination/Remote Copy (Push)#
rsync -avz /path/to/source/ <user>@<host>:/path/to/destination/Remote Copy (Pull)#
rsync -avz <user>@<host>:/path/to/source/ /path/to/destination/Mirror with Deletion#
rsync -av --delete /path/to/source/ /path/to/destination/The --delete flag removes files from the destination that no longer exist in the source.
5. Trailing Slash Behavior#
The presence or absence of a trailing slash on the source path changes rsync's behavior significantly. This is a common source of errors.
| Command | Result |
|---|---|
rsync -av /data/ /backup/ | Copies the contents of /data/ into /backup/ |
rsync -av /data /backup/ | Copies the directory itself into /backup/data/ |
Examples:
# Source: /data/ contains file1.txt, file2.txt
# WITH trailing slash: files end up at /backup/file1.txt, /backup/file2.txt
rsync -av /data/ /backup/
# WITHOUT trailing slash: files end up at /backup/data/file1.txt, /backup/data/file2.txt
rsync -av /data /backup/Rule of thumb: use a trailing slash on the source when you want the contents, omit it when you want the directory itself.
6. Common Options Reference#
| Option | Description |
|---|---|
-a | Archive mode; equivalent to -rlptgoD. Preserves permissions, ownership, timestamps, symlinks. |
-v | Verbose output |
-z | Compress data during transfer |
-P | Equivalent to --partial --progress. Keeps partial files and shows progress. |
--progress | Show transfer progress per file |
-h | Human-readable output |
-n / --dry-run | Simulate the transfer without making changes |
--delete | Delete files in destination not present in source |
--delete-after | Delete after transfer completes (safer than default) |
-e "ssh -p <port>" | Specify remote shell and options |
--partial | Keep partially transferred files for resuming |
--append-verify | Resume partial transfers and verify with checksum |
-c / --checksum | Use checksum instead of mod-time and size for comparison |
-H | Preserve hardlinks |
-A | Preserve ACLs |
-X | Preserve extended attributes |
--bwlimit=<kbps> | Limit bandwidth in KiB/s |
--compress-level=<n> | Set compression level (0-9) |
--stats | Print detailed transfer statistics |
--log-file=<path> | Write detailed log to a file |
--backup --backup-dir=<dir> | Move overwritten files to a backup directory |
7. Remote Transfers Over SSH#
Basic SSH Transfer#
rsync -avz -e ssh /path/to/source/ <user>@<host>:/path/to/destination/Custom SSH Port and Key#
rsync -avz -e "ssh -p <port> -i /path/to/key" /data/ <user>@<host>:/backup/SSH Key Setup for Passwordless Transfers#
# Generate key pair (if not already existing)
ssh-keygen -t ed25519 -f ~/.ssh/id_rsync -N ""
# Copy public key to remote host
ssh-copy-id -i ~/.ssh/id_rsync.pub <user>@<host>
# Test
rsync -avz -e "ssh -i ~/.ssh/id_rsync" /data/ <user>@<host>:/backup/Restricted SSH Key for Rsync Only#
In ~/.ssh/authorized_keys on the remote host:
command="rsync --server --sender -logDtprze.iLsfxCIvu . /allowed/path/",restrict ssh-ed25519 AAAA... rsync@client8. Bandwidth Throttling#
Using --bwlimit#
# Limit to 5 MiB/s
rsync -avz --bwlimit=5120 /data/ <user>@<host>:/backup/
# Limit to 1 MiB/s
rsync -avz --bwlimit=1024 /data/ <user>@<host>:/backup/The value is in KiB/s. Common values:
| Speed | --bwlimit value |
|---|---|
| 1 MiB/s | 1024 |
| 5 MiB/s | 5120 |
| 10 MiB/s | 10240 |
| 50 MiB/s | 51200 |
Scheduling Bandwidth#
Use different limits during business hours vs off-hours in your backup script:
#!/bin/bash
HOUR=$(date +%H)
if [ "$HOUR" -ge 8 ] && [ "$HOUR" -lt 18 ]; then
BWLIMIT="--bwlimit=2048" # 2 MiB/s during business hours
else
BWLIMIT="" # Unlimited at night
fi
rsync -avz $BWLIMIT /data/ backup@server:/backup/9. Excluding Files and Directories#
Command-Line Excludes#
rsync -av --exclude='*.tmp' --exclude='.cache' --exclude='node_modules' /data/ /backup/Exclude File#
Create /etc/rsync/excludes.txt:
*.tmp
*.swp
*~
.cache/
.thumbnails/
__pycache__/
node_modules/
.git/
/proc/
/sys/
/dev/
/tmp/
/run/Use it:
rsync -av --exclude-from='/etc/rsync/excludes.txt' /data/ /backup/Include/Exclude Combinations#
Process files matching specific patterns while excluding everything else:
rsync -av --include='*.conf' --include='*/' --exclude='*' /etc/ /backup/etc/This copies only .conf files while preserving the directory structure.
10. Checksum Mode#
By default, rsync uses file modification time and size to determine if a file needs to be transferred. The --checksum flag forces rsync to compute and compare checksums instead.
rsync -avc /data/ /backup/When to use --checksum:
- When file modification times are unreliable (e.g., after a restore or timezone change)
- When verifying backup integrity
- When syncing between systems with different timestamp precision
Trade-off: checksum mode is slower because rsync must read every file on both sides to compute checksums, even if the file has not changed. For large datasets, this can significantly increase sync time.
11. Daemon Mode (rsyncd)#
Running rsync as a daemon allows clients to connect without SSH, using the native rsync protocol on port 873.
Server Configuration#
Create /etc/rsyncd.conf:
# Global settings
uid = nobody
gid = nogroup
use chroot = yes
max connections = 10
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
# Module definitions
[backups]
path = /srv/backups
comment = Backup storage
read only = no
auth users = backup
secrets file = /etc/rsyncd.secrets
hosts allow = 192.0.2.0/24 198.51.100.0/24
hosts deny = *
[public]
path = /srv/public
comment = Public files
read only = yesSecrets File#
Create /etc/rsyncd.secrets:
backup:<your-password>Set permissions:
chmod 600 /etc/rsyncd.secretsStart the Daemon#
# Systemd
sudo systemctl enable --now rsync
# Or manually
rsync --daemon --config=/etc/rsyncd.confClient Connection#
# List modules on a daemon
rsync rsync://<host>/
# Pull from a daemon module
rsync -avz rsync://backup@<host>/backups/ /local/backup/
# Push to a daemon module
rsync -avz /data/ rsync://backup@<host>/backups/
# With password file (for automation)
echo "<your-password>" > ~/.rsync-password
chmod 600 ~/.rsync-password
rsync -avz --password-file=~/.rsync-password /data/ rsync://backup@<host>/backups/12. Automating Backups with Cron and Systemd#
Cron#
# Daily backup at 2:00 AM
0 2 * * * /usr/bin/rsync -avz --delete --log-file=/var/log/rsync-backup.log /data/ backup@server:/backup/
# Weekly full sync on Sunday at 3:00 AM
0 3 * * 0 /usr/bin/rsync -avc --delete /data/ /backup/Systemd Timer#
Create /etc/systemd/system/rsync-backup.service:
[Unit]
Description=Rsync backup
Wants=network-online.target
After=network-online.target
[Service]
Type=oneshot
Nice=19
IOSchedulingClass=idle
ExecStart=/usr/bin/rsync -avz --delete --log-file=/var/log/rsync-backup.log /data/ backup@server:/backup/Create /etc/systemd/system/rsync-backup.timer:
[Unit]
Description=Run rsync backup daily
[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=30m
[Install]
WantedBy=timers.targetEnable:
sudo systemctl daemon-reload
sudo systemctl enable --now rsync-backup.timer13. Real-Time Synchronization with inotify#
inotify is a Linux kernel subsystem that monitors filesystem events. Combined with rsync, it enables near-real-time file synchronization.
Prerequisites#
# Debian/Ubuntu
sudo apt install inotify-tools
# Arch Linux
sudo pacman -S inotify-toolsBasic inotify + Rsync Script#
#!/bin/bash
SOURCE="/path/to/source/"
DESTINATION="<user>@<host>:/path/to/destination/"
LOGFILE="/var/log/rsync-inotify.log"
inotifywait -m -r -e modify,create,delete,move --format '%w%f' "${SOURCE}" | while read file; do
echo "$(date): Change detected: ${file}" >> "${LOGFILE}"
rsync -avz --delete "${SOURCE}" "${DESTINATION}" >> "${LOGFILE}" 2>&1
doneImproved Script with Debouncing#
The basic script triggers rsync on every single event, which can cause excessive syncs. Use a debounce approach:
#!/bin/bash
SOURCE="/path/to/source/"
DESTINATION="<user>@<host>:/path/to/destination/"
LOGFILE="/var/log/rsync-inotify.log"
DEBOUNCE=5 # seconds
inotifywait -m -r -e modify,create,delete,move --format '%w%f' "${SOURCE}" | while read file; do
# Kill any pending sync, reset the timer
if [ -n "${SYNC_PID}" ] && kill -0 "${SYNC_PID}" 2>/dev/null; then
kill "${SYNC_PID}" 2>/dev/null
fi
(
sleep ${DEBOUNCE}
echo "$(date): Syncing after change to ${file}" >> "${LOGFILE}"
rsync -avz --delete "${SOURCE}" "${DESTINATION}" >> "${LOGFILE}" 2>&1
) &
SYNC_PID=$!
doneSystemd Service for inotify Sync#
Create /etc/systemd/system/rsync-watch.service:
[Unit]
Description=Real-time rsync via inotify
After=network-online.target
[Service]
Type=simple
ExecStart=/usr/local/bin/rsync-inotify.sh
Restart=always
RestartSec=10
[Install]
WantedBy=multi-user.targetTuning inotify Limits#
For large directory trees, increase the inotify watch limit:
# Check current limit
cat /proc/sys/fs/inotify/max_user_watches
# Increase temporarily
echo 524288 | sudo tee /proc/sys/fs/inotify/max_user_watches
# Persist in /etc/sysctl.d/90-inotify.conf
fs.inotify.max_user_watches = 524288Troubleshooting#
| Issue | Cause | Solution |
|---|---|---|
permission denied on remote | SSH key or user permissions | Check SSH key, user ownership, and directory permissions on remote |
| Files not updating | Timestamp/size unchanged (but content different) | Use --checksum flag to compare by content |
| Trailing slash confusion | Source path inconsistency | Review Trailing Slash Behavior section |
rsync: connection unexpectedly closed | SSH timeout, network drop, or out of disk space | Add --timeout=300; check disk space; use -P for resumption |
| Very slow transfer | No compression on WAN, or bandwidth saturation | Add -z for compression; use --bwlimit to prevent saturation |
max connections reached on daemon | Too many concurrent clients | Increase max connections in rsyncd.conf |
chroot failed in daemon mode | rsyncd running as non-root | Run daemon as root, or set use chroot = no |
| Files deleted unexpectedly | --delete with wrong trailing slash | Test with --dry-run first; verify source path |
inotify No space left on device | Too many inotify watches | Increase fs.inotify.max_user_watches |