Rsync guide and cheatsheet

Introduction

Rsync is a powerful and versatile command-line utility for synchronizing and transferring files efficiently between directories, systems, or over a network. It uses delta encoding to transfer only the differences between the source and destination files, minimizing bandwidth usage. This makes Rsync particularly effective for backups, file mirroring, and keeping files in sync across different systems.

Rsync supports a wide variety of options for customization, including compression, preservation of file permissions and attributes, exclusion of files, and remote transfers using SSH or rsync-specific daemons. Rsync is an indispensable tool for efficient file synchronization and backups. With its vast array of options and customization capabilities, it can handle everything from simple local transfers to complex remote backups. Mastering Rsync not only improves workflow efficiency but also ensures the integrity and safety of your data.

General Syntax

Syntax:

rsync [options] <source> <destination>

• source: The directory or files to copy.
• destination: The target directory or location for the files.

Example:

rsync -av /local/path/ user@remote:/remote/path/

Most useful options

Here’s a list with the most useful Rsync options.

• -a: Archive mode. Preserves symbolic links, file permissions, modification times, and directories.
• -v: Verbose mode. Outputs detailed information during the transfer.
• -z: Enables compression for faster transfers over slow networks.
• –progress: Displays progress for each file during the transfer.
• –delete: Deletes files in the destination directory that are not in the source directory.
• –exclude: Excludes specified files or directories from transfer.
• –include: Includes specific files or directories (used with --exclude to selectively include items).
• -e ssh: Uses SSH for remote transfers.
• –dry-run: Simulates the command without making any changes.

Usage Examples

Synchronizing a Local Directory

Synchronizes /source/ with /destination/, including permissions and timestamps.

rsync -av /source/ /destination/

Backing Up Files to a Remote Server

Copies files from the local directory to a remote server over SSH.

rsync -avz -e ssh /local/path/ user@remote:/remote/path/

Excluding Specific Files

Excludes all .log files during synchronization.

rsync -av --exclude "*.log" /source/ /destination/

Deleting Extraneous Files

Keeps the destination directory identical to the source by deleting files not present in the source.

rsync -av --delete /source/ /destination/

Pulling Files from a Remote Server

Fetches files from a remote server to the local system.

rsync -av user@remote:/remote/path/ /local/path/

Note: Rsync daemon is a networked service mode that enables data synchronization and file transfer over a network without requiring SSH. It operates using its own protocol, which is lightweight and efficient. The daemon listens for incoming connections, allowing remote systems to pull or push files to the configured directories. It is highly configurable, supporting features like authentication, access control, and logging. Unlike standadr Rsync over SSH, the daemon allows finer control over access and can work on custom TCP ports.

Using Rsync Daemon

Transfers files using an Rsync daemon with a custom configuration.

rsync -av rsync://remote_host/module /local/path/

Note: for a more specialized guide about Rsync Daemon see: Rsync Daemon guide and cheatsheet.

Advanced Features

Bandwidth Limiting

Limit the bandwidth used during transfer to 500 KB/s.

rsync --bwlimit=500 -av /source/ /destination/

Logging

Output a detailed log file of the transfer.

rsync -av --log-file=/path/to/logfile /source/ /destination/

Partial Transfers

Resume incomplete file transfers.

rsync -av --partial /source/ /destination/

Hard Link Preservation

Preserve hard links during transfer.

rsync -avH /source/ /destination/

Best Practices

Besides its usefulness, Rsync has 4 key points that when used and implemented can be seen as its best practices/uses:

1. Test Before Syncing: Use --dry-run to simulate the command and ensure the output matches your expectations.
2. Secure Transfers: Use SSH (-e ssh) for encrypting remote file transfers.
3. Incremental Backups: Rsync is ideal for creating incremental backups by combining it with options like --link-dest.
4. Automation: Schedule Rsync tasks with cron or other job schedulers for regular synchronization or backups.