Sync Remote Directories

rsync command

Use rsync on the command line:

$ rsync -av -e "ssh -p [NON_DEFAULT_SSH_PORT]" [USER]@[REMOTE_ADDRESS]:/path/to/remote/dir/ /path/to/local/dir

Arguments

• a flag: Stands for "archive" and copies recursively, preserving permissions, symbolic links, etc.
• n flag: "Dry run" option for testing purposes.
• v flag: Verbose.
• u flag: Only copy files with a newer modification time (or size difference if the times are equal)
• P flag: Show progress
• i flag: Itemize changes.
• e flag: Method for using a non-standard ssh port if the remote host is configured with a non-standard ssh port.

• --exclude specifies one subdirectory to exclude from the operation. The directory name must be followed with at trailing slash.
• --exclude-from specifies a text file containing a list of directories and files to exclude from the sync. This may be a more manageable option than multiple --exclude directives.
• --delete option will cause target files and directories to be deleted if they are not found within the source directory.
• --size-only will cause rsync to ignore metadata when comparing files and only look at file size.

Source and target paths

N.B. The trailing slash on the source path is required when copying directories. For the target path, no trailing slash will cause the directory to be the root of where files will be copied. A trailing slash on the target path would cause the source directory to be copied into the target directory, e.g. /path/to/local/dir/dir.

The order of remote vs local paths matters. Local path must be on the left in order push files from local to remote. Similarly, remote must be on the left to push files onto local.

Home directory on remote can be specified with ~ after the semicolon, e.g. [USER]@[REMOTE_ADDR]:~/path/below/home/

Whitespaces in paths

On linux, whitespace is handled with the --protect-args argument. This argument isn't included in the Mac OS distribution of rsync.

On Mac, quote the path and escape white spaces, e.g. '[USER]@[REMOTE]:~/path/to/directory\ with\ spaces/'

Previewing files that would be transferred

rsync's -i or --itemize-changes parameter will list the files that are different between the two locations.

Use egrep to filter out the files that differ in file size only:

$ rsync -ain [source] [dest] --exclude [excluded_path] | egrep "^>f[\+\.][\+s]"
# or, to include files that will be deleted
$ egrep "^[<>\*](?:f\.s|f\+\+|\delet)"

This page explains the output of the itemize changes flag.

This StackOverflow answer has a concise description of the relevant parts of the flags.

>f.st......

> - the item is received
f - it is a regular file
s - the file size is different
t - the time stamp is different

.d..t......

. - the item is not being updated (though it might have attributes 
    that are being modified)
d - it is a directory
t - the time stamp is different

>f+++++++++

> - the item is received
f - a regular file
+++++++++ - this is a newly created item

Troubleshooting

Symptom

Error message when running rsync: rsync error: error in rsync protocol data stream (code 12)

Possible solution

Provide path to rsync executable on the remote machine with the --rsync-path parameter, e.g.:

$ rsync --rsync-path="/bin/rsync" -avn -e "ssh -p [NON_DEFAULT_PORT]" /path/to/local/file [REMOTE_USER]:[REMOTE_HOST]:~/