Rsync tips and tricks

This post is about tips and tricks for Rsync.
Rsync is a great tool used by many for backup, or just copying data from one server to another and even for local copying of multiple files or syncing folders.
In this post I will show you some examples of what I use rsync for every day
Normally you would use the rsync program like this, to copy files from one folder to another:

rsync -zrav /source/dir/ /destination/dir

What does the parameters do?
z = Compress the data stream, this can improve performance if you copy from one server to another over the network
r = recurse into directories
a = Keep the attributes, this is for example the timestamp on which the file was last modified, or created.
v = Verbose, this just makes sure you see some helpful info while it copies.
If you want to sync and folder to another server via. ssh. you can use the following:

rsync -zrav /local/source/dir/ root@remote-server:/destination/dir/on/remote/server/

There are some other useful parameters you can append to the commands above. Here are some examples that I use a lot:

—delelete

This will keep the folders in sync, and delete files from the destination if they do not exist on the source.
Let’s say I have the following files in my source directory:

file1.txt
file2.txt
file4.txt
file5.txt

And I have the following in the destination directory:

file1.txt
file3.txt

If I run the following commmand:

rsync -zrav --delete /source/ /destionation/

It will delete the file “file3.txt” in the destination, and copy over the other files from source to destination so they both look like the source at the end:

file1.txt
file2.txt
file4.txt
file5.txt

file1.txt will only be overwritten if the file on source is newer than file1.txt in the destination.

—bwlimit

I have run into the problem before that rsync’ing large files to another department in the company can result in using up all the bandwidth to that department. This was a problem since there was not enough bandwidth for emails and other programs making all other systems slow.
This is really only useful if you copy from one server to another.
To prevent this, you can set a limit in KB/second simply by adding the following parameter:

--bwlimit=

In this example, I will limit the file copy to 5MB/s or 5000KB/s (5Mbit):

rsync -zrav --bwlimit=5000 /source/dir/ root@destination-server:/destination/dir/

—progress

Another reason I use rsync instead of a simple copy, is because it can show progress!!
Simply add the following parameter:

--progress

And you will se nice detailed progress, with speed, time left, file size and everything you need.
Here is and example:

rsync -zrav --progress /source/dir/ /destination/dir/

—dry-run

Are you not sure you typed the command correctly?
Maybe you are nervous that you delete sensitive production data by using the –delete parameter?
Then this is the parameter to use!
Add the following to the command:

--dry-run --progress

And the rsync script will not copy, delete or do any file modifcations at all, it will just show the output, as if it did.
This is great if you want to be sure you typed everything correctly, because you can view what the results is going to be before you do it.

Leave a Reply

Your email address will not be published.