Sync files from remote system via command line - Linux, Ubuntu, Debian

If you are using more than one laptop/computers in your day to day life for eg: Your personal laptop and Office laptop. Chances of needing a file from one laptop on the other laptop is very common. We don't want to copy the files over the system again and again. We just want the changes to the files made in one system to reflect on the other system.

In this post we will see a simple way to keep files in sync between your multiple laptops/computers. Linux has an wonderful utility called rsync which can be used to sync files/folders between systems in a very optimised way (copy only modified and new files/folders)

If you don't have rsync utility installed on your system, install it by

$ sudo apt-get update
$ sudo apt-get install rsync

Also we are planning to sync files from remote systems, So lets have the openssh-server installed on the both machines

$ sudo apt-get install openssh-server

openssh-client will be by default installed on linux machines, if not install it

$ sudo apt-get install openssh-client

Basic Usage of rsync

$ rsync source destination

The above command will simply send/sync all the files recursively under the directory 'source' to the 'destination'
To send/sync files to a remote systems

$ rsync source username@host:destination

Note: "username@host:"  will point to home directory of username in the host (Please note the colon (:) at the end of hostname)
With some more options from rsync we can create a single line file syncronizer as follows

$ rsync -avzP username@host:source destination

Options used:

-a     - archive mode
-v     - increase verbosity
-P     - To see the file transfer progress with its speed
-z     - compress file data during transfer
         Use this option so that you can utilise your network bandwidth efficiently

Note: I have changed the username@host term to the source option to intimate that remote host can be either source or destination.

Skip Some files from Sync/Copy

$ rsync -avzP --exclude "*.conf" username@host:source destination

--exclude option will exclude the specified file(s) syncing from source to destination. You can give a file name, Directory name or even file globs like (*.something)

To skip more files

Enter your list of files to be skipped/excluded in a text file and use that in rsync with  --exclude-from option

# File exclude-list.txt
*.conf
bin
blib

Now pass this file name to --exclude-from option

$ rsync -avzP --exclude-from exclude-list.txt username@host:source destination

Some more options to make it more finer tool

Sometimes we might have just changed the permissions/groups for a file/folder but we don't want to reflect that change on the other machine.

$ rsync -avzP --no-group --no-perms --no-owner username@host:source destination

You can create an alias of the above command and make life even more easier. rsync utility has many other options available please do check manual of rsync for more options (man rsync).