Transfer a 1GB file in 17 seconds using rsync
How to use rsync to transfer a 1GB file in 17 seconds.
September 24, 2023
rsync stands for "Remote Sync" and is a fast file transfer utility. It only transfers changed parts of files, making it incredibly efficient. Now, let's get into how you can use rsync to transfer a 1GB file in just 17 seconds.
mkfile command comes bundled on macOS. It creates a file on your computer of any size you choose. It can be used to create temporary files for testing.
mkfile command takes 2 arguments, size and filename:
mkfile -n size[b|k|m|g] filename
To create a 1GB file called 'useless_file' type the following in the terminal:
mkfile -n 1g useless_file
A new 1GB file is created in the current working directory. It will be made up of all 0's (this ties into using the
rsync command below).
rsync command also comes bundled on macOS. It can transfer files locally or remotely. It supports a number of features including compression and SSH to copy files over a network.
Here is the command to copy a file within the same directory:
rsync useless_file useless_file_copied
This makes a duplicate of
On my SSD this took about 10 seconds.
To copy this over a network use this command:
rsync -e ssh useless_file <username>@<destination-ip>:~
sshare used together to enable a secure ssh connection.
<destination-ip>are your username credential and address of the destination computer.
Directly after the
:is the save directory on the destination computer. I currently have
useless_filesaving to the users
This took over 2 minutes and 26 seconds to transfer on a local network. To speed this up use the -z flag which enables compression. Because useless_file is all 0's it will compress more than a typical file.
Here is the updated command with compression enabled:
rsync -z -e ssh useless_file <username>@<destination-ip>:~
This time it took 17 seconds to transfer a 1GB file to another computer.
Pretty neat, eh?
But wait… what about using
rsync in the 'real world'?
Because this 1GB file is made up of all 0's, how does this compare to sending something useful?
I took a 234MB .dmg and transfered it with and without compression, here are the results:
- 25 seconds without compression
- 23 seconds with compression
.dmg file was already heavily compressed by Oracle. It is the Java installer
.dmg from their website.
Okay, how about a 730MB
.wav file (people use those):
- 1 minute and 26 seconds without compression
- 41 seconds with compression
That's less than half the time needed to transfer the same file. Pretty sweet, isn't it?
So to conclude, the
rsync command was used to transfer 3 different files to a remote computer on a local network. The first file was 1GB and made using the
mkfile command. It was composed of all 0's and took 17 seconds to transfer.
The second and third transfer showed results using more useful files. A file that is already compressed such as a
.zip, etc) will probably not benefit from adding the
-z compression flag.
But uncompressed files such a
.wav transferred in less than half the time.
rsync is also the backbone of the popular Carbon Copy Cloner software for macOS. It is open source and credited on their website.
Next time you reach for the popular
scp command to copy a file, check out
rsync. It might speed up your file transfers.
Originally published at blog.jdwilson.ca on March 30, 2017.