Introduction
Last time I had the task to copy massive amounts of data between two file share systems. With massive amounts I talk about Terabytes 🙂 In addition, one of the systems was productive and the other one the new system, so the solution needed to be reliable and stable. So I asked myself, how can I transfer these files and folders? The answer is Robocopy: How to Copy Massive Amounts of Data
So I tried several things and had a look at the possibilites of the systems. I ended up using the simple Robocopy command with the file shares. Of course the client/server executing the Robocopy command had to be in the same local network in order to ensure a good copy speed.
Robocopy offers you the opportunity to define a lot of parameters for your copy job. For example you can define if the Access Control List (NTFS Permissions) should be copied to the destination host, or if you want to create the Access Groups from scratch. Imagine that on the old system there was a lot of direct access set for the users instead of Active Directory groups – in this case building the Access Groups from scratch can be very helpful to get a clean file share.
Another great thing about Robocopy is that it has basically a Resume option because it copies only the changed or new files after comparing source and destination host.
The Robocopy Command
But let’s talk about the command itself which is pretty simple and you can fulfill nearly all requirements with a One-Liner!
Basically you can split the command as following:
- Source
- Destination
- Source Options
- Copy Options
So let’s start with the probably most basic command. We want to copy a local directory from one partition to another
robocopy c:\temp\test d:\temp\
This simple command will copy the folder test and all files to the location d:\temp – but not subfolders!
Copy Everything (including permissions)
If you simply want to copy everything (also subfolders, empty subfolders, file owners) from the source to destination including the NTFS permission (Windows Access Control list) you can use this command.
robocopy c:\temp\test d:\temp\ /E /COPYALL
Copying over Network
If you want to use the Robocopy command for copying over the network, like in my case, I recommend you to use the following options and switches to get a good result.
/E -> copy all subfolders, including empty ones.
/V -> produce Verbose output, showing skipped files.
/ZB -> use restartable mode; if access denied use Backup mode.
/R:1 -> number of Retries on failed copies
/W:1 -> Wait time between retries: default is 30 seconds
/NP -> No Progress – don’t display percentage copied.
/COPYALL -> COPY ALL file info (equivalent to /COPY: DATSOU)
/LOG -> LOG location
robocopy \\source_share\folder \\destination_share\folder /E /V /ZB /R:1 /W:1 /NP /COPYALL /LOG:c:\robocopy.log
So, this ultimative copy job will copy all subfolders, copy all file information, produces verbose output to show skipped files, will retry failed copies and uses the restartable mode with backup mode in case of failed access. In Addition a log file will be generated with all the details of every single file copied. It is simply great! 🙂
Please find the whole documentation of the Robocopy command directly at Microsoft Docs
Robocopy: How to Copy Massive Amounts of Data
