Tuesday, June 10, 2014

Backup your data

When you've got a collection of files - pictures, videos, documents etc, it's a pain to lose it. If it's personal - Pictures and videos of family for instance, then it's priceless.

I recently lost a couple drives in my main PC. Thankfully, a backup on another machine pulled through.

There's multiple solutions that can be engineered. A good solution will backup data on another remote server automatically.

Backup on the same machine is ill advised. In fact, let's not call this backup...

If it's the same drive, it's not very useful at all. I only do this if I'm editing a file and want the original as I edit. I certainly don't do this for storage.

If it's on another drive it's better, but not ideal. I've lost data on multiple drives at once. Sometimes it's not the drive that goes bad. A faulty harddrive controller was responsible for the first incident. Recently I lost data on multiple drives again, due to the operating system driver becoming corrupt. Once formatted and reloaded, new drives worked just fine.

It's nice to be able to recover data I removed intentionally. If I've deleted a RAW or video and want it. It's also nice to be able to recover data fully or initiate backup procedures at will.


let's consider Synctoy.



Synctoy is a Microsoft tool for syncing folders.
Get it here 




It's pretty easy to understand. Syncs can be done in either direction, or even both ways depending on date. Errors in copying files will not cause it to stop.

Since this works from folder to folder, you're not really limited to syncing this from a Windows workstation to windows. You can have your alternate run Mac, Linux, Solaris - it doesn't matter so long as you share the backup drive, and map it in the windows machine.



Open explorer, and click on tools->Map network drive.
The above gui shows a very simple setup which will mount the "d$" share from server 192.168.1.18 as drive "Z:" locally.

Now, when I'm browsing what looks like the "Z:" drive on my PC, it's actually the share from the other machine - 192.168.1.18.


Great - So now you have details on how to establish connection to the backup server (map the drive) and setup the sync and run manually (synctoy).
Next - automate it.


Command line is where it's at for automation. 

C:\Program Files\SyncToy 2.1>SyncToyCmd.exe -R "backup"
Preview of backup (E:\, H:\) in time 00:00:26:712.
SyncToy action was 'Echo'
Found 33 actions to perform.
Found 120,270 files that did not require action.
Analyzed 4,502.4 files per second.
Avoided copying 2,779,264,431,572 bytes in 120,270 files.
Saved approximately 08:02:08:00 by not copying any files.

SyncToy run of backup (E:\, H:\) completed at 6/10/2014 12:12:28 PM.
SyncToy action was 'Echo'.
SyncToy options were:
        Active for run all
        All files included
        No files excluded
        Do not check file contents
        Include read-only files
        Include hidden files
        Include system files
        Backup older files (send to Recycle Bin)
        All subfolders included
SyncToy run took 00:01:00:425.
Copied 7,452,599,872 bytes in 21 files in 00:01:00:425.
Bytes per second 123,335,434.1, files per second 0.3.
Avoided copying 2,779,264,431,572 bytes in 120,270 files that did not require ac
tion.
Saved approximately 06:15:34:193 by not copying all files.
Warning: 8 failures occured.
You can retry by selecting "Run" again or select "Preview" to see
the operations that remain to be performed.

The Sync operation completed successfully on folder pair 'backup' but some files
 were skipped. Please look at the logs for more details.



OK - next step, automate that command, and display the errors for you.
Lets make a task:

Open Microsoft Task Scheduler and create a new task:

Key here is to run if the user is logged in or not.

Next - set the trigger as a scheduled time:


I've set this to run daily, at 2:00AM.

Finally, set the command:


I've created backup.bat to run the command:
::run backup
"C:\Program Files\SyncToy 2.1\SyncToyCmd.exe" -R "backup"

Click OK and you'll be prompted for your password to save the task.

There ARE some ways to alert yourself of errors with the backup.

After calling the synctoycmd.exe command to sync, we can have the bat file run a command to alert us only if there's a problem.

How can we run a command, IF the log contains "error"?
C:\>findstr "Error" %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToy
Log.log && echo "RUNS IF ERROR FOUND"

%LOCALAPPDATA% is the Windows Environment variable for the current user's AppData\Local folder.

This will check the synctoy log and if there's text returned, it will run the echo command. If not, it wont run the echo.
So lets say we wanted this to copy the log to the desktop:

C:\>findstr "Error" %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToy
Log.log && copy %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.
log %USERPROFILE%\Desktop\synctoy_ERROR.txt

Now, the log will grow continuously, so after this, it's probably best to rename the log, so that the old errors wont trigger a later successful backup.

e.g. the BAT file may look like this:

::run backup
"C:\Program Files\SyncToy 2.1\SyncToyCmd.exe" -R "backup"

::create error log on desktop if there's errors
findstr 'Error' %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.log && copy %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.log %USERPROFILE%\Desktop\synctoy_ERROR.txt

::archive the last log
type %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.log >> %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.log.ARCHIVE.txt

::delete the log so old data wont trigger the file copy 
del %LOCALAPPDATA%\Microsoft\SyncToy\2.0\SyncToyLog.log

i.e Backup, create error report, archive the log.
Edit: I've made a few changes to use environment variables.
Just copy the above text to a file - say "backup.bat" and run it at a scheduled time.

There are more obvious ways to let you know a backup has failed, such as displaying the errors in the log on the windows background using samurize.
Read more here at lifehacker.