Ticket #87 (closed enhancement: fixed)

Opened 11 years ago

Last modified 10 years ago

use robocopy for backups

Reported by: joshuadf Owned by: k1@…
Priority: major Milestone:
Component: systems Version:
Keywords: Cc: k1@…

Description

The current backups are kept in since very large files, accessible only with Microsoft's very basic Backup tool. We should switch to robocopy for its simplicity:

As an alternative, you could use a command line copy program like robocopy.exe with the scheduler, which should run just fine without user interaction. It will keep the individual files seperate and in the same folder structure on the backup drive instead of packing them into a single archive folder. 

This should only require writing the batch files.

JE will set this up on Jim's computer dendrite as a pilot.

Change History

comment:1 Changed 11 years ago by k1@…

The test run on Jim's machine as a local backup was a success, here's the template for the batch file used:

@ECHO OFF SETLOCAL

SET _source=c:/sourcefolder

SET _dest=d:/destinationfolder

SET _what=/COPYALL /B /SEC :: /COPYALL :: COPY ALL file info :: /B :: copy files in Backup mode. :: /SEC :: copy files with SECurity

SET _options=/R:0 /W:0 /LOG+:MyLogfile.txt /NFL /NDL :: /R:n :: number of Retries :: /W:n :: Wait time between retries :: /LOG :: Output log file :: /NFL :: No file logging :: /NDL :: No dir logging

ROBOCOPY %_source% %_dest% %_what% %_options%

Also note that windows 2000 doesn't have robocopy.exe installed by default, but windows XP does. Robocopy can be easily found for download if it needs to be installed (search google).

comment:2 Changed 11 years ago by k1@…

As for the network backups, some changes will need to be made to the parameters for backup data transfer over the network, such as follows:

@ECHO OFF SETLOCAL

SET _source=
UserComputer?\c$\users

SET _dest=
BackupServer?\e$\Backup\users

SET _what=/COPYALL /B /SEC /Z

:: /COPYALL :: COPY ALL file info

:: /B :: copy files in Backup mode.

:: /SEC :: copy files with SECurity

:: /Z :: copy files in restartable mode (survive network glitch).

SET _options=/R:15 /W:5 /LOG+:ComputerNameBackupLog.txt /NFL /NDL

:: /R:n :: number of Retries

:: /W:n :: Wait time between retries

:: /LOG :: Output log file

:: /NFL :: No file logging

:: /NDL :: No dir logging

ROBOCOPY %_source% %_dest% %_what% %_options%

Depending on how well this script runs further modifications will follow, including possibly scripting a system for incremental backups that can go several versions back.

comment:3 Changed 11 years ago by k1@…

The script has been modified for use over the network, and the following machines are currently being backed up using robocopy:

Atlas Cardia Cyton Dendrite Ligament Oculus

They are being stored on Vomer and the scripts are being run on that server as well.

Currently, all the old backup scripts are still being run until the new system is finalized, but it looks like it's running well.

The robocopy script executes backups daily, and only is stored on a single drive on vomer. A system to possibly store an additional copy on another drive (or preferably on another server) and to also be able to retrieve files that are older than a day old still needs to be implemented.

comment:4 Changed 11 years ago by joshuadf

This is great, JE, thanks. I would recommend making available \\vomer\BackupData\ as a share so our we and users can more easily get to these files, and adding this information to the WindowsOverview page. (And getting the rest of the Windows machines backing up this way, but I assume you're already working on that.)

comment:5 Changed 11 years ago by k1@…

  • Status changed from new to assigned

Some updates regarding our windows backup with robocopy:

  • The wiki is updated with accurate details on how robocopy works and how to recover data from the server.
  • There is now a script that emails the ropocopy logs to admin email addresses.
  • I've tried to get the number of errors reported in the emails down by filtering out certain folders and reducing the number of retry attempts for locked or failed file transfers, but it seems that I need to adjust the wild cards on those filters.

comment:6 Changed 11 years ago by joshuadf

JE,

Once you get the old BKF backups restored, please explode them and robocopy the whole tree to
vomer\BackupData?\archive\ so, for example, I should be able to browse to archive\thorax\users\ben\ and directly copy files. Then we'll have the files in at least 2 places so we can delete the BKF files.

Also, please move the backups for machines we are not using anymore to the archive (oculus, old cardia, atlas, cyton).

comment:7 Changed 10 years ago by joshuadf

  • Status changed from assigned to closed
  • Resolution set to fixed
Note: See TracTickets for help on using tickets.