PDA

View Full Version : disk to disk to tape and offsite thoughts...



mawhin
April 29th, 2010, 04:57 AM
Hiya...

I'm planning a new amanda installation. We'll have a server capable of storing let's say ten vtapes. Raided, etc.

But we want, weekly, to take a physical tape offsite. Now I don't want, every wednesday night (for thursday is the day that the tape goes offsite), to do a full backup separately to the normal backup. Nor do I want to force all the backups to full. This would push our backup window out, and doesn't seem necessary...

What I'd like to do is use the backups that I've already got to construct a full set for the tape.

I'm thinking along the lines of doing the usual amdump run, using amadmin info to work out which files on which vtapes I'll need for the offsite tape, copying them to a place I've identified, then using another amanda config to back that up straight to tape...

Is this just silly, is there a better way?

Mart

mawhin
April 30th, 2010, 03:45 PM
I run amdump DailySet1...

I run the following which prints a list of files on vtapes...

and cp -al all of them to an archive directory.

Which I'll probably just write straight to tape. Although there could be advantages to running them from within amanda as a second backup set...


So basically I get an offsite tape with all the level 0 dumps and all the differentials needed to bring it up to date. A bit of a pain to restore, but this offsite malarky isn't about restoring individual files from maybe last june. It's about restoring a whole site to as recently as possible...

Thoughts? Better ways?

Mart


-----snip----


#!/usr/bin/perl

$SET = "DailySet1";
$VTAPEROOT = "/export/amanda-vtape/test/mytape";

@export = `amadmin $SET export \| egrep -v "^#"`;
chop @export;
foreach $line (@export) {
# print $line . "\n";
if ( $line =~ /^host:/ ) {
( $dummy, $host ) = split(/ /, $line);
}
if ( $line =~ /^disk:/ ) {
( $dummy, $disk ) = split(/ /, $line);
}
if ( $line =~ /^stats:/ ) {
( $dummy, $level, $dummy, $dummy, $dummy, $dummy, $numontape, $tape ) = split(/ /, $line);
$disk =~ s/\//_/g;
$where = `find $VTAPEROOT -name "00000-$tape"`;
chop $where;
$where =~ s/00000-$tape//;
$out = sprintf( "%s%05d.%s.%s.%s", $where, $numontape, $host, $disk, $level);
print "$out\n";
}
}

RobertT
May 10th, 2010, 06:22 PM
Mart,
Hmmm, nice job getting that information out of the esoteric output. :)

I'm running 2.6 and the file name looks to have changed slightly (a . instead of a -).

Do you use a tape split or memory chunking for no-holding-disk backups? If so you'll have more than one file for the large backups.

I added in an extra find to look for all the backups on the vtapes that look to have the right name format for the backup.



# $out = sprintf( "%s%05d.%s.%s.%s", $where, $numontape, $host, $disk, $level);
$findwhat = "*$host.$disk.$level";
$out = `find $where -name "$findwhat"`;
print "$out";


I had some newlines turn up in the output with that change. I used grep to get rid of them but you may prefer perl. [EDIT: Removed newline from the print statement. Fixed.]

Also I suggest adding in the 00000* tape headers and a backup of your /etc/amanda/DailySet1 (which I think has all your tape index information). They're not very big and I think those would help with getting Amanda to do auto tape changing and extracting backups from the tape chunks.

Robert

RobertT
June 22nd, 2010, 11:12 PM
Added in a line to print out the file name of each tape's header

Here's the script for 2.6:


#!/usr/bin/perl

$SET = "Daily";
# Directory all the vtapes are in (ie. where slotXX folders are)
$VTAPEROOT = "/Backup01/amanda/Daily/vtapes";

@export = `amadmin $SET export \| egrep -v "^#"`;
chop @export;
foreach $line (@export) {
# print $line . "\n";
if ( $line =~ /^host:/ ) {
( $dummy, $host ) = split(/ /, $line);
}
if ( $line =~ /^disk:/ ) {
( $dummy, $disk ) = split(/ /, $line);
}
if ( $line =~ /^stats:/ ) {
( $dummy, $level, $dummy, $dummy, $dummy, $dummy, $numontape, $tape ) = split(/ /, $line);
$disk =~ s/\//_/g;
$where = `find $VTAPEROOT -name "00000.$tape"`;
chop $where;
print "$where\n";
$where =~ s/00000.$tape//;
$out = sprintf( "%s%05d.%s.%s.%s", $where, $numontape, $host, $disk, $level);
# $out = sprintf( "%s%05d.%s.%s.%s", $where, $numontape, $host, $disk, $level);
$findwhat = "*$host.$disk.$level";
$out = `find $where -name "$findwhat"`;
print "$out";
}
}

amanda_fan
June 23rd, 2010, 07:46 AM
Hi,

I have a requirement to do pretty much exactly the same thing and actually posted a thread at

http://forums.zmanda.com/showthread.php?p=10360#post10360

a couple of days ago, to which RobertT replied (thanks Robert), pointing me at this thread. The script you have written, combined with the subsequent cp command and write to tape operation appears to do exactly what I require.

I would be very grateful if you could clarify a couple of things for me so that I can confidently use this script, if appropriate:

(i) A section of one of your earlier posts (April 30) states


I run the following which prints a list of files on vtapes...

and cp -al all of them to an archive directory.

Which I'll probably just write straight to tape. Although there could be advantages to running them from within amanda as a second backup set...


So basically I get an offsite tape with all the level 0 dumps and all the differentials needed to bring it up to date. A bit of a pain to restore, but this offsite malarky isn't about restoring individual files from maybe last june. It's about restoring a whole site to as recently as possible...

When you say that 'you'll probably just write straight to tape' - I presume you mean that you just use 'tar -cvf' or some such command to write the appropriate vtape file list to a physical tape?

(ii) In the above-mentioned quoted section, you also state that you "get an offsite tape with all the level 0 dumps and all the differentials needed to bring it up to date". Am I correct in my interpretation of what you mean by this that what you get is basically level 0 dumps and sufficient differentials to bring you up to Wednesday night, in advance of your Thursday morning offsite requirement.

(iii) Any thoughts on how you will restore from this tape since obviously (or at least 'what I think is' :) ), the VTAPEROOT location when the files are extracted from the tape will be different from the original $VTAPEROOT location, so a different amanda.conf will be required. I think a new config with a new amanda.conf (referring to a new VTAPEROOT location) may be one way of doing this but I may be wrong.

Just getting my head around Amanda at the moment, as you probably realised but any feedback would be very welcome!!

Thanks,
amanda_fan.

RobertT
June 23rd, 2010, 06:00 PM
IMPORTANT NOTE: The script does not cope with backups split across multiple tapes. It will only list the files on the first tape of a particular DLE backup.

I was just having another look at how it works and the script is getting the list of tapes from amadmin export which looks to only list the first tape used.

Mart didn't seem to be using tape splitting so he wouldn't have come across it.

I think adding support for split tapes could be done fairly easily using the /etc/amanda/<config>/tapelist to work out what tapes are from the same date and adding in an extra loop.

Re: (ii)
Mart's script is pulling out the file for the most recent backups of each level (reported by amadmin export). You would need to restore from the level 0, then the level 1 etc.

Re: (iii)
I think it should just be a matter of copying the /etc/amanda/DailySet1 to /etc/amanda/Restore and then changing the paths in Restore/amanda.conf appropriately.
You should then be able to connect using amrecover Restore

I suggest making sure you have a backup of the config folder along with your backups (regular and off site) which will make getting Amanda to handle your restores automatically much easier.

RobertT
June 23rd, 2010, 09:26 PM
I've updated the script to try to support tape splitting.

Seeing as I'm working under 2.6 and am not sure of what the path/file differences are in 2.5 I've made a new thread for my version on the 2.6 board.

http://forums.zmanda.com/showthread.php?p=10376

If anyone modifies it for 2.5 please post it back here.

Note: I haven't done any testing apart from seeing that it gives output without erroring.

Robert