PDA

View Full Version : Verifying AMANDA backups



tamiral
June 6th, 2007, 08:33 AM
I'm a SysAdmin from a relatively medium-sized company dealing with DVRs.
I've been searching for a suitable backup solution and got AMANDA highly recommended by our company's old-timers...

After a while, I now have an AMANDA 2.5.1p fully-functional server (on Debian 4.0) which successfully (according to the logs), backs-up whatever I want to VTapes (on a 500GB RAID5 array storage-space).

Few questions though:
- How can I verify that AMANDA has successfully backed-up files? I'm aware of amverify, but I'd like to verify this with an
actual restore command and compare the files myself. Is there a way to restore AMANDA backups to a different client
other than the original location which the files were taken from?
- I'm quite worried about file permission issues. If a file is not readable by 'backup' user, thus cannot be read for backup
purposes, will it appear on the logs?
- How can I restore specific files from a successful backup rather than the whole backup/tape?

I'll probably have more questions along the way, but enough is enough for now...

Thanks!
Tamir.

shailen
June 6th, 2007, 09:04 AM
Hi Tamir,

Here are the answers to your questions.

- How can I verify that AMANDA has successfully backed-up files? I'm aware of amverify, but I'd like to verify this with an actual restore command and compare the files myself. Is there a way to restore AMANDA backups to a different client
other than the original location which the files were taken from?

You can use amrecover for this. You can run amrecover from either the original client or a different client, as long as you specify the correct backup server/configuration name. You can find more information here:
http://wiki.zmanda.com/index.php/Amrecover

Please remember if you restore to the original client, amrecover will restore the data to the directory you are running amrecover from by default. So you can create a directory called /restore and cd into that directory and run amrecover from there. Be sure to setup the amanda-client.conf (on the client side) as described in the above link.


- I'm quite worried about file permission issues. If a file is not readable by 'backup' user, thus cannot be read for backup purposes, will it appear on the logs?

Amcheck will pick this up. So if amcheck works successfully for your backup set(s), you should have no problems during backup time. if there are issues thats crop up between the time you ran amcheck and the time the backup runs they will crop up in the logs.

- How can I restore specific files from a successful backup rather than the whole backup/tape?

Amrecover can help you do this, once you run, amrecover is an interactive shell where you can select which files/directories to extract (or you can select to extract all files/directories).

Hope that helps to answer your questions!

-Shailen

tamiral
June 6th, 2007, 09:53 AM
Great, thank you. I was only aware of amrestore.

Another issue which concerned me was part of my AMADA server configuration.

As I mentioned, I have a 500GB storage space, and I specified 25 VTapes to be used by AMANDA:

Initially I've calculated 450GB / 25vtapes = 18000MB per vtape.
That didn't work out for me. I kept on receiving "dump larger than available tape space" on the logs, so I changed to:


define tapetype HARDDISK {
length 1645200 mbytes
}

Besides that, I've specified:


holdingdisk md1 {
use 500000 Mb }

Everything's seems to be working fine, but I'm concerned that because my defined tape size is larger than the amount of diskspace I have (tapesize x 25), AMANDA will get somehow confused...

Should I be?

shailen
June 6th, 2007, 12:13 PM
Great, thank you. I was only aware of amrestore.

Another issue which concerned me was part of my AMADA server configuration.

As I mentioned, I have a 500GB storage space, and I specified 25 VTapes to be used by AMANDA:

Initially I've calculated 450GB / 25vtapes = 18000MB per vtape.
That didn't work out for me. I kept on receiving "dump larger than available tape space" on the logs, so I changed to:


define tapetype HARDDISK {
length 1645200 mbytes
}

Besides that, I've specified:


holdingdisk md1 {
use 500000 Mb }

Everything's seems to be working fine, but I'm concerned that because my defined tape size is larger than the amount of diskspace I have (tapesize x 25), AMANDA will get somehow confused...

Should I be?

1 )if you are backing up to disk (vtapes) you shouldn't need a holding disk

2) How much is one full backup, of all the clients in your backup set? also, are you using any compression.

3) How often do you want to perform a full backup? and how long do you want to retain your backups?

Oversizing your vtapes won't confuse amanda. That size just lets amanda know that it can use UP to that for each tape. It may never hit that limit. But you should do your best to size your tapes based on the answers for questions 2 and 3 above.

Hope that helps!

-Shailen

dustin
June 6th, 2007, 12:43 PM
1 )if you are backing up to disk (vtapes) you shouldn't need a holding disk

Note, however, that without a holding disk you can only write one dump at a time to your vtapes (since they are emulating tapes, and tapes can only write one thing at a time)


Oversizing your vtapes won't confuse amanda. That size just lets amanda know that it can use UP to that for each tape. It may never hit that limit. But you should do your best to size your tapes based on the answers for questions 2 and 3 above.

Note, however, that Amanda will try to use as much tape space as possible, by promoting dumps it could do as incrementals to full dumps. As Shailen says, you should make sure your vtapes can all fit on disk when they are completely full, or things will probably go wrong eventually.

Regarding your initial question, about dumps not fitting on disk -- see
* http://wiki.zmanda.com/index.php/How_To:Split_Dumps_Across_Tapes
* http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists

tamiral
June 7th, 2007, 12:29 AM
1 )if you are backing up to disk (vtapes) you shouldn't need a holding disk

2) How much is one full backup, of all the clients in your backup set? also, are you using any compression.

3) How often do you want to perform a full backup? and how long do you want to retain your backups?

Oversizing your vtapes won't confuse amanda. That size just lets amanda know that it can use UP to that for each tape. It may never hit that limit. But you should do your best to size your tapes based on the answers for questions 2 and 3 above.


I think I'd prefer using holdingdisk space, in order to perform faster backups, wouldn't I?

I'm using comp-user-tar as my compression method. All of my backup clients (uncompressed) are roughly ~300GB. Should that cause a problem with my current vtapes configuration? Should I use a different compression method? I'm basically backing-up directories, not whole disks.

How do I set level 0 backups? I was sure that each time AMANDA cycles its tapes it performs a level 0 backup... I would actually prefer doing level 0 backups once each week (hopefully without compromising too much valuable diskspace).

Regardless to that, is there a way to completely erase all saved backup files and start level 0 backups all over again?

One more question that came to mind: While trying to configure and test AMANDA, I've specified a certain PC which didn't really needed to be backed up. It takes ~60GB of the storage diskspace and I don't want it backed up anymore (that's why I asked about completely deleting all saved backups).
If I remove that DLE, how will AMANDA actually handle this? Will it wait until the tapes which holds those files will be overwritten? Is there a manual way to delete those files from the disk?

dustin
June 7th, 2007, 08:08 AM
I'm using comp-user-tar as my compression method. All of my backup clients (uncompressed) are roughly ~300GB. Should that cause a problem with my current vtapes configuration? Should I use a different compression method? I'm basically backing-up directories, not whole disks.

You have 500G of vtapes, so that should be fine, if a little tight (when you add more systems, you may need more space).


How do I set level 0 backups? I was sure that each time AMANDA cycles its tapes it performs a level 0 backup... I would actually prefer doing level 0 backups once each week (hopefully without compromising too much valuable diskspace).

See
http://wiki.zmanda.com/index.php/FAQ:How_do_I_make_Amanda_do_full_backups_on_weeken ds_and_incrementals_during_the_week%3F


Regardless to that, is there a way to completely erase all saved backup files and start level 0 backups all over again?

Yes, but I'm not sure why you'd need to do that except to wipe out a test setup. If you want to do that, just delete all of your vtapes and your curinfo and logs to revert to the state your system was in before the first run.


One more question that came to mind: While trying to configure and test AMANDA, I've specified a certain PC which didn't really needed to be backed up. It takes ~60GB of the storage diskspace and I don't want it backed up anymore (that's why I asked about completely deleting all saved backups).
If I remove that DLE, how will AMANDA actually handle this? Will it wait until the tapes which holds those files will be overwritten? Is there a manual way to delete those files from the disk?

If you remove it, you will not be able to restore it, but Amanda will not try to go back to the tapes and "delete" the dumps for that host. You can delete the files manually, but I would suggest just emptying them, rather than deleting them, with:

:> /path/to/vtape/00007.host._path_to_backup.0

(that ':>' is part of the command!)

dustin
June 8th, 2007, 03:00 PM
I added some additional entries to the wiki based on this thread. Please feel free to edit / amend if you see anything missing:
* http://wiki.zmanda.com/index.php/Dump_larger_than_available_tape_space
* http://wiki.zmanda.com/index.php?title=How_To:Verify_backups

tamiral
June 10th, 2007, 12:31 AM
Great, thanks.

During the weekend I stumbled into another problem;
Due to all of my tests I ended up writing Thursday's backup to the last tape on the cycle (labeled DailySet1-25).
On Friday, AMANDA didn't cycle to the first vtape, and sent me a message "A TAPE ERROR OCCURRED: [No writable valid tape found]".

I had to:


$ /usr/sbin/amtape DailySet1 slot first

for the changer to use the first tape again.
I'm still not sure that this will fix the problem, I'm running amflush now and once it's done I'll run amdump again,
but considering that this is the problem (it makes sense...), is there a way for AMANDA to cycle to the first slot automatically once it has reached the last tape?

tamiral
June 10th, 2007, 11:58 AM
One more thing;
I tried to find an understandable document about the logs generated by amdump and sent via e-mail.
I have some things misunderstood there.

dustin
June 11th, 2007, 08:43 AM
Re: having to reset the changer -- Amanda will manipulate the changer itself as necessary, so I think that there may be a problem with your tapecycle. Can you post the relevant parts of your amanda.conf? (dumpcycle, tapecycle, and so on)

Re: log files -- see http://wiki.zmanda.com/index.php/Amanda_log_files. Also, the troubleshooting section lists common causes for many error messages.

tamiral
June 12th, 2007, 06:40 AM
Re: having to reset the changer -- Amanda will manipulate the changer itself as necessary, so I think that there may be a problem with your tapecycle. Can you post the relevant parts of your amanda.conf? (dumpcycle, tapecycle, and so on)

I hope so... I'm already at tape 19 of 25, so I'll probably wait for another cycle and see whether I have to manually set it to slot1. If not, I'll consult here.

I wanted to ask something else though:
It turns out that I have to backup the whole filesystem on one of our servers.
Basically, I'd rather use 'dd' for those kind of jobs (even unmount the filesystem so it won't change), but I guess I can't with AMANDA.

What is the recommended dumptype for this kind of backup (I'm usually using comp-user-tar)? I basically want to backup a root (/) directory with minimum restore time.
Compression shouldn't be considered an issue since it's relatively small (~20GB).

Thanks.

tamiral
June 16th, 2007, 10:52 AM
UP!

I really need help with the recommended dumptype for root folders or devices...

Thanks.

paddy
June 16th, 2007, 04:37 PM
If your criteria is minimal restoration time, you will have to

- do full dumps
- no compression
- no encryption

Please note that this will increase backup time and backup media needed.

If the root filesystem only contains system files (users have different filesystem), the data can restored from distribution. You can just do incremental dumps. But, the restoration time will be longer.

Paddy