Centrally managing exclude lists from Amanda server?

[Andrew Rakowski]

Hi folks...I'm trying to get Amanda up and running for a number of systems (perhaps 100 or so per Amanda backup server), and have perhaps a dozen systems backing up at the moment without any major problems. I have Linux i386 and Solaris working, and still need to build Amanda clients for IRIX and other flavors of Linux (ia64,

However, I'm realizing that there are a number of things that I'd like to exclude from backups. In particular, I'd like to be able to keep the list of "we always want to exclude these" items on the central Amanda backup server. That way, if we implement some new software that uses a directory or file we don't want to backup, I can change the master exclude list in one place and have it take effect on all our backup clients.

I've read the Wiki page on Excludes at:

http://wiki.zmanda.com/index.php/Exclude_and_include_lists

and the "exclude" directive entry in the Amanda.conf document at:

http://wiki.zmanda.com/index.php/Amanda.conf

but it seems that it is NOT behaving as (at least I) expected.

Here's an example showing my dumptypes with several exclude directives included. The "exclude list" and "exclude file" entries seem to only apply to files that reside on the client, and I don't see any way to implement a "central" file (or even list of dumptypes directives) that will let me control things.

So, on the Amanda server, we have:

....
-sh-3.00$ head -22 EMSLdumptypes.conf
define dumptype global {
comment "Global definitions"
#
# Enable the two ssh lines below when we want to move to ssh
# connections between server and clients.
#
# ssh_keys "/var/lib/amanda/.ssh/id_rsa_amdump"
# auth "ssh"
auth "bsdtcp"
}

define dumptype EMSL-full-weekly-inc-daily {
global
program "GNUTAR"
comment "Weekly full backup with daily incrementals via tar"
tape_splitsize 1 Gb
priority medium
index
dumpcycle 7
exclude file append "/etc/amanda/exclude-list"
exclude append "./vicepb"
exclude list optional append ".amanda.excludes"
-sh-3.00$
....

and when I test on the server using amcheck:

....
-sh-3.00$ amcheck EMSLdaily -c maxwell

Amanda Backup Client Hosts Check
--------------------------------
Client check: 1 host checked in 0.312 seconds, 0 problems found

(brought to you by Amanda 2.5.1p2)
-sh-3.00$
....

and then go look on the client, here's what I see being excluded (I've included the test contents of what's on the client files - obviously, I'd actually want stuff like /proc, /mnt, /media, /cdrom, etc. in my excludes):

....
maxwell{417}# cat /etc/amanda/exclude-list
./usr/vice/BLAH-from_etc_exclude-list_file/*
maxwell{418}#
maxwell{418}# cat /.amanda.excludes
./usr/vicepa-from-local-exclude-list
maxwell{419}#
maxwell{419}# \rm -rf /tmp/amanda/
maxwell{420}#
maxwell{420}# cat /tmp/amanda/selfcheck._.20070105141536.exclude
./usr/vice/BLAH-from_etc_exclude-list_file/*
lost+found
./usr/vicepa-from-local-exclude-list
maxwell{421}#
....

So, I don't see "./vicepb" in there anywhere (that would be a centrally specified file to exclude), but I do see both "list" and "file" entries listed as excluded.

Oh, the Amanda server is running on RedHat Enterprise Linux WS release 4. The client is running Solaris-10, and using "tar (GNU tar) 1.16.1" for its backup program. Both are running Amanda-2.5.1p2 software.

Finally, I *think* the definitions of "list" vs "file" aren't describing what I'm seeing. Both seem to specify files (on the client) that contain exclude patterns used by GNU tar to exclude objects from backups. My understanding was that the "exclude list" should let me specify individual excluded items.

So, can anyone take a look at the above and tell me either what I'm doing wrong, or what I need to do to all my to specify what I want excluded only from the Amanda backup server?

Thanks in advance for any thoughts, ideas, pointers, RTFMs indicating the proper Fine Manual to read, etc.

Cheers, and happy weekend!

-Andrew

ps: sorry for the long post, but I wanted to get as many details that might be needed in as possible. Please let me know what I managed to forget! :)

[ktill]

Hi,

exclude file, e.g. /etc/amanda/exclude-list in the case, needs to resides on each client.
If the pattern of the files are known, exclude list is what you need.

Kevin Till

[Andrew Rakowski]

Hi,

exclude file, e.g. /etc/amanda/exclude-list in the case, needs to resides on each client.
If the pattern of the files are known, exclude list is what you need.


Hi Kevin, thanks for the reply.

What I'm trying to do is avoid having to push changes to configuration (exclude-list) files on several hundred backup clients when I realize I need to exclude another item (I'm moving from Networker backups to Amanda and trying to make the transition without losing more Networker features than necessary.)

I agree that the Exclude List entry refers to something on the CLIENT, and that seems to work fine for me. What I was referring to was the ability to directly exclude CLIENT directories from an entry in the SERVER's dumptype section. I'm assuming that the dumptype entries only exist on the Amanda server, not the client (but I'm new to Amanda, so I could easily be misunderstanding that.)

In the http://wiki.zmanda.com/index.php/Exclude_and_include_lists documentation, it says:

The easiest way to exclude a file or directory is to specify it with the "exclude" option in the dumptype. This option accepts an argument of the file or directory to be excluded. AMANDA allows only one exclude option in any dumptype at a time.

UPDATE: Recent AMANDA-releases bring the option "exclude append" which enables the administrator to define more than one exclusion-pattern within one dumptype without using a exclude-list. Please look at the amanda.conf manpage for details.

Any path specified to be excluded must be encapsulated with quotes. Continuing with our example from above /var/log/somefile and using the same dumptype as above, the dumptype would now look like:

define dumptype exclude-test {
comment "test dumptype for documentation"
priority high
program "GNUTAR"
exclude "./log/somefile"
}


which seems to imply that you can exclude at least one file or directory directly in the dumptype entry (and the "Update" would imply that you can do multiples of these if you use the new "append" option to the "exclude" directive.) In the documented example, it seems that I could exclude all the "./log/somefile" files on client via that entry on the server. That, combined with the ability to do more than one (via the "append" option) would be perfect in my case.

Cheers,

-Andrew

[ktill]

Hi,

exclude list FILENAME1
exclude list append FILENAME2

both FILENAME1 and FILENAME2 must reside on the client. Amanda builds the list on the client and make use of --exclude-from option of gnutar.

If I know what file to exclude, I will do :

exclude file "/proc"
exclude file append "/tmp/log"

Hope this helps!

--Kevin Till

[Andrew Rakowski]

{...snip...}
If I know what file to exclude, I will do :

exclude file "/proc"
exclude file append "/tmp/log"


Hmmm, could I have found a bug in 2.5.1p2 perhaps? I'm leaning towards operator error, but read below for what I'm doing that's not working. I see the "exclude list" directive specifying which file(s) on the CLIENT contain lists of things to exclude - that part works fine for me.

However, the "exclude file" directive (which you show as being used to exclude selected files) seems to just specify addition CLIENT-side files containing other files to exclude. For instance, my exclude directives from my SERVER dumptypes file are:

-sh-3.00$ grep exclude EMSLdumptypes.conf
exclude file "/vicepb"
exclude file append "/etc/amanda/exclude-list"
exclude list optional append ".amanda.excludes"
-sh-3.00$

but I do NOT see /vicepb being excluded when I look at the "selfcheck*.exclude" debug output:

maxwell{477}# cat /tmp/amanda/selfcheck._.20070108151944.exclude
lost+found
./usr/vice/cache/*
./tmp
./proc
./mnt/*
./media/*
./cdrom*
./vol/*
./file_to_exclude_via_optional_.amanda.excludes_file
maxwell{478}#

That list shows files that are listed in the *contents* of both an "exclude list" and an "exclude file" specified file. The "/vicepb" entry doesn't show up at all, even though it's the first "exclude file" entry in the dumptypes file.

Now, if I'm misunderstanding what the selfcheck.*.exclude file is showing (I assumed that was what gnutar would be using as the exclude list), perhaps everything is fine (I didn't want to run the actual backup and suck up many gigabytes of raw AFS volume data), but it sure looks like both "exclude file" and "exclude list" are doing the exact same thing, and not allowing me to specify individual files to exclude.

Sorry for the continuing stream of questions, and thanks for your patience in answering, but I thought what I was using should be allowing some central exclusions, just like your example indicates.

Cheers,

-Andrew

[ktill]

The behavior I am seeing is that either "exclude list" OR "exclude file" can be used. Not both.
Amcheck should at least issue a warning.

[Andrew Rakowski]

The behavior I am seeing is that either "exclude list" OR "exclude file" can be used. Not both.
Amcheck should at least issue a warning.

That's the problem - if I comment out the "exclude list" directive, leaving just the "exclude file" entries, the specified files show as being excluded (and I suspect, probably need the leading "./" on them). If I uncomment the "exclude list" directive, then all the file names apparently become names of client files containing excluded files. That also explains the "lost+found" excluded file that was being listed - it would be the first directory entry in the /vicepb directory when opened the directory is opened as a file.

I get NO errors from amcheck when I use any of those directives mixed together. If this is not supposed to be allowed, I guess that's a small bug in amcheck perhaps?

So, it seems that if I want the ability to use the equivalent of Networker ".nsr" skip directives to exclude certain things on only certain clients, I can't do any centralized exclusion, and need to come up with a way of distributing and updating at least a couple of files on the client systems.

Thanks for finding the problem for me Kevin! Would you call this a bug, or just a feature that needs to be explained a bit more in depth in the Amanda.conf document?

-Andrew

[ktill]

>Thanks for finding the problem for me Kevin! Would you call this a bug, or just a feature >that needs to be explained a bit more in depth in the Amanda.conf document?

Andrew,
I think it's a bug, hopefully, we can address it in the next release.

Kevin Till

[martineau]

Could you try the attached patch, it enable the use of both exclude list and exclude file.
Only the server need to be patched

[Andrew Rakowski]

Could you try the attached patch, it enable the use of both exclude list and exclude file.
Only the server need to be patched

Hi, I haven't forgotten this, but I've been tied up in rolling out the backup clients on all our Linux, Solaris-(8,9,10), IRIX, and AIX clients (not done yet though...) I haven't yet figured out how to build the Linux RPMs from the source (I've been building the non-Linux clients from the source tarball, but moving out Linux with the pre-built packages so far...) I'll let you know how it works as soon as I can give it a test.

Am I correct in assuming that the excludes that I see listed in the Amanda debug sendsize.*.exclude files is correctly reflecting what an actual backup will be excluding, so I can just do "amcheck" on a selected client and verify both central and local excludes work?

-Andrew

[Andrew Rakowski]

Could you try the attached patch, it enable the use of both exclude list and exclude file.
Only the server need to be patched

Hi - sorry for the long delay, but I did get to a point where I could apply this patch (and the other for the security-util.c), and it DOES indeed fix the problem of not being able to have both local and central exclude lists. I've now got a new dumptype called "GLOBAL-excluded" that I can include in all my dumptypes (or put in the "global" dumptype - I wanted to preserve options of having different lists of centrally excluded files). For example:

....
define dumptype GLOBAL-excluded {
# Explicit entries for things to be excluded:
exclude append "./proc"
exclude append "./tmp"
exclude append ".usr/vice/cache/*"
exclude append "./nsr/core/*"
#
# Define places ON THE CLIENT where excluded items can also be listed:
exclude list "/etc/amanda/exclude-list"
exclude list optional append ".amanda.excludes"
}

define dumptype MYLAB-full-weekly-inc-daily {
global
GLOBAL-excluded
program "GNUTAR"
comment "Weekly full backup with daily incrementals via tar"
dumpcycle 7
}
....

will exclude any files in the centrally excluded list, as well as files listed on the client in the /etc/amanda/exclude-list file, as well as files listed in the optional client-side file .amanda.exclude in the root of the DLE being backed up.

Is there any "nesting" limit to dumptypes including other dumptypes? For instance, could "global" have included the "GLOBAL-excluded" dumptype, or is there only one level of inclusion?

In any case, I can now happily exclude all the known (and newly discovered - like the /hw/ directory on IRIX - sort of like /proc on Linux or Solaris) items easily from the Amanda backup server - just what the doctor ordered.

Thank you very much Jean-Louis (and Kevin) for the help - and once again, sorry for the delay in verifying your patches work - you folks are great! At the moment, I've got 192 DLEs backing up to 7.2TB of hard drive (virtual tape) space on a daily basis - sweet. Now to properly document for the rest of the admins how to actually use this whole system, and prove it's not a write-only backup system.

-Andrew

[martineau]

Is there any "nesting" limit to dumptypes including other dumptypes? For instance, could "global" have included the "GLOBAL-excluded" dumptype, or is there only one level of inclusion?

There
is no limit on the number of nesting dumptypes.