Results 1 to 6 of 6

Thread: Amanda overwriting level-0s, leaving me with none, and I can't figure out why

  1. #1
    Join Date
    Jul 2012
    Posts
    3

    Default Amanda overwriting level-0s, leaving me with none, and I can't figure out why

    Hi all,

    I've got a long-running amanda install here, currently on 3.3.0, which occasionally leaves us without level-0 backups and I can't sort out why. It happened last night, so all of the output and logs below are of last night's backup.

    We have six tapes in our onsite rotation, and we run four backups a week, at 0:45 AM on Tues, Wed, Thurs and Saturday. (At 0:45 AM on Friday, we run a different backup job which is all level-0 for offsites.)

    amoverview points out the problem:

    Code:
    :) root@backup# amoverview fbhq-onsite 
             date                 07  07  07  07  07  07  07  07  07 
    host     disk                 18  19  20  21  22  23  24  25  26 
    backup.2 /                    1   1   -   2   -   -   2   3   3  
    backup.2 /boot                1   1   -   1   -   -   1   1   1  
    bigmac.2 /etc/swupd           1   1   -   1   -   -   1   1   1  
    fw02.2nd /                    1   1   -   1   -   -   1   2   2  
    fw02.2nd /boot                1   1   -   1   -   -   1   1   1  
    fw02.2nd /var/log             1   1   -   2   -   -   2   3   3  
    git.2nds /                    1   1   -   2   -   -   2   3   3  
    git.2nds /boot                1   1   -   1   -   -   1   1   1  
    git.2nds /home                1   1   -   1   -   -   1   1   1  
    grapher. /                    1   1   -   2   -   -   2   3   3  
    grapher. /boot                1   1   -   1   -   -   1   1   1  
    iosbuild /Library             1   1   -   1   -   -   1   1   1  
    iosbuild /etc                 1   1   -   1   -   -   1   1   1  
    mailgate /                    1   1   -   2   -   -   2   0   1  
    mailgate /boot                1   1   -   1   -   -   1   1   1  
    megatron /                    1   1   -   1   -   -   1   1   1  
    megatron /boot                1   1   -   1   -   -   1   1   1  
    megatron /home                2   2   -   3   -   -   0   1   1  
    megatron /var                 1   1   -   2   -   -   2   3   3  
    nas01.2n /data/dev            1   1   -   2   -   -   2   3   3  
    nas01.2n /data/misc           1   1   -   1   -   -   1   2   2  
    nas01.2n /home                1   1   -   1   -   -   1   1   1  
    nas01.2n /mysql/mysqldump     2   3   -   0   -   -   1   1   2  
    optimus. /                    1   1   -   1   -   -   1   1   1  
    optimus. /boot                1   1   -   1   -   -   1   1   1  
    optimus. /boot2               1   1   -   1   -   -   1   1   1  
    optimus. /home                1   2   -   2   -   -   0   1   1  
    optimus. /usr                 1   1   -   1   -   -   1   1   2  
    optimus. /var                 1   1   -   2   -   -   2   3   3  
    optimus. /var/lib/mysqldump   1   1   -   2   -   -   2   3   0  
    qback.2n /                    1   1   -   2   -   -   2   3   3  
    quickboo C:/Users/quickbooks  1   1   -   2   -   -   2   3   3  
    ratchet. /                    1   1   -   1   -   -   1   1   1  
    ratchet. /boot                1   1   -   1   -   -   1   1   1  
    ratchet. /boot2               1   1   -   1   -   -   1   1   1  
    ratchet. /home                1   1   -   1   -   -   1   1   1  
    ratchet. /usr                 1   1   -   1   -   -   1   1   1  
    ratchet. /var                 1   1   -   2   -   -   2   3   3  
    schedgy. /                    1   1   -   2   -   -   2   3   3  
    schedgy. /boot                1   1   -   1   -   -   1   1   1  
    syslist. /                    1   1   -   2   -   -   2   3   3  
    syslist. /boot                1   1   -   1   -   -   1   1   1  
    vm.2ndsi C:/Documents and Set 1   1   -   2   -   -   2   3   3  
    vm.2ndsi C:/Program Files/Ava 1   1   -   1   -   -   1   1   1  
    vm.2ndsi D:/                  1   1   -   2   -   -   2   3   3
    Our tape cycle looks like so:
    Code:
    dumpcycle 7
    runspercycle 4
    tapecycle 6 tapes
    runtapes 1
    Tapelist matches:
    Code:
    20120726004501 fbhq-onsite-5 reuse
    20120725004502 fbhq-onsite-4 reuse
    20120724004501 fbhq-onsite-3 reuse
    20120721004501 fbhq-onsite-2 reuse
    20120719004501 fbhq-onsite-1 reuse
    20120718004501 fbhq-onsite-0 reuse
    The holding disk is practically empty:

    Code:
    /dev/mapper/data-amanda
                          1.8T  196M  1.7T   1% /var/spool/amanda
    I've made last night's amdump.1 and log.20120726004501.0 available on my Dropbox, I couldn't get the attachment bits to work here:

    [url]http://dl.dropbox.com/u/3585831/amanda/amdump.1.txt[/url]
    [url]http://dl.dropbox.com/u/3585831/amanda/log.20120726004501.0.txt[/url]

    I see the warnings about backups being overwritten on this run, so Amanda knows what's up, I just don't understand why it would need to when it's got 6 tapes to cover 4 runs per dumpcycle.

    One thing that did catch my eye was this in amdump.1:

    Code:
    DUMP megatron.2ndsiteinc.com ffffffff9ffeffffffff7f0000 /home 20120726004501 1 1 2012:7:24:4:45:1 7849430 6232940 2692 2315 "Skipping: a full is not planned, so can't dump in degraded mode"
    DUMP nas01.2ndsiteinc.com ffffffff9efefbffffffffff0f /mysql/mysqldump 20120726004501 1 2 2012:7:25:4:45:2 5200927 5173246 800 6462 "Skipping: a full is not planned, so can't dump in degraded mode"
    DUMP optimus.2ndsiteinc.com ffffffff9ffeffffffff7f0000 /home 20120726004501 1 1 2012:7:24:4:45:1 75621210 3999731 16374 244 "Skipping: a full is not planned, so can't dump in degraded mode"
    (which I don't understand at all -- Why isn't a full planned? Why is it in degraded mode (the tape was in)? What is it skipping?)

    But not to get distracted by those small questions -- the main problem here is, why is amanda overwriting level-0s leaving me with none?

    Thanks in advance for your help. If I've omitted anything from the details here please let me know!
    Last edited by mendel; July 26th, 2012 at 07:09 AM.

  2. #2
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    Do you overwrite the dumpcycle in the dumptype?
    What's the output of: amadmin fbhq-onsite disklist | grep -i dumpcycle

  3. #3
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    And to be sure the global is set as you says: amgetconf fbhq-onsite dumpcycle

  4. #4
    Join Date
    Jul 2012
    Posts
    3

    Default

    Quote Originally Posted by martineau View Post
    Do you overwrite the dumpcycle in the dumptype?
    What's the output of: amadmin fbhq-onsite disklist | grep -i dumpcycle
    Code:
    :) root@backup# su amandabackup -c 'amadmin fbhq-onsite disklist' | grep -i dumpcycle
            dumpcycle 10
            dumpcycle 10
            dumpcycle 10
    [etc]
    Well that sure sounds relevant! But I don't understand where it's getting that from. Here's the dumptypes that (I think) are relevant:

    Here's a couple of affected DLEs:
    Code:
    ratchet.2ndsiteinc.com / comp-root-amgtarold
    ratchet.2ndsiteinc.com /var comp-root-amgtarold-excludes
    From common.conf:
    Code:
    define dumptype global {
        comment "Global definitions"
        index yes
        record yes
        auth "bsdtcp"
        script "wakeonlan"
        estimate server
    }
    
    define dumptype comp-root-amgtarold {
        root-amgtarold
        comment "Root partitions with compression"
        compress client fast
    }
    
    define dumptype comp-root-amgtarold-excludes {
        comp-root-amgtarold
        exclude list ".amanda.excludes"
    }
    
    define dumptype root-amgtarold {
        global
        dt_amgtarold
        comment "root partitions dumped with amgtarold"
        compress none
    }
    
    define dumptype dt_amgtarold {
        program "APPLICATION"
        application "app_amgtarold"
    }
    And here's the full output from 'amadmin disklist' for one of those DLEs:

    Code:
    line 46:
        host ratchet.2ndsiteinc.com:
            interface default
        disk /:
            program "APPLICATION"
            application "app_amgtarold"
            data-path AMANDA
            priority 1
            dumpcycle 10
            maxdumps 1
            maxpromoteday 10000
            bumpsize 10240
            bumpdays 2
            bumpmult 1.500000
            strategy STANDARD
            ignore NO
            estimate SERVER
            compress CLIENT FAST
            comprate 0.50 0.50
            encrypt NONE
            auth "bsdtcp"
            kencrypt NO
            amandad-path ""
            client-username ""
            client-port ""
            ssh-keys ""
            holdingdisk AUTO
            record YES
            index YES
            starttime 0000
            fallback-splitsize 10Mb
            skip-incr NO
            skip-full NO
            allow-split YES
            spindle -1
    The only place "dumpcycle" is mentioned in /etc/amanda.common.conf is in our "always-full" dumptype which isn't used here. Any idea where that 10 is coming from?

    And to be sure the global is set as you says: amgetconf fbhq-onsite dumpcycle
    This is, thankfully, okay:

    Code:
    :) root@backup# amgetconf fbhq-onsite dumpcycle
    7
    Thanks for your help -- this is the most progress I've had on this issue since it started happening!

  5. #5
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    10 is the default value.
    Do you define the dumptype before you set the global dumcycle?

  6. #6
    Join Date
    Jul 2012
    Posts
    3

    Default

    Quote Originally Posted by martineau View Post
    10 is the default value.
    Do you define the dumptype before you set the global dumcycle?
    AHA!

    Yes, yes we do. Moving the "includefile" statement which brings in the dumptypes to the bottom of the amanda.conf makes things work as expected.

    Thanks so much for your help. The odds of us figuring this one out on our own was about zero!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •