Page 2 of 2 FirstFirst 12
Results 11 to 17 of 17

Thread: OK, split dumpo confusion

  1. #11
    Join Date
    Jan 2009
    Posts
    44

    Default

    Quote Originally Posted by pyeatman View Post
    You need to enter a line for any clients that you want to span tapes in your disklist file which is at /etc/amanda/<config>/disklist in the format as you mentioned in your first comment

    server /drive-to-backup user-tar-span

    I think you are still getting an error concerning your amanda.conf file when you run an amcheck. What is line 33 in your amanda.conf? Does /etc/amanda/template.d/dumptypes exist?

    Paul
    Thanks Paul. OK, the disklist for that set does have:

    backup /drive-to-split user-tar-span
    backup /drive-to split comp-user-tar

    I originally did it with the user-tar-span selected form the addclient tool. When this failed, I added the comp-user-tar to see exactly what would happen. Got the same error.

    Yes, /etc/amanda/template.d/dumptypes does exist. The file is thus:

    # dumptype global defined in $config/amanda.conf

    define dumptype always-full {
    global
    comment "Full dump of this filesystem always"
    compress none
    priority high
    dumpcycle 0
    }

    # Dumptypes for star
    define dumptype root-star {
    global
    program "STAR"
    comment "root partitions dumped with star"
    compress none
    index
    # exclude list "/var/lib/amanda/exclude.star"
    priority low
    }

    define dumptype user-star {
    root-star
    comment "user partitions dumped with star"
    priority medium
    }

    define dumptype user-star-span {
    root-star
    tape_splitsize 3 Gb
    comment "tape-spanning user partitions dumped with star"
    priority medium
    }

    define dumptype high-star {
    root-star
    comment "partitions dumped with star"
    priority high
    }

    define dumptype comp-root-star {
    root-star
    comment "Root partitions with compression"
    compress client fast
    }

    define dumptype comp-user-star {
    user-star
    compress client fast
    }

    define dumptype comp-user-star-span {
    user-star-span
    compress client fast
    }

    # Dumptypes for gnutar

    define dumptype root-tar {
    global
    program "GNUTAR"
    comment "root partitions dumped with tar"
    compress none
    index
    priority low
    }


    define dumptype user-tar {
    root-tar
    comment "user partitions dumped with tar"
    priority medium
    }


    define dumptype user-tar-span {
    root-tar
    tape_splitsize 3 Gb
    comment "tape-spanning user partitions dumped with tar"
    priority medium
    }



    define dumptype high-tar {
    root-tar
    comment "partitions dumped with tar"
    priority high
    }

    define dumptype comp-root-tar {
    root-tar
    comment "Root partitions with compression dumped with tar"
    compress client fast
    }

    define dumptype comp-user-tar {
    user-tar
    compress client fast
    }

    define dumptype comp-user-tar-span {
    user-tar-span
    compress client fast
    }


    define dumptype holding-disk {
    global
    comment "The master-host holding disk itself"
    holdingdisk never # do not use the holding disk
    priority medium
    }

    define dumptype comp-user {
    global
    comment "Non-root partitions on reasonably fast machines"
    compress client fast
    priority medium
    }

    define dumptype comp-user-span {
    global
    tape_splitsize 5 Gb
    comment "Tape-spanning non-root partitions on reasonably fast machines"
    compress client fast
    priority medium
    }


    define dumptype nocomp-user {
    comp-user
    comment "Non-root partitions on slow machines"
    compress none
    }

    define dumptype nocomp-user-span {
    comp-user-span
    comment "Tape-spanning non-root partitions on slow machines"
    compress none
    }


    define dumptype comp-root {
    global
    comment "Root partitions with compression"
    compress client fast
    priority low
    }

    define dumptype nocomp-root {
    comp-root
    comment "Root partitions without compression"
    compress none
    }

    define dumptype comp-high {
    global
    comment "very important partitions on fast machines"
    compress client best
    priority high
    }

    define dumptype nocomp-high {
    comp-high
    comment "very important partitions on slow machines"
    compress none
    }

    define dumptype nocomp-test {
    global
    comment "test dump without compression, no /etc/dumpdates recording"
    compress none
    record no
    priority medium
    }

    define dumptype comp-test {
    nocomp-test
    comment "test dump with compression, no /etc/dumpdates recording"
    compress client fast
    }

    define dumptype nocomp-ssh {
    root-tar
    comment "ssh authorization and dumped with tar"
    auth "ssh"
    ssh_keys "/var/lib/amanda/.ssh/id_rsa_amdump"
    compress none
    }


    define dumptype custom-compress {
    root-tar
    comment "custom client compression, dumped with tar"
    compress client custom
    client_custom_compress "/usr/bin/bzip2"
    }

    # amcrypt requires aespipe and uuencode
    define dumptype encrypt-fast {
    root-tar
    comment "fast client compression and server symmetric encryption, dumped with tar"
    compress client fast
    encrypt server
    server_encrypt "/usr/sbin/amcrypt"
    server_decrypt_option "-d"
    }


    # amcryptsimple use gpg symmetric encryption. gpg does compress with zlib by default.
    # Thus, specify compress none.
    define dumptype encrypt-simple-nocomp {
    root-tar
    comment "client simple symmetric encryption, dumped with tar"
    compress none
    encrypt client
    client_encrypt "/usr/sbin/amcryptsimple"
    client_decrypt_option "-d"
    }

    # To use gpg public-key encryption, gpg does compress with zlib by default.
    # Thus, specify compress none.

    define dumptype gpg-encrypt-nocomp {
    root-tar
    comment "server public-key encryption, dumped with tar"
    compress none
    encrypt server
    server_encrypt "/usr/sbin/amgpgcrypt"
    server_decrypt_option "-d"
    }

    # The following dumptypes are for ZMC
    # dumptype gui-base defined in $config/amanda.conf

    define dumptype gui-default {
    gui-base
    comment "gui default dumptype"
    compress none
    encrypt none
    }

    define dumptype gui-compress {
    gui-base
    comment "gui dumptype with compression"
    compress client fast
    encrypt none
    }

    define dumptype gui-encrypt {
    gui-base
    comment "gui dumptype with encryption"
    compress none
    encrypt server
    server_encrypt "/usr/sbin/amcryptsimple"
    }

    define dumptype gui-encrypt-compress {
    gui-base
    comment "gui dumptype with compression and encryption"
    compress client fast
    encrypt server
    server_encrypt "/usr/sbin/amcryptsimple"
    }
    # windows definitions for ZWC clients

    define dumptype zwc-normal {
    global
    program "DUMP"
    }

    define dumptype zwc-compress {
    global
    compress client fast
    program "DUMP"
    }


    Line 33 in amanda.conf has :

    define dumptype user-tar-span {
    root-tar
    tape_splitsize 100 Mb
    comment "tape-spanning user partitions dumped with tar"
    }

    That was based on what I understood from the instructions under the How To's. Is this not supposed to be there?

    TA I A

  2. #12
    Join Date
    Aug 2008
    Location
    Sunnyvale, CA
    Posts
    306

    Default

    Okay, Caesonia, the problem is simply that the dumptype definition that the documentation is informing you of putting in your amanda.conf is being placed too early in your conf file. This dumptype is referencing the "root-tar" dumptype which is not defined until the

    includefile "/etc/amanda/template.d/dumptypes"

    line of your amanda.conf. Putting this new definition after this line will allow "root-tar" to be then recognized as a dumptype. Also, user-tar-span is already defined in this same includefile so, you need to give the new dumptype you are defining a slightly different name, possibly my-user-tar-span, or Amanda will complain about the dumptype being already defined. Thus, I would remove the user-tar-span definition you have in your amanda.conf now and put something like the following instead at the end of your amanda.conf

    define dumptype my-user-tar-span {
    comment "my dumptype for tape-spanning user partitions dumped with tar"
    user-tar-span
    tape_splitsize 100 Mb
    }

    then use this dumptype name in your disklist file for your backup entries.

    I'll just mention that, if this preferred tape_splitsize is only for this one backup entry, you can do this same thing in the entry for this backup in the disklist file (without creating a new dumptype). If you want to use this preferred tape_splitsize value with multiple backup entires, then creating a new dumptype is a good way to go. You could even put it in /etc/amanda/template.d/dumptypes (beneath user-tar-span) if you want to use it with multiple configs.

    Paul

  3. #13
    Join Date
    Jan 2009
    Posts
    44

    Default

    Quote Originally Posted by pyeatman View Post
    Okay, Caesonia, the problem is simply that the dumptype definition that the documentation is informing you of putting in your amanda.conf is being placed too early in your conf file. This dumptype is referencing the "root-tar" dumptype which is not defined until the

    includefile "/etc/amanda/template.d/dumptypes"

    line of your amanda.conf. Putting this new definition after this line will allow "root-tar" to be then recognized as a dumptype. Also, user-tar-span is already defined in this same includefile so, you need to give the new dumptype you are defining a slightly different name, possibly my-user-tar-span, or Amanda will complain about the dumptype being already defined. Thus, I would remove the user-tar-span definition you have in your amanda.conf now and put something like the following instead at the end of your amanda.conf

    define dumptype my-user-tar-span {
    comment "my dumptype for tape-spanning user partitions dumped with tar"
    user-tar-span
    tape_splitsize 100 Mb
    }

    then use this dumptype name in your disklist file for your backup entries.

    I'll just mention that, if this preferred tape_splitsize is only for this one backup entry, you can do this same thing in the entry for this backup in the disklist file (without creating a new dumptype). If you want to use this preferred tape_splitsize value with multiple backup entires, then creating a new dumptype is a good way to go. You could even put it in /etc/amanda/template.d/dumptypes (beneath user-tar-span) if you want to use it with multiple configs.

    Paul
    OK, Paul, after one or two more snarls, I seem to have amcheck accepting without errors. It appears that because it is in the included file, it only creates a confusion to add it.

    Whether or not it does it properly is another matter. We'll see tonight. TA TA and TA again for your help.

    Still confused on the bit thats going on with the CentOS client.

  4. #14
    Join Date
    Aug 2008
    Location
    Sunnyvale, CA
    Posts
    306

    Default

    Cool, very glad you got things working! -Paul

  5. #15
    Join Date
    Jan 2009
    Posts
    44

    Default

    Quote Originally Posted by pyeatman View Post
    Cool, very glad you got things working! -Paul
    OK, now...some more questions so that when the documentation is upgraded, some important things are made clear. If I have these questions, other people probably will too!

    I can't seem to see how many tapes are being used to actually do the back up for the mongo big drive. I can guess, but I am not really sure, and its important if I want to know I have enough tapes available with enough space to go back a week.

    Initially, I was seeing thus in my email log:

    Hostname: backup-server
    Org : DailySet1
    Config : DailySet1
    Date : March 17, 2009

    These dumps were to tape DailySet1-5.
    The next tape Amanda expects to use is: DailySet1-6.


    STATISTICS:
    Total Full Incr.
    -------- -------- --------
    Estimate Time (hrs:min) 0:00
    Run Time (hrs:min) 0:28
    Dump Time (hrs:min) 0:28 0:27 0:01
    Output Size (meg) 13113.4 13113.3 0.0
    Original Size (meg) 16758.9 16758.4 0.5
    Avg Compressed Size (%) 78.2 78.2 6.4 (level:#disks ...)
    Filesystems Dumped 3 1 2 (1:2)
    Avg Dump Rate (k/s) 7967.6 8272.0 0.5

    Tape Time (hrs:min) 0:28 0:27 0:01
    Tape Size (meg) 13113.4 13113.3 0.0
    Tape Used (%) 72.9 72.9 0.0 (level:#disks ...)
    Filesystems Taped 3 1 2 (1:2)
    (level:#chunks ...)
    Chunks Taped 3 1 2 (1:2)
    Avg Tp Write Rate (k/s) 7971.5 8275.3 0.5

    USAGE BY TAPE:
    Label Time Size % Nb Nc
    DailySet1-5 0:28 13113M 72.9 3 3


    NOTES:
    planner: Adding new disk backup-server:/split-drive.
    big estimate: Vista-client C:/Backup 1
    est: 0M out 0M
    big estimate: Linux-client /home/user 1
    est: 1M out 0M
    small estimate: backup-server /split-drive 0
    est: 8379M out 13113M


    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-MB OUT-MB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- -------------
    backup-server /split-drive 0 16758 13113 78.2 27:03 8272.0 27:03 8275.3
    Vista Client "C:/Backup" 1 0 0 -- 1:02 0.0 1:02 0.0
    Linux-Client -e/user 1 1 0 6.2 0:00 343.2 0:00 538.8

    (brought to you by Amanda version 2.6.0p2)

    Initially, this was with assigning massive space to the virtual tap drives. Like 18 GB.

    Now, though, since big drive was split across tapes, I see.....



    These dumps were to tape DailySet1-10.
    The next 12 tapes Amanda expects to use are: DailySet1-11, DailySet1-12, DailySet1-1, DailySet1-2, DailySet1-3, DailySet1-4, DailySet1-5, DailySet1-6, DailySet1-7, DailySet1-8, DailySet1-9, DailySet1-10.


    Yes, I did change the config to use all 12 tapes, because I wasn't sure what was happening, but now I never know if they are ALL being overwritten, or just some but it doesn't say, or if I can't hold 1 week's worth of data...or what?
    It doesn't even seem as if it is doing incremental dumps during the week, with only one total dump once a week.

    I think this sort of stuff should be clear so people understand exactly what is going on when they end up splitting a big drive across several tapes.

    TA.

  6. #16
    Join Date
    Aug 2008
    Location
    Sunnyvale, CA
    Posts
    306

    Default

    It sounds like you have 12 tapes and you set runtapes to 12? This is in essence like telling Amanda that you have one tape. Nonetheless, it will only use as many of these as it needs to hold the backup images from the run. If the first tape will hold everything, this is all it will use. But now with one tape used, it won't have 12 available tapes for the next run and so I'm guessing refuses to run from not finding a "tape" to use.

    In the backup report, you can see your tape usage in the section "USAGE BY TAPE:". For the one you listed previously, 72.9% of the tape was used. If your runtapes is > 1, it will list the usage for each tape used for that run. For the backup entries listed above, it seems like it doesn't need more than the first tape.

    How big is "big drive" and how much space do you have for virtual tapes? And what is your current virtual tape size?

    Paul

  7. #17
    Join Date
    Jan 2009
    Posts
    44

    Default

    Quote Originally Posted by pyeatman View Post
    It sounds like you have 12 tapes and you set runtapes to 12? This is in essence like telling Amanda that you have one tape. Nonetheless, it will only use as many of these as it needs to hold the backup images from the run. If the first tape will hold everything, this is all it will use. But now with one tape used, it won't have 12 available tapes for the next run and so I'm guessing refuses to run from not finding a "tape" to use.

    In the backup report, you can see your tape usage in the section "USAGE BY TAPE:". For the one you listed previously, 72.9% of the tape was used. If your runtapes is > 1, it will list the usage for each tape used for that run. For the backup entries listed above, it seems like it doesn't need more than the first tape.

    How big is "big drive" and how much space do you have for virtual tapes? And what is your current virtual tape size?

    Paul
    Paul, sorry getting back so slowly. Had an accident and was in hospital from a big conk on the head. Still very knackered from it all.

    Anyways, that explanation certainly helped understand it much more. I don't really see it explained very well when I try and look at what exactly happens when you make that selection.

    The big drive is about 17gig all together, broken down into one 15 gig directory, and 2 1 gig directories. Initially, I had set it to one, and just gave each tape enough size to cover those directories and of other machines, but I would rather do it say...with 4 tapes, and then have it go incrementally.

    And maybe that's another problem: I thought I was getting a weekly back up with one total dump and incrementals for the rest of the week, but perhaps that's not the way it was working. Instead it it just doing a complete backup each time?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •