Page 1 of 3 123 LastLast
Results 1 to 10 of 24

Thread: trouble with spanning backups on 3.2.2

  1. #1
    Join Date
    Feb 2011
    Posts
    20

    Default trouble with spanning backups on 3.2.2

    I have used Amanda for many years to do my backups, and I've always been able to make adjustments for the changes in version.
    Since the update to 3.2.2, I am getting errors when a backup is multiple vtapes. It will span across two or three and then error out. the amcheckdump shows an error, and a restore shows missing files.

    I have 12 backups running, each varies in size but none are larger than 4Tb.
    This error only occurs when they start running over multiples.

    Broken pipe, no space left on device.....
    theres plenty of space on the device, there's no log file on server or client side to show any broken pipe or other communication error. It occurs on more than one client.

    Results:

    Hostname: rmlamanda.#####.###.###
    Org : rtsset12
    Config : rtsset12
    Date : March 29, 2011

    These dumps were to tapes rtsset12-40, rtsset12-11.
    The next 40 tapes Amanda expects to use are: 27 new tapes, rtsset12-1, rtsset12-2, rtsset12-3, rtsset12-4, rtsset12-5, rtsset12-6, rtsset12-7, rtsset12-8, rtsset12-35, rtsset12-9, rtsset12-10, rtsset12-40, rtsset12-11.
    The next 27 new tapes already labelled are: rtsset12-12, rtsset12-13, rtsset12-14, rtsset12-15, rtsset12-16, rtsset12-17, rtsset12-18, rtsset12-19, rtsset12-20, rtsset12-21, rtsset12-22, rtsset12-23, rtsset12-24, rtsset12-25, rtsset12-26, rtsset12-27, rtsset12-28, rtsset12-29, rtsset12-30, rtsset12-31, rtsset12-32, rtsset12-33, rtsset12-34, rtsset12-36, rtsset12-37, rtsset12-38, rtsset12-39
    FAILURE DUMP SUMMARY:
    redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.#####.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device
    redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.#####.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device



    STATISTICS:
    Total Full Incr. Level:#
    -------- -------- -------- --------
    Estimate Time (hrs:min) 0:01
    Run Time (hrs:min) 4:53
    Dump Time (hrs:min) 0:00 0:00 0:00
    Output Size (meg) 209345.5 209345.5 0.0
    Original Size (meg) 0.0 0.0 0.0
    Avg Compressed Size (%) -- -- --
    DLEs Dumped 0 0 0
    Avg Dump Rate (k/s) -- -- --

    Tape Time (hrs:min) 4:52 4:52 0:00
    Tape Size (meg) 209345.5 209345.5 0.0
    Tape Used (%) 199.4 199.4 0.0
    DLEs Taped 2 2 0
    Parts Taped 20936 20936 0
    Avg Tp Write Rate (k/s) 12242.0 12242.0 --

    USAGE BY TAPE:
    Label Time Size % DLEs Parts
    rtsset12-40 2:24 102G 99.7 1 10468
    rtsset12-11 2:26 102G 99.7 1 10468

    FAILED DUMP DETAILS:
    /-- redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.#####.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------
    /-- redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.#####.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------

    NOTES:
    planner: Forcing full dump of redux.#####.###.###:/import/data/solid_data6 as directed.
    taper: tape rtsset12-40 kb 107184896 fm 10468 [OK]
    taper: tape rtsset12-11 kb 107184896 fm 10468 [OK]


    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- --------------
    redux.#####. -olid_data6 0 102 -- PARTIAL 144:29 12364.2 PARTIAL

    (brought to you by Amanda version 3.2.2)


    amanda.conf


    org "rtsset12" # your organization name for reports
    mailto "********" # space separated list of operators at your site
    dumpcycle 2week # the number of days in the normal dump cycle
    runspercycle 14 # the number of amdump runs in dumpcycle days
    tapecycle 40 # the number of tapes in rotation
    runtapes 40 # number of tapes to be used in a single run of amdump

    tpchanger "chg-disk:/amanda3/vtapes/rtsset12" # the tape-changer glue script

    #tapedev "file://amanda3/vtapes/rtsset12" # the no-rewind tape device
    changerfile "/etc/amanda/rtsset12/changer.conf" # tape changer configuration parameter file

    changerdev "/dev/null" # tape changer configuration parameter device

    tapetype HARDDISK # what kind of tape it is
    labelstr "rtsset12" # label constraint regex: all tapes must match
    dtimeout 18000 # number of idle seconds before a dump is aborted
    ctimeout 30 # max number of secconds amcheck waits for each client
    etimeout 300 # number of seconds per filesystem for estimates
    define dumptype global {
    comment "Global definitions"
    auth "bsdtcp"
    }
    define dumptype gui-base {
    global
    program "GNUTAR"
    comment "gui base dumptype dumped with tar"
    compress none
    index yes
    }
    define tapetype HARDDISK {
    comment "Virtual Tapes"
    length 105000 mbytes
    part_size 10 mbytes
    }

    #holdingdisk hd1 {
    # directory "/amanda/holding"
    # use 1 gbytes
    # chunksize 10 mbyte
    #}

    includefile "advanced.conf"
    includefile "/etc/amanda/template.d/dumptypes"
    includefile "/etc/amanda/template.d/tapetypes"

    available space

    Filesystem Size Used Avail Use% Mounted on

    /dev/mapper/log_vol3-lvol0
    3.9T 1.3T 2.4T 35% /amanda3

    amdumpcheck gives results
    "Reading volume rtsset12-11 file 1....10468"
    "Validation errors:"
    "/usr/bin/gzip exited with status 1"
    "/bin/tar exited with status 2"


    I can't figure out whats causing the error.

    Thanks
    ~Kelly

  2. #2
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    looks like a bug.

    Do you still have the tape-splitsize in the dumptype? re-add it.
    You should set it (and part-size) to something bigger (1g)

  3. #3
    Join Date
    Feb 2011
    Posts
    20

    Default

    ok, I added tape-splitsize to dumptype. set it to 100Mb for testing. will increase size to 1g after testing is complete. I manually restarted the backup, and will let you know what the results are.

    Thank you

  4. #4
    Join Date
    Feb 2011
    Posts
    20

    Default

    on second thought, I will set tape-splitsize to 1 gb and part-size to 1gb.
    trying again.
    =)

  5. #5
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    It's not a bug, it is a misconfiguration.

    When doing dup direct to tape on a device that require caching, you needs to specify the caching mechanism with the tapetype part-cache-type set to memory or disk.

    Some device do not require caching, it this case, you can set the LEOM device property to yes, it is better than setting part-cache-type.

    vtape do not require caching, you should add:
    device-property "LEOM" "yes"

  6. #6
    Join Date
    Feb 2011
    Posts
    20

    Default

    That gets added where? dumptype, or amanda.conf?

  7. #7
    Join Date
    Feb 2011
    Posts
    20

    Default

    didnt work.

    changed Amanda.conf

    org "rtsset12" # your organization name for reports
    mailto "***" # space separated list of operators at your site
    dumpcycle 2week # the number of days in the normal dump cycle
    runspercycle 14 # the number of amdump runs in dumpcycle days
    tapecycle 40 # the number of tapes in rotation
    runtapes 40 # number of tapes to be used in a single run of amdump

    tpchanger "chg-disk:/amanda3/vtapes/rtsset12" # the tape-changer glue script

    #tapedev "file://amanda3/vtapes/rtsset12" # the no-rewind tape device
    changerfile "/etc/amanda/rtsset12/changer.conf" # tape changer configuration parameter file

    changerdev "/dev/null" # tape changer configuration parameter device

    device-property "LEOM" "yes"

    tapetype HARDDISK # what kind of tape it is
    labelstr "rtsset12" # label constraint regex: all tapes must match
    dtimeout 18000 # number of idle seconds before a dump is aborted
    ctimeout 30 # max number of secconds amcheck waits for each client
    etimeout 300 # number of seconds per filesystem for estimates
    define dumptype global {
    comment "Global definitions"
    auth "bsdtcp"
    }
    define dumptype gui-base {
    global
    program "GNUTAR"
    comment "gui base dumptype dumped with tar"
    compress none
    index yes
    }
    define tapetype HARDDISK {
    comment "Virtual Tapes"
    length 105000 mbytes
    part_size 1 gbyte
    }

    #holdingdisk hd1 {
    # directory "/amanda/holding"
    # use 1 gbytes
    # chunksize 10 mbyte
    #}

    includefile "advanced.conf"
    includefile "/etc/amanda/template.d/dumptypes"
    includefile "/etc/amanda/template.d/tapetypes"


    results are the same. still fails.

    Hostname: rmlamanda.#####.###.###
    Org : rtsset12
    Config : rtsset12
    Date : March 31, 2011

    These dumps were to tapes rtsset12-17, rtsset12-18.
    The next 40 tapes Amanda expects to use are: 20 new tapes, rtsset12-1, rtsset12-2, rtsset12-3, rtsset12-4, rtsset12-5, rtsset12-6, rtsset12-7, rtsset12-8, rtsset12-35, rtsset12-9, rtsset12-10, rtsset12-40, rtsset12-11, rtsset12-12, rtsset12-13, rtsset12-14, rtsset12-15, rtsset12-16, rtsset12-17, rtsset12-18.
    The next 20 new tapes already labelled are: rtsset12-19, rtsset12-20, rtsset12-21, rtsset12-22, rtsset12-23, rtsset12-24, rtsset12-25, rtsset12-26, rtsset12-27, rtsset12-28, rtsset12-29, rtsset12-30, rtsset12-31, rtsset12-32, rtsset12-33, rtsset12-34, rtsset12-36, rtsset12-37, rtsset12-38, rtsset12-39
    FAILURE DUMP SUMMARY:
    redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.#####.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device
    redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.#####.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device



    STATISTICS:
    Total Full Incr. Level:#
    -------- -------- -------- --------
    Estimate Time (hrs:min) 0:02
    Run Time (hrs:min) 4:46
    Dump Time (hrs:min) 0:00 0:00 0:00
    Output Size (meg) 209993.3 209993.3 0.0
    Original Size (meg) 0.0 0.0 0.0
    Avg Compressed Size (%) -- -- --
    DLEs Dumped 0 0 0
    Avg Dump Rate (k/s) -- -- --

    Tape Time (hrs:min) 4:44 4:44 0:00
    Tape Size (meg) 209993.3 209993.3 0.0
    Tape Used (%) 200.0 200.0 0.0
    DLEs Taped 2 2 0
    Parts Taped 206 206 0
    Avg Tp Write Rate (k/s) 12606.7 12606.7 --

    USAGE BY TAPE:
    Label Time Size % DLEs Parts
    rtsset12-17 2:22 103G 100.0 1 103
    rtsset12-18 2:22 103G 100.0 1 103

    FAILED DUMP DETAILS:
    /-- redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.#####.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------
    /-- redux.#####.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.#####.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------

    NOTES:
    planner: Forcing full dump of redux.#####.###.###:/import/data/solid_data6 as directed.
    taper: tape rtsset12-17 kb 107516576 fm 103 [OK]
    taper: tape rtsset12-18 kb 107516576 fm 103 [OK]


    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- --------------
    redux.###. -olid_data6 0 103 -- PARTIAL 142:05 12611.9 PARTIAL

    (brought to you by Amanda version 3.2.2)


    I'm running out of ideas, but I havent had a full backup complete since the upgrade to 3.2.2 .

  8. #8
    Join Date
    Nov 2005
    Location
    Canada
    Posts
    1,049

    Default

    What's the output of: grep 'preparing to write' taper.DATESTAMP.debug

  9. #9
    Join Date
    Feb 2011
    Posts
    20

    Default

    -sh-3.2$ grep 'preparing to write' taper.20110331114224.debug
    Thu Mar 31 11:43:53 2011: taper: Amanda::Taper::Scribe preparing to write, part size 1073741824, using LEOM detection (no caching) (splitter) (LEOM supported)

  10. #10
    Join Date
    Feb 2011
    Posts
    20

    Default

    my backups are still failing once it reaches the third tape.
    Hostname: rmlamanda.#####.###.###
    Org : rtsset12
    Config : rtsset12
    Date : April 4, 2011

    These dumps were to tapes rtsset12-26, rtsset12-27.
    The next 40 tapes Amanda expects to use are: 11 new tapes, rtsset12-1, rtsset12-2, rtsset12-3, rtsset12-4, rtsset12-5, rtsset12-6, rtsset12-7, rtsset12-8, rtsset12-35, rtsset12-9, rtsset12-10, rtsset12-40, rtsset12-11, rtsset12-12, rtsset12-13, rtsset12-14, rtsset12-15, rtsset12-16, rtsset12-17, rtsset12-18, rtsset12-19, rtsset12-20, rtsset12-21, rtsset12-22, rtsset12-23, rtsset12-24, rtsset12-25, rtsset12-26, rtsset12-27.
    The next 11 new tapes already labelled are: rtsset12-28, rtsset12-29, rtsset12-30, rtsset12-31, rtsset12-32, rtsset12-33, rtsset12-34, rtsset12-36, rtsset12-37, rtsset12-38, rtsset12-39
    FAILURE DUMP SUMMARY:
    redux.###.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.###.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device
    redux.###.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    redux.###.###.### /import/data/solid_data6 lev 0 partial taper: No space left on device



    STATISTICS:
    Total Full Incr. Level:#
    -------- -------- -------- --------
    Estimate Time (hrs:min) 0:01
    Run Time (hrs:min) 4:47
    Dump Time (hrs:min) 0:00 0:00 0:00
    Output Size (meg) 209993.3 209993.3 0.0
    Original Size (meg) 0.0 0.0 0.0
    Avg Compressed Size (%) -- -- --
    DLEs Dumped 0 0 0
    Avg Dump Rate (k/s) -- -- --

    Tape Time (hrs:min) 4:46 4:46 0:00
    Tape Size (meg) 209993.3 209993.3 0.0
    Tape Used (%) 200.0 200.0 0.0
    DLEs Taped 2 2 0
    Parts Taped 206 206 0
    Avg Tp Write Rate (k/s) 12547.2 12547.2 --

    USAGE BY TAPE:
    Label Time Size % DLEs Parts
    rtsset12-26 2:23 103G 100.0 1 103
    rtsset12-27 2:23 103G 100.0 1 103

    FAILED DUMP DETAILS:
    /-- redux.###.###.### /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.###.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------
    /-- redux.niaid.nih.gov /import/data/solid_data6 lev 0 FAILED [data write: Broken pipe]
    sendbackup: start [redux.###.###.###:/import/data/solid_data6 level 0]
    sendbackup: info BACKUP=/bin/tar
    sendbackup: info RECOVER_CMD=/bin/tar -xpGf - ...
    sendbackup: info end
    ? data compress:
    ? data compress: gzip: stdout: Broken pipe
    \--------

    NOTES:
    planner: Forcing full dump of redux.###.###.###:/import/data/solid_data6 as directed.
    taper: tape rtsset12-26 kb 107516576 fm 103 [OK]
    taper: tape rtsset12-27 kb 107516576 fm 103 [OK]


    DUMP SUMMARY:
    DUMPER STATS TAPER STATS
    HOSTNAME DISK L ORIG-GB OUT-GB COMP% MMM:SS KB/s MMM:SS KB/s
    -------------------------- ------------------------------------- --------------
    redux.###. -olid_data6 0 103 -- PARTIAL 142:46 12551.5 PARTIAL

    (brought to you by Amanda version 3.2.2)

    Do you have any other suggestions for resolving this? It was running fine with 3.2.1. I'll change whatever I need to to get it running.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •