PDA

View Full Version : Amanda 3.1.2 64-bit dumper crashing



tomaszc
August 19th, 2010, 05:49 AM
Hello,

I have recently migrated my company backup system from Amanda 2.6.1p2 to 3.1.2, and I am seeing sporadic dumper crashes. I have now tried both compiling from a tarball (my preferred method) and installing the rpm. While the rpm seems more stable, I am still seeing crashes.

I get printouts in /var/log/messages like this:
dumper[15589] general protection ip:7f243214ef19 sp:7fff1dbcad60 error:0 in libc-2.11.1.so[7f24320d5000+154000]

printouts in amdump:
driver: dumper0 pid 13762 exited with signal 11
driver: dumper2 pid 11482 exited with signal 6

The system is running SLES11_SP1 64-bit

I have a similar system running SLES11 32-bit with the same Amanda version compiled from tarball, and no problems at all.

Is anybody else seeing the same?

tomaszc
August 20th, 2010, 05:09 AM
When using the version compiled from the tarball, I get a more exhaustive error message:

*** glibc detected *** dumper1: corrupted double-linked list: 0x000000000062fe60 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x75018)[0x7f260ef75018]
/lib64/libc.so.6(+0x786b8)[0x7f260ef786b8]
/lib64/libc.so.6(__libc_malloc+0x77)[0x7f260ef7a057]
/usr/local/lib/amanda/libamanda-3.1.2.so(debug_alloc+0x1e)[0x7f2610c1067e]
/usr/local/lib/amanda/libamanda-3.1.2.so(tcpm_recv_token+0xf6)[0x7f2610c2fe76]
/usr/local/lib/amanda/libamanda-3.1.2.so(+0x376be)[0x7f2610c346be]
/usr/local/lib/amanda/libamanda-3.1.2.so(+0x2832f)[0x7f2610c2532f]
/usr/lib64/libglib-2.0.so.0(g_main_context_dispatch+0x24a)[0x7f260fc6060a]
/usr/lib64/libglib-2.0.so.0(+0x3ee88)[0x7f260fc63e88]
/usr/lib64/libglib-2.0.so.0(g_main_context_iteration+0x6c)[0x7f260fc6403c]
/usr/local/lib/amanda/libamanda-3.1.2.so(+0x2846f)[0x7f2610c2546f]
dumper1(main+0x15b0)[0x405580]
/lib64/libc.so.6(__libc_start_main+0xe6)[0x7f260ef1ebc6]
dumper1[0x402fe9]
======= Memory map: ========
00400000-0040a000 r-xp 00000000 08:09 557222 /usr/local/libexec/amanda/dumper
00609000-0060a000 r--p 00009000 08:09 557222 /usr/local/libexec/amanda/dumper
0060a000-0060b000 rw-p 0000a000 08:09 557222 /usr/local/libexec/amanda/dumper
0060b000-00674000 rw-p 00000000 00:00 0 [heap]
7f2608000000-7f2608021000 rw-p 00000000 00:00 0
7f2608021000-7f260c000000 ---p 00000000 00:00 0
7f260e69f000-7f260e6b5000 r-xp 00000000 08:07 548984 /lib64/libgcc_s.so.1
7f260e6b5000-7f260e8b4000 ---p 00016000 08:07 548984 /lib64/libgcc_s.so.1
7f260e8b4000-7f260e8b5000 r--p 00015000 08:07 548984 /lib64/libgcc_s.so.1
7f260e8b5000-7f260e8b6000 rw-p 00016000 08:07 548984 /lib64/libgcc_s.so.1
7f260e8b6000-7f260e8cb000 r-xp 00000000 08:07 548910 /lib64/libz.so.1.2.3
7f260e8cb000-7f260eaca000 ---p 00015000 08:07 548910 /lib64/libz.so.1.2.3
7f260eaca000-7f260eacb000 r--p 00014000 08:07 548910 /lib64/libz.so.1.2.3
7f260eacb000-7f260eacc000 rw-p 00015000 08:07 548910 /lib64/libz.so.1.2.3
7f260eacc000-7f260eafb000 r-xp 00000000 08:09 819656 /usr/lib64/libpcre.so.0.0.1
7f260eafb000-7f260ecfa000 ---p 0002f000 08:09 819656 /usr/lib64/libpcre.so.0.0.1
7f260ecfa000-7f260ecfb000 r--p 0002e000 08:09 819656 /usr/lib64/libpcre.so.0.0.1
7f260ecfb000-7f260ecfc000 rw-p 0002f000 08:09 819656 /usr/lib64/libpcre.so.0.0.1
7f260ecfc000-7f260ecfe000 r-xp 00000000 08:07 548879 /lib64/libdl-2.11.1.so
7f260ecfe000-7f260eefe000 ---p 00002000 08:07 548879 /lib64/libdl-2.11.1.so
7f260eefe000-7f260eeff000 r--p 00002000 08:07 548879 /lib64/libdl-2.11.1.so
7f260eeff000-7f260ef00000 rw-p 00003000 08:07 548879 /lib64/libdl-2.11.1.so
7f260ef00000-7f260f054000 r-xp 00000000 08:07 548873 /lib64/libc-2.11.1.so
7f260f054000-7f260f254000 ---p 00154000 08:07 548873 /lib64/libc-2.11.1.so
7f260f254000-7f260f258000 r--p 00154000 08:07 548873 /lib64/libc-2.11.1.so
7f260f258000-7f260f259000 rw-p 00158000 08:07 548873 /lib64/libc-2.11.1.so
7f260f259000-7f260f25e000 rw-p 00000000 00:00 0
7f260f25e000-7f260f275000 r-xp 00000000 08:07 548899 /lib64/libpthread-2.11.1.so
7f260f275000-7f260f475000 ---p 00017000 08:07 548899 /lib64/libpthread-2.11.1.so
7f260f475000-7f260f476000 r--p 00017000 08:07 548899 /lib64/libpthread-2.11.1.so
7f260f476000-7f260f477000 rw-p 00018000 08:07 548899 /lib64/libpthread-2.11.1.so
7f260f477000-7f260f47b000 rw-p 00000000 00:00 0
7f260f47b000-7f260f5cf000 r-xp 00000000 08:09 819572 /usr/lib64/libcrypto.so.0.9.8
7f260f5cf000-7f260f7cf000 ---p 00154000 08:09 819572 /usr/lib64/libcrypto.so.0.9.8
7f260f7cf000-7f260f7dc000 r--p 00154000 08:09 819572 /usr/lib64/libcrypto.so.0.9.8
7f260f7dc000-7f260f7f3000 rw-p 00161000 08:09 819572 /usr/lib64/libcrypto.so.0.9.8
7f260f7f3000-7f260f7f6000 rw-p 00000000 00:00 0
7f260f7f6000-7f260f809000 r-xp 00000000 08:07 548901 /lib64/libresolv-2.11.1.so
7f260f809000-7f260fa09000 ---p 00013000 08:07 548901 /lib64/libresolv-2.11.1.so
7f260fa09000-7f260fa0a000 r--p 00013000 08:07 548901 /lib64/libresolv-2.11.1.so
7f260fa0a000-7f260fa0b000 rw-p 00014000 08:07 548901 /lib64/libresolv-2.11.1.so
7f260fa0b000-7f260fa0d000 rw-p 00000000 00:00 0
7f260fa0d000-7f260fa22000 r-xp 00000000 08:07 548884 /lib64/libnsl-2.11.1.so
7f260fa22000-7f260fc21000 ---p 00015000 08:07 548884 /lib64/libnsl-2.11.1.so
7f260fc21000-7f260fc22000 r--p 00014000 08:07 548884 /lib64/libnsl-2.11.1.so
7f260fc22000-7f260fc23000 rw-p 00015000 08:07 548884 /lib64/libnsl-2.11.1.so
7f260fc23000-7f260fc25000 rw-p 00000000 00:00 0
7f260fc25000-7f260fce9000 r-xp 00000000 08:09 819873 /usr/lib64/libglib-2.0.so.0.2200.5
7f260fce9000-7f260fee8000 ---p 000c4000 08:09 819873 /usr/lib64/libglib-2.0.so.0.2200.5
7f260fee8000-7f260fee9000 r--p 000c3000 08:09 819873 /usr/lib64/libglib-2.0.so.0.2200.5
7f260fee9000-7f260feea000 rw-p 000c4000 08:09 819873 /usr/lib64/libglib-2.0.so.0.2200.5
7f260feea000-7f260feeb000 rw-p 00000000 00:00 0
7f260feeb000-7f260fef3000 r-xp 00000000 08:07 548903 /lib64/librt-2.11.1.so
7f260fef3000-7f26100f2000 ---p 00008000 08:07 548903 /lib64/librt-2.11.1.so
7f26100f2000-7f26100f3000 r--p 00007000 08:07 548903 /lib64/librt-2.11.1.so
7f26100f3000-7f26100f4000 rw-p 00008000 08:07 548903 /lib64/librt-2.11.1.so
7f26100f4000-7f26100f8000 r-xp 00000000 08:09 819956 /usr/lib64/libgthread-2.0.so.0.2200.5
7f26100f8000-7f26102f7000 ---p 00004000 08:09 819956 /usr/lib64/libgthread-2.0.so.0.2200.5
7f26102f7000-7f26102f8000 r--p 00003000 08:09 819956 /usr/lib64/libgthread-2.0.so.0.2200.5
7f26102f8000-7f26102f9000 rw-p 00004000 08:09 819956 /usr/lib64/libgthread-2.0.so.0.2200.5
7f26102f9000-7f261033b000 r-xp 00000000 08:09 819958 /usr/lib64/libgobject-2.0.so.0.2200.5
7f261033b000-7f261053a000 ---p 00042000 08:09 819958 /usr/lib64/libgobject-2.0.so.0.2200.5
7f261053a000-7f261053b000 r--p 00041000 08:09 819958 /usr/lib64/libgobject-2.0.so.0.2200.5
7f261053b000-7f261053c000 rw-p 00042000 08:09 819958 /usr/lib64/libgobject-2.0.so.0.2200.5
7f261053c000-7f261053d000 rw-p 00000000 00:00 0
7f261053d000-7f2610540000 r-xp 00000000 08:09 819960 /usr/lib64/libgmodule-2.0.so.0.2200.5
7f2610540000-7f261073f000 ---p 00003000 08:09 819960 /usr/lib64/libgmodule-2.0.so.0.2200.5
7f261073f000-7f2610740000 r--p 00002000 08:09 819960 /usr/lib64/libgmodule-2.0.so.0.2200.5
7f2610740000-7f2610741000 rw-p 00003000 08:09 819960 /usr/lib64/libgmodule-2.0.so.0.2200.5
7f2610741000-7f2610796000 r-xp 00000000 08:07 548881 /lib64/libm-2.11.1.so
7f2610796000-7f2610995000 ---p 00055000 08:07 548881 /lib64/libm-2.11.1.so
7f2610995000-7f2610996000 r--p 00054000 08:07 548881 /lib64/libm-2.11.1.so
7f2610996000-7f2610997000 rw-p 00055000 08:07 548881 /lib64/libm-2.11.1.so
7f2610997000-7f26109f4000 r-xp 00000000 08:09 557179 /usr/local/lib/amanda/libndmlib-3.1.2.so
7f26109f4000-7f2610bf3000 ---p 0005d000 08:09 557179 /usr/local/lib/amanda/libndmlib-3.1.2.so
7f2610bf3000-7f2610bf4000 r--p 0005c000 08:09 557179 /usr/local/lib/amanda/libndmlib-3.1.2.so
7f2610bf4000-7f2610bfd000 rw-p 0005d000 08:09 557179 /usr/local/lib/amanda/libndmlib-3.1.2.so
7f2610bfd000-7f2610c5e000 r-xp 00000000 08:09 557158 /usr/local/lib/amanda/libamanda-3.1.2.so
7f2610c5e000-7f2610e5d000 ---p 00061000 08:09 557158 /usr/local/lib/amanda/libamanda-3.1.2.so
7f2610e5d000-7f2610e5e000 r--p 00060000 08:09 557158 /usr/local/lib/amanda/libamanda-3.1.2.so
7f2610e5e000-7f2610e62000 rw-p 00061000 08:09 557158 /usr/local/lib/amanda/libamanda-3.1.2.so
7f2610e62000-7f2610e89000 rw-p 00000000 00:00 0
7f2610e89000-7f2610e94000 r-xp 00000000 08:09 557172 /usr/local/lib/amanda/libamxfer-3.1.2.so
7f2610e94000-7f2611093000 ---p 0000b000 08:09 557172 /usr/local/lib/amanda/libamxfer-3.1.2.so
7f2611093000-7f2611094000 r--p 0000a000 08:09 557172 /usr/local/lib/amanda/libamxfer-3.1.2.so
7f2611094000-7f2611095000 rw-p 0000b000 08:09 557172 /usr/local/lib/amanda/libamxfer-3.1.2.so
7f2611095000-7f26110c0000 r-xp 00000000 08:09 557213 /usr/local/lib/amanda/libamdevice-3.1.2.so
7f26110c0000-7f26112c0000 ---p 0002b000 08:09 557213 /usr/local/lib/amanda/libamdevice-3.1.2.so
7f26112c0000-7f26112c1000 r--p 0002b000 08:09 557213 /usr/local/lib/amanda/libamdevice-3.1.2.so
7f26112c1000-7f26112c2000 rw-p 0002c000 08:09 557213 /usr/local/lib/amanda/libamdevice-3.1.2.so
7f26112c2000-7f26112e2000 r-xp 00000000 08:09 557251 /usr/local/lib/amanda/libamserver-3.1.2.so
7f26112e2000-7f26114e1000 ---p 00020000 08:09 557251 /usr/local/lib/amanda/libamserver-3.1.2.so
7f26114e1000-7f26114e2000 r--p 0001f000 08:09 557251 /usr/local/lib/amanda/libamserver-3.1.2.so
7f26114e2000-7f26114e3000 rw-p 00020000 08:09 557251 /usr/local/lib/amanda/libamserver-3.1.2.so
7f26114e3000-7f26114e5000 rw-p 00000000 00:00 0
7f26114e5000-7f2611504000 r-xp 00000000 08:07 548866 /lib64/ld-2.11.1.so
7f2611652000-7f2611654000 rw-p 00000000 00:00 0
7f2611654000-7f2611689000 r--s 00000000 08:08 786487 /var/run/nscd/services
7f2611689000-7f26116be000 r--s 00000000 08:08 786488 /var/run/nscd/db9sUrSm (deleted)
7f26116be000-7f26116c6000 rw-p 00000000 00:00 0
7f26116c9000-7f26116cb000 rw-p 00000000 00:00 0
7f26116cb000-7f2611700000 r--s 00000000 08:08 786484 /var/run/nscd/passwd
7f2611700000-7f2611703000 rw-p 00000000 00:00 0
7f2611703000-7f2611704000 r--p 0001e000 08:07 548866 /lib64/ld-2.11.1.so
7f2611704000-7f2611705000 rw-p 0001f000 08:07 548866 /lib64/ld-2.11.1.so
7f2611705000-7f2611706000 rw-p 00000000 00:00 0
7fff61275000-7fff6128a000 rw-p 00000000 00:00 0 [stack]
7fff61320000-7fff61321000 r-xp 00000000 00:00 0 [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]


(...)


driver: dumper0 pid 13278 exited with signal 11
driver: dumper1 pid 13279 exited with signal 6
driver: dumper2 pid 13280 exited with signal 11

^^ Above "exited with signal 11" means two of the dumpers exited with a coredump.