amrecover: Connection reset after first packet

[Two9A]

I'm trying to connect to the amanda server through amrecover, running on the same computer. A backup exists for the configuration I'm trying to access, but it doesn't connect.

First of all, would it help if I tried this from another computer? (Is amindexd tied to the idea that the client is a different machine?) Secondly, if it's not an issue of the box being the same for client and server, is there another issue that would cause amindexd to throw away the connection as soon as it comes in?

Amanda server is 192.168.1.90, client is running on the same computer and trying to connect to 192.168.1.90; backup configuration name is KH8. Relevant files and output below, let me know if anything more is required for debugging.

# amrecover KH8
AMRECOVER Version 2.6.0p2. Contacting server on 192.168.1.90 ...
[request failed: recv error: Connection reset by peer]

/var/spool/amanda/.amandahosts:
192.168.1.90 amanda amdump amindexd amidxtaped amandabackup
192.168.1.90 root amdump amindexd amidxtaped amandabackup

/etc/amanda/KH8/amanda-client.conf:
conf "KH8"
index_server "192.168.1.90"
tape_server "192.168.1.90"
auth "bsdtcp"
debug_auth 1
debug_amandad 1

/var/tmp/amanda/client/KH8/amrecover.20100715144207.debug:
1279201327.978119: amrecover: pid 5559 ruid 0 euid 0: start at Thu Jul 15 14:42:07 2010
1279201327.978248: amrecover: warning: errors processing config file "/etc/amanda/amanda-client.conf" (non-fatal)
1279201327.978507: amrecover: pid 5559 ruid 0 euid 0: rename at Thu Jul 15 14:42:07 2010
1279201327.978543: amrecover: security_getdriver(name=bsdtcp) returns 0xb777bc60
1279201332.986944: amrecover: bsdtcp: bsdtcp_connect: 192.168.1.90
1279201332.986980: amrecover: security_handleinit(handle=0x959f3c8, driver=0xb777bc60 (BSDTCP))
1279201332.987058: amrecover: security_streaminit(stream=0x95a2c38, driver=0xb777bc60 (BSDTCP))
1279201332.987068: amrecover: sec_tcp_conn_get: 192.168.1.90
1279201332.987075: amrecover: sec_tcp_conn_get: creating new handle
1279201332.987082: amrecover: sec: stream_client: connected to stream 2
1279201332.987655: amrecover: connect_port: Skip port 512: owned by exec.
1279201332.987802: amrecover: connect_port: Skip port 513: owned by login.
1279201332.987950: amrecover: connect_port: Skip port 514: owned by shell.
1279201332.988107: amrecover: connect_port: Skip port 515: owned by printer.
1279201332.988250: amrecover: connect_port: Skip port 516: owned by videotex.
1279201332.988393: amrecover: connect_port: Skip port 517: owned by talk.
1279201332.988553: amrecover: connect_port: Skip port 518: owned by ntalk.
1279201332.988715: amrecover: connect_port: Skip port 519: owned by utime.
1279201332.988860: amrecover: connect_port: Skip port 520: owned by efs.
1279201332.989024: amrecover: connect_port: Skip port 521: owned by ripng.
1279201332.989177: amrecover: connect_port: Skip port 522: owned by ulp.
1279201332.989323: amrecover: connect_port: Skip port 523: owned by ibm-db2.
1279201332.989470: amrecover: connect_port: Skip port 524: owned by ncp.
1279201332.989617: amrecover: connect_port: Skip port 525: owned by timed.
1279201332.989763: amrecover: connect_port: Skip port 526: owned by tempo.
1279201332.990121: amrecover: connect_port: Try port 527: available - Success
1279201335.990193: amrecover: connected to 192.168.1.90.10080
1279201335.990223: amrecover: our side is 0.0.0.0.527
1279201335.990234: amrecover: try_socksize: send buffer size is 65536
1279201335.990240: amrecover: try_socksize: receive buffer size is 65536
1279201335.990267: amrecover: sec: stream_sendpkt: enter
1279201335.990323: amrecover: sec: stream_sendpkt: REQ (0) pkt_t (len 70) contains:

"SERVICE amindexd
OPTIONS features=ffffffff9ffeffffffff00;auth=bsdtcp;
"

1279201335.990331: amrecover: sec: stream_write: writing 91 bytes to 192.168.1.90:2 3
1279201335.990339: amrecover: tcpm_send_token: write 91 bytes to handle 2
1279201335.990389: amrecover: sec: recvpkt registered for 192.168.1.90
1279201335.990398: amrecover: sec: conn_read registering event handler for 192.168.1.90
1279201336.535877: amrecover: sec: conn_read_callback
1279201336.535905: amrecover: net_read: begin 8
1279201336.535912: amrecover: net_read: while 8
1279201336.535920: amrecover: net_read_fillbuf: begin
1279201336.535928: amrecover: net_read_fillbuf: case 1
1279201336.535940: amrecover: db: net_read: end return(-1)
1279201336.535952: amrecover: tcpm_recv_token: A return(-1)
1279201336.535958: amrecover: sec: conn_read_callback: tcpm_recv_token returned -1
1279201336.535966: amrecover: sec: stream_read_callback: handle 2
1279201336.535974: amrecover: sec: conn_read_cancel: decremented ev_read_refcnt to 0 for 192.168.1.90
1279201336.535980: amrecover: sec: conn_read_cancel: releasing event handler for 192.168.1.90
1279201336.535986: amrecover: sec: stream_read_callback: recv error: Connection reset by peer
1279201336.535995: amrecover: security_stream_seterr(0x95a2c38, recv error: Connection reset by peer)
1279201336.536002: amrecover: sec_tcp_conn_put: decrementing refcnt for 192.168.1.90 to 1
1279201336.536008: amrecover: sec: recvpkt_callback: -1
1279201336.536014: amrecover: sec: cancelling recvpkt for 192.168.1.90
1279201336.536021: amrecover: security_seterror(handle=0x959f3c8, driver=0xb777bc60 (BSDTCP) error=recv error: Connection reset by peer)
1279201336.536041: amrecover: security_close(handle=0x959f3c8, driver=0xb777bc60 (BSDTCP))
1279201336.536048: amrecover: sec: closing handle to 192.168.1.90
1279201336.536054: amrecover: sec: cancelling recvpkt for 192.168.1.90
1279201336.536059: amrecover: security_stream_close(0x95a2c38)
1279201336.536065: amrecover: sec: tcpma_stream_close: closing stream 2
1279201336.536082: amrecover: sec: conn_read_callback: event_wakeup return 1


tcpdump output for connection attempt:
# tcpdump -vvvv -XX -s 1500 -i lo dst port 10080 or src port 10080
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 1500 bytes
14:42:07.981739 IP (tos 0x0, ttl 64, id 64154, offset 0, flags [DF], proto TCP (6), length 60) keele1.527 > keele1.amanda: S, cksum 0x4d0c (correct), 2366240129:2366240129(0) win 32792 <mss 16396,sackOK,timestamp 23531291 0,nop,wscale 9>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 003c fa9a 4000 4006 bc1c c0a8 015a c0a8 .<[email protected]@......Z..
0x0020: 015a 020f 2760 8d09 f581 0000 0000 a002 .Z..'`..........
0x0030: 8018 4d0c 0000 0204 400c 0402 080a 0167 [email protected]
0x0040: 0f1b 0000 0000 0103 0309 ..........
14:42:07.981751 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 40) keele1.amanda > keele1.527: R, cksum 0x7fd0 (correct), 0:0(0) ack 2366240130 win 0
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 0028 0000 4000 4006 b6cb c0a8 015a c0a8 .([email protected]@......Z..
0x0020: 015a 2760 020f 0000 0000 8d09 f582 5014 .Z'`..........P.
0x0030: 0000 7fd0 0000 ......
14:42:12.990147 IP (tos 0x0, ttl 64, id 6797, offset 0, flags [DF], proto TCP (6), length 60) keele1.527 > keele1.amanda: S, cksum 0x2a71 (correct), 2444496525:2444496525(0) win 32792 <mss 16396,sackOK,timestamp 23532544 0,nop,wscale 9>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 003c 1a8d 4000 4006 9c2a c0a8 015a c0a8 .<[email protected]@..*...Z..
0x0020: 015a 020f 2760 91b4 0e8d 0000 0000 a002 .Z..'`..........
0x0030: 8018 2a71 0000 0204 400c 0402 080a 0167 ..*[email protected]
0x0040: 1400 0000 0000 0103 0309 ..........
14:42:15.989915 IP (tos 0x0, ttl 64, id 6798, offset 0, flags [DF], proto TCP (6), length 60) keele1.527 > keele1.amanda: S, cksum 0x2783 (correct), 2444496525:2444496525(0) win 32792 <mss 16396,sackOK,timestamp 23533294 0,nop,wscale 9>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 003c 1a8e 4000 4006 9c29 c0a8 015a c0a8 .<[email protected]@..)...Z..
0x0020: 015a 020f 2760 91b4 0e8d 0000 0000 a002 .Z..'`..........
0x0030: 8018 2783 0000 0204 400c 0402 080a 0167 ..'[email protected]
0x0040: 16ee 0000 0000 0103 0309 ..........
14:42:15.989935 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60) keele1.amanda > keele1.527: S, cksum 0x8c07 (correct), 2481516356:2481516356(0) ack 2444496526 win 32768 <mss 16396,sackOK,timestamp 23533294 23533294,nop,wscale 9>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 003c 0000 4000 4006 b6b7 c0a8 015a c0a8 .<[email protected]@......Z..
0x0020: 015a 2760 020f 93e8 ef44 91b4 0e8e a012 .Z'`.....D......
0x0030: 8000 8c07 0000 0204 400c 0402 080a 0167 [email protected]
0x0040: 16ee 0167 16ee 0103 0309 ...g......
14:42:15.989952 IP (tos 0x0, ttl 64, id 6799, offset 0, flags [DF], proto TCP (6), length 52) keele1.527 > keele1.amanda: ., cksum 0x74ed (correct), 1:1(0) ack 1 win 65 <nop,nop,timestamp 23533294 23533294>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 0034 1a8f 4000 4006 9c30 c0a8 015a c0a8 [email protected]@..0...Z..
0x0020: 015a 020f 2760 91b4 0e8e 93e8 ef45 8010 .Z..'`.......E..
0x0030: 0041 74ed 0000 0101 080a 0167 16ee 0167 .At........g...g
0x0040: 16ee ..
14:42:15.990366 IP (tos 0x0, ttl 64, id 6800, offset 0, flags [DF], proto TCP (6), length 151) keele1.527 > keele1.amanda: P, cksum 0x848e (incorrect (-> 0x68c7), 1:100(99) ack 1 win 65 <nop,nop,timestamp 23533294 23533294>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 0097 1a90 4000 4006 9bcc c0a8 015a c0a8 [email protected]@......Z..
0x0020: 015a 020f 2760 91b4 0e8e 93e8 ef45 8018 .Z..'`.......E..
0x0030: 0041 848e 0000 0101 080a 0167 16ee 0167 .A.........g...g
0x0040: 16ee 0000 005b 0000 0002 0053 4543 5552 .....[.....SECUR
0x0050: 4954 5920 5553 4552 2072 6f6f 740a 5345 ITY.USER.root.SE
0x0060: 5256 4943 4520 616d 696e 6465 7864 0a4f RVICE.amindexd.O
0x0070: 5054 494f 4e53 2066 6561 7475 7265 733d PTIONS.features=
0x0080: 6666 6666 6666 6666 3966 6665 6666 6666 ffffffff9ffeffff
0x0090: 6666 6666 3030 3b61 7574 683d 6273 6474 ffff00;auth=bsdt
0x00a0: 6370 3b0a 00 cp;..
14:42:15.990384 IP (tos 0x0, ttl 64, id 63915, offset 0, flags [DF], proto TCP (6), length 52) keele1.amanda > keele1.527: ., cksum 0x748b (correct), 1:1(0) ack 100 win 64 <nop,nop,timestamp 23533294 23533294>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 0034 f9ab 4000 4006 bd13 c0a8 015a c0a8 [email protected]@......Z..
0x0020: 015a 2760 020f 93e8 ef45 91b4 0ef1 8010 .Z'`.....E......
0x0030: 0040 748b 0000 0101 080a 0167 16ee 0167 [email protected]
0x0040: 16ee ..
14:42:16.535852 IP (tos 0x0, ttl 64, id 63916, offset 0, flags [DF], proto TCP (6), length 52) keele1.amanda > keele1.527: R, cksum 0x73ff (correct), 1:1(0) ack 100 win 64 <nop,nop,timestamp 23533430 23533294>
0x0000: 0000 0000 0000 0000 0000 0000 0800 4500 ..............E.
0x0010: 0034 f9ac 4000 4006 bd12 c0a8 015a c0a8 [email protected]@......Z..
0x0020: 015a 2760 020f 93e8 ef45 91b4 0ef1 8014 .Z'`.....E......
0x0030: 0040 73ff 0000 0101 080a 0167 1776 0167 [email protected]
0x0040: 16ee ..

[martineau]

Do amandad is executed? if no, then what's the xinetd configuration?
If yes, post its debug file.

[Two9A]

There doesn't seem to be a new amandad file when I restart xinetd and then try to amrecover, so I assume it's not being started.

/etc/xinetd.d/amanda:
service amanda
{
socket_type = stream
protocol = tcp
wait = yes
user = amanda
group = amanda
groups = yes
server = /usr/libexec/amanda/amandad
server-args = -auth=ssh amdump amindexd amidxtaped
disable = no
}

service amandaidx
{
socket_type = stream
protocol = tcp
wait = no
user = amanda
group = amanda
groups = yes
server = /usr/libexec/amanda/amindexd
disable = no
}

service amidxtape
{
socket_type = stream
protocol = tcp
wait = no
user = amanda
group = amanda
groups = yes
server = /usr/libexec/amanda/amidxtaped
disable = no
}

[Two9A]

Well, that's the issue, isn't it. ssh authentication.

I've changed /etc/amanda/KH8/amanda-client.conf to use ssh auth, with the key for the amanda user, but I'm getting back "Host key verification failed". I've put the public keys for amanda and root in ~amanda/.ssh/authorized_keys, and I can connect to localhost as both users, but amrecover doesn't seem to accept that. Any ideas?