PDA

View Full Version : amanda and OpenSolaris b80+ (SUN_SSH)



Bob Lawhead
March 21st, 2008, 05:23 PM
Head's-up for OpenSolaris/Amanda users. What is actually an SSHd problem causes amanda errors.

Problem: 3 out of 5 of the default SUN_SSH ciphers cause fatal errors re-keying.

Software: (distributed in snv_b80)
sshd version Sun_SSH_1.2
Sun_SSH_1.2, SSH protocols 1.5/2.0, OpenSSL 0x0090801f

Observations:
I initially noticed that some Amanda backups (using SSH transport)
intermittently failed after BFUing SXDE (b79b) to snv_b84. After
noting "Disconnecting: Protocol error: expected packet type 31, got
20" in the Amanda server logs, I realized that the errors were due to
Amanda's use of SSH for transport. Of particular interest, was that
packet type 20 was re-key request. Errors never occurred backups less
than 1GB, and always occurred in files > 2GB. In Amanda reports,
errors were noted as "lev 0 FAILED [missing size line from
sendbackup]" and as "sendbackup: critical (fatal): index tee cannot
write [Broken pipe]" in the client debug logs.

I noted the same symptoms when running Amanda on builds b83 & 85.
I eventually confirmed that the errors also occurred in b80.
In the b80 release notes, I found:

Issues Resolved: PSARC case 2007/034 : ssh/sshd resync with OpenSSH
BUG/RFE:5040151ssh(1) and sshd(1M) should re-key
periodically as per-recent recommendations

If I replaced the SUN_SSH server by an OpenSSH 4.7p1 server, the
symptoms no longer occurred. Further analysis revealed that the issue
occurred only for certain ciphers. I'm now able to demonstrate the
problem independent Amanda, as seen below.

Reproduce by:

% for l in 1G 10M; do
for c in arcfour aes128-ctr aes128-cbc 3des-cbc blowfish-cbc; do
echo ------ cipher=$c rekey=$l ------
ssh -o "Ciphers $c" -o "RekeyLimit 1G" $remote 'tar cf - .' >/dev/null
echo status=$?
done
done
------ cipher=arcfour rekey=1G ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255
------ cipher=aes128-ctr rekey=1G ------
status=0
------ cipher=aes128-cbc rekey=1G ------
status=0
------ cipher=3des-cbc rekey=1G ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255
------ cipher=blowfish-cbc rekey=1G ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255

------ cipher=arcfour rekey=10M ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255
------ cipher=aes128-ctr rekey=10M ------
status=0
------ cipher=aes128-cbc rekey=10M ------
status=0
------ cipher=3des-cbc rekey=10M ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255
------ cipher=blowfish-cbc rekey=10M ------
Disconnecting: Protocol error: expected packet type 31, got 20
status=255

Bob Lawhead
March 25th, 2008, 11:40 AM
It looks like the problem with arcfour, 3des and blowfish only occur when
RekeyLimit >= 1G, so setting it to 1023M in ~amanda/.ssh/config should help.