PDA

View Full Version : socket-copy.pl & socket-server.pl failure

sfrank
December 18th, 2006, 06:18 AM
Hi,

I've been trying to set up a remote backup and here is what happens:

a) In the first case I'm using 'raw' backups with full backup leve.
"""
root@devserver:/etc/mysql# mysql-zrm-scheduler --now --backup-set cage --backup-level 0
Logging to /var/log/mysql-zrm/mysql-zrm-scheduler.log
INFO: ZRM for MySQL Community Edition - version 1.1.2
INFO: Input Parameters Used {
INFO: quiet=0
INFO: verbose=1
INFO: retention-policy=10D
INFO: backup-level=0
INFO: all-databases=1
INFO: destination=/var/lib/mysql-zrm
INFO: port=3306
INFO: host=192.168.1.102
INFO: backup-mode=raw
INFO: password=******
INFO: comment=Backups from cage
INFO: compress-plugin=/bin/gzip
INFO: user=zrmuser
INFO: mysql-binlog-path=/var/log/mysql
INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
INFO: }
INFO: Getting mysql variables
INFO: mysqladmin --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" variables
INFO: datadir is /var/lib/mysql/
INFO: mysql_version is 5.0.24a-Debian_9-log
INFO: log_bin=ON
INFO: InnoDB data file are /var/lib/mysql/ibdata1
INFO: InnoDB log dir is /var/lib/mysql/.
INFO: backup set being used is cage
INFO: backup-set=cage
INFO: backup-date=20061218144314
INFO: host=192.168.1.102
INFO: backup-date-epoch=1166449394
INFO: retention-policy=10D
INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.1.2
INFO: mysql-version=5.0.24a-Debian_9-log
INFO: backup-directory=/var/lib/mysql-zrm/cage/20061218144314
INFO: comment=Backups from cage
INFO: Executing pre-backup-plugin
INFO: Flushing the logs
INFO: flush-logs
INFO: Getting master logname using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master status"
INFO: backup-level=0
INFO: Getting list of Databases
INFO: mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show databases;"
INFO: Command used for getting engine type mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show table status from mysql"
INFO: For database mysql
INFO: engine
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: Command used for raw backup is /usr/share/mysql-zrm/plugins/socket-copy.pl --mysqlhotcopy --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" --quiet mysql "/var/lib/mysql-zrm/cage/20061218144314" > /tmp/RxBlZiwVem 2>&1
ERROR: Output of command: 'mysqlhotcopy' is {

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
}
ERROR: mysqlhotcopy command did not succeed.
Command used is /usr/share/mysql-zrm/plugins/socket-copy.pl --mysqlhotcopy --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" --quiet mysql "/var/lib/mysql-zrm/cage/20061218144314" > /tmp/RxBlZiwVem 2>&1
Return value is 512
INFO: backup-status=Backup failed
INFO: Backup failed
ERROR: /usr/bin/mysql-zrm did not finish successfully
"""

b) In the second case I've switched to logical backups but using incremental in order to have something to move with the socket-copy plugin.
"""
root@devserver:/etc/mysql# mysql-zrm-scheduler --now --backup-set cage --backup-level 1
Logging to /var/log/mysql-zrm/mysql-zrm-scheduler.log
INFO: ZRM for MySQL Community Edition - version 1.1.2
INFO: Input Parameters Used {
INFO: quiet=0
INFO: verbose=1
INFO: retention-policy=10D
INFO: backup-level=1
INFO: all-databases=1
INFO: destination=/var/lib/mysql-zrm
INFO: port=3306
INFO: host=192.168.1.102
INFO: backup-mode=logical
INFO: password=******
INFO: comment=Backups from cage
INFO: compress-plugin=/bin/gzip
INFO: user=zrmuser
INFO: mysql-binlog-path=/var/log/mysql
INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
INFO: }
INFO: Getting mysql variables
INFO: mysqladmin --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" variables
INFO: datadir is /var/lib/mysql/
INFO: mysql_version is 5.0.24a-Debian_9-log
INFO: log_bin=ON
INFO: InnoDB data file are /var/lib/mysql/ibdata1
INFO: InnoDB log dir is /var/lib/mysql/.
INFO: backup set being used is cage
INFO: backup-set=cage
INFO: backup-date=20061218144228
INFO: host=192.168.1.102
INFO: backup-date-epoch=1166449348
INFO: retention-policy=10D
INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.1.2
INFO: mysql-version=5.0.24a-Debian_9-log
INFO: backup-directory=/var/lib/mysql-zrm/cage/20061218144228
INFO: comment=Backups from cage
INFO: Executing pre-backup-plugin
INFO: Flushing the logs
INFO: flush-logs
INFO: Getting master logname using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master status"
INFO: backup-level=1
INFO: Getting list of binary logs using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master logs"

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
ERROR: Could not copy file /var/log/mysql/mysql-bin.000027
ERROR: copy-plugin exited with error 512
ERROR: Could not copy bin log file mysql-bin.000027
INFO: incremental=mysql-bin.[0-9]*
INFO: next-binlog=mysql-bin.000028
INFO: last-backup=/var/lib/mysql-zrm/cage/20061218144155
INFO: backup-size=0.00 MB
INFO: Executing post-backup-plugin
INFO: read-locks-time=00:00:00
INFO: flush-logs-time=00:00:00
INFO: backup-time=00:00:00
INFO: backup-status=Backup done but with errors
INFO: Backup done but with errors
ERROR: /usr/bin/mysql-zrm did not finish successfully
"""

I've also tried to run the socket-copy.pl alone with the necessary parameters to move a file around, but it gave me the same gzip & tar error message.

Here is the content of the /tmp/client.log on the socket-server's side:
"""
Client started
ERROR: Bad data in
"""

I'm running Ubuntu Edgy on the MySQL server and Debian Sarge on the ZRM host. The host that I tried to hand-run the socket-copy.pl was a Debian testing which quite resembles to my MySQL host.

Any suggestions, hints are welcome.
kkg
December 18th, 2006, 06:31 AM
Hi,

I've been trying to set up a remote backup and here is what happens:

a) In the first case I'm using 'raw' backups with full backup leve.
"""
root@devserver:/etc/mysql# mysql-zrm-scheduler --now --backup-set cage --backup-level 0
Logging to /var/log/mysql-zrm/mysql-zrm-scheduler.log
INFO: ZRM for MySQL Community Edition - version 1.1.2
INFO: Input Parameters Used {
INFO: quiet=0
INFO: verbose=1
INFO: retention-policy=10D
INFO: backup-level=0
INFO: all-databases=1
INFO: destination=/var/lib/mysql-zrm
INFO: port=3306
INFO: host=192.168.1.102
INFO: backup-mode=raw
INFO: password=******
INFO: comment=Backups from cage
INFO: compress-plugin=/bin/gzip
INFO: user=zrmuser
INFO: mysql-binlog-path=/var/log/mysql
INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
INFO: }
INFO: Getting mysql variables
INFO: mysqladmin --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" variables
INFO: datadir is /var/lib/mysql/
INFO: mysql_version is 5.0.24a-Debian_9-log
INFO: log_bin=ON
INFO: InnoDB data file are /var/lib/mysql/ibdata1
INFO: InnoDB log dir is /var/lib/mysql/.
INFO: backup set being used is cage
INFO: backup-set=cage
INFO: backup-date=20061218144314
INFO: host=192.168.1.102
INFO: backup-date-epoch=1166449394
INFO: retention-policy=10D
INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.1.2
INFO: mysql-version=5.0.24a-Debian_9-log
INFO: backup-directory=/var/lib/mysql-zrm/cage/20061218144314
INFO: comment=Backups from cage
INFO: Executing pre-backup-plugin
INFO: Flushing the logs
INFO: flush-logs
INFO: Getting master logname using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master status"
INFO: backup-level=0
INFO: Getting list of Databases
INFO: mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show databases;"
INFO: Command used for getting engine type mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show table status from mysql"
INFO: For database mysql
INFO: engine
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: MyISAM
INFO: Command used for raw backup is /usr/share/mysql-zrm/plugins/socket-copy.pl --mysqlhotcopy --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" --quiet mysql "/var/lib/mysql-zrm/cage/20061218144314" > /tmp/RxBlZiwVem 2>&1
ERROR: Output of command: 'mysqlhotcopy' is {

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
}
ERROR: mysqlhotcopy command did not succeed.
Command used is /usr/share/mysql-zrm/plugins/socket-copy.pl --mysqlhotcopy --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" --quiet mysql "/var/lib/mysql-zrm/cage/20061218144314" > /tmp/RxBlZiwVem 2>&1
Return value is 512
INFO: backup-status=Backup failed
INFO: Backup failed
ERROR: /usr/bin/mysql-zrm did not finish successfully
"""

b) In the second case I've switched to logical backups but using incremental in order to have something to move with the socket-copy plugin.
"""
root@devserver:/etc/mysql# mysql-zrm-scheduler --now --backup-set cage --backup-level 1
Logging to /var/log/mysql-zrm/mysql-zrm-scheduler.log
INFO: ZRM for MySQL Community Edition - version 1.1.2
INFO: Input Parameters Used {
INFO: quiet=0
INFO: verbose=1
INFO: retention-policy=10D
INFO: backup-level=1
INFO: all-databases=1
INFO: destination=/var/lib/mysql-zrm
INFO: port=3306
INFO: host=192.168.1.102
INFO: backup-mode=logical
INFO: password=******
INFO: comment=Backups from cage
INFO: compress-plugin=/bin/gzip
INFO: user=zrmuser
INFO: mysql-binlog-path=/var/log/mysql
INFO: copy-plugin=/usr/share/mysql-zrm/plugins/socket-copy.pl
INFO: }
INFO: Getting mysql variables
INFO: mysqladmin --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" variables
INFO: datadir is /var/lib/mysql/
INFO: mysql_version is 5.0.24a-Debian_9-log
INFO: log_bin=ON
INFO: InnoDB data file are /var/lib/mysql/ibdata1
INFO: InnoDB log dir is /var/lib/mysql/.
INFO: backup set being used is cage
INFO: backup-set=cage
INFO: backup-date=20061218144228
INFO: host=192.168.1.102
INFO: backup-date-epoch=1166449348
INFO: retention-policy=10D
INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 1.1.2
INFO: mysql-version=5.0.24a-Debian_9-log
INFO: backup-directory=/var/lib/mysql-zrm/cage/20061218144228
INFO: comment=Backups from cage
INFO: Executing pre-backup-plugin
INFO: Flushing the logs
INFO: flush-logs
INFO: Getting master logname using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master status"
INFO: backup-level=1
INFO: Getting list of binary logs using command mysql --user="zrmuser" --password="*****" --host="192.168.1.102" --port="3306" -e "show master logs"

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
ERROR: Could not copy file /var/log/mysql/mysql-bin.000027
ERROR: copy-plugin exited with error 512
ERROR: Could not copy bin log file mysql-bin.000027
INFO: incremental=mysql-bin.[0-9]*
INFO: next-binlog=mysql-bin.000028
INFO: last-backup=/var/lib/mysql-zrm/cage/20061218144155
INFO: backup-size=0.00 MB
INFO: Executing post-backup-plugin
INFO: read-locks-time=00:00:00
INFO: flush-logs-time=00:00:00
INFO: backup-time=00:00:00
INFO: backup-status=Backup done but with errors
INFO: Backup done but with errors
ERROR: /usr/bin/mysql-zrm did not finish successfully
"""

I've also tried to run the socket-copy.pl alone with the necessary parameters to move a file around, but it gave me the same gzip & tar error message.

Here is the content of the /tmp/client.log on the socket-server's side:
"""
Client started
ERROR: Bad data in
"""

I'm running Ubuntu Edgy on the MySQL server and Debian Sarge on the ZRM host. The host that I tried to hand-run the socket-copy.pl was a Debian testing which quite resembles to my MySQL host.

Any suggestions, hints are welcome.

The client.pl only allows the following characters to be passed to it.
a-z A-Z 0-9 _ - / . = " ; and space. This is for security reasons so that arbitrary data is not passed to it. Any other character in the input stram to client.pl will generate the bad data error. So if your password has other characters, this error will be generated.

If your password has other characters, you may want to modify the list of characters that can be passed to client.pl.

Let me kow if this works for you.
--kkg
sfrank
December 18th, 2006, 07:02 AM
Unfortunately it did not help. My password for the test setup is 'zmpass' which matches the scheme your regexp determines.

Also see the following which does not include any tricky argument:
"""
root@devserver:/etc/mysql-zrm# /usr/share/mysql-zrm/plugins/socket-copy.pl --source-host=192.168.1.102 --source-file=/tmp/salala --destination-host=localhost --destination-directory=.

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
"""

Furthermore is it normal that there's 'Bad data in ' in my client.log while socket-server.pl supposed to print in what the bad data is. At least according to this:
"""
&printAndDie("Bad data in $_[0]\n");
"""
sfrank
December 18th, 2006, 07:15 AM
OK, here's the deal. I've downloaded 1.1.1 tar.gz release and used the client.pl, and it works now. However my problem with the mysql-zrm-socket-server.deb still persists (socket-server.pl).

I'm happy to give you guys more info on my setup if needed.
kkg
December 18th, 2006, 10:02 AM
OK, here's the deal. I've downloaded 1.1.1 tar.gz release and used the client.pl, and it works now. However my problem with the mysql-zrm-socket-server.deb still persists (socket-server.pl).

I'm happy to give you guys more info on my setup if needed.


Could you please set the TMPDIR env variable to your tmp directory (such as /tmp) and run mysql-zrm and let us know if that fixes the problem?

--kkg
kkg
December 18th, 2006, 11:27 AM
Could you please set the TMPDIR env variable to your tmp directory (such as /tmp) and run mysql-zrm and let us know if that fixes the problem?

--kkg

Rereading my earlier email I realized that I should have added an explanation on why I asked to specify TMPDIR.

In 1.1.2 we have added a functionality to be able to specify the tmp directory to use for all tmp file creation. This is also read by the copy plugin and passed to client.pl. Now if tmpdir is not specified, there is a bug in 1.1.2 where the socket-copy.pl sends a blank line to client.pl and this causes the error in client.pl and why the error in client.log shows that the bad data is empty. I did not pay attention to that when I initially replied. Sorry about that.

This issue is fixed in the latest tree and will be available in the next release.

The work around in 1.1.2 is either to specify the TMPDIR env variable or provide the tmpdir option to ZRM either in the config file or on the command line.

--kkg
sfrank
December 18th, 2006, 01:57 PM
I'll give it a try tomorrow when I'm in my office. Thanks for your clear explanation anyways.

One more quick question though: could you please tell me how client.pl, socket-copy.pl and socket-server.pl relate to each other? I've read a couple of manuals one in the tgz release, one in the deb file that comes with the socket-server and one in the zrm-server .deb package and I'm really confused now.

From what I understand is that earlier versions use client.pl on both (the MySQL and ZRM's) side, but newer installs should stick with socket-server.pl and socket-copy.pl setup, right?

Please clarify.
paddy
December 18th, 2006, 02:04 PM
If you are using MySQL ZRM 1.1.2, please install mysql-zrm-socket-server debian
package on the machine running MySQL server and mysql-zrm package on MySQL ZRM machine.

For MySQL ZRM 1.1.2, socket-server.pl should be installed on the MySQL server and
socket-copy.pl on the machine running MySQL ZRM.

Thanks,
Paddy
sfrank
December 18th, 2006, 02:09 PM
I did what you asked. I've uncommented the line '$ENV{'TMPDIR'}="/tmp";' and it works now.

/tmp/client.log says:
"""
Client started
Client clean exit
"""

Altough if you still have a second, please answer my question above. Thanks for your help.
sfrank
December 18th, 2006, 02:11 PM
If you are using MySQL ZRM 1.1.2, please install mysql-zrm-socket-server debian
package on the machine running MySQL server and mysql-zrm package on MySQL ZRM machine.

For MySQL ZRM 1.1.2, socket-server.pl should be installed on the MySQL server and
socket-copy.pl on the machine running MySQL ZRM.

Thanks,
Paddy
This is exactly what I've started with, but without declaring the TMPDIR variable in the socket-copy.pl, it did not work. Please see above. But I'm set for now. Thanks.
kkg
December 18th, 2006, 10:31 PM
I'll give it a try tomorrow when I'm in my office. Thanks for your clear explanation anyways.

One more quick question though: could you please tell me how client.pl, socket-copy.pl and socket-server.pl relate to each other? I've read a couple of manuals one in the tgz release, one in the deb file that comes with the socket-server and one in the zrm-server .deb package and I'm really confused now.

From what I understand is that earlier versions use client.pl on both (the MySQL and ZRM's) side, but newer installs should stick with socket-server.pl and socket-copy.pl setup, right?

Please clarify.

Hi,

socket-server.pl and client.pl are the same. We have renamed client.pl to socket-server.pl in the release so that we get more consistent and proper naming. Hence newer installs should stick with socket-server.pl and socket-copy.pl.

mysql-zrm uses the copy-plugin to do
a) Run mysqlhotcopy on the remote machine.
b) copy data from the remote machine to the backup machine and vice versa.

The socket-copy.pl is one of two copy plugins that we provide.
The socket-copy.pl talks to socket-server.pl on port 25300.
As of 1.1.2, the socket-server.pl expects to be passed 5 parameters
a) The version of the socket-copy.pl talking to it. This is mainly to ensure that there is no version mismatch.
b) The action to be performed. This is either of "mysqlhotcopy", "copy to" or "copy from"
c) The relevant parameters for the above action
d) The tmp directory to use for storing temporary files
e) The path where mysql client binaries can be found.

Once these parameters are received by socket-server.pl, it performs the action requested and exits.

Let me know if if you need any more information or if my explanation is not clear.
--kkg
sfrank
December 19th, 2006, 03:16 AM
That's clear now. Thanks for the clarification. And I also add, that having the TMPDIR defined, it works fine with the socket-server.pl also.

Edit: Probably the naming 'client.pl' confused me, because the scenario you described is rather a client - server model where the server is the machine running the MySQL to be backed up, while you used to call the script 'client.pl' (which later became socket-server.pl properly). But now it's clear.
lakshmi
June 30th, 2008, 06:21 AM
I took full logical remote database backup successfully but I am getting below error when I try to take logical incremental backup.

gzip: stdin: unexpected end of file
/bin/tar: Child returned status 1
/bin/tar: Error exit delayed from previous errors
close of pipe failed
test_backup:backup:ERROR: Could not copy file /var/lib/mysql/mysql_log.000019
test_backup:backup:ERROR: copy-plugin exited with error 512
test_backup:backup:ERROR: Could not copy bin log file mysql_log.000019
test_backup:backup:INFO: incremental=mysql_log.[0-9]*
test_backup:backup:INFO: next-binlog=mysql_log.000020
test_backup:backup:INFO: last-backup=/var/lib/mysql-zrm/test_backup/20080701065237
test_backup:backup:INFO: backup-size=0.00 MB
test_backup:backup:INFO: read-locks-time=00:00:00
test_backup:backup:INFO: flush-logs-time=00:00:01
test_backup:backup:INFO: backup-time=00:00:03
test_backup:backup:INFO: backup-status=Backup done but with errors
test_backup:backup:INFO: Backup done but with errors
test_backup:backup:INFO: Executing post-backup-plugin
test_backup:backup:INFO: mailing file /tmp/IbqNC8acNa
test_backup:backup:INFO: mail command is cat "/tmp/IbqNC8acNa"|mail -s "[ZRM for MySQL Report] backup-set test_backup" root@localhost
test_backup:backup:INFO: mailing file /tmp/qmm0x4qPwr
test_backup:backup:INFO: mail command is cat "/tmp/qmm0x4qPwr"|mail -s "[ZRM for MySQL Report] ERROR during backup of backup-set test_backup" root@localhost

I checked the content of the /tmp/mysql.sock on the socket-server's side(remote mysql server) but nothing was there.

Also password is very simple and it has all allowed characters only.

I set the TMPDIR env variable to tmp directory (such as /tmp) in mysql-zrm installed machine and run mysql-zrm but by this time also same error I am getting.

For your information, I am using MySQL ZRM 1.2.1-1 with mysql 5.0.18-Max version on Suse Linux 10.

Can you please suggest me what may be the reason for it.