Results 1 to 2 of 2

Thread: Sudden failed snapshots

  1. #1
    Join Date
    Aug 2014
    Posts
    1

    Default Sudden failed snapshots

    After upgrading a machine my snapshot/ssh-copy based raw backups fail. I have confirmed the mysql uid/gids are uniform, rsa keys are shared and working, all of the zrm plugins including the socket server are owned by mysql.mysql. I've been working on this for 3 days and our backups are stale. I've had zrm running for years and have never encountered this. Here's the resulting log, any help would be greatly appreciated:

    Mon Aug 18 15:48:41 2014: backup:INFO: Reading options from file /etc/mysql-zrm/mysql-zrm.conf
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: START OF BACKUP
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: PHASE START: Initialization
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: Reading options from file /etc/mysql-zrm/aux/mysql-zrm.conf
    Mon Aug 18 15:48:41 2014: aux:backup:WARNING: The lvm-snapshot option is deprecated. Use snapshot-size instead
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: Mail address: zrm-backups is ok
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: ZRM Temporary configuration file = /etc/mysql-zrm/aux/tmpT41IQ.conf
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: {
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: mail-policy=only-on-error
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: html-reports=backup-status-info
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: retention-policy=7D
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: destination=/data/mysql-zrm
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: backup-level=0
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: user=zrm
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: snapshot-plugin=/usr/share/mysql-zrm/plugins/lvm-snapshot.pl
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: verbose=1
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: host=dione
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: all-databases=1
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: tmpdir=/data/mysql-zrm
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: replication=0
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: socket=/opt/md3/mysql/auxdb/auxdb.sock
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: backup-mode=raw
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: html-report-directory=/var/www/sys/zrm/reports/aux
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: logical-parallel=0
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: snapshot-size=5000M
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: compress-plugin=/usr/bin/pigz
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: rss-header-location=/etc/mysql-zrm
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: lvm-snapshot=5000M
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: mailto=zrm-backups
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: port=3307
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: password=******
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: mysql-binlog-path=/opt/md3/mysql/auxdb/logs/
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: copy-plugin=/usr/share/mysql-zrm/plugins/ssh-copy.pl
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: backup-type=regular
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: }
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: Getting mysql variables
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: mysqladmin --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" variables
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: datadir is /opt/md3/mysql/auxdb/db/
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: mysql_version is 5.5.17-log
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: InnoDB data file are /opt/md3/mysql/auxdb/db/ibdata1
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: InnoDB log dir is /opt/md3/mysql/auxdb/db/.
    Mon Aug 18 15:48:41 2014: aux:backup:INFO: Running snapshot verification using command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action verify-config 2>/data/mysql-zrm/8FH0BhVmVf
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup set being used is aux
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-set=aux
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-date=20140818154841
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: mysql-server-os=Linux/Unix
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-type=regular
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: host=dione
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-date-epoch=1408394921
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: retention-policy=7D
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: mysql-zrm-version=ZRM for MySQL Community Edition - version 3.0
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: mysql-version=5.5.17-log
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-directory=/data/mysql-zrm/aux/20140818154841
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-level=0
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup-mode=raw
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE END: Initialization
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE START: Running pre backup plugin
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Executing pre-backup-plugin
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE END: Running pre backup plugin
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE START: Flushing logs
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Flushing the logs
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: mysqladmin --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" flush-logs
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Getting master logname using command mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show master status"
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE END: Flushing logs
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Getting list of Databases
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show databases;"
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: backup of the following databases will be done catyblog cmail_db_prod1 deleteddata mysql others performance_schema test ums ums_db
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE START: Creating snapshot based backup
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: checking if links should be created using command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action supports-regular 2>/data/mysql-zrm/8FH0BhVmVf
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: innodb on lvm = 0
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: getting device details using command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action get-vm-device-details --directory /opt/md3/mysql/auxdb/db/. --sname zrm7kwjacinpH --host dione 2>/data/mysql-zrm/8FH0BhVmVf
    Mon Aug 18 15:48:42 2014: aux:backup:ERROR: Command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action get-vm-device-details --directory /opt/md3/mysql/auxdb/db/. --sname zrm7kwjacinpH --host dione 2>/data/mysql-zrm/8FH0BhVmVf returned error
    Mon Aug 18 15:48:42 2014: aux:backup:WARNING: Error getting device details for /opt/md3/mysql/auxdb/db/.
    Mon Aug 18 15:48:42 2014: aux:backup:WARNING: Output of command: 'Getting Device Details' is {
    tar: --same-order option cannot be used with -c
    Try 'tar --help' or 'tar --usage' for more information.
    tar: This does not look like a tar archive
    tar: Error exit delayed from previous errors
    Unable to send conf file to dione
    copy-plugin returned error

    }
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE END: Creating snapshot based backup
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: PHASE START: Find table type
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`catyblog\`"
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: For database catyblog
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: engine
    Mon Aug 18 15:48:42 2014: aux:backup:WARNING: Database catyblog is empty and hence will not be backedup
    Mon Aug 18 15:48:42 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`cmail_db_prod1\`"
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: For database cmail_db_prod1
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`deleteddata\`"
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: For database deleteddata
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: engine
    Mon Aug 18 15:48:43 2014: aux:backup:WARNING: Database deleteddata is empty and hence will not be backedup
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`mysql\`"
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: For database mysql
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: engine
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: CSV
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: CSV
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: MyISAM
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`others\`"
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: For database others
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: engine
    Mon Aug 18 15:48:43 2014: aux:backup:WARNING: Database others is empty and hence will not be backedup
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: Command used for getting engine type mysql --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" -e "show table status from \`performance_schema\`"
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: For database performance_schema
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: engine
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: InnoDB
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: PHASE END: Find table type
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: PHASE START: Creating raw backup
    Mon Aug 18 15:48:43 2014: aux:backup:INFO: Command used for raw backup is /usr/share/mysql-zrm/plugins/ssh-copy.pl --mysqlhotcopy --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" --quiet mysql "/data/mysql-zrm/aux/20140818154841" > /data/mysql-zrm/8FH0BhVmVf 2>&1
    Mon Aug 18 15:48:45 2014: aux:backup:INFO: raw-databases=mysql
    Mon Aug 18 15:48:45 2014: aux:backup:INFO: PHASE END: Creating raw backup
    Mon Aug 18 15:48:45 2014: aux:backup:INFO: PHASE START: Creating logical backup
    Mon Aug 18 15:48:45 2014: aux:backup:WARNING: The database(s) cmail_db_prod1 performance_schema ums_db will be backed up in logical mode since they contain tables that use a transactional engine.
    Mon Aug 18 15:48:45 2014: aux:backup:INFO: Command used for logical backup is mysqldump --opt --extended-insert --create-options --default-character-set=utf8 --single-transaction --user="zrm" --password="*****" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock"bash -c 'mysqldump --opt --extended-insert --create-options --default-character-set=utf8 --single-transaction --user="zrm" --password="p4ss" --host="dione" --port="3307" --socket="/opt/md3/mysql/auxdb/auxdb.sock" --databases cmail_db_prod1 performance_schema ums_db > "/data/mysql-zrm/aux/20140818154841/backup.sql" 2>>/data/mysql-zrm/ou0FQqiYrU;echo ${PIPESTATUS[@]} > /data/mysql-zrm/mYEBbvu7yu 2>>/data/mysql-zrm/ou0FQqiYrU'
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: Command exit status is : 2
    Mon Aug 18 15:48:55 2014: aux:backup:WARNING: Could not read file /data/mysql-zrm/mYEBbvu7yu
    Mon Aug 18 15:48:55 2014: aux:backup:ERROR: mysqldump did not succeed.
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: PHASE START: Cleanup
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: backup-status=Backup failed
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: Backup failed
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: mailing file /data/mysql-zrm/QjAsYE9lFd
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: mail command is cat "/data/mysql-zrm/QjAsYE9lFd"|mail -s "[ZRM for MySQL Report] ERROR during backup of backup-set aux" zrm-backups
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: PHASE END: Cleanup
    Mon Aug 18 15:48:55 2014: aux:backup:INFO: END OF BACKUP

  2. #2
    Join Date
    Nov 2014
    Posts
    1

    Default

    Is there any way to set from header in notifaction mails ?


    ___________________
    We provide guarantee to pass [url=http://www.keiseruniversity.edu/]Keiser University[/url] with online exam training [url=http://www.pass4sure.co.uk/312-49v8.html]pass4sure.co.uk[/url] and [url=http://www.arm.com/]ARM[/url], you can also get best quality [url=http://www.stanford.edu/]Stanford University[/url] along with [url=http://www.youtube.com/user/vmwaretv]Youtube[/url] for your guaranteed success.
    Last edited by Sxandy; November 26th, 2014 at 10:34 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •