PDA

View Full Version : LVM snapshot restore failure ("Table is full")



czander
December 5th, 2008, 03:30 PM
I'm getting the following error when trying to restore from an lvm snapshot created using the mysql-zrm utility.

The following is the command I'm running:

mysql-zrm --action restore --source-directory /opt/mysql/20081205050022/

The following is my output:

restore:INFO: ZRM for MySQL Community Edition - version 1.2
BackupSet1:restore:INFO: Mail address: clint@vmix.com is ok
BackupSet1:restore:INFO: ZRM Temporary configuration file = /etc/mysql-zrm/BackupSet1/tmpapegZ.conf
BackupSet1:restore:INFO: {
BackupSet1:restore:INFO: verbose=1
BackupSet1:restore:INFO: retention-policy=7D
BackupSet1:restore:INFO: tmpdir=/opt
BackupSet1:restore:INFO: mysql-binpath=/usr/local/bin
BackupSet1:restore:INFO: default-character-set=utf8
BackupSet1:restore:INFO: all-databases=1
BackupSet1:restore:INFO: destination=/logs/mysql-zrm
BackupSet1:restore:INFO: source-directory=/opt/mysql/20081205050022
BackupSet1:restore:INFO: routines=1
BackupSet1:restore:INFO: password=******
BackupSet1:restore:INFO: backup-mode=raw
BackupSet1:restore:INFO: snapshot-plugin=/usr/share/mysql-zrm/plugins/lvm-snapshot.pl
BackupSet1:restore:INFO: compress-plugin=/usr/bin/gzip
BackupSet1:restore:INFO: user=root
BackupSet1:restore:INFO: backup-level=0
BackupSet1:restore:INFO: lvm-snapshot=100M
BackupSet1:restore:INFO: mailto=********
BackupSet1:restore:INFO: port=3306
BackupSet1:restore:INFO: host=localhost
BackupSet1:restore:INFO: socket=/var/lib/mysql/mysql.sock
BackupSet1:restore:INFO: html-report-directory=/logs/mysql-zrm/reports
BackupSet1:restore:INFO: compress=/usr/bin/gzip
BackupSet1:restore:INFO: mysql-binlog-path=/logs
BackupSet1:restore:INFO: }
BackupSet1:restore:INFO: Getting mysql variables
BackupSet1:restore:INFO: "/usr/local/bin"/mysqladmin --user="root" --password="*****" --host="localhost" --port="3306" --socket="/var/lib/mysql/mysql.sock" variables
BackupSet1:restore:INFO: datadir is /opt/mysql/
BackupSet1:restore:INFO: mysql_version is 5.0.45-log
BackupSet1:restore:INFO: log_bin=OFF
BackupSet1:restore:WARNING: Binary logging is off.
BackupSet1:restore:INFO: InnoDB data file are /opt/mysql/ibdata1
BackupSet1:restore:INFO: InnoDB log dir is /opt/mysql
BackupSet1:restore:INFO: Command used is 'cat "/opt/mysql/20081205050022/backup-data" | "/usr/bin/gzip" -d | tar --same-owner -xpsC "/opt/mysql/20081205050022" 2>/opt/gzWOs446Ns'



BackupSet1:restore:INFO: restoring using command "/usr/local/bin"/mysql --user="****" --password="*****" --host="localhost" --port="3306" --socket="/var/lib/mysql/mysql.sock" -e "set character_set_client=utf8;set character_set_connection=utf8;set character_set_database=utf8;set character_set_results=utf8;set character_set_server=utf8;source /opt/mnZfZNtfoX;"


ERROR 1114 (HY000) at line 20724 in file: '/opt/mnZfZNtfoX': The table 'play_location_ids' is full
BackupSet1:restore:ERROR: Restore from logical backup failed
BackupSet1:restore:INFO: Removing all of the uncompressed/unencrypted data
BackupSet1:restore:ERROR: Restore failed




The backup is getting about 80-90% complete then I get this 'table xxx is full' error. These tables are InnoDB and don't seem to be getting close to any filesize limitations related to the file system. On a previous run of this restoration job, I dropped the suspect table and ran another backup. When restoring off the second backup, I ended up with an error message reporting the same problem with a different table this time. The disk has space for the operation, so I'm not seeing what the problem is.

Another thing to note, is that the mysql is configured to store InnoDB tables to their own files, instead of the ibdata file. Not sure if that is related.

Any input would be greatly appreciated.

Thanks,

C

gsat
December 5th, 2008, 06:56 PM
from earlier posts and mysql documentation, You simply have filled up all the available memory based on your DataMemory setting. IF you have more ram available in each of your datanodes, you need to increase these settings to accommodate more data.


check config.ini and increase DataMemory and IndexMemory and try again..

NoOfReplicas=2 # Number of replicas
DataMemory=2500M # How much memory to allocate for data storage IndexMemory=100M # How much memory to allocate for index storage

czander
December 8th, 2008, 09:59 AM
What you are referring to is specific the the NDB storage engine and the config settings associated with that storage engine. The tables that are failing in this regard are all InnoDB based.

While I really appreciate the response, I'm not sure it is valid in this case.

If you happened to know of any related issues to InnoDB an why this would happen, any input would be greatly appreciated.

Thanks again,

C

paddy
December 11th, 2008, 01:00 PM
/opt/mnZfZNtfoX;"


ERROR 1114 (HY000) at line 20724 in file: '/opt/mnZfZNtfoX': The table 'play_location_ids' is full
BackupSet1:restore:ERROR: Restore from logical backup failed
BackupSet1:restore:INFO: Removing all of the uncompressed/unencrypted data
BackupSet1:restore:ERROR: Restore failed



This is an error message that is difficult to figure out what happened. One of the reasons for "the table is full" message is related to number of open transactions. See http://bugs.mysql.com/bug.php?id=26590

Are you encountering this problem?

thanks,
paddy