PDA

View Full Version : Testing 2.6.1b1 witn ZFS/FreeBSD



glowkrantz
December 16th, 2008, 12:20 PM
I am testing the 2.6.1b1 series and have found one thing and have one question.

The thing I have found is that the Zfs.pm module use the Solaris specific output of df to map mountpoint to zfs file system. This I have fixed by a oneliner script that I use with the amzfs-snapshot and amzfs-sendrecv applications:

# cat /usr/local/bin/amdf
#!/bin/sh

/bin/df $* | /usr/bin/awk '{ if ($1 != "Filesystem" ) printf "%s (%s ): %s blocks\n", $6, $1, $2 }'

The problem I have is that the snapshot name is somehow empty, so I get

1229457426.752518: Amzfs_sendrecv: pid 54977 ruid 2 euid 2 version 2.6.1b1: start at Tue Dec 16 20:57:06 2008
1229457426.778462: Amzfs_sendrecv: application: Amzfs_sendrecv

1229457426.778537: Amzfs_sendrecv: command: selfcheck
1229457426.778647: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint /var
1229457426.793497: Amzfs_sendrecv: running: /usr/local/bin/amdf /var
1229457426.799345: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint system/var|
1229457426.813772: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.828074: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.842604: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.857010: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.871407: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.885707: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.900640: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|
1229457426.915266: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/var@|

and so on, the destroy command is looping in some way.
This is with the 20081115 drop.

Any pointers where I should look for to find out what's going on?

/glz

dustin
December 16th, 2008, 12:31 PM
I don't know much about the amzfs scripts, but I'm happy to hear that you're testing them.

I do know that it would be great if you could put together a patch to help parse either format of 'df' output.

martineau
December 16th, 2008, 01:10 PM
Try the attached patch, it should avoid the recursive loop on error.

glowkrantz
December 16th, 2008, 01:47 PM
The patch didn't fix the loop, I still get a loop if it exits the Zfs.pm on line 158:
if ($zmountpoint ne $self->{mountpoint}) {
debug "2-1";
$self->print_to_server_and_die($action,
"mountpoint from 'df' ($self->{mountpoint}) and 'zfs list' ($zmountpoint) differ",
$Amanda::Script_App::ERROR);
}

I found two problems that result in the failure, on is OK but needs to be logged better: if the zfs command fails with insufficient privilege

The second problem is that we fail on legacy mount points, they must be specified as pool/fs, you can't use mountpoint. This cases the error at line 158 in Zfs.pm

/glz

martineau
December 16th, 2008, 02:10 PM
Sorry, try this patch instead.

A mountpoint should work, why do you changed the df command?

amcheck print the complete stderr of the zfs command, if zfs print the permission error on stderr then you should get it.

glowkrantz
December 16th, 2008, 02:31 PM
This patch was good. Thanks!

This is the response now when I use a legacy mountpoint:

Amanda Backup Client Hosts Check
--------------------------------
ERROR: skade: mountpoint from 'df' (/) and 'zfs list' (legacy) differ
ERROR: skade: cannot open 'system/boot@': invalid dataset name
ERROR: skade: unknown response:
ERROR: skade: Application 'amzfs-sendrecv': Use of uninitialized value in concatenation (.) or string at /usr/local/lib/perl5/site_perl/5.8.8/Amanda/Application/Zfs.pm line 218.
ERROR: skade: Application 'amzfs-sendrecv': exited with status 1
Client check: 5 hosts checked in 13.987 seconds. 5 problems found.

It's the root system:
# df -h /
Filesystem Size Used Avail Capacity Mounted on
system/boot 42G 22M 42G 0% /

Debug file content:
1229466219.421667: Amzfs_sendrecv: pid 60899 ruid 2 euid 2 version 2.6.1b1: start at Tue Dec 16 23:23:39 2008
1229466219.455313: Amzfs_sendrecv: application: Amzfs_sendrecv

1229466219.455388: Amzfs_sendrecv: command: selfcheck
1229466219.455546: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint /
1229466219.496721: Amzfs_sendrecv: running: /usr/local/bin/amdf /
1229466219.508363: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint system/boot|
1229466219.571173: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/boot@|

and the mountpoint property:
# zfs get -H -o value mountpoint system/boot
legacy

So this fails on line 158 as
$zmountpoint ne $self->{mountpoint}
legacy ne / as it says in the report above.

Hope this helps.

/glz

martineau
December 17th, 2008, 05:14 AM
What is a 'legacy mountpoint'?
Is it possible for 'zfs' to return a real mountpoint?
What is the output of:
/sbin/zfs get -H -o value mountpoint legacy
/sbin/zfs get -H -o value mountpoint

glowkrantz
December 17th, 2008, 08:15 AM
A legacy mountpoint is when you use a zfs file system like any other filesystem, i.e. it's mountpoint is in fstab

# cat /etc/fstab
# Device Mountpoint FStype Options Dump Pass#
system/boot / zfs rw,noatime 0 0
/dev/ufs/root /bootdir ufs rw,noatime 1 1
/bootdir/boot /boot nullfs rw,noatime 0 0

There may be times you want to have the mount -a command mount some zfs file systems. In the current case, I have the boot file system on ufs as zfsboot is not really finished.

# zfs get -H -o value mountpoint system/boot
legacy

# zfs get -H -o value mountpoint legacy
cannot open 'legacy': dataset does not exist

# zfs get -H -o value mountpoint
none
legacy
-
/usr
/usr/local
/usr/local/jails
/usr/local/pgsql
/usr/obj
/usr/ports
/usr/src
/usr/tmp
/var
/var/log
/var/tmp

I think I have a patch for Zfs.pm that will fix it. That plus the different df outputs. I will test tonight and submit something.

/glz

glowkrantz
December 17th, 2008, 09:13 AM
Here is the result of a check with legacy mount point:
1229533407.607015: Amzfs_sendrecv: pid 73227 ruid 2 euid 2 version 2.6.1b1: start at Wed Dec 17 18:03:27 2008
1229533407.660584: Amzfs_sendrecv: application: Amzfs_sendrecv

1229533407.660666: Amzfs_sendrecv: command: selfcheck
1229533407.660828: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint /
1229533407.706900: Amzfs_sendrecv: running: /usr/local/bin/amdf /
1229533407.717896: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint system/boot|
1229533407.763988: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs snapshot system/boot@_
1229533408.107545: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/boot@_|


Attached is the patch for Zfs.pm supporting both this and the differences in df.

1229533716.579981: Amzfs_sendrecv: pid 73364 ruid 2 euid 2 version 2.6.1b1: start at Wed Dec 17 18:08:36 2008
1229533716.622152: Amzfs_sendrecv: application: Amzfs_sendrecv

1229533716.622251: Amzfs_sendrecv: command: selfcheck
1229533716.622421: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint /
1229533716.684710: Amzfs_sendrecv: running: /bin/df /
1229533716.691038: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs get -H -o value mountpoint system/boot|
1229533716.739164: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs snapshot system/boot@_
1229533717.083238: Amzfs_sendrecv: running: /usr/local/bin/sudo /sbin/zfs destroy system/boot@_|


/glz

martineau
December 17th, 2008, 01:45 PM
Thanks for the patch, I committed it, can you try latest SVN tree?

glowkrantz
December 17th, 2008, 02:22 PM
autogen!!!

/glz

dustin
December 17th, 2008, 06:06 PM
The full list of development dependencies is here:
http://wiki.zmanda.com/index.php/Minimum_Versions

glowkrantz
December 18th, 2008, 12:31 PM
I did build the svn version and also installed the 20081218 drop, both works OK in the tests that failed in the earlier versions. I will let 20081218 run the dump tonight to see how it works and report back.

/glz

glowkrantz
December 19th, 2008, 01:52 AM
The 20081218 works fine with dump and zfs send on FreeBSD STABLE and CURRENT.