[Ve] xfs: test xfs_metadump for leaked strings

Commit Message

Eric Sandeen Aug. 2, 2017, 3:45 a.m. UTC

xfs_metadump by default sanitizes the image so that all strings
longer than 5 chars are obfusccated, and all stale data in metadata
blocks (i.e. unused/unwritten data) is zeroed out.  We didn't have
a test for this, though, so this does it.

It patterns 256M of the scratch device, then uses djwong's
populate infrastructure to write all types of metadata,
metadumps & mdrestores it, then looks for either the leaked
pre-pattern or any leaked strings or filenames.

The strings we look for are, unfortunately, a bit ad-hoc based on
what is currently used in the populate routines.

Signed-off-by: Eric Sandeen <[e-mail]>
---

V2: include .out file, minor whitespace & typo edits
V3: remove populate debug stuff that snuck in

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Comments

Darrick J. Wong Aug. 2, 2017, 5:29 p.m. UTC | #1

On Tue, Aug 01, 2017 at 10:45:06PM -0500, Eric Sandeen wrote:
> xfs_metadump by default sanitizes the image so that all strings
> longer than 5 chars are obfusccated, and all stale data in metadata
> blocks (i.e. unused/unwritten data) is zeroed out.  We didn't have
> a test for this, though, so this does it.
> 
> It patterns 256M of the scratch device, then uses djwong's
> populate infrastructure to write all types of metadata,
> metadumps & mdrestores it, then looks for either the leaked
> pre-pattern or any leaked strings or filenames.
> 
> The strings we look for are, unfortunately, a bit ad-hoc based on
> what is currently used in the populate routines.
> 
> Signed-off-by: Eric Sandeen <[e-mail]>
> ---
> 
> V2: include .out file, minor whitespace & typo edits
> V3: remove populate debug stuff that snuck in
> 
> diff --git a/tests/xfs/425 b/tests/xfs/425
> new file mode 100755
> index 0000000..dd11236
> --- /dev/null
> +++ b/tests/xfs/425
> @@ -0,0 +1,121 @@
> +#! /bin/bash
> +# FS QA Test 425
> +#
> +# Look for stale data leaks in an xfs_metadump
> +#
> +# If this fails, get the byte offset of the leaked strings
> +# which are found, then on the restored image in $TEST_DIR,
> +# do:
> +#
> +# xfs_db> blockget -n
> +# xfs_db> convert byte $BYTE daddr
> +# $RESULT
> +# xfs_db> daddr $RESULT
> +# xfs_db> blockuse -n
> +#
> +# to see information about the metadata block which contains the
> +# leaked strings
> +#
> +#-----------------------------------------------------------------------
> +# Copyright (c) 2017 Red Hat, Inc.  All Rights Reserved.
> +#
> +# This program is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU General Public License as
> +# published by the Free Software Foundation.
> +#
> +# This program is distributed in the hope that it would be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, write the Free Software Foundation,
> +# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
> +#-----------------------------------------------------------------------
> +#
> +
> +seq=`basename $0`
> +seqres=$RESULT_DIR/$seq
> +echo "QA output created by $seq"
> +
> +here=`pwd`
> +tmp=/tmp/$$
> +status=1	# failure is the default!
> +trap "_cleanup; exit \$status" 0 1 2 3 15
> +
> +_cleanup()
> +{
> +	cd /
> +	rm -f $tmp.*
> +}
> +
> +# get standard environment, filters and checks
> +. ./common/rc
> +. ./common/filter
> +. ./common/populate
> +
> +# remove previous $seqres.full before test
> +rm -f $seqres.full
> +
> +# real QA test starts here
> +
> +# Modify as appropriate.
> +_supported_fs xfs
> +_supported_os Linux
> +_require_test
> +_require_scratch
> +_require_populate_commands
> +
> +METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
> +MDRESTORE_FILE="${TEST_DIR}/${seq}_mdrestore"
> +
> +echo "Silence is golden"
> +
> +# Pattern the scratch disk, mkfs, and restore.
> +$XFS_IO_PROG -d -c "pwrite 0 256M" $SCRATCH_DEV > $seqres.full 2>&1

pwrite -b 1m to speed this up a bit...

> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
> +_scratch_populate nofill >> $seqres.full 2>&1
> +
> +# populate unmounts the fs for us
> +
> +_scratch_metadump $METADUMP_FILE
> +xfs_mdrestore $METADUMP_FILE $MDRESTORE_FILE
> +
> +# Grep for stale data (leaked cd cd pattern) or strings
> +# from populate routine
> +
> +hexdump -C $MDRESTORE_FILE | grep \

_require_command /usr/bin/hexdump "hexdump" ?

> +"cd cd cd cd\| \
> [phone]\|\
> [phone]\|\
> [phone]\|\
> [phone]\|\
> [phone]\|\
> [phone]\|\
> [phone]\|\
> [phone]\|\
> +dummy\|\
> +S_IF\|\
> +FMT_\|\
> +INLINE\|\
> +BLOCK\|\
> +LEAF\|\
> +NODE\|\
> +BTREE\|\
> +LOCAL\|\
> +EXTENTS\|\
> +REMOTE\|\
> +ATTR\|\
> +SYSTEM\|\
> +TRUSTED\|\
> +SECURITY\|\
> +attrvalfile\|\
> +unused\|\
> +BNOBT\|\
> +RMAPBT\|\
> +RTRMAPBT\|\
> +REFCOUNTBT" && echo "Leaked data found; see comments in test to debug"

/me wonders if this grep pattern ought to be some kind of _filter
function in common/populate?  That way if the xfs populate command adds
more weird strings, the search pattern is right there in the rc file,
not buried in tests/xfs/ somewhere.

OTOH a good counterargument is that this is the only test that cares
about that, so why make it common code?  <sigh>

--D

> +
> +# success, all done
> +status=0
> +exit
> diff --git a/tests/xfs/425.out b/tests/xfs/425.out
> new file mode 100644
> index 0000000..7a9714b
> --- /dev/null
> +++ b/tests/xfs/425.out
> @@ -0,0 +1,2 @@
> +QA output created by 425
> +Silence is golden
> diff --git a/tests/xfs/group b/tests/xfs/group
> index 185487d..d0d26ee 100644
> --- a/tests/xfs/group
> +++ b/tests/xfs/group
> @@ -422,3 +422,4 @@
>  422 dangerous_scrub dangerous_online_repair
>  423 dangerous_scrub
>  424 auto quick dump
> +425 auto metadata
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Eric Sandeen Aug. 4, 2017, 2:21 a.m. UTC | #2

On 8/2/17 12:29 PM, Darrick J. Wong wrote:
> On Tue, Aug 01, 2017 at 10:45:06PM -0500, Eric Sandeen wrote:
>> xfs_metadump by default sanitizes the image so that all strings
>> longer than 5 chars are obfusccated, and all stale data in metadata
>> blocks (i.e. unused/unwritten data) is zeroed out.  We didn't have
>> a test for this, though, so this does it.
>>
>> It patterns 256M of the scratch device, then uses djwong's
>> populate infrastructure to write all types of metadata,
>> metadumps & mdrestores it, then looks for either the leaked
>> pre-pattern or any leaked strings or filenames.
>>
>> The strings we look for are, unfortunately, a bit ad-hoc based on
>> what is currently used in the populate routines.
>>
>> Signed-off-by: Eric Sandeen <[e-mail]>
>> ---
>>
>> V2: include .out file, minor whitespace & typo edits
>> V3: remove populate debug stuff that snuck in


>> +echo "Silence is golden"
>> +
>> +# Pattern the scratch disk, mkfs, and restore.
>> +$XFS_IO_PROG -d -c "pwrite 0 256M" $SCRATCH_DEV > $seqres.full 2>&1
> 
> pwrite -b 1m to speed this up a bit...

ok
 
>> +_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
>> +_scratch_populate nofill >> $seqres.full 2>&1
>> +
>> +# populate unmounts the fs for us
>> +
>> +_scratch_metadump $METADUMP_FILE
>> +xfs_mdrestore $METADUMP_FILE $MDRESTORE_FILE
>> +
>> +# Grep for stale data (leaked cd cd pattern) or strings
>> +# from populate routine
>> +
>> +hexdump -C $MDRESTORE_FILE | grep \
> 
> _require_command /usr/bin/hexdump "hexdump" ?

sure? other tests don't bother, but ok.

> 
>> +"cd cd cd cd\| \
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> [phone]\|\
>> +dummy\|\
>> +S_IF\|\
>> +FMT_\|\
>> +INLINE\|\
>> +BLOCK\|\
>> +LEAF\|\
>> +NODE\|\
>> +BTREE\|\
>> +LOCAL\|\
>> +EXTENTS\|\
>> +REMOTE\|\
>> +ATTR\|\
>> +SYSTEM\|\
>> +TRUSTED\|\
>> +SECURITY\|\
>> +attrvalfile\|\
>> +unused\|\
>> +BNOBT\|\
>> +RMAPBT\|\
>> +RTRMAPBT\|\
>> +REFCOUNTBT" && echo "Leaked data found; see comments in test to debug"
> 
> /me wonders if this grep pattern ought to be some kind of _filter
> function in common/populate?  That way if the xfs populate command adds
> more weird strings, the search pattern is right there in the rc file,
> not buried in tests/xfs/ somewhere.
> 
> OTOH a good counterargument is that this is the only test that cares
> about that, so why make it common code?  <sigh>

um, I totally agree with both your arguments... :/

i'll put it in populate I guess.

> 
> --D
--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/tests/xfs/425 b/tests/xfs/425
new file mode 100755
index 0000000..dd11236
--- /dev/null
+++ b/tests/xfs/425
@@ -0,0 +1,121 @@ 
+#! /bin/bash
+# FS QA Test 425
+#
+# Look for stale data leaks in an xfs_metadump
+#
+# If this fails, get the byte offset of the leaked strings
+# which are found, then on the restored image in $TEST_DIR,
+# do:
+#
+# xfs_db> blockget -n
+# xfs_db> convert byte $BYTE daddr
+# $RESULT
+# xfs_db> daddr $RESULT
+# xfs_db> blockuse -n
+#
+# to see information about the metadata block which contains the
+# leaked strings
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2017 Red Hat, Inc.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1	# failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+	cd /
+	rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+. ./common/populate
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+# Modify as appropriate.
+_supported_fs xfs
+_supported_os Linux
+_require_test
+_require_scratch
+_require_populate_commands
+
+METADUMP_FILE="${TEST_DIR}/${seq}_metadump"
+MDRESTORE_FILE="${TEST_DIR}/${seq}_mdrestore"
+
+echo "Silence is golden"
+
+# Pattern the scratch disk, mkfs, and restore.
+$XFS_IO_PROG -d -c "pwrite 0 256M" $SCRATCH_DEV > $seqres.full 2>&1
+_scratch_mkfs_sized $((256 * 1024 * 1024)) >> $seqres.full 2>&1
+_scratch_populate nofill >> $seqres.full 2>&1
+
+# populate unmounts the fs for us
+
+_scratch_metadump $METADUMP_FILE
+xfs_mdrestore $METADUMP_FILE $MDRESTORE_FILE
+
+# Grep for stale data (leaked cd cd pattern) or strings
+# from populate routine
+
+hexdump -C $MDRESTORE_FILE | grep \
+"cd cd cd cd\| \
[phone]\|\
[phone]\|\
[phone]\|\
[phone]\|\
[phone]\|\
[phone]\|\
[phone]\|\
[phone]\|\
+dummy\|\
+S_IF\|\
+FMT_\|\
+INLINE\|\
+BLOCK\|\
+LEAF\|\
+NODE\|\
+BTREE\|\
+LOCAL\|\
+EXTENTS\|\
+REMOTE\|\
+ATTR\|\
+SYSTEM\|\
+TRUSTED\|\
+SECURITY\|\
+attrvalfile\|\
+unused\|\
+BNOBT\|\
+RMAPBT\|\
+RTRMAPBT\|\
+REFCOUNTBT" && echo "Leaked data found; see comments in test to debug"
+
+# success, all done
+status=0
+exit
diff --git a/tests/xfs/425.out b/tests/xfs/425.out
new file mode 100644
index 0000000..7a9714b
--- /dev/null
+++ b/tests/xfs/425.out
@@ -0,0 +1,2 @@ 
+QA output created by 425
+Silence is golden
diff --git a/tests/xfs/group b/tests/xfs/group
index 185487d..d0d26ee 100644
--- a/tests/xfs/group
+++ b/tests/xfs/group
@@ -422,3 +422,4 @@ 
 422 dangerous_scrub dangerous_online_repair
 423 dangerous_scrub
 424 auto quick dump
+425 auto metadata