xfs_db: add -R option

Commit Message

[e-mail] May 4, 2018, 2:02 p.m. UTC

From: Hal Pomeranz <[e-mail]>

-R is similar to -r, but allows for blockget on a mounted file system
or "dirty" file system image with pending log entries.  This makes
xfs_db more useful for forensics where we are often dealing with these
types of images.  Flushing log entries to disk by mounting/unmounting
the file system would allow us to use blockget, but would make changes
to the file system state which are not desirable in forensics
contexts.

Signed-off-by: Hal Pomeranz <[e-mail]>
---
 db/init.c         | 7 +++++--
 db/init.h         | 1 +
 db/sb.c           | 2 +-
 man/man8/xfs_db.8 | 8 ++++++++
 4 files changed, 15 insertions(+), 3 deletions(-)

Comments

Darrick J. Wong May 4, 2018, 3:32 p.m. UTC | #1

On Fri, May 04, 2018 at 09:02:44AM -0500, [e-mail] wrote:
> From: Hal Pomeranz <[e-mail]>
> 
> -R is similar to -r, but allows for blockget on a mounted file system
> or "dirty" file system image with pending log entries.  This makes
> xfs_db more useful for forensics where we are often dealing with these
> types of images.  Flushing log entries to disk by mounting/unmounting
> the file system would allow us to use blockget, but would make changes
> to the file system state which are not desirable in forensics
> contexts.
> 
> Signed-off-by: Hal Pomeranz <[e-mail]>
> ---
>  db/init.c         | 7 +++++--
>  db/init.h         | 1 +
>  db/sb.c           | 2 +-
>  man/man8/xfs_db.8 | 8 ++++++++
>  4 files changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/db/init.c b/db/init.c
> index 29fc344..96a0e78 100644
> --- a/db/init.c
> +++ b/db/init.c
> @@ -36,6 +36,7 @@ int			blkbb;
>  int			exitcode;
>  int			expert_mode;
>  int			force;
> +int                     ignore_dirty;

tabs vs. spaces...

>  struct xfs_mount	xmount;
>  struct xfs_mount	*mp;
>  struct xlog		xlog;
> @@ -46,7 +47,7 @@ static void
>  usage(void)
>  {
>  	fprintf(stderr, _(
> -		"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n"
> +		"Usage: %s [-ifFrRxV] [-p prog] [-l logdev] [-c cmd]... device\n"
>  		), progname);
>  	exit(1);
>  }
> @@ -66,7 +67,7 @@ init(
>  	textdomain(PACKAGE);
>  
>  	progname = basename(argv[0]);
> -	while ((c = getopt(argc, argv, "c:fFip:rxVl:")) != EOF) {
> +	while ((c = getopt(argc, argv, "c:fFip:rRxVl:")) != EOF) {
>  		switch (c) {
>  		case 'c':
>  			cmdline = xrealloc(cmdline, (ncmdline+1)*sizeof(char*));
> @@ -84,6 +85,8 @@ init(
>  		case 'p':
>  			progname = optarg;
>  			break;
> +		case 'R':
> +		        ignore_dirty = 1;

In general, fall through cases should have a comment to document that
explicitly.

/* fall through */

>  		case 'r':
>  			x.isreadonly = LIBXFS_ISREADONLY;
>  			break;
> diff --git a/db/init.h b/db/init.h
> index b09389e..f6bfda9 100644
> --- a/db/init.h
> +++ b/db/init.h
> @@ -20,6 +20,7 @@ extern char		*fsdevice;
>  extern int		blkbb;
>  extern int		exitcode;
>  extern int		expert_mode;
> +extern int              ignore_dirty;
>  extern xfs_mount_t	*mp;
>  extern libxfs_init_t	x;
>  extern xfs_agnumber_t	cur_agno;
> diff --git a/db/sb.c b/db/sb.c
> index c7fbfd6..4c04d79 100644
> --- a/db/sb.c
> +++ b/db/sb.c
> @@ -251,7 +251,7 @@ sb_logcheck(void)
>  	if (dirty == -1) {
>  		dbprintf(_("ERROR: cannot find log head/tail, run xfs_repair\n"));
>  		return 0;
> -	} else if (dirty == 1) {
> +	} else if (dirty == 1 && ignore_dirty == 0) {
>  		dbprintf(_(
>  "ERROR: The filesystem has valuable metadata changes in a log which needs to\n"
>  "be replayed.  Mount the filesystem to replay the log, and unmount it before\n"

Why skip the warning?  I think xfs_db should always warn on a dirty log,
but perhaps we could relax the 'return 0' at the end of this hunk if the
fs was opened readonly?

i.e.

dbprintf(_("ERROR: The filesystem has..."));
return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;

which would also make adding the -R option unnecessary.

--D

> diff --git a/man/man8/xfs_db.8 b/man/man8/xfs_db.8
> index 524b1ef..89be1aa 100644
> --- a/man/man8/xfs_db.8
> +++ b/man/man8/xfs_db.8
> @@ -90,6 +90,14 @@ It is only necessary to omit this flag if a command that changes data
>  .RB ( write ", " blocktrash ", " crc )
>  is to be used.
>  .TP
> +.B -R
> +Like
> +.B -r
> +but allows the 
> +.B blockget
> +command even if the file system is "dirty" (unreconciled transactions
> +in the log).
> +.TP
>  .B \-x
>  Specifies expert mode.
>  This enables the
> -- 
> 1.8.3.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[e-mail] May 7, 2018, 11:35 a.m. UTC | #2

> Why skip the warning?  I think xfs_db should always warn on a dirty log,
> but perhaps we could relax the 'return 0' at the end of this hunk if the
> fs was opened readonly?
> 
> i.e.
> 
> dbprintf(_("ERROR: The filesystem has..."));
> return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;
> 
> which would also make adding the -R option unnecessary.

I like this solution very much. I'll send in this patch in a
separate email.

Hal Pomeranz
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[e-mail] May 7, 2018, 12:56 p.m. UTC | #3

Darrick's suggested patch to sb_logcheck() is the way to accomplish
what I want with the minimum amount of code changes.  But as I look at
the code, I'm not sure doing it the way Darrick suggests actually
expresses what we're trying to accomplish.

Let's state the problem as "Recognize that the log is dirty but allow
blockget to proceed if '-r' is used". If you accept that problem
statement, then sb_logcheck() should just tell the caller whether the
log is dirty or not. It's up to the caller to decide what to do with
that information-- in this case the init() function in db/check.c.

However, right now sb_logcheck() returns 0 for the case where 
there's a fatal error (like not being able to find the log) and for
the log being dirty. init() would need to distinguish between those
two cases and act appropriately. That means changing the return
semantics of sb_logcheck() to something like -1 on error, 0 on dirty,
and 1 on clean. 

Note that sb_logcheck() is also called by sb_logzero() and
that code would have to be tweaked to handle the new return values.

So the end result is more code changes to do it this way, but I
feel like it's a more "correct" fix. I have patches created for
both scenarios-- does anybody have a strong opinion about which way
to proceed?

--Hal


On Mon May 07 06:35, [e-mail] wrote:
> > Why skip the warning?  I think xfs_db should always warn on a dirty log,
> > but perhaps we could relax the 'return 0' at the end of this hunk if the
> > fs was opened readonly?
> > 
> > i.e.
> > 
> > dbprintf(_("ERROR: The filesystem has..."));
> > return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;
> > 
> > which would also make adding the -R option unnecessary.
> 
> I like this solution very much. I'll send in this patch in a
> separate email.
> 
> Hal Pomeranz

--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Darrick J. Wong May 8, 2018, 1:21 a.m. UTC | #4

On Mon, May 07, 2018 at 07:56:21AM -0500, [e-mail] wrote:
> Darrick's suggested patch to sb_logcheck() is the way to accomplish
> what I want with the minimum amount of code changes.  But as I look at
> the code, I'm not sure doing it the way Darrick suggests actually
> expresses what we're trying to accomplish.
> 
> Let's state the problem as "Recognize that the log is dirty but allow
> blockget to proceed if '-r' is used". If you accept that problem
> statement, then sb_logcheck() should just tell the caller whether the
> log is dirty or not. It's up to the caller to decide what to do with
> that information-- in this case the init() function in db/check.c.
> 
> However, right now sb_logcheck() returns 0 for the case where 
> there's a fatal error (like not being able to find the log) and for
> the log being dirty. init() would need to distinguish between those
> two cases and act appropriately. That means changing the return
> semantics of sb_logcheck() to something like -1 on error, 0 on dirty,
> and 1 on clean. 
> 
> Note that sb_logcheck() is also called by sb_logzero() and
> that code would have to be tweaked to handle the new return values.

Ultimately I defer to Eric on this one, but I don't see how sb_logzero
would run to completion in readonly mode, since the only caller of it is
uuid_f, which bails out if we're in readonly mode.

(Not to mention the buffer writes should fail in ro mode).

--D

> 
> So the end result is more code changes to do it this way, but I
> feel like it's a more "correct" fix. I have patches created for
> both scenarios-- does anybody have a strong opinion about which way
> to proceed?
> 
> --Hal
> 
> 
> On Mon May 07 06:35, [e-mail] wrote:
> > > Why skip the warning?  I think xfs_db should always warn on a dirty log,
> > > but perhaps we could relax the 'return 0' at the end of this hunk if the
> > > fs was opened readonly?
> > > 
> > > i.e.
> > > 
> > > dbprintf(_("ERROR: The filesystem has..."));
> > > return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;
> > > 
> > > which would also make adding the -R option unnecessary.
> > 
> > I like this solution very much. I'll send in this patch in a
> > separate email.
> > 
> > Hal Pomeranz
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
> the body of a message to [e-mail]
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Eric Sandeen May 8, 2018, 1:55 p.m. UTC | #5

On 5/4/18 9:02 AM, [e-mail] wrote:
> From: Hal Pomeranz <[e-mail]>
> 
> -R is similar to -r, but allows for blockget on a mounted file system
> or "dirty" file system image with pending log entries.

Hm, so this probably makes it possible that blockget will wander off into
inconsistencies and fail; there's a /reason/ for the check.  So if anything
this is probably best-effort.  I doubt you can rely on blockget even completing
without error, let alone being correct.  In that case, is it still useful
to you?

Of course if you simply desire to not perturb the image you're analyzing
you'd be operating on a copy in any case, and so that's presumably not the
issue here; I assume you wish to analyze the state of the filesystem before
any log changes are applied.

What sort of information do you hope to gather after running blockget without
a replay?

I guess what I'm getting at here is yes, we could short-circuit the check
in one way or another, but I want to be sure that it really would advance
your goals.

In any case:

Using -r seems a bit odd; blockget is already a readonly operation, so
using "-r" / readonly as a modifier seems out of place.  (The same warning
would come up for -c check).

Given that xfs_db is a command for experts, and forensics analysis on an
unreplayed filesystem even more so, what about modifying the logformat
command to allow zeroing of a dirty log with some type of force option,
which would then allow blkget to proceed?  (assuming you were working on
a copy, of course.)  That would make things pretty explicit.

Is that too convoluted?

Otherwise I might suggest adding a switch specific to the blockget_f
function, which would just skip the sb_logcheck() altogether.  That would
be more targeted, and wouldn't be some global switch which affects
every current and future caller of sb_logcheck.  A warning about how
the log is being ignored and results may be inconsistent could then
be added specifically to blockget_f.

-Eric

>  This makes
> xfs_db more useful for forensics where we are often dealing with these
> types of images.  Flushing log entries to disk by mounting/unmounting
> the file system would allow us to use blockget, but would make changes
> to the file system state which are not desirable in forensics
> contexts.
> 
> Signed-off-by: Hal Pomeranz <[e-mail]>
> ---
>  db/init.c         | 7 +++++--
>  db/init.h         | 1 +
>  db/sb.c           | 2 +-
>  man/man8/xfs_db.8 | 8 ++++++++
>  4 files changed, 15 insertions(+), 3 deletions(-)
> 
> diff --git a/db/init.c b/db/init.c
> index 29fc344..96a0e78 100644
> --- a/db/init.c
> +++ b/db/init.c
> @@ -36,6 +36,7 @@ int			blkbb;
>  int			exitcode;
>  int			expert_mode;
>  int			force;
> +int                     ignore_dirty;
>  struct xfs_mount	xmount;
>  struct xfs_mount	*mp;
>  struct xlog		xlog;
> @@ -46,7 +47,7 @@ static void
>  usage(void)
>  {
>  	fprintf(stderr, _(
> -		"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n"
> +		"Usage: %s [-ifFrRxV] [-p prog] [-l logdev] [-c cmd]... device\n"
>  		), progname);
>  	exit(1);
>  }
> @@ -66,7 +67,7 @@ init(
>  	textdomain(PACKAGE);
>  
>  	progname = basename(argv[0]);
> -	while ((c = getopt(argc, argv, "c:fFip:rxVl:")) != EOF) {
> +	while ((c = getopt(argc, argv, "c:fFip:rRxVl:")) != EOF) {
>  		switch (c) {
>  		case 'c':
>  			cmdline = xrealloc(cmdline, (ncmdline+1)*sizeof(char*));
> @@ -84,6 +85,8 @@ init(
>  		case 'p':
>  			progname = optarg;
>  			break;
> +		case 'R':
> +		        ignore_dirty = 1;
>  		case 'r':
>  			x.isreadonly = LIBXFS_ISREADONLY;
>  			break;
> diff --git a/db/init.h b/db/init.h
> index b09389e..f6bfda9 100644
> --- a/db/init.h
> +++ b/db/init.h
> @@ -20,6 +20,7 @@ extern char		*fsdevice;
>  extern int		blkbb;
>  extern int		exitcode;
>  extern int		expert_mode;
> +extern int              ignore_dirty;
>  extern xfs_mount_t	*mp;
>  extern libxfs_init_t	x;
>  extern xfs_agnumber_t	cur_agno;
> diff --git a/db/sb.c b/db/sb.c
> index c7fbfd6..4c04d79 100644
> --- a/db/sb.c
> +++ b/db/sb.c
> @@ -251,7 +251,7 @@ sb_logcheck(void)
>  	if (dirty == -1) {
>  		dbprintf(_("ERROR: cannot find log head/tail, run xfs_repair\n"));
>  		return 0;
> -	} else if (dirty == 1) {
> +	} else if (dirty == 1 && ignore_dirty == 0) {
>  		dbprintf(_(
>  "ERROR: The filesystem has valuable metadata changes in a log which needs to\n"
>  "be replayed.  Mount the filesystem to replay the log, and unmount it before\n"
> diff --git a/man/man8/xfs_db.8 b/man/man8/xfs_db.8
> index 524b1ef..89be1aa 100644
> --- a/man/man8/xfs_db.8
> +++ b/man/man8/xfs_db.8
> @@ -90,6 +90,14 @@ It is only necessary to omit this flag if a command that changes data
>  .RB ( write ", " blocktrash ", " crc )
>  is to be used.
>  .TP
> +.B -R
> +Like
> +.B -r
> +but allows the 
> +.B blockget
> +command even if the file system is "dirty" (unreconciled transactions
> +in the log).
> +.TP
>  .B \-x
>  Specifies expert mode.
>  This enables the
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [e-mail]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Patch

diff --git a/db/init.c b/db/init.c
index 29fc344..96a0e78 100644
--- a/db/init.c
+++ b/db/init.c
@@ -36,6 +36,7 @@  int			blkbb;
 int			exitcode;
 int			expert_mode;
 int			force;
+int                     ignore_dirty;
 struct xfs_mount	xmount;
 struct xfs_mount	*mp;
 struct xlog		xlog;
@@ -46,7 +47,7 @@  static void
 usage(void)
 {
 	fprintf(stderr, _(
-		"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n"
+		"Usage: %s [-ifFrRxV] [-p prog] [-l logdev] [-c cmd]... device\n"
 		), progname);
 	exit(1);
 }
@@ -66,7 +67,7 @@  init(
 	textdomain(PACKAGE);
 
 	progname = basename(argv[0]);
-	while ((c = getopt(argc, argv, "c:fFip:rxVl:")) != EOF) {
+	while ((c = getopt(argc, argv, "c:fFip:rRxVl:")) != EOF) {
 		switch (c) {
 		case 'c':
 			cmdline = xrealloc(cmdline, (ncmdline+1)*sizeof(char*));
@@ -84,6 +85,8 @@  init(
 		case 'p':
 			progname = optarg;
 			break;
+		case 'R':
+		        ignore_dirty = 1;
 		case 'r':
 			x.isreadonly = LIBXFS_ISREADONLY;
 			break;
diff --git a/db/init.h b/db/init.h
index b09389e..f6bfda9 100644
--- a/db/init.h
+++ b/db/init.h
@@ -20,6 +20,7 @@  extern char		*fsdevice;
 extern int		blkbb;
 extern int		exitcode;
 extern int		expert_mode;
+extern int              ignore_dirty;
 extern xfs_mount_t	*mp;
 extern libxfs_init_t	x;
 extern xfs_agnumber_t	cur_agno;
diff --git a/db/sb.c b/db/sb.c
index c7fbfd6..4c04d79 100644
--- a/db/sb.c
+++ b/db/sb.c
@@ -251,7 +251,7 @@  sb_logcheck(void)
 	if (dirty == -1) {
 		dbprintf(_("ERROR: cannot find log head/tail, run xfs_repair\n"));
 		return 0;
-	} else if (dirty == 1) {
+	} else if (dirty == 1 && ignore_dirty == 0) {
 		dbprintf(_(
 "ERROR: The filesystem has valuable metadata changes in a log which needs to\n"
 "be replayed.  Mount the filesystem to replay the log, and unmount it before\n"
diff --git a/man/man8/xfs_db.8 b/man/man8/xfs_db.8
index 524b1ef..89be1aa 100644
--- a/man/man8/xfs_db.8
+++ b/man/man8/xfs_db.8
@@ -90,6 +90,14 @@  It is only necessary to omit this flag if a command that changes data
 .RB ( write ", " blocktrash ", " crc )
 is to be used.
 .TP
+.B -R
+Like
+.B -r
+but allows the 
+.B blockget
+command even if the file system is "dirty" (unreconciled transactions
+in the log).
+.TP
 .B \-x
 Specifies expert mode.
 This enables the