diff mbox series

[09/10] xfs: validate feature support when recovering rmap/refcount/bmap intents

Message ID 160704435080.734470.11175993745850698818.stgit@magnolia (mailing list archive)
State Superseded, archived
Headers show
Series xfs: strengthen log intent validation | expand

Commit Message

Darrick J. Wong Dec. 4, 2020, 1:12 a.m. UTC 
From: Darrick J. Wong <darrick.wong@oracle.com>

The bmap, rmap, and refcount log intent items were added to support the
rmap and reflink features.  Because these features come with changes to
the ondisk format, the log items aren't tied to a log incompat flag.

However, the log recovery routines don't actually check for those
feature flags.  The kernel has no business replayng an intent item for a
feature that isn't enabled, so check that as part of recovered log item
validation.  (Note that kernels pre-dating rmap and reflink will fail
the mount on the unknown log item type code.)

Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 fs/xfs/xfs_bmap_item.c     |    4 ++++
 fs/xfs/xfs_refcount_item.c |    3 +++
 fs/xfs/xfs_rmap_item.c     |    3 +++
 3 files changed, 10 insertions(+)

Comments

Brian Foster Dec. 4, 2020, 2 p.m. UTC | #1 
On Thu, Dec 03, 2020 at 05:12:30PM -0800, Darrick J. Wong wrote:
> From: Darrick J. Wong <darrick.wong@oracle.com>
> 
> The bmap, rmap, and refcount log intent items were added to support the
> rmap and reflink features.  Because these features come with changes to
> the ondisk format, the log items aren't tied to a log incompat flag.
> 
> However, the log recovery routines don't actually check for those
> feature flags.  The kernel has no business replayng an intent item for a
> feature that isn't enabled, so check that as part of recovered log item
> validation.  (Note that kernels pre-dating rmap and reflink will fail
> the mount on the unknown log item type code.)
> 
> Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> ---
>  fs/xfs/xfs_bmap_item.c     |    4 ++++
>  fs/xfs/xfs_refcount_item.c |    3 +++
>  fs/xfs/xfs_rmap_item.c     |    3 +++
>  3 files changed, 10 insertions(+)
> 
> 
> diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
> index 78346d47564b..4ea9132716c6 100644
> --- a/fs/xfs/xfs_bmap_item.c
> +++ b/fs/xfs/xfs_bmap_item.c
> @@ -425,6 +425,10 @@ xfs_bui_validate(
>  {
>  	struct xfs_map_extent		*bmap;
>  
> +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb) &&
> +	    !xfs_sb_version_hasreflink(&mp->m_sb))
> +		return false;
> +

Took me a minute to realize we use the map/unmap for extent swap if rmap
is enabled. That does make me wonder a bit.. had we made this kind of
recovery feature validation change before that came around (such that we
probably would have only checked _hasreflink() here), would we have
created an unnecessary backwards incompatibility?

Brian

>  	/* Only one mapping operation per BUI... */
>  	if (buip->bui_format.bui_nextents != XFS_BUI_MAX_FAST_EXTENTS)
>  		return false;
> diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
> index 8ad6c81f6d8f..2b28f5643c0b 100644
> --- a/fs/xfs/xfs_refcount_item.c
> +++ b/fs/xfs/xfs_refcount_item.c
> @@ -423,6 +423,9 @@ xfs_cui_validate_phys(
>  	struct xfs_mount		*mp,
>  	struct xfs_phys_extent		*refc)
>  {
> +	if (!xfs_sb_version_hasreflink(&mp->m_sb))
> +		return false;
> +
>  	if (refc->pe_flags & ~XFS_REFCOUNT_EXTENT_FLAGS)
>  		return false;
>  
> diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
> index f296ec349936..2628bc0080fe 100644
> --- a/fs/xfs/xfs_rmap_item.c
> +++ b/fs/xfs/xfs_rmap_item.c
> @@ -466,6 +466,9 @@ xfs_rui_validate_map(
>  	struct xfs_mount		*mp,
>  	struct xfs_map_extent		*rmap)
>  {
> +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb))
> +		return false;
> +
>  	if (rmap->me_flags & ~XFS_RMAP_EXTENT_FLAGS)
>  		return false;
>  
>
Darrick J. Wong Dec. 6, 2020, 11:08 p.m. UTC | #2 
On Fri, Dec 04, 2020 at 09:00:36AM -0500, Brian Foster wrote:
> On Thu, Dec 03, 2020 at 05:12:30PM -0800, Darrick J. Wong wrote:
> > From: Darrick J. Wong <darrick.wong@oracle.com>
> > 
> > The bmap, rmap, and refcount log intent items were added to support the
> > rmap and reflink features.  Because these features come with changes to
> > the ondisk format, the log items aren't tied to a log incompat flag.
> > 
> > However, the log recovery routines don't actually check for those
> > feature flags.  The kernel has no business replayng an intent item for a
> > feature that isn't enabled, so check that as part of recovered log item
> > validation.  (Note that kernels pre-dating rmap and reflink will fail
> > the mount on the unknown log item type code.)
> > 
> > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > ---
> >  fs/xfs/xfs_bmap_item.c     |    4 ++++
> >  fs/xfs/xfs_refcount_item.c |    3 +++
> >  fs/xfs/xfs_rmap_item.c     |    3 +++
> >  3 files changed, 10 insertions(+)
> > 
> > 
> > diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
> > index 78346d47564b..4ea9132716c6 100644
> > --- a/fs/xfs/xfs_bmap_item.c
> > +++ b/fs/xfs/xfs_bmap_item.c
> > @@ -425,6 +425,10 @@ xfs_bui_validate(
> >  {
> >  	struct xfs_map_extent		*bmap;
> >  
> > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb) &&
> > +	    !xfs_sb_version_hasreflink(&mp->m_sb))
> > +		return false;
> > +
> 
> Took me a minute to realize we use the map/unmap for extent swap if rmap
> is enabled. That does make me wonder a bit.. had we made this kind of
> recovery feature validation change before that came around (such that we
> probably would have only checked _hasreflink() here), would we have
> created an unnecessary backwards incompatibility?

Yes.

I confess to cheating a little here -- technically the bmap intents were
introduced with reflink in 4.9, whereas rmap was introduced in 4.8.  The
proper solution is probably to introduce a new log incompat bit for bmap
intents when reflink isn't enabled, but TBH there were enough other rmap
bugs in 4.8 (not to mention the EXPERIMENTAL warning) that nobody should
be running that old of a kernel on a production system.

(Also we don't enable rmap by default yet whereas reflink has been
enabled by default since 4.18, so the number of people affected probably
isn't very high...)

Secondary question: should we patch 4.9 and 4.14 to disable rmap and
reflink support, since they both still have EXPERIMENTAL warnings?

--D

> Brian
> 
> >  	/* Only one mapping operation per BUI... */
> >  	if (buip->bui_format.bui_nextents != XFS_BUI_MAX_FAST_EXTENTS)
> >  		return false;
> > diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
> > index 8ad6c81f6d8f..2b28f5643c0b 100644
> > --- a/fs/xfs/xfs_refcount_item.c
> > +++ b/fs/xfs/xfs_refcount_item.c
> > @@ -423,6 +423,9 @@ xfs_cui_validate_phys(
> >  	struct xfs_mount		*mp,
> >  	struct xfs_phys_extent		*refc)
> >  {
> > +	if (!xfs_sb_version_hasreflink(&mp->m_sb))
> > +		return false;
> > +
> >  	if (refc->pe_flags & ~XFS_REFCOUNT_EXTENT_FLAGS)
> >  		return false;
> >  
> > diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
> > index f296ec349936..2628bc0080fe 100644
> > --- a/fs/xfs/xfs_rmap_item.c
> > +++ b/fs/xfs/xfs_rmap_item.c
> > @@ -466,6 +466,9 @@ xfs_rui_validate_map(
> >  	struct xfs_mount		*mp,
> >  	struct xfs_map_extent		*rmap)
> >  {
> > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb))
> > +		return false;
> > +
> >  	if (rmap->me_flags & ~XFS_RMAP_EXTENT_FLAGS)
> >  		return false;
> >  
> > 
>
Brian Foster Dec. 7, 2020, 2:02 p.m. UTC | #3 
On Sun, Dec 06, 2020 at 03:08:42PM -0800, Darrick J. Wong wrote:
> On Fri, Dec 04, 2020 at 09:00:36AM -0500, Brian Foster wrote:
> > On Thu, Dec 03, 2020 at 05:12:30PM -0800, Darrick J. Wong wrote:
> > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > 
> > > The bmap, rmap, and refcount log intent items were added to support the
> > > rmap and reflink features.  Because these features come with changes to
> > > the ondisk format, the log items aren't tied to a log incompat flag.
> > > 
> > > However, the log recovery routines don't actually check for those
> > > feature flags.  The kernel has no business replayng an intent item for a
> > > feature that isn't enabled, so check that as part of recovered log item
> > > validation.  (Note that kernels pre-dating rmap and reflink will fail
> > > the mount on the unknown log item type code.)
> > > 
> > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > ---
> > >  fs/xfs/xfs_bmap_item.c     |    4 ++++
> > >  fs/xfs/xfs_refcount_item.c |    3 +++
> > >  fs/xfs/xfs_rmap_item.c     |    3 +++
> > >  3 files changed, 10 insertions(+)
> > > 
> > > 
> > > diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
> > > index 78346d47564b..4ea9132716c6 100644
> > > --- a/fs/xfs/xfs_bmap_item.c
> > > +++ b/fs/xfs/xfs_bmap_item.c
> > > @@ -425,6 +425,10 @@ xfs_bui_validate(
> > >  {
> > >  	struct xfs_map_extent		*bmap;
> > >  
> > > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb) &&
> > > +	    !xfs_sb_version_hasreflink(&mp->m_sb))
> > > +		return false;
> > > +
> > 
> > Took me a minute to realize we use the map/unmap for extent swap if rmap
> > is enabled. That does make me wonder a bit.. had we made this kind of
> > recovery feature validation change before that came around (such that we
> > probably would have only checked _hasreflink() here), would we have
> > created an unnecessary backwards incompatibility?
> 
> Yes.
> 
> I confess to cheating a little here -- technically the bmap intents were
> introduced with reflink in 4.9, whereas rmap was introduced in 4.8.  The
> proper solution is probably to introduce a new log incompat bit for bmap
> intents when reflink isn't enabled, but TBH there were enough other rmap
> bugs in 4.8 (not to mention the EXPERIMENTAL warning) that nobody should
> be running that old of a kernel on a production system.
> 
> (Also we don't enable rmap by default yet whereas reflink has been
> enabled by default since 4.18, so the number of people affected probably
> isn't very high...)
> 

Hmm, so this all has me a a bit concerned over the value proposition for
these particular feature checks. The current reflink/rmap feature
situation may work out Ok in practice, but it sounds like that is partly
due to timing and a little bit of luck around when the implementations
and interdependencies landed. This code will ultimately introduce a
verification pattern that will likely be followed for new features,
associated log item types, etc. and it's not totally clear to me that
we'd always get it right (as opposed to something more granular like
incompat bits for intent formats). Is this addressing a real problem
we've seen in the wild or more of a fuzzing thing?

> Secondary question: should we patch 4.9 and 4.14 to disable rmap and
> reflink support, since they both still have EXPERIMENTAL warnings?
> 

That sounds like an odd thing to do to a stable kernel, but that's just
my .02.

Brian

> --D
> 
> > Brian
> > 
> > >  	/* Only one mapping operation per BUI... */
> > >  	if (buip->bui_format.bui_nextents != XFS_BUI_MAX_FAST_EXTENTS)
> > >  		return false;
> > > diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
> > > index 8ad6c81f6d8f..2b28f5643c0b 100644
> > > --- a/fs/xfs/xfs_refcount_item.c
> > > +++ b/fs/xfs/xfs_refcount_item.c
> > > @@ -423,6 +423,9 @@ xfs_cui_validate_phys(
> > >  	struct xfs_mount		*mp,
> > >  	struct xfs_phys_extent		*refc)
> > >  {
> > > +	if (!xfs_sb_version_hasreflink(&mp->m_sb))
> > > +		return false;
> > > +
> > >  	if (refc->pe_flags & ~XFS_REFCOUNT_EXTENT_FLAGS)
> > >  		return false;
> > >  
> > > diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
> > > index f296ec349936..2628bc0080fe 100644
> > > --- a/fs/xfs/xfs_rmap_item.c
> > > +++ b/fs/xfs/xfs_rmap_item.c
> > > @@ -466,6 +466,9 @@ xfs_rui_validate_map(
> > >  	struct xfs_mount		*mp,
> > >  	struct xfs_map_extent		*rmap)
> > >  {
> > > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb))
> > > +		return false;
> > > +
> > >  	if (rmap->me_flags & ~XFS_RMAP_EXTENT_FLAGS)
> > >  		return false;
> > >  
> > > 
> > 
>
Darrick J. Wong Dec. 7, 2020, 4:50 p.m. UTC | #4 
On Mon, Dec 07, 2020 at 09:02:12AM -0500, Brian Foster wrote:
> On Sun, Dec 06, 2020 at 03:08:42PM -0800, Darrick J. Wong wrote:
> > On Fri, Dec 04, 2020 at 09:00:36AM -0500, Brian Foster wrote:
> > > On Thu, Dec 03, 2020 at 05:12:30PM -0800, Darrick J. Wong wrote:
> > > > From: Darrick J. Wong <darrick.wong@oracle.com>
> > > > 
> > > > The bmap, rmap, and refcount log intent items were added to support the
> > > > rmap and reflink features.  Because these features come with changes to
> > > > the ondisk format, the log items aren't tied to a log incompat flag.
> > > > 
> > > > However, the log recovery routines don't actually check for those
> > > > feature flags.  The kernel has no business replayng an intent item for a
> > > > feature that isn't enabled, so check that as part of recovered log item
> > > > validation.  (Note that kernels pre-dating rmap and reflink will fail
> > > > the mount on the unknown log item type code.)
> > > > 
> > > > Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
> > > > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > > > ---
> > > >  fs/xfs/xfs_bmap_item.c     |    4 ++++
> > > >  fs/xfs/xfs_refcount_item.c |    3 +++
> > > >  fs/xfs/xfs_rmap_item.c     |    3 +++
> > > >  3 files changed, 10 insertions(+)
> > > > 
> > > > 
> > > > diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
> > > > index 78346d47564b..4ea9132716c6 100644
> > > > --- a/fs/xfs/xfs_bmap_item.c
> > > > +++ b/fs/xfs/xfs_bmap_item.c
> > > > @@ -425,6 +425,10 @@ xfs_bui_validate(
> > > >  {
> > > >  	struct xfs_map_extent		*bmap;
> > > >  
> > > > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb) &&
> > > > +	    !xfs_sb_version_hasreflink(&mp->m_sb))
> > > > +		return false;
> > > > +
> > > 
> > > Took me a minute to realize we use the map/unmap for extent swap if rmap
> > > is enabled. That does make me wonder a bit.. had we made this kind of
> > > recovery feature validation change before that came around (such that we
> > > probably would have only checked _hasreflink() here), would we have
> > > created an unnecessary backwards incompatibility?
> > 
> > Yes.
> > 
> > I confess to cheating a little here -- technically the bmap intents were
> > introduced with reflink in 4.9, whereas rmap was introduced in 4.8.  The
> > proper solution is probably to introduce a new log incompat bit for bmap
> > intents when reflink isn't enabled, but TBH there were enough other rmap
> > bugs in 4.8 (not to mention the EXPERIMENTAL warning) that nobody should
> > be running that old of a kernel on a production system.
> > 
> > (Also we don't enable rmap by default yet whereas reflink has been
> > enabled by default since 4.18, so the number of people affected probably
> > isn't very high...)
> > 
> 
> Hmm, so this all has me a a bit concerned over the value proposition for
> these particular feature checks. The current reflink/rmap feature
> situation may work out Ok in practice, but it sounds like that is partly
> due to timing and a little bit of luck around when the implementations
> and interdependencies landed. This code will ultimately introduce a
> verification pattern that will likely be followed for new features,
> associated log item types, etc. and it's not totally clear to me that
> we'd always get it right (as opposed to something more granular like
> incompat bits for intent formats). Is this addressing a real problem
> we've seen in the wild or more of a fuzzing thing?

Neither, it was just me doing some code review over thanksgiving.

It also occurred to me to (re)consider this in terms of "What are we
protecting against?"  Adding feature checks to the CUI/RUI recovery
functions makes sense since we can't replay something into a feature
that isn't enabled.  For BUI items however, the bmap has existed forever
so we're really not guarding much.  If someone out there has (for
example) a V4 filesystem with a dirty BUI to replay, why not replay it?

So I guess I could just drop the feature check from the BUI recovery
function.

--D

> > Secondary question: should we patch 4.9 and 4.14 to disable rmap and
> > reflink support, since they both still have EXPERIMENTAL warnings?
> > 
> 
> That sounds like an odd thing to do to a stable kernel, but that's just
> my .02.
> 
> Brian
> 
> > --D
> > 
> > > Brian
> > > 
> > > >  	/* Only one mapping operation per BUI... */
> > > >  	if (buip->bui_format.bui_nextents != XFS_BUI_MAX_FAST_EXTENTS)
> > > >  		return false;
> > > > diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
> > > > index 8ad6c81f6d8f..2b28f5643c0b 100644
> > > > --- a/fs/xfs/xfs_refcount_item.c
> > > > +++ b/fs/xfs/xfs_refcount_item.c
> > > > @@ -423,6 +423,9 @@ xfs_cui_validate_phys(
> > > >  	struct xfs_mount		*mp,
> > > >  	struct xfs_phys_extent		*refc)
> > > >  {
> > > > +	if (!xfs_sb_version_hasreflink(&mp->m_sb))
> > > > +		return false;
> > > > +
> > > >  	if (refc->pe_flags & ~XFS_REFCOUNT_EXTENT_FLAGS)
> > > >  		return false;
> > > >  
> > > > diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
> > > > index f296ec349936..2628bc0080fe 100644
> > > > --- a/fs/xfs/xfs_rmap_item.c
> > > > +++ b/fs/xfs/xfs_rmap_item.c
> > > > @@ -466,6 +466,9 @@ xfs_rui_validate_map(
> > > >  	struct xfs_mount		*mp,
> > > >  	struct xfs_map_extent		*rmap)
> > > >  {
> > > > +	if (!xfs_sb_version_hasrmapbt(&mp->m_sb))
> > > > +		return false;
> > > > +
> > > >  	if (rmap->me_flags & ~XFS_RMAP_EXTENT_FLAGS)
> > > >  		return false;
> > > >  
> > > > 
> > > 
> > 
>
diff mbox series

Patch

diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c
index 78346d47564b..4ea9132716c6 100644
--- a/fs/xfs/xfs_bmap_item.c
+++ b/fs/xfs/xfs_bmap_item.c
@@ -425,6 +425,10 @@  xfs_bui_validate(
 {
 	struct xfs_map_extent		*bmap;
 
+	if (!xfs_sb_version_hasrmapbt(&mp->m_sb) &&
+	    !xfs_sb_version_hasreflink(&mp->m_sb))
+		return false;
+
 	/* Only one mapping operation per BUI... */
 	if (buip->bui_format.bui_nextents != XFS_BUI_MAX_FAST_EXTENTS)
 		return false;
diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c
index 8ad6c81f6d8f..2b28f5643c0b 100644
--- a/fs/xfs/xfs_refcount_item.c
+++ b/fs/xfs/xfs_refcount_item.c
@@ -423,6 +423,9 @@  xfs_cui_validate_phys(
 	struct xfs_mount		*mp,
 	struct xfs_phys_extent		*refc)
 {
+	if (!xfs_sb_version_hasreflink(&mp->m_sb))
+		return false;
+
 	if (refc->pe_flags & ~XFS_REFCOUNT_EXTENT_FLAGS)
 		return false;
 
diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c
index f296ec349936..2628bc0080fe 100644
--- a/fs/xfs/xfs_rmap_item.c
+++ b/fs/xfs/xfs_rmap_item.c
@@ -466,6 +466,9 @@  xfs_rui_validate_map(
 	struct xfs_mount		*mp,
 	struct xfs_map_extent		*rmap)
 {
+	if (!xfs_sb_version_hasrmapbt(&mp->m_sb))
+		return false;
+
 	if (rmap->me_flags & ~XFS_RMAP_EXTENT_FLAGS)
 		return false;