[0/8] Fix VMA tree modification under mmap read lock

Message

Liam R. Howlett March 27, 2023, 6:55 p.m. UTC

Syzbot reported a BUG_ON in mm/mmap.c which was found to be caused
by an inconsistency between threads walking the VMA maple tree.
The inconsistency is caused by the page fault handler modifying the
maple tree while holding the mmap_lock for read.

This only happens for stack VMAs.  We had thought this was safe as it only
modifies a single pivot in the tree.  Unfortunately, syzbot constructed
a test case where the stack had no guard page and grew the stack to abut
the next VMA.  This causes us to delete the NULL entry between the two
VMAs and rewrite the node.

We considered several options for fixing this, including dropping the
mmap_lock, then reacquiring it for write; and relaxing the definition of
the tree to permit a zero-length NULL entry in the node.  We decided the
best option was to backport some of the RCU patches from -next, which
solve the problem by allocating a new node and RCU-freeing the old node.
Since the problem exists in 6.1, we preferred a solution which is
similar to the one we intended to merge next merge window.

These patches have been in -next since next-20230301, and have received
intensive testing in Android as part of the RCU page fault patchset.
They were also sent as part of the "Per-VMA locks" v4 patch series.
Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
RCU mode for the tree.

Performance v6.3-rc3 vs patched v6.3-rc3:
Running these changes through mmtests showed there was a 15-20%
performance decrease in will-it-scale/brk1-processes.  This tests creating
and inserting a single VMA repeatedly through the brk interface and
isn't representative of any real world applications.

Liam R. Howlett (8):
  maple_tree: be more cautious about dead nodes
  maple_tree: detect dead nodes in mas_start()
  maple_tree: fix freeing of nodes in rcu mode
  maple_tree: remove extra smp_wmb() from mas_dead_leaves()
  maple_tree: fix write memory barrier of nodes once dead for RCU mode
  maple_tree: add smp_rmb() to dead node detection
  maple_tree: add RCU lock checking to rcu callback functions
  mm: enable maple tree RCU mode by default.

 include/linux/mm_types.h         |   3 +-
 kernel/fork.c                    |   3 +
 lib/maple_tree.c                 | 269 +++++++++++++++++++++----------
 mm/mmap.c                        |   3 +-
 tools/testing/radix-tree/maple.c |  16 ++
 5 files changed, 207 insertions(+), 87 deletions(-)

Comments

Andrew Morton March 27, 2023, 7:35 p.m. UTC | #1

On Mon, 27 Mar 2023 14:55:24 -0400 "Liam R. Howlett" <[e-mail]> wrote:

> These patches have been in -next since next-20230301, and have received
> intensive testing in Android as part of the RCU page fault patchset.
> They were also sent as part of the "Per-VMA locks" v4 patch series.
> Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
> RCU mode for the tree.

What's happening here?  I assume you've decided that the first 8
patches of the "Per-VMA locks v4" series should be fast-tracked into
6.3-rcX and backported?  And we retain the rest of that series for
6.4-rc1?

Patch [3/8] hasn't come through to me, to linux-mm or to linux-kernel.

Liam R. Howlett March 27, 2023, 7:48 p.m. UTC | #2

* Andrew Morton <[e-mail]> [230327 15:35]:
> On Mon, 27 Mar 2023 14:55:24 -0400 "Liam R. Howlett" <[e-mail]> wrote:
> 
> > These patches have been in -next since next-20230301, and have received
> > intensive testing in Android as part of the RCU page fault patchset.
> > They were also sent as part of the "Per-VMA locks" v4 patch series.
> > Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
> > RCU mode for the tree.
> 
> What's happening here?  I assume you've decided that the first 8
> patches of the "Per-VMA locks v4" series should be fast-tracked into
> 6.3-rcX and backported?  And we retain the rest of that series for
> 6.4-rc1?

Yes, they need to be backported and fast tracked to fix the issue syzbot
found.

> 
> Patch [3/8] hasn't come through to me, to linux-mm or to linux-kernel.

Should arrive shortly, I received it from one of the ML.

Vlastimil Babka March 28, 2023, 9:10 a.m. UTC | #3

On 3/27/23 21:48, Liam R. Howlett wrote:
> * Andrew Morton <[e-mail]> [230327 15:35]:
>> On Mon, 27 Mar 2023 14:55:24 -0400 "Liam R. Howlett" <[e-mail]> wrote:
>> 
>> > These patches have been in -next since next-20230301, and have received
>> > intensive testing in Android as part of the RCU page fault patchset.
>> > They were also sent as part of the "Per-VMA locks" v4 patch series.
>> > Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
>> > RCU mode for the tree.
>> 
>> What's happening here?  I assume you've decided that the first 8
>> patches of the "Per-VMA locks v4" series should be fast-tracked into
>> 6.3-rcX and backported?  And we retain the rest of that series for
>> 6.4-rc1?
> 
> Yes, they need to be backported and fast tracked to fix the issue syzbot
> found.

Stable usually wants the "mainline first" which means fast tracking first,
then once it's in mainline, they pick it and annotate with mainline commit id.

One question is how Linus would feel about this now for rc5.

Another question is if we should really deviate in the patch 8/8 backport
just because it's not necessary for stable. Generally they would also prefer
not to deviate, unless there's a strong reason.

>> 
>> Patch [3/8] hasn't come through to me, to linux-mm or to linux-kernel.
> 
> Should arrive shortly, I received it from one of the ML.
> 
>

Liam R. Howlett March 28, 2023, 1:02 p.m. UTC | #4

* Vlastimil Babka <[e-mail]> [230328 05:11]:
> On 3/27/23 21:48, Liam R. Howlett wrote:
> > * Andrew Morton <[e-mail]> [230327 15:35]:
> >> On Mon, 27 Mar 2023 14:55:24 -0400 "Liam R. Howlett" <[e-mail]> wrote:
> >> 
> >> > These patches have been in -next since next-20230301, and have received
> >> > intensive testing in Android as part of the RCU page fault patchset.
> >> > They were also sent as part of the "Per-VMA locks" v4 patch series.
> >> > Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
> >> > RCU mode for the tree.
> >> 
> >> What's happening here?  I assume you've decided that the first 8
> >> patches of the "Per-VMA locks v4" series should be fast-tracked into
> >> 6.3-rcX and backported?  And we retain the rest of that series for
> >> 6.4-rc1?
> > 
> > Yes, they need to be backported and fast tracked to fix the issue syzbot
> > found.
> 
> Stable usually wants the "mainline first" which means fast tracking first,
> then once it's in mainline, they pick it and annotate with mainline commit id.

Right.  I meant these patches won't cleanly apply to 6.1/6.2 and will
need more than just a cherry-pick due to the vma iterator changes.  I
have those modified patches ready to go as well.

> 
> One question is how Linus would feel about this now for rc5.
> 
> Another question is if we should really deviate in the patch 8/8 backport
> just because it's not necessary for stable. Generally they would also prefer
> not to deviate, unless there's a strong reason.

Just to clarify, the change is to remove something that isn't necessary
at all.

> 
> >> 
> >> Patch [3/8] hasn't come through to me, to linux-mm or to linux-kernel.
> > 
> > Should arrive shortly, I received it from one of the ML.
> > 
> > 
> 
> 
> -- 
> maple-tree mailing list
> [e-mail]
> https://lists.infradead.org/mailman/listinfo/maple-tree

Liam R. Howlett April 3, 2023, 7:44 p.m. UTC | #5

* Liam R. Howlett <[e-mail]> [230328 09:02]:
> * Vlastimil Babka <[e-mail]> [230328 05:11]:
> > On 3/27/23 21:48, Liam R. Howlett wrote:
> > > * Andrew Morton <[e-mail]> [230327 15:35]:
> > >> On Mon, 27 Mar 2023 14:55:24 -0400 "Liam R. Howlett" <[e-mail]> wrote:
> > >> 
> > >> > These patches have been in -next since next-20230301, and have received
> > >> > intensive testing in Android as part of the RCU page fault patchset.
> > >> > They were also sent as part of the "Per-VMA locks" v4 patch series.
> > >> > Patches 1 to 7 are bug fixes for RCU mode of the tree and patch 8 enables
> > >> > RCU mode for the tree.
> > >> 
> > >> What's happening here?  I assume you've decided that the first 8
> > >> patches of the "Per-VMA locks v4" series should be fast-tracked into
> > >> 6.3-rcX and backported?  And we retain the rest of that series for
> > >> 6.4-rc1?
> > > 
> > > Yes, they need to be backported and fast tracked to fix the issue syzbot
> > > found.
> > 
> > Stable usually wants the "mainline first" which means fast tracking first,
> > then once it's in mainline, they pick it and annotate with mainline commit id.
> 
> Right.  I meant these patches won't cleanly apply to 6.1/6.2 and will
> need more than just a cherry-pick due to the vma iterator changes.  I
> have those modified patches ready to go as well.
> 
> > 
> > One question is how Linus would feel about this now for rc5.
> > 
> > Another question is if we should really deviate in the patch 8/8 backport
> > just because it's not necessary for stable. Generally they would also prefer
> > not to deviate, unless there's a strong reason.
> 
> Just to clarify, the change is to remove something that isn't necessary
> at all.
> 

Andrew,

I just wanted to know where we stand with these patches?

I understand that it's late in the cycle, but this is a bug that affects
6.1, 6.2, 6.3-rc5 and can be triggered from userspace.

I'm asking because the LTS 6.1 is starting to be picked up by
distributions, although I don't know the scale of the install, and
getting these upstream will allow for the backported fixes to be picked
up by stable quicker.

Thanks,
Liam

Andrew Morton April 3, 2023, 8:19 p.m. UTC | #6

On Mon, 3 Apr 2023 15:44:43 -0400 "Liam R. Howlett" <[e-mail]> wrote:

> I just wanted to know where we stand with these patches?

They're in the mm.git hotfixes queue.  I'll be asking Linus
to pull them this week.