| Message ID | 1586877001-19138-1-git-send-email-agordeev@linux.ibm.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | mm/gup: dereference page table entry using helper | expand |
On Tue, Apr 14, 2020 at 05:10:01PM +0200, Alexander Gordeev wrote: > Commit 0005d20 ("mm/gup: Move page table entry dereference > into helper function") wrapped access to page table entries > larger than sizeof(long) into a race-aware accessor. One of > the two dereferences in gup_fast path was however overlooked. > > CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > CC: linux-mm@kvack.org > Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> > mm/gup.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/mm/gup.c b/mm/gup.c > index d53f7dd..eceb98b 100644 > +++ b/mm/gup.c > @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > if (!head) > goto pte_unmap; > > - if (unlikely(pte_val(pte) != pte_val(*ptep))) { > + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { It doesn't seem like this needs the special helper as it is just checking that the pte hasn't changed, it doesn't need to be read exactly. But it probably should technically still be a READ_ONCE. Although I think the atomic inside try_grab_compound_head prevents any real problems. Jason
On Tue, Apr 14, 2020 at 01:32:34PM -0300, Jason Gunthorpe wrote: > On Tue, Apr 14, 2020 at 05:10:01PM +0200, Alexander Gordeev wrote: > > Commit 0005d20 ("mm/gup: Move page table entry dereference > > into helper function") wrapped access to page table entries > > larger than sizeof(long) into a race-aware accessor. One of > > the two dereferences in gup_fast path was however overlooked. > > > > CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > > CC: linux-mm@kvack.org > > Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> > > mm/gup.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/mm/gup.c b/mm/gup.c > > index d53f7dd..eceb98b 100644 > > +++ b/mm/gup.c > > @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > > if (!head) > > goto pte_unmap; > > > > - if (unlikely(pte_val(pte) != pte_val(*ptep))) { > > + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { > > It doesn't seem like this needs the special helper as it is just > checking that the pte hasn't changed, it doesn't need to be read > exactly. > > But it probably should technically still be a READ_ONCE. Although I > think the atomic inside try_grab_compound_head prevents any real > problems. I think we should go for consistency here and use the helper function. Ira > > Jason >
On Tue, Apr 14, 2020 at 11:58:29AM -0700, Ira Weiny wrote: > On Tue, Apr 14, 2020 at 01:32:34PM -0300, Jason Gunthorpe wrote: > > On Tue, Apr 14, 2020 at 05:10:01PM +0200, Alexander Gordeev wrote: > > > Commit 0005d20 ("mm/gup: Move page table entry dereference > > > into helper function") wrapped access to page table entries > > > larger than sizeof(long) into a race-aware accessor. One of > > > the two dereferences in gup_fast path was however overlooked. > > > > > > CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > > > CC: linux-mm@kvack.org > > > Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> > > > mm/gup.c | 2 +- > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > diff --git a/mm/gup.c b/mm/gup.c > > > index d53f7dd..eceb98b 100644 > > > +++ b/mm/gup.c > > > @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > > > if (!head) > > > goto pte_unmap; > > > > > > - if (unlikely(pte_val(pte) != pte_val(*ptep))) { > > > + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { > > > > It doesn't seem like this needs the special helper as it is just > > checking that the pte hasn't changed, it doesn't need to be read > > exactly. > > > > But it probably should technically still be a READ_ONCE. Although I > > think the atomic inside try_grab_compound_head prevents any real > > problems. > > I think we should go for consistency here and use the helper function. It seems quite expensive to do two more unncessary barriers.. Jason
On Tue, Apr 14, 2020 at 04:06:20PM -0300, Jason Gunthorpe wrote: > On Tue, Apr 14, 2020 at 11:58:29AM -0700, Ira Weiny wrote: > > On Tue, Apr 14, 2020 at 01:32:34PM -0300, Jason Gunthorpe wrote: > > > On Tue, Apr 14, 2020 at 05:10:01PM +0200, Alexander Gordeev wrote: > > > > Commit 0005d20 ("mm/gup: Move page table entry dereference > > > > into helper function") wrapped access to page table entries > > > > larger than sizeof(long) into a race-aware accessor. One of > > > > the two dereferences in gup_fast path was however overlooked. > > > > > > > > CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > > > > CC: linux-mm@kvack.org > > > > Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> > > > > mm/gup.c | 2 +- > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > diff --git a/mm/gup.c b/mm/gup.c > > > > index d53f7dd..eceb98b 100644 > > > > +++ b/mm/gup.c > > > > @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > > > > if (!head) > > > > goto pte_unmap; > > > > > > > > - if (unlikely(pte_val(pte) != pte_val(*ptep))) { > > > > + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { > > > > > > It doesn't seem like this needs the special helper as it is just > > > checking that the pte hasn't changed, it doesn't need to be read > > > exactly. > > > > > > But it probably should technically still be a READ_ONCE. Although I > > > think the atomic inside try_grab_compound_head prevents any real > > > problems. > > > > I think we should go for consistency here and use the helper function. > > It seems quite expensive to do two more unncessary barriers.. But won't a failure to read the 'real' pte result in falling back to GUP slow? Not sure which is worse? And most arch's don't suffer from this... Ira
On Tue, Apr 14, 2020 at 12:39:53PM -0700, Ira Weiny wrote: > On Tue, Apr 14, 2020 at 04:06:20PM -0300, Jason Gunthorpe wrote: > > On Tue, Apr 14, 2020 at 11:58:29AM -0700, Ira Weiny wrote: > > > On Tue, Apr 14, 2020 at 01:32:34PM -0300, Jason Gunthorpe wrote: > > > > On Tue, Apr 14, 2020 at 05:10:01PM +0200, Alexander Gordeev wrote: > > > > > Commit 0005d20 ("mm/gup: Move page table entry dereference > > > > > into helper function") wrapped access to page table entries > > > > > larger than sizeof(long) into a race-aware accessor. One of > > > > > the two dereferences in gup_fast path was however overlooked. > > > > > > > > > > CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> > > > > > CC: linux-mm@kvack.org > > > > > Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> > > > > > mm/gup.c | 2 +- > > > > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > > diff --git a/mm/gup.c b/mm/gup.c > > > > > index d53f7dd..eceb98b 100644 > > > > > +++ b/mm/gup.c > > > > > @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, > > > > > if (!head) > > > > > goto pte_unmap; > > > > > > > > > > - if (unlikely(pte_val(pte) != pte_val(*ptep))) { > > > > > + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { > > > > > > > > It doesn't seem like this needs the special helper as it is just > > > > checking that the pte hasn't changed, it doesn't need to be read > > > > exactly. > > > > > > > > But it probably should technically still be a READ_ONCE. Although I > > > > think the atomic inside try_grab_compound_head prevents any real > > > > problems. > > > > > > I think we should go for consistency here and use the helper function. > > > > It seems quite expensive to do two more unncessary barriers.. > > But won't a failure to read the 'real' pte result in falling back to GUP slow? If there is no concurrent writer then the direct read will give the same result. If there is a concurrent writer then it is a random race if fallback to gup slow is required. Jason
diff --git a/mm/gup.c b/mm/gup.c index d53f7dd..eceb98b 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2208,7 +2208,7 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end, if (!head) goto pte_unmap; - if (unlikely(pte_val(pte) != pte_val(*ptep))) { + if (unlikely(pte_val(pte) != pte_val(gup_get_pte(ptep)))) { put_compound_head(head, 1, flags); goto pte_unmap; }
Commit 0005d20 ("mm/gup: Move page table entry dereference into helper function") wrapped access to page table entries larger than sizeof(long) into a race-aware accessor. One of the two dereferences in gup_fast path was however overlooked. CC: Kirill A. Shutemov <kirill.shutemov@linux.intel.com> CC: linux-mm@kvack.org Signed-off-by: Alexander Gordeev <agordeev@linux.ibm.com> --- mm/gup.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)