| Message ID | 20191216222537.491123-7-jhubbard@nvidia.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm/gup: track dma-pinned pages: FOLL_PIN | expand |
On Mon, Dec 16, 2019 at 02:25:18PM -0800, John Hubbard wrote: > As it says in the updated comment in gup.c: current FOLL_LONGTERM > behavior is incompatible with FAULT_FLAG_ALLOW_RETRY because of the > FS DAX check requirement on vmas. > > However, the corresponding restriction in get_user_pages_remote() was > slightly stricter than is actually required: it forbade all > FOLL_LONGTERM callers, but we can actually allow FOLL_LONGTERM callers > that do not set the "locked" arg. > > Update the code and comments to loosen the restriction, allowing > FOLL_LONGTERM in some cases. > > Also, copy the DAX check ("if a VMA is DAX, don't allow long term > pinning") from the VFIO call site, all the way into the internals > of get_user_pages_remote() and __gup_longterm_locked(). That is: > get_user_pages_remote() calls __gup_longterm_locked(), which in turn > calls check_dax_vmas(). This check will then be removed from the VFIO > call site in a subsequent patch. > > Thanks to Jason Gunthorpe for pointing out a clean way to fix this, > and to Dan Williams for helping clarify the DAX refactoring. > > Tested-by: Alex Williamson <alex.williamson@redhat.com> > Acked-by: Alex Williamson <alex.williamson@redhat.com> > Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> > Reviewed-by: Ira Weiny <ira.weiny@intel.com> > Suggested-by: Jason Gunthorpe <jgg@ziepe.ca> > Cc: Dan Williams <dan.j.williams@intel.com> > Cc: Jerome Glisse <jglisse@redhat.com> > Signed-off-by: John Hubbard <jhubbard@nvidia.com> > --- > mm/gup.c | 27 ++++++++++++++++++++++----- > 1 file changed, 22 insertions(+), 5 deletions(-) > > diff --git a/mm/gup.c b/mm/gup.c > index 3ecce297a47f..c0c56888e7cc 100644 > --- a/mm/gup.c > +++ b/mm/gup.c > @@ -29,6 +29,13 @@ struct follow_page_context { > unsigned int page_mask; > }; > > +static __always_inline long __gup_longterm_locked(struct task_struct *tsk, > + struct mm_struct *mm, > + unsigned long start, > + unsigned long nr_pages, > + struct page **pages, > + struct vm_area_struct **vmas, > + unsigned int flags); Any particular reason for the forward declaration? Maybe move get_user_pages_remote() down? > /* > * Return the compound head page with ref appropriately incremented, > * or NULL if that failed. > @@ -1179,13 +1186,23 @@ long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, > struct vm_area_struct **vmas, int *locked) > { > /* > - * FIXME: Current FOLL_LONGTERM behavior is incompatible with > + * Parts of FOLL_LONGTERM behavior are incompatible with > * FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on > - * vmas. As there are no users of this flag in this call we simply > - * disallow this option for now. > + * vmas. However, this only comes up if locked is set, and there are > + * callers that do request FOLL_LONGTERM, but do not set locked. So, > + * allow what we can. > */ > - if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) > - return -EINVAL; > + if (gup_flags & FOLL_LONGTERM) { > + if (WARN_ON_ONCE(locked)) > + return -EINVAL; > + /* > + * This will check the vmas (even if our vmas arg is NULL) > + * and return -ENOTSUPP if DAX isn't allowed in this case: > + */ > + return __gup_longterm_locked(tsk, mm, start, nr_pages, pages, > + vmas, gup_flags | FOLL_TOUCH | > + FOLL_REMOTE); > + } > > return __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, > locked, > -- > 2.24.1 >
On 12/18/19 8:19 AM, Kirill A. Shutemov wrote: ... >> diff --git a/mm/gup.c b/mm/gup.c >> index 3ecce297a47f..c0c56888e7cc 100644 >> --- a/mm/gup.c >> +++ b/mm/gup.c >> @@ -29,6 +29,13 @@ struct follow_page_context { >> unsigned int page_mask; >> }; >> >> +static __always_inline long __gup_longterm_locked(struct task_struct *tsk, >> + struct mm_struct *mm, >> + unsigned long start, >> + unsigned long nr_pages, >> + struct page **pages, >> + struct vm_area_struct **vmas, >> + unsigned int flags); > > Any particular reason for the forward declaration? Maybe move > get_user_pages_remote() down? > Yes, that's exactly why: I was thinking it would be cleaner to put in the forward declaration, rather than moving code blocks, but either way seems reasonable. I'll go ahead and move the code blocks and delete the forward declaration, now that someone has weighed in in favor of that. thanks,
diff --git a/mm/gup.c b/mm/gup.c index 3ecce297a47f..c0c56888e7cc 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -29,6 +29,13 @@ struct follow_page_context { unsigned int page_mask; }; +static __always_inline long __gup_longterm_locked(struct task_struct *tsk, + struct mm_struct *mm, + unsigned long start, + unsigned long nr_pages, + struct page **pages, + struct vm_area_struct **vmas, + unsigned int flags); /* * Return the compound head page with ref appropriately incremented, * or NULL if that failed. @@ -1179,13 +1186,23 @@ long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm, struct vm_area_struct **vmas, int *locked) { /* - * FIXME: Current FOLL_LONGTERM behavior is incompatible with + * Parts of FOLL_LONGTERM behavior are incompatible with * FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on - * vmas. As there are no users of this flag in this call we simply - * disallow this option for now. + * vmas. However, this only comes up if locked is set, and there are + * callers that do request FOLL_LONGTERM, but do not set locked. So, + * allow what we can. */ - if (WARN_ON_ONCE(gup_flags & FOLL_LONGTERM)) - return -EINVAL; + if (gup_flags & FOLL_LONGTERM) { + if (WARN_ON_ONCE(locked)) + return -EINVAL; + /* + * This will check the vmas (even if our vmas arg is NULL) + * and return -ENOTSUPP if DAX isn't allowed in this case: + */ + return __gup_longterm_locked(tsk, mm, start, nr_pages, pages, + vmas, gup_flags | FOLL_TOUCH | + FOLL_REMOTE); + } return __get_user_pages_locked(tsk, mm, start, nr_pages, pages, vmas, locked,
As it says in the updated comment in gup.c: current FOLL_LONGTERM behavior is incompatible with FAULT_FLAG_ALLOW_RETRY because of the FS DAX check requirement on vmas. However, the corresponding restriction in get_user_pages_remote() was slightly stricter than is actually required: it forbade all FOLL_LONGTERM callers, but we can actually allow FOLL_LONGTERM callers that do not set the "locked" arg. Update the code and comments to loosen the restriction, allowing FOLL_LONGTERM in some cases. Also, copy the DAX check ("if a VMA is DAX, don't allow long term pinning") from the VFIO call site, all the way into the internals of get_user_pages_remote() and __gup_longterm_locked(). That is: get_user_pages_remote() calls __gup_longterm_locked(), which in turn calls check_dax_vmas(). This check will then be removed from the VFIO call site in a subsequent patch. Thanks to Jason Gunthorpe for pointing out a clean way to fix this, and to Dan Williams for helping clarify the DAX refactoring. Tested-by: Alex Williamson <alex.williamson@redhat.com> Acked-by: Alex Williamson <alex.williamson@redhat.com> Reviewed-by: Jason Gunthorpe <jgg@mellanox.com> Reviewed-by: Ira Weiny <ira.weiny@intel.com> Suggested-by: Jason Gunthorpe <jgg@ziepe.ca> Cc: Dan Williams <dan.j.williams@intel.com> Cc: Jerome Glisse <jglisse@redhat.com> Signed-off-by: John Hubbard <jhubbard@nvidia.com> --- mm/gup.c | 27 ++++++++++++++++++++++----- 1 file changed, 22 insertions(+), 5 deletions(-)