| Message ID | 20200730015755.1827498-1-edumazet@google.com (mailing list archive) |
|---|---|
| State | Mainlined |
| Commit | 928da37a229f344424ffc89c9a58feb2368bb018 |
| Delegated to: | Jason Gunthorpe |
| Headers | show |
| Series | [net] RDMA/umem: add a schedule point in ib_umem_get() | expand |
On Wed, Jul 29, 2020 at 06:57:55PM -0700, Eric Dumazet wrote: > Mapping as little as 64GB can take more than 10 seconds, > triggering issues on kernels with CONFIG_PREEMPT_NONE=y. > > ib_umem_get() already splits the work in 2MB units on x86_64, > adding a cond_resched() in the long-lasting loop is enough > to solve the issue. > > Note that sg_alloc_table() can still use more than 100 ms, > which is also problematic. This might be addressed later > in ib_umem_add_sg_table(), adding new blocks in sgl > on demand. I have seen some patches in progress to do exactly this, the motivation is to reduce the memory consumption if a lot of pages are combined. > Signed-off-by: Eric Dumazet <edumazet@google.com> > Cc: Doug Ledford <dledford@redhat.com> > Cc: Jason Gunthorpe <jgg@ziepe.ca> > Cc: linux-rdma@vger.kernel.org > --- > drivers/infiniband/core/umem.c | 1 + > 1 file changed, 1 insertion(+) Why [PATCH net] ? Anyhow, applied to rdma for-next Thanks, Jason
On Fri, Jul 31, 2020 at 10:17 AM Jason Gunthorpe <jgg@nvidia.com> wrote: > > On Wed, Jul 29, 2020 at 06:57:55PM -0700, Eric Dumazet wrote: > > Mapping as little as 64GB can take more than 10 seconds, > > triggering issues on kernels with CONFIG_PREEMPT_NONE=y. > > > > ib_umem_get() already splits the work in 2MB units on x86_64, > > adding a cond_resched() in the long-lasting loop is enough > > to solve the issue. > > > > Note that sg_alloc_table() can still use more than 100 ms, > > which is also problematic. This might be addressed later > > in ib_umem_add_sg_table(), adding new blocks in sgl > > on demand. > > I have seen some patches in progress to do exactly this, the > motivation is to reduce the memory consumption if a lot of pages are > combined. Nice ;) > > > Signed-off-by: Eric Dumazet <edumazet@google.com> > > Cc: Doug Ledford <dledford@redhat.com> > > Cc: Jason Gunthorpe <jgg@ziepe.ca> > > Cc: linux-rdma@vger.kernel.org > > --- > > drivers/infiniband/core/umem.c | 1 + > > 1 file changed, 1 insertion(+) > > Why [PATCH net] ? Sorry, I used a script that I normally use for net submissions, forgot to remove this tag ;) > > Anyhow, applied to rdma for-next Thanks ! > > Thanks, > Jason
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 82455a1392f1d19c96ae956f0bd4e93e3a52d29c..831bff8d52e547834e9e04064127fbb280595126 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -261,6 +261,7 @@ struct ib_umem *ib_umem_get(struct ib_device *device, unsigned long addr, sg = umem->sg_head.sgl; while (npages) { + cond_resched(); ret = pin_user_pages_fast(cur_base, min_t(unsigned long, npages, PAGE_SIZE /
Mapping as little as 64GB can take more than 10 seconds, triggering issues on kernels with CONFIG_PREEMPT_NONE=y. ib_umem_get() already splits the work in 2MB units on x86_64, adding a cond_resched() in the long-lasting loop is enough to solve the issue. Note that sg_alloc_table() can still use more than 100 ms, which is also problematic. This might be addressed later in ib_umem_add_sg_table(), adding new blocks in sgl on demand. Signed-off-by: Eric Dumazet <edumazet@google.com> Cc: Doug Ledford <dledford@redhat.com> Cc: Jason Gunthorpe <jgg@ziepe.ca> Cc: linux-rdma@vger.kernel.org --- drivers/infiniband/core/umem.c | 1 + 1 file changed, 1 insertion(+)