From patchwork Tue Jan 15 18:12:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F08E6C5 for ; Tue, 15 Jan 2019 18:13:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 21F262D467 for ; Tue, 15 Jan 2019 18:13:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 15BAE2D4D8; Tue, 15 Jan 2019 18:13:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3EC0B2D4AD for ; Tue, 15 Jan 2019 18:13:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 639238E0002; Tue, 15 Jan 2019 13:13:28 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5C22C8E0004; Tue, 15 Jan 2019 13:13:28 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 462778E0002; Tue, 15 Jan 2019 13:13:28 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f199.google.com (mail-pl1-f199.google.com [209.85.214.199]) by kanga.kvack.org (Postfix) with ESMTP id F073E8E0004 for ; Tue, 15 Jan 2019 13:13:27 -0500 (EST) Received: by mail-pl1-f199.google.com with SMTP id a10so2097674plp.14 for ; Tue, 15 Jan 2019 10:13:27 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=SsAwls3m+BbQBzGu0J1VI6jQoTAF/QxYAcJI1UXFsXk=; b=mQM4cXIgfmhmEeBFTBbeOhltr2crexb0h2oi3/4t9/D9F8jascf8jkLcAELAr8TQeQ T7xyn4e96Ea5fm9PL4DDC3/yuoYCI2AAnV5QD3914iXGw/SkMRdncxqu/rx8656+Jpeb 4jCEk3FG62tJUsGKdPSu85ck9dGBwXqGrdiey9blZWh3LY88hTa6B159GIx0gmda6F70 7uFg4L5H2PJmwWZ9y6DewF+3uaDP/qWLbBow99/7FwqfRv2OA4OiYvAZDyJKHX1Z7MKR MYSPOwpY1bxgOWcv5S/yQfy9nAVU0oYpfVbQUlGRq1VBil2/nsqx0ByZOu/DYqC5cb2J SWpw== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdslECN+JxM6ETWi18GzgLAX/Yn1nwTXuSzk4CuN+akgHXRXFBI aU41L/3ZPCE+7zg3PwAmqSMLDmN2sxEjnjGvU5IVJga/fgzOR8kVt/PeirQD4nfevO+GBcpJ3yp +DXUeSZ/DtUhB/50uVhC+pG5j6QjgNmK9JzhAn6iRKofktiSxL0caLKwQcmlz2ag= X-Received: by 2002:a17:902:968d:: with SMTP id n13mr5449011plp.109.1547576007607; Tue, 15 Jan 2019 10:13:27 -0800 (PST) X-Google-Smtp-Source: ALg8bN57gVWwj8FNQoDYhSfcbn4iUARjhsWpDr8Oa629fwuHUxwahHxaTAUlsrTc+bnMqIeJDHjZ X-Received: by 2002:a17:902:968d:: with SMTP id n13mr5448917plp.109.1547576006398; Tue, 15 Jan 2019 10:13:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576006; cv=none; d=google.com; s=arc-20160816; b=YwfUkNxtpymcRAoUBSZikqNEc/GliK04O28FKUftrX55sSS6Sa5dx0LxNxQnAbpeMa dTenJ93bGJMSkKZ8UiHIBKID/BKLMZ7NKLez1WDojW2qjFjph6JKeoGK4onv0WBd4Rxl xOHuMmMQNpG0ro/AjIYWMCo2VHMBEPJlUJ++khrls31nCEDbvBS/JRMtKsZuf+4z/iLX FeWvfFZNTHrhMKOnyPM6pE3mil5EUO+g540wQeVEB5PDevx5WXlQrdwjh3DADHlbcOq4 cgrIb3yD52dxemQwzjizu7C/LBa85IbWzFVUTJVA/diRMmvFhzVfqEazG42faw/Uo7mI Ydsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=SsAwls3m+BbQBzGu0J1VI6jQoTAF/QxYAcJI1UXFsXk=; b=dxXxxczW4Rkwzpitas1XHWcY9JatOOp1g/qB+MAevu10BWaq4ia/T1H34+go+XmbAq dGoFnF2tYCgvQPoGPdnKcUACF171VJDx8+CzRytEH02j27fZQOLD3aQdh82RdU7WNsc3 O+twAYveOLHBhzRB9fvUd0JaKx3puQVX2rj9H8eQDg3VtpMFESe3z+mcL7C7b7wI0Tpr vs/ItWp3RldP/ThfRy+0crgl8T//74utK/0dNjtV0vepTek2YFWZvJnDmtpmoxXHh55P ApCmxEQrTohCqHGCS8WsHKejZ1ZyPaHLybKcOP8gcHaRl95VhpN0jXWte0vG7JicYXWF Ovcw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id s8si623264pgm.508.2019.01.15.10.13.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:26 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:19 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, Davidlohr Bueso Subject: [PATCH 1/6] mm: make mm->pinned_vm an atomic counter Date: Tue, 15 Jan 2019 10:12:55 -0800 Message-Id: <20190115181300.27547-2-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Taking a sleeping lock to _only_ increment a variable is quite the overkill, and pretty much all users do this. Furthermore, some drivers (ie: infiniband and scif) that need pinned semantics can go to quite some trouble to actually delay via workqueue (un)accounting for pinned pages when not possible to acquire it. By making the counter atomic we no longer need to hold the mmap_sem and can simply some code around it for pinned_vm users. Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny --- drivers/infiniband/core/umem.c | 12 ++++++------ drivers/infiniband/hw/hfi1/user_pages.c | 6 +++--- drivers/infiniband/hw/qib/qib_user_pages.c | 4 ++-- drivers/infiniband/hw/usnic/usnic_uiom.c | 8 ++++---- drivers/misc/mic/scif/scif_rma.c | 6 +++--- fs/proc/task_mmu.c | 2 +- include/linux/mm_types.h | 2 +- kernel/events/core.c | 8 ++++---- kernel/fork.c | 2 +- mm/debug.c | 3 ++- 10 files changed, 27 insertions(+), 26 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index c6144df47ea4..bf556215aa7e 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -161,13 +161,13 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; down_write(&mm->mmap_sem); - if (check_add_overflow(mm->pinned_vm, npages, &new_pinned) || - (new_pinned > lock_limit && !capable(CAP_IPC_LOCK))) { + new_pinned = atomic_long_read(&mm->pinned_vm) + npages; + if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) { up_write(&mm->mmap_sem); ret = -ENOMEM; goto out; } - mm->pinned_vm = new_pinned; + atomic_long_set(&mm->pinned_vm, new_pinned); up_write(&mm->mmap_sem); cur_base = addr & PAGE_MASK; @@ -229,7 +229,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, __ib_umem_release(context->device, umem, 0); vma: down_write(&mm->mmap_sem); - mm->pinned_vm -= ib_umem_num_pages(umem); + atomic_long_sub(ib_umem_num_pages(umem), &mm->pinned_vm); up_write(&mm->mmap_sem); out: if (vma_list) @@ -258,7 +258,7 @@ static void ib_umem_release_defer(struct work_struct *work) struct ib_umem *umem = container_of(work, struct ib_umem, work); down_write(&umem->owning_mm->mmap_sem); - umem->owning_mm->pinned_vm -= ib_umem_num_pages(umem); + atomic_long_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); up_write(&umem->owning_mm->mmap_sem); __ib_umem_release_tail(umem); @@ -297,7 +297,7 @@ void ib_umem_release(struct ib_umem *umem) } else { down_write(&umem->owning_mm->mmap_sem); } - umem->owning_mm->pinned_vm -= ib_umem_num_pages(umem); + atomic_long_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); up_write(&umem->owning_mm->mmap_sem); __ib_umem_release_tail(umem); diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index e341e6dcc388..df86a596d746 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -92,7 +92,7 @@ bool hfi1_can_pin_pages(struct hfi1_devdata *dd, struct mm_struct *mm, size = DIV_ROUND_UP(size, PAGE_SIZE); down_read(&mm->mmap_sem); - pinned = mm->pinned_vm; + pinned = atomic_long_read(&mm->pinned_vm); up_read(&mm->mmap_sem); /* First, check the absolute limit against all pinned pages. */ @@ -112,7 +112,7 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np return ret; down_write(&mm->mmap_sem); - mm->pinned_vm += ret; + atomic_long_add(ret, &mm->pinned_vm); up_write(&mm->mmap_sem); return ret; @@ -131,7 +131,7 @@ void hfi1_release_user_pages(struct mm_struct *mm, struct page **p, if (mm) { /* during close after signal, mm can be NULL */ down_write(&mm->mmap_sem); - mm->pinned_vm -= npages; + atomic_long_sub(npages, &mm->pinned_vm); up_write(&mm->mmap_sem); } } diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 16543d5e80c3..981795b23b73 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -75,7 +75,7 @@ static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, goto bail_release; } - current->mm->pinned_vm += num_pages; + atomic_long_add(num_pages, ¤t->mm->pinned_vm); ret = 0; goto bail; @@ -156,7 +156,7 @@ void qib_release_user_pages(struct page **p, size_t num_pages) __qib_release_user_pages(p, num_pages, 1); if (current->mm) { - current->mm->pinned_vm -= num_pages; + atomic_long_sub(num_pages, ¤t->mm->pinned_vm); up_write(¤t->mm->mmap_sem); } } diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c index 49275a548751..22c40c432b9e 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.c +++ b/drivers/infiniband/hw/usnic/usnic_uiom.c @@ -129,7 +129,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, uiomr->owning_mm = mm = current->mm; down_write(&mm->mmap_sem); - locked = npages + current->mm->pinned_vm; + locked = npages + atomic_long_read(¤t->mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { @@ -188,7 +188,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, if (ret < 0) usnic_uiom_put_pages(chunk_list, 0); else { - mm->pinned_vm = locked; + atomic_long_set(&mm->pinned_vm, locked); mmgrab(uiomr->owning_mm); } @@ -442,7 +442,7 @@ static void usnic_uiom_release_defer(struct work_struct *work) container_of(work, struct usnic_uiom_reg, work); down_write(&uiomr->owning_mm->mmap_sem); - uiomr->owning_mm->pinned_vm -= usnic_uiom_num_pages(uiomr); + atomic_long_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); up_write(&uiomr->owning_mm->mmap_sem); __usnic_uiom_release_tail(uiomr); @@ -470,7 +470,7 @@ void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, } else { down_write(&uiomr->owning_mm->mmap_sem); } - uiomr->owning_mm->pinned_vm -= usnic_uiom_num_pages(uiomr); + atomic_long_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); up_write(&uiomr->owning_mm->mmap_sem); __usnic_uiom_release_tail(uiomr); diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c index 749321eb91ae..a92b4d6f099c 100644 --- a/drivers/misc/mic/scif/scif_rma.c +++ b/drivers/misc/mic/scif/scif_rma.c @@ -285,7 +285,7 @@ __scif_dec_pinned_vm_lock(struct mm_struct *mm, } else { down_write(&mm->mmap_sem); } - mm->pinned_vm -= nr_pages; + atomic_long_sub(nr_pages, &mm->pinned_vm); up_write(&mm->mmap_sem); return 0; } @@ -299,7 +299,7 @@ static inline int __scif_check_inc_pinned_vm(struct mm_struct *mm, return 0; locked = nr_pages; - locked += mm->pinned_vm; + locked += atomic_long_read(&mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { dev_err(scif_info.mdev.this_device, @@ -307,7 +307,7 @@ static inline int __scif_check_inc_pinned_vm(struct mm_struct *mm, locked, lock_limit); return -ENOMEM; } - mm->pinned_vm = locked; + atomic_long_set(&mm->pinned_vm, locked); return 0; } diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f0ec9edab2f3..f5ac0b7fadcb 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -59,7 +59,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) SEQ_PUT_DEC("VmPeak:\t", hiwater_vm); SEQ_PUT_DEC(" kB\nVmSize:\t", total_vm); SEQ_PUT_DEC(" kB\nVmLck:\t", mm->locked_vm); - SEQ_PUT_DEC(" kB\nVmPin:\t", mm->pinned_vm); + SEQ_PUT_DEC(" kB\nVmPin:\t", atomic_long_read(&mm->pinned_vm)); SEQ_PUT_DEC(" kB\nVmHWM:\t", hiwater_rss); SEQ_PUT_DEC(" kB\nVmRSS:\t", total_rss); SEQ_PUT_DEC(" kB\nRssAnon:\t", anon); diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 2c471a2c43fa..38b1c5dc6d82 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -405,7 +405,7 @@ struct mm_struct { unsigned long total_vm; /* Total pages mapped */ unsigned long locked_vm; /* Pages that have PG_mlocked set */ - unsigned long pinned_vm; /* Refcount permanently increased */ + atomic_long_t pinned_vm; /* Refcount permanently increased */ unsigned long data_vm; /* VM_WRITE & ~VM_SHARED & ~VM_STACK */ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE & ~VM_STACK */ unsigned long stack_vm; /* VM_STACK */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 3cd13a30f732..af6ed973e9ee 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5459,7 +5459,7 @@ static void perf_mmap_close(struct vm_area_struct *vma) /* now it's safe to free the pages */ atomic_long_sub(rb->aux_nr_pages, &mmap_user->locked_vm); - vma->vm_mm->pinned_vm -= rb->aux_mmap_locked; + atomic_long_sub(rb->aux_mmap_locked, &vma->vm_mm->pinned_vm); /* this has to be the last one */ rb_free_aux(rb); @@ -5532,7 +5532,7 @@ static void perf_mmap_close(struct vm_area_struct *vma) */ atomic_long_sub((size >> PAGE_SHIFT) + 1, &mmap_user->locked_vm); - vma->vm_mm->pinned_vm -= mmap_locked; + atomic_long_sub(mmap_locked, &vma->vm_mm->pinned_vm); free_uid(mmap_user); out_put: @@ -5680,7 +5680,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) lock_limit = rlimit(RLIMIT_MEMLOCK); lock_limit >>= PAGE_SHIFT; - locked = vma->vm_mm->pinned_vm + extra; + locked = atomic_long_read(&vma->vm_mm->pinned_vm) + extra; if ((locked > lock_limit) && perf_paranoid_tracepoint_raw() && !capable(CAP_IPC_LOCK)) { @@ -5721,7 +5721,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) unlock: if (!ret) { atomic_long_add(user_extra, &user->locked_vm); - vma->vm_mm->pinned_vm += extra; + atomic_long_add(extra, &vma->vm_mm->pinned_vm); atomic_inc(&event->mmap_count); } else if (rb) { diff --git a/kernel/fork.c b/kernel/fork.c index b69248e6f0e0..cb34bf852231 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -981,7 +981,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, mm_pgtables_bytes_init(mm); mm->map_count = 0; mm->locked_vm = 0; - mm->pinned_vm = 0; + atomic_long_set(&mm->pinned_vm, 0); memset(&mm->rss_stat, 0, sizeof(mm->rss_stat)); spin_lock_init(&mm->page_table_lock); spin_lock_init(&mm->arg_lock); diff --git a/mm/debug.c b/mm/debug.c index 0abb987dad9b..d2dc74e83cd5 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -166,7 +166,8 @@ void dump_mm(const struct mm_struct *mm) mm_pgtables_bytes(mm), mm->map_count, mm->hiwater_rss, mm->hiwater_vm, mm->total_vm, mm->locked_vm, - mm->pinned_vm, mm->data_vm, mm->exec_vm, mm->stack_vm, + atomic_long_read(&mm->pinned_vm), + mm->data_vm, mm->exec_vm, mm->stack_vm, mm->start_code, mm->end_code, mm->start_data, mm->end_data, mm->start_brk, mm->brk, mm->start_stack, mm->arg_start, mm->arg_end, mm->env_start, mm->env_end, From patchwork Tue Jan 15 18:12:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764931 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8C33F6C5 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7E37F2D467 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 722842D4D8; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 10D3A2D467 for ; Tue, 15 Jan 2019 18:13:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE0C78E0005; Tue, 15 Jan 2019 13:13:31 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CC7BF8E0004; Tue, 15 Jan 2019 13:13:31 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B45318E0005; Tue, 15 Jan 2019 13:13:31 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id 6F48C8E0004 for ; Tue, 15 Jan 2019 13:13:31 -0500 (EST) Received: by mail-pf1-f198.google.com with SMTP id y88so2556467pfi.9 for ; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=mUM3N24aT9PyrD+2U70aULJmOFDBVISgGQ/w+hNmTgM=; b=KFDVrTOgBzWkuHN5ltAbVs+YYVLMBbKKkJuiO8CbA5GgryyJGPhuM9u7d43Gt/BGia n+sBLbSaao2A3jCMKql5n6OTaO/WjGgt6re9WLZvnXm+v072RLAobL2zGq5NzAVbZq8D lH+SfY2twaeToKl/gCbF88Qx13HxIIcAGr8lPgwGceVeNfJEACxF3bKgyDwDrL9QkJk/ hSEOvJeQDzoaHaVfPgTeOGY5h2DyKwE+8iZYEPZ2HQTKhzJr+sPyY9cfW3wpBxnmBhww jIwddmNzcA86DGyUOHZB27vQhsg8WajxkX2NbrAHGkwazE1Pai7DY81Tl6/2IBnf+S+T u/Pw== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdgity4eaaiwFMGgcqkuLqGsMCW0TdsD+wAOen0vPLVygfoucjA 0stoaqT8nwmKVdlNcmr1fO2RcE6Odk4wvCfwuz0BNtNh7nFk+aw1gC3Ta0y/o/egbRGaM1MNq8i QjeVI3jtKjCyKopDH8TtUfUVB4CLDlvSCuln43OGIwOxMhl0sj6bFPrgx9Bq7CRE= X-Received: by 2002:a63:30c8:: with SMTP id w191mr5075329pgw.120.1547576011073; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-Smtp-Source: ALg8bN4Dw8jAAcmipob90C7MYp6E8J2tt/mvtWxEoHJxzQojqgmfGXNhPw03pKy/csQKkUJH/Ozp X-Received: by 2002:a63:30c8:: with SMTP id w191mr5075264pgw.120.1547576010181; Tue, 15 Jan 2019 10:13:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576010; cv=none; d=google.com; s=arc-20160816; b=oDCtjCGTz3Ylm1oGkYl+hgxDNZc8SdQWymz3T5lOrkyP492EZu+nMHUil2EI8aM5EM 01okuSQS6AbyTO9FMaI4uPLaieIhnc+n1mszZfzCYdaAp0kscoU6B7dkpuHPXtTQr9CS jhXWVbQ0PmkYb3vTtOtb4N9L2ciAG1WhHBx2FoHYVcGghjuJbNh+gdLU8bTR4rvvguMs MWN+mY4hV0mmYsIkTkXoFVsO8qbGaeln7VEDGuiRdv4nOpWco8Gb/TliB4cnl5z9vj4r U9AGOvICEab/P8cnxgg1nTVbrQejjD9fqLjvSaYbqoonsESIqpV8lAbrldEF31tJYfhl nmMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=mUM3N24aT9PyrD+2U70aULJmOFDBVISgGQ/w+hNmTgM=; b=u2xesUA7vv2CMc6d1TzGnph+11mjq/7yA1rtgxInz9de010pCvV/lP5nFXbx+4UnAU jiu2Pt7SI1UmnbKUsXopumN4p6IhiD4OhRPnf3uiqgRghTHR2HtpW53fIvYY1f3phnT3 V87pFfYtEjFxnTlj0wZTTcq8AASxHOpfwg2nn4prtLEipZ2Y38AZpmGJoBAOYgb3R8Gy BVj5y9banUOlGbPf6JOUm0mmWRHE7IuBlp6xav3bziikYcgi2HNEsmVTqJZgUximmFY+ hAL6dWlL1Rv1+EEH0lLVlyjffTW0Tt9RlHU6+C7jutYs6pSiJ3KPiMi1SFFXknGHn5NH pYfw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id i1si3780777pgi.480.2019.01.15.10.13.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:30 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:24 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Davidlohr Bueso Subject: [PATCH 3/6] drivers/IB,qib: do not use mmap_sem Date: Tue, 15 Jan 2019 10:12:57 -0800 Message-Id: <20190115181300.27547-4-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The driver uses mmap_sem for both pinned_vm accounting and get_user_pages(). By using gup_fast() and letting the mm handle the lock if needed, we can no longer rely on the semaphore and simplify the whole thing as the pinning is decoupled from the lock. This also fixes a bug that __qib_get_user_pages was not taking into account the current value of pinned_vm. Cc: dennis.dalessandro@intel.com Cc: mike.marciniszyn@intel.com Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny --- drivers/infiniband/hw/qib/qib_user_pages.c | 67 ++++++++++-------------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 981795b23b73..4b7a5be782e6 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -49,43 +49,6 @@ static void __qib_release_user_pages(struct page **p, size_t num_pages, } } -/* - * Call with current->mm->mmap_sem held. - */ -static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, - struct page **p) -{ - unsigned long lock_limit; - size_t got; - int ret; - - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) { - ret = -ENOMEM; - goto bail; - } - - for (got = 0; got < num_pages; got += ret) { - ret = get_user_pages(start_page + got * PAGE_SIZE, - num_pages - got, - FOLL_WRITE | FOLL_FORCE, - p + got, NULL); - if (ret < 0) - goto bail_release; - } - - atomic_long_add(num_pages, ¤t->mm->pinned_vm); - - ret = 0; - goto bail; - -bail_release: - __qib_release_user_pages(p, got, 0); -bail: - return ret; -} - /** * qib_map_page - a safety wrapper around pci_map_page() * @@ -137,26 +100,40 @@ int qib_map_page(struct pci_dev *hwdev, struct page *page, dma_addr_t *daddr) int qib_get_user_pages(unsigned long start_page, size_t num_pages, struct page **p) { + unsigned long locked, lock_limit; + size_t got; int ret; - down_write(¤t->mm->mmap_sem); + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + locked = atomic_long_add_return(num_pages, ¤t->mm->pinned_vm); - ret = __qib_get_user_pages(start_page, num_pages, p); + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + ret = -ENOMEM; + goto bail; + } - up_write(¤t->mm->mmap_sem); + for (got = 0; got < num_pages; got += ret) { + ret = get_user_pages_fast(start_page + got * PAGE_SIZE, + num_pages - got, + FOLL_WRITE | FOLL_FORCE, + p + got); + if (ret < 0) + goto bail_release; + } + return 0; +bail_release: + __qib_release_user_pages(p, got, 0); +bail: + atomic_long_sub(num_pages, ¤t->mm->pinned_vm); return ret; } void qib_release_user_pages(struct page **p, size_t num_pages) { - if (current->mm) /* during close after signal, mm can be NULL */ - down_write(¤t->mm->mmap_sem); - __qib_release_user_pages(p, num_pages, 1); - if (current->mm) { + if (current->mm) { /* during close after signal, mm can be NULL */ atomic_long_sub(num_pages, ¤t->mm->pinned_vm); - up_write(¤t->mm->mmap_sem); } } From patchwork Tue Jan 15 18:12:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 297666C5 for ; Tue, 15 Jan 2019 18:13:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CD2E2D467 for ; Tue, 15 Jan 2019 18:13:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10B772D4D8; Tue, 15 Jan 2019 18:13:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 976582D467 for ; Tue, 15 Jan 2019 18:13:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 662F48E0004; Tue, 15 Jan 2019 13:13:32 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5C4F08E0006; Tue, 15 Jan 2019 13:13:32 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43E488E0004; Tue, 15 Jan 2019 13:13:32 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id EEE8A8E0006 for ; Tue, 15 Jan 2019 13:13:31 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id q64so2531173pfa.18 for ; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=aA+9kNc02EQmqMkRS/WiCkbnf0PIV936miLHkJ8CwR4=; b=ORNCT5d26U9EJnegARHAoPu5Mqwy5SbBjUpBcrZPdI7axmqT7BWQhTdBsZ74GC013D uU3OsDMrB857XpVyYO5sRFZXZU/TdKbWuKH0ZM+Z/XjEEfHKrjTZbDgsf+P7zYzz2wdP ArUa1EfCsBmO6nvmBfGunr/hxw7rDbIqDXJBxAvEwHkQh/bSzQaW2qQp5SfrfPWsKVCg F5zgQl9GSMpD8O2X2mxA6zrVlyKrR65UG9P3Ik/yk+8dEuMslu9PiS39q2X5R5LBIe1g k09ngJOIR5MxEqEYdcn7NAjNUL7D1TfSMTqXH5oDSGgtNCE6aCxM8jAkfiRTc2IHW/Lp UD6w== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdZrdQQv+eKh4K51bMgDPSl9cYODZXtO2p8QBCMouIFpC4ORi2G LEYImNplYUE51nOrzGN7+68rD7dg135x76bMX3+JwyrYf0WcbErXWg8SzRh5zOnfU0yywQHHULy dN3132F7gCG32sekFkb2WQx5GLF1GhsqQrI4lWHCxCvja+RCF3wgCZ/86nv+9MJM= X-Received: by 2002:a17:902:f20d:: with SMTP id gn13mr5223099plb.11.1547576011629; Tue, 15 Jan 2019 10:13:31 -0800 (PST) X-Google-Smtp-Source: ALg8bN6G6YRRek+Ob6JVU9VIr3PBW+UEUNLGbaTpgqw39iMh507iW3lroyaX/bbaNE0NPp/3u0AH X-Received: by 2002:a17:902:f20d:: with SMTP id gn13mr5223051plb.11.1547576010863; Tue, 15 Jan 2019 10:13:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576010; cv=none; d=google.com; s=arc-20160816; b=D2KDZn+Rjm/3a+U7NO43Bbkhd5nCzcjbkSfgo5nOCoaH2LOJ5/hGxCvmqfcTIdb5nt zIaaqBPcQkqSThACa5XHsFYSBzV5Bh4v9JmPy5g/QVV3JOADCUouwmaEy8z6+KDAj9nd qvoNGWS+kE5hO1kuawq38mYWgFRmOmOUzf0Nigoh6BfQSwRlZd7G+yYi4QPjO7GMJAox y8vgQ75vP5YWhgvATVfUCHP1AEnyOtgsUxoappkKxnjXLcu85Wn0x6arLE1mHT3WwJAc R5TcubA6FNHg9OVlFYrpfHlVHV3OicIZ2gKJwykOz5i9G4z/lN7oBl/6M/TFzL1YNs51 1MGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=aA+9kNc02EQmqMkRS/WiCkbnf0PIV936miLHkJ8CwR4=; b=pKnDrzzkpj0fIiL1Q498ubppnd4ayiUmkKn/l+Lz/ZeEEa/ti+YrPso8zRZf9uS1k1 RYqCiYkAQxTJI3GaECRxbtQINs3x3T9USpotvbp/Rcv6YkHvC1S0BDh9s0soN898HvFa 2v5tR2ixbFtl17YgSGdIb4dmx9TXwv8X7h7yeV+JgLRoSliInILaUtM6ls4gAfKuJr0y XUgaWF8Ls2N3kq22ZfcNWurfJX4bS2g4Kzd/SBtFNH4ty56tnWpiSAEZs6lanxhzP98D 9nUuDuA5qV17XZ8ufwB6kzoIhblXKP7GLGnYsTtAhcnUIJsXsy5cLmIhB5Hbj9Q0npG/ 4DvA== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id v75si3889866pfd.157.2019.01.15.10.13.30 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:30 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:26 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, Davidlohr Bueso Subject: [PATCH 4/6] drivers/IB,hfi1: do not se mmap_sem Date: Tue, 15 Jan 2019 10:12:58 -0800 Message-Id: <20190115181300.27547-5-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This driver already uses gup_fast() and thus we can just drop the mmap_sem protection around the pinned_vm counter. Note that the window between when hfi1_can_pin_pages() is called and the actual counter is incremented remains the same as mmap_sem was _only_ used for when ->pinned_vm was touched. Cc: mike.marciniszyn@intel.com Cc: dennis.dalessandro@intel.com Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny --- drivers/infiniband/hw/hfi1/user_pages.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index df86a596d746..f0c6f219f575 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -91,9 +91,7 @@ bool hfi1_can_pin_pages(struct hfi1_devdata *dd, struct mm_struct *mm, /* Convert to number of pages */ size = DIV_ROUND_UP(size, PAGE_SIZE); - down_read(&mm->mmap_sem); pinned = atomic_long_read(&mm->pinned_vm); - up_read(&mm->mmap_sem); /* First, check the absolute limit against all pinned pages. */ if (pinned + npages >= ulimit && !can_lock) @@ -111,9 +109,7 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np if (ret < 0) return ret; - down_write(&mm->mmap_sem); atomic_long_add(ret, &mm->pinned_vm); - up_write(&mm->mmap_sem); return ret; } @@ -130,8 +126,6 @@ void hfi1_release_user_pages(struct mm_struct *mm, struct page **p, } if (mm) { /* during close after signal, mm can be NULL */ - down_write(&mm->mmap_sem); atomic_long_sub(npages, &mm->pinned_vm); - up_write(&mm->mmap_sem); } } From patchwork Tue Jan 15 18:12:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764939 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E1E3F6C5 for ; Tue, 15 Jan 2019 18:13:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D42A92D467 for ; Tue, 15 Jan 2019 18:13:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C800F2D4D8; Tue, 15 Jan 2019 18:13:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 44D1B2D467 for ; Tue, 15 Jan 2019 18:13:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D66338E0009; Tue, 15 Jan 2019 13:13:41 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C78F88E0007; Tue, 15 Jan 2019 13:13:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B406A8E0008; Tue, 15 Jan 2019 13:13:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f200.google.com (mail-pf1-f200.google.com [209.85.210.200]) by kanga.kvack.org (Postfix) with ESMTP id 682918E0006 for ; Tue, 15 Jan 2019 13:13:41 -0500 (EST) Received: by mail-pf1-f200.google.com with SMTP id f69so2571626pff.5 for ; Tue, 15 Jan 2019 10:13:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=OsrANWbLtw5jX9/sTFFyY9hdWjcAlO+uaJD2IzHls/4=; b=tvt9Aik74/uQmjvjhSzoV/xZm6+HAlHlliUHH+xgTmSOlqYh9j135XDY2aaxmyuLkT 4jRE10OVpgN/qIYialXdftOrRaexmfiEMrNXnHblzxVp95ut4QQPaJnepr5MB9Z+tXYq FoGxmy4srunhVJs45vi+WXiLss15U1TXYyhGoDPvPfEKSgLGEtCzt5KUNthCkTVeOyfb 9QNZbzJ4yQGxaBKxu+YSevM0Ol6EPX2NsNOSk2N6P4aP9z6p9FB6IY+qmLQWyN72k5gz FbQ5s9/kjoLAM11OAuiYqZ9OM5H7LfVCCbz5qGMiPVjsJTTVibi8N5xCFClX0luAqYTN R5DQ== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukeCJPLDtoW2VoPNFq72lg8gFq76cYnLu1UXgpzxiCT0AsBPOhf6 6fc88LsBQWHNZWpjh2Oa+1Z6/+7WlerV/f2eNHJgSiWVunrNsezdMUo8sBM0ilb4jEWhs/HyKMT RSHPRsFx0SASt9FcHW5c1/j3HolxFIfl8x6SpbT9KLcvS4bjJ7OKB9Ukq2YWztCk= X-Received: by 2002:a63:5761:: with SMTP id h33mr4889887pgm.283.1547576021063; Tue, 15 Jan 2019 10:13:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN48TlZG3CEO1nZUyCZwbY+5ZXYPJ31OkSswaoWEqZ78hwasQlhZkjQHhzM/eDUPbwC8QAUD X-Received: by 2002:a63:5761:: with SMTP id h33mr4889830pgm.283.1547576020269; Tue, 15 Jan 2019 10:13:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576020; cv=none; d=google.com; s=arc-20160816; b=xmXTF6CkRyt41+jnHLa87CLiu8rMM9H2ZY+vZZn/g9FjK4N6hwJVeS2gv8BrfaxzcL inyqAtFt01xlQqcG+S08GP9flsmR9CnHd8wA/KLOXnUoGu7pdFLqMvAW7Kk64ohEKe1u UDNFXecOYnXQxKXtI7rTJYRXFqLzzI3/4tc58cqn5HfIYwQtVxtkwVVMhY9ljtMOWzcg eot9Rr4TMeHkLuKqr5FbhrJCESllcrVXp0R9IuWkcWquo40LNBkpwOdH/1X8nZpe/tCv mWDIB2SiJhsaD2CTkHMyvbyAyS3NRuVQKZFHeLt/KlQPPPsvAZJwR0oFfYVTB3cXWiVQ x4YQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=OsrANWbLtw5jX9/sTFFyY9hdWjcAlO+uaJD2IzHls/4=; b=FXcfbiugtpPW90lNbLMpwPddz0BPonSbyeXOyneXK6mRrMVVeRviMbmAEKtZ8KqrsT f6nAGypgSIzCIUms28Q4ITpaIxSOCpO/dbrYuJLyYYD/SBBdyjLIEmgw7/q5nGRiumja 9YJMAyxascm4BXDV2Fpl/3MhOvNA+onxmV4Mo7+lZAzcjWdQD4KFCEzn6Mhxk+oF91Tq JTC3cniBnIEOKaF+2OUCvTWikflIl9ZZo5qFPbutimMkZtzdU5vEJO1vY91jpfb9VSJr O4KHMUUiB+zHcPYuesbujU5q9BLD5FF39+3iD4knLyIHY4kIHbl3GG0STqLwMyN42HTw 8Vyw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id u186si3679201pgd.131.2019.01.15.10.13.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:40 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:29 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, benve@cisco.com, neescoba@cisco.com, pkaustub@cisco.com, Davidlohr Bueso Subject: [PATCH 5/6] drivers/IB,usnic: reduce scope of mmap_sem Date: Tue, 15 Jan 2019 10:12:59 -0800 Message-Id: <20190115181300.27547-6-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP usnic_uiom_get_pages() uses gup_longterm() so we cannot really get rid of mmap_sem altogether in the driver, but we can get rid of some complexity that mmap_sem brings with only pinned_vm. We can get rid of the wq altogether as we no longer need to defer work to unpin pages as the counter is now atomic. Cc: benve@cisco.com Cc: neescoba@cisco.com Cc: pkaustub@cisco.com Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny Acked-by: Parvi Kaustubhi --- drivers/infiniband/hw/usnic/usnic_ib_main.c | 2 -- drivers/infiniband/hw/usnic/usnic_uiom.c | 54 +++-------------------------- drivers/infiniband/hw/usnic/usnic_uiom.h | 1 - 3 files changed, 4 insertions(+), 53 deletions(-) diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c index b2323a52a0dd..64bc4fda36bf 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_main.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c @@ -691,7 +691,6 @@ static int __init usnic_ib_init(void) out_pci_unreg: pci_unregister_driver(&usnic_ib_pci_driver); out_umem_fini: - usnic_uiom_fini(); return err; } @@ -704,7 +703,6 @@ static void __exit usnic_ib_destroy(void) unregister_inetaddr_notifier(&usnic_ib_inetaddr_notifier); unregister_netdevice_notifier(&usnic_ib_netdevice_notifier); pci_unregister_driver(&usnic_ib_pci_driver); - usnic_uiom_fini(); } MODULE_DESCRIPTION("Cisco VIC (usNIC) Verbs Driver"); diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c index 22c40c432b9e..555d7bc93e72 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.c +++ b/drivers/infiniband/hw/usnic/usnic_uiom.c @@ -47,8 +47,6 @@ #include "usnic_uiom.h" #include "usnic_uiom_interval_tree.h" -static struct workqueue_struct *usnic_uiom_wq; - #define USNIC_UIOM_PAGE_CHUNK \ ((PAGE_SIZE - offsetof(struct usnic_uiom_chunk, page_list)) /\ ((void *) &((struct usnic_uiom_chunk *) 0)->page_list[1] - \ @@ -129,7 +127,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, uiomr->owning_mm = mm = current->mm; down_write(&mm->mmap_sem); - locked = npages + atomic_long_read(¤t->mm->pinned_vm); + locked = atomic_long_add_return(npages, ¤t->mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { @@ -185,12 +183,11 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, } out: - if (ret < 0) + if (ret < 0) { usnic_uiom_put_pages(chunk_list, 0); - else { - atomic_long_set(&mm->pinned_vm, locked); + atomic_long_sub(npages, ¤t->mm->pinned_vm); + } else mmgrab(uiomr->owning_mm); - } up_write(&mm->mmap_sem); free_page((unsigned long) page_list); @@ -436,43 +433,12 @@ static inline size_t usnic_uiom_num_pages(struct usnic_uiom_reg *uiomr) return PAGE_ALIGN(uiomr->length + uiomr->offset) >> PAGE_SHIFT; } -static void usnic_uiom_release_defer(struct work_struct *work) -{ - struct usnic_uiom_reg *uiomr = - container_of(work, struct usnic_uiom_reg, work); - - down_write(&uiomr->owning_mm->mmap_sem); - atomic_long_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); - up_write(&uiomr->owning_mm->mmap_sem); - - __usnic_uiom_release_tail(uiomr); -} - void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, struct ib_ucontext *context) { __usnic_uiom_reg_release(uiomr->pd, uiomr, 1); - /* - * We may be called with the mm's mmap_sem already held. This - * can happen when a userspace munmap() is the call that drops - * the last reference to our file and calls our release - * method. If there are memory regions to destroy, we'll end - * up here and not be able to take the mmap_sem. In that case - * we defer the vm_locked accounting to a workqueue. - */ - if (context->closing) { - if (!down_write_trylock(&uiomr->owning_mm->mmap_sem)) { - INIT_WORK(&uiomr->work, usnic_uiom_release_defer); - queue_work(usnic_uiom_wq, &uiomr->work); - return; - } - } else { - down_write(&uiomr->owning_mm->mmap_sem); - } atomic_long_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); - up_write(&uiomr->owning_mm->mmap_sem); - __usnic_uiom_release_tail(uiomr); } @@ -601,17 +567,5 @@ int usnic_uiom_init(char *drv_name) return -EPERM; } - usnic_uiom_wq = create_workqueue(drv_name); - if (!usnic_uiom_wq) { - usnic_err("Unable to alloc wq for drv %s\n", drv_name); - return -ENOMEM; - } - return 0; } - -void usnic_uiom_fini(void) -{ - flush_workqueue(usnic_uiom_wq); - destroy_workqueue(usnic_uiom_wq); -} diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.h b/drivers/infiniband/hw/usnic/usnic_uiom.h index b86a9731071b..c88cfa087e3a 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.h +++ b/drivers/infiniband/hw/usnic/usnic_uiom.h @@ -93,5 +93,4 @@ struct usnic_uiom_reg *usnic_uiom_reg_get(struct usnic_uiom_pd *pd, void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, struct ib_ucontext *ucontext); int usnic_uiom_init(char *drv_name); -void usnic_uiom_fini(void); #endif /* USNIC_UIOM_H_ */ From patchwork Tue Jan 15 18:13:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10764941 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D74E86C5 for ; Tue, 15 Jan 2019 18:13:46 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C9A352D467 for ; Tue, 15 Jan 2019 18:13:46 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDBEC2D4D8; Tue, 15 Jan 2019 18:13:46 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D0552D467 for ; Tue, 15 Jan 2019 18:13:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 091EF8E0006; Tue, 15 Jan 2019 13:13:42 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id D63288E0008; Tue, 15 Jan 2019 13:13:41 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C04998E0006; Tue, 15 Jan 2019 13:13:41 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pg1-f197.google.com (mail-pg1-f197.google.com [209.85.215.197]) by kanga.kvack.org (Postfix) with ESMTP id 788D38E0007 for ; Tue, 15 Jan 2019 13:13:41 -0500 (EST) Received: by mail-pg1-f197.google.com with SMTP id q62so2168928pgq.9 for ; Tue, 15 Jan 2019 10:13:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=Z5C2fSyH+OydW+iAWPs1EpjubN8brXYpZjwDPvPzivc=; b=ASBp18XfDJKfcHaImXhtg3+ptlRiYBfVL20UY51I5VTc/bzxQODTd52vtTzUk/y1HB vN8hoYHKgZmXC+mBSd+ejtXG+WmN5CtcrG3Ejk31Ek7j+PLTbTDYZfhTWllp5G+i7GrN QTSzHcbTzOAjJXXYDaFEFJaoUufUqc3gO6L0wTNcOdPtUnMWb0xkWtjBCXlqqaG7e1Sf c7LzJfQtT5esEDRI1Sh1dtwFu/3Z/EO/mer0HMmEtwt7z1sgzB7IDeqYrDm6yYaRaE+Z osNpow5FBWPU0FHqVKbdjH+W+P4m4Rjc3TizQMxZCMCgh4ss0Bdd7N9qWv9rCwzA3Orv xHXw== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukfjYM9BQTR4o6hvT2SJHcm6ASsqka++tR9crj5mSvwgpt8ARJxJ hOP9XsrLdIotrIKHk6ufTF8CYihpJqvYpWskmnBVaIkTvoJP42YVxgvdYEvJ2AOB6m+fIjeilJa 4q9e0kRlQrOj4Q/zRrAyFLmrWZmFI3sxBy0S5cWwHkDzjZGt5sxpEnqmZSXgeWNQ= X-Received: by 2002:a17:902:1122:: with SMTP id d31mr5388213pla.246.1547576021141; Tue, 15 Jan 2019 10:13:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN5K+AZznoYzg8jmrj877Mik5NRK7V5K43RbolDf1CnrwCayUjmQgzjPOE8SIxnLDXGpm3Ep X-Received: by 2002:a17:902:1122:: with SMTP id d31mr5388160pla.246.1547576020389; Tue, 15 Jan 2019 10:13:40 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547576020; cv=none; d=google.com; s=arc-20160816; b=dW6d2uK/R+fmZV7/1NUnYwM2k9nCB1xaJ2zSXrS61HuSlfO7bqkRl3OIlKUiRXky14 FVYI+R95yYQqi1OGkm5KnbIqy04pB+YeNnNfIamgw1DnsFyKZ8oods6n6Boh2WE4tmD5 Ta0wlsiGLeNggyx0kCLE59ch9OhhkNUhs2zyRTanQIOdVm9wm0COBGuq6iFf8a6Gi23e 6rXD/O5OQEFdIkbfz9DHlQ6e2VgsN1AXYHWSfxclJ4h+Vb/MS5/HHPxB9B1fgqdaOWFW bBt0Kr8F3/wnjk2xWMSvSlDAuOcXVhdVUUxBdKR9VYTPRJQogV/M9atnSsyML/lvrgjt hqCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=Z5C2fSyH+OydW+iAWPs1EpjubN8brXYpZjwDPvPzivc=; b=WEunCIgMPNWLY4Iw7umTFl6vFZkDnvSoWM1JVl6CICKV8k+OHgkad1ovZuqSZ3IoJ4 L8Et0ePW1Q1J04VojxOSdP2Hs7/FR0zd2IosMy7XxEegh/5WFcqfzqf7fkViaR01/U/u 3QxQK10MTpAAbPVycKOSTbAHnKyzHUm0FmhWf+1DaxdsQRWqAcC/Pmy+Kn8eX5eCS25j vxtYByFuw54mw7MuL6wUxDj13KkxXjKp6cOWQ5F/kRZWeT+M9orshvdqCktO1oRRu8Iu PjAPDyVUNVHSNGE/OPJZuf47HW63RRHshfsem3v6JWv3PsHQ80ukC0y6bcYqJ7VTC2CY ziuA== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id u4si3692634pga.91.2019.01.15.10.13.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 15 Jan 2019 10:13:40 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Tue, 15 Jan 2019 11:13:32 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, dave@stgolabs.net, Davidlohr Bueso Subject: [PATCH 6/6] drivers/IB,core: reduce scope of mmap_sem Date: Tue, 15 Jan 2019 10:13:00 -0800 Message-Id: <20190115181300.27547-7-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190115181300.27547-1-dave@stgolabs.net> References: <20190115181300.27547-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP ib_umem_get() uses gup_longterm() and relies on the lock to stabilze the vma_list, so we cannot really get rid of mmap_sem altogether, but now that the counter is atomic, we can get of some complexity that mmap_sem brings with only pinned_vm. Signed-off-by: Davidlohr Bueso Reviewed-by: Ira Weiny --- drivers/infiniband/core/umem.c | 41 ++--------------------------------------- 1 file changed, 2 insertions(+), 39 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index bf556215aa7e..baa2412bf6fb 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -160,15 +160,12 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - down_write(&mm->mmap_sem); - new_pinned = atomic_long_read(&mm->pinned_vm) + npages; + new_pinned = atomic_long_add_return(npages, &mm->pinned_vm); if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) { - up_write(&mm->mmap_sem); + atomic_long_sub(npages, &mm->pinned_vm); ret = -ENOMEM; goto out; } - atomic_long_set(&mm->pinned_vm, new_pinned); - up_write(&mm->mmap_sem); cur_base = addr & PAGE_MASK; @@ -228,9 +225,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr, umem_release: __ib_umem_release(context->device, umem, 0); vma: - down_write(&mm->mmap_sem); atomic_long_sub(ib_umem_num_pages(umem), &mm->pinned_vm); - up_write(&mm->mmap_sem); out: if (vma_list) free_page((unsigned long) vma_list); @@ -253,25 +248,12 @@ static void __ib_umem_release_tail(struct ib_umem *umem) kfree(umem); } -static void ib_umem_release_defer(struct work_struct *work) -{ - struct ib_umem *umem = container_of(work, struct ib_umem, work); - - down_write(&umem->owning_mm->mmap_sem); - atomic_long_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); - up_write(&umem->owning_mm->mmap_sem); - - __ib_umem_release_tail(umem); -} - /** * ib_umem_release - release memory pinned with ib_umem_get * @umem: umem struct to release */ void ib_umem_release(struct ib_umem *umem) { - struct ib_ucontext *context = umem->context; - if (umem->is_odp) { ib_umem_odp_release(to_ib_umem_odp(umem)); __ib_umem_release_tail(umem); @@ -280,26 +262,7 @@ void ib_umem_release(struct ib_umem *umem) __ib_umem_release(umem->context->device, umem, 1); - /* - * We may be called with the mm's mmap_sem already held. This - * can happen when a userspace munmap() is the call that drops - * the last reference to our file and calls our release - * method. If there are memory regions to destroy, we'll end - * up here and not be able to take the mmap_sem. In that case - * we defer the vm_locked accounting a workqueue. - */ - if (context->closing) { - if (!down_write_trylock(&umem->owning_mm->mmap_sem)) { - INIT_WORK(&umem->work, ib_umem_release_defer); - queue_work(ib_wq, &umem->work); - return; - } - } else { - down_write(&umem->owning_mm->mmap_sem); - } atomic_long_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); - up_write(&umem->owning_mm->mmap_sem); - __ib_umem_release_tail(umem); } EXPORT_SYMBOL(ib_umem_release);