From patchwork Mon Jan 21 17:42:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774257 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AE78D139A for ; Mon, 21 Jan 2019 17:42:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9784B29F0F for ; Mon, 21 Jan 2019 17:42:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8B08C2A6E0; Mon, 21 Jan 2019 17:42:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9D16429F0F for ; Mon, 21 Jan 2019 17:42:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE1468E0005; Mon, 21 Jan 2019 12:42:57 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C919E8E0001; Mon, 21 Jan 2019 12:42:57 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B82078E0005; Mon, 21 Jan 2019 12:42:57 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by kanga.kvack.org (Postfix) with ESMTP id 90E298E0001 for ; Mon, 21 Jan 2019 12:42:57 -0500 (EST) Received: by mail-oi1-f197.google.com with SMTP id b18so10138911oii.1 for ; Mon, 21 Jan 2019 09:42:57 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=DSSmYJsDrzWuJXdRSA9CUQxZnj7ORJ/iwPIrkctOYYc=; b=d4N3g5zCpMxE3cte2a+IgkZ9tySz2KZVkwW5cf+eDG11LQqTVJIjYhFxtBqzPCRdU6 8b5t4QbIzhtDr6Zud+XHMtp5L2G/nDfmL8SlHV8U0Cn6RPVF7vUxbzAGF56wvgKQZQaM 1myG4zOWuAYO55QNTlzLY7Kyoa/9r8D3q3c2rxSxoFpSfRVsmtiWmO1x1etZnDNQs5jA io3ht5Vc6FnZRus1EiKBLBQeYftV1MfSh2tp8wrH5UxNLSIio3JuSppLljYyOkyB0eks QchIGtQFaqBWcHYOpcxvnMutDdvy15K9d9w/FK5VW7kOpZzTvhP48mOLkChsp/H7c877 wnpA== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukesjHCeVgdztFhJi0IVsqJPyfQb5/dfXMkmJ5kntqhAfRgPIRLH 78ri0fFY/Ed1heKXpJf2qMJ73EvU6K352C5aW4AwGMxFJPBP58aU66WVH+LNU3MP6wG0LmTDswf T911oCx6lLXtX0xJJbTu4xc2RqlD2tRIGe4M7HMqblEw2qD6DIeFsdoJ2m4m0PH4= X-Received: by 2002:aca:a897:: with SMTP id r145mr6438244oie.253.1548092577305; Mon, 21 Jan 2019 09:42:57 -0800 (PST) X-Google-Smtp-Source: ALg8bN54c5iKp0nse25ND04DzPfvnZ2Q6UW/VZjXpqBjvOnLpKb99erVa6wOH8EpNqQLwFRLDGMS X-Received: by 2002:aca:a897:: with SMTP id r145mr6438200oie.253.1548092576105; Mon, 21 Jan 2019 09:42:56 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092576; cv=none; d=google.com; s=arc-20160816; b=g4otPJ4a7+cnX6f2RBxV6bBHmu7US4TkgaG+7tHQ8IfJrvJpW92kfdH3tD5CKNtGWI a+Mrg3HEPN2KOeLnGun16H3YpZoRQpWGkFrX2PU/alAbJYnQvMNtprldHkksUqTBh6QI 0bTsfFzNMpa7AN5eIcx+L5/fh1lMgz7Mh8ClNq45ZLzGDkT5T9vXxCO0AKMc6a91YXj5 E4c+eSVLmEGxsy/5ww8FwEU21Aj5V+l/gFqsfBy66jpY2RxVhbkYkzblIdA09M7kqMrp sE4EUdhc55lLHJJvw0JjsKSFtv1SAQcR6BwHHBcTH2DvNQgmx/5IsigkW+DEHyJODJ/4 NFBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=DSSmYJsDrzWuJXdRSA9CUQxZnj7ORJ/iwPIrkctOYYc=; b=sogtHVXPV0z+HNrRbBWd2aFx2gQohZl+rMfGBlO6tl9T12T93RznVEe9a/GZntZv1B dEGuyLdTu8mEy9PhuWBqQ2I4VRn246zkOdLKSHuIGIDUMrUpoqvndhao7wQ8gpRynPKy VkbH+MLbsufaVMvI2O26GeEMmVNyFhmN92uZn3S1NkrlYaC5frdmAcHIKNdxDznBM0tn 73yv1dnORZ4tbeLBM8B52Uilpuim+CCkMFLNZ0Zbkr7CeHSg9iWp6JeAOkEANIOxecXA l4dXZ0dIMTVWuuAOB44CekxDHPBIQicO9yZXFtdoPN5JnM8Q8ZU38WHkpPOwEpI5VVIb kmfg== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id 93si6849753ots.304.2019.01.21.09.42.55 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:42:56 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:42:44 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, Davidlohr Bueso Subject: [PATCH 1/6] mm: make mm->pinned_vm an atomic64 counter Date: Mon, 21 Jan 2019 09:42:15 -0800 Message-Id: <20190121174220.10583-2-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Taking a sleeping lock to _only_ increment a variable is quite the overkill, and pretty much all users do this. Furthermore, some drivers (ie: infiniband and scif) that need pinned semantics can go to quite some trouble to actually delay via workqueue (un)accounting for pinned pages when not possible to acquire it. By making the counter atomic we no longer need to hold the mmap_sem and can simply some code around it for pinned_vm users. The counter is 64-bit such that we need not worry about overflows such as rdma user input controlled from userspace. Signed-off-by: Davidlohr Bueso Reviewed-by: Christoph Lameter Reviewed-by: Jan Kara Reviewed-by: Daniel Jordan --- drivers/infiniband/core/umem.c | 12 ++++++------ drivers/infiniband/hw/hfi1/user_pages.c | 6 +++--- drivers/infiniband/hw/qib/qib_user_pages.c | 4 ++-- drivers/infiniband/hw/usnic/usnic_uiom.c | 8 ++++---- drivers/misc/mic/scif/scif_rma.c | 6 +++--- fs/proc/task_mmu.c | 2 +- include/linux/mm_types.h | 2 +- kernel/events/core.c | 8 ++++---- kernel/fork.c | 2 +- mm/debug.c | 3 ++- 10 files changed, 27 insertions(+), 26 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 1efe0a74e06b..678abe1afcba 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -166,13 +166,13 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; down_write(&mm->mmap_sem); - if (check_add_overflow(mm->pinned_vm, npages, &new_pinned) || - (new_pinned > lock_limit && !capable(CAP_IPC_LOCK))) { + new_pinned = atomic64_read(&mm->pinned_vm) + npages; + if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) { up_write(&mm->mmap_sem); ret = -ENOMEM; goto out; } - mm->pinned_vm = new_pinned; + atomic64_set(&mm->pinned_vm, new_pinned); up_write(&mm->mmap_sem); cur_base = addr & PAGE_MASK; @@ -234,7 +234,7 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, __ib_umem_release(context->device, umem, 0); vma: down_write(&mm->mmap_sem); - mm->pinned_vm -= ib_umem_num_pages(umem); + atomic64_sub(ib_umem_num_pages(umem), &mm->pinned_vm); up_write(&mm->mmap_sem); out: if (vma_list) @@ -263,7 +263,7 @@ static void ib_umem_release_defer(struct work_struct *work) struct ib_umem *umem = container_of(work, struct ib_umem, work); down_write(&umem->owning_mm->mmap_sem); - umem->owning_mm->pinned_vm -= ib_umem_num_pages(umem); + atomic64_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); up_write(&umem->owning_mm->mmap_sem); __ib_umem_release_tail(umem); @@ -302,7 +302,7 @@ void ib_umem_release(struct ib_umem *umem) } else { down_write(&umem->owning_mm->mmap_sem); } - umem->owning_mm->pinned_vm -= ib_umem_num_pages(umem); + atomic64_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); up_write(&umem->owning_mm->mmap_sem); __ib_umem_release_tail(umem); diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index e341e6dcc388..40a6e434190f 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -92,7 +92,7 @@ bool hfi1_can_pin_pages(struct hfi1_devdata *dd, struct mm_struct *mm, size = DIV_ROUND_UP(size, PAGE_SIZE); down_read(&mm->mmap_sem); - pinned = mm->pinned_vm; + pinned = atomic64_read(&mm->pinned_vm); up_read(&mm->mmap_sem); /* First, check the absolute limit against all pinned pages. */ @@ -112,7 +112,7 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np return ret; down_write(&mm->mmap_sem); - mm->pinned_vm += ret; + atomic64_add(ret, &mm->pinned_vm); up_write(&mm->mmap_sem); return ret; @@ -131,7 +131,7 @@ void hfi1_release_user_pages(struct mm_struct *mm, struct page **p, if (mm) { /* during close after signal, mm can be NULL */ down_write(&mm->mmap_sem); - mm->pinned_vm -= npages; + atomic64_sub(npages, &mm->pinned_vm); up_write(&mm->mmap_sem); } } diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 16543d5e80c3..602387bf98e7 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -75,7 +75,7 @@ static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, goto bail_release; } - current->mm->pinned_vm += num_pages; + atomic64_add(num_pages, ¤t->mm->pinned_vm); ret = 0; goto bail; @@ -156,7 +156,7 @@ void qib_release_user_pages(struct page **p, size_t num_pages) __qib_release_user_pages(p, num_pages, 1); if (current->mm) { - current->mm->pinned_vm -= num_pages; + atomic64_sub(num_pages, ¤t->mm->pinned_vm); up_write(¤t->mm->mmap_sem); } } diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c index ce01a59fccc4..854436a2b437 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.c +++ b/drivers/infiniband/hw/usnic/usnic_uiom.c @@ -129,7 +129,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, uiomr->owning_mm = mm = current->mm; down_write(&mm->mmap_sem); - locked = npages + current->mm->pinned_vm; + locked = npages + atomic64_read(¤t->mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { @@ -187,7 +187,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, if (ret < 0) usnic_uiom_put_pages(chunk_list, 0); else { - mm->pinned_vm = locked; + atomic64_set(&mm->pinned_vm, locked); mmgrab(uiomr->owning_mm); } @@ -441,7 +441,7 @@ static void usnic_uiom_release_defer(struct work_struct *work) container_of(work, struct usnic_uiom_reg, work); down_write(&uiomr->owning_mm->mmap_sem); - uiomr->owning_mm->pinned_vm -= usnic_uiom_num_pages(uiomr); + atomic64_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); up_write(&uiomr->owning_mm->mmap_sem); __usnic_uiom_release_tail(uiomr); @@ -469,7 +469,7 @@ void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, } else { down_write(&uiomr->owning_mm->mmap_sem); } - uiomr->owning_mm->pinned_vm -= usnic_uiom_num_pages(uiomr); + atomic64_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); up_write(&uiomr->owning_mm->mmap_sem); __usnic_uiom_release_tail(uiomr); diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c index 749321eb91ae..2448368f181e 100644 --- a/drivers/misc/mic/scif/scif_rma.c +++ b/drivers/misc/mic/scif/scif_rma.c @@ -285,7 +285,7 @@ __scif_dec_pinned_vm_lock(struct mm_struct *mm, } else { down_write(&mm->mmap_sem); } - mm->pinned_vm -= nr_pages; + atomic64_sub(nr_pages, &mm->pinned_vm); up_write(&mm->mmap_sem); return 0; } @@ -299,7 +299,7 @@ static inline int __scif_check_inc_pinned_vm(struct mm_struct *mm, return 0; locked = nr_pages; - locked += mm->pinned_vm; + locked += atomic64_read(&mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { dev_err(scif_info.mdev.this_device, @@ -307,7 +307,7 @@ static inline int __scif_check_inc_pinned_vm(struct mm_struct *mm, locked, lock_limit); return -ENOMEM; } - mm->pinned_vm = locked; + atomic64_set(&mm->pinned_vm, locked); return 0; } diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 6976e17dba68..640ae8a47c73 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -59,7 +59,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) SEQ_PUT_DEC("VmPeak:\t", hiwater_vm); SEQ_PUT_DEC(" kB\nVmSize:\t", total_vm); SEQ_PUT_DEC(" kB\nVmLck:\t", mm->locked_vm); - SEQ_PUT_DEC(" kB\nVmPin:\t", mm->pinned_vm); + SEQ_PUT_DEC(" kB\nVmPin:\t", atomic64_read(&mm->pinned_vm)); SEQ_PUT_DEC(" kB\nVmHWM:\t", hiwater_rss); SEQ_PUT_DEC(" kB\nVmRSS:\t", total_rss); SEQ_PUT_DEC(" kB\nRssAnon:\t", anon); diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 6312b02d65ed..0c8be6f9c92d 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -404,7 +404,7 @@ struct mm_struct { unsigned long total_vm; /* Total pages mapped */ unsigned long locked_vm; /* Pages that have PG_mlocked set */ - unsigned long pinned_vm; /* Refcount permanently increased */ + atomic64_t pinned_vm; /* Refcount permanently increased */ unsigned long data_vm; /* VM_WRITE & ~VM_SHARED & ~VM_STACK */ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE & ~VM_STACK */ unsigned long stack_vm; /* VM_STACK */ diff --git a/kernel/events/core.c b/kernel/events/core.c index 3cd13a30f732..8df0b77a4687 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -5459,7 +5459,7 @@ static void perf_mmap_close(struct vm_area_struct *vma) /* now it's safe to free the pages */ atomic_long_sub(rb->aux_nr_pages, &mmap_user->locked_vm); - vma->vm_mm->pinned_vm -= rb->aux_mmap_locked; + atomic64_sub(rb->aux_mmap_locked, &vma->vm_mm->pinned_vm); /* this has to be the last one */ rb_free_aux(rb); @@ -5532,7 +5532,7 @@ static void perf_mmap_close(struct vm_area_struct *vma) */ atomic_long_sub((size >> PAGE_SHIFT) + 1, &mmap_user->locked_vm); - vma->vm_mm->pinned_vm -= mmap_locked; + atomic64_sub(mmap_locked, &vma->vm_mm->pinned_vm); free_uid(mmap_user); out_put: @@ -5680,7 +5680,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) lock_limit = rlimit(RLIMIT_MEMLOCK); lock_limit >>= PAGE_SHIFT; - locked = vma->vm_mm->pinned_vm + extra; + locked = atomic64_read(&vma->vm_mm->pinned_vm) + extra; if ((locked > lock_limit) && perf_paranoid_tracepoint_raw() && !capable(CAP_IPC_LOCK)) { @@ -5721,7 +5721,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma) unlock: if (!ret) { atomic_long_add(user_extra, &user->locked_vm); - vma->vm_mm->pinned_vm += extra; + atomic64_add(extra, &vma->vm_mm->pinned_vm); atomic_inc(&event->mmap_count); } else if (rb) { diff --git a/kernel/fork.c b/kernel/fork.c index c48e9e244a89..a68de9032ced 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -981,7 +981,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, mm_pgtables_bytes_init(mm); mm->map_count = 0; mm->locked_vm = 0; - mm->pinned_vm = 0; + atomic64_set(&mm->pinned_vm, 0); memset(&mm->rss_stat, 0, sizeof(mm->rss_stat)); spin_lock_init(&mm->page_table_lock); spin_lock_init(&mm->arg_lock); diff --git a/mm/debug.c b/mm/debug.c index 0abb987dad9b..bcf70e365a77 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -166,7 +166,8 @@ void dump_mm(const struct mm_struct *mm) mm_pgtables_bytes(mm), mm->map_count, mm->hiwater_rss, mm->hiwater_vm, mm->total_vm, mm->locked_vm, - mm->pinned_vm, mm->data_vm, mm->exec_vm, mm->stack_vm, + atomic64_read(&mm->pinned_vm), + mm->data_vm, mm->exec_vm, mm->stack_vm, mm->start_code, mm->end_code, mm->start_data, mm->end_data, mm->start_brk, mm->brk, mm->start_stack, mm->arg_start, mm->arg_end, mm->env_start, mm->env_end, From patchwork Mon Jan 21 17:42:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774259 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 925B96C2 for ; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BC8D29F0F for ; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D43F2A6E0; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAB6C29F0F for ; Mon, 21 Jan 2019 17:43:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2345A8E0006; Mon, 21 Jan 2019 12:43:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 20A2C8E0001; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 122588E0006; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by kanga.kvack.org (Postfix) with ESMTP id DE4628E0001 for ; Mon, 21 Jan 2019 12:43:06 -0500 (EST) Received: by mail-oi1-f197.google.com with SMTP id r131so9935386oia.7 for ; Mon, 21 Jan 2019 09:43:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=BvaEORAmuWSzID7o6vf8zI8PBOEpqDPHoqVpDewB9Qg=; b=VCCA/u1Ey633rnpDk2g7QyNujPjSzJfrCfQijTATde32hH11KOFSYnVkhLz0+r4FUi s0gGsIViRdBomR8Lg3vUul54Z49BEleuBgfwxjwn8dsdRkIWt886Os17UDiQN+kTTAb/ sHzIA8zILFJCpqDb/enr6X4kExWxcYPDmFfV3gYaIYkTZD7JdjCn8AFLFuFNR9fBt4yQ 7tOiLzXIW5+JMNZyZAxWwYPo8huLSmU1QoFXjVKhfeYxkjjQbAczWzUWWKZVjuF+mZxT KEjXqlaAvYhPBvTwULVbpu4pqMIFlNTcoUsR6I/Ihyr7DJb8UuYULh+T1ixoLFU63+0L CkLg== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdw5SDjEiZvHnqYFMqoJDZTswjriVTGZIA7Bmoi2sRghfA+yNUy y6BQ/rDun7htDFJI0tDSGi95kcN4mkj0SgSGDbd30nu/WdrEnd7UAUpSftkJd1nckxKo2g9bzxs Wm//9o/NshuQcrqXlpMw0WU2Wytdc6WOL8YVGQUp5ikAY4JHFAE2B4crtfHLQQKU= X-Received: by 2002:aca:f389:: with SMTP id r131mr6470732oih.58.1548092586649; Mon, 21 Jan 2019 09:43:06 -0800 (PST) X-Google-Smtp-Source: ALg8bN4+4B0JvhqCsz2mZJPsP8Q6Q0iIBBL+5ktSEWCjS7ReSAX1hMWyWJ+2VE3OUMaFRDtjlj5p X-Received: by 2002:aca:f389:: with SMTP id r131mr6470699oih.58.1548092585872; Mon, 21 Jan 2019 09:43:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092585; cv=none; d=google.com; s=arc-20160816; b=DV9wyNnut740gB1s4za16kd1MSBkT6hsaI59GBL8nkpBpQxovQqkfetOmEgyrX5fjW L9Zpbrnf4B55fgEhXyKJbh84NDtDDGX7kHalnC6MJrDpVz3IS5+TrI93xCe2IJB/YDmr eYQhcu9Z09wE/v5csyI1vZyw/AqBUvd/1YUGoEtdjh9CFSPmzETUPatWnPHvXM4/JuMq X0BRmvxnJ+nqZx0SenU4pdh1w/hzVifpyxMf1zz1FPnkDgmir2I7MTraNKsvtRum1MLA wS6+j+TgPLNK3ITRmoysgplwY6/8CpL9U2bPwQdB5o7//M26vjjnVCZ6uiP4f6mSyU5y oMmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=BvaEORAmuWSzID7o6vf8zI8PBOEpqDPHoqVpDewB9Qg=; b=ya2i9oj6o/d8fdDDyJqUMgfd0zhgRluGtozJT885dpSnp7rWxYXMKMG3mssEJA2ISM cmuU19J9HN6b/B1Gpu6ZjSxUgcBCSeoVGZmLIKrYwSoRGUSlNBZdGfDHLq9E1sRddkEK +wM+HMWaggJjuWmw8SwEKV7ctcT2SVIh/Bw/w1nSSizdZs7LQDPjPVCxdyy7YokjRX4l 6G5fMzFAngLD8bLdspZoXYmiIk2JAkU4bbk8QfMqqsqhOC7W6vIQ404cdrWd/pDt2RZp xDq4Vm8TqwI7XdvCRxA3fVCB7tyUXPzRDS9pcj174XR0bI/80XSi3i3QQ07hqNPGegs3 2AEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id f24si6470486otq.97.2019.01.21.09.43.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:43:05 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:42:52 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Davidlohr Bueso Subject: [PATCH 3/6] drivers/IB,qib: do not use mmap_sem Date: Mon, 21 Jan 2019 09:42:17 -0800 Message-Id: <20190121174220.10583-4-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The driver uses mmap_sem for both pinned_vm accounting and get_user_pages(). By using gup_fast() and letting the mm handle the lock if needed, we can no longer rely on the semaphore and simplify the whole thing as the pinning is decoupled from the lock. This also fixes a bug that __qib_get_user_pages was not taking into account the current value of pinned_vm. Cc: dennis.dalessandro@intel.com Cc: mike.marciniszyn@intel.com Reviewed-by: Ira Weiny Signed-off-by: Davidlohr Bueso --- drivers/infiniband/hw/qib/qib_user_pages.c | 67 ++++++++++-------------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 602387bf98e7..e4114aad4a2f 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -49,43 +49,6 @@ static void __qib_release_user_pages(struct page **p, size_t num_pages, } } -/* - * Call with current->mm->mmap_sem held. - */ -static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, - struct page **p) -{ - unsigned long lock_limit; - size_t got; - int ret; - - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) { - ret = -ENOMEM; - goto bail; - } - - for (got = 0; got < num_pages; got += ret) { - ret = get_user_pages(start_page + got * PAGE_SIZE, - num_pages - got, - FOLL_WRITE | FOLL_FORCE, - p + got, NULL); - if (ret < 0) - goto bail_release; - } - - atomic64_add(num_pages, ¤t->mm->pinned_vm); - - ret = 0; - goto bail; - -bail_release: - __qib_release_user_pages(p, got, 0); -bail: - return ret; -} - /** * qib_map_page - a safety wrapper around pci_map_page() * @@ -137,26 +100,40 @@ int qib_map_page(struct pci_dev *hwdev, struct page *page, dma_addr_t *daddr) int qib_get_user_pages(unsigned long start_page, size_t num_pages, struct page **p) { + unsigned long locked, lock_limit; + size_t got; int ret; - down_write(¤t->mm->mmap_sem); + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + locked = atomic64_add_return(num_pages, ¤t->mm->pinned_vm); - ret = __qib_get_user_pages(start_page, num_pages, p); + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + ret = -ENOMEM; + goto bail; + } - up_write(¤t->mm->mmap_sem); + for (got = 0; got < num_pages; got += ret) { + ret = get_user_pages_fast(start_page + got * PAGE_SIZE, + num_pages - got, + FOLL_WRITE | FOLL_FORCE, + p + got); + if (ret < 0) + goto bail_release; + } + return 0; +bail_release: + __qib_release_user_pages(p, got, 0); +bail: + atomic64_sub(num_pages, ¤t->mm->pinned_vm); return ret; } void qib_release_user_pages(struct page **p, size_t num_pages) { - if (current->mm) /* during close after signal, mm can be NULL */ - down_write(¤t->mm->mmap_sem); - __qib_release_user_pages(p, num_pages, 1); - if (current->mm) { + if (current->mm) { /* during close after signal, mm can be NULL */ atomic64_sub(num_pages, ¤t->mm->pinned_vm); - up_write(¤t->mm->mmap_sem); } } From patchwork Mon Jan 21 17:42:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774261 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B5A08139A for ; Mon, 21 Jan 2019 17:43:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A070D29F0F for ; Mon, 21 Jan 2019 17:43:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 946C12A6E0; Mon, 21 Jan 2019 17:43:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3206C29F0F for ; Mon, 21 Jan 2019 17:43:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5F088E0007; Mon, 21 Jan 2019 12:43:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A26B38E0001; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 824188E0007; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f70.google.com (mail-ot1-f70.google.com [209.85.210.70]) by kanga.kvack.org (Postfix) with ESMTP id 5B38F8E0001 for ; Mon, 21 Jan 2019 12:43:07 -0500 (EST) Received: by mail-ot1-f70.google.com with SMTP id p4so8528617otl.10 for ; Mon, 21 Jan 2019 09:43:07 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=XS5BzN3Kh9mSORepzR9fJCwL4sFTMLrvBU/dIXdFFj4=; b=uBfBWzKGBgwCHE3WgnoGj5y2HiQ6Z2csTI0SBvIu746MVg6hLHJN60s3IzITenDHup x4IOb0kfXdDeS9dRixkSH4mZTM9dQ2fgCfbIdx2G9WWNJPOMUBJhYwdHgKhft9TES5Dz s5Jm4uy+wqv3GqgdK/KPlrp0A3PUK5nX+8JPPc7m9lOHZgoldojIpiemi5OTElhJ6OKN 6hmeLmu/ieVC+Gscnyr7dtjehyu3AoQKLmQprCRQia+cQHmwKeKIlFdmLhdtt6q2jPVM d0w4r7sE4zNyZheRCe8Q82Jkz2ZwQ93eW8i5YO7G1f0tVi8BQQIId+hBU26c/V4mo/Iq a/UQ== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukcVwzAM3aNP6Z6tBnuWwxaFgAqcz79jNJ4GfO2E77Lj+NxdGeHs T1gDH8ku6a1/0SBbLjjtYvcVRqpClytanRNn1YcbvJ4uzYIFmWdr8VyOOjOIF/OnIVkRXPckkD2 Lej+r20/YgHEtxWjLf1KJPcjqG3WBTge8VQHwHO+I1lwr6ASE9FsAkQuqzjZrFKc= X-Received: by 2002:a9d:4a87:: with SMTP id i7mr20515441otf.178.1548092587035; Mon, 21 Jan 2019 09:43:07 -0800 (PST) X-Google-Smtp-Source: ALg8bN4pDISLLe5Cw/x1B+Kh3Bu57GjYhNn8XVOn5fhUzsnzGt/CvndZoLJw7iKeXjvb6pRiyB1d X-Received: by 2002:a9d:4a87:: with SMTP id i7mr20515406otf.178.1548092586399; Mon, 21 Jan 2019 09:43:06 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092586; cv=none; d=google.com; s=arc-20160816; b=nsppjvPv/2fp8AFf9fFKKMs0ac3MXkGlKdReNm1i6evzHpnGtJsClvKPOzszfOel7F evve0+7UrRgBhqxvxGFh9coFGvDOLrhr1Z6mvIyWqBv0CYAXoaV14Z0FdhfH8+Gz3luN 5ln8HotyyT8YqjfyFYTkeNSQl7p2WPaUuBYAbK/+9wYp/YO1uOuAmqRdOq9DYni0sczx MSQVACz0CyaUIRE9MH7MlaATEPKMuGX7DZeiUVJLwIBsq3GVtx+Pls6+xiDyGZunHucO Xsc4/4rU7WuCX4hJ13MFC+oXWA1BOhQEoE1QYrWCnuX+BAq98ldDe6UtfHp1yksC2F8O i9Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=XS5BzN3Kh9mSORepzR9fJCwL4sFTMLrvBU/dIXdFFj4=; b=VS8LDqQgQEGALHZZCil6aagg7jyHx6JPg59Y4yijQrB2fU5BNZbC4eWQSNi7isaTzh ba7Vhemije2+Hw8IGT2mUSNdNIaxkTCkPbI6vNz2rZ7/QQ3ofNtu34MS7ogHoXSjhEqd o/U/wrbWOZ6i286J6uwjKwds3CT1WhvLRT2QvTvMDn2xtFtQxzCk9HkmRUBRAVdEZ+Hk GwoYqfQH2tRjXrdnusEfbnGA2UKocLWAvt88ebV23JT/zkKOmRdbxQDOeyR0NXs7izGB L72Hnwj7TbvBz163kf26Sx5ohX9NdnTX88lsC69kPlJgt2r+q1K1WS/yjCG2LRyE8c1Z V4Fw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id g14si5859734otj.14.2019.01.21.09.43.06 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:43:06 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:42:57 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, Davidlohr Bueso Subject: [PATCH 4/6] drivers/IB,hfi1: do not se mmap_sem Date: Mon, 21 Jan 2019 09:42:18 -0800 Message-Id: <20190121174220.10583-5-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP This driver already uses gup_fast() and thus we can just drop the mmap_sem protection around the pinned_vm counter. Note that the window between when hfi1_can_pin_pages() is called and the actual counter is incremented remains the same as mmap_sem was _only_ used for when ->pinned_vm was touched. Cc: mike.marciniszyn@intel.com Cc: dennis.dalessandro@intel.com Reviewed-by: Ira Weiny Signed-off-by: Davidlohr Bueso --- drivers/infiniband/hw/hfi1/user_pages.c | 6 ------ 1 file changed, 6 deletions(-) diff --git a/drivers/infiniband/hw/hfi1/user_pages.c b/drivers/infiniband/hw/hfi1/user_pages.c index 40a6e434190f..24b592c6522e 100644 --- a/drivers/infiniband/hw/hfi1/user_pages.c +++ b/drivers/infiniband/hw/hfi1/user_pages.c @@ -91,9 +91,7 @@ bool hfi1_can_pin_pages(struct hfi1_devdata *dd, struct mm_struct *mm, /* Convert to number of pages */ size = DIV_ROUND_UP(size, PAGE_SIZE); - down_read(&mm->mmap_sem); pinned = atomic64_read(&mm->pinned_vm); - up_read(&mm->mmap_sem); /* First, check the absolute limit against all pinned pages. */ if (pinned + npages >= ulimit && !can_lock) @@ -111,9 +109,7 @@ int hfi1_acquire_user_pages(struct mm_struct *mm, unsigned long vaddr, size_t np if (ret < 0) return ret; - down_write(&mm->mmap_sem); atomic64_add(ret, &mm->pinned_vm); - up_write(&mm->mmap_sem); return ret; } @@ -130,8 +126,6 @@ void hfi1_release_user_pages(struct mm_struct *mm, struct page **p, } if (mm) { /* during close after signal, mm can be NULL */ - down_write(&mm->mmap_sem); atomic64_sub(npages, &mm->pinned_vm); - up_write(&mm->mmap_sem); } } From patchwork Mon Jan 21 17:42:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774267 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CC1746C2 for ; Mon, 21 Jan 2019 17:43:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B62EF29F0F for ; Mon, 21 Jan 2019 17:43:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A9E2C2A6E0; Mon, 21 Jan 2019 17:43:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 033F829F0F for ; Mon, 21 Jan 2019 17:43:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C95ED8E0001; Mon, 21 Jan 2019 12:43:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id C6A408E0008; Mon, 21 Jan 2019 12:43:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B57C18E0001; Mon, 21 Jan 2019 12:43:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-ot1-f71.google.com (mail-ot1-f71.google.com [209.85.210.71]) by kanga.kvack.org (Postfix) with ESMTP id 8C2228E0008 for ; Mon, 21 Jan 2019 12:43:17 -0500 (EST) Received: by mail-ot1-f71.google.com with SMTP id w4so8537804otj.2 for ; Mon, 21 Jan 2019 09:43:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=XiUZYKXD4Tx97CxjR3rvojV2CPcEUyCQz+9PQbH7q4Q=; b=XwvWBZdPUETHfvhxLRzuZAGsxubyrfVHMf05AA/WmBLMY67idUQdVhlRLRr711JCrl LMeSm0b8Js6QI/y2dVaFONovGfbHCp2sOpvPSp1DcQfWw/48+1bXFy7K0gbPZx7Kps/9 Ef+juEvycX3fJI9x9MDhKIUvda9XvwaxQaPnBSp4YUqSyp3SvtT4eqO5kOHAFkClfTV6 /2AT0/brftagdazn9psgHgZjPQa/KexZ9W/DyvfU5fie1qqCg7abyqZVUKmzDDmiJBwH Y+GqMoJn0Z9udnJByUmvVSBV7kbyaxHQjtp01Hv4XYZ1gpo7JMeAdrnycioE4WqqlWDm U5Cg== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukcHtkkdO9Ie/UoYGwafeKErPWFB7DsHTgv6xsnQ2s8P59H+yBcT Os+cYFtm9M0emqo1DXwptgSxyKZ7VbW1D6uP7j+N1wLJFuxZYBgwWqAjZmDOCtRlIWS7Dsw0rS2 XuEdZEjR10CSfU5DgIgrunpMmaWrCmI7U2VK+hDT4ND2uaDCNThh6EW5bu30bJ/g= X-Received: by 2002:aca:aad8:: with SMTP id t207mr5859429oie.90.1548092597296; Mon, 21 Jan 2019 09:43:17 -0800 (PST) X-Google-Smtp-Source: ALg8bN5ntdR8Y4wp+ma2a00hmV+Xkq66HP9uGvj+2KkcLcYpYI6rgGFGkiYGPAZGX9I398wjBnO2 X-Received: by 2002:aca:aad8:: with SMTP id t207mr5859389oie.90.1548092596477; Mon, 21 Jan 2019 09:43:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092596; cv=none; d=google.com; s=arc-20160816; b=OIhgn9i+RXRXs83B7wGlIpWU9ukf7HuRVaxePf6NoVn9e6+iMFRt+49f43Zj8cnZOU QaGv/BtSX8xbuc0fKh9TN++wuH0flB18sxzoippQ2mQwoZp0lb6zVmznGAKo1lz80ki+ 6EmbsAo0b4LzfX7pr5lnYXTbMmnz5MxVgCzWkqbztMdVZafmAiw3woO9B2AA9ngumLxp UkRZn3BwsinxMCsBRXocw2yEGP+FeBA09x7+G9LMoD0jb1DAmh5H7pNouNP88gv5NkVu Vio0sUK8WGdIArYGuY615RQmBWj30AK9ZAxECB08Zi6OBGVFW1UZiU6F2GTtmukLK+fZ sk7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=XiUZYKXD4Tx97CxjR3rvojV2CPcEUyCQz+9PQbH7q4Q=; b=I5cyTvSS8795QKMyMC/A+v5SwZgP5EoY684y4RTGOYH8y+p6bWGAx5doJqMemo+7ow eZWI7k9TAYfWKerFy28ax48Dst6KKL55RPfrNVBQNpeZ+qdK9BP2ekdGY7iou5UeVA0S k8liYtGf0ZnbsUKRuzzWvNv3k+bSG7TfcUKlN9O5gS9NOucj7siFOkk0W4JzXQiD9mx0 lclXn5WhY38c06lSiMJqhqZZNtbezX5GHXd9/H93H5BrIwN1g9rZOVCk8tlNLsMfiuXj iEaEk3+h/NEKRpx6Uv90vb9pRKKX/RNQr8qne2Q9FdMUz1tmDekHCiDNa6qPUKocSnwX ZU2A== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id h22si6184263otm.149.2019.01.21.09.43.16 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:43:16 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:43:02 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, benve@cisco.com, neescoba@cisco.com, pkaustub@cisco.com, Davidlohr Bueso Subject: [PATCH 5/6] drivers/IB,usnic: reduce scope of mmap_sem Date: Mon, 21 Jan 2019 09:42:19 -0800 Message-Id: <20190121174220.10583-6-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP usnic_uiom_get_pages() uses gup_longterm() so we cannot really get rid of mmap_sem altogether in the driver, but we can get rid of some complexity that mmap_sem brings with only pinned_vm. We can get rid of the wq altogether as we no longer need to defer work to unpin pages as the counter is now atomic. Cc: benve@cisco.com Cc: neescoba@cisco.com Cc: pkaustub@cisco.com Reviewed-by: Ira Weiny Signed-off-by: Davidlohr Bueso --- drivers/infiniband/hw/usnic/usnic_ib_main.c | 2 -- drivers/infiniband/hw/usnic/usnic_uiom.c | 54 +++-------------------------- drivers/infiniband/hw/usnic/usnic_uiom.h | 1 - 3 files changed, 4 insertions(+), 53 deletions(-) diff --git a/drivers/infiniband/hw/usnic/usnic_ib_main.c b/drivers/infiniband/hw/usnic/usnic_ib_main.c index 3201dd1899c7..1d363b706314 100644 --- a/drivers/infiniband/hw/usnic/usnic_ib_main.c +++ b/drivers/infiniband/hw/usnic/usnic_ib_main.c @@ -691,7 +691,6 @@ static int __init usnic_ib_init(void) out_pci_unreg: pci_unregister_driver(&usnic_ib_pci_driver); out_umem_fini: - usnic_uiom_fini(); return err; } @@ -704,7 +703,6 @@ static void __exit usnic_ib_destroy(void) unregister_inetaddr_notifier(&usnic_ib_inetaddr_notifier); unregister_netdevice_notifier(&usnic_ib_netdevice_notifier); pci_unregister_driver(&usnic_ib_pci_driver); - usnic_uiom_fini(); } MODULE_DESCRIPTION("Cisco VIC (usNIC) Verbs Driver"); diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c index 854436a2b437..505252298b52 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.c +++ b/drivers/infiniband/hw/usnic/usnic_uiom.c @@ -47,8 +47,6 @@ #include "usnic_uiom.h" #include "usnic_uiom_interval_tree.h" -static struct workqueue_struct *usnic_uiom_wq; - #define USNIC_UIOM_PAGE_CHUNK \ ((PAGE_SIZE - offsetof(struct usnic_uiom_chunk, page_list)) /\ ((void *) &((struct usnic_uiom_chunk *) 0)->page_list[1] - \ @@ -129,7 +127,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, uiomr->owning_mm = mm = current->mm; down_write(&mm->mmap_sem); - locked = npages + atomic64_read(¤t->mm->pinned_vm); + locked = atomic64_add_return(npages, ¤t->mm->pinned_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if ((locked > lock_limit) && !capable(CAP_IPC_LOCK)) { @@ -184,12 +182,11 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable, } out: - if (ret < 0) + if (ret < 0) { usnic_uiom_put_pages(chunk_list, 0); - else { - atomic64_set(&mm->pinned_vm, locked); + atomic64_sub(npages, ¤t->mm->pinned_vm); + } else mmgrab(uiomr->owning_mm); - } up_write(&mm->mmap_sem); free_page((unsigned long) page_list); @@ -435,43 +432,12 @@ static inline size_t usnic_uiom_num_pages(struct usnic_uiom_reg *uiomr) return PAGE_ALIGN(uiomr->length + uiomr->offset) >> PAGE_SHIFT; } -static void usnic_uiom_release_defer(struct work_struct *work) -{ - struct usnic_uiom_reg *uiomr = - container_of(work, struct usnic_uiom_reg, work); - - down_write(&uiomr->owning_mm->mmap_sem); - atomic64_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); - up_write(&uiomr->owning_mm->mmap_sem); - - __usnic_uiom_release_tail(uiomr); -} - void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, struct ib_ucontext *context) { __usnic_uiom_reg_release(uiomr->pd, uiomr, 1); - /* - * We may be called with the mm's mmap_sem already held. This - * can happen when a userspace munmap() is the call that drops - * the last reference to our file and calls our release - * method. If there are memory regions to destroy, we'll end - * up here and not be able to take the mmap_sem. In that case - * we defer the vm_locked accounting to a workqueue. - */ - if (context->closing) { - if (!down_write_trylock(&uiomr->owning_mm->mmap_sem)) { - INIT_WORK(&uiomr->work, usnic_uiom_release_defer); - queue_work(usnic_uiom_wq, &uiomr->work); - return; - } - } else { - down_write(&uiomr->owning_mm->mmap_sem); - } atomic64_sub(usnic_uiom_num_pages(uiomr), &uiomr->owning_mm->pinned_vm); - up_write(&uiomr->owning_mm->mmap_sem); - __usnic_uiom_release_tail(uiomr); } @@ -600,17 +566,5 @@ int usnic_uiom_init(char *drv_name) return -EPERM; } - usnic_uiom_wq = create_workqueue(drv_name); - if (!usnic_uiom_wq) { - usnic_err("Unable to alloc wq for drv %s\n", drv_name); - return -ENOMEM; - } - return 0; } - -void usnic_uiom_fini(void) -{ - flush_workqueue(usnic_uiom_wq); - destroy_workqueue(usnic_uiom_wq); -} diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.h b/drivers/infiniband/hw/usnic/usnic_uiom.h index b86a9731071b..c88cfa087e3a 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom.h +++ b/drivers/infiniband/hw/usnic/usnic_uiom.h @@ -93,5 +93,4 @@ struct usnic_uiom_reg *usnic_uiom_reg_get(struct usnic_uiom_pd *pd, void usnic_uiom_reg_release(struct usnic_uiom_reg *uiomr, struct ib_ucontext *ucontext); int usnic_uiom_init(char *drv_name); -void usnic_uiom_fini(void); #endif /* USNIC_UIOM_H_ */ From patchwork Mon Jan 21 17:42:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774265 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BC361139A for ; Mon, 21 Jan 2019 17:43:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A652029F0F for ; Mon, 21 Jan 2019 17:43:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9A56C2A6E0; Mon, 21 Jan 2019 17:43:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88F6829F0F for ; Mon, 21 Jan 2019 17:43:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8CE208E0009; Mon, 21 Jan 2019 12:43:17 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 899578E0001; Mon, 21 Jan 2019 12:43:17 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7B38E8E0008; Mon, 21 Jan 2019 12:43:17 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f198.google.com (mail-oi1-f198.google.com [209.85.167.198]) by kanga.kvack.org (Postfix) with ESMTP id 4EAD38E0001 for ; Mon, 21 Jan 2019 12:43:17 -0500 (EST) Received: by mail-oi1-f198.google.com with SMTP id s204so9916588oib.11 for ; Mon, 21 Jan 2019 09:43:17 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=waI1WmmPLoTveHPlnnfdMx1fmeBn1F1egAaxgnJemTY=; b=k+OTOC129RyfjIfiwkmbE8O2jhYAdlL8+SH5tnowww/4BtHh7T4EpG+cGqR/Itw7Fm YG+mNjAIfFCSJKaqELf7fY7YWKyZnNdRGpkbwN1uFyVOewTYLNzVOw8I/dEDmUuHen+n HeJXitetSPON3JwyGoBFwLlJ30PJmD7dDGLndBwoGd1Ovucjf3j1xRj0OSAThV7oGB+H T8jeQhvMAn+1gHzAhUMyK0sbvA98vAXto/xWLPZIH9IxlCcfCBjnO8zF/m97xhtyXq2n bC1TVeHflL01Q5HU+dFdvxiZQpc/ezfjJs7r8DgeYH5UWZUGE144e251ec4L9A84s5zY hG0Q== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukcFBZ686sPR5Bezx77OGimDg9gewFgV8laqy0Q1cKFnCCLYHwbo hZTRS57N2v88m+TYuBS9xTbV1VRtkd57HZjjz3Y/G5/kuaRwO6CJrShaF6+5E3WhIuUF00CrnmI n+lluSoTFkGfB/6+WV6/C8neP6AwB2utBzNTpKqe5pPy78QVs3wd1t7B4lmzff18= X-Received: by 2002:aca:293:: with SMTP id 141mr6428654oic.28.1548092597024; Mon, 21 Jan 2019 09:43:17 -0800 (PST) X-Google-Smtp-Source: ALg8bN5IHk/LGWr8bEZskkT3QG41qPw2zzJi8dOoV86y04otYgHZ4IlqIOD4VhWTPyQlxIKPf91w X-Received: by 2002:aca:293:: with SMTP id 141mr6428617oic.28.1548092595959; Mon, 21 Jan 2019 09:43:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092595; cv=none; d=google.com; s=arc-20160816; b=fFjTMcgbd2IGeOqXfgXjq71m+RZS3ozJFwdPjEhbq8kXYuPBjXKvzGpHee97180WZV jpg7kXJkiqgXY2Zkoaie+V95hqZ+/25jqt9sowVo+1/qlAcUpg6MYv+DOrOHZ8sePopo F8vFl7Gona1rJyYX2jjJeoS9qrC66B61b5d4NpvdnVM+GIbKRcgXIL1RquOz8BZMwDl6 aRl0D8AgXf9R774cHExbuajpYj9WDtKwNqr9cS1ozitqqQWpgENqKL6QJJVn74XWwrex kr03hekbj4PSKnY7EqPaiUe/E1yjG6CQql4BHk9T+5RoRrxuhCi/9UnR0WlwJQGRj/jX wMKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=waI1WmmPLoTveHPlnnfdMx1fmeBn1F1egAaxgnJemTY=; b=I3FXG9Fe7a0aGp9A9WSbffAIa1g22W7Cbcczz248+JZHep3SaZu9JMSo70I614NXEt K/8BmyCGxILO3g7skegLg16MSom7PWSCAP6tzK6iawywPHbJHtMFydRgOCXnS2v9dNIr cIo1Qj6iRlAId9Fs5hbgh/Zodb+53CYfhnjDZ8Q/511eHCEQLzOe3lzAGPNxUvJ3AJ8O 1e4PkVWXzxeyqtUHDEwQDDdvDRQajgHlS1k8/Qi+SJJGAXcsiNsjiQWPgAIA3rdMCqnC 7XGP+o6BkwHUKzzo6e5feV8SH2OCuDtCPI/YBcKcutZ7HnVV3MU4DoBrrF1EqLK4lIoY DMyw== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id r3si6200808oif.92.2019.01.21.09.43.15 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:43:15 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:43:07 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, Davidlohr Bueso Subject: [PATCH 6/6] drivers/IB,core: reduce scope of mmap_sem Date: Mon, 21 Jan 2019 09:42:20 -0800 Message-Id: <20190121174220.10583-7-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP ib_umem_get() uses gup_longterm() and relies on the lock to stabilze the vma_list, so we cannot really get rid of mmap_sem altogether, but now that the counter is atomic, we can get of some complexity that mmap_sem brings with only pinned_vm. Reviewed-by: Ira Weiny Signed-off-by: Davidlohr Bueso --- drivers/infiniband/core/umem.c | 41 ++--------------------------------------- 1 file changed, 2 insertions(+), 39 deletions(-) diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c index 678abe1afcba..b69d3efa8712 100644 --- a/drivers/infiniband/core/umem.c +++ b/drivers/infiniband/core/umem.c @@ -165,15 +165,12 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - down_write(&mm->mmap_sem); - new_pinned = atomic64_read(&mm->pinned_vm) + npages; + new_pinned = atomic64_add_return(npages, &mm->pinned_vm); if (new_pinned > lock_limit && !capable(CAP_IPC_LOCK)) { - up_write(&mm->mmap_sem); + atomic64_sub(npages, &mm->pinned_vm); ret = -ENOMEM; goto out; } - atomic64_set(&mm->pinned_vm, new_pinned); - up_write(&mm->mmap_sem); cur_base = addr & PAGE_MASK; @@ -233,9 +230,7 @@ struct ib_umem *ib_umem_get(struct ib_udata *udata, unsigned long addr, umem_release: __ib_umem_release(context->device, umem, 0); vma: - down_write(&mm->mmap_sem); atomic64_sub(ib_umem_num_pages(umem), &mm->pinned_vm); - up_write(&mm->mmap_sem); out: if (vma_list) free_page((unsigned long) vma_list); @@ -258,25 +253,12 @@ static void __ib_umem_release_tail(struct ib_umem *umem) kfree(umem); } -static void ib_umem_release_defer(struct work_struct *work) -{ - struct ib_umem *umem = container_of(work, struct ib_umem, work); - - down_write(&umem->owning_mm->mmap_sem); - atomic64_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); - up_write(&umem->owning_mm->mmap_sem); - - __ib_umem_release_tail(umem); -} - /** * ib_umem_release - release memory pinned with ib_umem_get * @umem: umem struct to release */ void ib_umem_release(struct ib_umem *umem) { - struct ib_ucontext *context = umem->context; - if (umem->is_odp) { ib_umem_odp_release(to_ib_umem_odp(umem)); __ib_umem_release_tail(umem); @@ -285,26 +267,7 @@ void ib_umem_release(struct ib_umem *umem) __ib_umem_release(umem->context->device, umem, 1); - /* - * We may be called with the mm's mmap_sem already held. This - * can happen when a userspace munmap() is the call that drops - * the last reference to our file and calls our release - * method. If there are memory regions to destroy, we'll end - * up here and not be able to take the mmap_sem. In that case - * we defer the vm_locked accounting a workqueue. - */ - if (context->closing) { - if (!down_write_trylock(&umem->owning_mm->mmap_sem)) { - INIT_WORK(&umem->work, ib_umem_release_defer); - queue_work(ib_wq, &umem->work); - return; - } - } else { - down_write(&umem->owning_mm->mmap_sem); - } atomic64_sub(ib_umem_num_pages(umem), &umem->owning_mm->pinned_vm); - up_write(&umem->owning_mm->mmap_sem); - __ib_umem_release_tail(umem); } EXPORT_SYMBOL(ib_umem_release);