From patchwork Tue Apr 2 20:41:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882321 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D182D184E for ; Tue, 2 Apr 2019 20:50:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD1D928856 for ; Tue, 2 Apr 2019 20:50:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9CCCE2846F; Tue, 2 Apr 2019 20:50:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4251B2846F for ; Tue, 2 Apr 2019 20:50:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4EB46B0275; Tue, 2 Apr 2019 16:50:30 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DFFB66B0276; Tue, 2 Apr 2019 16:50:30 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CA0496B0277; Tue, 2 Apr 2019 16:50:30 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f198.google.com (mail-it1-f198.google.com [209.85.166.198]) by kanga.kvack.org (Postfix) with ESMTP id A83AA6B0275 for ; Tue, 2 Apr 2019 16:50:30 -0400 (EDT) Received: by mail-it1-f198.google.com with SMTP id v11so2437865itb.1 for ; Tue, 02 Apr 2019 13:50:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=QrP2gOWfQDscn5FJYkHkueLx9eSDmR294rMG9/wfMMc=; b=nzkfElKXzMJs0U/O+r9eXGsffXh6RplJrzcaToVunhxqmEiAJMr8iLtjQuQVXo6lDq GnFEZenXrY/py3OQDZHcptdQVd7x32OQMiBgVsPSrVzgImoY7Na4gOpbU4N3KmBz65GH Ir5n86qc50s9iV2oAsPMfxq1Dgz4/5sXgW+CryFxJJvR4Oow5xQvrjpaXzdYaU+rI7wt 3nMqXUMi3tf44OY6oVhDYDRYan7dZSxF6zrl15/BnCDjilghbemFcJnqeKHx0JXB3rbm 4YVKLVO/28q/gAfepOaljEx1IcdUrhz11ztUgTipYW07+xw5V9JGCpw5XOqH9V4h9EA3 urfA== X-Gm-Message-State: APjAAAUYxSJh7QxvkT2kog+On/Cggy//1vOJRkXO2ALUcuqKa8oGg1bf 7l+kXwVkjX+x5C4e4c5N+QzUD5mclmnzUyb5JH6loUC4oC0qepnIZD2mrFM+P2MF8y0dsb8/6Yv Ot4kjUJpUG/2SBKfY5FNf8BFePylB5paDqw/MuO0EbFUU2VJN+DcB2pKFroLKiwz8Pw== X-Received: by 2002:a24:e089:: with SMTP id c131mr5723280ith.29.1554238230299; Tue, 02 Apr 2019 13:50:30 -0700 (PDT) X-Google-Smtp-Source: APXvYqxzZCFn4swj9bFVZ9xqlgw/wXd7aiJvPxaQ+TaAP4nCtsfmMYy6fmPRAH+FpMsN4fUe5mBo X-Received: by 2002:a24:e089:: with SMTP id c131mr5723209ith.29.1554238228770; Tue, 02 Apr 2019 13:50:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554238228; cv=none; d=google.com; s=arc-20160816; b=rlsA5G9+jCUzrKE9mfOZofZLoK/4IgPrpnlfxdSjIUwAuGf1NB/bUrwKGjjlN7muF1 +gjiLdxH3Pd7zyKDpJqTCq4Iy3ZZ5dmaloqyrve2muzcpMbUPVcDUCRHkIxU9MDdu1Dn KG729eENb1rH+eO4+5dpIZmWqYs5W869POmadDgjKnRM2Ei2xRoPfL0wNGbwjZ+lg92f woQCmlVCEH/dl+Ue6COoWp16DF6Rq275F1Ak2hA5kL7EpSolUpAv5Im/gTiuwOvcVKC4 1xunDnuKn3v64m626hPJDh4opW7JNp2R8Rr3AxXIVlYjacb5yvE1MlDs7nvOqtDwzd+W OVkw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=QrP2gOWfQDscn5FJYkHkueLx9eSDmR294rMG9/wfMMc=; b=UuxM/vRjzMgQPoN8ElGuZYxKfPDdH/rsrq7zpb4w+7H3NnoiPCgKi29aDZFbzqEALM Ps84rEN+UO56rGPaKpkch6KkJShF81RFYpvTjx6AbgFmqYx7kFarmfExf2XxsZJ0x1Lj J45fMwmXMJjZYz6OhN0Nu+Y7qSjJMiuTG/uk7mvy9f9yFzi3lPfjF8mufy/TfFRX3d7k CfNBkDPET7wcp/Ae33ZYUEL/eOHC9v/ufhH8S1f/rOvEgi54AujNJKPbPWd3iyH770CX Nx+Js4Bdsm+eivUWprBk7PEg+e0gVArvyROtnAovtunUGkLv2DeT4o6StnePT/n8Y4zm Ex2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="myGa/v1+"; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id 8si7156366jae.26.2019.04.02.13.50.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:50:28 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b="myGa/v1+"; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd5vJ164029; Tue, 2 Apr 2019 20:48:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=QrP2gOWfQDscn5FJYkHkueLx9eSDmR294rMG9/wfMMc=; b=myGa/v1+cXj9m6AltRx8x8N/Ns8VESUwT2Hc85+ihYiT5eC9brN+i+WnjTDEwVy1pYLn reN1Et2KHRyu3dwotNeaDwjahLaZ8mNrnn7GRFzF9B/j+MgAc7woMB4xsARkV/Yh0WAj 5AiIL7M3X+jgYeyLklCN86BBH0o1K06JROTDOe+CT7WdsmmgIzc0VVmKRoMa8tx3BEMv siJcvxFym81a3hQr36vYyL2jdIKr/FStDIeU+nJsAxTll+TOOM22gWMA7onCf9nEQ5Q5 5LE6WU5aUdfUl8VBml2GiCpsHLnB2fHhWdbddvvlT0XzudJEaCFeAxRI9CzOEoqTAd42 2w== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 2rj0dnm18u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:48:18 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfJvu103109; Tue, 2 Apr 2019 20:42:18 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 2rm9mhp3mn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:18 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x32KgFHa030189; Tue, 2 Apr 2019 20:42:15 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:14 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alan Tull , Alexey Kardashevskiy , Alex Williamson , Benjamin Herrenschmidt , Christoph Lameter , Davidlohr Bueso , Michael Ellerman , Moritz Fischer , Paul Mackerras , Wu Hao , linux-mm@kvack.org, kvm@vger.kernel.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/6] mm: change locked_vm's type from unsigned long to atomic64_t Date: Tue, 2 Apr 2019 16:41:53 -0400 Message-Id: <20190402204158.27582-2-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Taking and dropping mmap_sem to modify a single counter, locked_vm, is overkill when the counter could be synchronized separately. Make mmap_sem a little less coarse by changing locked_vm to an atomic, the 64-bit variety to avoid issues with overflow on 32-bit systems. Signed-off-by: Daniel Jordan Cc: Alan Tull Cc: Alexey Kardashevskiy Cc: Alex Williamson Cc: Andrew Morton Cc: Benjamin Herrenschmidt Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Michael Ellerman Cc: Moritz Fischer Cc: Paul Mackerras Cc: Wu Hao Cc: Cc: Cc: Cc: Cc: Cc: --- arch/powerpc/kvm/book3s_64_vio.c | 14 ++++++++------ arch/powerpc/mm/mmu_context_iommu.c | 15 ++++++++------- drivers/fpga/dfl-afu-dma-region.c | 18 ++++++++++-------- drivers/vfio/vfio_iommu_spapr_tce.c | 17 +++++++++-------- drivers/vfio/vfio_iommu_type1.c | 10 ++++++---- fs/proc/task_mmu.c | 2 +- include/linux/mm_types.h | 2 +- kernel/fork.c | 2 +- mm/debug.c | 5 +++-- mm/mlock.c | 4 ++-- mm/mmap.c | 18 +++++++++--------- mm/mremap.c | 6 +++--- 12 files changed, 61 insertions(+), 52 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c index f02b04973710..e7fdb6d10eeb 100644 --- a/arch/powerpc/kvm/book3s_64_vio.c +++ b/arch/powerpc/kvm/book3s_64_vio.c @@ -59,32 +59,34 @@ static unsigned long kvmppc_stt_pages(unsigned long tce_pages) static long kvmppc_account_memlimit(unsigned long stt_pages, bool inc) { long ret = 0; + s64 locked_vm; if (!current || !current->mm) return ret; /* process exited */ down_write(¤t->mm->mmap_sem); + locked_vm = atomic64_read(¤t->mm->locked_vm); if (inc) { unsigned long locked, lock_limit; - locked = current->mm->locked_vm + stt_pages; + locked = locked_vm + stt_pages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) ret = -ENOMEM; else - current->mm->locked_vm += stt_pages; + atomic64_add(stt_pages, ¤t->mm->locked_vm); } else { - if (WARN_ON_ONCE(stt_pages > current->mm->locked_vm)) - stt_pages = current->mm->locked_vm; + if (WARN_ON_ONCE(stt_pages > locked_vm)) + stt_pages = locked_vm; - current->mm->locked_vm -= stt_pages; + atomic64_sub(stt_pages, ¤t->mm->locked_vm); } pr_debug("[%d] RLIMIT_MEMLOCK KVM %c%ld %ld/%ld%s\n", current->pid, inc ? '+' : '-', stt_pages << PAGE_SHIFT, - current->mm->locked_vm << PAGE_SHIFT, + atomic64_read(¤t->mm->locked_vm) << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index e7a9c4f6bfca..8038ac24a312 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -55,30 +55,31 @@ static long mm_iommu_adjust_locked_vm(struct mm_struct *mm, unsigned long npages, bool incr) { long ret = 0, locked, lock_limit; + s64 locked_vm; if (!npages) return 0; down_write(&mm->mmap_sem); - + locked_vm = atomic64_read(&mm->locked_vm); if (incr) { - locked = mm->locked_vm + npages; + locked = locked_vm + npages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) ret = -ENOMEM; else - mm->locked_vm += npages; + atomic64_add(npages, &mm->locked_vm); } else { - if (WARN_ON_ONCE(npages > mm->locked_vm)) - npages = mm->locked_vm; - mm->locked_vm -= npages; + if (WARN_ON_ONCE(npages > locked_vm)) + npages = locked_vm; + atomic64_sub(npages, &mm->locked_vm); } pr_debug("[%d] RLIMIT_MEMLOCK HASH64 %c%ld %ld/%ld\n", current ? current->pid : 0, incr ? '+' : '-', npages << PAGE_SHIFT, - mm->locked_vm << PAGE_SHIFT, + atomic64_read(&mm->locked_vm) << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK)); up_write(&mm->mmap_sem); diff --git a/drivers/fpga/dfl-afu-dma-region.c b/drivers/fpga/dfl-afu-dma-region.c index e18a786fc943..08132fd9b6b7 100644 --- a/drivers/fpga/dfl-afu-dma-region.c +++ b/drivers/fpga/dfl-afu-dma-region.c @@ -45,6 +45,7 @@ void afu_dma_region_init(struct dfl_feature_platform_data *pdata) static int afu_dma_adjust_locked_vm(struct device *dev, long npages, bool incr) { unsigned long locked, lock_limit; + s64 locked_vm; int ret = 0; /* the task is exiting. */ @@ -53,24 +54,25 @@ static int afu_dma_adjust_locked_vm(struct device *dev, long npages, bool incr) down_write(¤t->mm->mmap_sem); + locked_vm = atomic64_read(¤t->mm->locked_vm); if (incr) { - locked = current->mm->locked_vm + npages; + locked = locked_vm + npages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) ret = -ENOMEM; else - current->mm->locked_vm += npages; + atomic64_add(npages, ¤t->mm->locked_vm); } else { - if (WARN_ON_ONCE(npages > current->mm->locked_vm)) - npages = current->mm->locked_vm; - current->mm->locked_vm -= npages; + if (WARN_ON_ONCE(npages > locked_vm)) + npages = locked_vm; + atomic64_sub(npages, ¤t->mm->locked_vm); } - dev_dbg(dev, "[%d] RLIMIT_MEMLOCK %c%ld %ld/%ld%s\n", current->pid, + dev_dbg(dev, "[%d] RLIMIT_MEMLOCK %c%ld %lld/%lu%s\n", current->pid, incr ? '+' : '-', npages << PAGE_SHIFT, - current->mm->locked_vm << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK), - ret ? "- exceeded" : ""); + (s64)atomic64_read(¤t->mm->locked_vm) << PAGE_SHIFT, + rlimit(RLIMIT_MEMLOCK), ret ? "- exceeded" : ""); up_write(¤t->mm->mmap_sem); diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index 8dbb270998f4..e7d787e5d839 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -36,7 +36,8 @@ static void tce_iommu_detach_group(void *iommu_data, static long try_increment_locked_vm(struct mm_struct *mm, long npages) { - long ret = 0, locked, lock_limit; + long ret = 0, lock_limit; + s64 locked; if (WARN_ON_ONCE(!mm)) return -EPERM; @@ -45,16 +46,16 @@ static long try_increment_locked_vm(struct mm_struct *mm, long npages) return 0; down_write(&mm->mmap_sem); - locked = mm->locked_vm + npages; + locked = atomic64_read(&mm->locked_vm) + npages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) ret = -ENOMEM; else - mm->locked_vm += npages; + atomic64_add(npages, &mm->locked_vm); pr_debug("[%d] RLIMIT_MEMLOCK +%ld %ld/%ld%s\n", current->pid, npages << PAGE_SHIFT, - mm->locked_vm << PAGE_SHIFT, + atomic64_read(&mm->locked_vm) << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); @@ -69,12 +70,12 @@ static void decrement_locked_vm(struct mm_struct *mm, long npages) return; down_write(&mm->mmap_sem); - if (WARN_ON_ONCE(npages > mm->locked_vm)) - npages = mm->locked_vm; - mm->locked_vm -= npages; + if (WARN_ON_ONCE(npages > atomic64_read(&mm->locked_vm))) + npages = atomic64_read(&mm->locked_vm); + atomic64_sub(npages, &mm->locked_vm); pr_debug("[%d] RLIMIT_MEMLOCK -%ld %ld/%ld\n", current->pid, npages << PAGE_SHIFT, - mm->locked_vm << PAGE_SHIFT, + atomic64_read(&mm->locked_vm) << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK)); up_write(&mm->mmap_sem); } diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 73652e21efec..5b2878697286 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -270,18 +270,19 @@ static int vfio_lock_acct(struct vfio_dma *dma, long npage, bool async) if (!ret) { if (npage > 0) { if (!dma->lock_cap) { + s64 locked_vm = atomic64_read(&mm->locked_vm); unsigned long limit; limit = task_rlimit(dma->task, RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (mm->locked_vm + npage > limit) + if (locked_vm + npage > limit) ret = -ENOMEM; } } if (!ret) - mm->locked_vm += npage; + atomic64_add(npage, &mm->locked_vm); up_write(&mm->mmap_sem); } @@ -401,6 +402,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, long ret, pinned = 0, lock_acct = 0; bool rsvd; dma_addr_t iova = vaddr - dma->vaddr + dma->iova; + atomic64_t *locked_vm = ¤t->mm->locked_vm; /* This code path is only user initiated */ if (!current->mm) @@ -418,7 +420,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, * pages are already counted against the user. */ if (!rsvd && !vfio_find_vpfn(dma, iova)) { - if (!dma->lock_cap && current->mm->locked_vm + 1 > limit) { + if (!dma->lock_cap && atomic64_read(locked_vm) + 1 > limit) { put_pfn(*pfn_base, dma->prot); pr_warn("%s: RLIMIT_MEMLOCK (%ld) exceeded\n", __func__, limit << PAGE_SHIFT); @@ -445,7 +447,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, if (!rsvd && !vfio_find_vpfn(dma, iova)) { if (!dma->lock_cap && - current->mm->locked_vm + lock_acct + 1 > limit) { + atomic64_read(locked_vm) + lock_acct + 1 > limit) { put_pfn(pfn, dma->prot); pr_warn("%s: RLIMIT_MEMLOCK (%ld) exceeded\n", __func__, limit << PAGE_SHIFT); diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index 92a91e7816d8..61da4b24d0e0 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -58,7 +58,7 @@ void task_mem(struct seq_file *m, struct mm_struct *mm) swap = get_mm_counter(mm, MM_SWAPENTS); SEQ_PUT_DEC("VmPeak:\t", hiwater_vm); SEQ_PUT_DEC(" kB\nVmSize:\t", total_vm); - SEQ_PUT_DEC(" kB\nVmLck:\t", mm->locked_vm); + SEQ_PUT_DEC(" kB\nVmLck:\t", atomic64_read(&mm->locked_vm)); SEQ_PUT_DEC(" kB\nVmPin:\t", atomic64_read(&mm->pinned_vm)); SEQ_PUT_DEC(" kB\nVmHWM:\t", hiwater_rss); SEQ_PUT_DEC(" kB\nVmRSS:\t", total_rss); diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 7eade9132f02..5059b99a0827 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -410,7 +410,7 @@ struct mm_struct { unsigned long hiwater_vm; /* High-water virtual memory usage */ unsigned long total_vm; /* Total pages mapped */ - unsigned long locked_vm; /* Pages that have PG_mlocked set */ + atomic64_t locked_vm; /* Pages that have PG_mlocked set */ atomic64_t pinned_vm; /* Refcount permanently increased */ unsigned long data_vm; /* VM_WRITE & ~VM_SHARED & ~VM_STACK */ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE & ~VM_STACK */ diff --git a/kernel/fork.c b/kernel/fork.c index 9dcd18aa210b..56be8cdc7b4a 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -979,7 +979,7 @@ static struct mm_struct *mm_init(struct mm_struct *mm, struct task_struct *p, mm->core_state = NULL; mm_pgtables_bytes_init(mm); mm->map_count = 0; - mm->locked_vm = 0; + atomic64_set(&mm->locked_vm, 0); atomic64_set(&mm->pinned_vm, 0); memset(&mm->rss_stat, 0, sizeof(mm->rss_stat)); spin_lock_init(&mm->page_table_lock); diff --git a/mm/debug.c b/mm/debug.c index eee9c221280c..b9cd71927d3c 100644 --- a/mm/debug.c +++ b/mm/debug.c @@ -136,7 +136,7 @@ void dump_mm(const struct mm_struct *mm) #endif "mmap_base %lu mmap_legacy_base %lu highest_vm_end %lu\n" "pgd %px mm_users %d mm_count %d pgtables_bytes %lu map_count %d\n" - "hiwater_rss %lx hiwater_vm %lx total_vm %lx locked_vm %lx\n" + "hiwater_rss %lx hiwater_vm %lx total_vm %lx locked_vm %llx\n" "pinned_vm %llx data_vm %lx exec_vm %lx stack_vm %lx\n" "start_code %lx end_code %lx start_data %lx end_data %lx\n" "start_brk %lx brk %lx start_stack %lx\n" @@ -167,7 +167,8 @@ void dump_mm(const struct mm_struct *mm) atomic_read(&mm->mm_count), mm_pgtables_bytes(mm), mm->map_count, - mm->hiwater_rss, mm->hiwater_vm, mm->total_vm, mm->locked_vm, + mm->hiwater_rss, mm->hiwater_vm, mm->total_vm, + (u64)atomic64_read(&mm->locked_vm), (u64)atomic64_read(&mm->pinned_vm), mm->data_vm, mm->exec_vm, mm->stack_vm, mm->start_code, mm->end_code, mm->start_data, mm->end_data, diff --git a/mm/mlock.c b/mm/mlock.c index 080f3b36415b..e492a155c51a 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -562,7 +562,7 @@ static int mlock_fixup(struct vm_area_struct *vma, struct vm_area_struct **prev, nr_pages = -nr_pages; else if (old_flags & VM_LOCKED) nr_pages = 0; - mm->locked_vm += nr_pages; + atomic64_add(nr_pages, &mm->locked_vm); /* * vm_flags is protected by the mmap_sem held in write mode. @@ -687,7 +687,7 @@ static __must_check int do_mlock(unsigned long start, size_t len, vm_flags_t fla if (down_write_killable(¤t->mm->mmap_sem)) return -EINTR; - locked += current->mm->locked_vm; + locked += atomic64_read(¤t->mm->locked_vm); if ((locked > lock_limit) && (!capable(CAP_IPC_LOCK))) { /* * It is possible that the regions requested intersect with diff --git a/mm/mmap.c b/mm/mmap.c index 41eb48d9b527..03576c1d530c 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1339,7 +1339,7 @@ static inline int mlock_future_check(struct mm_struct *mm, /* mlock MCL_FUTURE? */ if (flags & VM_LOCKED) { locked = len >> PAGE_SHIFT; - locked += mm->locked_vm; + locked += atomic64_read(&mm->locked_vm); lock_limit = rlimit(RLIMIT_MEMLOCK); lock_limit >>= PAGE_SHIFT; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) @@ -1825,7 +1825,7 @@ unsigned long mmap_region(struct file *file, unsigned long addr, vma == get_gate_vma(current->mm)) vma->vm_flags &= VM_LOCKED_CLEAR_MASK; else - mm->locked_vm += (len >> PAGE_SHIFT); + atomic64_add(len >> PAGE_SHIFT, &mm->locked_vm); } if (file) @@ -2301,7 +2301,7 @@ static int acct_stack_growth(struct vm_area_struct *vma, if (vma->vm_flags & VM_LOCKED) { unsigned long locked; unsigned long limit; - locked = mm->locked_vm + grow; + locked = atomic64_read(&mm->locked_vm) + grow; limit = rlimit(RLIMIT_MEMLOCK); limit >>= PAGE_SHIFT; if (locked > limit && !capable(CAP_IPC_LOCK)) @@ -2395,7 +2395,7 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address) */ spin_lock(&mm->page_table_lock); if (vma->vm_flags & VM_LOCKED) - mm->locked_vm += grow; + atomic64_add(grow, &mm->locked_vm); vm_stat_account(mm, vma->vm_flags, grow); anon_vma_interval_tree_pre_update_vma(vma); vma->vm_end = address; @@ -2475,7 +2475,7 @@ int expand_downwards(struct vm_area_struct *vma, */ spin_lock(&mm->page_table_lock); if (vma->vm_flags & VM_LOCKED) - mm->locked_vm += grow; + atomic64_add(grow, &mm->locked_vm); vm_stat_account(mm, vma->vm_flags, grow); anon_vma_interval_tree_pre_update_vma(vma); vma->vm_start = address; @@ -2796,11 +2796,11 @@ int __do_munmap(struct mm_struct *mm, unsigned long start, size_t len, /* * unlock any mlock()ed ranges before detaching vmas */ - if (mm->locked_vm) { + if (atomic64_read(&mm->locked_vm)) { struct vm_area_struct *tmp = vma; while (tmp && tmp->vm_start < end) { if (tmp->vm_flags & VM_LOCKED) { - mm->locked_vm -= vma_pages(tmp); + atomic64_sub(vma_pages(tmp), &mm->locked_vm); munlock_vma_pages_all(tmp); } @@ -3043,7 +3043,7 @@ static int do_brk_flags(unsigned long addr, unsigned long len, unsigned long fla mm->total_vm += len >> PAGE_SHIFT; mm->data_vm += len >> PAGE_SHIFT; if (flags & VM_LOCKED) - mm->locked_vm += (len >> PAGE_SHIFT); + atomic64_add(len >> PAGE_SHIFT, &mm->locked_vm); vma->vm_flags |= VM_SOFTDIRTY; return 0; } @@ -3115,7 +3115,7 @@ void exit_mmap(struct mm_struct *mm) up_write(&mm->mmap_sem); } - if (mm->locked_vm) { + if (atomic64_read(&mm->locked_vm)) { vma = mm->mmap; while (vma) { if (vma->vm_flags & VM_LOCKED) diff --git a/mm/mremap.c b/mm/mremap.c index e3edef6b7a12..9a4046bb2875 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -422,7 +422,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, } if (vm_flags & VM_LOCKED) { - mm->locked_vm += new_len >> PAGE_SHIFT; + atomic64_add(new_len >> PAGE_SHIFT, &mm->locked_vm); *locked = true; } @@ -473,7 +473,7 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, if (vma->vm_flags & VM_LOCKED) { unsigned long locked, lock_limit; - locked = mm->locked_vm << PAGE_SHIFT; + locked = atomic64_read(&mm->locked_vm) << PAGE_SHIFT; lock_limit = rlimit(RLIMIT_MEMLOCK); locked += new_len - old_len; if (locked > lock_limit && !capable(CAP_IPC_LOCK)) @@ -679,7 +679,7 @@ SYSCALL_DEFINE5(mremap, unsigned long, addr, unsigned long, old_len, vm_stat_account(mm, vma->vm_flags, pages); if (vma->vm_flags & VM_LOCKED) { - mm->locked_vm += pages; + atomic64_add(pages, &mm->locked_vm); locked = true; new_addr = addr; } From patchwork Tue Apr 2 20:41:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882301 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3212C139A for ; Tue, 2 Apr 2019 20:44:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1AA282878F for ; Tue, 2 Apr 2019 20:44:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0E2BF28800; Tue, 2 Apr 2019 20:44:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 645462878F for ; Tue, 2 Apr 2019 20:44:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52F786B0275; Tue, 2 Apr 2019 16:44:22 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 4B4AE6B0276; Tue, 2 Apr 2019 16:44:22 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 356EA6B0277; Tue, 2 Apr 2019 16:44:22 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-yw1-f69.google.com (mail-yw1-f69.google.com [209.85.161.69]) by kanga.kvack.org (Postfix) with ESMTP id 137436B0275 for ; Tue, 2 Apr 2019 16:44:22 -0400 (EDT) Received: by mail-yw1-f69.google.com with SMTP id x185so10715196ywd.4 for ; Tue, 02 Apr 2019 13:44:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=DAV8hZYjlHWVUW/ro2G5efrumxLnILCMzYubKdb6RAA=; b=h9+QKK5e4ma7Oj1InWfBUsNrJtnoRA0lucoja8HrdhSQODMYo4wCID8rn/97hh+uyH 2LBD9ncxQZe/Ip+6GRabzZ4L+TVnm2gyL5dT3lCOX9W1XtVFCf8SXz98H889abZwz25c N/NXBra75Yar8YZuH9vgjsNqcJ01eHWPlGYMjK0PAf5w0yVBtv6n56wgUZ6aoNlmFlNn olURUiaxrQcoXbSEAGsExrYTclMv2UgUStCTXKint77/WeFu9+q6O29os9aLMzI9BnBH G3Z6OuncHrH5yLZeihl59/UhW3Av8oaMCqgCowqiKKT3HKoCu7U0HK17wl1rlxuEie9s sTTw== X-Gm-Message-State: APjAAAVqBu7WaR7zSuqc/v83vLJZ0T7dMUZY1twt9yib5MCxyXjSoRdv uHnItETJ+ggsUWOECmj1dEjc9R7uisY8sYVkIV/5lorCPDPi2VdRd3b43YmZRtpR7XYv7U5zNKo IF6iNY3XKw2jqJa/vuEx5SHJ1+9/+SYMxjDL16ElTMnJZ4fEQ/TjKn6paeY36ap86fg== X-Received: by 2002:a25:a049:: with SMTP id x67mr25639577ybh.3.1554237861801; Tue, 02 Apr 2019 13:44:21 -0700 (PDT) X-Google-Smtp-Source: APXvYqxYMK6fHsqAN/9502Vts9qe2d4rPEYnPWSJV4htTLAJwLCWuq0IoDFj+JVL+bWMUICYO8Tm X-Received: by 2002:a25:a049:: with SMTP id x67mr25639515ybh.3.1554237860513; Tue, 02 Apr 2019 13:44:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554237860; cv=none; d=google.com; s=arc-20160816; b=x77NuNpGRpSZscmDjWbcl4JSsn8hSmQ6fucI3Q5LPuvkcU/eSVlMqC5ast3y+KRWKt mhG4+ASg4+1PP02HfaM8GRYHPfm2honi5vFsBxnMwWSZj2X12LZVZ/pNATl0s+kYwjjP 5yjijZUsRBmHZy2B1owaOx2dcd6gO4cE/x6FY9WEE9A3OYnj0OxccbgneIQkgZpsAR6e Q9Ilmn9EiBvJ/zT0qS5frSC5iqu/HcXQflBb8Y8FKygqAmZevK0aISR5pE2TIn5CZL7c HSV47QZc8i2XXJ92s7XQS4MnK8cX5iZeuYmkSuq6QC8vguIBrjmMEhgnM6dktQhtXqWu BWFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=DAV8hZYjlHWVUW/ro2G5efrumxLnILCMzYubKdb6RAA=; b=Q8D+2/OQV8WQ7fNwyPTE85WylOEneYbPs5B5YAVY5dxvONneN8c58HEKkuA3Rriu+B 4DYqbpHd9+9FpwbUbzCDCwTVoYCnPfQIPjT0o872C4GFedcMfSt3aNFdFvgNW11fyllR GXVXcOP3KNWKERqGC8QXqqIoIrHwBrU4Ngdi1u88aHSVdmL2x9PAh8mxy2kQanMPYJv/ 3Xx/xOIe9/lC7DTQcNoA0EnLsjDbbuF7kXkkqxiHFvhbWeVf1aTD1UOp23TWelciHjar Rf/yM5hMw/JEwOFdee6K++uLrcdiJzT26ej1+YoG36JY1cTwl4/sw9rJq+QUCPw0FWdj 7dGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=DYQqFz1E; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id 188si8721685ybr.271.2019.04.02.13.44.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:44:20 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=DYQqFz1E; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd3AR163985; Tue, 2 Apr 2019 20:44:18 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=DAV8hZYjlHWVUW/ro2G5efrumxLnILCMzYubKdb6RAA=; b=DYQqFz1Es0eTsjHefSJinfHkZj9GG0R6x09s5f53oykcMd5iF0EII+BlUpn+1XKZyqpy ehpuKE1KPh8f8+SX199aDH/x+N+c4l1g9WFEU5iJKbb/WhMXAR/+JG3Jtrq7vyhLEzuS yj7xGVmGvwzZdzDFVKtrc2EK1k9cDlWim983X0ss0owuMARRJf/y4yPvXj/O2j6nRI8d 69y1hsXZe8f7O5frVrxqBnD9WhKwIdQB0yL8z2p5IwSrNFxRb/FBjMXEZfuExAqgXB8I NEDh13MtMZ+JfeVS5BTlZdVsZxd8eJpF+zupdafHyI8+jDyhML92kzB4j6V4IQwvewMS 9g== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 2rj0dnm0b9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:44:18 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfI5b103059; Tue, 2 Apr 2019 20:42:18 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 2rm9mhp3mp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:17 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x32KgHK2029977; Tue, 2 Apr 2019 20:42:17 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:17 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alex Williamson , Christoph Lameter , Davidlohr Bueso , linux-mm@kvack.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] vfio/type1: drop mmap_sem now that locked_vm is atomic Date: Tue, 2 Apr 2019 16:41:54 -0400 Message-Id: <20190402204158.27582-3-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With locked_vm now an atomic, there is no need to take mmap_sem as writer. Delete and refactor accordingly. Signed-off-by: Daniel Jordan Cc: Alex Williamson Cc: Andrew Morton Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Cc: Cc: --- drivers/vfio/vfio_iommu_type1.c | 27 +++++++++------------------ 1 file changed, 9 insertions(+), 18 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 5b2878697286..a227de6d9c4c 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -257,7 +257,8 @@ static int vfio_iova_put_vfio_pfn(struct vfio_dma *dma, struct vfio_pfn *vpfn) static int vfio_lock_acct(struct vfio_dma *dma, long npage, bool async) { struct mm_struct *mm; - int ret; + s64 locked_vm; + int ret = 0; if (!npage) return 0; @@ -266,25 +267,15 @@ static int vfio_lock_acct(struct vfio_dma *dma, long npage, bool async) if (!mm) return -ESRCH; /* process exited */ - ret = down_write_killable(&mm->mmap_sem); - if (!ret) { - if (npage > 0) { - if (!dma->lock_cap) { - s64 locked_vm = atomic64_read(&mm->locked_vm); - unsigned long limit; - - limit = task_rlimit(dma->task, - RLIMIT_MEMLOCK) >> PAGE_SHIFT; + locked_vm = atomic64_add_return(npage, &mm->locked_vm); - if (locked_vm + npage > limit) - ret = -ENOMEM; - } + if (npage > 0 && !dma->lock_cap) { + unsigned long limit = task_rlimit(dma->task, RLIMIT_MEMLOCK) >> + PAGE_SHIFT; + if (locked_vm > limit) { + atomic64_sub(npage, &mm->locked_vm); + ret = -ENOMEM; } - - if (!ret) - atomic64_add(npage, &mm->locked_vm); - - up_write(&mm->mmap_sem); } if (async) From patchwork Tue Apr 2 20:41:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882303 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3738D139A for ; Tue, 2 Apr 2019 20:44:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 240342878F for ; Tue, 2 Apr 2019 20:44:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 177C528800; Tue, 2 Apr 2019 20:44:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 12DF42878F for ; Tue, 2 Apr 2019 20:44:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E30A66B0276; Tue, 2 Apr 2019 16:44:24 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DB6B66B0277; Tue, 2 Apr 2019 16:44:24 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C5BB96B0278; Tue, 2 Apr 2019 16:44:24 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f197.google.com (mail-it1-f197.google.com [209.85.166.197]) by kanga.kvack.org (Postfix) with ESMTP id 9B3856B0276 for ; Tue, 2 Apr 2019 16:44:24 -0400 (EDT) Received: by mail-it1-f197.google.com with SMTP id e124so4018585ita.4 for ; Tue, 02 Apr 2019 13:44:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=S3/NdFYbeXH44L9JBwqlhAYuzDTUjFxD7HwZ/en50So=; b=qcody47LZS1HlHFOfQZH4Wg3rFSiHMcc8tabb19cyjSIhhyycWEhxFe/6VHnlna5fn Tbj4aMkTxc3wITH5A5ZN+nECHjtk57KcJ/xvqCbh/SQtRy7ItdehjQyKiPyGZ2K54zcc zrI1vjXCaBIjPfSL/TOl51k4CYKjQzxyNvQikkRCv8g3H+HOrtgWwD1HVZNjTGWwowjQ mp3qYr3Enu/1+J90QjdUKcmlTj7G/aEUPp6SEVMThecDE8dpa22T1WFbYESBvAY9Ekj6 sCAb7hr4ihTA5OEyHxlhJlQR5XzgAtmmdr9fZXxHL8QjAjEj9AWUQkRkUNaBcj091UVk KqJQ== X-Gm-Message-State: APjAAAW62KqnJ0flNz/l0FYi21jXSM0yI2kw5dlNnoi2I9FzHIvirALr btpZe5ziutlw2JHxMxSm5TOf2a9vFMTfBX+7rsc7DRB8PmsOOFNEnW7bCcpj1QGWdQ1MI9WWlaq +U66PmByZNkfzlU619U5fEvyF3txBlXq1jJTqKypsdFAWpQ4ZxeUKTsfN5Mbu8QW9Wg== X-Received: by 2002:a24:1f50:: with SMTP id d77mr5755590itd.25.1554237864403; Tue, 02 Apr 2019 13:44:24 -0700 (PDT) X-Google-Smtp-Source: APXvYqz04hEuKEWCX445WVrjU98lllzT9ofzZXSnF/Pscv5AGeYYEmMFgaEomRUlkZ76ep83N/vQ X-Received: by 2002:a24:1f50:: with SMTP id d77mr5755537itd.25.1554237863287; Tue, 02 Apr 2019 13:44:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554237863; cv=none; d=google.com; s=arc-20160816; b=PKpf/W3nYLTHPrFKmSK+8QnTUs/Ppt25T4y28AS6x0i8vjocwqLk8qwre1CZ6dGtBl STAA0N5qcK06vtuZjDahFusjzqGom301pjOodA7uU5f9ZErKBhJesTjLkuBa5UMmSUqy dmPI92r1XDyUCKtvXif+VmcINKrRVnV4EZ7kDS+1/mzoVuNQbugcGDj7o12nAOWR0OQT juINodSEQ1voyVMUX4OdrxG0138CzeotGDnUOkAp6OMMNBMhsxwmimNSCDegZ4sOae6E EJH5bCzHMSXFlZA5DFBU6zHAU35GyW+FMJYDmRZo+hmmoDe0+MnxZTIRw7vZQK6dBKig 4JIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=S3/NdFYbeXH44L9JBwqlhAYuzDTUjFxD7HwZ/en50So=; b=XjpSYlqsQHvmKz1R3kVI2x6c8GGCF6HbrMEox++k2fmLwxrV1m4ZjDZLSWQLp3/D0j XEGPfbMV4TxDyMCcP0EcewMw6Tg6xUPGy31C/1zofjVS2wYom7jrSoBiVghdNF7AYLgt EutIhGnx24fmLuZYxN0ucWYZhBWXxaIRNSHkwFc5r/VvGZPpnGqAT0q4pUXc8cuiN8Oh TQ4D5jK9PPCPxx57+e3t88DhtqoC9LcMf9cEsqgr8IxulNK2zKEaWDNOrCWKZLhGwLrl o8IZ+HWkIWITet0L3buTAj3Er1wxcup9AZ+r+iig4nuTJOgCEdc2azAtVg7HgKY+Wm/y r7FA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=zHZtVFnZ; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id u186si6821500ioe.104.2019.04.02.13.44.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:44:23 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=zHZtVFnZ; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd5We164028; Tue, 2 Apr 2019 20:44:21 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=S3/NdFYbeXH44L9JBwqlhAYuzDTUjFxD7HwZ/en50So=; b=zHZtVFnZgKoe6ZZd0qgP5F20qgLYeOWz6WFuuVG1k0+N66MfV1nudJiuz+2DI13M9n+/ o8CI9whu2KiVn5rsBvVo65JdlCfmh/T4jcsa/xM75AbqoeT7a4yrN2L2Bng8UYCQ6a2H De5IK40aPpSq6rT8IeBkUYHuDbtluEkDJy2L7Dltwn9fOJZ6KxL0Rk012ZdOtvzjspr1 ZzGaDPdeUr4ec142YXkU37CGcy1HoStLQN7kJRtmodw6pDGhK5iwrUlP8utTFopsOkcs B4H4S1vHoByx7sZUZwJ2bDnQ8wcEYq95jACbPC/r64EKK7ZZzW4zat4swwLjYtWQhc+g bA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 2rj0dnm0bj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:44:21 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfGOl064850; Tue, 2 Apr 2019 20:42:21 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 2rm8f4yjyv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:20 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x32KgJlj031428; Tue, 2 Apr 2019 20:42:19 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:19 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alexey Kardashevskiy , Alex Williamson , Christoph Lameter , Davidlohr Bueso , linux-mm@kvack.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/6] vfio/spapr_tce: drop mmap_sem now that locked_vm is atomic Date: Tue, 2 Apr 2019 16:41:55 -0400 Message-Id: <20190402204158.27582-4-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=29 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With locked_vm now an atomic, there is no need to take mmap_sem as writer. Delete and refactor accordingly. Signed-off-by: Daniel Jordan Cc: Alexey Kardashevskiy Cc: Alex Williamson Cc: Andrew Morton Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Cc: Cc: --- drivers/vfio/vfio_iommu_spapr_tce.c | 36 ++++++++++++----------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/vfio/vfio_iommu_spapr_tce.c b/drivers/vfio/vfio_iommu_spapr_tce.c index e7d787e5d839..7675a3b28410 100644 --- a/drivers/vfio/vfio_iommu_spapr_tce.c +++ b/drivers/vfio/vfio_iommu_spapr_tce.c @@ -36,8 +36,9 @@ static void tce_iommu_detach_group(void *iommu_data, static long try_increment_locked_vm(struct mm_struct *mm, long npages) { - long ret = 0, lock_limit; + long ret = 0; s64 locked; + unsigned long lock_limit; if (WARN_ON_ONCE(!mm)) return -EPERM; @@ -45,39 +46,32 @@ static long try_increment_locked_vm(struct mm_struct *mm, long npages) if (!npages) return 0; - down_write(&mm->mmap_sem); - locked = atomic64_read(&mm->locked_vm) + npages; + locked = atomic64_add_return(npages, &mm->locked_vm); lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked > lock_limit && !capable(CAP_IPC_LOCK)) + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { ret = -ENOMEM; - else - atomic64_add(npages, &mm->locked_vm); - - pr_debug("[%d] RLIMIT_MEMLOCK +%ld %ld/%ld%s\n", current->pid, - npages << PAGE_SHIFT, - atomic64_read(&mm->locked_vm) << PAGE_SHIFT, - rlimit(RLIMIT_MEMLOCK), - ret ? " - exceeded" : ""); + atomic64_sub(npages, &mm->locked_vm); + } - up_write(&mm->mmap_sem); + pr_debug("[%d] RLIMIT_MEMLOCK +%ld %lld/%lu%s\n", current->pid, + npages << PAGE_SHIFT, locked << PAGE_SHIFT, + lock_limit, ret ? " - exceeded" : ""); return ret; } static void decrement_locked_vm(struct mm_struct *mm, long npages) { + s64 locked; + if (!mm || !npages) return; - down_write(&mm->mmap_sem); - if (WARN_ON_ONCE(npages > atomic64_read(&mm->locked_vm))) - npages = atomic64_read(&mm->locked_vm); - atomic64_sub(npages, &mm->locked_vm); - pr_debug("[%d] RLIMIT_MEMLOCK -%ld %ld/%ld\n", current->pid, - npages << PAGE_SHIFT, - atomic64_read(&mm->locked_vm) << PAGE_SHIFT, + locked = atomic64_sub_return(npages, &mm->locked_vm); + WARN_ON_ONCE(locked < 0); + pr_debug("[%d] RLIMIT_MEMLOCK -%ld %lld/%lu\n", current->pid, + npages << PAGE_SHIFT, locked << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK)); - up_write(&mm->mmap_sem); } /* From patchwork Tue Apr 2 20:41:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E08FF139A for ; Tue, 2 Apr 2019 20:44:31 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CB5CA2878F for ; Tue, 2 Apr 2019 20:44:31 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BDD8128800; Tue, 2 Apr 2019 20:44:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2F0D82878F for ; Tue, 2 Apr 2019 20:44:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B49CD6B0278; Tue, 2 Apr 2019 16:44:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id AA7866B0279; Tue, 2 Apr 2019 16:44:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 94F9D6B027A; Tue, 2 Apr 2019 16:44:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-it1-f197.google.com (mail-it1-f197.google.com [209.85.166.197]) by kanga.kvack.org (Postfix) with ESMTP id 63CB56B0278 for ; Tue, 2 Apr 2019 16:44:29 -0400 (EDT) Received: by mail-it1-f197.google.com with SMTP id 190so3996325itv.3 for ; Tue, 02 Apr 2019 13:44:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=SWwbtSuBuVRAsBvShPwVRYWt3xedhjkFQ6ROdmYUzro=; b=enwPbXTNGLuJBl7hEfF49lF8qDuKteghTf+0SEmHUzBxt2wqWqsb2R4QZKPpTkYkWW VNDqcUHc9whvZBfaKD7mD0tElq7QdOhWxnR3dMLefzoRd/DUne6RRBWaGnNO7bayTWXV Xi1Vr35KAYdSV02v9IB5jHLP2u29GIu9LWl0d06FQi1QTITS5YmQMMCxXV8dkIUNxkW4 iiSSgkN2c8roMOm76AuDT9vThPLJi/OPs56n0A66OpvLKfOFg5dZ5/MvorTv6Dre/l2C LmcCFZMK20afxUkBsQdr5AtXbqc5nnaPLrjzufnDo7ttMpiqoLb6b7JN1N2XMyXrgWVm tf7Q== X-Gm-Message-State: APjAAAXIqNxc9A+MjCBHf6GDA3D8NV8p3w4MO7T56CXRvK1mzppS7TIR Z8HkJNtyfM/PfA+o/QJKj/7FzC2S+5mGxS5I3XqBRNr9BaVpRhE4wpYhHlvHEfA0RffK+bx8QKg cWZGXmsNzJp6secix/iadUkpr27RSVwIM3MEWtHUbWHPc0h6KQBUDjOlR6ExVUJB8ew== X-Received: by 2002:a6b:cd89:: with SMTP id d131mr47440300iog.213.1554237869147; Tue, 02 Apr 2019 13:44:29 -0700 (PDT) X-Google-Smtp-Source: APXvYqxwC5OeuJbiVc0vSfNCLYafXdUtFwCF9i1KRvBf9dcwItabUjvC83Z+hlVL6HdVySbJsmO8 X-Received: by 2002:a6b:cd89:: with SMTP id d131mr47440261iog.213.1554237868071; Tue, 02 Apr 2019 13:44:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554237868; cv=none; d=google.com; s=arc-20160816; b=y+ZY5tNnOJDLPsvWPuaCLY5Y9xHSB5ja8igNm0gPyniLlkTb49vj8hRH0XRAsmlirc cjCwoq5yHTMK3+QN43ZGbBcQqc+juyjhrnry6hQeYKojR7M+JY3fJXXUr6EeZKmUMovy VRNmXDzeWFnoCTnKMHY5cnh4kvYUSShRKjzWepBXyi8bnBW8hQx8YbsV62hbpnxc1elQ LbskrkL+x2+0EbPEXO/iSfL75p+V5Cmeegt79R/SIkkvydtkmzzIa2sNVeAiak8YYfGM bGj9+ouiFW99pjVL1vQeXI+cfbqcuLJrAKO+LiM4GnCXhPY8Rhfayx7qkOe8lWpFXXa8 8BEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=SWwbtSuBuVRAsBvShPwVRYWt3xedhjkFQ6ROdmYUzro=; b=RQRSplMgdPi0Gay30yPQ7IOzvysPO5opCpwzDr4cIFg/DdPbfDxkrLeVA4h8NbJ4GQ h+GAVo1gYKheHsntH0gr/edHrZNhk0QoE7cA9zxnMxaFs6cmlhYjyFqUTUB9mk2WlQbS bb+pv7HQhuL7bxI9+8VZkvGXwisyqye4WiVshiOtQWJwJBMvQIKUcOIoaikeE13+8Jzh vHC88pxzTqq9np8ky+I35CYTuoXIDSiAguvnZ2UdhjYMEPPWq8Alfk4a31WROQUtHUHD OnqwJ1o/IStWBkCkBMccPUWuVi6C0+SuUG7Qdwad3ZLh61QFmA1mNLFxrcuKkyb5WZpO OiqQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ZHUgm2Mu; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id z14si6780471ioj.131.2019.04.02.13.44.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:44:28 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=ZHUgm2Mu; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd3oq163977; Tue, 2 Apr 2019 20:44:23 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=SWwbtSuBuVRAsBvShPwVRYWt3xedhjkFQ6ROdmYUzro=; b=ZHUgm2MuOOV0krty4T7QXlZIWISL8Smr5VxapxX5M+iGYwDtJrdgceBsKxYJad72jdFs WS6PXNlId7SQt1sMM422gZWgqsqdoOXN4TuZV9XoD+5A4MgP2ZHg4kYngmD/wvkJ+Xgj OcxQ0Ed7eusFEwspi+mZ4o+M15eUeGihEv2NnqjMkMR9M/SgYecFbMS2YxvOnsYMm27K DbDjihLSkP5+pHWFMCMIJK0R0jhEKczACn8l1Bgh/JSxMplw+viZEOvNtNzKtkwi8JNd G8lMfgtiWXb7/DUUffMG3VNE5jGReclP09wmyGjuhYIpBk/SM3T6LIXX7CzgD5iDuFEY xw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 2rj0dnm0bs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:44:23 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfJxi065003; Tue, 2 Apr 2019 20:42:23 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by aserp3030.oracle.com with ESMTP id 2rm8f4yk08-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:22 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x32KgLio031444; Tue, 2 Apr 2019 20:42:21 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:21 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alan Tull , Christoph Lameter , Davidlohr Bueso , Moritz Fischer , Wu Hao , linux-mm@kvack.org, linux-fpga@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/6] fpga/dlf/afu: drop mmap_sem now that locked_vm is atomic Date: Tue, 2 Apr 2019 16:41:56 -0400 Message-Id: <20190402204158.27582-5-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=3 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With locked_vm now an atomic, there is no need to take mmap_sem as writer. Delete and refactor accordingly. Signed-off-by: Daniel Jordan Cc: Alan Tull Cc: Andrew Morton Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Moritz Fischer Cc: Wu Hao Cc: Cc: Cc: --- drivers/fpga/dfl-afu-dma-region.c | 40 ++++++++++++------------------- 1 file changed, 15 insertions(+), 25 deletions(-) diff --git a/drivers/fpga/dfl-afu-dma-region.c b/drivers/fpga/dfl-afu-dma-region.c index 08132fd9b6b7..81e3e3a71758 100644 --- a/drivers/fpga/dfl-afu-dma-region.c +++ b/drivers/fpga/dfl-afu-dma-region.c @@ -35,46 +35,36 @@ void afu_dma_region_init(struct dfl_feature_platform_data *pdata) * afu_dma_adjust_locked_vm - adjust locked memory * @dev: port device * @npages: number of pages - * @incr: increase or decrease locked memory * * Increase or decrease the locked memory size with npages input. * * Return 0 on success. * Return -ENOMEM if locked memory size is over the limit and no CAP_IPC_LOCK. */ -static int afu_dma_adjust_locked_vm(struct device *dev, long npages, bool incr) +static int afu_dma_adjust_locked_vm(struct device *dev, long pages) { - unsigned long locked, lock_limit; + unsigned long lock_limit; s64 locked_vm; int ret = 0; /* the task is exiting. */ - if (!current->mm) + if (!current->mm || !pages) return 0; - down_write(¤t->mm->mmap_sem); - - locked_vm = atomic64_read(¤t->mm->locked_vm); - if (incr) { - locked = locked_vm + npages; + locked_vm = atomic64_add_return(pages, ¤t->mm->locked_vm); + WARN_ON_ONCE(locked_vm < 0); + if (pages > 0 && !capable(CAP_IPC_LOCK)) { lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - if (locked > lock_limit && !capable(CAP_IPC_LOCK)) + if (locked_vm > lock_limit) { ret = -ENOMEM; - else - atomic64_add(npages, ¤t->mm->locked_vm); - } else { - if (WARN_ON_ONCE(npages > locked_vm)) - npages = locked_vm; - atomic64_sub(npages, ¤t->mm->locked_vm); + atomic64_sub(pages, ¤t->mm->locked_vm); + } } dev_dbg(dev, "[%d] RLIMIT_MEMLOCK %c%ld %lld/%lu%s\n", current->pid, - incr ? '+' : '-', npages << PAGE_SHIFT, - (s64)atomic64_read(¤t->mm->locked_vm) << PAGE_SHIFT, - rlimit(RLIMIT_MEMLOCK), ret ? "- exceeded" : ""); - - up_write(¤t->mm->mmap_sem); + (pages > 0) ? '+' : '-', pages << PAGE_SHIFT, + locked_vm << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK), + ret ? "- exceeded" : ""); return ret; } @@ -94,7 +84,7 @@ static int afu_dma_pin_pages(struct dfl_feature_platform_data *pdata, struct device *dev = &pdata->dev->dev; int ret, pinned; - ret = afu_dma_adjust_locked_vm(dev, npages, true); + ret = afu_dma_adjust_locked_vm(dev, npages); if (ret) return ret; @@ -123,7 +113,7 @@ static int afu_dma_pin_pages(struct dfl_feature_platform_data *pdata, free_pages: kfree(region->pages); unlock_vm: - afu_dma_adjust_locked_vm(dev, npages, false); + afu_dma_adjust_locked_vm(dev, -npages); return ret; } @@ -143,7 +133,7 @@ static void afu_dma_unpin_pages(struct dfl_feature_platform_data *pdata, put_all_pages(region->pages, npages); kfree(region->pages); - afu_dma_adjust_locked_vm(dev, npages, false); + afu_dma_adjust_locked_vm(dev, -npages); dev_dbg(dev, "%ld pages unpinned\n", npages); } From patchwork Tue Apr 2 20:41:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882311 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5B98F922 for ; Tue, 2 Apr 2019 20:44:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4425E2852E for ; Tue, 2 Apr 2019 20:44:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 387D92878F; Tue, 2 Apr 2019 20:44:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E64292852E for ; Tue, 2 Apr 2019 20:44:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B486A6B027A; Tue, 2 Apr 2019 16:44:40 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id ACFC66B027B; Tue, 2 Apr 2019 16:44:40 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 971CA6B027C; Tue, 2 Apr 2019 16:44:40 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) by kanga.kvack.org (Postfix) with ESMTP id 78BB86B027A for ; Tue, 2 Apr 2019 16:44:40 -0400 (EDT) Received: by mail-io1-f69.google.com with SMTP id e126so11760838ioa.8 for ; Tue, 02 Apr 2019 13:44:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=fAYAxmDQUAFF/V4uXClSMnViHRm6i0GISHNgrEn8/T0=; b=nYyp5NHLktVn6P1cfxUZUL94qwHXcjVvqsRleKpVm4h0NsjM9NCtTSzMyyq7e9wb19 lrYUU3ClkvTGN2UimrWJIqNFV3NLToQjwdM/qJ50el8UlOhpddfYVKoZsJT1ow0M7gSz Ys8IjJFnJ5x5ZSpPAv+6jfBR9gKQdXXR8GRG7acySL38b7Y1L/rpUGJNcXxilvqN2yZ7 qXZMOXoDPFJ1kvrCzrI7MblDg5Mlc0U9KboeBOkMrtphIzaVmIXNuXGPE2ZThXgGMYS7 UegdiiRy3XxhydsM3JPYp2C34Dc4wpVczWW/QxJ1h7luEbb0IhDokgR56INy4tE0jfQo 3URA== X-Gm-Message-State: APjAAAU9uTcuzS22flr/e6dXrEwjmjIYQI6oUgFELGtZeeR89EKU8GGY A0w+9+4K8MaIUyLyHASjVUy6gCG4QEGWD3fyYjZtSyAxB2C/qQlU17Av58ysPzGfiuyXl8tV/Kx MYi2lHAHQNhG0XsFYu0ji97LXSniFGpmXDu1/l8QdgFGPVwzQPKWC/KF473jvetqK/g== X-Received: by 2002:a6b:7108:: with SMTP id q8mr32709347iog.85.1554237880270; Tue, 02 Apr 2019 13:44:40 -0700 (PDT) X-Google-Smtp-Source: APXvYqwKZQ3XEXcr3F/nlh9kFRrHnmok/ZGXhEExvmSfe/9psibXI2u0e152KwSTVK+S+KBGcytt X-Received: by 2002:a6b:7108:: with SMTP id q8mr32709298iog.85.1554237879003; Tue, 02 Apr 2019 13:44:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554237879; cv=none; d=google.com; s=arc-20160816; b=l3pKmYd/GG1srds0k8zYolkg4MD42+ZpwuNsBj+6d9B1uw1dibdfaHCOwQZYIPvCtN HZLjGFMlpwgoCkelOV0WOeD+1QbcfCDBheckAIyVhVijYWNHKM6tYyl53KD0K1kc9msa B0GIhzQZFyY1H4ahBvVDvJALfH59v4SRkW35VKtZL/AgZZVLiC5ecXNCHo4vsTBc0m24 Z+yaZzTM0uxuf+6ihuxbeKNbBJQjbY5Nhf8Gv4jfZs7f1iCFAeXKItZ6ynvZrKti8kFm eTGe6aAuej3VeG7HjV96Lyx8MtnbFxJWY0pHlbSaHym2Rw/IG/KnnTqu5cBbDZ6gGPIw HnIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=fAYAxmDQUAFF/V4uXClSMnViHRm6i0GISHNgrEn8/T0=; b=O3EHHd1VIAK57nB1PVj4pME+GaFEVsTwWMo1rNYXFGKGrdDQOuFbzLlZ0KMaYngIvg WhAQuPk6t8pECucEo9XxyUJyDx8ywc4iwA4UtGwV3vYHUpa39S87gE6/yQonD01054wj KqOWvOS2R9Jk4kUWpMDwTAIu42TQKlwZamKRqvTo8P0ryBEc28OLDKMF87REhXMWFRE4 P+VhosFs724io3A3L2pyg5EMW9DEVl/kslAjBD46R8RKoA53ogEUG04yrb3OQN5RmeP1 pl2dSP76YPPR+HensvWPml16FKaoJ2HDxAUOmCquSP07QMOE3jevRnxGUSaJJc0G+vTT CnXg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=1Db4i7+5; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id m2si6936875ioj.9.2019.04.02.13.44.38 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:44:38 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=1Db4i7+5; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd3or163977; Tue, 2 Apr 2019 20:44:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=fAYAxmDQUAFF/V4uXClSMnViHRm6i0GISHNgrEn8/T0=; b=1Db4i7+5QTdvuvzmOABCOx/rKduKmOFCPGRVprrFpS4dMFTzLojfS7ykp8TBYOzgp6zY 8P35z5ZLaWeBPaHuenEidBxoo2DL5tmyrNWrdyFhv4XeDZAq5KtPCKKdaA2NY1HjuW0b UuvnV3+uYld/y+6agytQUQm2fU3HwT7sOULBUeaR5XyYYSJrmn85pitVdxeCDkBtHkan Eejglhyb4LcN/RILi+Mo3LVfwShqb//q1rCvklXxwVJuxd0TCmDsWlN0sCjdC000J1cN BT8XsWnEWY+bD0/jjXtxx7WlF8eSrJlpiSlwZhHMb1njCz6VzdqdcpH6zqpx3ASn5w7L 6w== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 2rj0dnm0cb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:44:32 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfgSY004926; Tue, 2 Apr 2019 20:42:31 GMT Received: from aserv0122.oracle.com (aserv0122.oracle.com [141.146.126.236]) by userp3030.oracle.com with ESMTP id 2rm8f4qkf5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:30 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0122.oracle.com (8.14.4/8.14.4) with ESMTP id x32KgN95019599; Tue, 2 Apr 2019 20:42:28 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:23 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alexey Kardashevskiy , Benjamin Herrenschmidt , Christoph Lameter , Davidlohr Bueso , Michael Ellerman , Paul Mackerras , linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/6] powerpc/mmu: drop mmap_sem now that locked_vm is atomic Date: Tue, 2 Apr 2019 16:41:57 -0400 Message-Id: <20190402204158.27582-6-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With locked_vm now an atomic, there is no need to take mmap_sem as writer. Delete and refactor accordingly. Signed-off-by: Daniel Jordan Cc: Alexey Kardashevskiy Cc: Andrew Morton Cc: Benjamin Herrenschmidt Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Michael Ellerman Cc: Paul Mackerras Cc: Cc: Cc: --- arch/powerpc/mm/mmu_context_iommu.c | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/arch/powerpc/mm/mmu_context_iommu.c b/arch/powerpc/mm/mmu_context_iommu.c index 8038ac24a312..a4ef22b67c07 100644 --- a/arch/powerpc/mm/mmu_context_iommu.c +++ b/arch/powerpc/mm/mmu_context_iommu.c @@ -54,34 +54,29 @@ struct mm_iommu_table_group_mem_t { static long mm_iommu_adjust_locked_vm(struct mm_struct *mm, unsigned long npages, bool incr) { - long ret = 0, locked, lock_limit; + long ret = 0; + unsigned long lock_limit; s64 locked_vm; if (!npages) return 0; - down_write(&mm->mmap_sem); - locked_vm = atomic64_read(&mm->locked_vm); if (incr) { - locked = locked_vm + npages; lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked > lock_limit && !capable(CAP_IPC_LOCK)) + locked_vm = atomic64_add_return(npages, &mm->locked_vm); + if (locked_vm > lock_limit && !capable(CAP_IPC_LOCK)) { ret = -ENOMEM; - else - atomic64_add(npages, &mm->locked_vm); + atomic64_sub(npages, &mm->locked_vm); + } } else { - if (WARN_ON_ONCE(npages > locked_vm)) - npages = locked_vm; - atomic64_sub(npages, &mm->locked_vm); + locked_vm = atomic64_sub_return(npages, &mm->locked_vm); + WARN_ON_ONCE(locked_vm < 0); } - pr_debug("[%d] RLIMIT_MEMLOCK HASH64 %c%ld %ld/%ld\n", - current ? current->pid : 0, - incr ? '+' : '-', - npages << PAGE_SHIFT, - atomic64_read(&mm->locked_vm) << PAGE_SHIFT, + pr_debug("[%d] RLIMIT_MEMLOCK HASH64 %c%lu %lld/%lu\n", + current ? current->pid : 0, incr ? '+' : '-', + npages << PAGE_SHIFT, locked_vm << PAGE_SHIFT, rlimit(RLIMIT_MEMLOCK)); - up_write(&mm->mmap_sem); return ret; } From patchwork Tue Apr 2 20:41:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Jordan X-Patchwork-Id: 10882309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 01E3E139A for ; Tue, 2 Apr 2019 20:44:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E2D302852E for ; Tue, 2 Apr 2019 20:44:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D6BDE2878F; Tue, 2 Apr 2019 20:44:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B07202852E for ; Tue, 2 Apr 2019 20:44:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 665806B0279; Tue, 2 Apr 2019 16:44:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5EE3A6B027A; Tue, 2 Apr 2019 16:44:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 467CC6B027B; Tue, 2 Apr 2019 16:44:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f72.google.com (mail-io1-f72.google.com [209.85.166.72]) by kanga.kvack.org (Postfix) with ESMTP id 287B36B0279 for ; Tue, 2 Apr 2019 16:44:35 -0400 (EDT) Received: by mail-io1-f72.google.com with SMTP id p143so11958947iod.19 for ; Tue, 02 Apr 2019 13:44:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=sUcjXY3uzftbRPA/qx9LGu6+ef2CXjVd6ceZM9+b18U=; b=QSYDguhg1ca1vUAxdAVUumOyhjnbU5mGoSThvv4/zDEURG9YG/CasnV4P8ls6doD6o pU5du4akei+uKWS3Fe0f9rX0UrcKQVViQPUuH3L68HOEx1mVA/8DdJRcN2glX81e2VRA jqkhLwoBegnRVmdHxahFOnzWR6KfuK26GGjNGuKzH5a1+BUSvscZ8RXcUM+q3Zi9X7Dl 3LMEVys5QwylRmwBZUNw7lNN7eWhfX8En998uU2BFbybG8htP4UZEZCvXEZ3sKwxdld3 EB4cg/B06TI9t41q1o90dhE5qhibmztMO9T8XXn/6v9fYDOzaZJzIzrINR/QJOP/OAl2 rjhw== X-Gm-Message-State: APjAAAVpvLCSR6TUx61ndikRLWHVPsKSn5vQ3162hU1DadEV1/nmEsU+ EfO+gKg8b5caZnXO8lxp/kyQzlgUMsZNv4+LHZeTXh9otHohmlTM7hhq6tR0GxRMEFTrgbemiJN 3Mqrw+npXCExjOSbJ+Hk3d8h4B6wuvuqrqEUlwvgK+A7ndVEEHwFsuWBbgcLheEAIiw== X-Received: by 2002:a05:6638:9a:: with SMTP id v26mr8086536jao.82.1554237874904; Tue, 02 Apr 2019 13:44:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqwbjPDJA/9iSLnKvLkE6WsvJjyks97QKSueAistGxxArWekins+NQZym9uqS+m3KAbQW0aR X-Received: by 2002:a05:6638:9a:: with SMTP id v26mr8086464jao.82.1554237873507; Tue, 02 Apr 2019 13:44:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1554237873; cv=none; d=google.com; s=arc-20160816; b=dNt02LEPxFEqQfXuwBBwuwLrm330dtlBbwZcPgIbnr45Cp/WV5RFx3kVX7U2qRuOFL pjg40lu80nteFd7Y1Xoi6BPqWEDuG1c3/7EOx7/JEflRGT+PRc9kgcrOc3mVHSSUMESX kmcLEJmvZ/1NPLmkiTpB8MtiI+JWn+7vFz5Me2qn+WeC6ofCWNpW/3yJ0xjSUJM41lWQ Vg1N4XHO8OwGnh9madwhlyLX/d9u8fvPEmx5PxINucpxgdzM2OsoM1JUMd6lcHarIfy/ 3LcT0FDFS0+XpBaiHCco/h2boROCGrDEJX6UiXD58OnYvV8m/5TXFeMzBrg9T2S9C0wR vpcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:dkim-signature; bh=sUcjXY3uzftbRPA/qx9LGu6+ef2CXjVd6ceZM9+b18U=; b=Cr2Qe2vjJQPQTgJqwQrnJ71N4BcWzyxDWO2OKPvmyNtqFZjwNkt8qVPv7vfGM5mKuK k7nwOoGE7Z5HXlyGdNRKdxzAxR9jmmpY/DR95fvWUkldVRFyfCQUqBzz0TQ39vWAjaY3 j8pwzJCnLQu01tcH71NaiPNIXtFHvTXdTSerIjJKnXiid4nm3BLK2eip80WlXhW+zMGC HKt86TjBcqxkxeFvzEW8zlGFuapb9Kp5zTn368VXX1fpF3qk7cEY+B5QdBhYsCPbw/Ko ePKD/d1T2TUaXrdFKk4rUgvzx2Fx4Sn42DM0ddmaa6FJB5Hv78pdnFFOr//BCnk/sPEa PFlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=xBffgtoj; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from aserp2120.oracle.com (aserp2120.oracle.com. [141.146.126.78]) by mx.google.com with ESMTPS id d5si6680779iob.137.2019.04.02.13.44.33 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 02 Apr 2019 13:44:33 -0700 (PDT) Received-SPF: pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) client-ip=141.146.126.78; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2018-07-02 header.b=xBffgtoj; spf=pass (google.com: domain of daniel.m.jordan@oracle.com designates 141.146.126.78 as permitted sender) smtp.mailfrom=daniel.m.jordan@oracle.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32Kd3X1163998; Tue, 2 Apr 2019 20:42:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=corp-2018-07-02; bh=sUcjXY3uzftbRPA/qx9LGu6+ef2CXjVd6ceZM9+b18U=; b=xBffgtojFULgQywk6dg++phJ4dRh6ZFh7wsyFjoG1MnC+TBYtmtYjJElPMLapMw7cm8c VErNSijyy6Pj+8n4yE9yB/x8mMPgiR2/kTuoakitYDqe98KORwjyj0/R2IX7F7kwJfuS 35MlOD9NugPRLgKZF+gPHWJCVtOk/RiUVaDNeVrsrVRH3bJCwL/Oa8L7vMS1F5iukMRh EoCVrL95xhmbuHu7GtL70vFi58jRPIMVnNuSTNy8PI6N34JZS3qvld0+RTBNxhy9t/AR loky0Zi2Y94RsT48wmNjpC2uJc3HpzEyMRCi3pu1G0E5UgmKcN0G/stq5GDGiP6E6oxt IQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 2rj0dnkywp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:26 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.27/8.16.0.27) with SMTP id x32KfI3M103083; Tue, 2 Apr 2019 20:42:25 GMT Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by aserp3020.oracle.com with ESMTP id 2rm9mhp3nq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 02 Apr 2019 20:42:25 +0000 Received: from abhmp0009.oracle.com (abhmp0009.oracle.com [141.146.116.15]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id x32KgPpX030145; Tue, 2 Apr 2019 20:42:25 GMT Received: from localhost.localdomain (/73.60.114.248) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Tue, 02 Apr 2019 13:42:25 -0700 From: Daniel Jordan To: akpm@linux-foundation.org Cc: daniel.m.jordan@oracle.com, Alexey Kardashevskiy , Benjamin Herrenschmidt , Christoph Lameter , Davidlohr Bueso , Michael Ellerman , Paul Mackerras , linux-mm@kvack.org, kvm-ppc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [PATCH 6/6] kvm/book3s: drop mmap_sem now that locked_vm is atomic Date: Tue, 2 Apr 2019 16:41:58 -0400 Message-Id: <20190402204158.27582-7-daniel.m.jordan@oracle.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190402204158.27582-1-daniel.m.jordan@oracle.com> References: <20190402204158.27582-1-daniel.m.jordan@oracle.com> MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=29 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=9215 signatures=668685 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 suspectscore=29 phishscore=0 bulkscore=0 spamscore=0 clxscore=1011 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1810050000 definitions=main-1904020138 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP With locked_vm now an atomic, there is no need to take mmap_sem as writer. Delete and refactor accordingly. Signed-off-by: Daniel Jordan Cc: Alexey Kardashevskiy Cc: Andrew Morton Cc: Benjamin Herrenschmidt Cc: Christoph Lameter Cc: Davidlohr Bueso Cc: Michael Ellerman Cc: Paul Mackerras Cc: Cc: Cc: Cc: --- arch/powerpc/kvm/book3s_64_vio.c | 34 +++++++++++--------------------- 1 file changed, 12 insertions(+), 22 deletions(-) diff --git a/arch/powerpc/kvm/book3s_64_vio.c b/arch/powerpc/kvm/book3s_64_vio.c index e7fdb6d10eeb..8e034c3a5d25 100644 --- a/arch/powerpc/kvm/book3s_64_vio.c +++ b/arch/powerpc/kvm/book3s_64_vio.c @@ -56,7 +56,7 @@ static unsigned long kvmppc_stt_pages(unsigned long tce_pages) return tce_pages + ALIGN(stt_bytes, PAGE_SIZE) / PAGE_SIZE; } -static long kvmppc_account_memlimit(unsigned long stt_pages, bool inc) +static long kvmppc_account_memlimit(unsigned long pages, bool inc) { long ret = 0; s64 locked_vm; @@ -64,33 +64,23 @@ static long kvmppc_account_memlimit(unsigned long stt_pages, bool inc) if (!current || !current->mm) return ret; /* process exited */ - down_write(¤t->mm->mmap_sem); - - locked_vm = atomic64_read(¤t->mm->locked_vm); if (inc) { - unsigned long locked, lock_limit; + unsigned long lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - locked = locked_vm + stt_pages; - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - if (locked > lock_limit && !capable(CAP_IPC_LOCK)) + locked_vm = atomic64_add_return(pages, ¤t->mm->locked_vm); + if (locked_vm > lock_limit && !capable(CAP_IPC_LOCK)) { ret = -ENOMEM; - else - atomic64_add(stt_pages, ¤t->mm->locked_vm); + atomic64_sub(pages, ¤t->mm->locked_vm); + } } else { - if (WARN_ON_ONCE(stt_pages > locked_vm)) - stt_pages = locked_vm; - - atomic64_sub(stt_pages, ¤t->mm->locked_vm); + locked_vm = atomic64_sub_return(pages, ¤t->mm->locked_vm); + WARN_ON_ONCE(locked_vm < 0); } - pr_debug("[%d] RLIMIT_MEMLOCK KVM %c%ld %ld/%ld%s\n", current->pid, - inc ? '+' : '-', - stt_pages << PAGE_SHIFT, - atomic64_read(¤t->mm->locked_vm) << PAGE_SHIFT, - rlimit(RLIMIT_MEMLOCK), - ret ? " - exceeded" : ""); - - up_write(¤t->mm->mmap_sem); + pr_debug("[%d] RLIMIT_MEMLOCK KVM %c%lu %lld/%lu%s\n", current->pid, + inc ? '+' : '-', pages << PAGE_SHIFT, + locked_vm << PAGE_SHIFT, + rlimit(RLIMIT_MEMLOCK), ret ? " - exceeded" : ""); return ret; }