From patchwork Mon Jan 21 17:42:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 10774259 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 925B96C2 for ; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7BC8D29F0F for ; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6D43F2A6E0; Mon, 21 Jan 2019 17:43:08 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EAB6C29F0F for ; Mon, 21 Jan 2019 17:43:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2345A8E0006; Mon, 21 Jan 2019 12:43:07 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 20A2C8E0001; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 122588E0006; Mon, 21 Jan 2019 12:43:07 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi1-f197.google.com (mail-oi1-f197.google.com [209.85.167.197]) by kanga.kvack.org (Postfix) with ESMTP id DE4628E0001 for ; Mon, 21 Jan 2019 12:43:06 -0500 (EST) Received: by mail-oi1-f197.google.com with SMTP id r131so9935386oia.7 for ; Mon, 21 Jan 2019 09:43:06 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=BvaEORAmuWSzID7o6vf8zI8PBOEpqDPHoqVpDewB9Qg=; b=VCCA/u1Ey633rnpDk2g7QyNujPjSzJfrCfQijTATde32hH11KOFSYnVkhLz0+r4FUi s0gGsIViRdBomR8Lg3vUul54Z49BEleuBgfwxjwn8dsdRkIWt886Os17UDiQN+kTTAb/ sHzIA8zILFJCpqDb/enr6X4kExWxcYPDmFfV3gYaIYkTZD7JdjCn8AFLFuFNR9fBt4yQ 7tOiLzXIW5+JMNZyZAxWwYPo8huLSmU1QoFXjVKhfeYxkjjQbAczWzUWWKZVjuF+mZxT KEjXqlaAvYhPBvTwULVbpu4pqMIFlNTcoUsR6I/Ihyr7DJb8UuYULh+T1ixoLFU63+0L CkLg== X-Original-Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net X-Gm-Message-State: AJcUukdw5SDjEiZvHnqYFMqoJDZTswjriVTGZIA7Bmoi2sRghfA+yNUy y6BQ/rDun7htDFJI0tDSGi95kcN4mkj0SgSGDbd30nu/WdrEnd7UAUpSftkJd1nckxKo2g9bzxs Wm//9o/NshuQcrqXlpMw0WU2Wytdc6WOL8YVGQUp5ikAY4JHFAE2B4crtfHLQQKU= X-Received: by 2002:aca:f389:: with SMTP id r131mr6470732oih.58.1548092586649; Mon, 21 Jan 2019 09:43:06 -0800 (PST) X-Google-Smtp-Source: ALg8bN4+4B0JvhqCsz2mZJPsP8Q6Q0iIBBL+5ktSEWCjS7ReSAX1hMWyWJ+2VE3OUMaFRDtjlj5p X-Received: by 2002:aca:f389:: with SMTP id r131mr6470699oih.58.1548092585872; Mon, 21 Jan 2019 09:43:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548092585; cv=none; d=google.com; s=arc-20160816; b=DV9wyNnut740gB1s4za16kd1MSBkT6hsaI59GBL8nkpBpQxovQqkfetOmEgyrX5fjW L9Zpbrnf4B55fgEhXyKJbh84NDtDDGX7kHalnC6MJrDpVz3IS5+TrI93xCe2IJB/YDmr eYQhcu9Z09wE/v5csyI1vZyw/AqBUvd/1YUGoEtdjh9CFSPmzETUPatWnPHvXM4/JuMq X0BRmvxnJ+nqZx0SenU4pdh1w/hzVifpyxMf1zz1FPnkDgmir2I7MTraNKsvtRum1MLA wS6+j+TgPLNK3ITRmoysgplwY6/8CpL9U2bPwQdB5o7//M26vjjnVCZ6uiP4f6mSyU5y oMmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=BvaEORAmuWSzID7o6vf8zI8PBOEpqDPHoqVpDewB9Qg=; b=ya2i9oj6o/d8fdDDyJqUMgfd0zhgRluGtozJT885dpSnp7rWxYXMKMG3mssEJA2ISM cmuU19J9HN6b/B1Gpu6ZjSxUgcBCSeoVGZmLIKrYwSoRGUSlNBZdGfDHLq9E1sRddkEK +wM+HMWaggJjuWmw8SwEKV7ctcT2SVIh/Bw/w1nSSizdZs7LQDPjPVCxdyy7YokjRX4l 6G5fMzFAngLD8bLdspZoXYmiIk2JAkU4bbk8QfMqqsqhOC7W6vIQ404cdrWd/pDt2RZp xDq4Vm8TqwI7XdvCRxA3fVCB7tyUXPzRDS9pcj174XR0bI/80XSi3i3QQ07hqNPGegs3 2AEQ== ARC-Authentication-Results: i=1; mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from smtp2.provo.novell.com (smtp2.provo.novell.com. [137.65.250.81]) by mx.google.com with ESMTPS id f24si6470486otq.97.2019.01.21.09.43.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 09:43:05 -0800 (PST) Received-SPF: softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) client-ip=137.65.250.81; Authentication-Results: mx.google.com; spf=softfail (google.com: domain of transitioning dave@stgolabs.net does not designate 137.65.250.81 as permitted sender) smtp.mailfrom=dave@stgolabs.net Received: from localhost.localdomain (prv-ext-foundry1int.gns.novell.com [137.65.251.240]) by smtp2.provo.novell.com with ESMTP (TLS encrypted); Mon, 21 Jan 2019 10:42:52 -0700 From: Davidlohr Bueso To: akpm@linux-foundation.org Cc: dledford@redhat.com, jgg@mellanox.com, jack@suse.de, ira.weiny@intel.com, linux-rdma@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, dave@stgolabs.net, dennis.dalessandro@intel.com, mike.marciniszyn@intel.com, Davidlohr Bueso Subject: [PATCH 3/6] drivers/IB,qib: do not use mmap_sem Date: Mon, 21 Jan 2019 09:42:17 -0800 Message-Id: <20190121174220.10583-4-dave@stgolabs.net> X-Mailer: git-send-email 2.16.4 In-Reply-To: <20190121174220.10583-1-dave@stgolabs.net> References: <20190121174220.10583-1-dave@stgolabs.net> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The driver uses mmap_sem for both pinned_vm accounting and get_user_pages(). By using gup_fast() and letting the mm handle the lock if needed, we can no longer rely on the semaphore and simplify the whole thing as the pinning is decoupled from the lock. This also fixes a bug that __qib_get_user_pages was not taking into account the current value of pinned_vm. Cc: dennis.dalessandro@intel.com Cc: mike.marciniszyn@intel.com Reviewed-by: Ira Weiny Signed-off-by: Davidlohr Bueso --- drivers/infiniband/hw/qib/qib_user_pages.c | 67 ++++++++++-------------------- 1 file changed, 22 insertions(+), 45 deletions(-) diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c index 602387bf98e7..e4114aad4a2f 100644 --- a/drivers/infiniband/hw/qib/qib_user_pages.c +++ b/drivers/infiniband/hw/qib/qib_user_pages.c @@ -49,43 +49,6 @@ static void __qib_release_user_pages(struct page **p, size_t num_pages, } } -/* - * Call with current->mm->mmap_sem held. - */ -static int __qib_get_user_pages(unsigned long start_page, size_t num_pages, - struct page **p) -{ - unsigned long lock_limit; - size_t got; - int ret; - - lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; - - if (num_pages > lock_limit && !capable(CAP_IPC_LOCK)) { - ret = -ENOMEM; - goto bail; - } - - for (got = 0; got < num_pages; got += ret) { - ret = get_user_pages(start_page + got * PAGE_SIZE, - num_pages - got, - FOLL_WRITE | FOLL_FORCE, - p + got, NULL); - if (ret < 0) - goto bail_release; - } - - atomic64_add(num_pages, ¤t->mm->pinned_vm); - - ret = 0; - goto bail; - -bail_release: - __qib_release_user_pages(p, got, 0); -bail: - return ret; -} - /** * qib_map_page - a safety wrapper around pci_map_page() * @@ -137,26 +100,40 @@ int qib_map_page(struct pci_dev *hwdev, struct page *page, dma_addr_t *daddr) int qib_get_user_pages(unsigned long start_page, size_t num_pages, struct page **p) { + unsigned long locked, lock_limit; + size_t got; int ret; - down_write(¤t->mm->mmap_sem); + lock_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT; + locked = atomic64_add_return(num_pages, ¤t->mm->pinned_vm); - ret = __qib_get_user_pages(start_page, num_pages, p); + if (locked > lock_limit && !capable(CAP_IPC_LOCK)) { + ret = -ENOMEM; + goto bail; + } - up_write(¤t->mm->mmap_sem); + for (got = 0; got < num_pages; got += ret) { + ret = get_user_pages_fast(start_page + got * PAGE_SIZE, + num_pages - got, + FOLL_WRITE | FOLL_FORCE, + p + got); + if (ret < 0) + goto bail_release; + } + return 0; +bail_release: + __qib_release_user_pages(p, got, 0); +bail: + atomic64_sub(num_pages, ¤t->mm->pinned_vm); return ret; } void qib_release_user_pages(struct page **p, size_t num_pages) { - if (current->mm) /* during close after signal, mm can be NULL */ - down_write(¤t->mm->mmap_sem); - __qib_release_user_pages(p, num_pages, 1); - if (current->mm) { + if (current->mm) { /* during close after signal, mm can be NULL */ atomic64_sub(num_pages, ¤t->mm->pinned_vm); - up_write(¤t->mm->mmap_sem); } }