From patchwork Fri Oct 12 06:00:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: john.hubbard@gmail.com X-Patchwork-Id: 10637917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D635715E2 for ; Fri, 12 Oct 2018 06:00:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C7F7A2BB24 for ; Fri, 12 Oct 2018 06:00:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BB6E92BFF5; Fri, 12 Oct 2018 06:00:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6DA712BB24 for ; Fri, 12 Oct 2018 06:00:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727761AbeJLNbg (ORCPT ); Fri, 12 Oct 2018 09:31:36 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:43206 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727723AbeJLNbY (ORCPT ); Fri, 12 Oct 2018 09:31:24 -0400 Received: by mail-pl1-f195.google.com with SMTP id 30-v6so5363570plb.10; Thu, 11 Oct 2018 23:00:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=r6gf/3+FxFoo0uFxF1HQUrQtpnIyPUJrRDNGoh0sv3k=; b=svScXGM87F7pkDY1zQnXkoh0/E1jU6dk0eyAzQUvDSh5Wugzl9qEuvG7+Xu4dUWb8s SR8mYLdKbQHzzLAk5dDx+sjo76Sddq81tsZSt/tD+C3LvQ5k9pQhTlRjHosPb+d3GSXm /cGIXS5lMJJw+Ap7p+ExA/KOOSa5eRnZ+9tIIUXRqApUVtWqowKabLgR1wH7OvyBpnDH IVfAm5wSDyiM5HD46w4QSJQ4oRl/9xFNCEoKmBmFC6f8UFrgAJvf6Jc+fIm3+nTsnSFB NwtJ71tK1cupi2Fq3gN5qKUJegKEAjP41drN38u8eWJQbpGVxp5gVE24v49WVEZNnrlK LbbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=r6gf/3+FxFoo0uFxF1HQUrQtpnIyPUJrRDNGoh0sv3k=; b=Z8yLJNtZYBXMUYWEu4JduPz+06oWWz5Tq38nYEmLfxXqHhMNKs1HTGroKxLWfZ0J5t dunXjP1R0smLrqb8+z5Xr4zBy7RjbWNGPUVk6SFUzK1l5oCLIFOudCCtL5oktZPbnJn/ Dqj/UGJItf+hdKQLFzV55bFncv9Vykhe3izSNg9kodtYd1NXtW2nuNHEiH4bhjz00dkH yrYZsGqad+naPnPYzJ1JEOJqd6LP0I4qnOFAGo6lopWmznagqGYzpJ6dnRowHDYWm6mh Nq+QiPNFngdG0bNtRuCPgUBRGRRLRhSfACj1jaxMFMo5zkOQVO5TtGLpibTzkVMCryOj sxXA== X-Gm-Message-State: ABuFfohKe3AWGk638NTv2JXrH+U8deYRlWdveoJaDNGdl/T2JiE6A0ki QLK72IIq+d2bqaeyKjQ1+iQ= X-Google-Smtp-Source: ACcGV63SVIxTN5usiPLLB6gc3qBl/+Vh3uNHa6giBPTmGkEMBVLUqMufOt9kVZRfmZGKcEL9+tf8rg== X-Received: by 2002:a17:902:124:: with SMTP id 33-v6mr4618125plb.205.1539324035839; Thu, 11 Oct 2018 23:00:35 -0700 (PDT) Received: from blueforge.nvidia.com (searspoint.nvidia.com. [216.228.112.21]) by smtp.gmail.com with ESMTPSA id z3-v6sm368579pfm.150.2018.10.11.23.00.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 11 Oct 2018 23:00:34 -0700 (PDT) From: john.hubbard@gmail.com X-Google-Original-From: jhubbard@nvidia.com To: Matthew Wilcox , Michal Hocko , Christopher Lameter , Jason Gunthorpe , Dan Williams , Jan Kara Cc: linux-mm@kvack.org, Andrew Morton , LKML , linux-rdma , linux-fsdevel@vger.kernel.org, John Hubbard Subject: [PATCH 5/6] mm: introduce zone_gup_lock, for dma-pinned pages Date: Thu, 11 Oct 2018 23:00:13 -0700 Message-Id: <20181012060014.10242-6-jhubbard@nvidia.com> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181012060014.10242-1-jhubbard@nvidia.com> References: <20181012060014.10242-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: John Hubbard The page->dma_pinned_flags and _count fields require lock protection. A lock at approximately the granularity of the zone_lru_lock is called for, but adding to the locking contention of zone_lru_lock is undesirable, because that is a pre-existing hot spot. Fortunately, these new dma_pinned_* fields can use an independent lock, so this patch creates an entirely new lock, right next to the zone_lru_lock. Why "zone_gup_lock"? Most of the naming refers to "DMA-pinned pages", but "zone DMA lock" has other meanings already, so this is called zone_gup_lock instead. The "dma pinning" is a result of get_user_pages (gup) being called, so the name still helps explain its use. Signed-off-by: John Hubbard --- include/linux/mmzone.h | 6 ++++++ mm/page_alloc.c | 1 + 2 files changed, 7 insertions(+) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index d4b0c79d2924..971a63f84ad5 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -661,6 +661,7 @@ typedef struct pglist_data { enum zone_type kswapd_classzone_idx; int kswapd_failures; /* Number of 'reclaimed == 0' runs */ + spinlock_t pinned_dma_lock; #ifdef CONFIG_COMPACTION int kcompactd_max_order; @@ -730,6 +731,11 @@ static inline spinlock_t *zone_lru_lock(struct zone *zone) return &zone->zone_pgdat->lru_lock; } +static inline spinlock_t *zone_gup_lock(struct zone *zone) +{ + return &zone->zone_pgdat->pinned_dma_lock; +} + static inline struct lruvec *node_lruvec(struct pglist_data *pgdat) { return &pgdat->lruvec; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e2ef1c17942f..850f90223cc7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6225,6 +6225,7 @@ static void __meminit pgdat_init_internals(struct pglist_data *pgdat) pgdat_page_ext_init(pgdat); spin_lock_init(&pgdat->lru_lock); + spin_lock_init(&pgdat->pinned_dma_lock); lruvec_init(node_lruvec(pgdat)); }