From patchwork Thu May 24 00:58:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 10422559 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 51F5E6019D for ; Thu, 24 May 2018 01:00:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3FD85292EA for ; Thu, 24 May 2018 01:00:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 346E6292EC; Thu, 24 May 2018 01:00:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEBF0292EA for ; Thu, 24 May 2018 01:00:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B5826B026B; Wed, 23 May 2018 21:00:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 73D646B026C; Wed, 23 May 2018 21:00:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 60A176B026D; Wed, 23 May 2018 21:00:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f199.google.com (mail-pf0-f199.google.com [209.85.192.199]) by kanga.kvack.org (Postfix) with ESMTP id 15BCA6B026B for ; Wed, 23 May 2018 21:00:41 -0400 (EDT) Received: by mail-pf0-f199.google.com with SMTP id d20-v6so13950649pfn.16 for ; Wed, 23 May 2018 18:00:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=y9KJcXxv1uiT8FNG4pi9wPWcvlBsW239KbT00nSccw0=; b=L+OT6U8v2vvglRYoevCfHHcASBo/wZcW1qX2tGHbKsChcYUwlIL75hreD16KcDecI/ D+G/AG9lIyl2ydhGDiU4Gnn3pkVZrS9dNUWveBDbOBUPh32BYH/RV/8LAGxR0D0ZJagx Sz5cTR68QcVStzbdInXi1LYg6+uO4DNvDF72kWvJJ71m0z1or3I5xcGsPO0rn7WXJVqX 3k6yGRnWFGrCZ7j6C4zfeQjN5nuAYbjU27uchbmcOq2SNTBvC81ew7WG6Bv2tidh1UZq 6bWv/Lagfyxg/FbMWJGZME2YPM4LWnjhBXko5/dXuhe1IxT9EJBlEGEOkhbQoBtArEDn AV5Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: ALKqPwdUDckghGRrrLbGshHGNanZit4t09CxdQbrp3+b5hjc3rA1fOLO gXMzvqdJPIYneBZPgBZmHUepqizPKvdMFvW7dVem31BRc9rctdvNdKazoXm5ergQV/j6ZIGxgC/ XaYje4oVr5iDBl7P9IcFGmP4u4li+sNBTyHGnD8VcnzvVAsmH2cy1GFXcToOfB0DFVQ== X-Received: by 2002:a63:b601:: with SMTP id j1-v6mr4069797pgf.335.1527123640738; Wed, 23 May 2018 18:00:40 -0700 (PDT) X-Google-Smtp-Source: AB8JxZrVqsGHbg9zQJFGfZrnjizoW7iSAWBuWiLkoB7u0k2w8WWsA/V8NMkyoDMUR/ajQ94nVuVV X-Received: by 2002:a63:b601:: with SMTP id j1-v6mr4069742pgf.335.1527123639511; Wed, 23 May 2018 18:00:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527123639; cv=none; d=google.com; s=arc-20160816; b=JQs6CCaI4bEFrK2U03/I4i/od2O1dEjzd5doX2vEeWg2uc4av2KtY2eWLRUqecL2Zi FwYDoElyHZscmXgeohmtyAJEBk8lodXOcV844u6tWZ/QHKbDFVaqhr7Ure3LAniPcxLS elnQj7GbfNx8GZrdRfaubKFKjjZalGothpEIffTdfzzc8jN7vcqmksBXrKujY+A7MHf5 wylXzJi/k/ldbY3zqMdkuZ37lGRN1bCOdJx328i0/qAUg6PeOUVzczSCIQnnd7JpDI8h UOi3ir3u8yf8D5nUVtna+IWr8t7n8cvmm5JnMk4KFc17wnVKA0OsJcHKxyFyJ1Xn+vwI NFsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=y9KJcXxv1uiT8FNG4pi9wPWcvlBsW239KbT00nSccw0=; b=PrPVtqG15HQqXNBxq0yFtO+TaoqVwvCQE8uzsOAz5v+DcUuNi/FPjrR/SeQk0Q70ki ShSX5TyBM53dJvFArN8bQ7npVYUARwaSGJY7HP5X5xSQxxNwVBWpYbH7WHy5njBvzMyr aMHuOnB8SvKJYcXKF3yxrbN0A/nOO7tkfyIb4U5eWgkhLlHgcB3ZBa6ZRVsM8QkaPr4p r4G7DpWrhUWPYpeaBq0v4tFFca4nHhUMy69eJeGP37ZZu7c2y6+EH1DVPmKL/00mwJ66 HfEHFS5/qp5gSW6vbxDY98wFgvmXoTPPJiJ6gsq/v62ToPWtnruDKpFkURlIbYQBYPeK 1IGQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga18.intel.com (mga18.intel.com. [134.134.136.126]) by mx.google.com with ESMTPS id o1-v6si19635860pld.424.2018.05.23.18.00.39 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 May 2018 18:00:39 -0700 (PDT) Received-SPF: pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) client-ip=134.134.136.126; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ying.huang@intel.com designates 134.134.136.126 as permitted sender) smtp.mailfrom=ying.huang@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 23 May 2018 18:00:39 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.49,435,1520924400"; d="scan'208";a="42362576" Received: from yhuang6-ux31a.sh.intel.com ([10.239.197.97]) by fmsmga008.fm.intel.com with ESMTP; 23 May 2018 18:00:36 -0700 From: "Huang, Ying" To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Mike Kravetz , Michal Hocko , David Rientjes , Andrea Arcangeli , "Kirill A. Shutemov" , Andi Kleen , Jan Kara , Matthew Wilcox , Hugh Dickins , Minchan Kim , Shaohua Li , Christopher Lameter , "Aneesh Kumar K.V" , Punit Agrawal , Anshuman Khandual Subject: [PATCH -V2 -mm 4/4] mm, hugetlbfs: Pass fault address to cow handler Date: Thu, 24 May 2018 08:58:51 +0800 Message-Id: <20180524005851.4079-5-ying.huang@intel.com> X-Mailer: git-send-email 2.16.1 In-Reply-To: <20180524005851.4079-1-ying.huang@intel.com> References: <20180524005851.4079-1-ying.huang@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Huang Ying This is to take better advantage of the general huge page copying optimization. Where, the target subpage will be copied last to avoid the cache lines of target subpage to be evicted when copying other subpages. This works better if the address of the target subpage is available when copying huge page. So hugetlbfs page fault handlers are changed to pass that information to hugetlb_cow(). This will benefit workloads which don't access the begin of the hugetlbfs huge page after the page fault under heavy cache contention. Signed-off-by: "Huang, Ying" Cc: Mike Kravetz Cc: Michal Hocko Cc: David Rientjes Cc: Andrea Arcangeli Cc: "Kirill A. Shutemov" Cc: Andi Kleen Cc: Jan Kara Cc: Matthew Wilcox Cc: Hugh Dickins Cc: Minchan Kim Cc: Shaohua Li Cc: Christopher Lameter Cc: "Aneesh Kumar K.V" Cc: Punit Agrawal Cc: Anshuman Khandual Reviewed-by: Mike Kravetz --- mm/hugetlb.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ad3bec2ed269..1df974af34c1 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -3500,7 +3500,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma, * Keep the pte_same checks anyway to make transition from the mutex easier. */ static int hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, - unsigned long haddr, pte_t *ptep, + unsigned long address, pte_t *ptep, struct page *pagecache_page, spinlock_t *ptl) { pte_t pte; @@ -3509,6 +3509,7 @@ static int hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, int ret = 0, outside_reserve = 0; unsigned long mmun_start; /* For mmu_notifiers */ unsigned long mmun_end; /* For mmu_notifiers */ + unsigned long haddr = address & huge_page_mask(h); pte = huge_ptep_get(ptep); old_page = pte_page(pte); @@ -3583,7 +3584,7 @@ static int hugetlb_cow(struct mm_struct *mm, struct vm_area_struct *vma, goto out_release_all; } - copy_user_huge_page(new_page, old_page, haddr, vma, + copy_user_huge_page(new_page, old_page, address, vma, pages_per_huge_page(h)); __SetPageUptodate(new_page); set_page_huge_active(new_page); @@ -3817,7 +3818,7 @@ static int hugetlb_no_page(struct mm_struct *mm, struct vm_area_struct *vma, hugetlb_count_add(pages_per_huge_page(h), mm); if ((flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) { /* Optimization, do the COW without a second fault */ - ret = hugetlb_cow(mm, vma, haddr, ptep, page, ptl); + ret = hugetlb_cow(mm, vma, address, ptep, page, ptl); } spin_unlock(ptl); @@ -3971,7 +3972,7 @@ int hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, if (flags & FAULT_FLAG_WRITE) { if (!huge_pte_write(entry)) { - ret = hugetlb_cow(mm, vma, haddr, ptep, + ret = hugetlb_cow(mm, vma, address, ptep, pagecache_page, ptl); goto out_put_page; }