From patchwork Thu Jan 5 10:18:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Houghton X-Patchwork-Id: 13089640 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 349CCC53210 for ; Thu, 5 Jan 2023 10:19:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C512A900004; Thu, 5 Jan 2023 05:19:00 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C01798E0001; Thu, 5 Jan 2023 05:19:00 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA147900004; Thu, 5 Jan 2023 05:19:00 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 88B928E0001 for ; Thu, 5 Jan 2023 05:19:00 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 6650C1A0786 for ; Thu, 5 Jan 2023 10:19:00 +0000 (UTC) X-FDA: 80320347240.02.86DD742 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf26.hostedemail.com (Postfix) with ESMTP id C5C73140003 for ; Thu, 5 Jan 2023 10:18:58 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NmgS3tlh; spf=pass (imf26.hostedemail.com: domain of 3EaS2YwoKCFg9J7EK67JED6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--jthoughton.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3EaS2YwoKCFg9J7EK67JED6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672913938; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8fBYHLCr9oSB4M/a/ngoaaKhcuSrl4k3J29XD8vYb2Q=; b=XdsRzbYlXkduALKHm9juae7A69hkBljuSuVHzPDhIHIqOWPjJdn8MIcHrachLK+T3b3Lel h180wxwOae6J6vWDi1RIqMdYrwNLa48agass/ECiKo4rcW3Op+uu5CVpc+TQyJ+YF/Ff6w dGrSQpSqois3TUiftIsLd6m4H4POZG8= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=NmgS3tlh; spf=pass (imf26.hostedemail.com: domain of 3EaS2YwoKCFg9J7EK67JED6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--jthoughton.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3EaS2YwoKCFg9J7EK67JED6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--jthoughton.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672913938; a=rsa-sha256; cv=none; b=38ZJCBBeer8neXtOzNtk2i73LxVnrRqvCUBVaDh57zrvSVafcHfFqkmSqFGR57/Jkgwo3v aBmDnDRzwEl0u2SCG12XwP2N7dVAFezq/PvLGgP4J9zWLX0iOj5YHgNPWP4ix4UiMLdBig ABrM+hOvAcr+UhnPV5QQVqHcv5nw4Oo= Received: by mail-yb1-f202.google.com with SMTP id s6-20020a259006000000b00706c8bfd130so36458840ybl.11 for ; Thu, 05 Jan 2023 02:18:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=8fBYHLCr9oSB4M/a/ngoaaKhcuSrl4k3J29XD8vYb2Q=; b=NmgS3tlh8WUTbw91DNXzILqULTbddBWnPQn0Wz57ppipw1WObH6FkSpTww6yztM+jA 6e8g5ZzoA5lVCAREHkwcp9nrWOI+Q0S0zYB3GUuo7cG78xBZHsz/CSyv57wBUHMoyAZn uMRawDWwRYn9gVztU7/tvicjuzxjqEUsaG6Q2Nh3CiNSMC3lromhxCi7P1Rw/hwbHL1F YmJbdMcEkKBiPCX4t46+UvV1gVJE1sWQPNJLWRGQKmx7VESDn0Z4IGBEAOvcilWvJwx0 097HlvspkiuYJIAb7OcrrOPaSLfJxD3uYwyJHjA4LcqVHz5gZ4GhB3QpiX2mLa2tpK7P fY/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=8fBYHLCr9oSB4M/a/ngoaaKhcuSrl4k3J29XD8vYb2Q=; b=hS0stWYJd8pcVe/8Hh4Geh/sgE7H+hG3RW5bjfohSCU+djRaK4fZnT2HUY5u0gPiCd 1DhD/rBw+6mTxIZ8z/ZyqRrwfML+BlPuFlNxq8OJSA1FUxqlLYHmxQnonrFGCyKRMLgQ pl36RFP3W1WgCI5qWJH8trfbgyTlzaLmVYesTVP2pdjae+2LjrfnH7feMiorHSSFuOUX C0ZrZLYLGsqn2UuRWe+Cxp9hH31SXznQJj6nupR6dFOXAn2nwjVhJrkatBiQ5UaZIXHC WlVrUQOkZ+VyDxzqJ6JtWSjPH6PC11y7NVEzrF6uI93W490iK6euCBBGkKiQku3Bj0yX oCrA== X-Gm-Message-State: AFqh2kp8Grml+bUpXYDp8zwMLC+Up2JiEDWWAnVHWdG0vf7h9hnVQTrj nlKwr0dGsH2KrqAqFM6fexpvkoZXJoYs8FZG X-Google-Smtp-Source: AMrXdXtKSspcz64KdJshT2Zl/+9+WpY5XaxA4H4cZjlRHhAILIZdFfHdFt4D44YNQqdRUNZd3I9uCvlbO/Onyj2D X-Received: from jthoughton.c.googlers.com ([fda3:e722:ac3:cc00:14:4d90:c0a8:2a4f]) (user=jthoughton job=sendgmr) by 2002:a25:f311:0:b0:6ff:84be:717 with SMTP id c17-20020a25f311000000b006ff84be0717mr5385597ybs.314.1672913937929; Thu, 05 Jan 2023 02:18:57 -0800 (PST) Date: Thu, 5 Jan 2023 10:18:02 +0000 In-Reply-To: <20230105101844.1893104-1-jthoughton@google.com> Mime-Version: 1.0 References: <20230105101844.1893104-1-jthoughton@google.com> X-Mailer: git-send-email 2.39.0.314.g84b9a713c41-goog Message-ID: <20230105101844.1893104-5-jthoughton@google.com> Subject: [PATCH 04/46] hugetlb: only adjust address ranges when VMAs want PMD sharing From: James Houghton To: Mike Kravetz , Muchun Song , Peter Xu Cc: David Hildenbrand , David Rientjes , Axel Rasmussen , Mina Almasry , "Zach O'Keefe" , Manish Mishra , Naoya Horiguchi , "Dr . David Alan Gilbert" , "Matthew Wilcox (Oracle)" , Vlastimil Babka , Baolin Wang , Miaohe Lin , Yang Shi , Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org, James Houghton X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C5C73140003 X-Rspam-User: X-Stat-Signature: b5b4zkfkzkr6q7bswk837ts964xnxst7 X-HE-Tag: 1672913938-510185 X-HE-Meta: U2FsdGVkX1/nn9bPhUDf2rpWXaORAcKbbvGW8pMedbERCmEyCTRZrWI8SuQm5k+w6s9kXpk4finD8vR0Y/emLGBAaEuJ73pAPcHMxm4/wdK2IOpex2oO+rw4R+57/vAy39eWF9srsXEDjdHiKW/zHUT9Ygj/725q4MaHaLqfX2V+BvJlpx91iyemTiw/Lk3S9kk3cs2wQWLJVdhyA/5HWoEYMEvET3HtkG8gD0arjxcm/QsEhneIirxcJjKC4GzEP5GbjUEm0AKY1Hg19BBgETMAnqtEYB0l4vwtfVVwqRiDSMXrmLDZjUBasQENsUpNAo8xrd+53aqbycdClaVwDDtwHH/12/Ig/ehdXBCRUqKXTMQyrbI9Z/GSMt0Du4GxcEW5EvliNje16Y4Vc8K7H4SQTzFtILHZfFYw4ZJ9ddI3+EtZpPAy8NtNpctJ4sb+A4IMmpfHF6kCtbJmzUic4CjoNcpRmKqx4o78Lv7PbxacjyrMNsW2wXNh/bjX3URzUrcKlzzDSvkz5ZdIl2UkX5s3kRy84IBSUzgxJHJ0LKpej2Fph0V9JBqNUdXrfutTfwRJCePJO7aLw7TSrg0EP0wKLEkZ4aan91IxvC4yI7PHPwNGZrevln/m53vV/2LFaEcA6muDo9alVu+/x3OJ1DtIdiIRWUMlv/arK7Pcz2iK3QtlYkrYYh6R2bQLMgxrZwKQEM7dul2y6q794RYMs0Orwy4wAyKUjvMd+5eptV8Pk0iwQTiClIF00s2GqXNb4nJAZwdoGPdjW2EyrROsnzb1qy1mhEO6fl/DwaKbDgriOoUJ27dldq+dDbdLdHgkfPUf1bYsXN6QUfzycFx1q+iNFw2M5Cg/uSXgDVPH2fc6erRQD0XJvovVN9rLj3SaoQgJCt0Uunct3ZTEqtIIGUzXnuP1nxa17uPLJafEvSTsJAhVe7OZg4zch6gNEk92meVPCH5wPG66Cq+fZ8I VdU/xcSl DBAJ6L75FyibY8o8gwb0dzPB83QC25aBBYqWyJxrLt/aCJT7EbIYS2UiX3MY20A3y78Scz6sIUeKxvTy7F8Dd6rXGI85tCD9pQYg5ZvES9U7FVHIoWHwBS1JzkqWsQkZB+L+4wEqoUPNgfyQM9AdpvFRaH12hP1CQ2BuJRa03xNWdpRrW3uOwuC1BwGPOD/+GoL/cVF6lzEywNQDzmbDaOCSWHPW+ucUjRb/99lx2/9q/3oYjJLCLmMsOKRqWT+ojlogrNCP17fH3KLV/qjsAVK9/f+JqINf2avPK9vX/dhw8OE0bDGdWPuvb9XFrlFG9p0Hmojw7oM8lR6f6+V/EV2MY6mfcBuARdt38GSvlEwYlbD/sDlGcxoFd61c7/gbiio+JPassberj9UA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently this check is overly aggressive. For some userfaultfd VMAs, VMA sharing is disabled, yet we still widen the address range, which is used for flushing TLBs and sending MMU notifiers. This is done now, as HGM VMAs also have sharing disabled, yet would still have flush ranges adjusted. Overaggressively flushing TLBs and triggering MMU notifiers is particularly harmful with lots of high-granularity operations. Acked-by: Peter Xu Reviewed-by: Mike Kravetz Signed-off-by: James Houghton --- mm/hugetlb.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 7e9793b602ac..99fadd7680ec 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6961,22 +6961,31 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma, return saddr; } -bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +static bool pmd_sharing_possible(struct vm_area_struct *vma) { - unsigned long start = addr & PUD_MASK; - unsigned long end = start + PUD_SIZE; - #ifdef CONFIG_USERFAULTFD if (uffd_disable_huge_pmd_share(vma)) return false; #endif /* - * check on proper vm_flags and page table alignment + * Only shared VMAs can share PMDs. */ if (!(vma->vm_flags & VM_MAYSHARE)) return false; if (!vma->vm_private_data) /* vma lock required for sharing */ return false; + return true; +} + +bool want_pmd_share(struct vm_area_struct *vma, unsigned long addr) +{ + unsigned long start = addr & PUD_MASK; + unsigned long end = start + PUD_SIZE; + /* + * check on proper vm_flags and page table alignment + */ + if (!pmd_sharing_possible(vma)) + return false; if (!range_in_vma(vma, start, end)) return false; return true; @@ -6997,7 +7006,7 @@ void adjust_range_if_pmd_sharing_possible(struct vm_area_struct *vma, * vma needs to span at least one aligned PUD size, and the range * must be at least partially within in. */ - if (!(vma->vm_flags & VM_MAYSHARE) || !(v_end > v_start) || + if (!pmd_sharing_possible(vma) || !(v_end > v_start) || (*end <= v_start) || (*start >= v_end)) return;