From patchwork Thu Jun 14 02:49:31 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10463231 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 4933360348 for ; Thu, 14 Jun 2018 02:49:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2EF6528B5D for ; Thu, 14 Jun 2018 02:49:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2105D28B64; Thu, 14 Jun 2018 02:49:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 832ED28B5D for ; Thu, 14 Jun 2018 02:49:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7265F6B0005; Wed, 13 Jun 2018 22:49:41 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 6D75F6B0006; Wed, 13 Jun 2018 22:49:41 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5C6E96B0007; Wed, 13 Jun 2018 22:49:41 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf0-f197.google.com (mail-pf0-f197.google.com [209.85.192.197]) by kanga.kvack.org (Postfix) with ESMTP id 1BF8B6B0005 for ; Wed, 13 Jun 2018 22:49:41 -0400 (EDT) Received: by mail-pf0-f197.google.com with SMTP id u21-v6so2287327pfn.0 for ; Wed, 13 Jun 2018 19:49:41 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=R5p7AbKU2jjzhdJXZ3QBYGGRaZb93kpUqzc4IWfZr40=; b=Rjg6zr6Mwv0IETzv9TO554bAIU+I99DhClM5Jz+k20T50RIyst1qlXM5MU36XCqE4O sM+/UE5DDa4/4Pwq5dzumreJoBYuMSkelvAAF80yGNbKxMYV7nlfUGIYZPS4oONwneRo 3Wl1O/JfUv4Y4QVGw+E0ViILZrRhN9nuYnlKwIBaJuKPJoe26lyqHrIkSPGyviuTJNsO dZMe5aIldvFSWgYRg9txkDBSSjezQnlJzauBo0LwsI+Kn+HXTiFCxn1f5R8Kpot4l8Qn wkkkeWzMyj710MD9pjLiIrE1PO9zjP9EYIaB3zcdEJcEU0a8FBaSysB3KHsiAoQ8sol6 a3rw== X-Gm-Message-State: APt69E0MYa+Fg71kh5zRnaJdvTk3WuynUS3Jx60wx2KgZ/XMHnB07EUl 9ruVZIAtfPT73ADYC9veUsM6FphdmFd4HlR0VQobotXoo7WYfhvRH76sdpgVHK3GVpnGQg3I9v7 R/b8m4jYir/Wkh7VlpX4eYygZaLV4hOL1Or371xHVJP4d/xDIkK3iJmQpyCXuVPgPb5tBxebvSw NrbpKe6rDJ/GLaOmdA4/fdJ/1AIRyCxexr48bRS1e2zahkpQzqg/F2lURmc6tcBgfp2sFpkR82N RWjPGUehWWu/DnZPNMbYX6ALPQSbGbfd5+4GfCHZxdpA0qmBZcc9U1E+WGTyXBtwuvzAEvxUmV/ CdqIfe6t04qPy+zoOPACts/Gseusyv5dnhgN8pY8J75ZuYvSoZQH19xdRPmg+0lS3qibs4O3+Dy H X-Received: by 2002:a63:6107:: with SMTP id v7-v6mr680492pgb.264.1528944580698; Wed, 13 Jun 2018 19:49:40 -0700 (PDT) X-Received: by 2002:a63:6107:: with SMTP id v7-v6mr680443pgb.264.1528944579446; Wed, 13 Jun 2018 19:49:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528944579; cv=none; d=google.com; s=arc-20160816; b=ZYqk+499L/37bxUrVrtVG7OwTmwLzefau3BBtJIWiyeCi8q7qtCdiU0zuc+84s2D9C TZ/aPYGyOvHVHNp0PdRBzW5oO0hh4Zy7XLxK/xT6ba+1up2WGNstxKURoHecBaoZ2k8L TDC+dIy1AEOBhNFbX35Kho6CrrSgj3GKiqmVUYBGHwGlDw6MQGycfvnC65FyVxC68TRZ OpoPiSeUg/xz4JSRm9Y39Z9BCAMeXzdwSEJ/+CC6asxfW02T0Qaa9UP7otbLzxIUlBfv YongTkBZ6b5THdn+t2Ut15O+njjRJNh9rhA6eqZ/cV3mP9TzI0Oz2THNlQ4fqDsmQkfG Mu/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:dkim-signature :arc-authentication-results; bh=R5p7AbKU2jjzhdJXZ3QBYGGRaZb93kpUqzc4IWfZr40=; b=WcW9ljmNw9EClxMFWxkNYqpGJ+7Q1W5cymtwp66QEiwn543gZZgAbB1RWPQjcXjmVC 5h6qc79yjFl6NgOLdozn5frQLh2u3lI7TqnJjdypXN7F0N2YjunuKm3DmJreVGOPdJvc h5C9rghuFwgnQTabeWWgPU7AqjE2cvXSPa8ALicuETRgifbE7Gvqs3gD11WqyaZJ9aML Znv7HeQGMscXea33utKgbpda1nqrLezJyi27uvIUd0c4a2yCJmJcYEWYtbMf/SElWUXC L16Z4hiF4F56R7L9VPPzRCcJ95oXTsQYM5PJVuQg0Ads9bsVp4sK6aCaFxvcj/ot227g 1J8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=T4QJhT4o; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id y18-v6sor1349736pfj.44.2018.06.13.19.49.39 for (Google Transport Security); Wed, 13 Jun 2018 19:49:39 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=T4QJhT4o; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=R5p7AbKU2jjzhdJXZ3QBYGGRaZb93kpUqzc4IWfZr40=; b=T4QJhT4oAM7163D1/UvXtANXLeNpYM+6vpsaWkzJ/1gJM/WD+DJvFaHBYTiGNz+KA9 wjDSALnByX4UOyYzHHwqDoTPPNbARsI86Rx1NG/VKuR0FQZ/etAYSa63qXHlcKQ1btZS MQaQBwapU97vj+LXngXqfK7U8UDYaEaQURqjev5M2EC6nxiJIUntmwvnOTxJ0zWHYPO7 3fFd5XLzWILj2LxqWQtiodmP1eQRcOVSCnZL2iscp/+RrxOOoofVB1H/hRsvjVtBp/9Y 2lL7+BjaBM5MmCGuHiWxi3t9Y+3J1lSzkNVcLpH3XnMZ/JH9oVLfXmhPN4KHHqLgovuB MQ8g== X-Google-Smtp-Source: ADUXVKISYxy7NcTC/PAktfKpAg+iZJUz13v49oxUvaJxqnksQIJeixLaCpY7j/EszvQ+cq8F/HkcxA== X-Received: by 2002:a62:f615:: with SMTP id x21-v6mr7474268pfh.43.1528944579146; Wed, 13 Jun 2018 19:49:39 -0700 (PDT) Received: from roar.ozlabs.ibm.com (59-102-70-78.tpgi.com.au. [59.102.70.78]) by smtp.gmail.com with ESMTPSA id w7-v6sm4402830pgs.68.2018.06.13.19.49.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 13 Jun 2018 19:49:38 -0700 (PDT) Date: Thu, 14 Jun 2018 12:49:31 +1000 From: Nicholas Piggin To: Linus Torvalds Cc: linux-mm , ppc-dev , linux-arch , "Aneesh Kumar K. V" , Minchan Kim , Mel Gorman , Nadav Amit , Andrew Morton Subject: Re: [RFC PATCH 3/3] powerpc/64s/radix: optimise TLB flush with precise TLB ranges in mmu_gather Message-ID: <20180614124931.703e5b54@roar.ozlabs.ibm.com> In-Reply-To: References: <20180612071621.26775-1-npiggin@gmail.com> <20180612071621.26775-4-npiggin@gmail.com> <20180613083131.139a3c34@roar.ozlabs.ibm.com> <20180613090950.50566245@roar.ozlabs.ibm.com> <20180613101241.004fd64e@roar.ozlabs.ibm.com> X-Mailer: Claws Mail 3.16.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP On Tue, 12 Jun 2018 18:10:26 -0700 Linus Torvalds wrote: > On Tue, Jun 12, 2018 at 5:12 PM Nicholas Piggin wrote: > > > > > > And in _theory_, maybe you could have just used "invalpg" with a > > > targeted address instead. In fact, I think a single invlpg invalidates > > > _all_ caches for the associated MM, but don't quote me on that. > > Confirmed. The SDK says > > "INVLPG also invalidates all entries in all paging-structure caches > associated with the current PCID, regardless of the linear addresses > to which they correspond" Interesting, so that's very much like powerpc. > so if x86 wants to do this "separate invalidation for page directory > entryes", then it would want to > > (a) remove the __tlb_adjust_range() operation entirely from > pud_free_tlb() and friends Revised patch below (only the generic part this time, but powerpc implementation gives the same result as the last patch). > > (b) instead just have a single field for "invalidate_tlb_caches", > which could be a boolean, or could just be one of the addresses Yeah well powerpc hijacks one of the existing bools in the mmu_gather for exactly that, and sets it when a page table page is to be freed. > and then the logic would be that IFF no other tlb invalidate is done > due to an actual page range, then we look at that > invalidate_tlb_caches field, and do a single INVLPG instead. > > I still am not sure if this would actually make a difference in > practice, but I guess it does mean that x86 could at least participate > in some kind of scheme where we have architecture-specific actions for > those page directory entries. I think it could. But yes I don't know how much it would help, I think x86 tlb invalidation is very fast, and I noticed this mostly at exec time when you probably lose all your TLBs anyway. > > And we could make the default behavior - if no architecture-specific > tlb page directory invalidation function exists - be the current > "__tlb_adjust_range()" case. So the default would be to not change > behavior, and architectures could opt in to something like this. > > Linus Yep, is this a bit more to your liking? --- include/asm-generic/tlb.h | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index faddde44de8c..fa44321bc8dd 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -262,36 +262,49 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb, * architecture to do its own odd thing, not cause pain for others * http://lkml.kernel.org/r/CA+55aFzBggoXtNXQeng5d_mRoDnaMBE5Y+URs+PHR67nUpMtaw@mail.gmail.com * + * Powerpc (Book3S 64-bit) with the radix MMU has an architected "page + * walk cache" that is invalidated with a specific instruction. It uses + * need_flush_all to issue this instruction, which is set by its own + * __p??_free_tlb functions. + * * For now w.r.t page table cache, mark the range_size as PAGE_SIZE */ +#ifndef pte_free_tlb #define pte_free_tlb(tlb, ptep, address) \ do { \ __tlb_adjust_range(tlb, address, PAGE_SIZE); \ __pte_free_tlb(tlb, ptep, address); \ } while (0) +#endif +#ifndef pmd_free_tlb #define pmd_free_tlb(tlb, pmdp, address) \ do { \ - __tlb_adjust_range(tlb, address, PAGE_SIZE); \ + __tlb_adjust_range(tlb, address, PAGE_SIZE); \ __pmd_free_tlb(tlb, pmdp, address); \ } while (0) +#endif #ifndef __ARCH_HAS_4LEVEL_HACK +#ifndef pud_free_tlb #define pud_free_tlb(tlb, pudp, address) \ do { \ __tlb_adjust_range(tlb, address, PAGE_SIZE); \ __pud_free_tlb(tlb, pudp, address); \ } while (0) #endif +#endif #ifndef __ARCH_HAS_5LEVEL_HACK +#ifndef p4d_free_tlb #define p4d_free_tlb(tlb, pudp, address) \ do { \ - __tlb_adjust_range(tlb, address, PAGE_SIZE); \ + __tlb_adjust_range(tlb, address, PAGE_SIZE); \ __p4d_free_tlb(tlb, pudp, address); \ } while (0) #endif +#endif #define tlb_migrate_finish(mm) do {} while (0)