From patchwork Thu Aug 23 08:47:08 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10573567 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7FB71112E for ; Thu, 23 Aug 2018 08:47:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6D1CD2BA27 for ; Thu, 23 Aug 2018 08:47:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 616272BA41; Thu, 23 Aug 2018 08:47:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F03B32BA27 for ; Thu, 23 Aug 2018 08:47:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D3B026B2924; Thu, 23 Aug 2018 04:47:29 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id CEA6B6B2925; Thu, 23 Aug 2018 04:47:29 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BB13D6B2926; Thu, 23 Aug 2018 04:47:29 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f72.google.com (mail-pl0-f72.google.com [209.85.160.72]) by kanga.kvack.org (Postfix) with ESMTP id 6A8E46B2924 for ; Thu, 23 Aug 2018 04:47:29 -0400 (EDT) Received: by mail-pl0-f72.google.com with SMTP id bg5-v6so2230177plb.20 for ; Thu, 23 Aug 2018 01:47:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=IRhZgLPRx4tzi/e3fxP2vHoZGo1V/TLqfKDUUNoAm38=; b=lXqnTK5HWhT3G4JSls5Fu9yIpnMdwDWRwXxbs9iwsX3gtsqlZHnn94UcbB3UIGExHo xxOtHlra5gHwEOYlKN0Knul1oNfBOQTH7Ml//qOWN83fbaWqUm7j4H3ZBTAuIOU+Y7eI QY4vFUth8AyPU+pDrF4DPlvWSmQnn0FBymLQ3OnuvCAMrnuBcyhd//3RtVLXhh/h81ws 4HxgTONWxPMe8cM2oh07hV6l4U7Wt3R1My841QvDM8Izmua6oBl2W9jz6Rqyd+9eEIwS EdBcY909tgfWTIjqmZFx49/LfmGnI4uMlIOlWJESGVrAKFFCabDuLE1/u8v/phs9t+zl ebyg== X-Gm-Message-State: AOUpUlEpU6ySkU4b6habz6r9cb3EEyaed2YdZLHva9izW1PfYcETUyVm 6oAea3plTJC8tCw5n0kHj1dYNoAqxVvktyf26p2DGAtT6alIYQ/H/VxWWz2AatTiKJ0Ue/zB5sW wBkn/rXWpOedDPxBOZ4owWnBbwQ63AoAUX/h5z+GlSI5bLSP3RfCzNgR+MUy/7g7tB0ATOySghL WbC0zmwupCclc+Mcc9IcJFQkJxpkn8s76DZT/W+HoAwee+SGW1U0g9eh5k3jebBowRWeliCRPC9 Y1bALrRRHrmnjBmPmgdyZh0P7Z857Ny5K4PXU0frh+yvHxguO3FtPt51hAmijiQ1ZYw/OtemawK bT+k8RQH4eyUAWm7X8/yN3+aNbbMnUQH9WhZ3tQrzvPRGHINpSabMlGqZm/eSirQiRjyFDdh1ZO 6 X-Received: by 2002:a63:5856:: with SMTP id i22-v6mr15368069pgm.232.1535014049021; Thu, 23 Aug 2018 01:47:29 -0700 (PDT) X-Received: by 2002:a63:5856:: with SMTP id i22-v6mr15368043pgm.232.1535014048328; Thu, 23 Aug 2018 01:47:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535014048; cv=none; d=google.com; s=arc-20160816; b=ctcKv0/tqLVoOe89Z5t/v9SeYiUh99iBh1Lm421/jFvVvV06Dv4Ugr8HIpswmaMp4V 7B72myqTc/gftXn5zJexoaEgjIiMSdO/TjWDnRSObRd9jSX4NRU37uw+PGCXbt7+6HA9 86VnLsgiNLa++f3ZE81WHtuQny+qWDcHdvaLXKJlVngVKTnHUrpQYxF9hwoEAtCzRH55 HWPx2K39PZvzoLeFDuUMKJLXLiFbwdBZHxtpb1bxMwTYSeVzeh1qdNdkUg42OLEKLzoi fQ3GJTWpAen1BETIrF+pI3ETmIeavIRdTmBuEfP0uLhhJZrSzdFGP9dReM3sVN7x4LBJ EfLA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=IRhZgLPRx4tzi/e3fxP2vHoZGo1V/TLqfKDUUNoAm38=; b=DUq6HPS3NqLGLpcrGYGE2FtXbJ8lbW3s702GeQRADlMIwGt/aHCfHsXQrZJNAjcPsN c5BuHdB+nENNfB7iTJNCzVzPPR3WZFvPM6ULIweWpMFRwrQquOimwLLXyIF4bpvvkKqo HPvit/RQg+5AYeBfL8NbjcySFXxP61vxAxuZIHMhlGL43lR4Nma7+eLYUd6vqzMzGemb KvPNLe4E/KobCzqq3ia1z8A/taB+3zeS6ORZieV/esxSAuKYeOGLu2dwOSJxt3pXhw/Z kie2APcXe8s3iW7FQg/0bAcD9ozwdJ6+mawgtcazcqH2VKnjZSbNlJ0YI3lohhspI+br 6Uvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=e85lCNW5; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id v2-v6sor1111968pgt.196.2018.08.23.01.47.28 for (Google Transport Security); Thu, 23 Aug 2018 01:47:28 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=e85lCNW5; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=IRhZgLPRx4tzi/e3fxP2vHoZGo1V/TLqfKDUUNoAm38=; b=e85lCNW5ZhdjUCei/RAj9byezOvy9sEmiiZkPkURq+UAHjwaE5/PxcfLUapq8XVJlx pWoX2yLTmaYHUL4Yfgg1iYWsY9wPSO8gwLR7qh58C5BrmBbJkiKMhK2whmZZpP3h+kd0 MRW0MDqKR/7vgpjZJrJ7FIHLKjH/U0pfrvPoBQZXbhEQ0LNW2c4OBwVhhYET0X5XChl1 YFAW706vZ4vJwGDdKbtiuwMAjyv7nyWXFnwwOMAjDaoOyMWikwDcvWDfaNLPI+JCr768 /xJSyNYzqbXu6PKmfyoDn30oJq6Z+etT3FcNl6LjVJtUjvhPdfu883Zi+1ZQ1NE3vj8Q Wrpg== X-Google-Smtp-Source: AA+uWPzgnKVeyGDrwd1X5Y4sSxuoZ5U4azvkGjVyP7iTgSuBQqhduD/7xVxtxH+hKFgoJPhXiXJFnA== X-Received: by 2002:a63:1c47:: with SMTP id c7-v6mr16501905pgm.38.1535014048116; Thu, 23 Aug 2018 01:47:28 -0700 (PDT) Received: from roar.ozlabs.ibm.com. ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id w5-v6sm5170503pfn.44.2018.08.23.01.47.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Aug 2018 01:47:27 -0700 (PDT) From: Nicholas Piggin To: Peter Zijlstra Cc: Nicholas Piggin , torvalds@linux-foundation.org, luto@kernel.org, x86@kernel.org, bp@alien8.de, will.deacon@arm.com, riel@surriel.com, jannh@google.com, ascannell@google.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Miller , Martin Schwidefsky , Michael Ellerman , linux-arch@vger.kernel.org Subject: [RFC PATCH 1/2] mm: move tlb_table_flush to tlb_flush_mmu_free Date: Thu, 23 Aug 2018 18:47:08 +1000 Message-Id: <20180823084709.19717-2-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180823084709.19717-1-npiggin@gmail.com> References: <20180823084709.19717-1-npiggin@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP There is no need to call this from tlb_flush_mmu_tlbonly, it logically belongs with tlb_flush_mmu_free. This allows some code consolidation with a subsequent fix. Signed-off-by: Nicholas Piggin Acked-by: Will Deacon Reviewed-by: Rik van Riel --- mm/memory.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 19f47d7b9b86..7c58310734eb 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -245,9 +245,6 @@ static void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) tlb_flush(tlb); mmu_notifier_invalidate_range(tlb->mm, tlb->start, tlb->end); -#ifdef CONFIG_HAVE_RCU_TABLE_FREE - tlb_table_flush(tlb); -#endif __tlb_reset_range(tlb); } @@ -255,6 +252,9 @@ static void tlb_flush_mmu_free(struct mmu_gather *tlb) { struct mmu_gather_batch *batch; +#ifdef CONFIG_HAVE_RCU_TABLE_FREE + tlb_table_flush(tlb); +#endif for (batch = &tlb->local; batch && batch->nr; batch = batch->next) { free_pages_and_swap_cache(batch->pages, batch->nr); batch->nr = 0; From patchwork Thu Aug 23 08:47:09 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicholas Piggin X-Patchwork-Id: 10573569 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6537B112E for ; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 50EA22BA27 for ; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 44F7D2BA41; Thu, 23 Aug 2018 08:47:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 24FC22BA27 for ; Thu, 23 Aug 2018 08:47:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C2D3D6B2926; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id BD7936B2927; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2DAE6B2928; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f197.google.com (mail-pf1-f197.google.com [209.85.210.197]) by kanga.kvack.org (Postfix) with ESMTP id 51C786B2926 for ; Thu, 23 Aug 2018 04:47:35 -0400 (EDT) Received: by mail-pf1-f197.google.com with SMTP id c8-v6so2847425pfn.2 for ; Thu, 23 Aug 2018 01:47:35 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:from:to:cc:subject:date :message-id:in-reply-to:references; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=hMs0UYqmAYuOLrmGOfcNsydLo4EVbB5NzHfLiiDEH3CfKTTZAoBSvyN9RIwu9Jc35U hNBg4jnZkj/3iO/+mW2/8nU9b9UI0HsBgeyoRoOutiB1hfOkmTzMo3azka7Ue52nEdh/ Ax5MZwoua/IZvjd9hCFHvHbzMeVR2CPtk36BnufFEcw+we2Qa3hpHlLBonu9dm4KN+wV WUhPvK5rpCUn1/SB9bDggn7912+KJmODHlGpcPVEjIEEme+NnCos3HdzaKN8hnhLOUUk iZ9q5+JLQG6pwS21HfsjM/EaIdqd058nf2/wN2WZXNxkU7D24v/ccLot4fLGl2romTf6 tYig== X-Gm-Message-State: AOUpUlGO/W6LDDTunuVfQPKEMIet/n+J9O4ShYz50V6GEID8avzfwupU SYnkcEbD0TTJxynvBe60EkDuy7pAEaeY94XR+z1hgMGxhBHONLNMxxVqCroiOvmz8Uhs8Jhq5Ta B8tnyB/TIbvVU1fcL5EyJfaM/ofshSI6koZQGFX4fE0gZW0wuuY7exZ6RaLYt6sVkeiSPTUGG1o ES1rTTkSlYgUvxsF5VnLIixRXpMdNNd2AhisfN7Tz1UymMSmTzzkhtYJjhl2sUHD3qGV7D3k+GR XoR6ln5lbr6piHOQUy24ZsbRV1H/eOQo+LwBjuDIbmq21gRL5rpbw3aKms+V8mqW+6bD8bRgf/L sQVjrP24exRa0sjYGFZLSqHaLsT15MVvSns4hYPFjoX5kSWzUZbtlsBAK0kYEzvYzdnZxyqtR1M H X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr62233762pfd.32.1535014054964; Thu, 23 Aug 2018 01:47:34 -0700 (PDT) X-Received: by 2002:a62:8559:: with SMTP id u86-v6mr62233723pfd.32.1535014054145; Thu, 23 Aug 2018 01:47:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535014054; cv=none; d=google.com; s=arc-20160816; b=SVG9459EtuClUImj1r+U5b+GqyWOqbluVS59DWFbrPxVnww0DYx2CotQNi5aBnprqC x1AotvFj6LMI0678ThioSPRWGTBBH05jdRtFvcxboCQoyo0yUnPniLKItz1VgMi78Veh /Z1gG7ydhUq8ffCZQYycSKAbEnPLiy0CZISTwrFreoho/TwMZwYaeblbQGCqZpc4tJvQ her9hjY/IW5XOEPv5oAcNr7DsHdJCPo1K/mrlXz1T8NjoQ9UTyCrH60v34dXJUe2y8zq zobH72VbSmg1VdpQ/IB+rRhfWvK7elPiGcKwo1F8l9cbc7lXBaxm/LSeVWryQ1h0UIQh /xYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=EreiySvDT3pTflUw+Rnry9uc42Ayr5rlpbuTyn4YwFEuszYJX34FGj4V7gCG9bcm5d +M1M7Pcfi5Q5duBezlHemEv5EHap+tG8HkpCJB7ZW5xasmmwZ494p50yJwi5gi58aA1Q 2hYxGm8S1L9L5cJ+aoOV7kOylYb9loS0Nrkde14WkYedU3iZ3MB6wAnm9s/jbouvlTTP fwK+d45b6rMLNnc1hRCZ2GOLCBV0jYwgxf2L6lVAF9wmoMIgpCw6SQv6j3qK44YXgzpa 1uBPkEcz6JEOW4aqCXSb6MkKMesVdAka0qlGDbMMVNEl2acmtFCiMRdObIOOcSIi12EA N9Cg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gEYBvPCj; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from mail-sor-f65.google.com (mail-sor-f65.google.com. [209.85.220.65]) by mx.google.com with SMTPS id r37-v6sor735631pgb.330.2018.08.23.01.47.34 for (Google Transport Security); Thu, 23 Aug 2018 01:47:34 -0700 (PDT) Received-SPF: pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) client-ip=209.85.220.65; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=gEYBvPCj; spf=pass (google.com: domain of npiggin@gmail.com designates 209.85.220.65 as permitted sender) smtp.mailfrom=npiggin@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=vm5WJYEScMr7UWDWXPbr88R9HWjvqSW7wxj8DCyVjYs=; b=gEYBvPCjdTml9yIgaT/W/ou3ACAqcWJwpC5ZL0E4EgrZoLqbUra2Rwg1DAYe8iKJhf zq3c95YV4k46qxBQyAKkdclcDfYUib1+cgop+zBiCUT1aekJgyWIbamjOYVe7O/+iXAv dOLLpwCgvXwZA2E8OixpCseW2sZuiHSRJBY6/fuOO0tBZ5/afFYhbAhwVul45WraIpvV HD75sXtJTgxrQ+i72/GsCGEUmKr5+KTPURlVS6aIMNqgVfAcIpT4Fzyt3IO0qOKyIFsa 4AXjJ3xm8omnlemZtY23Zhj3RYm1ae2mTuUPQszHgEJ1W6R9eM/3gSxzLjUE4K08Od5c 4yKg== X-Google-Smtp-Source: AA+uWPyGlyWnj3nMPrxY4SPnBTjWZjcKNxEdhGijGKj4PrMsrgME/v8IMpsaIZLBPJdZiyrMNgxmmA== X-Received: by 2002:a63:2e09:: with SMTP id u9-v6mr52325298pgu.294.1535014053904; Thu, 23 Aug 2018 01:47:33 -0700 (PDT) Received: from roar.ozlabs.ibm.com. ([122.99.82.10]) by smtp.gmail.com with ESMTPSA id w5-v6sm5170503pfn.44.2018.08.23.01.47.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 23 Aug 2018 01:47:33 -0700 (PDT) From: Nicholas Piggin To: Peter Zijlstra Cc: Nicholas Piggin , torvalds@linux-foundation.org, luto@kernel.org, x86@kernel.org, bp@alien8.de, will.deacon@arm.com, riel@surriel.com, jannh@google.com, ascannell@google.com, dave.hansen@intel.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, David Miller , Martin Schwidefsky , Michael Ellerman , linux-arch@vger.kernel.org Subject: [RFC PATCH 2/2] mm: mmu_notifier fix for tlb_end_vma Date: Thu, 23 Aug 2018 18:47:09 +1000 Message-Id: <20180823084709.19717-3-npiggin@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180823084709.19717-1-npiggin@gmail.com> References: <20180823084709.19717-1-npiggin@gmail.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP The generic tlb_end_vma does not call invalidate_range mmu notifier, and it resets resets the mmu_gather range, which means the notifier won't be called on part of the range in case of an unmap that spans multiple vmas. ARM64 seems to be the only arch I could see that has notifiers and uses the generic tlb_end_vma. I have not actually tested it. Signed-off-by: Nicholas Piggin Acked-by: Will Deacon Reported-by: Guenter Roeck Signed-off-by: Will Deacon --- include/asm-generic/tlb.h | 17 +++++++++++++---- mm/memory.c | 10 ---------- 2 files changed, 13 insertions(+), 14 deletions(-) diff --git a/include/asm-generic/tlb.h b/include/asm-generic/tlb.h index e811ef7b8350..79cb76b89c26 100644 --- a/include/asm-generic/tlb.h +++ b/include/asm-generic/tlb.h @@ -15,6 +15,7 @@ #ifndef _ASM_GENERIC__TLB_H #define _ASM_GENERIC__TLB_H +#include #include #include #include @@ -138,6 +139,16 @@ static inline void __tlb_reset_range(struct mmu_gather *tlb) } } +static inline void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) +{ + if (!tlb->end) + return; + + tlb_flush(tlb); + mmu_notifier_invalidate_range(tlb->mm, tlb->start, tlb->end); + __tlb_reset_range(tlb); +} + static inline void tlb_remove_page_size(struct mmu_gather *tlb, struct page *page, int page_size) { @@ -186,10 +197,8 @@ static inline void tlb_remove_check_page_size_change(struct mmu_gather *tlb, #define __tlb_end_vma(tlb, vma) \ do { \ - if (!tlb->fullmm && tlb->end) { \ - tlb_flush(tlb); \ - __tlb_reset_range(tlb); \ - } \ + if (!tlb->fullmm) \ + tlb_flush_mmu_tlbonly(tlb); \ } while (0) #ifndef tlb_end_vma diff --git a/mm/memory.c b/mm/memory.c index 7c58310734eb..c4a7b6cb17c8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -238,16 +238,6 @@ void arch_tlb_gather_mmu(struct mmu_gather *tlb, struct mm_struct *mm, __tlb_reset_range(tlb); } -static void tlb_flush_mmu_tlbonly(struct mmu_gather *tlb) -{ - if (!tlb->end) - return; - - tlb_flush(tlb); - mmu_notifier_invalidate_range(tlb->mm, tlb->start, tlb->end); - __tlb_reset_range(tlb); -} - static void tlb_flush_mmu_free(struct mmu_gather *tlb) { struct mmu_gather_batch *batch;