From patchwork Mon Jul 23 22:34:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Rientjes X-Patchwork-Id: 10541261 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 03B2B13BB for ; Mon, 23 Jul 2018 22:35:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E44DF284C7 for ; Mon, 23 Jul 2018 22:35:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D5AF32855D; Mon, 23 Jul 2018 22:35:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_NONE, USER_IN_DEF_DKIM_WL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6F8CC284C7 for ; Mon, 23 Jul 2018 22:35:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 21A556B0007; Mon, 23 Jul 2018 18:35:00 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1C7AC6B0008; Mon, 23 Jul 2018 18:35:00 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 090D66B000A; Mon, 23 Jul 2018 18:35:00 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl0-f71.google.com (mail-pl0-f71.google.com [209.85.160.71]) by kanga.kvack.org (Postfix) with ESMTP id B69B26B0007 for ; Mon, 23 Jul 2018 18:34:59 -0400 (EDT) Received: by mail-pl0-f71.google.com with SMTP id 66-v6so1317663plb.18 for ; Mon, 23 Jul 2018 15:34:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:dkim-signature:date:from:to:cc:subject :message-id:user-agent:mime-version; bh=KsV/kWTltH7eIg5UDgd4mIs/DmKDGJ0gizBcV5nwZVY=; b=UdglhSpScY6tpt3EqGlW+QRlTJBSR6ERZgnPnUgDkhNaexlKw1SPRGTYt/LjKjTPNv 5/hK2/Xgkcsd1JiK5ZVeRxDHGeh/CMACU6UZjKUcsj4CBwmXkYENDo47jKlXt3C+y9sv swpTPhSCaNQUdX+7iXUexQYphuMnClW/mqJB8WO3ZiAH5nHz2No5EeSHLZL2IBEzt7HW DM2LjtHkCbLMJzmVp77IzxGk6vP2MYldAy4iBHV09s7V0fxsMRBns5UtJOdjKyaIH4QH 7Y8AOoFG8b7GTr/i+zrhs4lnU/uuD1JRjXdrMhL5CkFR+ndpIBopwawjbIAmF4xaXa3m yFwg== X-Gm-Message-State: AOUpUlFd0wAWtP+tYU8wx5N0XJVR5LrCVO/VqpQDzowB2W2zqamO2AHr wgjtVLP9lF7hDcYMU5rENhRYYlJUeba9tuD5E3tG7PPWQHoBVPIlmNbAsWRBku5JtBaor0+nHHv nkT9XBP1OPtM68+1oIL5AD35ZX1/0S5GXD3rdxgnJu/cTYxH+4kJrTWlv0HRRC7Jmtav05ZiTbw /FHvey8BX+kGQNn9PMmFkd5/qWAW+JMdq1Zb8oZMVtfLWw//zA7soF8TP2mmrI2KPoQDhcDZ1yJ NbG1vQQuhiJmhxPN68zMxc39lmV/xXLJfQXC34+aaGyYIZLs8Dppz8BKiM5OEstVDYZu0fyjZ+w zmky/qrqDIwvYWb9L2UZjjMyYFZpPDrWKJ4gu5hEk5j7BOUMriPk5dlm02yaBEVXQGUztuotSqG e X-Received: by 2002:a17:902:820a:: with SMTP id x10-v6mr14600850pln.261.1532385299411; Mon, 23 Jul 2018 15:34:59 -0700 (PDT) X-Received: by 2002:a17:902:820a:: with SMTP id x10-v6mr14600821pln.261.1532385298682; Mon, 23 Jul 2018 15:34:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1532385298; cv=none; d=google.com; s=arc-20160816; b=nh2LQXUQkxx4ph0LP+D/xbPz6j0HdMvIvYuGnTdyi9sOesVavjCem5sh4OjMMM0zQu o0Th2e33xL6QjitLtJORa/lRTHzEYRrn0B7u5ibJkdpg9zhxW3kP6/knEABVRkTMazBu TKiqIMQWTICmb1ICcCtA7AhbKTmpxOe0iMuT/kAj3BwGO/PEjs+IO+OynhvGuO7bTMCP 2taHBsM0L2KPPlbglSKBb7rezSlw5WhPcWuhm3yPHXvOvwwm6OMvvpopvsCruK7Xk14c Nx/fjR9S7GhIsIwLB48C0BnQWkoS3X7N5bwvS6JESkYHAtXqjzyFX84NMpG4Ur7Z8mdB G6wQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:subject:cc:to:from:date :dkim-signature:arc-authentication-results; bh=KsV/kWTltH7eIg5UDgd4mIs/DmKDGJ0gizBcV5nwZVY=; b=zyttakcEwlSt2Inc6YAOMC6ajtTppBqSjgOkxHFzema0d2VuPGaXGisEbrwkEydZYN 6I+qe1h3vrl+ZwiOjOLFbk5/dp6og59MTE0cstuAuOHGZr6H7DY9If047AyikqlveI0D I+Ig7KmN+GyCBhjUbRTXDJUoIzbzGXzoP57AyRyUp/moPj6H0gHh/UTRfHaz9c41Fudo cgnJ/RHrB/3um1L9Eo0OOvOugg7mu8xC/bdvWkPvflE1qbs6ZuBoKxhCd7emDbGt5glq A4hNvfmb5xXPUBdyvmEZRbY6VRHv0Bd+QXE5FfrOhG4aADKojSxTdCZ1qI3VgsdIdDE6 Xuiw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="WnuFRI/s"; spf=pass (google.com: domain of rientjes@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id s24-v6sor2837912pfm.12.2018.07.23.15.34.58 for (Google Transport Security); Mon, 23 Jul 2018 15:34:58 -0700 (PDT) Received-SPF: pass (google.com: domain of rientjes@google.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b="WnuFRI/s"; spf=pass (google.com: domain of rientjes@google.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=rientjes@google.com; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:user-agent:mime-version; bh=KsV/kWTltH7eIg5UDgd4mIs/DmKDGJ0gizBcV5nwZVY=; b=WnuFRI/srZgi1qXZcF7qL9ahfPS5thNJAHRJLZ6I+dCVQ4HC1WUb7CbmPoqkhGLMvn ccLZj7LJNrFuDHcu3dmftdyLZEban8MbtjMTsxj1g4/aYZMJjt763d8w9zPL5BL2tFD7 MAboNjyVVzGL8PT0e5ESokJtdCqKun9W0yjTlR6QM9qXD0M9E44VtTs55WT8H35191lv 4j/V3LDxslR81eRMq63QE7ikuj6+nReUeuqGgrwThhGoRWxQMHLPydBR4cRlQoj5JnPK yoU8y740Tj0YUHtwuKIHYLXK+LFbUimC2QtlbRIbS+q0fQGNDvEkwc/Hr+Buny8k+QZl fnOA== X-Google-Smtp-Source: AAOMgpfeK42cHX+zvJsyeErXzfPIwopsmIQTKhxO+QjPNPznxtMMUPYxKTuzSLdXkt7Fj9TQDzeCzQ== X-Received: by 2002:a62:d544:: with SMTP id d65-v6mr14834768pfg.107.1532385298023; Mon, 23 Jul 2018 15:34:58 -0700 (PDT) Received: from [2620:15c:17:3:3a5:23a7:5e32:4598] ([2620:15c:17:3:3a5:23a7:5e32:4598]) by smtp.gmail.com with ESMTPSA id g86-v6sm14095338pfd.23.2018.07.23.15.34.57 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 23 Jul 2018 15:34:57 -0700 (PDT) Date: Mon, 23 Jul 2018 15:34:56 -0700 (PDT) From: David Rientjes X-X-Sender: rientjes@chino.kir.corp.google.com To: Andrew Morton cc: kbuild-all@01.org, lkp@intel.com, Johannes Weiner , linux-mm@kvack.org Subject: [patch -mm] mm, vmacache: hash addresses based on pmd fix Message-ID: User-Agent: Alpine 2.21 (DEB 202 2017-01-01) MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP - use PMD_SHIFT only if CONFIG_MMU is used, otherwise there is only pgdir, per kbuild test robot - fix vmacache_find_exact() for correct formal name, per me - only check vma->vm_mm == mm for CONFIG_DEBUG_VM_VMACACHE, per akpm Tested for {allnoconfig, defconfig} on alpha, arc, arm, arm64, c6x, h8300, i386, ia64, m68k, microblaze, mips, mips, nds32, nios2, parisc, powerpc, riscv, s390, sh, sparc, um, and xtensa. Reported-by: kbuild test robot Signed-off-by: David Rientjes --- mm/vmacache.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/mm/vmacache.c b/mm/vmacache.c --- a/mm/vmacache.c +++ b/mm/vmacache.c @@ -6,12 +6,18 @@ #include #include #include +#include /* - * Hash based on the pmd of addr. Provides a good hit rate for workloads with - * spatial locality. + * Hash based on the pmd of addr if configured with MMU, which provides a good + * hit rate for workloads with spatial locality. Otherwise, use pages. */ -#define VMACACHE_HASH(addr) ((addr >> PMD_SHIFT) & VMACACHE_MASK) +#ifdef CONFIG_MMU +#define VMACACHE_SHIFT PMD_SHIFT +#else +#define VMACACHE_SHIFT PAGE_SHIFT +#endif +#define VMACACHE_HASH(addr) ((addr >> VMACACHE_SHIFT) & VMACACHE_MASK) /* * Flush vma caches for threads that share a given mm. @@ -105,8 +111,10 @@ struct vm_area_struct *vmacache_find(struct mm_struct *mm, unsigned long addr) struct vm_area_struct *vma = current->vmacache.vmas[idx]; if (vma) { +#ifdef CONFIG_DEBUG_VM_VMACACHE if (WARN_ON_ONCE(vma->vm_mm != mm)) break; +#endif if (vma->vm_start <= addr && vma->vm_end > addr) { count_vm_vmacache_event(VMACACHE_FIND_HITS); return vma; @@ -124,7 +132,7 @@ struct vm_area_struct *vmacache_find_exact(struct mm_struct *mm, unsigned long start, unsigned long end) { - int idx = VMACACHE_HASH(addr); + int idx = VMACACHE_HASH(start); int i; count_vm_vmacache_event(VMACACHE_FIND_CALLS);