From patchwork Sat Jul 14 09:32:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Srivatsa S. Bhat" X-Patchwork-Id: 10524655 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7D9686028E for ; Sat, 14 Jul 2018 09:32:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6A30927BA5 for ; Sat, 14 Jul 2018 09:32:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5C57129123; Sat, 14 Jul 2018 09:32:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC87C27BA5 for ; Sat, 14 Jul 2018 09:32:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4ABA6B000D; Sat, 14 Jul 2018 05:32:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id E20BB6B000E; Sat, 14 Jul 2018 05:32:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D119C6B0010; Sat, 14 Jul 2018 05:32:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt0-f197.google.com (mail-qt0-f197.google.com [209.85.216.197]) by kanga.kvack.org (Postfix) with ESMTP id A3BE36B000D for ; Sat, 14 Jul 2018 05:32:14 -0400 (EDT) Received: by mail-qt0-f197.google.com with SMTP id x9-v6so5817367qto.18 for ; Sat, 14 Jul 2018 02:32:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:subject:from :to:cc:date:message-id:in-reply-to:references:user-agent :mime-version:content-transfer-encoding; bh=2c+V7XqZne2T1CJZKoomzUzu/8A2csiffY4Rc67oym4=; b=bjTkU9hfLVDuEyvKyYg+e7yF+OqeiVgqA4v6mh6VJdMB5ygA/4Ubx1KBc1/4OckBTQ 22UvrnamCeSO0J2YHv5KlGJUyR4E16QkcRuDqXgW39S/zVuHdro557J6P0bP5cl99JqJ hWSxYS/em04dtNoT2O2A8BCWTgJ/0aS6FF9Akpw1hF2RuCvbVXnhPShdlLQlVk+/LZty HLLETdWS9GKGf31LMkbs3ssik+aTa6HA0MK0nPv7788V07dnKGIeMSGhw8PXkBlwXlwb CFzWhKiKHdqEddEDZZwi9KlszldFP7GQzl75JdorKIVCSiBmEdSn7u+H+Q+bKW0DRt7x S+ow== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of srivatsa@csail.mit.edu designates 128.30.2.210 as permitted sender) smtp.mailfrom=srivatsa@csail.mit.edu; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=csail.mit.edu X-Gm-Message-State: AOUpUlGssWHIOdlV8Wop6xqXAp0ozRexjU9hMqkbrhuhS2wA6QtFBInI 8i8ZdMj6LAzVJPSjN6EHSqPHs1+KKqfQq6ICYHpevSVBB4UcZQQHvBwAU71hfEdiiuSyzJBLs8K FyZx2SUDzY5Y7O4BSGnEr6zY4Ha6eLT0aJQ/Fo1Ft+jlKMQGPmOcOnm7JWF5O5ChNQw== X-Received: by 2002:a0c:f643:: with SMTP id s3-v6mr10415294qvm.131.1531560734467; Sat, 14 Jul 2018 02:32:14 -0700 (PDT) X-Google-Smtp-Source: AAOMgpc9MPSZm5W3jKbMhi8eNvSAXv9T8GxtjQOpaDBn57FCQskalvjmiTvEXkbvgYxUULry4s6B X-Received: by 2002:a0c:f643:: with SMTP id s3-v6mr10415270qvm.131.1531560733803; Sat, 14 Jul 2018 02:32:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1531560733; cv=none; d=google.com; s=arc-20160816; b=UX4PCYZmPpdTx1CjgTgjUjgWHatrqKp05G4Lau9RNYDVR4imFEenmNL3ajheDptIFY 18bqDIdwqmGNxYBHBacZbSYs0aXjoPcabW4oYiNe/f0OUo5GFvN5E0TbYQ+w24PRya4O ki80EA+zGmEF+DUSpN9UI5lnHMRm630PsXcK3YwhLakWKaBlmXOsatZzJYZ3r7XZbmL2 Day3MQxz7vrD/aM18GivkilzZnTfhHTXBZ1gC5vntnpvzlb4xH+x9DffgTBluGBlY1b+ 5QUfircLh/jOAKdDQmqU8qYvsftv1jNOKX52QjyDk/HVmaoQJA+S0qCjofglzgXfP49J FdTA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:user-agent:references :in-reply-to:message-id:date:cc:to:from:subject :arc-authentication-results; bh=2c+V7XqZne2T1CJZKoomzUzu/8A2csiffY4Rc67oym4=; b=ICFOnvfvvPQc0lQEgPFaUpj04xW9mj3Fbkvk/8GlJ+SyoQfvzH5TB7IN4+bXA2LNvd +j0RSf01SnCx4zHqv9fa3QEkXAkDGfttUAuI5kz2la7BkPfszZBn9KrJ+DM5JZFN7e/r xp4p7LbPdRHfVhTMdIRfsXVjStqZL6rRqnLiz/Yypp3FMvLv43ljnNHeB3MvoZ3GHiSM HjnUvGd45Yz1z7w0XI/VH48wzFHhPknE7u9mRzQzifXfviLV2qYjCtoTpaYjMVGBmgNP QUX+90LgQBj9DlnW7BhWcuHjbMMCtgdjxiEFpT3Tn33aCFSVxumxdh5N9yZGnJXv9pwA PvIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of srivatsa@csail.mit.edu designates 128.30.2.210 as permitted sender) smtp.mailfrom=srivatsa@csail.mit.edu; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=csail.mit.edu Received: from outgoing-stata.csail.mit.edu (outgoing-stata.csail.mit.edu. [128.30.2.210]) by mx.google.com with ESMTP id k16-v6si4616qkh.243.2018.07.14.02.32.13 for ; Sat, 14 Jul 2018 02:32:13 -0700 (PDT) Received-SPF: pass (google.com: domain of srivatsa@csail.mit.edu designates 128.30.2.210 as permitted sender) client-ip=128.30.2.210; Authentication-Results: mx.google.com; spf=pass (google.com: domain of srivatsa@csail.mit.edu designates 128.30.2.210 as permitted sender) smtp.mailfrom=srivatsa@csail.mit.edu; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=csail.mit.edu Received: from c-71-231-194-96.hsd1.wa.comcast.net ([71.231.194.96] helo=[127.0.1.1]) by outgoing-stata.csail.mit.edu with esmtpsa (TLS1.2:DHE_RSA_AES_256_CBC_SHA256:256) (Exim 4.82) (envelope-from ) id 1feGuQ-000ZZ4-Lh; Sat, 14 Jul 2018 05:32:10 -0400 Subject: [PATCH 4.4.y 042/101] x86/mm: Give each mm TLB flush generation a unique ID From: "Srivatsa S. Bhat" To: gregkh@linuxfoundation.org, stable@vger.kernel.org Cc: Andy Lutomirski , Nadav Amit , Thomas Gleixner , Andrew Morton , Arjan van de Ven , Borislav Petkov , Dave Hansen , Linus Torvalds , Mel Gorman , Peter Zijlstra , Rik van Riel , linux-mm@kvack.org, Ingo Molnar , Tim Chen , "Matt Helsley \(VMware\)" , Alexey Makhalov , Bo Gan , matt.helsley@gmail.com, rostedt@goodmis.org, amakhalov@vmware.com, ganb@vmware.com, srivatsa@csail.mit.edu, srivatsab@vmware.com Date: Sat, 14 Jul 2018 02:32:07 -0700 Message-ID: <153156072694.10043.1719994417190491710.stgit@srivatsa-ubuntu> In-Reply-To: <153156030832.10043.13438231886571087086.stgit@srivatsa-ubuntu> References: <153156030832.10043.13438231886571087086.stgit@srivatsa-ubuntu> User-Agent: StGit/0.18 MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Andy Lutomirski commit f39681ed0f48498b80455095376f11535feea332 upstream. This adds two new variables to mmu_context_t: ctx_id and tlb_gen. ctx_id uniquely identifies the mm_struct and will never be reused. For a given mm_struct (and hence ctx_id), tlb_gen is a monotonic count of the number of times that a TLB flush has been requested. The pair (ctx_id, tlb_gen) can be used as an identifier for TLB flush actions and will be used in subsequent patches to reliably determine whether all needed TLB flushes have occurred on a given CPU. This patch is split out for ease of review. By itself, it has no real effect other than creating and updating the new variables. Signed-off-by: Andy Lutomirski Reviewed-by: Nadav Amit Reviewed-by: Thomas Gleixner Cc: Andrew Morton Cc: Arjan van de Ven Cc: Borislav Petkov Cc: Dave Hansen Cc: Linus Torvalds Cc: Mel Gorman Cc: Peter Zijlstra Cc: Rik van Riel Cc: linux-mm@kvack.org Link: http://lkml.kernel.org/r/413a91c24dab3ed0caa5f4e4d017d87b0857f920.1498751203.git.luto@kernel.org Signed-off-by: Ingo Molnar Signed-off-by: Tim Chen Signed-off-by: Greg Kroah-Hartman Signed-off-by: Srivatsa S. Bhat Reviewed-by: Matt Helsley (VMware) Reviewed-by: Alexey Makhalov Reviewed-by: Bo Gan --- arch/x86/include/asm/mmu.h | 15 +++++++++++++-- arch/x86/include/asm/mmu_context.h | 4 ++++ arch/x86/mm/tlb.c | 2 ++ 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/mmu.h b/arch/x86/include/asm/mmu.h index 7680b76..3359dfe 100644 --- a/arch/x86/include/asm/mmu.h +++ b/arch/x86/include/asm/mmu.h @@ -3,12 +3,18 @@ #include #include +#include /* - * The x86 doesn't have a mmu context, but - * we put the segment information here. + * x86 has arch-specific MMU state beyond what lives in mm_struct. */ typedef struct { + /* + * ctx_id uniquely identifies this mm_struct. A ctx_id will never + * be reused, and zero is not a valid ctx_id. + */ + u64 ctx_id; + #ifdef CONFIG_MODIFY_LDT_SYSCALL struct ldt_struct *ldt; #endif @@ -24,6 +30,11 @@ typedef struct { atomic_t perf_rdpmc_allowed; /* nonzero if rdpmc is allowed */ } mm_context_t; +#define INIT_MM_CONTEXT(mm) \ + .context = { \ + .ctx_id = 1, \ + } + void leave_mm(int cpu); #endif /* _ASM_X86_MMU_H */ diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_context.h index 1c4794f..effc127 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -11,6 +11,9 @@ #include #include #include + +extern atomic64_t last_mm_ctx_id; + #ifndef CONFIG_PARAVIRT static inline void paravirt_activate_mm(struct mm_struct *prev, struct mm_struct *next) @@ -105,6 +108,7 @@ static inline void enter_lazy_tlb(struct mm_struct *mm, struct task_struct *tsk) static inline int init_new_context(struct task_struct *tsk, struct mm_struct *mm) { + mm->context.ctx_id = atomic64_inc_return(&last_mm_ctx_id); init_new_context_ldt(tsk, mm); return 0; } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 7cad01af..efec198 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -29,6 +29,8 @@ * Implement flush IPI by CALL_FUNCTION_VECTOR, Alex Shi */ +atomic64_t last_mm_ctx_id = ATOMIC64_INIT(1); + struct flush_tlb_info { struct mm_struct *flush_mm; unsigned long flush_start;