From patchwork Tue Aug 14 17:17:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Greg Kroah-Hartman X-Patchwork-Id: 10565927 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 421401390 for ; Tue, 14 Aug 2018 17:46:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34450289EE for ; Tue, 14 Aug 2018 17:46:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 24B0A2A46F; Tue, 14 Aug 2018 17:46:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.3 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE,URIBL_SBL autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6EF2D289EE for ; Tue, 14 Aug 2018 17:46:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0C5626B0003; Tue, 14 Aug 2018 13:46:45 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 0764D6B0005; Tue, 14 Aug 2018 13:46:45 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ECE026B0006; Tue, 14 Aug 2018 13:46:44 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pf1-f198.google.com (mail-pf1-f198.google.com [209.85.210.198]) by kanga.kvack.org (Postfix) with ESMTP id ACF366B0003 for ; Tue, 14 Aug 2018 13:46:44 -0400 (EDT) Received: by mail-pf1-f198.google.com with SMTP id l15-v6so11735244pff.1 for ; Tue, 14 Aug 2018 10:46:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:user-agent :mime-version; bh=ySNr4rQnYgpG6Zas2Fju3ubHVDY0L++tI7E7/Amu8uE=; b=nqJMbMxMlypb7/UqLOhspW3OEnJNG3y9iOLyWHyvDwqqwVlvD3QvPGzFSfOj5zc7Jc BeIwmUfpTMzj7hcpeQFBfiiW2M7F2UMQo3RICOzW9Lwl8yLYI3MnH07cTMWSYmL/v4lF LJs54IwkaRGb97fM2Z4RhIesMRzaTkm+M83WVx8fvBRQf0rttQDPA5mGoopsMZpxlyx2 /X67p1w5ae0ARUN7ZbPfc9c5gVUkw3hRe+Mq336KLZZ32pFfvn8MPJtfHFtlCKJZYpEx 3fL67FfdGADoFqo2Pm05yLEebahvnkRWDWfbQT4X19tsaf9ULRN7N/Bwp3MwmfzwKrbq 9PeA== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org X-Gm-Message-State: AOUpUlHOfJq4VHnNz4c/sDbqfDnyvDDvbjFRZP01R5o3JLh8seKYKed1 ACzSc8xi/VPLQNp/2dEHO8JjQqXdop6kQdiOjWASZwt4iKwaUUSUn+j9wy6H6BE1JtvdY+w3LuF BiqG8tRLlYSQOs69oIAZNUI9jinOQWGSZVIZmz2FZHKLZsHg5FiG6swDqgFOq1g4XFQ== X-Received: by 2002:a63:f713:: with SMTP id x19-v6mr21946288pgh.233.1534268804366; Tue, 14 Aug 2018 10:46:44 -0700 (PDT) X-Google-Smtp-Source: AA+uWPx1q8UIFRLMucDyXguB2p+eMRIdG8erUqoz0QbW/umeBU6SNwjR5Hv8yPlX41Ln58AVPMC1 X-Received: by 2002:a63:f713:: with SMTP id x19-v6mr21946252pgh.233.1534268803442; Tue, 14 Aug 2018 10:46:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1534268803; cv=none; d=google.com; s=arc-20160816; b=A6BIzuEj3Ne5rjcR6YRyHJlQW9XHon9JfU8U6xG+kvxBe4gYfEMa0D0lmKlLMnwFCY DKA21bEysFnOZq7VjuQrLCp/Z2/Qq11Lp/eVaSz9Q4SZxIbE91rUJszYZ+qYRbuxM14e /BK/DLPqPdotqIIu+0YreJbvQhgJ8duCSJtfK4dxuUAtOYo+AJCnwlmm/orVLYkB+YAA tvX7FDvWVa+2WNuJXwmgI3IAURnYqH/XBrGtlhqjVZj1U3QHYnRXl0sgZb6CCpogdTcL HHFgbRRdfUjG5rmxBZWkKTMj4+1SRt2C5B7R6/iZEbvIpYIvIggQ/wYed54aZu9uW+rz HBiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=ySNr4rQnYgpG6Zas2Fju3ubHVDY0L++tI7E7/Amu8uE=; b=FJX64jiEIJOKAX+M/q/SnRnYS7949p0DGlW8anZXSyW9tGJG5GSGxR5KiuRZXEaI2/ kQsHL97ttQvsSIzX6uUNx+Hta2eQ8Vyll5+priGOsWyiBE7envQSvv9+VXzR02zgHQPv dB8rGgmfxs6rtxpSiRZwqq6qQNfra6OwNRQ4Kyl0Hzu0AKBOLKxYt3cHJRjDpj8vshHq nicLO+y3n7rFl1P0WIoS34Ql4r5vRlSSX6ZvcX+M/oyyX94wPzTWSblJJXlacD0x6LpM URi4h5JpmYgs1U9YYyWNShWTnSBAjazQtZ1vZI1bMxCvpq4JeVLdvOvgXYhXzaU50r1L 99kQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from mail.linuxfoundation.org (mail.linuxfoundation.org. [140.211.169.12]) by mx.google.com with ESMTPS id d7-v6si16711506pll.162.2018.08.14.10.46.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Aug 2018 10:46:43 -0700 (PDT) Received-SPF: pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) client-ip=140.211.169.12; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gregkh@linuxfoundation.org designates 140.211.169.12 as permitted sender) smtp.mailfrom=gregkh@linuxfoundation.org Received: from localhost (unknown [194.244.16.108]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 2675EAF3; Tue, 14 Aug 2018 17:46:42 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Dave Hansen , Andrew Morton , Andy Lutomirski , Borislav Petkov , Brian Gerst , Dave Hansen , Denys Vlasenko , "H. Peter Anvin" , Josh Poimboeuf , Linus Torvalds , "Luis R. Rodriguez" , Peter Zijlstra , Thomas Gleixner , Toshi Kani , dave.hansen@intel.com, linux-mm@kvack.org, mhocko@suse.com, Ingo Molnar , Guenter Roeck Subject: [PATCH 4.4 22/43] x86/mm: Move swap offset/type up in PTE to work around erratum Date: Tue, 14 Aug 2018 19:17:58 +0200 Message-Id: <20180814171518.589533556@linuxfoundation.org> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20180814171517.014285600@linuxfoundation.org> References: <20180814171517.014285600@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP 4.4-stable review patch. If anyone has any objections, please let me know. ------------------ From: Dave Hansen commit 00839ee3b299303c6a5e26a0a2485427a3afcbbf upstream This erratum can result in Accessed/Dirty getting set by the hardware when we do not expect them to be (on !Present PTEs). Instead of trying to fix them up after this happens, we just allow the bits to get set and try to ignore them. We do this by shifting the layout of the bits we use for swap offset/type in our 64-bit PTEs. It looks like this: bitnrs: | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| names: | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| before: | OFFSET (9-63) |0|X|X| TYPE(1-5) |0| after: | OFFSET (14-63) | TYPE (9-13) |0|X|X|X| X| X|X|X|0| Note that D was already a don't care (X) even before. We just move TYPE up and turn its old spot (which could be hit by the A bit) into all don't cares. We take 5 bits away from the offset, but that still leaves us with 50 bits which lets us index into a 62-bit swapfile (4 EiB). I think that's probably fine for the moment. We could theoretically reclaim 5 of the bits (1, 2, 3, 4, 7) but it doesn't gain us anything. Signed-off-by: Dave Hansen Cc: Andrew Morton Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Dave Hansen Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Josh Poimboeuf Cc: Linus Torvalds Cc: Luis R. Rodriguez Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Toshi Kani Cc: dave.hansen@intel.com Cc: linux-mm@kvack.org Cc: mhocko@suse.com Link: http://lkml.kernel.org/r/20160708001911.9A3FD2B6@viggo.jf.intel.com Signed-off-by: Ingo Molnar Signed-off-by: Guenter Roeck Signed-off-by: Greg Kroah-Hartman --- arch/x86/include/asm/pgtable_64.h | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-) --- a/arch/x86/include/asm/pgtable_64.h +++ b/arch/x86/include/asm/pgtable_64.h @@ -163,18 +163,32 @@ static inline int pgd_large(pgd_t pgd) { #define pte_offset_map(dir, address) pte_offset_kernel((dir), (address)) #define pte_unmap(pte) ((void)(pte))/* NOP */ -/* Encode and de-code a swap entry */ +/* + * Encode and de-code a swap entry + * + * | ... | 11| 10| 9|8|7|6|5| 4| 3|2|1|0| <- bit number + * | ... |SW3|SW2|SW1|G|L|D|A|CD|WT|U|W|P| <- bit names + * | OFFSET (14->63) | TYPE (10-13) |0|X|X|X| X| X|X|X|0| <- swp entry + * + * G (8) is aliased and used as a PROT_NONE indicator for + * !present ptes. We need to start storing swap entries above + * there. We also need to avoid using A and D because of an + * erratum where they can be incorrectly set by hardware on + * non-present PTEs. + */ +#define SWP_TYPE_FIRST_BIT (_PAGE_BIT_PROTNONE + 1) #define SWP_TYPE_BITS 5 -#define SWP_OFFSET_SHIFT (_PAGE_BIT_PROTNONE + 1) +/* Place the offset above the type: */ +#define SWP_OFFSET_FIRST_BIT (SWP_TYPE_FIRST_BIT + SWP_TYPE_BITS + 1) #define MAX_SWAPFILES_CHECK() BUILD_BUG_ON(MAX_SWAPFILES_SHIFT > SWP_TYPE_BITS) -#define __swp_type(x) (((x).val >> (_PAGE_BIT_PRESENT + 1)) \ +#define __swp_type(x) (((x).val >> (SWP_TYPE_FIRST_BIT)) \ & ((1U << SWP_TYPE_BITS) - 1)) -#define __swp_offset(x) ((x).val >> SWP_OFFSET_SHIFT) +#define __swp_offset(x) ((x).val >> SWP_OFFSET_FIRST_BIT) #define __swp_entry(type, offset) ((swp_entry_t) { \ - ((type) << (_PAGE_BIT_PRESENT + 1)) \ - | ((offset) << SWP_OFFSET_SHIFT) }) + ((type) << (SWP_TYPE_FIRST_BIT)) \ + | ((offset) << SWP_OFFSET_FIRST_BIT) }) #define __pte_to_swp_entry(pte) ((swp_entry_t) { pte_val((pte)) }) #define __swp_entry_to_pte(x) ((pte_t) { .pte = (x).val })