From patchwork Mon Jun 12 21:03:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Vishal Moola (Oracle)" X-Patchwork-Id: 13277347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63E34C7EE43 for ; Mon, 12 Jun 2023 21:04:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E7A198E000A; Mon, 12 Jun 2023 17:04:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E03428E0005; Mon, 12 Jun 2023 17:04:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C55D98E000A; Mon, 12 Jun 2023 17:04:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B7D9A8E0005 for ; Mon, 12 Jun 2023 17:04:55 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8D090AF6F6 for ; Mon, 12 Jun 2023 21:04:55 +0000 (UTC) X-FDA: 80895325350.26.5A7A957 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) by imf25.hostedemail.com (Postfix) with ESMTP id A45B5A000C for ; Mon, 12 Jun 2023 21:04:53 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=NyvAW5xc; spf=pass (imf25.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686603893; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=V5BTNYEJrMvEdgPHE9qghL8qlsXVOst+iBLMKLVFrSI=; b=tnU9lH37J172IMhj2hh7snHfRkFLbbXiC5dKCsSLCW/doH2+krj6UrjEloIuzggejiz6q9 x8tlrTLaZwLZdOO/VWScjUp4qJUhDVtIe0PG+6a5yAw/cf1HBd0xPbP7by5ENTRtzinKnt e3Yti4fnCB7X79swa/Wj9IE33SObivE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686603893; a=rsa-sha256; cv=none; b=2Ap+ExEJSJI3eSBWDw2CL/S/JxXI9fbWxsM1JkIk7PziBLHbGX+UE22WgP3mlVzHN+aooT A5WwNoJ9Ef3tnstv3+97l50B7VAC+6THVq6v6kqTTwhwHSJl0rpphzLi/SmUJuw+NmHib8 LnaDWEHjkDhjH+cpN81Y4a9iD9/8vdw= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20221208 header.b=NyvAW5xc; spf=pass (imf25.hostedemail.com: domain of vishal.moola@gmail.com designates 209.85.210.177 as permitted sender) smtp.mailfrom=vishal.moola@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-6549df4321aso4980251b3a.2 for ; Mon, 12 Jun 2023 14:04:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686603892; x=1689195892; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V5BTNYEJrMvEdgPHE9qghL8qlsXVOst+iBLMKLVFrSI=; b=NyvAW5xcJ/1RwE9QmSiFJ+QoLspe4N8bc4rT8FqwnwCneI2MtPt+zR2zPpLqApF4NL U1okS2jcBv5boWFVjS3ZWFOO57AXCrm8924Yzy4TWhUVvSSQaBnBXCQN1FwgNux+a5wh DsFZHudMr3gw7SxF9mDXLpRFyJexdPdYzbIm/1S5JJIGYV9GOI+NthRwDASTis3Fze4p V6ixmPm9cBdQ1cAVmEM6h/YPAwNhp4idZwgbxQNshhMy1ADo1GloFQRBJ0Q5sP9FFktv iWvvuohJvJZTnnhPK8vd97XGBVPC/qP/sL8+3CeVXYInAaBFtRMw40MKTPvSoWZiFPoO gspg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686603892; x=1689195892; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V5BTNYEJrMvEdgPHE9qghL8qlsXVOst+iBLMKLVFrSI=; b=gIZcuBlZKjMrJCw5Ikv2U8II38/bNdTXmhCNEaoy5iR13YW+kBMrRuaG/fsMU3bGbu B8rhGSLiyFriWwW+BU4Kv2jFme4tJgQSGmCd09IGfkvDRcaDvK9EEB+3n+98ZBhZ+rv/ noONdgr+vw6WUIlnEQ4WlK1l/PAonts9ee9K93IpY+DrIuV+9hmbsF7EGcin0W5RHGbC ePg49k+hKuDXan8Cji6OU0aojqCyheXCUQLdNbuY+gE0u10k9AYGFWULlfSXZM4alFzr k+45DzMVs7CZBtZdjLXuiCc+TzV+hqaj3LB2vmuCQ13VGnua7hR6bRqZK2XAHIgirjT7 5krg== X-Gm-Message-State: AC+VfDzpxwh17FeWcZk+IUis2dU6jEuRjmebJI82xS91CZJCWZevFvx9 YGX2XrTceeok+sPb5V8jOz8= X-Google-Smtp-Source: ACHHUZ7txlwQKhNIkJlSWkpy4O9/GHkXq934dfDZR66J7e8THGAZbbGg45zrTEyJTMim3NRAIaYTPA== X-Received: by 2002:a05:6a20:e617:b0:10c:2349:459e with SMTP id my23-20020a056a20e61700b0010c2349459emr10483492pzb.10.1686603892270; Mon, 12 Jun 2023 14:04:52 -0700 (PDT) Received: from unknowna0e70b2ca394.attlocal.net ([2600:1700:2f7d:1800::16]) by smtp.googlemail.com with ESMTPSA id s125-20020a817783000000b00569eb609458sm2757115ywc.81.2023.06.12.14.04.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 12 Jun 2023 14:04:51 -0700 (PDT) From: "Vishal Moola (Oracle)" To: Andrew Morton , Matthew Wilcox Cc: linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, loongarch@lists.linux.dev, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-openrisc@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-sh@vger.kernel.org, sparclinux@vger.kernel.org, linux-um@lists.infradead.org, xen-devel@lists.xenproject.org, kvm@vger.kernel.org, Hugh Dickins , "Vishal Moola (Oracle)" , David Hildenbrand , Claudio Imbrenda Subject: [PATCH v4 03/34] s390: Use pt_frag_refcount for pagetables Date: Mon, 12 Jun 2023 14:03:52 -0700 Message-Id: <20230612210423.18611-4-vishal.moola@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20230612210423.18611-1-vishal.moola@gmail.com> References: <20230612210423.18611-1-vishal.moola@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: A45B5A000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: tod7kn381ubfthb14qu7e3gyjkt7dg87 X-HE-Tag: 1686603893-81164 X-HE-Meta: U2FsdGVkX19je0avX3R6LI4dfiR9slnPiYEcvrcVRMhCH9rIndvihLig7KVjV/HKUcXF/qI3e+AIbbupE9IuUVT7Dr12ylS4qyWTxOZOHHjIDsgcyKgaXLN1Or8w0tCE7mIZJFeQT6w+b/KmyfnO5wdfakqBGkHQuxpbLCNT1E1Cj1FH1WqsvPTQFTOc9KIxpWuP8wneAMPy68QWgnQDZRDcZX/gs/vF2LFQ0kLxI5cut7kIAGg67cEkm1DPK4oRiOIqqTBFrfVZWk8QuOPyzyYDXmjdx8SDiTToADnSPvz0sjLpAKlFNWVanDVmm+7p3xPONfb3uaiQcbL1I35Q573YhKCNXRDMgIlu7/BnkQ+C9/aRQmnTJkFy3KVIbUcW0X+ISWJz2gb3jpLaw8RZtKgBg08ZJHANAsopSpFlfhYc4hWPLjW98NstrAgzt1av4h9nfqveHLfFFl3588aLbX1ix+rPhfu5atwWLXpY8VZwvuDmg8duatPwKUlWZjznEEyXIqYqzIg5G0TJ9XvQREmPMWBm0q5asRiOAvV4zf40G7abgPGp4XWTiUipFZ4Avaz341erqRemUdcUtu5RWRie5hgVjevZYE3HQZQGueeF7mX9pVz3/NuNHfHTSZu2PcGLX82nHmaiQdFxHfCpOpJNvU0t3QTUuYnB2q3aqeZkno/0bCGzabPoOByD+N/lYnvHKf5YpKWiFwCEHpeUJ8qNtj0GmRff36+VkWKaVQPmI+FmDy54Xtr4o4I5t0bx/PdrxJCYUX3wsvg2tn0vm2h/F0pqMuxpNKF2p9utGUfuWP6gBZnuxsEo7wFKmJ3zakvBauN7CyYL3WLeGFohlSNincCC1pt57/TRSNfafmyF5+1XFsOrwy8GppX7cmTmfYFXoFLMu+WenGbGL6hvkXmzVV0VdeZfuK5dKDsG5ZB+gnH902i/uWTC31GQDvq/rvWMofkR5uMRyocOVUe YLnDImtu lM8llSiGqjWNXu1pWJnJ/+WzfRffV8tCkRIJZbCg1TsHDO7q7wgyvUEb3zFvk6wc3Jpm6ejxWgs/O+Z6gb9MiL7f5qTSaQkXgFHKQ6d7lPNAOEa+pokzSi5UAGsQfTOejIlfpi7NNsEeTLH2alEidThaLKEuoqq0yiDpPdCR6nZdWK1aP1fIXnlqK/wrM66pc8/ZK3z8czb7tksJZfb/Ax29VzAabjg6XPgIkOUw6uxU2ozUH0Ve8pnPoZDLAPln/xLz6hHSjrjf36Pg74IRYRsi57wEv3rk/N0Fc3HklyX24erzG4QsoP/hyHfJilQI6R2U35QFg1Db7+Ux7kU9rC6uqF4m3I0z9eEfZ46BWlAiFah2BfSyjuLGpCeXQBCHw0MkOkS43VxAqLLwnhE9ICYCvZJllDi6EUfHfEIAhF1MqzsBAvVnpnAPD4I5r/09ppNqduno+kDeORyuKpxAtzhdJkjNwJLMSkTUZX1IZftDFvsueg42kQEFH6ndmLrGnUCLsfGj7YxKq+LTeJHAl4BrdPcpnIz5g541u6Tyybibuy1ww9xamnDapU48JToCZfuKrl0n5oRjymGA= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: s390 currently uses _refcount to identify fragmented page tables. The page table struct already has a member pt_frag_refcount used by powerpc, so have s390 use that instead of the _refcount field as well. This improves the safety for _refcount and the page table tracking. This also allows us to simplify the tracking since we can once again use the lower byte of pt_frag_refcount instead of the upper byte of _refcount. Signed-off-by: Vishal Moola (Oracle) Acked-by: Mike Rapoport (IBM) --- arch/s390/mm/pgalloc.c | 38 +++++++++++++++----------------------- 1 file changed, 15 insertions(+), 23 deletions(-) diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c index 66ab68db9842..6b99932abc66 100644 --- a/arch/s390/mm/pgalloc.c +++ b/arch/s390/mm/pgalloc.c @@ -182,20 +182,17 @@ void page_table_free_pgste(struct page *page) * As follows from the above, no unallocated or fully allocated parent * pages are contained in mm_context_t::pgtable_list. * - * The upper byte (bits 24-31) of the parent page _refcount is used + * The lower byte (bits 0-7) of the parent page pt_frag_refcount is used * for tracking contained 2KB-pgtables and has the following format: * * PP AA - * 01234567 upper byte (bits 24-31) of struct page::_refcount + * 01234567 upper byte (bits 0-7) of struct page::pt_frag_refcount * || || * || |+--- upper 2KB-pgtable is allocated * || +---- lower 2KB-pgtable is allocated * |+------- upper 2KB-pgtable is pending for removal * +-------- lower 2KB-pgtable is pending for removal * - * (See commit 620b4e903179 ("s390: use _refcount for pgtables") on why - * using _refcount is possible). - * * When 2KB-pgtable is allocated the corresponding AA bit is set to 1. * The parent page is either: * - added to mm_context_t::pgtable_list in case the second half of the @@ -243,11 +240,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm) if (!list_empty(&mm->context.pgtable_list)) { page = list_first_entry(&mm->context.pgtable_list, struct page, lru); - mask = atomic_read(&page->_refcount) >> 24; + mask = atomic_read(&page->pt_frag_refcount); /* * The pending removal bits must also be checked. * Failure to do so might lead to an impossible - * value of (i.e 0x13 or 0x23) written to _refcount. + * value of (i.e 0x13 or 0x23) written to + * pt_frag_refcount. * Such values violate the assumption that pending and * allocation bits are mutually exclusive, and the rest * of the code unrails as result. That could lead to @@ -259,8 +257,8 @@ unsigned long *page_table_alloc(struct mm_struct *mm) bit = mask & 1; /* =1 -> second 2K */ if (bit) table += PTRS_PER_PTE; - atomic_xor_bits(&page->_refcount, - 0x01U << (bit + 24)); + atomic_xor_bits(&page->pt_frag_refcount, + 0x01U << bit); list_del(&page->lru); } } @@ -281,12 +279,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm) table = (unsigned long *) page_to_virt(page); if (mm_alloc_pgste(mm)) { /* Return 4K page table with PGSTEs */ - atomic_xor_bits(&page->_refcount, 0x03U << 24); + atomic_xor_bits(&page->pt_frag_refcount, 0x03U); memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE); memset64((u64 *)table + PTRS_PER_PTE, 0, PTRS_PER_PTE); } else { /* Return the first 2K fragment of the page */ - atomic_xor_bits(&page->_refcount, 0x01U << 24); + atomic_xor_bits(&page->pt_frag_refcount, 0x01U); memset64((u64 *)table, _PAGE_INVALID, 2 * PTRS_PER_PTE); spin_lock_bh(&mm->context.lock); list_add(&page->lru, &mm->context.pgtable_list); @@ -323,22 +321,19 @@ void page_table_free(struct mm_struct *mm, unsigned long *table) * will happen outside of the critical section from this * function or from __tlb_remove_table() */ - mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit); if (mask & 0x03U) list_add(&page->lru, &mm->context.pgtable_list); else list_del(&page->lru); spin_unlock_bh(&mm->context.lock); - mask = atomic_xor_bits(&page->_refcount, 0x10U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x10U << bit); if (mask != 0x00U) return; half = 0x01U << bit; } else { half = 0x03U; - mask = atomic_xor_bits(&page->_refcount, 0x03U << 24); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U); } page_table_release_check(page, table, half, mask); @@ -368,8 +363,7 @@ void page_table_free_rcu(struct mmu_gather *tlb, unsigned long *table, * outside of the critical section from __tlb_remove_table() or from * page_table_free() */ - mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x11U << bit); if (mask & 0x03U) list_add_tail(&page->lru, &mm->context.pgtable_list); else @@ -391,14 +385,12 @@ void __tlb_remove_table(void *_table) return; case 0x01U: /* lower 2K of a 4K page table */ case 0x02U: /* higher 2K of a 4K page table */ - mask = atomic_xor_bits(&page->_refcount, mask << (4 + 24)); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, mask << 4); if (mask != 0x00U) return; break; case 0x03U: /* 4K page table with pgstes */ - mask = atomic_xor_bits(&page->_refcount, 0x03U << 24); - mask >>= 24; + mask = atomic_xor_bits(&page->pt_frag_refcount, 0x03U); break; }