From patchwork Mon Jul 17 16:03:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F589C04A94 for ; Mon, 17 Jul 2023 16:03:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564729.882393 (Exim 4.92) (envelope-from ) id 1qLQh4-0002co-Qo; Mon, 17 Jul 2023 16:03:26 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564729.882393; Mon, 17 Jul 2023 16:03:26 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh4-0002ca-Nd; Mon, 17 Jul 2023 16:03:26 +0000 Received: by outflank-mailman (input) for mailman id 564729; Mon, 17 Jul 2023 16:03:25 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh3-00027B-41 for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:25 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 74ad8102-24bb-11ee-b23a-6b7b168915f2; Mon, 17 Jul 2023 18:03:23 +0200 (CEST) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-52176fdad9dso3906486a12.0 for ; Mon, 17 Jul 2023 09:03:23 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:22 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 74ad8102-24bb-11ee-b23a-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609802; x=1692201802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BW+fp2cFFWVe/TGuiikouZekFdJrXU/1zeYhc/QEe/k=; b=bXNCVYC4zG2I77SeoN40sQkO3gZHN++CNGlR10/+7GAJV/JWL4E/DBSiQwn2xLXqz2 S+0zh8ADYioSbPkvErW8rtDhLY3i3iotarf0Lb+ivTtBmpBFZUZXA6RAS2KCl+DF7+vE 7o39ish/VPGMWkTfCEOEqi2/0gm/g7jtpzBi4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609802; x=1692201802; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BW+fp2cFFWVe/TGuiikouZekFdJrXU/1zeYhc/QEe/k=; b=girtGy7x32kAS9menGf39NpLZqJT7CWGDq2ap10ngR57xKuq7E50UP9pTqT3SsC/cj xKCEoxhM6OwEw4/4r34QKd/24FO8Wq9Vqj5EtJQCvfpAmy7otVMEyo8xHeB5UALasN2c +3PcN6iwr3F8BdLw2ju0P6N8fMEJoB330U0CYrxRJbpZG3fDgYAyQD5rvvEZ0DJCzhwh FPSmRWdVKAZjbpZM+umm0JX4vdOHjwGy0K9Skou4qeb+8fWRrT1u8edQrw7bHiwg7ZyN s7KCvcq1LYmfl5bu+oQEsHeD9UxIZMqu97iC4vB5EZyk/cz7TxdgPFWyi9UB4+3Lep7j v8Ew== X-Gm-Message-State: ABy/qLY6YoEeTtTrlq3NUSIRRFNB3gmDHtgQsKa7+/UCDLiGoIvoO51S g501gvG0xf+AI8bDD1R+mL8pUj2lTcWKCk0YO38= X-Google-Smtp-Source: APBJJlEx2LPDE6+tT77cmeCuRYNMOmeajBP0fDfUkbbY3LwD4Pi0hrl0Jv0rdvL7OZnWkG8c3fqWPA== X-Received: by 2002:aa7:d991:0:b0:51e:253e:1e0c with SMTP id u17-20020aa7d991000000b0051e253e1e0cmr182713eds.25.1689609802321; Mon, 17 Jul 2023 09:03:22 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH 1/8] mm/pdx: Add comments throughout the codebase for pdx Date: Mon, 17 Jul 2023 17:03:11 +0100 Message-Id: <20230717160318.2113-2-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Document the behaviour of the pdx machinery in Xen. Some logic is fairly opaque and hard to follow without it being documented anywhere. This explains the rationale behind compression and its relationship to frametable indexing and directmap management. While modifying the file: * Convert u64 -> uint64_t * Remove extern keyword from function prototypes No functional change. Signed-off-by: Alejandro Vallejo Reviewed-by: Jan Beulich --- This would've been v4 of the previously standalone patch: v4: * Adds 2 "may"s to mm.h as requested --- xen/common/pdx.c | 59 +++++++++++++++++- xen/include/xen/mm.h | 11 ++++ xen/include/xen/pdx.h | 139 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 202 insertions(+), 7 deletions(-) diff --git a/xen/common/pdx.c b/xen/common/pdx.c index c91875fabe..ec64d3d2ef 100644 --- a/xen/common/pdx.c +++ b/xen/common/pdx.c @@ -20,13 +20,56 @@ #include #include -/* Parameters for PFN/MADDR compression. */ +/* + * Diagram to make sense of the following variables. The masks and shifts + * are done on mfn values in order to convert to/from pdx: + * + * pfn_hole_mask + * pfn_pdx_hole_shift (mask bitsize) + * | + * |---------| + * | | + * V V + * -------------------------- + * |HHHHHHH|000000000|LLLLLL| <--- mfn + * -------------------------- + * ^ ^ ^ ^ + * | | |------| + * | | | + * | | pfn_pdx_bottom_mask + * | | + * |-------| + * | + * pfn_top_mask + * + * ma_{top,va_bottom}_mask is simply a shifted pfn_{top,pdx_bottom}_mask, + * where ma_top_mask has zeroes shifted in while ma_va_bottom_mask has + * ones. + */ + +/** Maximum (non-inclusive) usable pdx */ unsigned long __read_mostly max_pdx; + +/** Mask for the lower non-compressible bits of an mfn */ unsigned long __read_mostly pfn_pdx_bottom_mask = ~0UL; + +/** Mask for the lower non-compressible bits of an maddr or vaddr */ unsigned long __read_mostly ma_va_bottom_mask = ~0UL; + +/** Mask for the higher non-compressible bits of an mfn */ unsigned long __read_mostly pfn_top_mask = 0; + +/** Mask for the higher non-compressible bits of an maddr or vaddr */ unsigned long __read_mostly ma_top_mask = 0; + +/** + * Mask for a pdx compression bit slice. + * + * Invariant: valid(mfn) implies (mfn & pfn_hole_mask) == 0 + */ unsigned long __read_mostly pfn_hole_mask = 0; + +/** Number of bits of the "compressible" bit slice of an mfn */ unsigned int __read_mostly pfn_pdx_hole_shift = 0; unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS( @@ -42,7 +85,7 @@ bool __mfn_valid(unsigned long mfn) } /* Sets all bits from the most-significant 1-bit down to the LSB */ -static u64 __init fill_mask(u64 mask) +static uint64_t __init fill_mask(uint64_t mask) { while (mask & (mask + 1)) mask |= mask + 1; @@ -57,8 +100,18 @@ uint64_t __init pdx_init_mask(uint64_t base_addr) (uint64_t)1 << (MAX_ORDER + PAGE_SHIFT)) - 1); } -u64 __init pdx_region_mask(u64 base, u64 len) +uint64_t __init pdx_region_mask(uint64_t base, uint64_t len) { + /* + * We say a bit "moves" in a range if there exist 2 addresses in that + * range that have that bit both set and cleared respectively. We want + * to create a mask of _all_ moving bits in this range. We do this by + * comparing the first and last addresses in the range, discarding the + * bits that remain the same (this is logically an XOR operation). The + * MSB of the resulting expression is the most significant moving bit + * in the range. Then it's a matter of setting every bit in lower + * positions in order to get the mask of moving bits. + */ return fill_mask(base ^ (base + len - 1)); } diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h index b0dc3ba9c9..962ef216fd 100644 --- a/xen/include/xen/mm.h +++ b/xen/include/xen/mm.h @@ -31,6 +31,17 @@ * (i.e. all devices assigned to) a guest share a single DMA address space * and, by default, Xen will ensure dfn == pfn. * + * pdx: Page InDeX + * Indices into the frame table holding the per-page's book-keeping + * metadata. A compression scheme may be used, so there's a possibly non + * identity mapping between valid(mfn) <-> valid(pdx). See the comments + * in pdx.c for an in-depth explanation of that mapping. This may also + * have a knock-on effect on the directmap, as "compressed" pfns may not have + * corresponding mapped frames. + * + * maddr: Machine Address + * The physical address that corresponds to an mfn + * * WARNING: Some of these terms have changed over time while others have been * used inconsistently, meaning that a lot of existing code does not match the * definitions above. New code should use these terms as described here, and diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index 9fcfb0ce52..de5439a5e5 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -1,6 +1,72 @@ #ifndef __XEN_PDX_H__ #define __XEN_PDX_H__ +/* + * PDX (Page inDeX) + * + * This file deals with optimisations pertaining to frame table and + * directmap indexing, A pdx is an index into the frame table, which + * typically also means an index into the directmap[1]. However, having an + * identity relationship between mfn and pdx could waste copious amounts of + * memory in empty frame table entries and page tables. There are some + * techniques to bring memory wastage down. + * + * [1] Some ports apply further modifications to a pdx before indexing the + * directmap. This doesn't change the fact that the same compression + * present in the frame table is also present in the directmap + * whenever said map is present. + * + * ## PDX grouping + * + * The frame table may have some sparsity even on systems where the memory + * banks are tightly packed. This is due to system quirks (like the PCI + * hole) which might introduce several GiB of unused page frame numbers + * that uselessly waste memory in the frame table. PDX grouping addresses + * this by keeping a bitmap of the ranges in the frame table containing + * invalid entries and not allocating backing memory for them. + * + * ## PDX compression + * + * This is a technique to avoid wasting memory on machines known to have + * split their machine address space in several big discontinuous and highly + * disjoint chunks. + * + * In its uncompressed form the frame table must have book-keeping metadata + * structures for every page between [0, max_mfn) (whether they are backed + * by RAM or not), and a similar condition exists for the direct map. We + * know some systems, however, that have some sparsity in their address + * space, leading to a lot of wastage in the form of unused frame table + * entries. + * + * This is where compression becomes useful. The idea is to note that if + * you have several big chunks of memory sufficiently far apart you can + * ignore the middle part of the address because it will always contain + * zeroes. + * + * i.e: + * Consider 2 regions of memory. One starts at 0 while the other starts + * at offset 2^off_h. Furthermore, let's assume both regions are smaller + * than 2^off_l. This means that all addresses between [2^off_l, 2^off_h) + * are invalid and we can assume them to be zero on all valid addresses. + * + * off_h off_l + * | | + * V V + * -------------------------- + * |HHHHHHH|000000000|LLLLLL| <--- mfn + * -------------------------- + * ^ | + * | | (de)compression by adding/removing "useless" zeroes + * | V + * --------------- + * |HHHHHHHLLLLLL| <--- pdx + * --------------- + * + * This scheme also holds for multiple regions, where HHHHHHH acts as + * the region identifier and LLLLLL fully contains the span of every + * region involved. + */ + #ifdef CONFIG_HAS_PDX extern unsigned long max_pdx; @@ -13,22 +79,78 @@ extern unsigned long pfn_top_mask, ma_top_mask; (sizeof(*frame_table) & -sizeof(*frame_table))) extern unsigned long pdx_group_valid[]; -extern uint64_t pdx_init_mask(u64 base_addr); -extern u64 pdx_region_mask(u64 base, u64 len); +/** + * Calculates a mask covering "moving" bits of all addresses of a region + * + * The i-th bit of the mask must be set if there's 2 different addresses + * in the region that have different j-th bits. where j >= i. + * + * e.g: + * base=0x1B00000000 + * len+base=0x1B00042000 + * + * ought to return 0x000007FFFF, which implies that every bit position + * with a zero in the mask remains unchanged in every address of the + * region. + * + * @param base Base address of the region + * @param len Size in octets of the region + * @return Mask of moving bits at the bottom of all the region addresses + */ +uint64_t pdx_region_mask(uint64_t base, uint64_t len); -extern void set_pdx_range(unsigned long smfn, unsigned long emfn); +/** + * Creates the mask to start from when calculating non-compressible bits + * + * This function is intimately related to pdx_region_mask(), and together + * they are meant to calculate the mask of non-compressible bits given the + * current memory map. + * + * @param base_addr Address of the first maddr in the system + * @return An integer of the form 2^n - 1 + */ +uint64_t pdx_init_mask(uint64_t base_addr); + +/** + * Mark [smfn, emfn) as accesible in the frame table + * + * @param smfn Start mfn + * @param emfn End mfn + */ +void set_pdx_range(unsigned long smfn, unsigned long emfn); #define page_to_pdx(pg) ((pg) - frame_table) #define pdx_to_page(pdx) gcc11_wrap(frame_table + (pdx)) +/** + * Invoked to determine if an mfn has an associated valid frame table entry + * + * In order for it to be legal it must pass bounds, grouping and + * compression sanity checks. + * + * @param mfn To-be-checked mfn + * @return True iff all checks pass + */ bool __mfn_valid(unsigned long mfn); +/** + * Map pfn to its corresponding pdx + * + * @param pfn Frame number + * @return Obtained pdx after compressing the pfn + */ static inline unsigned long pfn_to_pdx(unsigned long pfn) { return (pfn & pfn_pdx_bottom_mask) | ((pfn & pfn_top_mask) >> pfn_pdx_hole_shift); } +/** + * Map a pdx to its corresponding pfn + * + * @param pdx Page index + * @return Obtained pfn after decompressing the pdx + */ static inline unsigned long pdx_to_pfn(unsigned long pdx) { return (pdx & pfn_pdx_bottom_mask) | @@ -38,7 +160,16 @@ static inline unsigned long pdx_to_pfn(unsigned long pdx) #define mfn_to_pdx(mfn) pfn_to_pdx(mfn_x(mfn)) #define pdx_to_mfn(pdx) _mfn(pdx_to_pfn(pdx)) -extern void pfn_pdx_hole_setup(unsigned long); +/** + * Initializes global variables with information about the compressible + * range of the current memory regions. + * + * @param mask This mask is the biggest pdx_mask of every region in the + * system ORed with all base addresses of every region in the + * system. This results in a mask where every zero in a bit + * position marks a potentially compressible bit. + */ +void pfn_pdx_hole_setup(unsigned long mask); #endif /* HAS_PDX */ #endif /* __XEN_PDX_H__ */ From patchwork Mon Jul 17 16:03:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 37FDFC0015E for ; Mon, 17 Jul 2023 16:03:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564728.882383 (Exim 4.92) (envelope-from ) id 1qLQh3-0002NQ-Kh; Mon, 17 Jul 2023 16:03:25 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564728.882383; Mon, 17 Jul 2023 16:03:25 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh3-0002NJ-Gk; Mon, 17 Jul 2023 16:03:25 +0000 Received: by outflank-mailman (input) for mailman id 564728; Mon, 17 Jul 2023 16:03:24 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh2-00027B-3i for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:24 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 74e3c3c0-24bb-11ee-b23a-6b7b168915f2; Mon, 17 Jul 2023 18:03:23 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-51e28b299adso6556664a12.2 for ; Mon, 17 Jul 2023 09:03:23 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:22 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 74e3c3c0-24bb-11ee-b23a-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609803; x=1692201803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8akak382Pvkzk8smYZShsg8P//0dNcL1l5JC56gewGo=; b=aBkBGRY5kmrQNcoLDo1E1l5XfJc7Yybzdet93GHz0y0BV+O9N+Y4ug/Xpz3GXRAVVM qql+KrY9+5VhMJKMtzGPgLdCcJqqMzyRZHAMbtiqqxkBe4W7XaSOYuYIYUh0fAlWcRzG qMT5/f0naP6oGbd9ebD9X09zH4hmI+ct9w+4I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609803; x=1692201803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8akak382Pvkzk8smYZShsg8P//0dNcL1l5JC56gewGo=; b=g9afuXjSlhHeEDbo9hG5tdU0pr0uv9WE1fv5gaPRIEf5HVeUEIF7dLgBeCLP/7id2S 8bjatQkkXQRg1l3YeekrTvN0oDQyWzknwEBxvfhnsKH+MXUnrKUrWSJDPJIpfKgu28V+ VysXmrVPdVznn1hIB1Utmz1cRtRdt8JOkaIie8jlXwZGHWVRnTF9X82W4Jw3o+TAe225 m07iWVo/dzTIhT27QJHOsp7R2z8Sn9pI2STC5YgjhMAxchFHw/BR0RCkg1JrchUCJhR8 3up2WbzrJA3m6lTifNdKjcgNdJNU+rSXmmFb2h2aJqZYBmUIGIxd8wZVQFoXkInwAuMt DrOQ== X-Gm-Message-State: ABy/qLa8V1CwbtjYejjzD3hfH3URO+wNdwwk6KB7weH4nRAuaDl9/IZ1 mfCdAfYPNuJUW4tMwRckc/SnRnDMhM/F/Ts8fZ8= X-Google-Smtp-Source: APBJJlFXq+YuZi7U2G9yLDx7wuxCaHoDw+Wc30wPyNw4rp1n8ZvEtJGZgBqr3VCUNh5wuoEqCh5dfA== X-Received: by 2002:aa7:cd16:0:b0:51b:c714:a2a1 with SMTP id b22-20020aa7cd16000000b0051bc714a2a1mr11744396edw.7.1689609802826; Mon, 17 Jul 2023 09:03:22 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk Subject: [PATCH 2/8] arm/mm: Document the differences between arm32 and arm64 directmaps Date: Mon, 17 Jul 2023 17:03:12 +0100 Message-Id: <20230717160318.2113-3-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 arm32 merely covers the XENHEAP, whereas arm64 currently covers anything in the frame table. These comments highlight why arm32 doesn't need to account for PDX compression in its __va() implementation while arm64 does. Signed-off-by: Alejandro Vallejo --- xen/arch/arm/include/asm/mm.h | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 4262165ce2..1a83f41879 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -280,6 +280,19 @@ static inline paddr_t __virt_to_maddr(vaddr_t va) #define virt_to_maddr(va) __virt_to_maddr((vaddr_t)(va)) #ifdef CONFIG_ARM_32 +/** + * Find the virtual address corresponding to a machine address + * + * Only memory backing the XENHEAP has a corresponding virtual address to + * be found. This is so we can save precious virtual space, as it's in + * short supply on arm32. This mapping is not subject to PDX compression + * because XENHEAP is known to be physically contiguous and can't hence + * jump over the PDX hole. This means we can avoid the roundtrips + * converting to/from pdx. + * + * @param ma Machine address + * @return Virtual address mapped to `ma` + */ static inline void *maddr_to_virt(paddr_t ma) { ASSERT(is_xen_heap_mfn(maddr_to_mfn(ma))); @@ -287,6 +300,20 @@ static inline void *maddr_to_virt(paddr_t ma) return (void *)(unsigned long) ma + XENHEAP_VIRT_START; } #else +/** + * Find the virtual address corresponding to a machine address + * + * The directmap covers all conventional memory accesible by the + * hypervisor. This means it's subject to PDX compression. + * + * More specifically to arm64, the directmap mappings start at the first + * GiB boundary containing valid RAM. This means there's an extra offset + * applied (directmap_base_pdx) on top of the regular PDX compression + * logic. + * + * @param ma Machine address + * @return Virtual address mapped to `ma` + */ static inline void *maddr_to_virt(paddr_t ma) { ASSERT((mfn_to_pdx(maddr_to_mfn(ma)) - directmap_base_pdx) < From patchwork Mon Jul 17 16:03:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 67B9BC001DF for ; Mon, 17 Jul 2023 16:03:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564730.882403 (Exim 4.92) (envelope-from ) id 1qLQh6-0002tQ-AD; Mon, 17 Jul 2023 16:03:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564730.882403; Mon, 17 Jul 2023 16:03:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh6-0002tD-5k; Mon, 17 Jul 2023 16:03:28 +0000 Received: by outflank-mailman (input) for mailman id 564730; Mon, 17 Jul 2023 16:03:26 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh4-00027B-4J for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:26 +0000 Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [2a00:1450:4864:20::52b]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 75277e3a-24bb-11ee-b23a-6b7b168915f2; Mon, 17 Jul 2023 18:03:23 +0200 (CEST) Received: by mail-ed1-x52b.google.com with SMTP id 4fb4d7f45d1cf-51e619bcbf9so6491891a12.3 for ; Mon, 17 Jul 2023 09:03:23 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:23 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 75277e3a-24bb-11ee-b23a-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609803; x=1692201803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r1Ds6GpX99KKIP7Xn+WLKam0uM2qlDi/O2AccjIBDFI=; b=kzQU5pUvrMIxMWwKhNFFloQfT3Fid9QUqdqwmdmy2o74vkeIaW9smWYYfEV9sACSS6 NZhlaU8DWiYfNw6psxWBx/XqBRkzYy7pKoWsYLbmA6N6hISS+k/vuUicJ4saBmwQ4ECx 4pCcUVmbDIGfqVSAJ4cSxxzuJ/IjuvshhA9vc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609803; x=1692201803; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r1Ds6GpX99KKIP7Xn+WLKam0uM2qlDi/O2AccjIBDFI=; b=BN4z2FxhydydOExwPeTorzEj2veQK+Hb0Q6deiW217Shyd6ofDw1sbeMHPyGBp1JO2 i4B+HS/fFmaMC1m9UzEjSgF+a8ryavhDw0foc7+gO8eZB/ujpYEVbJ4Ao6854hgEDOlz Wqi+1HsMwg4/JmcJ9oZDFtMMXNYwJR/skZWQY45OkQnFFh6cdMhez2BF06x8mW0m8Nt3 J2hkZnX7DZVN2Gdv3kEVGTUlM43tW4u7GZV3LVBWHevfPd83VWCKJQvnv0lzHkmHImEG nhn2gFa5S2mwkFFYUh6bYeAiAIGOjd3lf8IgRLw6AnNREq0Al+PXOkE5Qnx7LlOfR5Fi 1kvQ== X-Gm-Message-State: ABy/qLaJ2rRS5ewuPJmXG8FU1PBNtdMxvB4RFHQqkld+obL7rKoE8K8R O7RWT3JCWxcLOuNlYYhHS4dmi1KItgbRSKnxAWg= X-Google-Smtp-Source: APBJJlGXMxR+Y48i+AvaPfZNSSbKNoQ9OV+wgR9SMk5t9XhKnZ+4VMKWcARiO9Bct80MeXVomlzdgw== X-Received: by 2002:aa7:d5ca:0:b0:51d:a181:d78e with SMTP id d10-20020aa7d5ca000000b0051da181d78emr10646412eds.27.1689609803427; Mon, 17 Jul 2023 09:03:23 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH 3/8] pdx: Mark pdx hole description globals readonly after boot Date: Mon, 17 Jul 2023 17:03:13 +0100 Message-Id: <20230717160318.2113-4-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 They define where the compressible area of valid mfns is, and all of them are populated on boot (with the exception of max_pdx, that's updated on memory hotplug). No functional change. Signed-off-by: Alejandro Vallejo Reviewed-by: Jan Beulich --- xen/common/pdx.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/xen/common/pdx.c b/xen/common/pdx.c index ec64d3d2ef..99d4a90a50 100644 --- a/xen/common/pdx.c +++ b/xen/common/pdx.c @@ -47,30 +47,33 @@ * ones. */ -/** Maximum (non-inclusive) usable pdx */ +/** + * Maximum (non-inclusive) usable pdx. Must be + * modifiable after init due to memory hotplug + */ unsigned long __read_mostly max_pdx; /** Mask for the lower non-compressible bits of an mfn */ -unsigned long __read_mostly pfn_pdx_bottom_mask = ~0UL; +unsigned long __ro_after_init pfn_pdx_bottom_mask = ~0UL; /** Mask for the lower non-compressible bits of an maddr or vaddr */ -unsigned long __read_mostly ma_va_bottom_mask = ~0UL; +unsigned long __ro_after_init ma_va_bottom_mask = ~0UL; /** Mask for the higher non-compressible bits of an mfn */ -unsigned long __read_mostly pfn_top_mask = 0; +unsigned long __ro_after_init pfn_top_mask = 0; /** Mask for the higher non-compressible bits of an maddr or vaddr */ -unsigned long __read_mostly ma_top_mask = 0; +unsigned long __ro_after_init ma_top_mask = 0; /** * Mask for a pdx compression bit slice. * * Invariant: valid(mfn) implies (mfn & pfn_hole_mask) == 0 */ -unsigned long __read_mostly pfn_hole_mask = 0; +unsigned long __ro_after_init pfn_hole_mask = 0; /** Number of bits of the "compressible" bit slice of an mfn */ -unsigned int __read_mostly pfn_pdx_hole_shift = 0; +unsigned int __ro_after_init pfn_pdx_hole_shift = 0; unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS( (FRAMETABLE_NR + PDX_GROUP_COUNT - 1) / PDX_GROUP_COUNT)] = { [0] = 1 }; From patchwork Mon Jul 17 16:03:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 887BFC001B0 for ; Mon, 17 Jul 2023 16:03:51 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564731.882407 (Exim 4.92) (envelope-from ) id 1qLQh6-0002vU-JW; Mon, 17 Jul 2023 16:03:28 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564731.882407; Mon, 17 Jul 2023 16:03:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh6-0002v1-Du; Mon, 17 Jul 2023 16:03:28 +0000 Received: by outflank-mailman (input) for mailman id 564731; Mon, 17 Jul 2023 16:03:26 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh4-0002Zo-LD for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:26 +0000 Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com [2a00:1450:4864:20::131]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 75af6146-24bb-11ee-8611-37d641c3527e; Mon, 17 Jul 2023 18:03:24 +0200 (CEST) Received: by mail-lf1-x131.google.com with SMTP id 2adb3069b0e04-4fb73ba3b5dso7599782e87.1 for ; Mon, 17 Jul 2023 09:03:24 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:23 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 75af6146-24bb-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609804; x=1692201804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v0IhxmE72AYW3LaZbJhXXAiAR4Fd8XXkCy6sjFZ0aFM=; b=FtvKo5PCq617lmKRFyV0zgLNB30iOSu8auC6E2SOys3Xyqd1Rucss97XPbNM20PqXT SPQfBQbbQObshwWPAkZzbmIPg7N436jBLf7+3WQ6DXGTXHjCtYZmCRrZvig/7frXOgdI M+3JpXTV8FmMP83cXmg0nAcyJtUv1ZZtuwFNA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609804; x=1692201804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v0IhxmE72AYW3LaZbJhXXAiAR4Fd8XXkCy6sjFZ0aFM=; b=kAxBWSDNE5fv6w2AE3R8M8A1mWUG2Z3t1OkRZnGz5Lo+Uxp3mLSdOMpUcrIfdR9o1J QgfaKZWCw3EFUS+OUSEPFR1MlwUNwj0e8KtyxPPDwifmKYpgrrIYIl3QEXzJOKP99/9e bhtI0hJG5cMvFwhHjicZdMsuko6FOxVxU7JHbUeuWeTT0/yehL+UWCLOROwmI/8M9TtI WkXjy2P7QAez7W4J9WLUT70n+LIyQW7YrgUO2Y9qG+H8eSOcnt2iThPmMDGth4ASLMvE 7tf4EicvigZawKi6E1QhmZi4+gNurGt38rBp+ja72qdZxXdzm1Thv1enRGrpdBQIrPiE oBwQ== X-Gm-Message-State: ABy/qLZx+Ob8gHrCSWaT1GcWIlyOEATiV3b6w/DqNjkBKrOrHynawk/i CPevNMJCwyLDPN6pb5fwoNf8AYF2HPavChTT/Yw= X-Google-Smtp-Source: APBJJlGlOoLPXXCdptae0m4OTaZqL/bcr7BbXfoZ/2PlFQ6YxvMLg3XpFG5LN6Zp4BtfhPEDporJiw== X-Received: by 2002:a05:6512:3194:b0:4f9:586b:dba6 with SMTP id i20-20020a056512319400b004f9586bdba6mr10059097lfe.10.1689609804064; Mon, 17 Jul 2023 09:03:24 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 4/8] build: Remove CONFIG_HAS_PDX Date: Mon, 17 Jul 2023 17:03:14 +0100 Message-Id: <20230717160318.2113-5-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 It's set everywhere and can't be turned off because it's presence is assumed in several parts of the codebase. This is an initial patch towards adding a more fine-grained CONFIG_HAS_PDX_COMPRESSION that can actually be disabled on systems that don't typically benefit from it. No functional change. Signed-off-by: Alejandro Vallejo --- xen/arch/arm/Kconfig | 1 - xen/arch/x86/Kconfig | 1 - xen/common/Kconfig | 3 --- xen/common/Makefile | 2 +- xen/include/xen/pdx.h | 3 --- 5 files changed, 1 insertion(+), 9 deletions(-) diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index 439cc94f33..ea1949fbaa 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -14,7 +14,6 @@ config ARM select HAS_ALTERNATIVE select HAS_DEVICE_TREE select HAS_PASSTHROUGH - select HAS_PDX select HAS_PMAP select HAS_UBSAN select IOMMU_FORCE_PT_SHARE diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig index 92f3a627da..30df085d96 100644 --- a/xen/arch/x86/Kconfig +++ b/xen/arch/x86/Kconfig @@ -24,7 +24,6 @@ config X86 select HAS_PASSTHROUGH select HAS_PCI select HAS_PCI_MSI - select HAS_PDX select HAS_SCHED_GRANULARITY select HAS_UBSAN select HAS_VPCI if HVM diff --git a/xen/common/Kconfig b/xen/common/Kconfig index dd8d7c3f1c..40ec63c4b2 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -53,9 +53,6 @@ config HAS_IOPORTS config HAS_KEXEC bool -config HAS_PDX - bool - config HAS_PMAP bool diff --git a/xen/common/Makefile b/xen/common/Makefile index 46049eac35..0020cafb8a 100644 --- a/xen/common/Makefile +++ b/xen/common/Makefile @@ -29,7 +29,7 @@ obj-y += multicall.o obj-y += notifier.o obj-$(CONFIG_NUMA) += numa.o obj-y += page_alloc.o -obj-$(CONFIG_HAS_PDX) += pdx.o +obj-y += pdx.o obj-$(CONFIG_PERF_COUNTERS) += perfc.o obj-bin-$(CONFIG_HAS_PMAP) += pmap.init.o obj-y += preempt.o diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index de5439a5e5..67ae20e89c 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -67,8 +67,6 @@ * region involved. */ -#ifdef CONFIG_HAS_PDX - extern unsigned long max_pdx; extern unsigned long pfn_pdx_bottom_mask, ma_va_bottom_mask; extern unsigned int pfn_pdx_hole_shift; @@ -171,7 +169,6 @@ static inline unsigned long pdx_to_pfn(unsigned long pdx) */ void pfn_pdx_hole_setup(unsigned long mask); -#endif /* HAS_PDX */ #endif /* __XEN_PDX_H__ */ /* From patchwork Mon Jul 17 16:03:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id CCCC1EB64DC for ; Mon, 17 Jul 2023 16:03:40 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564732.882412 (Exim 4.92) (envelope-from ) id 1qLQh7-00032l-3R; Mon, 17 Jul 2023 16:03:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564732.882412; Mon, 17 Jul 2023 16:03:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh6-00030p-RD; Mon, 17 Jul 2023 16:03:28 +0000 Received: by outflank-mailman (input) for mailman id 564732; Mon, 17 Jul 2023 16:03:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh5-00027B-4c for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:27 +0000 Received: from mail-ed1-x52e.google.com (mail-ed1-x52e.google.com [2a00:1450:4864:20::52e]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 75f1a2cd-24bb-11ee-b23a-6b7b168915f2; Mon, 17 Jul 2023 18:03:25 +0200 (CEST) Received: by mail-ed1-x52e.google.com with SMTP id 4fb4d7f45d1cf-51e56749750so6312473a12.0 for ; Mon, 17 Jul 2023 09:03:25 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:24 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 75f1a2cd-24bb-11ee-b23a-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609804; x=1692201804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EaOSzOKwnaNDbtFFGDCLrWszAMCEvLERpiHBsiwXmN8=; b=VvXSrWXjavQXI5yrEBMvKrTHVuE2itZE0f9oFhSLvAVAI1RTJGLVA3uqhxBzf3MMSh zTyK49m5celffRs4jobunADI5Kc3uv3YaHzbiP4iy9ONqjGDUNA2QMg69bXyvuq5wjl5 3MYXJd4ATPjuKvZGXB/uxLoB04zVq8sxvM1l0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609804; x=1692201804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EaOSzOKwnaNDbtFFGDCLrWszAMCEvLERpiHBsiwXmN8=; b=dAjb9EUzeNiNBZDLzdWNmgn7m776iSFuQt1W4/ANG5Y7cDJnc7MMEBHdGkwFwCkLo+ h6qty6M6LhX+0eUqis74GlNhtN7UKC51JyV0qB7mCkN5xnOXYumwv2UBx5nfeIxGbV7Z aIwnY5k3We7LBYir2RDTFQu/PiGaRy998fgOKZnPGvZYb2IO1bAD4BN5y2cDHWovQGdN BzjViPuWxES8GI3p+XlFFdUGXyJobFyvgA2NdUnpSslQVarS5DdJ3XKRMh3rrCJniWdO oxEtSsmO3Tdb9pFLGzCA1sBPWECbYUW2yY7GlVJki2KEuoIXZ879VRhNlfQd4Fdkaa2m EZRQ== X-Gm-Message-State: ABy/qLYFMYNfCPmq2Hojanoxe2gc4y1dXKDxeisgK+xiEFNRHrXqKGYR rlv5+jd38kd2qR6cXcEm7YxDyJETIvdh09JF+cs= X-Google-Smtp-Source: APBJJlHxlHXDnujOKuyyvPzhR1A6cx4oXng80WRe0d6aM0pVJeTuzQusY90cvxRft80tFLU+BAPBZA== X-Received: by 2002:a05:6402:8c2:b0:51d:ece5:afd9 with SMTP id d2-20020a05640208c200b0051dece5afd9mr12464346edz.21.1689609804717; Mon, 17 Jul 2023 09:03:24 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Stefano Stabellini , Julien Grall , Bertrand Marquis , Volodymyr Babchuk , Andrew Cooper , George Dunlap , Jan Beulich , Wei Liu , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH 5/8] mm: Factor out the pdx compression logic in ma/va converters Date: Mon, 17 Jul 2023 17:03:15 +0100 Message-Id: <20230717160318.2113-6-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 This patch factors out the pdx compression logic hardcoded in both ports for the maddr<->vaddr conversion functions. Touches both x86 and arm ports. Signed-off-by: Alejandro Vallejo --- xen/arch/arm/include/asm/mm.h | 3 +-- xen/arch/x86/include/asm/x86_64/page.h | 28 +++++++++++--------------- xen/include/xen/pdx.h | 25 +++++++++++++++++++++++ 3 files changed, 38 insertions(+), 18 deletions(-) diff --git a/xen/arch/arm/include/asm/mm.h b/xen/arch/arm/include/asm/mm.h index 1a83f41879..78cb23858a 100644 --- a/xen/arch/arm/include/asm/mm.h +++ b/xen/arch/arm/include/asm/mm.h @@ -320,8 +320,7 @@ static inline void *maddr_to_virt(paddr_t ma) (DIRECTMAP_SIZE >> PAGE_SHIFT)); return (void *)(XENHEAP_VIRT_START - (directmap_base_pdx << PAGE_SHIFT) + - ((ma & ma_va_bottom_mask) | - ((ma & ma_top_mask) >> pfn_pdx_hole_shift))); + maddr_to_directmapoff(ma)); } #endif diff --git a/xen/arch/x86/include/asm/x86_64/page.h b/xen/arch/x86/include/asm/x86_64/page.h index 53faa7875b..b589c93e77 100644 --- a/xen/arch/x86/include/asm/x86_64/page.h +++ b/xen/arch/x86/include/asm/x86_64/page.h @@ -36,26 +36,22 @@ static inline unsigned long __virt_to_maddr(unsigned long va) { ASSERT(va < DIRECTMAP_VIRT_END); if ( va >= DIRECTMAP_VIRT_START ) - va -= DIRECTMAP_VIRT_START; - else - { - BUILD_BUG_ON(XEN_VIRT_END - XEN_VIRT_START != GB(1)); - /* Signed, so ((long)XEN_VIRT_START >> 30) fits in an imm32. */ - ASSERT(((long)va >> (PAGE_ORDER_1G + PAGE_SHIFT)) == - ((long)XEN_VIRT_START >> (PAGE_ORDER_1G + PAGE_SHIFT))); - - va += xen_phys_start - XEN_VIRT_START; - } - return (va & ma_va_bottom_mask) | - ((va << pfn_pdx_hole_shift) & ma_top_mask); + return directmapoff_to_maddr(va - DIRECTMAP_VIRT_START); + + BUILD_BUG_ON(XEN_VIRT_END - XEN_VIRT_START != GB(1)); + /* Signed, so ((long)XEN_VIRT_START >> 30) fits in an imm32. */ + ASSERT(((long)va >> (PAGE_ORDER_1G + PAGE_SHIFT)) == + ((long)XEN_VIRT_START >> (PAGE_ORDER_1G + PAGE_SHIFT))); + + return xen_phys_start + va - XEN_VIRT_START; } static inline void *__maddr_to_virt(unsigned long ma) { - ASSERT(pfn_to_pdx(ma >> PAGE_SHIFT) < (DIRECTMAP_SIZE >> PAGE_SHIFT)); - return (void *)(DIRECTMAP_VIRT_START + - ((ma & ma_va_bottom_mask) | - ((ma & ma_top_mask) >> pfn_pdx_hole_shift))); + /* Offset in the direct map, accounting for pdx compression */ + size_t va_offset = maddr_to_directmapoff(ma); + ASSERT(va_offset < DIRECTMAP_SIZE); + return (void *)(DIRECTMAP_VIRT_START + va_offset); } /* read access (should only be used for debug printk's) */ diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index 67ae20e89c..f8ca0f5821 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -158,6 +158,31 @@ static inline unsigned long pdx_to_pfn(unsigned long pdx) #define mfn_to_pdx(mfn) pfn_to_pdx(mfn_x(mfn)) #define pdx_to_mfn(pdx) _mfn(pdx_to_pfn(pdx)) +/** + * Computes the offset into the direct map of an maddr + * + * @param ma Machine address + * @return Offset on the direct map where that + * machine address can be accessed + */ +static inline unsigned long maddr_to_directmapoff(uint64_t ma) +{ + return ((ma & ma_top_mask) >> pfn_pdx_hole_shift) | + (ma & ma_va_bottom_mask); +} + +/** + * Computes a machine address given a direct map offset + * + * @param offset Offset into the direct map + * @return Corresponding machine address of that virtual location + */ +static inline uint64_t directmapoff_to_maddr(unsigned long offset) +{ + return ((offset << pfn_pdx_hole_shift) & ma_top_mask) | + (offset & ma_va_bottom_mask); +} + /** * Initializes global variables with information about the compressible * range of the current memory regions. From patchwork Mon Jul 17 16:03:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B300AC00528 for ; Mon, 17 Jul 2023 16:03:48 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564733.882419 (Exim 4.92) (envelope-from ) id 1qLQh7-00038N-EN; Mon, 17 Jul 2023 16:03:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564733.882419; Mon, 17 Jul 2023 16:03:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh7-00037B-8D; Mon, 17 Jul 2023 16:03:29 +0000 Received: by outflank-mailman (input) for mailman id 564733; Mon, 17 Jul 2023 16:03:27 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh5-0002Zo-Ih for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:27 +0000 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [2a00:1450:4864:20::530]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 764ff69d-24bb-11ee-8611-37d641c3527e; Mon, 17 Jul 2023 18:03:25 +0200 (CEST) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-5216569f9e3so4831001a12.0 for ; Mon, 17 Jul 2023 09:03:25 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 764ff69d-24bb-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609805; x=1692201805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HhpWCn/ri7wmy6U70OoVzKYS333ZuYeUMZy4TaZcJGg=; b=LC8OXhHYu2epObgaJF1GN8yE+taRZmj2Deix//fzqFxnm4zzgTsSRxqKU35v6fbcaY Fo3dh22EUX1fPQx8YgXsJt8gWAY5OeKljz9zY2CL4yNl9ksag1Y0Q0Wy4rmZSqD07dSt OHNePmKyjiZElFKHZrs6C2QYd+1pzVU4Ppjjo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609805; x=1692201805; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HhpWCn/ri7wmy6U70OoVzKYS333ZuYeUMZy4TaZcJGg=; b=dWAR5Hhju0ANpEtpW7+pxIjBSgcO24xiYdOa1+j/KjroscnrIWpD3J7VXbSzF889Qd neXSPIZZrJ2D+JWdttDqptH0e8cTwaCS8mWcMr/u+sSiqAJGz1KzvKqcUuZGjgUazosG 40fvk1r3Cs8KBryJKh9DU9jmIagUYdNvWQ1EOnIOU4JWplTjfqM+WurlXlLSDdb0Fe6g puGHCkYSn5A9vmxDI6ne8/3QYGdtb88O3D+EncWIdYLGFBvhvlktGZDEh1FfJZoHYlrn VB/HvuCrXdZRYCHJhpsUspCnVyXdIdeO4zAD+CCdrQZKUs+PjTG064Id0YxD1iJHKMnM jhpg== X-Gm-Message-State: ABy/qLY/mgRNmWM0A2KTZPakhNB2WXJyWDZweNRnGLb+q6GK6l8rXIAg 9h8N7aAdTF61BMygIGS5HFuwchzlSwoZNugTIYg= X-Google-Smtp-Source: APBJJlFnFUUmOQ+wqt6dusKepCAatkVsCd0yz8EvIbApX95MNtQnG6lkA4Vb2tobteJBuPe+BusbqA== X-Received: by 2002:a05:6402:335:b0:51e:288d:2a27 with SMTP id q21-20020a056402033500b0051e288d2a27mr12091129edw.11.1689609805273; Mon, 17 Jul 2023 09:03:25 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH 6/8] mm/pdx: Standardize region validation wrt pdx compression Date: Mon, 17 Jul 2023 17:03:16 +0100 Message-Id: <20230717160318.2113-7-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Regions must be occasionally validated for pdx compression validity. That is, whether any of the machine addresses spanning the region have a bit set in the pdx "hole" (which is expected to always contain zeroes). There are a few such tests through the code, and they all check for different things. This patch replaces all such occurences with a call to a centralized function that checks a region for validity. Signed-off-by: Alejandro Vallejo --- xen/arch/x86/x86_64/mm.c | 2 +- xen/common/efi/boot.c | 6 +++--- xen/common/pdx.c | 13 +++++++++++-- xen/include/xen/pdx.h | 9 +++++++++ 4 files changed, 24 insertions(+), 6 deletions(-) diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c index 60db439af3..914e65c26c 100644 --- a/xen/arch/x86/x86_64/mm.c +++ b/xen/arch/x86/x86_64/mm.c @@ -1168,7 +1168,7 @@ static int mem_hotadd_check(unsigned long spfn, unsigned long epfn) if ( (spfn | epfn) & ((1UL << PAGETABLE_ORDER) - 1) ) return 0; - if ( (spfn | epfn) & pfn_hole_mask ) + if ( !pdx_is_region_compressible(spfn, epfn) ) return 0; /* Make sure the new range is not present now */ diff --git a/xen/common/efi/boot.c b/xen/common/efi/boot.c index 24169b7b50..b098a8c030 100644 --- a/xen/common/efi/boot.c +++ b/xen/common/efi/boot.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #if EFI_PAGE_SIZE != PAGE_SIZE # error Cannot use xen/pfn.h here! @@ -1647,7 +1648,7 @@ static bool __init cf_check ram_range_valid(unsigned long smfn, unsigned long em { unsigned long sz = pfn_to_pdx(emfn - 1) / PDX_GROUP_COUNT + 1; - return !(smfn & pfn_hole_mask) && + return pdx_is_region_compressible(smfn, emfn) && find_next_bit(pdx_group_valid, sz, pfn_to_pdx(smfn) / PDX_GROUP_COUNT) < sz; } @@ -1759,8 +1760,7 @@ void __init efi_init_memory(void) prot |= _PAGE_NX; if ( pfn_to_pdx(emfn - 1) < (DIRECTMAP_SIZE >> PAGE_SHIFT) && - !(smfn & pfn_hole_mask) && - !((smfn ^ (emfn - 1)) & ~pfn_pdx_bottom_mask) ) + pdx_is_region_compressible(smfn, emfn)) { if ( (unsigned long)mfn_to_virt(emfn - 1) >= HYPERVISOR_VIRT_END ) prot &= ~_PAGE_GLOBAL; diff --git a/xen/common/pdx.c b/xen/common/pdx.c index 99d4a90a50..72845e4bab 100644 --- a/xen/common/pdx.c +++ b/xen/common/pdx.c @@ -88,7 +88,7 @@ bool __mfn_valid(unsigned long mfn) } /* Sets all bits from the most-significant 1-bit down to the LSB */ -static uint64_t __init fill_mask(uint64_t mask) +static uint64_t fill_mask(uint64_t mask) { while (mask & (mask + 1)) mask |= mask + 1; @@ -96,6 +96,15 @@ static uint64_t __init fill_mask(uint64_t mask) return mask; } +bool pdx_is_region_compressible(unsigned long smfn, unsigned long emfn) +{ + uint64_t base = smfn << PAGE_SHIFT; + uint64_t len = (emfn - smfn) << PAGE_SHIFT; + + return !(smfn & pfn_hole_mask) && + !(pdx_region_mask(base, len) & ~ma_va_bottom_mask); +} + /* We don't want to compress the low MAX_ORDER bits of the addresses. */ uint64_t __init pdx_init_mask(uint64_t base_addr) { @@ -103,7 +112,7 @@ uint64_t __init pdx_init_mask(uint64_t base_addr) (uint64_t)1 << (MAX_ORDER + PAGE_SHIFT)) - 1); } -uint64_t __init pdx_region_mask(uint64_t base, uint64_t len) +uint64_t pdx_region_mask(uint64_t base, uint64_t len) { /* * We say a bit "moves" in a range if there exist 2 addresses in that diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index f8ca0f5821..5378e664c2 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -77,6 +77,15 @@ extern unsigned long pfn_top_mask, ma_top_mask; (sizeof(*frame_table) & -sizeof(*frame_table))) extern unsigned long pdx_group_valid[]; +/** + * Validate a region's compatibility with the current compression runtime + * + * @param smfn Start mfn + * @param emfn End mfn (non-inclusive) + * @return True iff the region can be used with the current compression + */ +bool pdx_is_region_compressible(unsigned long smfn, unsigned long emfn); + /** * Calculates a mask covering "moving" bits of all addresses of a region * From patchwork Mon Jul 17 16:03:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BF137C04A6A for ; Mon, 17 Jul 2023 16:03:49 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564735.882433 (Exim 4.92) (envelope-from ) id 1qLQh8-0003US-LY; Mon, 17 Jul 2023 16:03:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564735.882433; Mon, 17 Jul 2023 16:03:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh8-0003Sr-Ap; Mon, 17 Jul 2023 16:03:30 +0000 Received: by outflank-mailman (input) for mailman id 564735; Mon, 17 Jul 2023 16:03:28 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh6-0002Zo-B9 for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:28 +0000 Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [2a00:1450:4864:20::135]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 76c48538-24bb-11ee-8611-37d641c3527e; Mon, 17 Jul 2023 18:03:26 +0200 (CEST) Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-4fba74870abso7440332e87.0 for ; Mon, 17 Jul 2023 09:03:26 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 76c48538-24bb-11ee-8611-37d641c3527e DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609806; x=1692201806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UgGX9V3xdVqjoBF9z1oNCiSxRVEcUKRWlOAV3af9M+4=; b=BNbG3GmEohiD+wkcErA+NgY75lkbwVbS38Sr2VrBTPEoBpQzxWm2irAT5/NNR58loy VVWQyPj/JrHX6BVI4DDkeX39jDBSk8Xm5uew71HUJi8cNb09gjRwGnzvSaJV66BSYtsm K0TUkLCyM6roQlTDVMYddaI90/ailSkcp5ouU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609806; x=1692201806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UgGX9V3xdVqjoBF9z1oNCiSxRVEcUKRWlOAV3af9M+4=; b=Zl3p13GSvqSMARnDjBv+FkRr8kEjCSFOD29gQhA5Ah4vrkJGwsKjMicHdhJDxQk7HQ FBEIghyYHO2XIaVv9VXNt0FqKLcMT0a5LMrM4U0Cry2a+LkENJLHttL99dFht7gXskyB Xkdu0RzH6uqTpGOcvDQrd9vBExWQSEPCr+aILVUNlvLdM0jC96YH6lUhF3B6I6qJOUFH sMkhUZpDHStz4VwX6fumoaVIbLGcoFHfbztdb9bZA2JL5+DtUar2qnL9IdpXFcAfqEOp 1SL3oCgImdh3LpG9ydFZGH6snTK9/X6QfY0TsNBB6EbzOndh5apKhf7vIhJ5Dg0JXx+Z BXMg== X-Gm-Message-State: ABy/qLYaj8EbemCz/smQ1BjA9PT79cBOAE9X0G3BKztE7oF4Xm5nPcPU KEhZvXKq+IKnEXp/UTDzUWSymqC1U4JU0YeJnek= X-Google-Smtp-Source: APBJJlFJxGwk4jlCMhmmUrOdLTtWrCITKqaYX+v7ey5mw1dcyCM/XhITGM6eD6gFd8eNs+sa/JQ8sQ== X-Received: by 2002:a05:6512:711:b0:4fb:751a:98d5 with SMTP id b17-20020a056512071100b004fb751a98d5mr7144788lfs.18.1689609805885; Mon, 17 Jul 2023 09:03:25 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH 7/8] pdx: Reorder pdx.[ch] Date: Mon, 17 Jul 2023 17:03:17 +0100 Message-Id: <20230717160318.2113-8-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 The next patch compiles out compression-related chunks, and it's helpful to have them grouped together beforehand. Signed-off-by: Alejandro Vallejo --- xen/common/pdx.c | 58 +++++++++++++++++++++---------------------- xen/include/xen/pdx.h | 37 +++++++++++++++++++++------ 2 files changed, 59 insertions(+), 36 deletions(-) diff --git a/xen/common/pdx.c b/xen/common/pdx.c index 72845e4bab..cc963a3cb3 100644 --- a/xen/common/pdx.c +++ b/xen/common/pdx.c @@ -20,6 +20,35 @@ #include #include +/** + * Maximum (non-inclusive) usable pdx. Must be + * modifiable after init due to memory hotplug + */ +unsigned long __read_mostly max_pdx; + +unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS( + (FRAMETABLE_NR + PDX_GROUP_COUNT - 1) / PDX_GROUP_COUNT)] = { [0] = 1 }; + +bool __mfn_valid(unsigned long mfn) +{ + if ( unlikely(evaluate_nospec(mfn >= max_page)) ) + return false; + return likely(!(mfn & pfn_hole_mask)) && + likely(test_bit(pfn_to_pdx(mfn) / PDX_GROUP_COUNT, + pdx_group_valid)); +} + +void set_pdx_range(unsigned long smfn, unsigned long emfn) +{ + unsigned long idx, eidx; + + idx = pfn_to_pdx(smfn) / PDX_GROUP_COUNT; + eidx = (pfn_to_pdx(emfn - 1) + PDX_GROUP_COUNT) / PDX_GROUP_COUNT; + + for ( ; idx < eidx; ++idx ) + __set_bit(idx, pdx_group_valid); +} + /* * Diagram to make sense of the following variables. The masks and shifts * are done on mfn values in order to convert to/from pdx: @@ -47,12 +76,6 @@ * ones. */ -/** - * Maximum (non-inclusive) usable pdx. Must be - * modifiable after init due to memory hotplug - */ -unsigned long __read_mostly max_pdx; - /** Mask for the lower non-compressible bits of an mfn */ unsigned long __ro_after_init pfn_pdx_bottom_mask = ~0UL; @@ -75,18 +98,6 @@ unsigned long __ro_after_init pfn_hole_mask = 0; /** Number of bits of the "compressible" bit slice of an mfn */ unsigned int __ro_after_init pfn_pdx_hole_shift = 0; -unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS( - (FRAMETABLE_NR + PDX_GROUP_COUNT - 1) / PDX_GROUP_COUNT)] = { [0] = 1 }; - -bool __mfn_valid(unsigned long mfn) -{ - if ( unlikely(evaluate_nospec(mfn >= max_page)) ) - return false; - return likely(!(mfn & pfn_hole_mask)) && - likely(test_bit(pfn_to_pdx(mfn) / PDX_GROUP_COUNT, - pdx_group_valid)); -} - /* Sets all bits from the most-significant 1-bit down to the LSB */ static uint64_t fill_mask(uint64_t mask) { @@ -127,17 +138,6 @@ uint64_t pdx_region_mask(uint64_t base, uint64_t len) return fill_mask(base ^ (base + len - 1)); } -void set_pdx_range(unsigned long smfn, unsigned long emfn) -{ - unsigned long idx, eidx; - - idx = pfn_to_pdx(smfn) / PDX_GROUP_COUNT; - eidx = (pfn_to_pdx(emfn - 1) + PDX_GROUP_COUNT) / PDX_GROUP_COUNT; - - for ( ; idx < eidx; ++idx ) - __set_bit(idx, pdx_group_valid); -} - void __init pfn_pdx_hole_setup(unsigned long mask) { unsigned int i, j, bottom_shift = 0, hole_shift = 0; diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index 5378e664c2..ce27177b56 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -68,15 +68,41 @@ */ extern unsigned long max_pdx; -extern unsigned long pfn_pdx_bottom_mask, ma_va_bottom_mask; -extern unsigned int pfn_pdx_hole_shift; -extern unsigned long pfn_hole_mask; -extern unsigned long pfn_top_mask, ma_top_mask; #define PDX_GROUP_COUNT ((1 << PDX_GROUP_SHIFT) / \ (sizeof(*frame_table) & -sizeof(*frame_table))) extern unsigned long pdx_group_valid[]; +/** + * Mark [smfn, emfn) as allocatable in the frame table + * + * @param smfn Start mfn + * @param emfn End mfn + */ +void set_pdx_range(unsigned long smfn, unsigned long emfn); + +/** + * Invoked to determine if an mfn has an associated valid frame table entry + * + * In order for it to be legal it must pass bounds, grouping and + * compression sanity checks. + * + * @param mfn To-be-checked mfn + * @return True iff all checks pass + */ +bool __mfn_valid(unsigned long mfn); + +#define page_to_pdx(pg) ((pg) - frame_table) +#define pdx_to_page(pdx) gcc11_wrap(frame_table + (pdx)) + +#define mfn_to_pdx(mfn) pfn_to_pdx(mfn_x(mfn)) +#define pdx_to_mfn(pdx) _mfn(pdx_to_pfn(pdx)) + +extern unsigned long pfn_pdx_bottom_mask, ma_va_bottom_mask; +extern unsigned int pfn_pdx_hole_shift; +extern unsigned long pfn_hole_mask; +extern unsigned long pfn_top_mask, ma_top_mask; + /** * Validate a region's compatibility with the current compression runtime * @@ -164,9 +190,6 @@ static inline unsigned long pdx_to_pfn(unsigned long pdx) ((pdx << pfn_pdx_hole_shift) & pfn_top_mask); } -#define mfn_to_pdx(mfn) pfn_to_pdx(mfn_x(mfn)) -#define pdx_to_mfn(pdx) _mfn(pdx_to_pfn(pdx)) - /** * Computes the offset into the direct map of an maddr * From patchwork Mon Jul 17 16:03:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alejandro Vallejo X-Patchwork-Id: 13315966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id DBF62C001B0 for ; Mon, 17 Jul 2023 16:03:41 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.564734.882429 (Exim 4.92) (envelope-from ) id 1qLQh8-0003PN-7d; Mon, 17 Jul 2023 16:03:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 564734.882429; Mon, 17 Jul 2023 16:03:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh7-0003Lw-V7; Mon, 17 Jul 2023 16:03:29 +0000 Received: by outflank-mailman (input) for mailman id 564734; Mon, 17 Jul 2023 16:03:28 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1qLQh6-00027B-4o for xen-devel@lists.xenproject.org; Mon, 17 Jul 2023 16:03:28 +0000 Received: from mail-lf1-x12d.google.com (mail-lf1-x12d.google.com [2a00:1450:4864:20::12d]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 773057de-24bb-11ee-b23a-6b7b168915f2; Mon, 17 Jul 2023 18:03:27 +0200 (CEST) Received: by mail-lf1-x12d.google.com with SMTP id 2adb3069b0e04-4fcd615d7d6so6903860e87.3 for ; Mon, 17 Jul 2023 09:03:27 -0700 (PDT) Received: from EMEAENGAAD19049.citrite.net (default-46-102-197-194.interdsl.co.uk. [46.102.197.194]) by smtp.gmail.com with ESMTPSA id bo28-20020a0564020b3c00b0051df5bd1cd8sm10099050edb.65.2023.07.17.09.03.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jul 2023 09:03:26 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 773057de-24bb-11ee-b23a-6b7b168915f2 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloud.com; s=cloud; t=1689609806; x=1692201806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7QGfkDVjG71aom9bAterc/1wKh4jZEIowYjFqXVaWh0=; b=J9UFqD4U6/+xFXgvL/eHS1K3ZsRFfObmThVd0Sx3haPrf1u/4zdURKFRc5ZJ9k+yI3 YbgVQ52ZFPB0UzsfMDnMrO+RojCfSZFbkTfIoBA6QTHyKQ/aSMngs1hVScSzvRMxXnub bDv923akwd+wLhxzq2A0UDY7tLIKqbIPS/vwc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689609806; x=1692201806; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7QGfkDVjG71aom9bAterc/1wKh4jZEIowYjFqXVaWh0=; b=W/FJiTum4ljF8alLQao5viIJ3dKu/DiVSdbzT4MxVsd1aJU3yBrM0ZRdBhiWlnci+1 sTs8f+qa5HqbHXqdfKyyK+y5qHOSVhSNFq3S6GwhXjojSczi8hEkoSrD5hyRGn+SjGRr cBeH/X97cKiwPcLEhJ+6H7NE29WCQPPIq3Rsi4nWVjB2nUmc7krF4C/4mob4mGE0FKSu yRf9eIbrWzb4tSJhMGoNbPCel4UcYJd0FaKvOm9TlJNlKVaDKG3yvjQYWSiqokr8cTxP oKaZ2BmaxB9lfxEYOjm2QxzkUCsqRPS5hmKZ46zhf6IPsd7L0sKDNkvE3mgu/LdQIpGV Fi4g== X-Gm-Message-State: ABy/qLa/BcmUYZ79Jc37Pq8Y+Grj3lLs7uvaX8rAzV+u0VLXsGvPD3HU wEvaNJJY4tZOWNlFi0aBapl3agKj29xUcZPKypw= X-Google-Smtp-Source: APBJJlF5hBqYxBEdbU/CdUkMNf+ZWkqlphrHQ0PjL8lO0H04R4PCEvyaCbBP88krTvazp26SQzFivA== X-Received: by 2002:a05:6512:2822:b0:4fc:dcff:773b with SMTP id cf34-20020a056512282200b004fcdcff773bmr12474057lfb.3.1689609806597; Mon, 17 Jul 2023 09:03:26 -0700 (PDT) From: Alejandro Vallejo To: Xen-devel Cc: Alejandro Vallejo , Jan Beulich , Andrew Cooper , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Wei Liu , George Dunlap , Julien Grall , Stefano Stabellini Subject: [PATCH 8/8] pdx: Add CONFIG_HAS_PDX_COMPRESSION as a Kconfig option Date: Mon, 17 Jul 2023 17:03:18 +0100 Message-Id: <20230717160318.2113-9-alejandro.vallejo@cloud.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230717160318.2113-1-alejandro.vallejo@cloud.com> References: <20230717160318.2113-1-alejandro.vallejo@cloud.com> MIME-Version: 1.0 Adds a new compile-time flag to allow disabling pdx compression and compiles out compression-related code/data. It also shorts the pdx<->pfn conversion macros and creates stubs for masking fucntions. Signed-off-by: Alejandro Vallejo --- xen/arch/x86/domain.c | 19 +++++++++++++------ xen/common/Kconfig | 10 ++++++++++ xen/common/pdx.c | 15 +++++++++++---- xen/include/xen/pdx.h | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 68 insertions(+), 10 deletions(-) diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index 39c2153165..c818ccc4d5 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -458,7 +458,7 @@ void domain_cpu_policy_changed(struct domain *d) } } -#ifndef CONFIG_BIGMEM +#if !defined(CONFIG_BIGMEM) && defined(CONFIG_HAS_PDX_COMPRESSION) /* * The hole may be at or above the 44-bit boundary, so we need to determine * the total bit count until reaching 32 significant (not squashed out) bits @@ -485,13 +485,20 @@ static unsigned int __init noinline _domain_struct_bits(void) struct domain *alloc_domain_struct(void) { struct domain *d; -#ifdef CONFIG_BIGMEM - const unsigned int bits = 0; -#else + /* - * We pack the PDX of the domain structure into a 32-bit field within - * the page_info structure. Hence the MEMF_bits() restriction. + * Without CONFIG_BIGMEM, we pack the PDX of the domain structure into + * a 32-bit field within the page_info structure. Hence the MEMF_bits() + * restriction. With PDX compression in place the number of bits must + * be calculated at runtime, but it's fixed otherwise. + * + * On systems with CONFIG_BIGMEM there's no packing, and so there's no + * such restriction. */ +#if defined(CONFIG_BIGMEM) || !defined(CONFIG_HAS_PDX_COMPRESSION) + const unsigned int bits = IS_ENABLED(CONFIG_BIGMEM) ? 0 : + 32 + PAGE_SHIFT; +#else static unsigned int __read_mostly bits; if ( unlikely(!bits) ) diff --git a/xen/common/Kconfig b/xen/common/Kconfig index 40ec63c4b2..6605a60ff7 100644 --- a/xen/common/Kconfig +++ b/xen/common/Kconfig @@ -23,6 +23,16 @@ config GRANT_TABLE If unsure, say Y. +config HAS_PDX_COMPRESSION + bool "PDX (Page inDeX) compression support" + default ARM + help + PDX compression is a technique that allows the hypervisor to + represent physical addresses in a very space-efficient manner. + This is very helpful reducing memory wastage in systems with + memory banks with base addresses far from each other, but carrier + a performance cost. + config ALTERNATIVE_CALL bool diff --git a/xen/common/pdx.c b/xen/common/pdx.c index cc963a3cb3..d0fac9d7c7 100644 --- a/xen/common/pdx.c +++ b/xen/common/pdx.c @@ -31,11 +31,15 @@ unsigned long __read_mostly pdx_group_valid[BITS_TO_LONGS( bool __mfn_valid(unsigned long mfn) { - if ( unlikely(evaluate_nospec(mfn >= max_page)) ) + bool invalid = mfn >= max_page; +#ifdef CONFIG_HAS_PDX_COMPRESSION + invalid |= mfn & pfn_hole_mask; +#endif + + if ( unlikely(evaluate_nospec(invalid)) ) return false; - return likely(!(mfn & pfn_hole_mask)) && - likely(test_bit(pfn_to_pdx(mfn) / PDX_GROUP_COUNT, - pdx_group_valid)); + + return test_bit(pfn_to_pdx(mfn) / PDX_GROUP_COUNT, pdx_group_valid); } void set_pdx_range(unsigned long smfn, unsigned long emfn) @@ -49,6 +53,8 @@ void set_pdx_range(unsigned long smfn, unsigned long emfn) __set_bit(idx, pdx_group_valid); } +#ifdef CONFIG_HAS_PDX_COMPRESSION + /* * Diagram to make sense of the following variables. The masks and shifts * are done on mfn values in order to convert to/from pdx: @@ -178,6 +184,7 @@ void __init pfn_pdx_hole_setup(unsigned long mask) pfn_top_mask = ~(pfn_pdx_bottom_mask | pfn_hole_mask); ma_top_mask = pfn_top_mask << PAGE_SHIFT; } +#endif /* CONFIG_HAS_PDX_COMPRESSION */ /* diff --git a/xen/include/xen/pdx.h b/xen/include/xen/pdx.h index ce27177b56..5531890d1c 100644 --- a/xen/include/xen/pdx.h +++ b/xen/include/xen/pdx.h @@ -98,6 +98,8 @@ bool __mfn_valid(unsigned long mfn); #define mfn_to_pdx(mfn) pfn_to_pdx(mfn_x(mfn)) #define pdx_to_mfn(pdx) _mfn(pdx_to_pfn(pdx)) +#ifdef CONFIG_HAS_PDX_COMPRESSION + extern unsigned long pfn_pdx_bottom_mask, ma_va_bottom_mask; extern unsigned int pfn_pdx_hole_shift; extern unsigned long pfn_hole_mask; @@ -225,7 +227,39 @@ static inline uint64_t directmapoff_to_maddr(unsigned long offset) * position marks a potentially compressible bit. */ void pfn_pdx_hole_setup(unsigned long mask); +#else /* CONFIG_HAS_PDX_COMPRESSION */ + +/* Without PDX compression we can skip some computations */ + +/* pdx<->pfn == identity */ +#define pdx_to_pfn(x) (x) +#define pfn_to_pdx(x) (x) + +/* directmap is indexed by by maddr */ +#define maddr_to_directmapoff(x) (x) +#define directmapoff_to_maddr(x) (x) + +static inline bool pdx_is_region_compressible(unsigned long smfn, + unsigned long emfn) +{ + return true; +} + +static inline uint64_t pdx_init_mask(uint64_t base_addr) +{ + return 0; +} + +static inline uint64_t pdx_region_mask(uint64_t base, uint64_t len) +{ + return 0; +} + +static inline void pfn_pdx_hole_setup(unsigned long mask) +{ +} +#endif /* CONFIG_HAS_PDX_COMPRESSION */ #endif /* __XEN_PDX_H__ */ /*