From patchwork Thu Nov 14 13:50:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875126 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 00374D68B18 for ; Thu, 14 Nov 2024 13:45:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JNrlvvL21ME0NOO4edlkQNtP9UaBe8f68aKe8AvVql8=; b=OPr2A9z60Jex8/Uj8N+Je8P5e0 OGRAOkNg5hw/JxDmLRTfhqKwSKHWGpvRJeFQk8OxcOJFLvW2UN5pWdQxzqZRwKccDqaHuglI3YG+t MlrD3PNqx4obUE6T2bcPeh3oNiXdi9QCYNrv3lTWhPqGbr++3arP6PrNeeI6Ln/TwguzHlFVTrTbG yydnOQGK3ao+LIC4JOLrjN+MD7+JV7b/euO3Pcx7PnWsR8MT3Y3aL9H/kNuWR5zGGCEBecdGZ2MHI s+LTlM0NIg/NXkq4GVTYh2NvogaqREw+gGSoe8wDlOsJ0+bSGts1qcNDuH7T3hIqKw2ozmeGEy8G5 xrgSTtwQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBaA8-0000000A6M8-11e3; Thu, 14 Nov 2024 13:45:32 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50H-3Zzt for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:52 +0000 Received: from mail.maildlp.com (unknown [172.19.163.174]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Xq1Np5tbYz10RqK; Thu, 14 Nov 2024 21:35:06 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id A97D9140157; Thu, 14 Nov 2024 21:37:37 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:36 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 1/6] arm_mpam: Set INTERNAL as needed when setting MSC controls Date: Thu, 14 Nov 2024 21:50:32 +0800 Message-ID: <20241114135037.918470-2-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053749_294171_2D4881F6 X-CRM114-Status: GOOD ( 13.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Dave Martin Currently, when an MSC implements PARTID narrowing, MPAMCFG_PART_SEL is left set to the reqPARTID while while programming resource controls in an MSC. The MPAM architecture does not guarantee that any particular resource controls will be updated correctly in this scenario. Instead, MPAMCFG_PART_SEL must be written with the corresponding intPARTID and with the INTERNAL bit set before attempting to program resource controls. Only the PARTID->intPARTID mappings can be written without the INTERNAL bit set in MPAMCFG_PART_SEL. Fix it, by rewriting MPAMCFG_PART_SEL appropriately after setting the intPARTID mapping. The MPAMCFG_INTPARTID_INTERNAL flag is not currently accepted as input to the __mpam_part_sel() helper. In the interest of keeping the code clean, break this helper up. Signed-off-by: Dave Martin Signed-off-by: James Morse --- drivers/platform/arm64/mpam/mpam_devices.c | 26 +++++++++++++++++----- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c index 954b0017eda8..e7ab1bfbf831 100644 --- a/drivers/platform/arm64/mpam/mpam_devices.c +++ b/drivers/platform/arm64/mpam/mpam_devices.c @@ -204,15 +204,27 @@ static u64 mpam_msc_read_esr(struct mpam_msc *msc) return (esr_high << 32) | esr_low; } +static void __mpam_part_sel_raw(u32 partsel, struct mpam_msc *msc) +{ + lockdep_assert_held(&msc->part_sel_lock); + mpam_write_partsel_reg(msc, PART_SEL, partsel); +} + static void __mpam_part_sel(u8 ris_idx, u16 partid, struct mpam_msc *msc) { - u32 partsel; + u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) | + FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid); - lockdep_assert_held(&msc->part_sel_lock); + __mpam_part_sel_raw(partsel, msc); +} - partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) | - FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, partid); - mpam_write_partsel_reg(msc, PART_SEL, partsel); +static void __mpam_intpart_sel(u8 ris_idx, u16 intpartid, struct mpam_msc *msc) +{ + u32 partsel = FIELD_PREP(MPAMCFG_PART_SEL_RIS, ris_idx) | + FIELD_PREP(MPAMCFG_PART_SEL_PARTID_SEL, intpartid) | + MPAMCFG_PART_SEL_INTERNAL; + + __mpam_part_sel_raw(partsel, msc); } int mpam_register_requestor(u16 partid_max, u8 pmg_max) @@ -1529,9 +1541,11 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid, mutex_lock(&msc->part_sel_lock); __mpam_part_sel(ris->ris_idx, partid, msc); - if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) + if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) { mpam_write_partsel_reg(msc, INTPARTID, MPAMCFG_INTPARTID_INTERNAL | partid); + __mpam_intpart_sel(ris->ris_idx, partid, msc); + } if (mpam_has_feature(mpam_feat_cpor_part, rprops)) { if (mpam_has_feature(mpam_feat_cpor_part, cfg)) From patchwork Thu Nov 14 13:50:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 75201D68B17 for ; Thu, 14 Nov 2024 13:43:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CJT73PtWLn1yoVL3zzJAz0QiSmoYcqtUjlF9S1hLSos=; b=MAiH4Kbkl68gngq2MT8YzVN6FG H9APq87cK88A9UaKzee9Gl/2ZJcZGw9mMdnrTIB+jbI+ymVFDhxd/KRW19eXkT+QwnchlFvI8MF2k J3+pY3mRd/ngWskrei0xo6zQUF2DTiE/2wHniMlTXkrRLXecZDeI7PGQ3qMPcdR89s4jfree/rAW6 4sZSe3025tleVHVLT92j0iqmkV914broM8v+7AgBldDuPgJAJoV8vlGea+UvLbsh8thnmO1jUHG8B H7Jc4z/Ro4T8uUsGSETiTqA6vYGeIabb+/65/8lxk1mLhQ9tHE0s7k1DfeMude8NiBOAO8OwMcvHO mZU8qeoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBa8J-0000000A65q-2klZ; Thu, 14 Nov 2024 13:43:39 +0000 Received: from szxga01-in.huawei.com ([45.249.212.187]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50C-2A50 for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:52 +0000 Received: from mail.maildlp.com (unknown [172.19.163.252]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Xq1PV5zXhz10Vxd; Thu, 14 Nov 2024 21:35:42 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id 53F651800A6; Thu, 14 Nov 2024 21:37:38 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:37 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 2/6] arm_mpam: Introduce the definitions of intPARTID and reqPARTID Date: Thu, 14 Nov 2024 21:50:33 +0800 Message-ID: <20241114135037.918470-3-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053748_991226_3BF9C654 X-CRM114-Status: GOOD ( 20.61 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The narrow-partid feature in MPAM allows for a more efficient use of PARTIDs by enabling a many-to-one mapping of reqpartids (requested PARTIDs) to intpartids (internal PARTIDs). This mapping reduces the number of unique PARTIDs needed, thus allowing more tasks or processes to be monitored and managed with the available resources. Intpartid(Internal PARTID) is an internal identifier used by the hardware to represent a specific resource partition. It is a low-level identifier that the hardware uses to track and manage resource allocation and monitoring. Reqpartid(Request PARTID) is an identifier provided by the software when requesting resources from the memory system. It indicates the desired partition for resource monitoring. By using reqpartids, software can monitor specific resources or allow the system to subdivide smaller granularity partitions within existing partitions to serve as monitoring partitions. Regarding intPARTID, MPAM uses it as the unit for control group configuration delivery. MPAM will synchronize the delivered configuration to all reqPARTIDs mapped to the same intPARTIDs. The number of intPARTIDs is indicated by MPAMF_PARTID_NRW_IDR.INTPARTID_MAX if implemented, or directly use the number of PARTID as intpartid_max. reqPARTIDs can be used to expand the number of monitors, for each control group is no longer simply restricted by the range of PMG. By mapping between intPARTID and reqPARTID, the number of monitors would be greatly expanded and more fine-grained monitoring under a control group will be achieved. After initialization, MPAM driver would select the minimum intpartid_max among all classes of MSCs as the number of CLOSIDs, and use partid_max as the number of reqPARTIDs. Signed-off-by: Zeng Heng --- drivers/platform/arm64/mpam/mpam_devices.c | 12 +++++++++++- drivers/platform/arm64/mpam/mpam_internal.h | 2 ++ drivers/platform/arm64/mpam/mpam_resctrl.c | 9 +++++++-- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c index e7ab1bfbf831..1bcdeb847a22 100644 --- a/drivers/platform/arm64/mpam/mpam_devices.c +++ b/drivers/platform/arm64/mpam/mpam_devices.c @@ -67,6 +67,7 @@ static DEFINE_MUTEX(mpam_cpuhp_state_lock); * Generating traffic outside this range will result in screaming interrupts. */ u16 mpam_partid_max; +u16 mpam_intpartid_max; u8 mpam_pmg_max; static bool partid_max_init, partid_max_published; static DEFINE_SPINLOCK(partid_max_lock); @@ -234,10 +235,16 @@ int mpam_register_requestor(u16 partid_max, u8 pmg_max) spin_lock(&partid_max_lock); if (!partid_max_init) { mpam_partid_max = partid_max; + /* + * Update mpam_intpartid_max here, in case the + * system doesn't have narrow-partid feature. + */ + mpam_intpartid_max = partid_max; mpam_pmg_max = pmg_max; partid_max_init = true; } else if (!partid_max_published) { mpam_partid_max = min(mpam_partid_max, partid_max); + mpam_intpartid_max = min(mpam_intpartid_max, partid_max); mpam_pmg_max = min(mpam_pmg_max, pmg_max); } else { /* New requestors can't lower the values */ @@ -988,7 +995,9 @@ static void mpam_ris_hw_probe(struct mpam_msc_ris *ris) u16 partid_max = FIELD_GET(MPAMF_PARTID_NRW_IDR_INTPARTID_MAX, nrwidr); mpam_set_feature(mpam_feat_partid_nrw, props); - msc->partid_max = min(msc->partid_max, partid_max); + msc->intpartid_max = min(msc->partid_max, partid_max); + } else { + msc->intpartid_max = msc->partid_max; } mpam_mon_sel_outer_unlock(msc); @@ -1050,6 +1059,7 @@ static int mpam_msc_hw_probe(struct mpam_msc *msc) spin_lock(&partid_max_lock); mpam_partid_max = min(mpam_partid_max, msc->partid_max); + mpam_intpartid_max = min(mpam_intpartid_max, msc->intpartid_max); mpam_pmg_max = min(mpam_pmg_max, msc->pmg_max); spin_unlock(&partid_max_lock); diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h index a7113ba12a22..db1d990bc42b 100644 --- a/drivers/platform/arm64/mpam/mpam_internal.h +++ b/drivers/platform/arm64/mpam/mpam_internal.h @@ -80,6 +80,7 @@ struct mpam_msc bool error_irq_requested; bool error_irq_hw_enabled; u16 partid_max; + u16 intpartid_max; u8 pmg_max; unsigned long ris_idxs[128 / BITS_PER_LONG]; u32 ris_max; @@ -455,6 +456,7 @@ static inline void mpam_assert_srcu_read_lock_held(void) /* System wide partid/pmg values */ extern u16 mpam_partid_max; +extern u16 mpam_intpartid_max; extern u8 mpam_pmg_max; /* Scheduled work callback to enable mpam once all MSC have been probed */ diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c index cc50337e10b0..a74133d7b402 100644 --- a/drivers/platform/arm64/mpam/mpam_resctrl.c +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c @@ -162,6 +162,11 @@ static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid) * only the system wide safe value is safe to use. */ u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored) +{ + return mpam_intpartid_max + 1; +} + +static u32 get_num_reqpartid(void) { return mpam_partid_max + 1; } @@ -169,9 +174,9 @@ u32 resctrl_arch_get_num_closid(struct rdt_resource *ignored) u32 resctrl_arch_system_num_rmid_idx(void) { u8 closid_shift = fls(mpam_pmg_max); - u32 num_partid = resctrl_arch_get_num_closid(NULL); + u32 num_reqpartid = get_num_reqpartid(); - return num_partid << closid_shift; + return num_reqpartid << closid_shift; } u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) From patchwork Thu Nov 14 13:50:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B134CD68B17 for ; Thu, 14 Nov 2024 13:40:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=fx2qbIVQCfquylI0hSKSiIRssUet/G2LCF/5OEUppKI=; b=0TLiW4J9qg4eLes6mw2lby/Kk7 QRvsZJaCjrGQ9KiYx5uiC9te/JbM2+LLOP0eYnkZawr6YTRCVY+6VBbE41lDz8HtktnD1T6tFn76o cr+Gq1gj0CKYyXb546DOsOMG1/pYMB9v39k3AGfQuo08TimMmNyD7GTNRvtLDYSTQAOoec3JTVlao C/cOOciKgWG3H20gYZEy8LtpPsJ0C0E7dBs2ISrF5NrVDJsHVFmhxPmCJVAZwkSAIB/Xhe0AdgqxG Q03vZAlq3WMQdn9FrjxNRP8fAD9XHHsxB+Sv6HNcJY1RJhe2fkZCb7wglnupsoo10nJiojir0cx7U 2nB0AyJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBa4c-0000000A5QA-1IgY; Thu, 14 Nov 2024 13:39:50 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50E-2quD for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:50 +0000 Received: from mail.maildlp.com (unknown [172.19.163.48]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Xq1PZ3H22zpb0m; Thu, 14 Nov 2024 21:35:46 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id EA53E18005F; Thu, 14 Nov 2024 21:37:38 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:38 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 3/6] arm_mpam: Create reqPARTIDs resource bitmap Date: Thu, 14 Nov 2024 21:50:34 +0800 Message-ID: <20241114135037.918470-4-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053748_897396_12EA562C X-CRM114-Status: GOOD ( 13.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The driver checks whether mpam_partid_max and mpam_intpartid_max are equal as a basis for supporting reqPARTID. If this feature is supported, use a bitmap to represent whether the target reqPARTID is available or not. Create the bitmap during monitor initialization, and the destructor is called during the monitor exit process. It is noted that the reqpartid_free_map reserves the first reqPARTID under each intPARTID (which is equal to the corresponding intPARTID itself). By default, assigns it to the corresponding control group for use in monitoring. Signed-off-by: Zeng Heng --- drivers/platform/arm64/mpam/mpam_resctrl.c | 33 ++++++++++++++++++++++ 1 file changed, 33 insertions(+) diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c index a74133d7b402..01ad6732eac5 100644 --- a/drivers/platform/arm64/mpam/mpam_resctrl.c +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c @@ -1000,6 +1000,33 @@ static void mpam_resctrl_monitor_init(struct mpam_class *class, return; } +static unsigned long *reqpartid_free_map; +static int reqpartid_free_map_len; + +static int reqpartid_create(void) +{ + u32 reqpartid_num = get_num_reqpartid(); + int i; + + reqpartid_free_map = bitmap_alloc(reqpartid_num, GFP_KERNEL); + if (!reqpartid_free_map) + return -ENOMEM; + + bitmap_fill(reqpartid_free_map, reqpartid_num); + + /* Reserved for the internal partIDs mapping */ + for (i = 0; i < resctrl_arch_get_num_closid(NULL); i++) + __clear_bit(i, reqpartid_free_map); + + reqpartid_free_map_len = reqpartid_num; + return 0; +} + +static void reqpartid_destroy(void) +{ + bitmap_free(reqpartid_free_map); +} + int mpam_resctrl_setup(void) { int err = 0; @@ -1047,6 +1074,10 @@ int mpam_resctrl_setup(void) cpus_read_unlock(); + err = reqpartid_create(); + if (err) + return err; + if (!err && !exposed_alloc_capable && !exposed_mon_capable) err = -EOPNOTSUPP; @@ -1075,6 +1106,8 @@ static void mpam_resctrl_exit(void) WRITE_ONCE(resctrl_enabled, false); resctrl_exit(); + + reqpartid_destroy(); } From patchwork Thu Nov 14 13:50:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875127 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7256FD68B18 for ; Thu, 14 Nov 2024 13:47:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Transfer-Encoding: Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC: To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=lwPl4iyoLfoOymnj7cnTHI3FlKS+nqEWFjuAsdT1MRo=; b=d23zxQ7aeOEulrCqEFP8BxVblg H+eonQhPu235zn0tD+GpuuTlS414PV8Nn+hyWIzHmSi1IUHhWjdzvZj28HFuo/qoLco/D1O9M/c3Q QJKt+6osAduY0LWXeFh0E/gffPLDeKZ+2tUkoT2ydlYBjpY1WgLABp1IMnhB6wPPI9BccVA8B9XRu wqv+wjvs3bKLcY8UnL98Hm/kRdRBOpMDDzLIYtM3KL9TdRbs/oH99pqv5U+ZcnY5H1IGhPwygfhOo IOPUweEL7lopVf36J19HbG9xACuG4fpqzgJ93qKkmUbTIM25+tosySydn+hzDS01IUy6EQcVmpQoB s2xdYYlQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBaBv-0000000A6Yg-00fo; Thu, 14 Nov 2024 13:47:23 +0000 Received: from szxga02-in.huawei.com ([45.249.212.188]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50A-2wLe for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:53 +0000 Received: from mail.maildlp.com (unknown [172.19.88.194]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Xq1Nf6ZwbzDrDW; Thu, 14 Nov 2024 21:34:58 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id 8B4471400FF; Thu, 14 Nov 2024 21:37:39 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:38 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 4/6] arm_mpam: Enhance the rmid allocation strategy Date: Thu, 14 Nov 2024 21:50:35 +0800 Message-ID: <20241114135037.918470-5-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053749_150728_F773E06F X-CRM114-Status: GOOD ( 18.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org resctrl_arch_alloc_rmid() provides the similar functionality as alloc_rmid(), but it is an enhanced version based on different architecture implementations. For the new rmid allocation strategy, it will check whether there is an available rmid of any reqPARTID which belongs to the input intPARTID. The MPAM driver statically assigns all reqPARTIDs to respective intPARTIDs, with a specific illustration as follows: m - Indicates the number of reqPARTIDs per intPARTID n - Indicates the total number of intPARTIDs (m * n) - Represents the total number of reqPARTIDs intPARTID_1 = 0 ├── reqPARTID_1_1 = 0 ├── reqPARTID_1_2 = 0 + n ├── ... └── reqPARTID_1_m = 0 + n * (m - 1) intPARTID_2 = 1 ├── reqPARTID_2_1 = 1 ├── reqPARTID_2_2 = 1 + n ├── ... └── reqPARTID_2_m = 1 + n * (m - 1) ... intPARTID_n = (n - 1) Each intPARTID has m reqPARTIDs, which are used to expand the number of monitoring groups under the control group. Therefore, the number of monitoring groups is no longer limited by the range of MPAM PMG, which enhances the extensibility of the system's monitoring capabilities. Signed-off-by: Zeng Heng --- arch/x86/kernel/cpu/resctrl/core.c | 15 ++++++++ drivers/platform/arm64/mpam/mpam_resctrl.c | 41 ++++++++++++++++++++-- fs/resctrl/internal.h | 2 -- fs/resctrl/pseudo_lock.c | 3 +- fs/resctrl/rdtgroup.c | 3 +- include/linux/resctrl.h | 16 +++++++++ 6 files changed, 71 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index b8aa6faf100a..60889ae61bfc 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -693,6 +693,21 @@ static void domain_remove_cpu(int cpu, struct rdt_resource *r) domain_remove_cpu_mon(cpu, r); } +int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid) +{ + int ret; + + if (!closid || !rmid) + return -EINVAL; + + ret = alloc_rmid(*closid); + if (ret < 0) + return ret; + + *rmid = ret; + return 0; +} + static void clear_closid_rmid(int cpu) { struct resctrl_pqr_state *state = this_cpu_ptr(&pqr_state); diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c index 01ad6732eac5..ecb3438379bc 100644 --- a/drivers/platform/arm64/mpam/mpam_resctrl.c +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c @@ -171,6 +171,11 @@ static u32 get_num_reqpartid(void) return mpam_partid_max + 1; } +static u32 get_num_reqpartid_per_closid(void) +{ + return get_num_reqpartid() / resctrl_arch_get_num_closid(NULL); +} + u32 resctrl_arch_system_num_rmid_idx(void) { u8 closid_shift = fls(mpam_pmg_max); @@ -182,10 +187,11 @@ u32 resctrl_arch_system_num_rmid_idx(void) u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) { u8 closid_shift = fls(mpam_pmg_max); + u32 pmg_mask = ~(~0 << closid_shift); BUG_ON(closid_shift > 8); - return (closid << closid_shift) | rmid; + return (closid << closid_shift) | (rmid & pmg_mask); } void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid) @@ -195,8 +201,10 @@ void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid, u32 *rmid) BUG_ON(closid_shift > 8); - *closid = idx >> closid_shift; - *rmid = idx & pmg_mask; + if (closid) + *closid = idx >> closid_shift; + if (rmid) + *rmid = idx & pmg_mask; } void resctrl_arch_sched_in(struct task_struct *tsk) @@ -1027,6 +1035,33 @@ static void reqpartid_destroy(void) bitmap_free(reqpartid_free_map); } +int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid) +{ + int closid_num = resctrl_arch_get_num_closid(NULL); + int i, reqpartid, pmg; + + if (!closid || !rmid) + return -EINVAL; + + /* The closid is out of the range of intPARTIDs */ + if (*closid >= closid_num) + return -EINVAL; + + for (i = 0; i < get_num_reqpartid_per_closid(); i++) { + reqpartid = i * closid_num + *closid; + pmg = alloc_rmid(reqpartid); + if (pmg >= 0) + break; + } + + if (pmg < 0) + return pmg; + + *closid = reqpartid; + *rmid = pmg; + return 0; +} + int mpam_resctrl_setup(void) { int err = 0; diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h index 4298d45d1851..08cf21186448 100644 --- a/fs/resctrl/internal.h +++ b/fs/resctrl/internal.h @@ -325,8 +325,6 @@ int closids_supported(void); void closid_free(int closid); -int alloc_rmid(u32 closid); - void free_rmid(u32 closid, u32 rmid); void resctrl_mon_resource_exit(void); diff --git a/fs/resctrl/pseudo_lock.c b/fs/resctrl/pseudo_lock.c index ef2a277764f9..11532c5a5fed 100644 --- a/fs/resctrl/pseudo_lock.c +++ b/fs/resctrl/pseudo_lock.c @@ -582,12 +582,11 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) int ret; if (resctrl_arch_mon_capable()) { - ret = alloc_rmid(rdtgrp->closid); + ret = resctrl_arch_alloc_rmid(&rdtgrp->closid, &rdtgrp->mon.rmid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; } - rdtgrp->mon.rmid = ret; } ret = rdtgroup_locksetup_user_restore(rdtgrp); diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c index 7f7586af8def..b5748fce933f 100644 --- a/fs/resctrl/rdtgroup.c +++ b/fs/resctrl/rdtgroup.c @@ -3280,12 +3280,11 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) if (!resctrl_arch_mon_capable()) return 0; - ret = alloc_rmid(rdtgrp->closid); + ret = resctrl_arch_alloc_rmid(&rdtgrp->closid, &rdtgrp->mon.rmid); if (ret < 0) { rdt_last_cmd_puts("Out of RMIDs\n"); return ret; } - rdtgrp->mon.rmid = ret; ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); if (ret) { diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index a1a96d928c37..ec679cfad5fb 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -510,6 +510,22 @@ static inline void resctrl_arch_rmid_read_context_check(void) might_sleep(); } +int alloc_rmid(u32 closid); +/** + * resctrl_arch_alloc_rmid() - Providing the similar functionality as + * alloc_rmid, but this function is an + * enhanced version based on different + * architecture implementations. + * @closid: closid that matches the rmid. Depending on the + * architecture, may update new closid and return by the + * pointer. + * @rmid: update available rmid by the pointer. + * + * Return: + * 0 on success, or -EINVAL, -ENOSPC etc on error. + */ +int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid); + /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid * and eventid. From patchwork Thu Nov 14 13:50:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875130 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E5BDED68B19 for ; Thu, 14 Nov 2024 13:51:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=vmdmvryAtGdMnFnA0R9JDPhwtiKl7cBkrTUvMBUE3pU=; b=Ak/Qrn9F26fvxSZIrTb6s8GO9v cdWZLCMmW/DKQOY6iMR0llPwobd6MlTWidbBQZ5nie9pww7FJccXejq6eWz60VcP7KiWR73uN/Q7a hgzNiCZ6Df/SdgHzVOGsifYjGxeaHt+hcNyz7TaKZtzWzW+pPhK7iN2erAi3HjsNTkGTkwJndThdT 3DfQdtTIxVjMaDFuYYBzBm02IefG0RCq9fHgXAat4Ony5tV8NUJ3BH4CIwRHz43tsMvDI3Oa/Fk1a BKdIywoZNtJ5QfxR+Y2xeD/oulh2hyiR695/VTLKGXwcJH3daiHT10xc0cskcJjHYrecK4JuygUst C9OHkrVA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBaFV-0000000A760-0TYd; Thu, 14 Nov 2024 13:51:05 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50G-2q4v for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:54 +0000 Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4Xq1Pc5Qvvz2Dgv3; Thu, 14 Nov 2024 21:35:48 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id 34A011401F4; Thu, 14 Nov 2024 21:37:40 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:39 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 5/6] arm_mpam: Call resctrl_sync_config() when allocate new reqPARTID Date: Thu, 14 Nov 2024 21:50:36 +0800 Message-ID: <20241114135037.918470-6-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053749_145651_49B29692 X-CRM114-Status: GOOD ( 29.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Add resctrl_sync_config() for establishing new mapping between intPARTID and reqPARTID for hardware or synchronizing their configuration by software. For the expansion of monitors using reqPARTID in the system: 1. If a MSC class supports the narrow-partid feature, then it is only necessary to associate the newly allocated reqPARTID with its corresponding intPARTID. 2. If a MSC class does not support the narrow-partid feature, the hardware can't associate reqPARTID with intPARTID, and the software needs to synchronize the configuration of the parent control group to the new PARTID (the newly allocated "reqpartid"). Additionally, for MSC class that doesn't support the narrow-partid feature, when the control group configuration is updated, the software needs to forcibly update its sub-monitoring groups (referencing the __write_config()). We establish a reqPARTID resource bitmap and check whether the reqPARTID is in use by reqpartid_is_busy() to reduce unnecessary I/O operations. In mpam_apply_config(), if closid is equal to intPARTID, we regard that it is updating new configuration for the control group. Otherwise, it is attempting to establish mapping and synchronize configuration for sub-monitoring groups. Signed-off-by: Zeng Heng --- arch/x86/kernel/cpu/resctrl/core.c | 5 ++ drivers/platform/arm64/mpam/mpam_devices.c | 47 +++++++++++++-- drivers/platform/arm64/mpam/mpam_internal.h | 4 ++ drivers/platform/arm64/mpam/mpam_resctrl.c | 66 ++++++++++++++++++++- fs/resctrl/internal.h | 2 - fs/resctrl/monitor.c | 16 ++++- fs/resctrl/pseudo_lock.c | 4 +- fs/resctrl/rdtgroup.c | 29 +++++++-- include/linux/resctrl.h | 14 +++++ 9 files changed, 170 insertions(+), 17 deletions(-) diff --git a/arch/x86/kernel/cpu/resctrl/core.c b/arch/x86/kernel/cpu/resctrl/core.c index 60889ae61bfc..1e8f46203bc4 100644 --- a/arch/x86/kernel/cpu/resctrl/core.c +++ b/arch/x86/kernel/cpu/resctrl/core.c @@ -708,6 +708,11 @@ int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid) return 0; } +void resctrl_arch_free_rmid(u32 closid, u32 rmid) +{ + free_rmid(closid, rmid); +} + static void clear_closid_rmid(int cpu) { struct resctrl_pqr_state *state = this_cpu_ptr(&pqr_state); diff --git a/drivers/platform/arm64/mpam/mpam_devices.c b/drivers/platform/arm64/mpam/mpam_devices.c index 1bcdeb847a22..23b375753372 100644 --- a/drivers/platform/arm64/mpam/mpam_devices.c +++ b/drivers/platform/arm64/mpam/mpam_devices.c @@ -1543,6 +1543,7 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid, u32 pri_val = 0; u16 cmax = MPAMCFG_CMAX_CMAX; u16 bwa_fract = MPAMCFG_MBW_MAX_MAX; + u16 intpartid = req2intpartid(partid); struct mpam_msc *msc = ris->vmsc->msc; struct mpam_props *rprops = &ris->props; u16 dspri = GENMASK(rprops->dspri_wd, 0); @@ -1553,8 +1554,14 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid, if (mpam_has_feature(mpam_feat_partid_nrw, rprops)) { mpam_write_partsel_reg(msc, INTPARTID, - MPAMCFG_INTPARTID_INTERNAL | partid); - __mpam_intpart_sel(ris->ris_idx, partid, msc); + MPAMCFG_INTPARTID_INTERNAL | + intpartid); + + /* Already finish mapping reqPARTID to intPARTID */ + if (partid != intpartid) + goto out; + + __mpam_intpart_sel(ris->ris_idx, intpartid, msc); } if (mpam_has_feature(mpam_feat_cpor_part, rprops)) { @@ -1611,6 +1618,7 @@ static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid, mpam_quirk_post_config_change(ris, partid, cfg); +out: mutex_unlock(&msc->part_sel_lock); } @@ -2975,10 +2983,34 @@ struct mpam_write_config_arg { static int __write_config(void *arg) { + int closid_num = resctrl_arch_get_num_closid(NULL); struct mpam_write_config_arg *c = arg; + struct mpam_props *rprops; + u32 reqpartid; + int i; mpam_reprogram_ris_partid(c->ris, c->partid, &c->comp->cfg[c->partid]); + if (c->partid != req2intpartid(c->partid)) + return 0; + + rprops = &c->ris->props; + + /* + * Synchronize the configuration to each working sub-monitoring + * group. + */ + for (i = 1; i < get_num_reqpartid_per_closid(); i++) { + reqpartid = i * closid_num + c->partid; + + if (!reqpartid_is_busy(reqpartid)) + continue; + + if (!mpam_has_feature(mpam_feat_partid_nrw, rprops)) + mpam_reprogram_ris_partid(c->ris, reqpartid, + &c->comp->cfg[c->partid]); + } + return 0; } @@ -3030,6 +3062,7 @@ static void mpam_extend_config(struct mpam_class *class, struct mpam_config *cfg int mpam_apply_config(struct mpam_component *comp, u16 partid, struct mpam_config *cfg) { + int intpartid = req2intpartid(partid); struct mpam_write_config_arg arg; struct mpam_msc_ris *ris; struct mpam_vmsc *vmsc; @@ -3040,10 +3073,16 @@ int mpam_apply_config(struct mpam_component *comp, u16 partid, mpam_extend_config(comp->class, cfg); - if (!memcmp(&comp->cfg[partid], cfg, sizeof(*cfg))) + if ((intpartid == partid) && + (!memcmp(&comp->cfg[intpartid], cfg, sizeof(*cfg)))) return 0; - comp->cfg[partid] = *cfg; + /* + * If the partid is not equal to its intpartid, then we continue + * to establish the association between reqpartid and intpartid. + */ + + comp->cfg[intpartid] = *cfg; arg.comp = comp; arg.partid = partid; diff --git a/drivers/platform/arm64/mpam/mpam_internal.h b/drivers/platform/arm64/mpam/mpam_internal.h index db1d990bc42b..ef5ce818464e 100644 --- a/drivers/platform/arm64/mpam/mpam_internal.h +++ b/drivers/platform/arm64/mpam/mpam_internal.h @@ -750,4 +750,8 @@ static inline void mpam_resctrl_teardown_class(struct mpam_class *class) { } */ #define MSMON_CAPT_EVNT_NOW BIT(0) +u32 get_num_reqpartid_per_closid(void); +u32 req2intpartid(u32 reqpartid); +bool reqpartid_is_busy(int reqpartid); + #endif /* MPAM_INTERNAL_H */ diff --git a/drivers/platform/arm64/mpam/mpam_resctrl.c b/drivers/platform/arm64/mpam/mpam_resctrl.c index ecb3438379bc..1f3478a7d965 100644 --- a/drivers/platform/arm64/mpam/mpam_resctrl.c +++ b/drivers/platform/arm64/mpam/mpam_resctrl.c @@ -171,11 +171,16 @@ static u32 get_num_reqpartid(void) return mpam_partid_max + 1; } -static u32 get_num_reqpartid_per_closid(void) +u32 get_num_reqpartid_per_closid(void) { return get_num_reqpartid() / resctrl_arch_get_num_closid(NULL); } +u32 req2intpartid(u32 reqpartid) +{ + return reqpartid % resctrl_arch_get_num_closid(NULL); +} + u32 resctrl_arch_system_num_rmid_idx(void) { u8 closid_shift = fls(mpam_pmg_max); @@ -1035,10 +1040,26 @@ static void reqpartid_destroy(void) bitmap_free(reqpartid_free_map); } +static int reqpartid_alloc(int reqpartid) +{ + __clear_bit(reqpartid, reqpartid_free_map); + return reqpartid; +} + +static void reqpartid_free(int reqpartid) +{ + __set_bit(reqpartid, reqpartid_free_map); +} + +bool reqpartid_is_busy(int reqpartid) +{ + return !test_bit(reqpartid, reqpartid_free_map); +} + int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid) { int closid_num = resctrl_arch_get_num_closid(NULL); - int i, reqpartid, pmg; + int i, ret, reqpartid, pmg; if (!closid || !rmid) return -EINVAL; @@ -1059,7 +1080,48 @@ int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid) *closid = reqpartid; *rmid = pmg; + + if (reqpartid_is_busy(reqpartid)) + return 0; + + ret = reqpartid_alloc(reqpartid); + if (ret < 0) + goto out_free_rmid; + + ret = resctrl_sync_config(reqpartid); + if (ret < 0) + goto out; + return 0; + +out: + reqpartid_free(reqpartid); +out_free_rmid: + free_rmid(reqpartid, pmg); + return ret; +} + +void resctrl_arch_free_rmid(u32 reqpartid, u32 pmg) +{ + int i; + + free_rmid(reqpartid, pmg); + + /* Always reserve the reqPARTID which equals to its intPARTID. */ + if (reqpartid == req2intpartid(reqpartid)) + return; + + /* + * Check whether every RMID belongs to this reqPARTID is freed or + * not. If all belonging rmids are freed, also free the reqPARTID + * self. + */ + for (i = 0; i <= mpam_pmg_max; i++) { + if (rmid_is_busy(reqpartid, i)) + return; + } + + reqpartid_free(reqpartid); } int mpam_resctrl_setup(void) diff --git a/fs/resctrl/internal.h b/fs/resctrl/internal.h index 08cf21186448..ca544143d6a9 100644 --- a/fs/resctrl/internal.h +++ b/fs/resctrl/internal.h @@ -325,8 +325,6 @@ int closids_supported(void); void closid_free(int closid); -void free_rmid(u32 closid, u32 rmid); - void resctrl_mon_resource_exit(void); void mon_event_count(void *info); diff --git a/fs/resctrl/monitor.c b/fs/resctrl/monitor.c index d9041d134d54..8258ceaddd24 100644 --- a/fs/resctrl/monitor.c +++ b/fs/resctrl/monitor.c @@ -282,10 +282,24 @@ int alloc_rmid(u32 closid) if (IS_ERR(entry)) return PTR_ERR(entry); - list_del(&entry->list); + list_del_init(&entry->list); return entry->rmid; } +bool rmid_is_busy(u32 closid, u32 rmid) +{ + u32 idx = resctrl_arch_rmid_idx_encode(closid, rmid); + struct rmid_entry *entry; + + entry = __rmid_entry(idx); + + /* + * If it's not in the rmid_free_lru list, + * the rmid is working. + */ + return list_empty(&entry->list); +} + static void add_rmid_to_limbo(struct rmid_entry *entry) { struct rdt_resource *r = resctrl_arch_get_resource(RDT_RESOURCE_L3); diff --git a/fs/resctrl/pseudo_lock.c b/fs/resctrl/pseudo_lock.c index 11532c5a5fed..69754ddada87 100644 --- a/fs/resctrl/pseudo_lock.c +++ b/fs/resctrl/pseudo_lock.c @@ -557,7 +557,7 @@ int rdtgroup_locksetup_enter(struct rdtgroup *rdtgrp) * anymore when this group would be used for pseudo-locking. This * is safe to call on platforms not capable of monitoring. */ - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); ret = 0; goto out; @@ -591,7 +591,7 @@ int rdtgroup_locksetup_exit(struct rdtgroup *rdtgrp) ret = rdtgroup_locksetup_user_restore(rdtgrp); if (ret) { - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); return ret; } diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c index b5748fce933f..2e8ec20f0415 100644 --- a/fs/resctrl/rdtgroup.c +++ b/fs/resctrl/rdtgroup.c @@ -2754,7 +2754,7 @@ static void free_all_child_rdtgrp(struct rdtgroup *rdtgrp) head = &rdtgrp->mon.crdtgrp_list; list_for_each_entry_safe(sentry, stmp, head, mon.crdtgrp_list) { - free_rmid(sentry->closid, sentry->mon.rmid); + resctrl_arch_free_rmid(sentry->closid, sentry->mon.rmid); list_del(&sentry->mon.crdtgrp_list); if (atomic_read(&sentry->waitcount) != 0) @@ -2794,7 +2794,7 @@ static void rmdir_all_sub(void) cpumask_or(&rdtgroup_default.cpu_mask, &rdtgroup_default.cpu_mask, &rdtgrp->cpu_mask); - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); kernfs_remove(rdtgrp->kn); list_del(&rdtgrp->rdtgroup_list); @@ -3289,7 +3289,7 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) ret = mkdir_mondata_all(rdtgrp->kn, rdtgrp, &rdtgrp->mon.mon_data_kn); if (ret) { rdt_last_cmd_puts("kernfs subdir error\n"); - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); return ret; } @@ -3299,7 +3299,24 @@ static int mkdir_rdt_prepare_rmid_alloc(struct rdtgroup *rdtgrp) static void mkdir_rdt_prepare_rmid_free(struct rdtgroup *rgrp) { if (resctrl_arch_mon_capable()) - free_rmid(rgrp->closid, rgrp->mon.rmid); + resctrl_arch_free_rmid(rgrp->closid, rgrp->mon.rmid); +} + +int resctrl_sync_config(u32 closid) +{ + struct resctrl_schema *s; + int ret; + + list_for_each_entry(s, &resctrl_schema_all, list) { + ret = resctrl_arch_update_domains(s->res, closid); + if (ret < 0) { + rdt_last_cmd_puts("Failed to synchronize partitions\n"); + goto out; + } + } + +out: + return ret; } static int mkdir_rdt_prepare(struct kernfs_node *parent_kn, @@ -3565,7 +3582,7 @@ static int rdtgroup_rmdir_mon(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) update_closid_rmid(tmpmask, NULL); rdtgrp->flags = RDT_DELETED; - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); /* * Remove the rdtgrp from the parent ctrl_mon group's list @@ -3612,7 +3629,7 @@ static int rdtgroup_rmdir_ctrl(struct rdtgroup *rdtgrp, cpumask_var_t tmpmask) cpumask_or(tmpmask, tmpmask, &rdtgrp->cpu_mask); update_closid_rmid(tmpmask, NULL); - free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); + resctrl_arch_free_rmid(rdtgrp->closid, rdtgrp->mon.rmid); closid_free(rdtgrp->closid); rdtgroup_ctrl_remove(rdtgrp); diff --git a/include/linux/resctrl.h b/include/linux/resctrl.h index ec679cfad5fb..ede5963cc5b0 100644 --- a/include/linux/resctrl.h +++ b/include/linux/resctrl.h @@ -511,6 +511,9 @@ static inline void resctrl_arch_rmid_read_context_check(void) } int alloc_rmid(u32 closid); +void free_rmid(u32 closid, u32 rmid); +bool rmid_is_busy(u32 closid, u32 rmid); + /** * resctrl_arch_alloc_rmid() - Providing the similar functionality as * alloc_rmid, but this function is an @@ -525,6 +528,17 @@ int alloc_rmid(u32 closid); * 0 on success, or -EINVAL, -ENOSPC etc on error. */ int resctrl_arch_alloc_rmid(u32 *closid, u32 *rmid); +/** + * resctrl_arch_free_rmid() - Manage rmid and reqpartid resources in + * conjunction with the + * resctrl_arch_alloc_rmid(). + * + * @closid: closid that matches the rmid. + * @rmid: The rmid would be free. + */ +void resctrl_arch_free_rmid(u32 closid, u32 rmid); + +int resctrl_sync_config(u32 closid); /** * resctrl_arch_reset_rmid() - Reset any private state associated with rmid From patchwork Thu Nov 14 13:50:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zeng Heng X-Patchwork-Id: 13875129 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B3BFED68B19 for ; Thu, 14 Nov 2024 13:49:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type: Content-Transfer-Encoding:MIME-Version:References:In-Reply-To:Message-ID:Date :Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=hlukbRcGK6foTZEM9wlyNxrgIpobwxVv71eGziTF4OE=; b=ysD+O8cm2HMr0j9xPadAB8sJLP 1j0+7I8N9wwgqqyKU4W++1mQ69TL08Z3/kLBKrJriWoe+F16knMy5WP3Kd8qiwbb5Um60EAKouwg6 sZ6HMBvCT7pwgamT8seU2mrMy97o3iITiprA8NMy86SAQCYA33cSYfo4jUT/dZclfMw6sae1CFGeB JpeSmF1vLnvotXCQ+lPuSuJOPXk3NAqmKvybAzG3OcSpB2AT1jTDt0n8MklFiRrTlqe/EsCxaPMRg bjJ0MyO9gENhn+xF2dUXOQkC8EA73GjcqAI2n5C0NapNeiBJHtsR1o0JNVgviJYGReX6Nhehzbmmv 01xD9mPA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tBaDi-0000000A6sD-2X0q; Thu, 14 Nov 2024 13:49:14 +0000 Received: from szxga04-in.huawei.com ([45.249.212.190]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tBa2e-0000000A50F-2q0J for linux-arm-kernel@lists.infradead.org; Thu, 14 Nov 2024 13:37:53 +0000 Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4Xq1Pd2rVjz2Dh42; Thu, 14 Nov 2024 21:35:49 +0800 (CST) Received: from kwepemf100008.china.huawei.com (unknown [7.202.181.222]) by mail.maildlp.com (Postfix) with ESMTPS id D3422140153; Thu, 14 Nov 2024 21:37:40 +0800 (CST) Received: from huawei.com (10.175.103.91) by kwepemf100008.china.huawei.com (7.202.181.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Thu, 14 Nov 2024 21:37:40 +0800 From: Zeng Heng To: , CC: , , , Subject: [RFC PATCH mpam mpam/snapshot/v6.11-rc1 6/6] fs/resctrl: Add the helper to check if the task exists in the target group Date: Thu, 14 Nov 2024 21:50:37 +0800 Message-ID: <20241114135037.918470-7-zengheng4@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241114135037.918470-1-zengheng4@huawei.com> References: <20241114135037.918470-1-zengheng4@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.103.91] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemf100008.china.huawei.com (7.202.181.222) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241114_053749_125439_F99D879C X-CRM114-Status: GOOD ( 16.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org After introducing the extension for monitoring feature, to check if a closid exists within the target control group, it is not only check if the closid of the control group is the same, but also to recursively check if there is a closid of sub-monitor groups is the same. Therefore, a new helper task_belongs_to_ctrl_group() is added. On the x86 side, the closid of the child monitor group is the same as its parent control group's (only the rmid is different), hence resctrl_arch_match_rmid() can directly use the closid of the child monitor group. Meanwhile, task_belongs_to_ctrl_group() can replace resctrl_arch_match_closid() to be compatible with the x86 arch. Signed-off-by: Zeng Heng --- fs/resctrl/rdtgroup.c | 52 +++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/fs/resctrl/rdtgroup.c b/fs/resctrl/rdtgroup.c index 2e8ec20f0415..03c5c9eb9298 100644 --- a/fs/resctrl/rdtgroup.c +++ b/fs/resctrl/rdtgroup.c @@ -589,19 +589,38 @@ static void update_task_closid_rmid(struct task_struct *t) static bool task_in_rdtgroup(struct task_struct *tsk, struct rdtgroup *rdtgrp) { - u32 closid, rmid = rdtgrp->mon.rmid; + u32 closid, rmid; - if (rdtgrp->type == RDTCTRL_GROUP) - closid = rdtgrp->closid; - else if (rdtgrp->type == RDTMON_GROUP) - closid = rdtgrp->mon.parent->closid; - else - return false; + closid = rdtgrp->closid; + rmid = rdtgrp->mon.rmid; return resctrl_arch_match_closid(tsk, closid) && resctrl_arch_match_rmid(tsk, closid, rmid); } +/** + * task_belongs_to_ctrl_group - the helper to check if the task exists in + * the target control group. + * @tsk: task to be checked + * @rdtgrp: target control group + */ +static bool task_belongs_to_ctrl_group(struct task_struct *tsk, struct rdtgroup *rdtgrp) +{ + struct rdtgroup *crdtgrp; + + /* Check whether exists in contrl group self */ + if (resctrl_arch_match_closid(tsk, rdtgrp->closid)) + return true; + + /* Check if exists in one of children monitor groups */ + list_for_each_entry(crdtgrp, &rdtgrp->mon.crdtgrp_list, mon.crdtgrp_list) { + if (resctrl_arch_match_closid(tsk, crdtgrp->closid)) + return true; + } + + return false; +} + static int __rdtgroup_move_task(struct task_struct *tsk, struct rdtgroup *rdtgrp) { @@ -618,17 +637,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk, * their parent CTRL group. */ if (rdtgrp->type == RDTMON_GROUP && - !resctrl_arch_match_closid(tsk, rdtgrp->mon.parent->closid)) { + !task_belongs_to_ctrl_group(tsk, rdtgrp->mon.parent)) { rdt_last_cmd_puts("Can't move task to different control group\n"); return -EINVAL; } - if (rdtgrp->type == RDTMON_GROUP) - resctrl_arch_set_closid_rmid(tsk, rdtgrp->mon.parent->closid, - rdtgrp->mon.rmid); - else - resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid, - rdtgrp->mon.rmid); + resctrl_arch_set_closid_rmid(tsk, rdtgrp->closid, + rdtgrp->mon.rmid); /* * Ensure the task's closid and rmid are written before determining if @@ -652,14 +667,13 @@ static int __rdtgroup_move_task(struct task_struct *tsk, static bool is_closid_match(struct task_struct *t, struct rdtgroup *r) { return (resctrl_arch_alloc_capable() && (r->type == RDTCTRL_GROUP) && - resctrl_arch_match_closid(t, r->closid)); + task_belongs_to_ctrl_group(t, r)); } static bool is_rmid_match(struct task_struct *t, struct rdtgroup *r) { return (resctrl_arch_mon_capable() && (r->type == RDTMON_GROUP) && - resctrl_arch_match_rmid(t, r->mon.parent->closid, - r->mon.rmid)); + resctrl_arch_match_rmid(t, r->closid, r->mon.rmid)); } /** @@ -909,7 +923,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, rdtg->mode != RDT_MODE_EXCLUSIVE) continue; - if (!resctrl_arch_match_closid(tsk, rdtg->closid)) + if (!task_belongs_to_ctrl_group(tsk, rdtg)) continue; seq_printf(s, "res:%s%s\n", (rdtg == &rdtgroup_default) ? "/" : "", @@ -917,7 +931,7 @@ int proc_resctrl_show(struct seq_file *s, struct pid_namespace *ns, seq_puts(s, "mon:"); list_for_each_entry(crg, &rdtg->mon.crdtgrp_list, mon.crdtgrp_list) { - if (!resctrl_arch_match_rmid(tsk, crg->mon.parent->closid, + if (!resctrl_arch_match_rmid(tsk, crg->closid, crg->mon.rmid)) continue; seq_printf(s, "%s", crg->kn->name);