| Message ID | 20241119135104.595630-1-zengheng4@huawei.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | arm_mpam: Introduce the Narrow-PARTID feature for MPAM driver | expand |
Hi, On Tue, Nov 19, 2024 at 09:50:58PM +0800, Zeng Heng wrote: > The patch set is applied for mpam/snapshot/v6.12-rc1 branch of > https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git > repository. > > This patch set is fully compatible with x86 RDT functionality. > > The narrow-partid feature in MPAM allows for a more efficient use of > PARTIDs by enabling a many-to-one mapping of reqpartids (requested PARTIDs) > to intpartids (internal PARTIDs). This mapping reduces the number of unique > PARTIDs needed, thus allowing more tasks or processes to be monitored and > managed with the available resources. > > Intpartid(Internal PARTID) is an internal identifier used by the hardware > to represent a specific resource partition. It is a low-level identifier > that the hardware uses to track and manage resource allocation and > monitoring. > > Reqpartid(Request PARTID) is an identifier provided by the software when > requesting resources from the memory system. It indicates the desired > partition for resource monitoring. By using reqpartids, software can > monitor specific resources or allow the system to subdivide smaller > granularity partitions within existing partitions to serve as monitoring > partitions. > > For the new rmid allocation strategy, it will check whether there is an > available rmid of any reqPARTID which belongs to the input intPARTID. > > The MPAM driver statically assigns all reqPARTIDs to respective intPARTIDs, > with a specific illustration as follows: > > m - Indicates the number of reqPARTIDs per intPARTID > n - Indicates the total number of intPARTIDs > (m * n) - Represents the total number of reqPARTIDs > > intPARTID_1 = 0 > ├── reqPARTID_1_1 = 0 > ├── reqPARTID_1_2 = 0 + n > ├── ... > └── reqPARTID_1_m = 0 + n * (m - 1) > > intPARTID_2 = 1 > ├── reqPARTID_2_1 = 1 > ├── reqPARTID_2_2 = 1 + n > ├── ... > └── reqPARTID_2_m = 1 + n * (m - 1) > > ... > > intPARTID_n = (n - 1) > > Each intPARTID has m reqPARTIDs, which are used to expand the number of > monitoring groups under the control group. Therefore, the number of > monitoring groups is no longer limited by the range of MPAM PMG, which > enhances the extensibility of the system's monitoring capabilities. The idea of mapping multiple reqPARTIDs to each resctrl control group looks like it can work, but I think that there are some issues that need to be considered: 1) There may be a mixture of MSCs in the system, some of which support PARTID Narrowing and some of which do not. Affected MSCs will not be able to regulate resource consumption for a single resctrl control group as a single unit, if multiple reqPARTIDs are used. This matters when an MSC that does not support PARTID Narrowing also has resource controls that are not of the "partition bitmap" type. (Consider a resctrl control partition that throttles the partition to 30% of memory bandwidth. How can the same behaviour be achieved if the partition is split arbitrarily across multiple reqPARTIDs?) Because the MPAM driver needs to be as general as possible, it may be hard to make the "right" decision about whether to group reqPARTIDs to provide more monitoring groups. because different use cases may have different requirments (e.g., number of control groups versus number of monitoring groups, and which types of control are useful). 2) The resctrl core code uses CLOSIDs and RMIDs to identify control groups and monitoring groups. If a particular driver wants to translate these into other values (reqPARTID, intPARTID, PMG) then it can do so, but this mapping logic should be encapsulated in the driver. This should be better for maintainability, since the details of the remapping will be arch-specific -- and in general not all arches are going to require it. With this in mind, I think that changes in the resctrl core code would be minimal (perhaps no changes at all). 3) How should the amount of reqPARTID grouping (your "n" parameter) be determined, in general? As with (1), the right answer may depend on the use case as well as on the hardware. From my investigations into this, I feel that some configuration parameters will probably be needed, at least at boot time. 4) If the mapping between reqPARTIDs and (CLOSID,RMID) pairs is static, is it necessary to track which reqPARTIDs are in use? Would it be simpler to treat all n reqPARTIDs as permanently assigned to the corresponding CLOSID? If reqPARTID usage is not tracked, then every control change on MSCs that do not support PARTID Narrowing would need to be replicated across all reqPARTIDs corresponding to the affected resctrl control partition. But control changes are a relatively rare event, so this approach feels acceptable as a way of keeping the driver complexity down. It partly depends on how large the "n" parameter can become. (Since PARTID Narrowing allows any arbitrary set of reqPARTIDs to be mapped to a given intPARTID, it might be possible to allocate reqPARTIDs completely dynamically. But this probably would require a change to the resctrl core interface. It is also not clear to me whether the "num_closids" and "num_rmids" values advertised to userspace can be satisfied. For now, static allocation seems the most straightforward way to to get better monitoring, but perhaps it could be enhanced later on.) [...] Cheers ---Dave
The patch set is applied for mpam/snapshot/v6.12-rc1 branch of https://git.kernel.org/pub/scm/linux/kernel/git/morse/linux.git repository. This patch set is fully compatible with x86 RDT functionality. The narrow-partid feature in MPAM allows for a more efficient use of PARTIDs by enabling a many-to-one mapping of reqpartids (requested PARTIDs) to intpartids (internal PARTIDs). This mapping reduces the number of unique PARTIDs needed, thus allowing more tasks or processes to be monitored and managed with the available resources. Intpartid(Internal PARTID) is an internal identifier used by the hardware to represent a specific resource partition. It is a low-level identifier that the hardware uses to track and manage resource allocation and monitoring. Reqpartid(Request PARTID) is an identifier provided by the software when requesting resources from the memory system. It indicates the desired partition for resource monitoring. By using reqpartids, software can monitor specific resources or allow the system to subdivide smaller granularity partitions within existing partitions to serve as monitoring partitions. For the new rmid allocation strategy, it will check whether there is an available rmid of any reqPARTID which belongs to the input intPARTID. The MPAM driver statically assigns all reqPARTIDs to respective intPARTIDs, with a specific illustration as follows: m - Indicates the number of reqPARTIDs per intPARTID n - Indicates the total number of intPARTIDs (m * n) - Represents the total number of reqPARTIDs intPARTID_1 = 0 ├── reqPARTID_1_1 = 0 ├── reqPARTID_1_2 = 0 + n ├── ... └── reqPARTID_1_m = 0 + n * (m - 1) intPARTID_2 = 1 ├── reqPARTID_2_1 = 1 ├── reqPARTID_2_2 = 1 + n ├── ... └── reqPARTID_2_m = 1 + n * (m - 1) ... intPARTID_n = (n - 1) Each intPARTID has m reqPARTIDs, which are used to expand the number of monitoring groups under the control group. Therefore, the number of monitoring groups is no longer limited by the range of MPAM PMG, which enhances the extensibility of the system's monitoring capabilities. --- compared with v1: - Rebase this patch set on latest MPAM driver of the v6.12-rc1 branch. --- Dave Martin (1): arm_mpam: Set INTERNAL as needed when setting MSC controls Zeng Heng (5): arm_mpam: Introduce the definitions of intPARTID and reqPARTID arm_mpam: Create reqPARTIDs resource bitmap arm_mpam: Enhance the rmid allocation strategy arm_mpam: Call resctrl_sync_config() when allocate new reqPARTID fs/resctrl: Add the helper to check if the task exists in the target group arch/x86/kernel/cpu/resctrl/core.c | 20 +++ drivers/platform/arm64/mpam/mpam_devices.c | 80 +++++++++-- drivers/platform/arm64/mpam/mpam_internal.h | 6 + drivers/platform/arm64/mpam/mpam_resctrl.c | 145 +++++++++++++++++++- fs/resctrl/internal.h | 4 - fs/resctrl/monitor.c | 16 ++- fs/resctrl/pseudo_lock.c | 7 +- fs/resctrl/rdtgroup.c | 84 ++++++++---- include/linux/resctrl.h | 30 ++++ 9 files changed, 342 insertions(+), 50 deletions(-) -- 2.25.1