From patchwork Sun Sep 1 14:21:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tao Xu X-Patchwork-Id: 11125295 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4D8A3112C for ; Sun, 1 Sep 2019 14:28:06 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1D63C20644 for ; Sun, 1 Sep 2019 14:28:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1D63C20644 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Received: from localhost ([::1]:57782 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i4Qpo-0002iZ-Uh for patchwork-qemu-devel@patchwork.kernel.org; Sun, 01 Sep 2019 10:28:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39580) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1i4Qjk-0004mg-9o for qemu-devel@nongnu.org; Sun, 01 Sep 2019 10:21:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1i4Qjh-0004uE-Rz for qemu-devel@nongnu.org; Sun, 01 Sep 2019 10:21:47 -0400 Received: from mga12.intel.com ([192.55.52.136]:16719) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1i4Qjh-0004jJ-Ax for qemu-devel@nongnu.org; Sun, 01 Sep 2019 10:21:45 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Sep 2019 07:21:45 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,455,1559545200"; d="scan'208";a="211453142" Received: from tao-optiplex-7060.sh.intel.com ([10.239.159.36]) by fmsmga002.fm.intel.com with ESMTP; 01 Sep 2019 07:21:43 -0700 From: Tao Xu To: imammedo@redhat.com, eblake@redhat.com, ehabkost@redhat.com Date: Sun, 1 Sep 2019 22:21:18 +0800 Message-Id: <20190901142119.20482-11-tao3.xu@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190901142119.20482-1-tao3.xu@intel.com> References: <20190901142119.20482-1-tao3.xu@intel.com> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 192.55.52.136 Subject: [Qemu-devel] [PATCH v10 10/11] numa: Extend CLI to provide memory side cache information X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jingqi.liu@intel.com, tao3.xu@intel.com, fan.du@intel.com, qemu-devel@nongnu.org, daniel@linux.ibm.com, jonathan.cameron@huawei.com, dan.j.williams@intel.com Errors-To: qemu-devel-bounces+patchwork-qemu-devel=patchwork.kernel.org@nongnu.org Sender: "Qemu-devel" From: Liu Jingqi Add -numa hmat-cache option to provide Memory Side Cache Information. These memory attributes help to build Memory Side Cache Information Structure(s) in ACPI Heterogeneous Memory Attribute Table (HMAT). Reviewed-by: Daniel Black Signed-off-by: Liu Jingqi Signed-off-by: Tao Xu --- No changes in v10. --- hw/core/numa.c | 67 +++++++++++++++++++++++++++++++++++ include/sysemu/numa.h | 2 ++ qapi/machine.json | 81 +++++++++++++++++++++++++++++++++++++++++-- qemu-options.hx | 16 +++++++-- 4 files changed, 162 insertions(+), 4 deletions(-) diff --git a/hw/core/numa.c b/hw/core/numa.c index 7387547793..6e4f6169cb 100644 --- a/hw/core/numa.c +++ b/hw/core/numa.c @@ -333,6 +333,67 @@ void parse_numa_hmat_lb(MachineState *ms, NumaHmatLBOptions *node, } } +void parse_numa_hmat_cache(MachineState *ms, NumaHmatCacheOptions *node, + Error **errp) +{ + int nb_numa_nodes = ms->numa_state->num_nodes; + HMAT_Cache_Info *hmat_cache = NULL; + + if (node->node_id >= nb_numa_nodes) { + error_setg(errp, "Invalid node-id=%" PRIu32 + ", it should be less than %d.", + node->node_id, nb_numa_nodes); + return; + } + + if (node->total > MAX_HMAT_CACHE_LEVEL) { + error_setg(errp, "Invalid total=%" PRIu8 + ", it should be less than or equal to %d.", + node->total, MAX_HMAT_CACHE_LEVEL); + return; + } + if (node->level > node->total) { + error_setg(errp, "Invalid level=%" PRIu8 + ", it should be less than or equal to" + " total=%" PRIu8 ".", + node->level, node->total); + return; + } + if (ms->numa_state->hmat_cache[node->node_id][node->level]) { + error_setg(errp, "Duplicate configuration of the side cache for " + "node-id=%" PRIu32 " and level=%" PRIu8 ".", + node->node_id, node->level); + return; + } + + if ((node->level > 1) && + ms->numa_state->hmat_cache[node->node_id][node->level - 1] && + (node->size >= + ms->numa_state->hmat_cache[node->node_id][node->level - 1]->size)) { + error_setg(errp, "Invalid size=0x%" PRIx64 + ", the size of level=%" PRIu8 + " should be less than the size(0x%" PRIx64 + ") of level=%" PRIu8 ".", + node->size, node->level, + ms->numa_state->hmat_cache[node->node_id] + [node->level - 1]->size, + node->level - 1); + return; + } + + hmat_cache = g_malloc0(sizeof(*hmat_cache)); + + hmat_cache->mem_proximity = node->node_id; + hmat_cache->size = node->size; + hmat_cache->total_levels = node->total; + hmat_cache->level = node->level; + hmat_cache->associativity = node->assoc; + hmat_cache->write_policy = node->policy; + hmat_cache->line_size = node->line; + + ms->numa_state->hmat_cache[node->node_id][node->level] = hmat_cache; +} + void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp) { Error *err = NULL; @@ -377,6 +438,12 @@ void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp) goto end; } break; + case NUMA_OPTIONS_TYPE_HMAT_CACHE: + parse_numa_hmat_cache(ms, &object->u.hmat_cache, &err); + if (err) { + goto end; + } + break; default: abort(); } diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h index c18e939395..700d7c0392 100644 --- a/include/sysemu/numa.h +++ b/include/sysemu/numa.h @@ -77,6 +77,8 @@ void set_numa_options(MachineState *ms, NumaOptions *object, Error **errp); void parse_numa_opts(MachineState *ms); void parse_numa_hmat_lb(MachineState *ms, NumaHmatLBOptions *node, Error **errp); +void parse_numa_hmat_cache(MachineState *ms, NumaHmatCacheOptions *node, + Error **errp); void numa_complete_configuration(MachineState *ms); void query_numa_node_mem(NumaNodeMem node_mem[], MachineState *ms); extern QemuOptsList qemu_numa_opts; diff --git a/qapi/machine.json b/qapi/machine.json index b6019335e8..088be81920 100644 --- a/qapi/machine.json +++ b/qapi/machine.json @@ -428,10 +428,12 @@ # # @hmat-lb: memory latency and bandwidth information (Since: 4.2) # +# @hmat-cache: memory side cache information (Since: 4.2) +# # Since: 2.1 ## { 'enum': 'NumaOptionsType', - 'data': [ 'node', 'dist', 'cpu', 'hmat-lb' ] } + 'data': [ 'node', 'dist', 'cpu', 'hmat-lb', 'hmat-cache' ] } ## # @NumaOptions: @@ -447,7 +449,8 @@ 'node': 'NumaNodeOptions', 'dist': 'NumaDistOptions', 'cpu': 'NumaCpuOptions', - 'hmat-lb': 'NumaHmatLBOptions' }} + 'hmat-lb': 'NumaHmatLBOptions', + 'hmat-cache': 'NumaHmatCacheOptions' }} ## # @NumaNodeOptions: @@ -648,6 +651,80 @@ '*latency': 'time', '*bandwidth': 'size' }} +## +# @HmatCacheAssociativity: +# +# Cache associativity in the Memory Side Cache +# Information Structure of HMAT +# +# For more information of @HmatCacheAssociativity see +# the chapter 5.2.27.5: Table 5-143 of ACPI 6.3 spec. +# +# @none: None +# +# @direct: Direct Mapped +# +# @complex: Complex Cache Indexing (implementation specific) +# +# Since: 4.2 +## +{ 'enum': 'HmatCacheAssociativity', + 'data': [ 'none', 'direct', 'complex' ] } + +## +# @HmatCacheWritePolicy: +# +# Cache write policy in the Memory Side Cache +# Information Structure of HMAT +# +# For more information of @HmatCacheWritePolicy see +# the chapter 5.2.27.5: Table 5-143: Field "Cache Attributes" of ACPI 6.3 spec. +# +# @none: None +# +# @write-back: Write Back (WB) +# +# @write-through: Write Through (WT) +# +# Since: 4.2 +## +{ 'enum': 'HmatCacheWritePolicy', + 'data': [ 'none', 'write-back', 'write-through' ] } + +## +# @NumaHmatCacheOptions: +# +# Set the memory side cache information for a given memory domain. +# +# For more information of @NumaHmatCacheOptions see +# the chapter 5.2.27.5: Table 5-143: Field "Cache Attributes" of ACPI 6.3 spec. +# +# @node-id: the memory proximity domain to which the memory belongs. +# +# @size: the size of memory side cache in bytes. +# +# @total: the total cache levels for this memory proximity domain. +# +# @level: the cache level described in this structure. +# +# @assoc: the cache associativity, none/direct-mapped/complex(complex cache indexing). +# +# @policy: the write policy, none/write-back/write-through. +# +# @line: the cache Line size in bytes. +# +# Since: 4.2 +## +{ 'struct': 'NumaHmatCacheOptions', + 'data': { + 'node-id': 'uint32', + 'size': 'size', + 'total': 'uint8', + 'level': 'uint8', + 'assoc': 'HmatCacheAssociativity', + 'policy': 'HmatCacheWritePolicy', + 'line': 'uint16' }} + ## # @HostMemPolicy: # diff --git a/qemu-options.hx b/qemu-options.hx index 252495c288..f30cf27559 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -169,7 +169,8 @@ DEF("numa", HAS_ARG, QEMU_OPTION_numa, "-numa node[,memdev=id][,cpus=firstcpu[-lastcpu]][,nodeid=node][,initiator=node]\n" "-numa dist,src=source,dst=destination,val=distance\n" "-numa cpu,node-id=node[,socket-id=x][,core-id=y][,thread-id=z]\n" - "-numa hmat-lb,initiator=node,target=node,hierarchy=memory|first-level|second-level|third-level,data-type=access-latency|read-latency|write-latency[,latency=lat][,bandwidth=bw]\n", + "-numa hmat-lb,initiator=node,target=node,hierarchy=memory|first-level|second-level|third-level,data-type=access-latency|read-latency|write-latency[,latency=lat][,bandwidth=bw]\n" + "-numa hmat-cache,node-id=node,size=size,total=total,level=level[,assoc=none|direct|complex][,policy=none|write-back|write-through][,line=size]\n", QEMU_ARCH_ALL) STEXI @item -numa node[,mem=@var{size}][,cpus=@var{firstcpu}[-@var{lastcpu}]][,nodeid=@var{node}][,initiator=@var{initiator}] @@ -177,6 +178,7 @@ STEXI @itemx -numa dist,src=@var{source},dst=@var{destination},val=@var{distance} @itemx -numa cpu,node-id=@var{node}[,socket-id=@var{x}][,core-id=@var{y}][,thread-id=@var{z}] @itemx -numa hmat-lb,initiator=@var{node},target=@var{node},hierarchy=@var{str},data-type=@var{str}[,latency=@var{lat}][,bandwidth=@var{bw}] +@itemx -numa hmat-cache,node-id=@var{node},size=@var{size},total=@var{total},level=@var{level}[,assoc=@var{str}][,policy=@var{str}][,line=@var{size}] @findex -numa Define a NUMA node and assign RAM and VCPUs to it. Set the NUMA distance from a source node to a destination node. @@ -282,11 +284,19 @@ if NUM is 0, means the corresponding latency or bandwidth information is not pro And if input numbers without any unit, the latency unit will be 'ps' and the bandwidth will be MB/s. +In @samp{hmat-cache} option, @var{node-id} is the NUMA-id of the memory belongs. +@var{size} is the size of memory side cache in bytes. @var{total} is the total cache levels. +@var{level} is the cache level described in this structure. @var{assoc} is the cache associativity, +the possible value is 'none/direct(direct-mapped)/complex(complex cache indexing)'. +@var{policy} is the write policy. @var{line} is the cache Line size in bytes. + For example, the following option assigns NUMA node 0 and 1. Node 0 has 2 cpus and a ram, node 1 has only a ram. The processors in node 0 access memory in node 0 with access-latency 5 nanoseconds, access-bandwidth is 200 MB/s; The processors in NUMA node 0 access memory in NUMA node 1 with access-latency 10 nanoseconds, access-bandwidth is 100 MB/s. +And for memory side cache information, NUMA node 0 and 1 both have 1 level memory +cache, size is 0x20000 bytes, policy is write-back, the cache Line size is 8 bytes: @example -machine hmat=on \ -m 2G \ @@ -300,7 +310,9 @@ nanoseconds, access-bandwidth is 100 MB/s. -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5ns \ -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=200M \ -numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-latency,latency=10ns \ --numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M +-numa hmat-lb,initiator=0,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=100M \ +-numa hmat-cache,node-id=0,size=0x20000,total=1,level=1,assoc=direct,policy=write-back,line=8 \ +-numa hmat-cache,node-id=1,size=0x20000,total=1,level=1,assoc=direct,policy=write-back,line=8 @end example ETEXI