From patchwork Wed Jan 16 17:58:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10766617 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9399C1580 for ; Wed, 16 Jan 2019 18:00:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 82E062F0EC for ; Wed, 16 Jan 2019 18:00:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 767C22F0EF; Wed, 16 Jan 2019 18:00:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BB3CB2F0F2 for ; Wed, 16 Jan 2019 18:00:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A9EEB8E000C; Wed, 16 Jan 2019 12:59:44 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 9D9B98E000D; Wed, 16 Jan 2019 12:59:44 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 826AB8E000C; Wed, 16 Jan 2019 12:59:44 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-pl1-f198.google.com (mail-pl1-f198.google.com [209.85.214.198]) by kanga.kvack.org (Postfix) with ESMTP id 39FAB8E000D for ; Wed, 16 Jan 2019 12:59:44 -0500 (EST) Received: by mail-pl1-f198.google.com with SMTP id o23so4297516pll.0 for ; Wed, 16 Jan 2019 09:59:44 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=t2b6xpOyb2bVHTLZNAWW/AEy+vaE5WGzHQgBNhAcNp8=; b=MI59qDXgA/T+c2K6qovM9IqOR9ojwd/Tqb62GYmwONNYOKw/Q25MO8uP01kHfouE7t R0FbaENOChE0K5xhdSd1QQqWmslDSZsZqBTZmVNlNLvNHZTtvkUJTbGqLEOOHxn/c4QG uLB0FF098NfcxuhjxmrXxRA44SHPa0C4NhwytUZs0i9Fp1UeWlV1cFmqJHygveQ77kXW 3uc393wad6wItbQusuoLnfv57+x/SoKyCJXjJHkmb7GimPoC0cVl0UXbg3m5LEiNAeyA ZX1HrLimaSMJUTP36W+kD3E0p+YD50RalW73L0VT13OYgNq6YFPjfhvG7jJWbQe3nx1q i51Q== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of keith.busch@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=keith.busch@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Gm-Message-State: AJcUukdXZJNJR8AvC4HqrGe2XDhj6z1AlEmv2+3KR1t0yxeeoUAQSfQw GvjBuAE4IRbCmYX7lf6QoVUJXBM6vqgKbF7p3akIriZFQwwVwnfkVO33IRfgeDu33cowX3oKVXu R/ZcW/cs6ffAAgpCKvNM3KWZZTkq82QhRf2ybCRfq3z7hC9qxJu2LBRCQ3fGwZWjZAw== X-Received: by 2002:a63:134f:: with SMTP id 15mr9925988pgt.19.1547661583766; Wed, 16 Jan 2019 09:59:43 -0800 (PST) X-Google-Smtp-Source: ALg8bN72Hf0iFhOC/b6ntFiZKMi0/GGfR6DwyWCIX7laVIb2rTYC1DIHVffYQNpiQN5RGJMDHL42 X-Received: by 2002:a63:134f:: with SMTP id 15mr9925910pgt.19.1547661582480; Wed, 16 Jan 2019 09:59:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547661582; cv=none; d=google.com; s=arc-20160816; b=oNX1S/LzMFaT9M806hVooHK6bYUn42vW9jVPJ4TU7yJAvc60GSQlAQQFQlJPPv0124 Hzv41MTVkS6VYwqAADazWSUWfZJ6xNdsjunOQsFZOww8hG0I62r0PCPDr1bub35BpM5A +5MbbsfNIOKQPrZ2HNR46NmSe32WELHup0S2X2uZoADNPdS1Mxu/AcoYomwF0pbamp5i iUKeWhLIFXrpsz4FLGAIB0buRsUAZAZqK2m+tXWdogKz6Q33h02gA8FsorAkAMUr01jw FLlCW31TRSmBvgngGHsVJCeKcYrXaM3TgsQbcPNIchwUiw+v7owUUu6K/kLc/IlhIz84 pCAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=t2b6xpOyb2bVHTLZNAWW/AEy+vaE5WGzHQgBNhAcNp8=; b=GRXwk4LH/QkwSk2wfjOV7Oob7DmenTRJEWDS5D0cHattJE6Cp3VpurQkj1zjpXOs8D pREzvrOpzVMzHVg1NCI42QZrTrPx5uKsnGChkf9sqlfkV1qkxU87AbR591Lh0WA9pPjY cTmwjEUyJSCqMlXryslHe4hFDJndLaCAFquZffRcSZTCw/Iz9YEOczTobi0FE01eJM4a 34DfcQUiiLVDOioI+85Q5Yv8yzLwJXUaBTs2Xd37Utp8w2GRX8ONyURN4/m3krYkE+tD Md80RLD8RY/uPbYTVZJ6lAAKdn5t79PbM4iGbHn/IP07BrpmSZHB+Iob4npdWHH/0Ew7 sihQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of keith.busch@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=keith.busch@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from mga01.intel.com (mga01.intel.com. [192.55.52.88]) by mx.google.com with ESMTPS id v9si4151090pfl.45.2019.01.16.09.59.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 16 Jan 2019 09:59:42 -0800 (PST) Received-SPF: pass (google.com: domain of keith.busch@intel.com designates 192.55.52.88 as permitted sender) client-ip=192.55.52.88; Authentication-Results: mx.google.com; spf=pass (google.com: domain of keith.busch@intel.com designates 192.55.52.88 as permitted sender) smtp.mailfrom=keith.busch@intel.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 16 Jan 2019 09:59:41 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,487,1539673200"; d="scan'208";a="117227800" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by fmsmga008.fm.intel.com with ESMTP; 16 Jan 2019 09:59:41 -0800 From: Keith Busch To: linux-kernel@vger.kernel.org, linux-acpi@vger.kernel.org, linux-mm@kvack.org Cc: Greg Kroah-Hartman , Rafael Wysocki , Dave Hansen , Dan Williams , Keith Busch Subject: [PATCHv4 10/13] node: Add memory caching attributes Date: Wed, 16 Jan 2019 10:58:01 -0700 Message-Id: <20190116175804.30196-11-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190116175804.30196-1-keith.busch@intel.com> References: <20190116175804.30196-1-keith.busch@intel.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP System memory may have side caches to help improve access speed to frequently requested address ranges. While the system provided cache is transparent to the software accessing these memory ranges, applications can optimize their own access based on cache attributes. Provide a new API for the kernel to register these memory side caches under the memory node that provides it. The new sysfs representation is modeled from the existing cpu cacheinfo attributes, as seen from /sys/devices/system/cpu/cpuX/side_cache/. Unlike CPU cacheinfo, though, the node cache level is reported from the view of the memory. A higher number is nearer to the CPU, while lower levels are closer to the backing memory. Also unlike CPU cache, it is assumed the system will handle flushing any dirty cached memory to the last level on a power failure if the range is persistent memory. The attributes we export are the cache size, the line size, associativity, and write back policy. Signed-off-by: Keith Busch --- drivers/base/node.c | 142 +++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/node.h | 39 ++++++++++++++ 2 files changed, 181 insertions(+) diff --git a/drivers/base/node.c b/drivers/base/node.c index 1e909f61e8b1..7ff3ed566d7d 100644 --- a/drivers/base/node.c +++ b/drivers/base/node.c @@ -191,6 +191,146 @@ void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, pr_info("failed to add performance attribute group to node %d\n", nid); } + +struct node_cache_info { + struct device dev; + struct list_head node; + struct node_cache_attrs cache_attrs; +}; +#define to_cache_info(device) container_of(device, struct node_cache_info, dev) + +#define CACHE_ATTR(name, fmt) \ +static ssize_t name##_show(struct device *dev, \ + struct device_attribute *attr, \ + char *buf) \ +{ \ + return sprintf(buf, fmt "\n", to_cache_info(dev)->cache_attrs.name);\ +} \ +DEVICE_ATTR_RO(name); + +CACHE_ATTR(size, "%llu") +CACHE_ATTR(level, "%u") +CACHE_ATTR(line_size, "%u") +CACHE_ATTR(associativity, "%u") +CACHE_ATTR(write_policy, "%u") + +static struct attribute *cache_attrs[] = { + &dev_attr_level.attr, + &dev_attr_associativity.attr, + &dev_attr_size.attr, + &dev_attr_line_size.attr, + &dev_attr_write_policy.attr, + NULL, +}; +ATTRIBUTE_GROUPS(cache); + +static void node_cache_release(struct device *dev) +{ + kfree(dev); +} + +static void node_cacheinfo_release(struct device *dev) +{ + struct node_cache_info *info = to_cache_info(dev); + kfree(info); +} + +static void node_init_cache_dev(struct node *node) +{ + struct device *dev; + + dev = kzalloc(sizeof(*dev), GFP_KERNEL); + if (!dev) + return; + + dev->parent = &node->dev; + dev->release = node_cache_release; + if (dev_set_name(dev, "side_cache")) + goto free_dev; + + if (device_register(dev)) + goto free_name; + + pm_runtime_no_callbacks(dev); + node->cache_dev = dev; + return; +free_name: + kfree_const(dev->kobj.name); +free_dev: + kfree(dev); +} + +void node_add_cache(unsigned int nid, struct node_cache_attrs *cache_attrs) +{ + struct node_cache_info *info; + struct device *dev; + struct node *node; + + if (!node_online(nid) || !node_devices[nid]) + return; + + node = node_devices[nid]; + list_for_each_entry(info, &node->cache_attrs, node) { + if (info->cache_attrs.level == cache_attrs->level) { + dev_warn(&node->dev, + "attempt to add duplicate cache level:%d\n", + cache_attrs->level); + return; + } + } + + if (!node->cache_dev) + node_init_cache_dev(node); + if (!node->cache_dev) + return; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return; + + dev = &info->dev; + dev->parent = node->cache_dev; + dev->release = node_cacheinfo_release; + dev->groups = cache_groups; + if (dev_set_name(dev, "index%d", cache_attrs->level)) + goto free_cache; + + info->cache_attrs = *cache_attrs; + if (device_register(dev)) { + dev_warn(&node->dev, "failed to add cache level:%d\n", + cache_attrs->level); + goto free_name; + } + pm_runtime_no_callbacks(dev); + list_add_tail(&info->node, &node->cache_attrs); + return; +free_name: + kfree_const(dev->kobj.name); +free_cache: + kfree(info); +} + +static void node_remove_caches(struct node *node) +{ + struct node_cache_info *info, *next; + + if (!node->cache_dev) + return; + + list_for_each_entry_safe(info, next, &node->cache_attrs, node) { + list_del(&info->node); + device_unregister(&info->dev); + } + device_unregister(node->cache_dev); +} + +static void node_init_caches(unsigned int nid) +{ + INIT_LIST_HEAD(&node_devices[nid]->cache_attrs); +} +#else +static void node_init_caches(unsigned int nid) { } +static void node_remove_caches(struct node *node) { } #endif #define K(x) ((x) << (PAGE_SHIFT - 10)) @@ -475,6 +615,7 @@ void unregister_node(struct node *node) { hugetlb_unregister_node(node); /* no-op, if memoryless node */ node_remove_classes(node); + node_remove_caches(node); device_unregister(&node->dev); } @@ -755,6 +896,7 @@ int __register_one_node(int nid) INIT_LIST_HEAD(&node_devices[nid]->class_list); /* initialize work queue for memory hot plug */ init_node_hugetlb_work(nid); + node_init_caches(nid); return error; } diff --git a/include/linux/node.h b/include/linux/node.h index e22940a593c2..8cdf2b2808e4 100644 --- a/include/linux/node.h +++ b/include/linux/node.h @@ -37,12 +37,47 @@ struct node_hmem_attrs { }; void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, unsigned class); + +enum cache_associativity { + NODE_CACHE_DIRECT_MAP, + NODE_CACHE_INDEXED, + NODE_CACHE_OTHER, +}; + +enum cache_write_policy { + NODE_CACHE_WRITE_BACK, + NODE_CACHE_WRITE_THROUGH, + NODE_CACHE_WRITE_OTHER, +}; + +/** + * struct node_cache_attrs - system memory caching attributes + * + * @associativity: The ways memory blocks may be placed in cache + * @write_policy: Write back or write through policy + * @size: Total size of cache in bytes + * @line_size: Number of bytes fetched on a cache miss + * @level: Represents the cache hierarchy level + */ +struct node_cache_attrs { + enum cache_associativity associativity; + enum cache_write_policy write_policy; + u64 size; + u16 line_size; + u8 level; +}; +void node_add_cache(unsigned int nid, struct node_cache_attrs *cache_attrs); #else static inline void node_set_perf_attrs(unsigned int nid, struct node_hmem_attrs *hmem_attrs, unsigned class) { } + +static inline void node_add_cache(unsigned int nid, + struct node_cache_attrs *cache_attrs) +{ +} #endif struct node { @@ -51,6 +86,10 @@ struct node { #if defined(CONFIG_MEMORY_HOTPLUG_SPARSE) && defined(CONFIG_HUGETLBFS) struct work_struct node_work; #endif +#ifdef CONFIG_HMEM_REPORTING + struct list_head cache_attrs; + struct device *cache_dev; +#endif }; struct memory_block;