From patchwork Mon Dec 3 23:35:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10710897 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5BA0614E2 for ; Mon, 3 Dec 2018 23:36:03 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4B3CA29267 for ; Mon, 3 Dec 2018 23:36:03 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3F3842B212; Mon, 3 Dec 2018 23:36:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5E7E029267 for ; Mon, 3 Dec 2018 23:36:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 235F96B6BAE; Mon, 3 Dec 2018 18:36:01 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1966C6B6BAF; Mon, 3 Dec 2018 18:36:01 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 03A246B6BB0; Mon, 3 Dec 2018 18:36:00 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f199.google.com (mail-qk1-f199.google.com [209.85.222.199]) by kanga.kvack.org (Postfix) with ESMTP id C79156B6BAE for ; Mon, 3 Dec 2018 18:36:00 -0500 (EST) Received: by mail-qk1-f199.google.com with SMTP id k203so14975907qke.2 for ; Mon, 03 Dec 2018 15:36:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=XeiKgLSePtBBt1YPT9+iZv2xdm7yAGzLblr3Z0Y6AK4=; b=axfhFd9VWdePVo3149f5ZHBqoKCP2IiDEf+oO6hMbu58jlTabqFHjx+9+6DzbhpzK3 QlSDSzyzmy9sXUh1rNNK8DCJcD4OvNWT3dfwytQx6dIa2rhZXzPdUl4McqIb+N8bP45/ X2yA98OdeVMiXbVVJieqXaoE19rXiH4QMV2vdo5zCHvcEi2qXhx+QMSub6Vo8URHoJa3 WLegkkGIHZMV23ykJDuNcle41esZNWXXuws4aDybNpIpSYwhddWu+imFpj4j2As96epJ MCwuhJrj+3dCquMALta3L4kiTKsE2wOWjWfFZ23lgTGY8Erc03Utk0AxATk0Z6IZArBh Sz7g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWbJShDUhpLWROMonrDPecwxwdpFbpiV962xnb9t95JyERDepHYY SAL0qQq4Lh9YPQ2Ed9yauKJtyQQkREzqtx+GsXC14InwlVHcWL3X7Cw9Mn7sQqMI+iLxHnbW3zW FlobIQeGYcQq78BDO0M8nfcHBh+dhJjnM7s6jS+/j6iHSSlREYyrgukakpegJbnvyxg== X-Received: by 2002:a37:17df:: with SMTP id 92mr17100744qkx.191.1543880160553; Mon, 03 Dec 2018 15:36:00 -0800 (PST) X-Google-Smtp-Source: AFSGD/XPIFh7APiihHcfK6e+KeVcmpVxm13KQSHF+6MYkeVcAoL10+yPFI0CCm7Bzq18RTRVM5gV X-Received: by 2002:a37:17df:: with SMTP id 92mr17100718qkx.191.1543880159822; Mon, 03 Dec 2018 15:35:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543880159; cv=none; d=google.com; s=arc-20160816; b=QpGjYFXlNE7+FN9ZlPMO68ZLD7vM/8WRO0dMGuYsnp4WXykpOL7LU4USGm9Qg8F15q M3vktIiuI72vOSHUKuUsCSnZReNHghMPw4dqaVRepHh1+HlZeLPZb/rQ4PEI8p4Iv1g5 92h34Ud/QmI8iYsZEHvPsIdiujt/BqU+n/GgepbqgbG1QyDVJLUmEOcoVDRzN2B+jMli wcg1MUfBqRz+2GZPcuTdj/otT0rLO/Oub2HZSKyAYxpfPw71mzc2GnXNAnpC3LUP2Rcq fS0JctVHYIOoSswhDZfl7sQukb8OTstxLvUjph1LWyAhaVELf0WH8uGQ0XkBtcjzg6k/ hqMA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=XeiKgLSePtBBt1YPT9+iZv2xdm7yAGzLblr3Z0Y6AK4=; b=wQRd8nx5Qstf7SxDy7wdUu+RzrHXarkXUXoIGKIhdAvcKBxAW0AatjDa54wmTSiWCq TCXb4g5JkQxlQ+L8LNmQgb9kdst9ppagz3Z0/uxtYTDT82wvZ2AbAGiDYu6iapP0BgRd GyGVENhKIH29w4aDS7psePHByu3J1q5MTDI2Pj2pkuo7XQJAaiBbCgHqZ6g9l8zkHEgn IGf/mYiTupXRgriyTFi5obK1YWwTbHqjIBCM+z1cJNqjkqeWJbL3w2bAttnRxGQkOjFp 4+KRtyLA++39cv9KcpTJeBz7dWVQNbo5aFOKOGcxyOw5i3wQ+6fiS+IklHJ+46IHEzJc 2/eA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id p45si5826356qta.144.2018.12.03.15.35.59 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Dec 2018 15:35:59 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9111F315486B; Mon, 3 Dec 2018 23:35:58 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-188.rdu2.redhat.com [10.10.120.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3CC23604A3; Mon, 3 Dec 2018 23:35:55 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Rafael J . Wysocki" , Ross Zwisler , Dan Williams , Dave Hansen , Haggai Eran , Balbir Singh , "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Felix Kuehling , Philip Yang , =?utf-8?q?Christian_K=C3=B6nig?= , Paul Blinzer , Logan Gunthorpe , John Hubbard , Ralph Campbell , Michal Hocko , Jonathan Cameron , Mark Hairgrove , Vivek Kini , Mel Gorman , Dave Airlie , Ben Skeggs , Andrea Arcangeli Subject: [RFC PATCH 05/14] mm/hms: add link to heterogeneous memory system infrastructure Date: Mon, 3 Dec 2018 18:35:00 -0500 Message-Id: <20181203233509.20671-6-jglisse@redhat.com> In-Reply-To: <20181203233509.20671-1-jglisse@redhat.com> References: <20181203233509.20671-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Mon, 03 Dec 2018 23:35:59 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse A link connect initiators (CPUs or devices) and targets memory with each others. It does necessarily match one to one with a physical inter-connect ie a given physical inter-connect by be presented as multiple links or multiple physical inter-connect can be presented as just one link. What matters is that the properties associated with the links applies to all initiators and targets listed as connected to that link. For example you can consider the PCIE bus if all initiators can peer to peer with each others than it can be presented as just one link with all the PCIE devices in it and the local CPU (ie CPU from which the PCIE lanes are coming from). If not all PCIE device can peer to peer than a link per peer to peer group is created and corresponding CPU is added to each. See HMS Documentation/vm/hms.txt for detail. Signed-off-by: Jérôme Glisse Cc: Rafael J. Wysocki Cc: Ross Zwisler Cc: Dan Williams Cc: Dave Hansen Cc: Haggai Eran Cc: Balbir Singh Cc: Aneesh Kumar K.V Cc: Benjamin Herrenschmidt Cc: Felix Kuehling Cc: Philip Yang Cc: Christian König Cc: Paul Blinzer Cc: Logan Gunthorpe Cc: John Hubbard Cc: Ralph Campbell Cc: Michal Hocko Cc: Jonathan Cameron Cc: Mark Hairgrove Cc: Vivek Kini Cc: Mel Gorman Cc: Dave Airlie Cc: Ben Skeggs Cc: Andrea Arcangeli --- drivers/base/Makefile | 2 +- drivers/base/hms-link.c | 183 ++++++++++++++++++++++++++++++++++++++++ include/linux/hms.h | 23 +++++ 3 files changed, 207 insertions(+), 1 deletion(-) create mode 100644 drivers/base/hms-link.c diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 6a1b5ab667bd..b8ff678fdae9 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -12,7 +12,7 @@ obj-y += power/ obj-$(CONFIG_ISA_BUS_API) += isa.o obj-y += firmware_loader/ obj-$(CONFIG_NUMA) += node.o -obj-$(CONFIG_HMS) += hms.o hms-target.o hms-initiator.o +obj-$(CONFIG_HMS) += hms.o hms-target.o hms-initiator.o hms-link.o obj-$(CONFIG_MEMORY_HOTPLUG_SPARSE) += memory.o ifeq ($(CONFIG_SYSFS),y) obj-$(CONFIG_MODULES) += module.o diff --git a/drivers/base/hms-link.c b/drivers/base/hms-link.c new file mode 100644 index 000000000000..58f4fdd8977c --- /dev/null +++ b/drivers/base/hms-link.c @@ -0,0 +1,183 @@ +/* + * Copyright 2018 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Authors: + * Jérôme Glisse + */ +/* Heterogeneous memory system (HMS) see Documentation/vm/hms.rst */ +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +struct hms_link *hms_object_to_link(struct hms_object *object) +{ + if (object == NULL) + return NULL; + + if (object->type != HMS_LINK) + return NULL; + return container_of(object, struct hms_link, object); +} + +static inline struct hms_link *device_to_hms_link(struct device *device) +{ + if (device == NULL) + return NULL; + + return hms_object_to_link(to_hms_object(device)); +} + +struct hms_link *hms_link_find_locked(unsigned uid) +{ + struct hms_object *object = hms_object_find_locked(uid); + struct hms_link *link; + + link = hms_object_to_link(object); + if (link) + return link; + hms_object_put(object); + return NULL; +} + +struct hms_link *hms_link_find(unsigned uid) +{ + struct hms_object *object = hms_object_find(uid); + struct hms_link *link; + + link = hms_object_to_link(object); + if (link) + return link; + hms_object_put(object); + return NULL; +} + +static void hms_link_release(struct device *device) + +{ + struct hms_link *link = device_to_hms_link(device); + + hms_object_release(&link->object); + kfree(link); +} + +static ssize_t hms_link_show_uid(struct device *device, + struct device_attribute *attr, + char *buf) +{ + struct hms_link *link = device_to_hms_link(device); + + if (link == NULL) + return -EINVAL; + + return sprintf(buf, "%d\n", link->object.uid); +} + +static DEVICE_ATTR(uid, 0444, hms_link_show_uid, NULL); + +static struct attribute *hms_link_attrs[] = { + &dev_attr_uid.attr, + NULL +}; + +static struct attribute_group hms_link_attr_group = { + .attrs = hms_link_attrs, +}; + +static const struct attribute_group *hms_link_attr_groups[] = { + &hms_link_attr_group, + NULL, +}; + +void hms_link_register(struct hms_link **linkp, struct device *parent, + unsigned version) +{ + struct hms_link *link; + + *linkp = NULL; + link = kzalloc(sizeof(*link), GFP_KERNEL); + if (link == NULL) + return; + + if (hms_object_init(&link->object, parent, HMS_LINK, version, + hms_link_release, hms_link_attr_groups)) { + kfree(link); + link = NULL; + } + + *linkp = link; +} +EXPORT_SYMBOL(hms_link_register); + +void hms_unlink_initiator(struct hms_link *link, + struct hms_initiator *initiator) +{ + if (link == NULL || initiator == NULL) + return; + if (link->object.type != HMS_LINK) + return; + if (initiator->object.type != HMS_INITIATOR) + return; + hms_object_unlink(&link->object, &initiator->object); +} +EXPORT_SYMBOL(hms_unlink_initiator); + +void hms_unlink_target(struct hms_link *link, struct hms_target *target) +{ + if (link == NULL || target == NULL) + return; + if (link->object.type != HMS_LINK || target->object.type != HMS_TARGET) + return; + hms_object_unlink(&link->object, &target->object); +} +EXPORT_SYMBOL(hms_unlink_target); + +int hms_link_initiator(struct hms_link *link, struct hms_initiator *initiator) +{ + if (link == NULL || initiator == NULL) + return -EINVAL; + if (link->object.type != HMS_LINK) + return -EINVAL; + if (initiator->object.type != HMS_INITIATOR) + return -EINVAL; + return hms_object_link(&link->object, &initiator->object); +} +EXPORT_SYMBOL(hms_link_initiator); + +int hms_link_target(struct hms_link *link, struct hms_target *target) +{ + if (link == NULL || target == NULL) + return -EINVAL; + if (link->object.type != HMS_LINK || target->object.type != HMS_TARGET) + return -EINVAL; + return hms_object_link(&link->object, &target->object); +} +EXPORT_SYMBOL(hms_link_target); + +void hms_link_unregister(struct hms_link **linkp) +{ + struct hms_link *link = *linkp; + + *linkp = NULL; + if (link == NULL) + return; + + hms_object_unregister(&link->object); +} +EXPORT_SYMBOL(hms_link_unregister); diff --git a/include/linux/hms.h b/include/linux/hms.h index 7a2823493f63..2a9e49a2d771 100644 --- a/include/linux/hms.h +++ b/include/linux/hms.h @@ -100,6 +100,21 @@ static inline void hms_target_put(struct hms_target *target) } +struct hms_link { + struct hms_object object; +}; + +struct hms_link *hms_object_to_link(struct hms_object *object); +void hms_unlink_initiator(struct hms_link *link, + struct hms_initiator *initiator); +void hms_unlink_target(struct hms_link *link, struct hms_target *target); +int hms_link_initiator(struct hms_link *link, struct hms_initiator *initiator); +int hms_link_target(struct hms_link *link, struct hms_target *target); +void hms_link_register(struct hms_link **linkp, struct device *parent, + unsigned version); +void hms_link_unregister(struct hms_link **linkp); + + int hms_init(void); @@ -116,6 +131,14 @@ int hms_init(void); #define hms_target_unregister(targetp) +#define hms_unlink_initiator(link, initiator) +#define hms_unlink_target(link, target) +#define hms_link_initiator(link, initiator) +#define hms_link_target(link, target) +#define hms_link_register(linkp) +#define hms_link_unregister(linkp) + + static inline int hms_init(void) { return 0;