From patchwork Mon Dec 3 23:35:01 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jerome Glisse X-Patchwork-Id: 10710899 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id ADA7814E2 for ; Mon, 3 Dec 2018 23:36:06 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9CC5829267 for ; Mon, 3 Dec 2018 23:36:06 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8FE652B222; Mon, 3 Dec 2018 23:36:06 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AD78D29267 for ; Mon, 3 Dec 2018 23:36:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 84EEE6B6BB0; Mon, 3 Dec 2018 18:36:04 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 7D85C6B6BB1; Mon, 3 Dec 2018 18:36:04 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 654576B6BB2; Mon, 3 Dec 2018 18:36:04 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f200.google.com (mail-qk1-f200.google.com [209.85.222.200]) by kanga.kvack.org (Postfix) with ESMTP id 35A186B6BB0 for ; Mon, 3 Dec 2018 18:36:04 -0500 (EST) Received: by mail-qk1-f200.google.com with SMTP id y83so14733007qka.7 for ; Mon, 03 Dec 2018 15:36:04 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=lteeHK8AAWG2SXyXToM8ykO2ukjRJHO6lSFifN80R68=; b=VTYmS+qWYBK8p2c4FUkyLpN0G4nB+f2DXdmd4RKhMeY9kJsf1mNr8o3izL6PuLQHu8 lI6/+Q6LzpnCQ8HmrA5SVg2P9+0kMkcx/kLM0v2BzU6D+lIAAcyccEWhsAPLYiun+bf8 sissRYeI9BHUcI4q1K5U8xTuw0KUnXqe0IuONidE8zvBgq4UjcY/jO5StKL0XpFKNoBv i4K9kMs7zUPiIwqpNJRIaQK2Dgl7bJnyuTpTdjeXEsIGm3VT9Of/XdNnwcCM0k5LJUbn mKe1c07ubINjpEbe0J4+BO0syuQiKBpfN2RAU8/G0Xm1JZ5mcvBPrZ5Uza8js+uVHwYR Yvfg== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AA+aEWYAemRFcWfwJtE0AN4zC9BgIttKlypN7gaj/KvGpjjUpio3itXp aCdEuUSwU70sp+on+yM2YKHBGq96cpOVFcC9ggjR27ZvTgcAre9fk1u0QrDiKm6efeIMe7vUuQz ZCnZAV1DNa519yGfpi75CSjIdeH6/CPF0v/lh6HRkEw6rF2ZfM8aRiL26ioOO15bizA== X-Received: by 2002:a0c:95b5:: with SMTP id s50mr17744892qvs.64.1543880163942; Mon, 03 Dec 2018 15:36:03 -0800 (PST) X-Google-Smtp-Source: AFSGD/UrAvkQBV/uFtmyPwziTZDg4NpOfjS9uHIt/xXOxhvwgn9rit3O5QG0vTa8qDkz2AZYVcTr X-Received: by 2002:a0c:95b5:: with SMTP id s50mr17744871qvs.64.1543880163231; Mon, 03 Dec 2018 15:36:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543880163; cv=none; d=google.com; s=arc-20160816; b=08d6auV99XDlvnGGEXxiuERKg8mGYc+dcYdEAMttv9kX40rdEJt+kY/VduqC6N2vgm kyGOkDAsMjgTcgDfUsavzvb4Jl3I3SoQFwYUhRcjcpRmQTWsA+gjgY2UUr9oyyJq7OK0 Rvc2GvlaC8sqlJ1rFsvspWIWLfbi44nDTk/QFtTLKpAnyLpXQiKbd41yqbxnRXndh4JT 2IfnQSBV3njrzASmhBHHHQXse5+3l2/FsPOadK0Duc65MuyHrc4M2J5qE0MBbXk6o8cn +Wjmv8BU8zmhiItJUTk/cNAUVsfCe7LMOof8so4tO7u6JW66+ob0jjNAjgpUmbal+TsR NDhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=lteeHK8AAWG2SXyXToM8ykO2ukjRJHO6lSFifN80R68=; b=opqcilcBbjybjNAPYYUs02AJrXzX/9sYhhbQThkkYZb2hG8L4lrd+XQRnrCfbV9Y3G DnFoqdbAYqsbcLwVgULb+JtvVQ7felraDtL7EHUkDtt6X0YY3drcIydMS40HbKU0NSVY NY197QG3zBPcH/GLAGs1LbN7kJPW0MPqLw6C3NWfr7HAJ4dApDVEdtp40Gt6dly9vh75 Bj8nEhsX7wHTKYdbCMgjtQgp7kBOjlj5KQVDDsYnzl30eZ/BPtkaVqwc2b84TCOCHGLb apQRIPw3wxyAQ126O9YBjc02CJhgyazsv2fmYbl0Pgi3HSnnsviOxvkmb1U4I0QxgMNA FcKQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id u11si101004qvl.90.2018.12.03.15.36.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 03 Dec 2018 15:36:03 -0800 (PST) Received-SPF: pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jglisse@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=jglisse@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 163EA2D7FB; Mon, 3 Dec 2018 23:36:02 +0000 (UTC) Received: from localhost.localdomain.com (ovpn-120-188.rdu2.redhat.com [10.10.120.188]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9873600C1; Mon, 3 Dec 2018 23:35:58 +0000 (UTC) From: jglisse@redhat.com To: linux-mm@kvack.org Cc: Andrew Morton , linux-kernel@vger.kernel.org, =?utf-8?b?SsOpcsO0bWUgR2xpc3Nl?= , "Rafael J . Wysocki" , Ross Zwisler , Dan Williams , Dave Hansen , Haggai Eran , Balbir Singh , "Aneesh Kumar K . V" , Benjamin Herrenschmidt , Felix Kuehling , Philip Yang , =?utf-8?q?Christian_K=C3=B6nig?= , Paul Blinzer , Logan Gunthorpe , John Hubbard , Ralph Campbell , Michal Hocko , Jonathan Cameron , Mark Hairgrove , Vivek Kini , Mel Gorman , Dave Airlie , Ben Skeggs , Andrea Arcangeli Subject: [RFC PATCH 06/14] mm/hms: add bridge to heterogeneous memory system infrastructure Date: Mon, 3 Dec 2018 18:35:01 -0500 Message-Id: <20181203233509.20671-7-jglisse@redhat.com> In-Reply-To: <20181203233509.20671-1-jglisse@redhat.com> References: <20181203233509.20671-1-jglisse@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]); Mon, 03 Dec 2018 23:36:02 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP From: Jérôme Glisse A bridge connect two links with each others and apply only to listed initiators. With links, this allows to describe any kind of system topology ie any kind of directed graph. Moreover with bridges the userspace can choose to use different bridges to load balance bandwidth usage accross multiple paths between targets memory and initiators. Note that explicit path selection is not always under the control of user space, some system might do load balancing in hardware. See HMS Documentation/vm/hms.txt for detail. Signed-off-by: Jérôme Glisse Cc: Rafael J. Wysocki Cc: Ross Zwisler Cc: Dan Williams Cc: Dave Hansen Cc: Haggai Eran Cc: Balbir Singh Cc: Aneesh Kumar K.V Cc: Benjamin Herrenschmidt Cc: Felix Kuehling Cc: Philip Yang Cc: Christian König Cc: Paul Blinzer Cc: Logan Gunthorpe Cc: John Hubbard Cc: Ralph Campbell Cc: Michal Hocko Cc: Jonathan Cameron Cc: Mark Hairgrove Cc: Vivek Kini Cc: Mel Gorman Cc: Dave Airlie Cc: Ben Skeggs Cc: Andrea Arcangeli --- drivers/base/Makefile | 2 +- drivers/base/hms-bridge.c | 197 ++++++++++++++++++++++++++++++++++++++ include/linux/hms.h | 24 +++++ 3 files changed, 222 insertions(+), 1 deletion(-) create mode 100644 drivers/base/hms-bridge.c diff --git a/drivers/base/Makefile b/drivers/base/Makefile index b8ff678fdae9..62695fdcd32f 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -12,7 +12,7 @@ obj-y += power/ obj-$(CONFIG_ISA_BUS_API) += isa.o obj-y += firmware_loader/ obj-$(CONFIG_NUMA) += node.o -obj-$(CONFIG_HMS) += hms.o hms-target.o hms-initiator.o hms-link.o +obj-$(CONFIG_HMS) += hms.o hms-target.o hms-initiator.o hms-link.o hms-bridge.o obj-$(CONFIG_MEMORY_HOTPLUG_SPARSE) += memory.o ifeq ($(CONFIG_SYSFS),y) obj-$(CONFIG_MODULES) += module.o diff --git a/drivers/base/hms-bridge.c b/drivers/base/hms-bridge.c new file mode 100644 index 000000000000..64732e923fba --- /dev/null +++ b/drivers/base/hms-bridge.c @@ -0,0 +1,197 @@ +/* + * Copyright 2018 Red Hat Inc. + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License as + * published by the Free Software Foundation; either version 2 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * Authors: + * Jérôme Glisse + */ +/* Heterogeneous memory system (HMS) see Documentation/vm/hms.rst */ +#include +#include +#include +#include +#include +#include +#include +#include +#include + + +static inline struct hms_bridge *hms_object_to_bridge(struct hms_object *object) +{ + if (object == NULL) + return NULL; + + if (object->type != HMS_BRIDGE) + return NULL; + return container_of(object, struct hms_bridge, object); +} + +static inline struct hms_bridge *device_to_hms_bridge(struct device *device) +{ + if (device == NULL) + return NULL; + + return hms_object_to_bridge(to_hms_object(device)); +} + +struct hms_bridge *hms_bridge_find_locked(unsigned uid) +{ + struct hms_object *object = hms_object_find_locked(uid); + struct hms_bridge *bridge; + + bridge = hms_object_to_bridge(object); + if (bridge) + return bridge; + hms_object_put(object); + return NULL; +} + +struct hms_bridge *hms_bridge_find(unsigned uid) +{ + struct hms_object *object = hms_object_find(uid); + struct hms_bridge *bridge; + + bridge = hms_object_to_bridge(object); + if (bridge) + return bridge; + hms_object_put(object); + return NULL; +} + +static void hms_bridge_release(struct device *device) +{ + struct hms_bridge *bridge = device_to_hms_bridge(device); + + hms_object_put(&bridge->linka->object); + hms_object_put(&bridge->linkb->object); + hms_object_release(&bridge->object); + kfree(bridge); +} + +static ssize_t hms_bridge_show_uid(struct device *device, + struct device_attribute *attr, + char *buf) +{ + struct hms_bridge *bridge = device_to_hms_bridge(device); + + if (bridge == NULL) + return -EINVAL; + + return sprintf(buf, "%d\n", bridge->object.uid); +} + +static DEVICE_ATTR(uid, 0444, hms_bridge_show_uid, NULL); + +static struct attribute *hms_bridge_attrs[] = { + &dev_attr_uid.attr, + NULL +}; + +static struct attribute_group hms_bridge_attr_group = { + .attrs = hms_bridge_attrs, +}; + +static const struct attribute_group *hms_bridge_attr_groups[] = { + &hms_bridge_attr_group, + NULL, +}; + +void hms_bridge_register(struct hms_bridge **bridgep, + struct device *parent, + struct hms_link *linka, + struct hms_link *linkb, + unsigned version) +{ + struct hms_bridge *bridge; + int ret; + + *bridgep = NULL; + + if (linka == NULL || linkb == NULL) + return; + linka = hms_object_to_link(hms_object_get(&linka->object)); + linkb = hms_object_to_link(hms_object_get(&linkb->object)); + if (linka == NULL || linkb == NULL) + goto error; + + bridge = kzalloc(sizeof(*bridge), GFP_KERNEL); + if (bridge == NULL) + goto error; + + if (hms_object_init(&bridge->object, parent, HMS_BRIDGE, version, + hms_bridge_release, hms_bridge_attr_groups)) { + kfree(bridge); + goto error; + } + + bridge->linka = linka; + bridge->linkb = linkb; + + ret = hms_object_link(&bridge->object, &linka->object); + if (ret) { + hms_bridge_unregister(&bridge); + return; + } + + ret = hms_object_link(&bridge->object, &linkb->object); + if (ret) { + hms_bridge_unregister(&bridge); + return; + } + + *bridgep = bridge; + return; + +error: + hms_object_put(&linka->object); + hms_object_put(&linkb->object); +} +EXPORT_SYMBOL(hms_bridge_register); + +void hms_unbridge_initiator(struct hms_bridge *bridge, + struct hms_initiator *initiator) +{ + if (bridge == NULL || initiator == NULL) + return; + if (bridge->object.type != HMS_BRIDGE) + return; + if (initiator->object.type != HMS_INITIATOR) + return; + hms_object_unlink(&bridge->object, &initiator->object); +} +EXPORT_SYMBOL(hms_unbridge_initiator); + +int hms_bridge_initiator(struct hms_bridge *bridge, + struct hms_initiator *initiator) +{ + if (bridge == NULL || initiator == NULL) + return -EINVAL; + if (bridge->object.type != HMS_BRIDGE) + return -EINVAL; + if (initiator->object.type != HMS_INITIATOR) + return -EINVAL; + return hms_object_link(&bridge->object, &initiator->object); +} +EXPORT_SYMBOL(hms_bridge_initiator); + +void hms_bridge_unregister(struct hms_bridge **bridgep) +{ + struct hms_bridge *bridge = *bridgep; + + *bridgep = NULL; + if (bridge == NULL) + return; + + hms_object_unregister(&bridge->object); +} +EXPORT_SYMBOL(hms_bridge_unregister); diff --git a/include/linux/hms.h b/include/linux/hms.h index 2a9e49a2d771..511b5363d8f2 100644 --- a/include/linux/hms.h +++ b/include/linux/hms.h @@ -115,6 +115,24 @@ void hms_link_register(struct hms_link **linkp, struct device *parent, void hms_link_unregister(struct hms_link **linkp); +struct hms_bridge { + struct hms_object object; + struct hms_link *linka; + struct hms_link *linkb; +}; + +void hms_unbridge_initiator(struct hms_bridge *bridge, + struct hms_initiator *initiator); +int hms_bridge_initiator(struct hms_bridge *bridge, + struct hms_initiator *initiator); +void hms_bridge_register(struct hms_bridge **bridgep, + struct device *parent, + struct hms_link *linka, + struct hms_link *linkb, + unsigned version); +void hms_bridge_unregister(struct hms_bridge **bridgep); + + int hms_init(void); @@ -139,6 +157,12 @@ int hms_init(void); #define hms_link_unregister(linkp) +#define hms_unbridge_initiator(bridge, initiator) +#define hms_bridge_initiator(bridge, initiator) +#define hms_bridge_register(bridgep) +#define hms_bridge_unregister(bridgep) + + static inline int hms_init(void) { return 0;