From patchwork Wed Jan 15 19:55:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940867 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78FB51D90B9; Wed, 15 Jan 2025 19:55:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970946; cv=none; b=FZJdX0IG4wsNx3fIVtq2KFmBuixGIJfRU3C5Tb4Hz+xK4NsNWDsxxtm+LCrdjPT3kGQvg7u00t2eymcZpR44zW+BQiuhcLM19Oo7r85iJGKnuYCA5GdvQedjrtB/2uEjSG9yQ886zD6kKNZd0dr1M1FIBTcMt1k3FRNEXDT6leA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970946; c=relaxed/simple; bh=M4jc7KywldEBMSriBSffmUb2Aakwp58Qk8yORJ0Ur5g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t12uk5odiREuNVdxNo4S3NWn+6rZAON+y60NS5MGuzYFsNeaiEzFH5eDt7fRcMUGICaSV9UAk1XyQAb/A2Hru+jXFQTyceKslKf4H6GBZ2393UkhpOWeSsoOcdz3jHKrFg9SRyG2jJAycpQk2DKH2b8hamuZEdbD4vHayW7pgXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ncWWYY7I; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ncWWYY7I" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX5T8022844; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=QAPQX+hFfDdUU7n4C initYfhGV0oNAyMPeW7d6O50js=; b=ncWWYY7IxGkRybImV3m+iTGxBp9XzZXDK GCSnOIp8UQs0oNlxYXqUkXvMWkNmizzmDcURqddZKUl/CDja7RLm0ecDp/L82JgU yPwM1X/xCkNrx1KJ/lQkG9rolpG+qe8lJbvReujssm4IhanD2VqE5NlakYzQrP9A PxPUcJZButWpiuONBry2dlkwyG29CxuP9NlyGA5UYLe3UA7SWhNpwkT4mlkenPus KCDYNKV7ay+vlJmsR4i6buV6dr2gjrDgom1qngYNhRg4934qxQZpxlmE0l/Qc3Xi N+N2yMms9iH0/lzY9QQ0eU7Bfa0eETKw4i6TaMXQf2K7BfZGBs5zg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy61-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJtW5T006726; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy5v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FIxDwg016499; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4445p1su9c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav06.fra02v.mail.ibm.com (smtpav06.fra02v.mail.ibm.com [10.20.54.105]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtRtM30147208 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:27 GMT Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B74B52004E; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav06.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8E4F020049; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav06.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 45CC4E04DC; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 1/7] net/ism: Create net/ism Date: Wed, 15 Jan 2025 20:55:21 +0100 Message-ID: <20250115195527.2094320-2-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: LAXKG9qPW7047XT2VfpIf8Ye_P_1xwEt X-Proofpoint-GUID: MLiaogtkvTyks_8K8F2iFA24uHGVnj1T X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=910 adultscore=0 bulkscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Create an 'ISM' shim layer that will provide generic functionality and declarations for ism device drivers and ism clients. Move the respective pieces from drivers/s390/net/ism_drv.* to net/ism/ When we need to distinguish between generic ism interfaces and specifically the s390 virtual pci ism device, it will be called 'ISM_vPCI'. No optimizations are done in this patch, it only moves pieces around. Following patch will further detangle ism_vpci and smc-d. Signed-off-by: Alexandra Winter --- MAINTAINERS | 7 ++ drivers/s390/net/Kconfig | 9 +-- drivers/s390/net/Makefile | 4 +- drivers/s390/net/ism_drv.c | 129 ++--------------------------- include/linux/ism.h | 8 ++ include/net/smc.h | 3 - net/Kconfig | 1 + net/Makefile | 1 + net/ism/Kconfig | 14 ++++ net/ism/Makefile | 7 ++ net/ism/ism_main.c | 162 +++++++++++++++++++++++++++++++++++++ 11 files changed, 213 insertions(+), 132 deletions(-) create mode 100644 net/ism/Kconfig create mode 100644 net/ism/Makefile create mode 100644 net/ism/ism_main.c diff --git a/MAINTAINERS b/MAINTAINERS index 4dcb849e6748..780db61f3f16 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12239,6 +12239,13 @@ F: Documentation/devicetree/bindings/hwmon/renesas,isl28022.yaml F: Documentation/hwmon/isl28022.rst F: drivers/hwmon/isl28022.c +ISM (INTERNAL SHARED MEMORY) +M: Alexandra Winter +L: netdev@vger.kernel.org +S: Supported +F: include/linux/ism.h +F: net/ism/ + ISOFS FILESYSTEM M: Jan Kara L: linux-fsdevel@vger.kernel.org diff --git a/drivers/s390/net/Kconfig b/drivers/s390/net/Kconfig index c61e6427384c..2e900d3087d4 100644 --- a/drivers/s390/net/Kconfig +++ b/drivers/s390/net/Kconfig @@ -100,15 +100,14 @@ config CCWGROUP tristate default (LCS || CTCM || QETH || SMC) -config ISM +config ISM_VPCI tristate "Support for ISM vPCI Adapter" - depends on PCI + depends on PCI && ISM imply SMC - default n + default y help Select this option if you want to use the Internal Shared Memory vPCI Adapter. The adapter can be used with the SMC network protocol. - To compile as a module choose M. The module name is ism. - If unsure, choose N. + To compile as a module choose M. The module name is ism_vpci. endmenu diff --git a/drivers/s390/net/Makefile b/drivers/s390/net/Makefile index bc55ec316adb..87461019184e 100644 --- a/drivers/s390/net/Makefile +++ b/drivers/s390/net/Makefile @@ -16,5 +16,5 @@ obj-$(CONFIG_QETH_L2) += qeth_l2.o qeth_l3-y += qeth_l3_main.o qeth_l3_sys.o obj-$(CONFIG_QETH_L3) += qeth_l3.o -ism-y := ism_drv.o -obj-$(CONFIG_ISM) += ism.o +ism_vpci-y += ism_drv.o +obj-$(CONFIG_ISM_VPCI) += ism_vpci.o diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index e36e3ea165d3..2eeccf5ef48d 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -4,7 +4,7 @@ * * Copyright IBM Corp. 2018 */ -#define KMSG_COMPONENT "ism" +#define KMSG_COMPONENT "ism-vpci" #define pr_fmt(fmt) KMSG_COMPONENT ": " fmt #include @@ -31,100 +31,7 @@ MODULE_DEVICE_TABLE(pci, ism_device_table); static debug_info_t *ism_debug_info; -#define NO_CLIENT 0xff /* must be >= MAX_CLIENTS */ -static struct ism_client *clients[MAX_CLIENTS]; /* use an array rather than */ - /* a list for fast mapping */ -static u8 max_client; -static DEFINE_MUTEX(clients_lock); static bool ism_v2_capable; -struct ism_dev_list { - struct list_head list; - struct mutex mutex; /* protects ism device list */ -}; - -static struct ism_dev_list ism_dev_list = { - .list = LIST_HEAD_INIT(ism_dev_list.list), - .mutex = __MUTEX_INITIALIZER(ism_dev_list.mutex), -}; - -static void ism_setup_forwarding(struct ism_client *client, struct ism_dev *ism) -{ - unsigned long flags; - - spin_lock_irqsave(&ism->lock, flags); - ism->subs[client->id] = client; - spin_unlock_irqrestore(&ism->lock, flags); -} - -int ism_register_client(struct ism_client *client) -{ - struct ism_dev *ism; - int i, rc = -ENOSPC; - - mutex_lock(&ism_dev_list.mutex); - mutex_lock(&clients_lock); - for (i = 0; i < MAX_CLIENTS; ++i) { - if (!clients[i]) { - clients[i] = client; - client->id = i; - if (i == max_client) - max_client++; - rc = 0; - break; - } - } - mutex_unlock(&clients_lock); - - if (i < MAX_CLIENTS) { - /* initialize with all devices that we got so far */ - list_for_each_entry(ism, &ism_dev_list.list, list) { - ism->priv[i] = NULL; - client->add(ism); - ism_setup_forwarding(client, ism); - } - } - mutex_unlock(&ism_dev_list.mutex); - - return rc; -} -EXPORT_SYMBOL_GPL(ism_register_client); - -int ism_unregister_client(struct ism_client *client) -{ - struct ism_dev *ism; - unsigned long flags; - int rc = 0; - - mutex_lock(&ism_dev_list.mutex); - list_for_each_entry(ism, &ism_dev_list.list, list) { - spin_lock_irqsave(&ism->lock, flags); - /* Stop forwarding IRQs and events */ - ism->subs[client->id] = NULL; - for (int i = 0; i < ISM_NR_DMBS; ++i) { - if (ism->sba_client_arr[i] == client->id) { - WARN(1, "%s: attempt to unregister '%s' with registered dmb(s)\n", - __func__, client->name); - rc = -EBUSY; - goto err_reg_dmb; - } - } - spin_unlock_irqrestore(&ism->lock, flags); - } - mutex_unlock(&ism_dev_list.mutex); - - mutex_lock(&clients_lock); - clients[client->id] = NULL; - if (client->id + 1 == max_client) - max_client--; - mutex_unlock(&clients_lock); - return rc; - -err_reg_dmb: - spin_unlock_irqrestore(&ism->lock, flags); - mutex_unlock(&ism_dev_list.mutex); - return rc; -} -EXPORT_SYMBOL_GPL(ism_unregister_client); static int ism_cmd(struct ism_dev *ism, void *cmd) { @@ -475,7 +382,7 @@ static void ism_handle_event(struct ism_dev *ism) entry = &ism->ieq->entry[ism->ieq_idx]; debug_event(ism_debug_info, 2, entry, sizeof(*entry)); - for (i = 0; i < max_client; ++i) { + for (i = 0; i < MAX_CLIENTS; ++i) { clt = ism->subs[i]; if (clt) clt->handle_event(ism, entry); @@ -524,7 +431,7 @@ static irqreturn_t ism_handle_irq(int irq, void *data) static int ism_dev_init(struct ism_dev *ism) { struct pci_dev *pdev = ism->pdev; - int i, ret; + int ret; ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI); if (ret <= 0) @@ -558,19 +465,7 @@ static int ism_dev_init(struct ism_dev *ism) else ism_v2_capable = false; - mutex_lock(&ism_dev_list.mutex); - mutex_lock(&clients_lock); - for (i = 0; i < max_client; ++i) { - if (clients[i]) { - clients[i]->add(ism); - ism_setup_forwarding(clients[i], ism); - } - } - mutex_unlock(&clients_lock); - - list_add(&ism->list, &ism_dev_list.list); - mutex_unlock(&ism_dev_list.mutex); - + ism_dev_register(ism); query_info(ism); return 0; @@ -649,17 +544,11 @@ static void ism_dev_exit(struct ism_dev *ism) int i; spin_lock_irqsave(&ism->lock, flags); - for (i = 0; i < max_client; ++i) + for (i = 0; i < MAX_CLIENTS; ++i) ism->subs[i] = NULL; spin_unlock_irqrestore(&ism->lock, flags); - mutex_lock(&ism_dev_list.mutex); - mutex_lock(&clients_lock); - for (i = 0; i < max_client; ++i) { - if (clients[i]) - clients[i]->remove(ism); - } - mutex_unlock(&clients_lock); + ism_dev_unregister(ism); if (ism_v2_capable) ism_del_vlan_id(ism, ISM_RESERVED_VLANID); @@ -668,8 +557,6 @@ static void ism_dev_exit(struct ism_dev *ism) free_irq(pci_irq_vector(pdev, 0), ism); kfree(ism->sba_client_arr); pci_free_irq_vectors(pdev); - list_del_init(&ism->list); - mutex_unlock(&ism_dev_list.mutex); } static void ism_remove(struct pci_dev *pdev) @@ -700,8 +587,6 @@ static int __init ism_init(void) if (!ism_debug_info) return -ENODEV; - memset(clients, 0, sizeof(clients)); - max_client = 0; debug_register_view(ism_debug_info, &debug_hex_ascii_view); ret = pci_register_driver(&ism_driver); if (ret) @@ -721,7 +606,7 @@ module_exit(ism_exit); /*************************** SMC-D Implementation *****************************/ -#if IS_ENABLED(CONFIG_SMC) +#if IS_ENABLED(CONFIG_SMC) // needed to avoid unused functions static int ism_query_rgid(struct ism_dev *ism, u64 rgid, u32 vid_valid, u32 vid) { diff --git a/include/linux/ism.h b/include/linux/ism.h index 5428edd90982..1462296e8ba7 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -9,6 +9,7 @@ #ifndef _ISM_H #define _ISM_H +#include #include struct ism_dmb { @@ -24,6 +25,7 @@ struct ism_dmb { /* Unless we gain unexpected popularity, this limit should hold for a while */ #define MAX_CLIENTS 8 +#define NO_CLIENT 0xff /* must be >= MAX_CLIENTS */ #define ISM_NR_DMBS 1920 struct ism_dev { @@ -76,6 +78,9 @@ static inline void *ism_get_priv(struct ism_dev *dev, return dev->priv[client->id]; } +int ism_dev_register(struct ism_dev *ism); +void ism_dev_unregister(struct ism_dev *ism); + static inline void ism_set_priv(struct ism_dev *dev, struct ism_client *client, void *priv) { dev->priv[client->id] = priv; @@ -87,6 +92,9 @@ int ism_unregister_dmb(struct ism_dev *dev, struct ism_dmb *dmb); int ism_move(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); +#define ISM_RESERVED_VLANID 0x1FFF +#define ISM_ERROR 0xFFFF + const struct smcd_ops *ism_get_smcd_ops(void); #endif /* _ISM_H */ diff --git a/include/net/smc.h b/include/net/smc.h index db84e4e35080..ab732b286f91 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -42,9 +42,6 @@ struct smcd_dmb { #define ISM_EVENT_GID 1 #define ISM_EVENT_SWR 2 -#define ISM_RESERVED_VLANID 0x1FFF - -#define ISM_ERROR 0xFFFF struct smcd_dev; diff --git a/net/Kconfig b/net/Kconfig index c3fca69a7c83..2dbe9655f7de 100644 --- a/net/Kconfig +++ b/net/Kconfig @@ -83,6 +83,7 @@ source "net/tls/Kconfig" source "net/xfrm/Kconfig" source "net/iucv/Kconfig" source "net/smc/Kconfig" +source "net/ism/Kconfig" source "net/xdp/Kconfig" config NET_HANDSHAKE diff --git a/net/Makefile b/net/Makefile index 60ed5190eda8..6f06cf00bfbb 100644 --- a/net/Makefile +++ b/net/Makefile @@ -51,6 +51,7 @@ obj-$(CONFIG_TIPC) += tipc/ obj-$(CONFIG_NETLABEL) += netlabel/ obj-$(CONFIG_IUCV) += iucv/ obj-$(CONFIG_SMC) += smc/ +obj-$(CONFIG_ISM) += ism/ obj-$(CONFIG_RFKILL) += rfkill/ obj-$(CONFIG_NET_9P) += 9p/ obj-$(CONFIG_CAIF) += caif/ diff --git a/net/ism/Kconfig b/net/ism/Kconfig new file mode 100644 index 000000000000..4329489cc1e9 --- /dev/null +++ b/net/ism/Kconfig @@ -0,0 +1,14 @@ +# SPDX-License-Identifier: GPL-2.0 +config ISM + tristate "ISM support" + default n + help + Internal Shared Memory (ISM) + A communication method that uses common physical memory for + synchronous direct access into a remote buffer. + + Select this option to provide the abstraction layer between + ISM devices and ISM users like the SMC protocol. + + To compile as a module choose M. The module name is ism. + If unsure, choose N. diff --git a/net/ism/Makefile b/net/ism/Makefile new file mode 100644 index 000000000000..b752baf72003 --- /dev/null +++ b/net/ism/Makefile @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# ISM class module +# + +ism-y += ism_main.o +obj-$(CONFIG_ISM) += ism.o diff --git a/net/ism/ism_main.c b/net/ism/ism_main.c new file mode 100644 index 000000000000..268408dbd691 --- /dev/null +++ b/net/ism/ism_main.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Internal Shared Memory + * + * Implementation of the ISM class module + * + * Copyright IBM Corp. 2024 + */ +#define KMSG_COMPONENT "ism" +#define pr_fmt(fmt) KMSG_COMPONENT ": " fmt + +#include +#include +#include +#include + +MODULE_DESCRIPTION("Internal Shared Memory class"); +MODULE_LICENSE("GPL"); + +static struct ism_client *clients[MAX_CLIENTS]; /* use an array rather than */ + /* a list for fast mapping */ +static u8 max_client; +static DEFINE_MUTEX(clients_lock); +struct ism_dev_list { + struct list_head list; + struct mutex mutex; /* protects ism device list */ +}; + +static struct ism_dev_list ism_dev_list = { + .list = LIST_HEAD_INIT(ism_dev_list.list), + .mutex = __MUTEX_INITIALIZER(ism_dev_list.mutex), +}; + +static void ism_setup_forwarding(struct ism_client *client, struct ism_dev *ism) +{ + unsigned long flags; + + spin_lock_irqsave(&ism->lock, flags); + ism->subs[client->id] = client; + spin_unlock_irqrestore(&ism->lock, flags); +} + +int ism_register_client(struct ism_client *client) +{ + struct ism_dev *ism; + int i, rc = -ENOSPC; + + mutex_lock(&ism_dev_list.mutex); + mutex_lock(&clients_lock); + for (i = 0; i < MAX_CLIENTS; ++i) { + if (!clients[i]) { + clients[i] = client; + client->id = i; + if (i == max_client) + max_client++; + rc = 0; + break; + } + } + mutex_unlock(&clients_lock); + + if (i < MAX_CLIENTS) { + /* initialize with all devices that we got so far */ + list_for_each_entry(ism, &ism_dev_list.list, list) { + ism->priv[i] = NULL; + client->add(ism); + ism_setup_forwarding(client, ism); + } + } + mutex_unlock(&ism_dev_list.mutex); + + return rc; +} +EXPORT_SYMBOL_GPL(ism_register_client); + +int ism_unregister_client(struct ism_client *client) +{ + struct ism_dev *ism; + unsigned long flags; + int rc = 0; + + mutex_lock(&ism_dev_list.mutex); + list_for_each_entry(ism, &ism_dev_list.list, list) { + spin_lock_irqsave(&ism->lock, flags); + /* Stop forwarding IRQs and events */ + ism->subs[client->id] = NULL; + for (int i = 0; i < ISM_NR_DMBS; ++i) { + if (ism->sba_client_arr[i] == client->id) { + WARN(1, "%s: attempt to unregister '%s' with registered dmb(s)\n", + __func__, client->name); + rc = -EBUSY; + goto err_reg_dmb; + } + } + spin_unlock_irqrestore(&ism->lock, flags); + } + mutex_unlock(&ism_dev_list.mutex); + + mutex_lock(&clients_lock); + clients[client->id] = NULL; + if (client->id + 1 == max_client) + max_client--; + mutex_unlock(&clients_lock); + return rc; + +err_reg_dmb: + spin_unlock_irqrestore(&ism->lock, flags); + mutex_unlock(&ism_dev_list.mutex); + return rc; +} +EXPORT_SYMBOL_GPL(ism_unregister_client); + +int ism_dev_register(struct ism_dev *ism) +{ + int i; + + mutex_lock(&ism_dev_list.mutex); + mutex_lock(&clients_lock); + for (i = 0; i < max_client; ++i) { + if (clients[i]) { + clients[i]->add(ism); + ism_setup_forwarding(clients[i], ism); + } + } + mutex_unlock(&clients_lock); + list_add(&ism->list, &ism_dev_list.list); + mutex_unlock(&ism_dev_list.mutex); + + return 0; +} +EXPORT_SYMBOL_GPL(ism_dev_register); + +void ism_dev_unregister(struct ism_dev *ism) +{ + int i; + + mutex_lock(&ism_dev_list.mutex); + mutex_lock(&clients_lock); + for (i = 0; i < max_client; ++i) { + if (clients[i]) + clients[i]->remove(ism); + } + mutex_unlock(&clients_lock); + list_del_init(&ism->list); + mutex_unlock(&ism_dev_list.mutex); +} +EXPORT_SYMBOL_GPL(ism_dev_unregister); + +static int __init ism_init(void) +{ + memset(clients, 0, sizeof(clients)); + max_client = 0; + + return 0; +} + +static void __exit ism_exit(void) +{ +} + +module_init(ism_init); +module_exit(ism_exit); From patchwork Wed Jan 15 19:55:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940864 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F087C1D86F2; Wed, 15 Jan 2025 19:55:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970944; cv=none; b=aZoZKjNIBfW0P5FYzo6ycnkx+wT0vfjB9IyoOO1v0iub9ejAkIsLaXUR3RBVGsvv3g1sLLgXBIBIN/q5iFYCHChWJn6ozOa30akFb+sf+XU0rzjdG53OF2xSErdygXejJvW8O7JeitJIwUVe3uvtm0VgWUUrWSAmcmLlp3RUvVU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970944; c=relaxed/simple; bh=hCjK59icEPJgQHXN30wyFrnd90eWmotLxsKPmA4AWhI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eB4F4JCvsEk9fay3MRhitRjnKfG6mAWMw9i3RiVbc88vyvynQCblOp+urzivusyw34G02mSK4DM/ZnFxYIaD97NngG/bX8zkaoJ2RZv5V3B3c0B+OjXDiQkhkng9DN5DmVwnZZPIyt2m6sKaed2hckYSgDug+Pnz7aQ1hojjt48= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=LJJEwUug; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="LJJEwUug" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX5DG024451; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=o9Bod7I6wEirlR+0z c2JpYns0G5mDBKkE489MdQ45rk=; b=LJJEwUug/yb6YQGaWa6V80N0hOM2Cs9HY IudnXuJO4DcuR7ERCv2n2TlT3h3js7oICer+kdQp8lazJs60KkliMI4hKzqzaRCO 98DZBhz1Jjcf7h7IZkZFmr7+DjBmfThID2VAKF0dpv6+pKvf9zySnOFqa/Bi3U4o D+zhKhdxIr4EJnic4oYRhsJ2Evg4vpWYRxIaLAHOHQ9XMMaj9fze1N+i8KwxQ+FT 2X8hL8vDFjJo4yMPxmWrsnAn7WO/r1+050uetXyXLuInmh2XR+ot0/0D0xHOuscT 1f40Xm0cJDV3HyqQ43XW3zs7LNbsT6FTzf9A1JpVh65EDXosiO1Kg== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44622hw1qx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from m0353729.ppops.net (m0353729.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJjvQk017712; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 44622hw1qr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FIqLZp016490; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4445p1su9d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtRB035782996 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:27 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A28362004B; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A97320040; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 48ECAE0D81; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 2/7] net/ism: Remove dependencies between ISM_VPCI and SMC Date: Wed, 15 Jan 2025 20:55:22 +0100 Message-ID: <20250115195527.2094320-3-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: S46kHLk4hUyUIJ6M_-Mlscp471qKswDG X-Proofpoint-ORIG-GUID: Tz712k5wofPJ9Hv6BLPoU8DCjOmORGwN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 impostorscore=0 bulkscore=0 lowpriorityscore=0 phishscore=0 suspectscore=0 priorityscore=1501 mlxlogscore=876 malwarescore=0 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC The modules ISM_VPCI and SMC should not depend on each other, instead they should both depend on the ISM layer module. use only ism_dmb: Now that SMC depends on ISM, we can safely remove the duplicate declaration of smcd_dmb and use only ism_dmb. Move smcd_ops away from ism_drv: move smcd_ops from drivers/s390/net/ism_drv.c to net/smc/smc_ism.c Less exported functions, no more dependencies between ISM_VPCI and SMC. Once ism_loopback is also moved to ism layer, a follow on patch can use ism_ops directly and remove smcd_ops. Now the ISM_VPCI module no longer needs to imply SMC. Note: - This patch temporarily moves smcd_gid to ism.h, a follow on patch (uuid_t gid) will restore this. - Added a comment that vlan handling in ism_drv.c and smc is incomplete. Should be fixed by a follow-on patch. Signed-off-by: Alexandra Winter --- drivers/s390/net/Kconfig | 1 - drivers/s390/net/ism.h | 1 - drivers/s390/net/ism_drv.c | 236 +++++++++++++------------------------ include/linux/ism.h | 146 +++++++++++++++++------ include/net/smc.h | 31 ++--- net/smc/smc_ism.c | 123 ++++++++++++++++++- net/smc/smc_loopback.c | 6 +- 7 files changed, 319 insertions(+), 225 deletions(-) diff --git a/drivers/s390/net/Kconfig b/drivers/s390/net/Kconfig index 2e900d3087d4..9bb3cc186510 100644 --- a/drivers/s390/net/Kconfig +++ b/drivers/s390/net/Kconfig @@ -103,7 +103,6 @@ config CCWGROUP config ISM_VPCI tristate "Support for ISM vPCI Adapter" depends on PCI && ISM - imply SMC default y help Select this option if you want to use the Internal Shared Memory diff --git a/drivers/s390/net/ism.h b/drivers/s390/net/ism.h index 047fa6101555..8b56e1d82e6b 100644 --- a/drivers/s390/net/ism.h +++ b/drivers/s390/net/ism.h @@ -6,7 +6,6 @@ #include #include #include -#include #include #define UTIL_STR_LEN 16 diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index 2eeccf5ef48d..112e0d67cdd6 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -191,11 +191,28 @@ static int ism_read_local_gid(struct ism_dev *ism) if (ret) goto out; - ism->local_gid = cmd.response.gid; + ism->gid.gid = cmd.response.gid; + ism->gid.gid_ext = 0; out: return ret; } +static int ism_query_rgid(struct ism_dev *ism, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) +{ + union ism_query_rgid cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.request.hdr.cmd = ISM_QUERY_RGID; + cmd.request.hdr.len = sizeof(cmd.request); + + cmd.request.rgid = rgid->gid; + cmd.request.vlan_valid = vid_valid; + cmd.request.vlan_id = vid; + + return ism_cmd(ism, &cmd); +} + static void ism_free_dmb(struct ism_dev *ism, struct ism_dmb *dmb) { clear_bit(dmb->sba_idx, ism->sba_bitmap); @@ -251,8 +268,8 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) return rc; } -int ism_register_dmb(struct ism_dev *ism, struct ism_dmb *dmb, - struct ism_client *client) +static int ism_register_dmb(struct ism_dev *ism, struct ism_dmb *dmb, + struct ism_client *client) { union ism_reg_dmb cmd; unsigned long flags; @@ -285,9 +302,8 @@ int ism_register_dmb(struct ism_dev *ism, struct ism_dmb *dmb, out: return ret; } -EXPORT_SYMBOL_GPL(ism_register_dmb); -int ism_unregister_dmb(struct ism_dev *ism, struct ism_dmb *dmb) +static int ism_unregister_dmb(struct ism_dev *ism, struct ism_dmb *dmb) { union ism_unreg_dmb cmd; unsigned long flags; @@ -311,7 +327,6 @@ int ism_unregister_dmb(struct ism_dev *ism, struct ism_dmb *dmb) out: return ret; } -EXPORT_SYMBOL_GPL(ism_unregister_dmb); static int ism_add_vlan_id(struct ism_dev *ism, u64 vlan_id) { @@ -339,14 +354,42 @@ static int ism_del_vlan_id(struct ism_dev *ism, u64 vlan_id) return ism_cmd(ism, &cmd); } +static int ism_set_vlan_required(struct ism_dev *ism) +{ + return ism_cmd_simple(ism, ISM_SET_VLAN); +} + +static int ism_reset_vlan_required(struct ism_dev *ism) +{ + return ism_cmd_simple(ism, ISM_RESET_VLAN); +} + +static int ism_signal_ieq(struct ism_dev *ism, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info) +{ + union ism_sig_ieq cmd; + + memset(&cmd, 0, sizeof(cmd)); + cmd.request.hdr.cmd = ISM_SIGNAL_IEQ; + cmd.request.hdr.len = sizeof(cmd.request); + + cmd.request.rgid = rgid->gid; + cmd.request.trigger_irq = trigger_irq; + cmd.request.event_code = event_code; + cmd.request.info = info; + + return ism_cmd(ism, &cmd); +} + static unsigned int max_bytes(unsigned int start, unsigned int len, unsigned int boundary) { return min(boundary - (start & (boundary - 1)), len); } -int ism_move(struct ism_dev *ism, u64 dmb_tok, unsigned int idx, bool sf, - unsigned int offset, void *data, unsigned int size) +static int ism_move(struct ism_dev *ism, u64 dmb_tok, unsigned int idx, + bool sf, unsigned int offset, void *data, + unsigned int size) { unsigned int bytes; u64 dmb_req; @@ -368,7 +411,19 @@ int ism_move(struct ism_dev *ism, u64 dmb_tok, unsigned int idx, bool sf, return 0; } -EXPORT_SYMBOL_GPL(ism_move); + +static int ism_supports_v2(void) +{ + return ism_v2_capable; +} + +static u16 ism_get_chid(struct ism_dev *ism) +{ + if (!ism || !ism->pdev) + return 0; + + return to_zpci(ism->pdev)->pchid; +} static void ism_handle_event(struct ism_dev *ism) { @@ -428,6 +483,20 @@ static irqreturn_t ism_handle_irq(int irq, void *data) return IRQ_HANDLED; } +static const struct ism_ops ism_vp_ops = { + .query_remote_gid = ism_query_rgid, + .register_dmb = ism_register_dmb, + .unregister_dmb = ism_unregister_dmb, + .add_vlan_id = ism_add_vlan_id, + .del_vlan_id = ism_del_vlan_id, + .set_vlan_required = ism_set_vlan_required, + .reset_vlan_required = ism_reset_vlan_required, + .signal_event = ism_signal_ieq, + .move_data = ism_move, + .supports_v2 = ism_supports_v2, + .get_chid = ism_get_chid, +}; + static int ism_dev_init(struct ism_dev *ism) { struct pci_dev *pdev = ism->pdev; @@ -465,6 +534,8 @@ static int ism_dev_init(struct ism_dev *ism) else ism_v2_capable = false; + ism->ops = &ism_vp_ops; + ism_dev_register(ism); query_info(ism); return 0; @@ -603,150 +674,3 @@ static void __exit ism_exit(void) module_init(ism_init); module_exit(ism_exit); - -/*************************** SMC-D Implementation *****************************/ - -#if IS_ENABLED(CONFIG_SMC) // needed to avoid unused functions -static int ism_query_rgid(struct ism_dev *ism, u64 rgid, u32 vid_valid, - u32 vid) -{ - union ism_query_rgid cmd; - - memset(&cmd, 0, sizeof(cmd)); - cmd.request.hdr.cmd = ISM_QUERY_RGID; - cmd.request.hdr.len = sizeof(cmd.request); - - cmd.request.rgid = rgid; - cmd.request.vlan_valid = vid_valid; - cmd.request.vlan_id = vid; - - return ism_cmd(ism, &cmd); -} - -static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, - u32 vid_valid, u32 vid) -{ - return ism_query_rgid(smcd->priv, rgid->gid, vid_valid, vid); -} - -static int smcd_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, - void *client) -{ - return ism_register_dmb(smcd->priv, (struct ism_dmb *)dmb, client); -} - -static int smcd_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) -{ - return ism_unregister_dmb(smcd->priv, (struct ism_dmb *)dmb); -} - -static int smcd_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) -{ - return ism_add_vlan_id(smcd->priv, vlan_id); -} - -static int smcd_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) -{ - return ism_del_vlan_id(smcd->priv, vlan_id); -} - -static int smcd_set_vlan_required(struct smcd_dev *smcd) -{ - return ism_cmd_simple(smcd->priv, ISM_SET_VLAN); -} - -static int smcd_reset_vlan_required(struct smcd_dev *smcd) -{ - return ism_cmd_simple(smcd->priv, ISM_RESET_VLAN); -} - -static int ism_signal_ieq(struct ism_dev *ism, u64 rgid, u32 trigger_irq, - u32 event_code, u64 info) -{ - union ism_sig_ieq cmd; - - memset(&cmd, 0, sizeof(cmd)); - cmd.request.hdr.cmd = ISM_SIGNAL_IEQ; - cmd.request.hdr.len = sizeof(cmd.request); - - cmd.request.rgid = rgid; - cmd.request.trigger_irq = trigger_irq; - cmd.request.event_code = event_code; - cmd.request.info = info; - - return ism_cmd(ism, &cmd); -} - -static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, - u32 trigger_irq, u32 event_code, u64 info) -{ - return ism_signal_ieq(smcd->priv, rgid->gid, - trigger_irq, event_code, info); -} - -static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, - bool sf, unsigned int offset, void *data, - unsigned int size) -{ - return ism_move(smcd->priv, dmb_tok, idx, sf, offset, data, size); -} - -static int smcd_supports_v2(void) -{ - return ism_v2_capable; -} - -static u64 ism_get_local_gid(struct ism_dev *ism) -{ - return ism->local_gid; -} - -static void smcd_get_local_gid(struct smcd_dev *smcd, - struct smcd_gid *smcd_gid) -{ - smcd_gid->gid = ism_get_local_gid(smcd->priv); - smcd_gid->gid_ext = 0; -} - -static u16 ism_get_chid(struct ism_dev *ism) -{ - if (!ism || !ism->pdev) - return 0; - - return to_zpci(ism->pdev)->pchid; -} - -static u16 smcd_get_chid(struct smcd_dev *smcd) -{ - return ism_get_chid(smcd->priv); -} - -static inline struct device *smcd_get_dev(struct smcd_dev *dev) -{ - struct ism_dev *ism = dev->priv; - - return &ism->dev; -} - -static const struct smcd_ops ism_ops = { - .query_remote_gid = smcd_query_rgid, - .register_dmb = smcd_register_dmb, - .unregister_dmb = smcd_unregister_dmb, - .add_vlan_id = smcd_add_vlan_id, - .del_vlan_id = smcd_del_vlan_id, - .set_vlan_required = smcd_set_vlan_required, - .reset_vlan_required = smcd_reset_vlan_required, - .signal_event = smcd_signal_ieq, - .move_data = smcd_move, - .supports_v2 = smcd_supports_v2, - .get_local_gid = smcd_get_local_gid, - .get_chid = smcd_get_chid, - .get_dev = smcd_get_dev, -}; - -const struct smcd_ops *ism_get_smcd_ops(void) -{ - return &ism_ops; -} -EXPORT_SYMBOL_GPL(ism_get_smcd_ops); -#endif diff --git a/include/linux/ism.h b/include/linux/ism.h index 1462296e8ba7..ede1a40b408e 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -12,6 +12,7 @@ #include #include +/* The remote peer rgid can use dmb_tok to write into this buffer. */ struct ism_dmb { u64 dmb_tok; u64 rgid; @@ -23,30 +24,9 @@ struct ism_dmb { dma_addr_t dma_addr; }; -/* Unless we gain unexpected popularity, this limit should hold for a while */ -#define MAX_CLIENTS 8 -#define NO_CLIENT 0xff /* must be >= MAX_CLIENTS */ -#define ISM_NR_DMBS 1920 - -struct ism_dev { - spinlock_t lock; /* protects the ism device */ - struct list_head list; - struct pci_dev *pdev; - - struct ism_sba *sba; - dma_addr_t sba_dma_addr; - DECLARE_BITMAP(sba_bitmap, ISM_NR_DMBS); - u8 *sba_client_arr; /* entries are indices into 'clients' array */ - void *priv[MAX_CLIENTS]; - - struct ism_eq *ieq; - dma_addr_t ieq_dma_addr; - - struct device dev; - u64 local_gid; - int ieq_idx; - - struct ism_client *subs[MAX_CLIENTS]; +struct smcd_gid { + u64 gid; + u64 gid_ext; }; struct ism_event { @@ -57,6 +37,12 @@ struct ism_event { u64 info; }; +#define ISM_EVENT_DMB 0 +#define ISM_EVENT_GID 1 +#define ISM_EVENT_SWR 2 + +struct ism_dev; + struct ism_client { const char *name; void (*add)(struct ism_dev *dev); @@ -73,28 +59,116 @@ struct ism_client { int ism_register_client(struct ism_client *client); int ism_unregister_client(struct ism_client *client); -static inline void *ism_get_priv(struct ism_dev *dev, - struct ism_client *client) { - return dev->priv[client->id]; -} + +/* Mandatory operations for all ism devices: + * int (*query_remote_gid)(struct ism_dev *dev, struct smcd_gid *rgid, + * u32 vid_valid, u32 vid); + * Query whether remote GID rgid is reachable via this device and this + * vlan id. Vlan id is only checked if vid_valid != 0. + * + * int (*register_dmb)(struct ism_dev *dev, struct ism_dmb *dmb, + * void *client); + * Register an ism_dmb buffer for this device and this client. + * + * int (*unregister_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); + * Unregister an ism_dmb buffer + * + * int (*move_data)(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, + * bool sf, unsigned int offset, void *data, + * unsigned int size); + * Use dev to write data of size at offset into a remote dmb + * identified by dmb_tok and idx. If signal flag (sf) then signal + * the remote peer that data has arrived in this dmb. + * + * int (*supports_v2)(void); + * + * u16 (*get_chid)(struct ism_dev *dev); + * Returns ism fabric identifier (channel id) of this device. + * Only devices on the same ism fabric can communicate. + * chid is unique per HW system, except for 0xFFFF, which denotes + * an ism_loopback device that can only communicate with itself. + * Use chid for fast negative checks, but only query_remote_gid() + * can give a reliable positive answer. + * + * struct device* (*get_dev)(struct ism_dev *dev); + * + * Optional operations: + * int (*add_vlan_id)(struct ism_dev *dev, u64 vlan_id); + * int (*del_vlan_id)(struct ism_dev *dev, u64 vlan_id); + * int (*set_vlan_required)(struct ism_dev *dev); + * int (*reset_vlan_required)(struct ism_dev *dev); + * VLAN handling is broken - don't use it + * Ability to assign dmbs to VLANs is missing + * - do we really want / need this? + * + * int (*signal_event)(struct ism_dev *dev, struct smcd_gid *rgid, + * u32 trigger_irq, u32 event_code, u64 info); + * Send a control event into the event queue of a remote gid (rgid) + * with (1) or without (0) triggering an interrupt at the remote gid. + */ + +struct ism_ops { + int (*query_remote_gid)(struct ism_dev *dev, struct smcd_gid *rgid, + u32 vid_valid, u32 vid); + int (*register_dmb)(struct ism_dev *dev, struct ism_dmb *dmb, + struct ism_client *client); + int (*unregister_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); + int (*move_data)(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, + bool sf, unsigned int offset, void *data, + unsigned int size); + int (*supports_v2)(void); + u16 (*get_chid)(struct ism_dev *dev); + struct device* (*get_dev)(struct ism_dev *dev); + + /* optional operations */ + int (*add_vlan_id)(struct ism_dev *dev, u64 vlan_id); + int (*del_vlan_id)(struct ism_dev *dev, u64 vlan_id); + int (*set_vlan_required)(struct ism_dev *dev); + int (*reset_vlan_required)(struct ism_dev *dev); + int (*signal_event)(struct ism_dev *dev, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info); +}; + +/* Unless we gain unexpected popularity, this limit should hold for a while */ +#define MAX_CLIENTS 8 +#define NO_CLIENT 0xff /* must be >= MAX_CLIENTS */ +#define ISM_NR_DMBS 1920 + +struct ism_dev { + const struct ism_ops *ops; + spinlock_t lock; /* protects the ism device */ + struct list_head list; + struct pci_dev *pdev; + + struct ism_sba *sba; + dma_addr_t sba_dma_addr; + DECLARE_BITMAP(sba_bitmap, ISM_NR_DMBS); + u8 *sba_client_arr; /* entries are indices into 'clients' array */ + void *priv[MAX_CLIENTS]; + + struct ism_eq *ieq; + dma_addr_t ieq_dma_addr; + + struct device dev; + struct smcd_gid gid; + int ieq_idx; + + struct ism_client *subs[MAX_CLIENTS]; +}; int ism_dev_register(struct ism_dev *ism); void ism_dev_unregister(struct ism_dev *ism); +static inline void *ism_get_priv(struct ism_dev *dev, + struct ism_client *client) { + return dev->priv[client->id]; +} static inline void ism_set_priv(struct ism_dev *dev, struct ism_client *client, void *priv) { dev->priv[client->id] = priv; } -int ism_register_dmb(struct ism_dev *dev, struct ism_dmb *dmb, - struct ism_client *client); -int ism_unregister_dmb(struct ism_dev *dev, struct ism_dmb *dmb); -int ism_move(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, - unsigned int offset, void *data, unsigned int size); - #define ISM_RESERVED_VLANID 0x1FFF #define ISM_ERROR 0xFFFF -const struct smcd_ops *ism_get_smcd_ops(void); - #endif /* _ISM_H */ diff --git a/include/net/smc.h b/include/net/smc.h index ab732b286f91..3d20c6c05056 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -27,35 +27,20 @@ struct smc_hashinfo { }; /* SMCD/ISM device driver interface */ -struct smcd_dmb { - u64 dmb_tok; - u64 rgid; - u32 dmb_len; - u32 sba_idx; - u32 vlan_valid; - u32 vlan_id; - void *cpu_addr; - dma_addr_t dma_addr; -}; - -#define ISM_EVENT_DMB 0 -#define ISM_EVENT_GID 1 -#define ISM_EVENT_SWR 2 - struct smcd_dev; -struct smcd_gid { - u64 gid; - u64 gid_ext; -}; - +//struct smcd_gid { +// u64 gid; +// u64 gid_ext; +//}; +// struct smcd_ops { int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, u32 vid_valid, u32 vid); - int (*register_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb, + int (*register_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb, void *client); - int (*unregister_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*unregister_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb); int (*move_data)(struct smcd_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); @@ -72,7 +57,7 @@ struct smcd_ops { int (*signal_event)(struct smcd_dev *dev, struct smcd_gid *rgid, u32 trigger_irq, u32 event_code, u64 info); int (*support_dmb_nocopy)(struct smcd_dev *dev); - int (*attach_dmb)(struct smcd_dev *dev, struct smcd_dmb *dmb); + int (*attach_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb); int (*detach_dmb)(struct smcd_dev *dev, u64 token); }; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 84f98e18c7db..6fbacad02f23 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -207,7 +207,7 @@ int smc_ism_put_vlan(struct smcd_dev *smcd, unsigned short vlanid) int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) { - struct smcd_dmb dmb; + struct ism_dmb dmb; int rc = 0; if (!dmb_desc->dma_addr) @@ -231,7 +231,7 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { - struct smcd_dmb dmb; + struct ism_dmb dmb; int rc; memset(&dmb, 0, sizeof(dmb)); @@ -263,7 +263,7 @@ bool smc_ism_support_dmb_nocopy(struct smcd_dev *smcd) int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, struct smc_buf_desc *dmb_desc) { - struct smcd_dmb dmb; + struct ism_dmb dmb; int rc = 0; if (!dev->ops->attach_dmb) @@ -481,9 +481,122 @@ static struct smcd_dev *smcd_alloc_dev(struct device *parent, const char *name, return smcd; } +static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 vid_valid, u32 vid) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->query_remote_gid(ism, rgid, vid_valid, vid); +} + +static int smcd_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, + void *client) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->register_dmb(ism, dmb, (struct ism_client *)client); +} + +static int smcd_unregister_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->unregister_dmb(ism, dmb); +} + +static int smcd_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->add_vlan_id(ism, vlan_id); +} + +static int smcd_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->del_vlan_id(ism, vlan_id); +} + +static int smcd_set_vlan_required(struct smcd_dev *smcd) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->set_vlan_required(ism); +} + +static int smcd_reset_vlan_required(struct smcd_dev *smcd) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->reset_vlan_required(ism); +} + +static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, + u32 trigger_irq, u32 event_code, u64 info) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->signal_event(ism, rgid, + trigger_irq, event_code, info); +} + +static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, + bool sf, unsigned int offset, void *data, + unsigned int size) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->move_data(ism, dmb_tok, idx, sf, offset, data, size); +} + +static int smcd_supports_v2(void) +{ + return smc_ism_v2_capable; +} + +static void smcd_get_local_gid(struct smcd_dev *smcd, + struct smcd_gid *smcd_gid) +{ + struct ism_dev *ism = smcd->priv; + + smcd_gid->gid = ism->gid.gid; + smcd_gid->gid_ext = ism->gid.gid_ext; +} + +static u16 smcd_get_chid(struct smcd_dev *smcd) +{ + struct ism_dev *ism = smcd->priv; + + return ism->ops->get_chid(ism); +} + +static inline struct device *smcd_get_dev(struct smcd_dev *dev) +{ + struct ism_dev *ism = dev->priv; + + return &ism->dev; +} + +static const struct smcd_ops ism_smcd_ops = { + .query_remote_gid = smcd_query_rgid, + .register_dmb = smcd_register_dmb, + .unregister_dmb = smcd_unregister_dmb, + .add_vlan_id = smcd_add_vlan_id, + .del_vlan_id = smcd_del_vlan_id, + .set_vlan_required = smcd_set_vlan_required, + .reset_vlan_required = smcd_reset_vlan_required, + .signal_event = smcd_signal_ieq, + .move_data = smcd_move, + .supports_v2 = smcd_supports_v2, + .get_local_gid = smcd_get_local_gid, + .get_chid = smcd_get_chid, + .get_dev = smcd_get_dev, +}; + static void smcd_register_dev(struct ism_dev *ism) { - const struct smcd_ops *ops = ism_get_smcd_ops(); + const struct smcd_ops *ops = &ism_smcd_ops; struct smcd_dev *smcd, *fentry; if (!ops) @@ -499,7 +612,7 @@ static void smcd_register_dev(struct ism_dev *ism) if (smc_pnetid_by_dev_port(&ism->pdev->dev, 0, smcd->pnetid)) smc_pnetid_by_table_smcd(smcd); - if (smcd->ops->supports_v2()) + if (ism->ops->supports_v2()) smc_ism_set_v2_capable(); mutex_lock(&smcd_dev_list.mutex); /* sort list: diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c index 3c5f64ca4115..c4020653ae20 100644 --- a/net/smc/smc_loopback.c +++ b/net/smc/smc_loopback.c @@ -51,7 +51,7 @@ static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, return 0; } -static int smc_lo_register_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb, +static int smc_lo_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, void *client_priv) { struct smc_lo_dmb_node *dmb_node, *tmp_node; @@ -129,7 +129,7 @@ static void __smc_lo_unregister_dmb(struct smc_lo_dev *ldev, wake_up(&ldev->ldev_release); } -static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) { struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; @@ -158,7 +158,7 @@ static int smc_lo_support_dmb_nocopy(struct smcd_dev *smcd) return SMC_LO_SUPPORT_NOCOPY; } -static int smc_lo_attach_dmb(struct smcd_dev *smcd, struct smcd_dmb *dmb) +static int smc_lo_attach_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) { struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; struct smc_lo_dev *ldev = smcd->priv; From patchwork Wed Jan 15 19:55:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940862 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67C681D86ED; Wed, 15 Jan 2025 19:55:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970943; cv=none; b=Z7zjf9xIot05Pj8D/q7D5GVgNITYGhS41nuNhlXyckxGWpCwvxuRd5Kr6qM2bFdK/nRFPZHys/qu5+oMbTIMmJw4x4gus+XidX/RXNsPXj3GuApc9dv8/l0XWlb2ch2pjffb9F8FpbIraiq8rDbGN+CdHB38Q/tA9TExQ9dcUOI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970943; c=relaxed/simple; bh=0xIbJg6ktBGbg8LQpYCIlByVKbE247d+Z8EHLMt3Ccw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ex40dfB0fpCvebgYvFCCnWsl1sOeC4VfClyFa/tQ6ak6dzrLc6KATW36Ti3ET1tPHhosSeO9dAZSbzilr0EFRAIrhoDiLP04ArCDFEV6NVmouZK8LyR25uZsj/iQpuYlF5Zyq789ROzJVZOFwc1XdXXHDd1QECMx/jnf6A7w7J4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=U8bWHOnk; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="U8bWHOnk" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX4fp024067; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=z6lUahnBOLW6mWPDB +99h59uUIwQEQOGhnQTZ92akko=; b=U8bWHOnk6BZoycUPzQXCOGvj7+ETGdThp lLHISqXWUfBykD+pM6Ft0FTeD16OrQ61YN90ThhrfOeu/OWGuBON1D+Q7xzywRin VsJRIxA/7sQTcVKSyYUII/J3hBKBt5Ew15VD7FHLkpct+1dOSLCfVtdd16v2qr7K 5qqb2uvSPxEstPbEVCIuV5aIQZ/WbvjLspO01a+CR9e1DIzRo6ILBS7yTaFUqNJv mPNtIK47uTUI4H9/IueGVMI+r51M0HVulZEttSentJhxM1TVE6qSqzAthJZocLPE EDIjXRnQyCH/fxH0TV8QXgQe3MeGm49pTC8g3TY1ZR/q8ctDWkGJA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4469733c40-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJpqHk030708; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4469733c3v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FIi3YT001330; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 44456k1xbm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtRxZ56820008 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:27 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AC2DD2004F; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8B37A2004B; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 4BB95E0D82; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 3/7] net/ism: Use uuid_t for ISM GID Date: Wed, 15 Jan 2025 20:55:23 +0100 Message-ID: <20250115195527.2094320-4-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: wuVzr37PZmx3A4zghoqmOg1YOekFppic X-Proofpoint-GUID: I6TvsXNnODiI5tIOD_mL6c69h3bRzsbC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 priorityscore=1501 phishscore=0 clxscore=1015 suspectscore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=523 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC SMC uses 64 Bit and 128 Bit Global Identifiers (GIDs) that need to be sent via the SMC protocol. When integers are used network endianness and host endianness need to be considered. Avoid this in the ISM layer by using uuid_t byte arrays. Follow on patches could do the same change for SMC, for now conversion helper functions are introduced. ISM-vPCI devices provide 64 Bit GIDs. Map them to ISM uuid_t GIDs like this: _________________________________________ | 64 Bit ISM-vPCI GID | 00000000_00000000 | ----------------------------------------- If interpreted as UUID, this would be interpreted as th UIID variant, that is reserved for NCS backward compatibility. So it will not collide with UUIDs that were generated according to the standard. Future ISM devices, shall use real UUIDs as 128 Bit GIDs. Note: - In this RFC patch smcd_gid is now moved back to smc.h, future patchset should avoid that. - ism_dmb and ism_event structs still contain 64 Bit rgid and info fields. A future patch could change them to uuid_t gids. This does not break anything, because ism_loopback does not use them. Signed-off-by: Alexandra Winter --- drivers/s390/net/ism.h | 9 +++++++++ drivers/s390/net/ism_drv.c | 16 ++++++++-------- include/linux/ism.h | 16 ++++++---------- include/net/smc.h | 12 ++++++------ net/smc/smc_ism.c | 13 ++++++++----- net/smc/smc_ism.h | 21 +++++++++++++++++++++ 6 files changed, 58 insertions(+), 29 deletions(-) diff --git a/drivers/s390/net/ism.h b/drivers/s390/net/ism.h index 8b56e1d82e6b..61cf10334170 100644 --- a/drivers/s390/net/ism.h +++ b/drivers/s390/net/ism.h @@ -64,6 +64,15 @@ union ism_reg_ieq { } response; } __aligned(16); +/* ISM-vPCI devices provide 64 Bit GIDs + * Map them to ISM UUID GIDs like this: + * _________________________________________ + * | 64 Bit ISM-vPCI GID | 00000000_00000000 | + * ----------------------------------------- + * This will be interpreted as a UIID variant, that is reserved + * for NCS backward compatibility. So it will not collide with + * proper UUIDs. + */ union ism_read_gid { struct { struct ism_req_hdr hdr; diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index 112e0d67cdd6..ab70debbdd9d 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -191,14 +191,14 @@ static int ism_read_local_gid(struct ism_dev *ism) if (ret) goto out; - ism->gid.gid = cmd.response.gid; - ism->gid.gid_ext = 0; + memset(&ism->gid, 0, sizeof(ism->gid)); + memcpy(&ism->gid, &cmd.response.gid, sizeof(cmd.response.gid)); out: return ret; } -static int ism_query_rgid(struct ism_dev *ism, struct smcd_gid *rgid, - u32 vid_valid, u32 vid) +static int ism_query_rgid(struct ism_dev *ism, uuid_t *rgid, u32 vid_valid, + u32 vid) { union ism_query_rgid cmd; @@ -206,7 +206,7 @@ static int ism_query_rgid(struct ism_dev *ism, struct smcd_gid *rgid, cmd.request.hdr.cmd = ISM_QUERY_RGID; cmd.request.hdr.len = sizeof(cmd.request); - cmd.request.rgid = rgid->gid; + memcpy(&cmd.request.rgid, rgid, sizeof(cmd.request.rgid)); cmd.request.vlan_valid = vid_valid; cmd.request.vlan_id = vid; @@ -364,8 +364,8 @@ static int ism_reset_vlan_required(struct ism_dev *ism) return ism_cmd_simple(ism, ISM_RESET_VLAN); } -static int ism_signal_ieq(struct ism_dev *ism, struct smcd_gid *rgid, - u32 trigger_irq, u32 event_code, u64 info) +static int ism_signal_ieq(struct ism_dev *ism, uuid_t *rgid, u32 trigger_irq, + u32 event_code, u64 info) { union ism_sig_ieq cmd; @@ -373,7 +373,7 @@ static int ism_signal_ieq(struct ism_dev *ism, struct smcd_gid *rgid, cmd.request.hdr.cmd = ISM_SIGNAL_IEQ; cmd.request.hdr.len = sizeof(cmd.request); - cmd.request.rgid = rgid->gid; + memcpy(&cmd.request.rgid, rgid, sizeof(cmd.request.rgid)); cmd.request.trigger_irq = trigger_irq; cmd.request.event_code = event_code; cmd.request.info = info; diff --git a/include/linux/ism.h b/include/linux/ism.h index ede1a40b408e..50975847248f 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -11,6 +11,7 @@ #include #include +#include /* The remote peer rgid can use dmb_tok to write into this buffer. */ struct ism_dmb { @@ -24,11 +25,6 @@ struct ism_dmb { dma_addr_t dma_addr; }; -struct smcd_gid { - u64 gid; - u64 gid_ext; -}; - struct ism_event { u32 type; u32 code; @@ -61,7 +57,7 @@ int ism_register_client(struct ism_client *client); int ism_unregister_client(struct ism_client *client); /* Mandatory operations for all ism devices: - * int (*query_remote_gid)(struct ism_dev *dev, struct smcd_gid *rgid, + * int (*query_remote_gid)(struct ism_dev *dev, uuid_t *rgid, * u32 vid_valid, u32 vid); * Query whether remote GID rgid is reachable via this device and this * vlan id. Vlan id is only checked if vid_valid != 0. @@ -101,14 +97,14 @@ int ism_unregister_client(struct ism_client *client); * Ability to assign dmbs to VLANs is missing * - do we really want / need this? * - * int (*signal_event)(struct ism_dev *dev, struct smcd_gid *rgid, + * int (*signal_event)(struct ism_dev *dev, uuid_t *rgid, * u32 trigger_irq, u32 event_code, u64 info); * Send a control event into the event queue of a remote gid (rgid) * with (1) or without (0) triggering an interrupt at the remote gid. */ struct ism_ops { - int (*query_remote_gid)(struct ism_dev *dev, struct smcd_gid *rgid, + int (*query_remote_gid)(struct ism_dev *dev, uuid_t *rgid, u32 vid_valid, u32 vid); int (*register_dmb)(struct ism_dev *dev, struct ism_dmb *dmb, struct ism_client *client); @@ -125,7 +121,7 @@ struct ism_ops { int (*del_vlan_id)(struct ism_dev *dev, u64 vlan_id); int (*set_vlan_required)(struct ism_dev *dev); int (*reset_vlan_required)(struct ism_dev *dev); - int (*signal_event)(struct ism_dev *dev, struct smcd_gid *rgid, + int (*signal_event)(struct ism_dev *dev, uuid_t *rgid, u32 trigger_irq, u32 event_code, u64 info); }; @@ -150,7 +146,7 @@ struct ism_dev { dma_addr_t ieq_dma_addr; struct device dev; - struct smcd_gid gid; + uuid_t gid; int ieq_idx; struct ism_client *subs[MAX_CLIENTS]; diff --git a/include/net/smc.h b/include/net/smc.h index 3d20c6c05056..91aab1d44166 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -15,7 +15,7 @@ #include #include #include -#include "linux/ism.h" +#include struct sock; @@ -30,11 +30,11 @@ struct smc_hashinfo { struct smcd_dev; -//struct smcd_gid { -// u64 gid; -// u64 gid_ext; -//}; -// +struct smcd_gid { + u64 gid; + u64 gid_ext; +}; + struct smcd_ops { int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, u32 vid_valid, u32 vid); diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 6fbacad02f23..a49da16bafd5 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -485,8 +485,10 @@ static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, u32 vid_valid, u32 vid) { struct ism_dev *ism = smcd->priv; + uuid_t ism_rgid; - return ism->ops->query_remote_gid(ism, rgid, vid_valid, vid); + copy_to_ismgid(&ism_rgid, rgid); + return ism->ops->query_remote_gid(ism, &ism_rgid, vid_valid, vid); } static int smcd_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, @@ -536,9 +538,11 @@ static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, u32 trigger_irq, u32 event_code, u64 info) { struct ism_dev *ism = smcd->priv; + uuid_t ism_rgid; - return ism->ops->signal_event(ism, rgid, - trigger_irq, event_code, info); + copy_to_ismgid(&ism_rgid, rgid); + return ism->ops->signal_event(ism, &ism_rgid, trigger_irq, + event_code, info); } static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, @@ -560,8 +564,7 @@ static void smcd_get_local_gid(struct smcd_dev *smcd, { struct ism_dev *ism = smcd->priv; - smcd_gid->gid = ism->gid.gid; - smcd_gid->gid_ext = ism->gid.gid_ext; + copy_to_smcdgid(smcd_gid, &ism->gid); } static u16 smcd_get_chid(struct smcd_dev *smcd) diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index 6763133dd8d0..d041e5a7c459 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -12,6 +12,7 @@ #include #include #include +#include #include "smc.h" @@ -94,4 +95,24 @@ static inline bool smc_ism_is_loopback(struct smcd_dev *smcd) return (smcd->ops->get_chid(smcd) == 0xFFFF); } +static inline void copy_to_smcdgid(struct smcd_gid *sgid, uuid_t *igid) +{ + __be64 temp; + + memcpy(&temp, igid, sizeof(sgid->gid)); + sgid->gid = ntohll(temp); + memcpy(&temp, igid + sizeof(sgid->gid), sizeof(sgid->gid_ext)); + sgid->gid_ext = ntohll(temp); +} + +static inline void copy_to_ismgid(uuid_t *igid, struct smcd_gid *sgid) +{ + __be64 temp; + + temp = htonll(sgid->gid); + memcpy(igid, &temp, sizeof(sgid->gid)); + temp = htonll(sgid->gid_ext); + memcpy(igid + sizeof(sgid->gid), &temp, sizeof(sgid->gid_ext)); +} + #endif From patchwork Wed Jan 15 19:55:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940865 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D9EF1D8A0D; Wed, 15 Jan 2025 19:55:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970945; cv=none; b=Eu1GPZ2ZINxCJrYJIe84liavGKs4+9VgXTSPuuypP1lCW2gafXeqci5WQueAk4JnRgB4NEOALFbP2XXWk/q/ACPWuBHxvKFIx9S34vJxjE0iQoRNaTScblKa75D/vWIIulxPpAwnqpX/0BF2hdTrkyL9UxXUYLQLVLNM32WMtFU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970945; c=relaxed/simple; bh=FUjlJDN+4Nksbi6eIo9rP1tSJz6LipiNnlHfrpQ3pRI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mpLNok7euy+EF8Otfm9RgrOGkD+G0kJ6jvNpS2Di6or0kCzQVLIcP1eyy4KjILotsr6U9UzoSxbnZWWX7jMzy2kWk8C+nyctpc5ztACJeT4V+Berer+mL+K8oPHDiqEIW67nNZwZDd+DMvYGcyezeeTy7MARgHLx8v42pJfvISk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=c6c3nm+G; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="c6c3nm+G" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHXS0k028583; Wed, 15 Jan 2025 19:55:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=E29i78O/hTJl7fkSQ LdRdQR4cRCFr5mDjPRMvTCVJqo=; b=c6c3nm+GjwlhF52S+zkx3ZMCVVS3fOOEu sG+BHAiAyxAkF0LbzB84H6+BoYCq9bbwMpheSoiHGxBsGWv1HQb3x6t2jpjVfpT0 zuccXokxli4vzfkjyidX1eTEF36VLu35khoAkd1jXB8UXgDgH3yk+B/JQ8XMPLJm iP507qjENuAbzU+GifjFYggSU6c3g6wbdGA55mhkrRCCI3aPPZ56S+cpDlgCgx2T D6BL48vDKTHBR+U/VPFLxyQDnuEEe+NQ4Spa5MRFz1bBR48lJ4Bm7EhmNw/jsLGb GypgeuUxfWKJID23Z/w037QvYdicgD8EeiTrykOOfeJ6G4MKD/Pew== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4465gjvcd7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from m0356517.ppops.net (m0356517.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJtXWR011500; Wed, 15 Jan 2025 19:55:33 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4465gjvcd5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHaxFb007364; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4443yna6ru-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav04.fra02v.mail.ibm.com (smtpav04.fra02v.mail.ibm.com [10.20.54.103]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtR5M50987410 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:27 GMT Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 99C702004B; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav04.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7ADFF20040; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav04.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 4E7C6E0D86; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 4/7] net/ism: Add kernel-doc comments for ism functions Date: Wed, 15 Jan 2025 20:55:24 +0100 Message-ID: <20250115195527.2094320-5-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: MY-skRTw3QyqdBbcymjsfS-DEPy2j0CA X-Proofpoint-GUID: e1nNPZiZHo9fi1SEkJK_9v7zxkxXLzFy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 clxscore=1015 suspectscore=0 lowpriorityscore=0 malwarescore=0 spamscore=0 priorityscore=1501 mlxlogscore=999 impostorscore=0 adultscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Note that in this RFC this patch is not complete, future versions of this patch need to contain comments for all ism_ops. Especially signal_event() and handle_event() need a good generic description. Signed-off-by: Alexandra Winter --- include/linux/ism.h | 115 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 105 insertions(+), 10 deletions(-) diff --git a/include/linux/ism.h b/include/linux/ism.h index 50975847248f..bc165d077071 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -13,11 +13,26 @@ #include #include -/* The remote peer rgid can use dmb_tok to write into this buffer. */ +/* + * DMB - Direct Memory Buffer + * ========================== + * An ism client provides an DMB as input buffer for a local receiving + * ism device for exactly one (remote) sending ism device. Only this + * sending device can send data into this DMB using move_data(). Sender + * and receiver can be the same device. + * TODO: Alignment and length rules (CPU and DMA). Device specific? + */ struct ism_dmb { + /* dmb_tok - Token for this dmb + * Used by remote sender to address this dmb. + * Provided by ism fabric in register_dmb(). + * Unique per ism fabric. + */ u64 dmb_tok; + /* rgid - GID of designated remote sending device */ u64 rgid; u32 dmb_len; + /* sba_idx - Index of this DMB on this receiving device */ u32 sba_idx; u32 vlan_valid; u32 vlan_id; @@ -25,6 +40,8 @@ struct ism_dmb { dma_addr_t dma_addr; }; +/* ISM event structure (currently device type specific) */ +// TODO: Define and describe generic event properties struct ism_event { u32 type; u32 code; @@ -33,38 +50,89 @@ struct ism_event { u64 info; }; +//TODO: use enum typedef #define ISM_EVENT_DMB 0 #define ISM_EVENT_GID 1 #define ISM_EVENT_SWR 2 struct ism_dev; +/* + * ISM clients + * =========== + * All ism clients have access to all ism devices + * and must provide the following functions to be called by + * ism device drivers: + */ struct ism_client { + /* client name for logging and debugging purposes */ const char *name; + /** + * add() - add an ism device + * @dev: device that was added + * + * Will be called during ism_register_client() for all existing + * ism devices and whenever a new ism device is registered. + * *dev is valid until ism_client->remove() is called. + */ void (*add)(struct ism_dev *dev); + /** + * remove() - remove an ism device + * @dev: device to be removed + * + * Will be called whenever an ism device is unregistered. + * Before this call the device is already inactive: It will + * no longer call client handlers. + * The client must not access *dev after this call. + */ void (*remove)(struct ism_dev *dev); + /** + * handle_event() - Handle control information sent by device + * @dev: device reporting the event + * @event: ism event structure + */ void (*handle_event)(struct ism_dev *dev, struct ism_event *event); - /* Parameter dmbemask contains a bit vector with updated DMBEs, if sent - * via ism_move_data(). Callback function must handle all active bits - * indicated by dmbemask. + /** + * handle_irq() - Handle signalling of a DMB + * @dev: device owns the dmb + * @bit: sba_idx=idx of the ism_dmb that got signalled + * TODO: Pass a priv pointer to ism_dmb instead of 'bit'(?) + * @dmbemask: ism signalling mask of the dmb + * + * Handle signalling of a dmb that was registered by this client + * for this device. + * The ism device can coalesce multiple signalling triggers into a + * single call of handle_irq(). dmbemask can be used to indicate + * different kinds of triggers. */ void (*handle_irq)(struct ism_dev *dev, unsigned int bit, u16 dmbemask); - /* Private area - don't touch! */ + /* client index - provided by ism layer */ u8 id; }; int ism_register_client(struct ism_client *client); int ism_unregister_client(struct ism_client *client); +//TODO: Pair descriptions with functions +/* + * ISM devices + * =========== + */ /* Mandatory operations for all ism devices: * int (*query_remote_gid)(struct ism_dev *dev, uuid_t *rgid, * u32 vid_valid, u32 vid); * Query whether remote GID rgid is reachable via this device and this * vlan id. Vlan id is only checked if vid_valid != 0. + * Returns 0 if remote gid is reachable. * * int (*register_dmb)(struct ism_dev *dev, struct ism_dmb *dmb, * void *client); - * Register an ism_dmb buffer for this device and this client. + * Allocate and register an ism_dmb buffer for this device and this client. + * The following fields of ism_dmb must be valid: + * rgid, dmb_len, vlan_*; Optionally:requested sba_idx (non-zero) + * Upon return the following fields will be valid: dmb_tok, sba_idx + * cpu_addr, dma_addr (if applicable) + * Returns zero on success * * int (*unregister_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); * Unregister an ism_dmb buffer @@ -81,10 +149,15 @@ int ism_unregister_client(struct ism_client *client); * u16 (*get_chid)(struct ism_dev *dev); * Returns ism fabric identifier (channel id) of this device. * Only devices on the same ism fabric can communicate. - * chid is unique per HW system, except for 0xFFFF, which denotes - * an ism_loopback device that can only communicate with itself. - * Use chid for fast negative checks, but only query_remote_gid() - * can give a reliable positive answer. + * chid is unique per HW system. Use chid for fast negative checks, + * but only query_remote_gid() can give a reliable positive answer: + * Different chid: ism is not possible + * Same chid: ism traffic may be possible or not + * (e.g. different HW systems) + * EXCEPTION: A value of 0xFFFF denotes an ism_loopback device + * that can only communicate with itself. Use GID or + * query_remote_gid()to determine whether sender and + * receiver use the same ism_loopback device. * * struct device* (*get_dev)(struct ism_dev *dev); * @@ -109,6 +182,28 @@ struct ism_ops { int (*register_dmb)(struct ism_dev *dev, struct ism_dmb *dmb, struct ism_client *client); int (*unregister_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); + /** + * move_data() - write into a remote dmb + * @dev: Local sending ism device + * @dmb_tok: Token of the remote dmb + * @idx: signalling index + * @sf: signalling flag; + * if true, idx will be turned on at target ism interrupt mask + * and target device will be signalled, if required. + * @offset: offset within target dmb + * @data: pointer to data to be sent + * @size: length of data to be sent + * + * Use dev to write data of size at offset into a remote dmb + * identified by dmb_tok. Data is moved synchronously, *data can + * be freed when this function returns. + * + * If signalling flag (sf) is true, bit number idx bit will be + * turned on in the ism signalling mask, that belongs to the + * target dmb, and handle_irq() of the ism client that owns this + * dmb will be called, if required. The target device may chose to + * coalesce multiple signalling triggers. + */ int (*move_data)(struct ism_dev *dev, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size); From patchwork Wed Jan 15 19:55:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940868 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C0DE1D934C; Wed, 15 Jan 2025 19:55:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970948; cv=none; b=VAml5bL/LTdshDZolNZlD3epBcT54p0Bg2lHk+jFEBiV48yV44j7EUu8Xd9qAXFFFZEgSGFJ58Dx3poL/JMIzz16HhgaKupniiYqSUrWTwxjMsKU5fznOqhNcMoCDbxY12o2IlHrTQED97gp88RHEZHBPhulmmQZy6d4/n5+U7c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970948; c=relaxed/simple; bh=apZrV6Cfv+ct32OZ59yXTMmHP/olwYgwfPwqmHxZUF0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EX+Ms4J6P5wHB1XmNfC+4zY/zo7PthFzCFpxhraDAqSTXLLMmHu1Nuo+H2hQK5Y1YoiKRsnrXAnxbRw3/YtdZ1KkKPuXgXOQ4tUWK3Z2kEYDGJbMCSxDM2K3TuIC5WxYqPpLbN+iEh7I/wUgo+2yFNFpB7Y3gSVGTwDki+ZPlRg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=lfbbDj3K; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="lfbbDj3K" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX4vK022499; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=SXHQ5gafw2hafsP/s bqTHR4KcTD0UjaOe3Ye2LqP9gU=; b=lfbbDj3KB7q1uh0ny8Axh2YSisZrjt/jk B6FwNH2cMWQTDiS6TTslFlMfm7K6ZCPrKvjx1NgIkFT+VKkt+0Y1x9odqHT/lehI eA5jKUksTE9sD0EbturfxbQ5eG0lef1qEcUF5BroQzNYJhMchnKooWP4S0DBmpN4 mwbjRpVlBSRFKB8B44TxCN9A1Q9syLvATRijVrqFV8RRBa/ifZZ1s0ch8tUe3YSl QdHL/kaISP4ba8W/cy6hEine6Wogu7FtWN1jqmGvcj8fccR8ksjOad8bKyUNL8Is qs+llkGOyt/H8Fv4wiTX58KIyx446zLjXzP2jn8nwRXeKN8G+P7HA== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy66-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJf0dN006338; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy5y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FGm2ST004551; Wed, 15 Jan 2025 19:55:32 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4442ystcdy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav03.fra02v.mail.ibm.com (smtpav03.fra02v.mail.ibm.com [10.20.54.102]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtSa551052806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:28 GMT Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F09B620043; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav03.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CAB732004D; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav03.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 51FF4E0D92; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 5/7] net/ism: Move ism_loopback to net/ism Date: Wed, 15 Jan 2025 20:55:25 +0100 Message-ID: <20250115195527.2094320-6-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: OsdRV2HZLOyjQnv5Wl-kRMRglcENHe5a X-Proofpoint-GUID: v9ewj9DO06AMUH2VoLBQbh2eVH7vyYZs X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 bulkscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC The first stage of ism_loopback was implemented as part of the SMC module [1]. Now that we have the ism layer, provide access to the ism_loopback device to all ism clients. Move ism_loopback.* from net/smc to net/ism. The following changes are required to ism_loopback.c: - Change ism_lo_move_data() to no longer schedule an smcd receive tasklet, but instead call ism_client->handle_irq(). Note: In this RFC patch ism_loppback is not fully generic. The smc-d client uses attached buffers, for moves without signalling. and not-attached buffers for moves with signalling. ism_lo_move_data() must not rely on that assumption. ism_lo_move_data() must be able to handle more than one ism client. In addition the following changes are required to unify ism_loopback and ism_vp: In ism layer and ism_vpci: ism_loopback is not backed by a pci device, so use dev instead of pdev in ism_dev. In smc-d: in smcd_alloc_dev(): - use kernel memory instead of device memory for smcd_dev and smcd->conn. An alternative would be to ask device to alloc the memory. - use different smcd_ops and max_dmbs for ism_vp and ism_loopback. A future patch can change smc-d to directly use ism_ops instead of smcd_ops. - use ism dev_name instead of pci dev name for ism_evt_wq name - allocate an event workqueue for ism_loopback, although it currently does not generate events. Link: https://lore.kernel.org/linux-kernel//20240428060738.60843-1-guwen@linux.alibaba.com/ [1] Signed-off-by: Alexandra Winter --- drivers/s390/net/ism.h | 6 +- drivers/s390/net/ism_drv.c | 31 ++- include/linux/ism.h | 59 +++++ include/net/smc.h | 4 +- net/ism/Kconfig | 13 ++ net/ism/Makefile | 1 + net/ism/ism_loopback.c | 366 +++++++++++++++++++++++++++++++ net/ism/ism_loopback.h | 59 +++++ net/ism/ism_main.c | 11 +- net/smc/Kconfig | 13 -- net/smc/Makefile | 1 - net/smc/af_smc.c | 12 +- net/smc/smc_ism.c | 108 +++++++--- net/smc/smc_loopback.c | 427 ------------------------------------- net/smc/smc_loopback.h | 60 ------ 15 files changed, 606 insertions(+), 565 deletions(-) create mode 100644 net/ism/ism_loopback.c create mode 100644 net/ism/ism_loopback.h delete mode 100644 net/smc/smc_loopback.c delete mode 100644 net/smc/smc_loopback.h diff --git a/drivers/s390/net/ism.h b/drivers/s390/net/ism.h index 61cf10334170..0deca6d0e328 100644 --- a/drivers/s390/net/ism.h +++ b/drivers/s390/net/ism.h @@ -202,7 +202,7 @@ struct ism_sba { static inline void __ism_read_cmd(struct ism_dev *ism, void *data, unsigned long offset, unsigned long len) { - struct zpci_dev *zdev = to_zpci(ism->pdev); + struct zpci_dev *zdev = to_zpci(to_pci_dev(ism->dev.parent)); u64 req = ZPCI_CREATE_REQ(zdev->fh, 2, 8); while (len > 0) { @@ -216,7 +216,7 @@ static inline void __ism_read_cmd(struct ism_dev *ism, void *data, static inline void __ism_write_cmd(struct ism_dev *ism, void *data, unsigned long offset, unsigned long len) { - struct zpci_dev *zdev = to_zpci(ism->pdev); + struct zpci_dev *zdev = to_zpci(to_pci_dev(ism->dev.parent)); u64 req = ZPCI_CREATE_REQ(zdev->fh, 2, len); if (len) @@ -226,7 +226,7 @@ static inline void __ism_write_cmd(struct ism_dev *ism, void *data, static inline int __ism_move(struct ism_dev *ism, u64 dmb_req, void *data, unsigned int size) { - struct zpci_dev *zdev = to_zpci(ism->pdev); + struct zpci_dev *zdev = to_zpci(to_pci_dev(ism->dev.parent)); u64 req = ZPCI_CREATE_REQ(zdev->fh, 0, size); return __zpci_store_block(data, req, dmb_req); diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index ab70debbdd9d..c0954d6dd9f5 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -88,7 +88,7 @@ static int register_sba(struct ism_dev *ism) dma_addr_t dma_handle; struct ism_sba *sba; - sba = dma_alloc_coherent(&ism->pdev->dev, PAGE_SIZE, &dma_handle, + sba = dma_alloc_coherent(ism->dev.parent, PAGE_SIZE, &dma_handle, GFP_KERNEL); if (!sba) return -ENOMEM; @@ -99,7 +99,7 @@ static int register_sba(struct ism_dev *ism) cmd.request.sba = dma_handle; if (ism_cmd(ism, &cmd)) { - dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, sba, dma_handle); + dma_free_coherent(ism->dev.parent, PAGE_SIZE, sba, dma_handle); return -EIO; } @@ -115,7 +115,7 @@ static int register_ieq(struct ism_dev *ism) dma_addr_t dma_handle; struct ism_eq *ieq; - ieq = dma_alloc_coherent(&ism->pdev->dev, PAGE_SIZE, &dma_handle, + ieq = dma_alloc_coherent(ism->dev.parent, PAGE_SIZE, &dma_handle, GFP_KERNEL); if (!ieq) return -ENOMEM; @@ -127,7 +127,7 @@ static int register_ieq(struct ism_dev *ism) cmd.request.len = sizeof(*ieq); if (ism_cmd(ism, &cmd)) { - dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, ieq, dma_handle); + dma_free_coherent(ism->dev.parent, PAGE_SIZE, ieq, dma_handle); return -EIO; } @@ -149,7 +149,7 @@ static int unregister_sba(struct ism_dev *ism) if (ret && ret != ISM_ERROR) return -EIO; - dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, + dma_free_coherent(ism->dev.parent, PAGE_SIZE, ism->sba, ism->sba_dma_addr); ism->sba = NULL; @@ -169,7 +169,7 @@ static int unregister_ieq(struct ism_dev *ism) if (ret && ret != ISM_ERROR) return -EIO; - dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, + dma_free_coherent(ism->dev.parent, PAGE_SIZE, ism->ieq, ism->ieq_dma_addr); ism->ieq = NULL; @@ -216,7 +216,7 @@ static int ism_query_rgid(struct ism_dev *ism, uuid_t *rgid, u32 vid_valid, static void ism_free_dmb(struct ism_dev *ism, struct ism_dmb *dmb) { clear_bit(dmb->sba_idx, ism->sba_bitmap); - dma_unmap_page(&ism->pdev->dev, dmb->dma_addr, dmb->dmb_len, + dma_unmap_page(ism->dev.parent, dmb->dma_addr, dmb->dmb_len, DMA_FROM_DEVICE); folio_put(virt_to_folio(dmb->cpu_addr)); } @@ -227,7 +227,7 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) unsigned long bit; int rc; - if (PAGE_ALIGN(dmb->dmb_len) > dma_get_max_seg_size(&ism->pdev->dev)) + if (PAGE_ALIGN(dmb->dmb_len) > dma_get_max_seg_size(ism->dev.parent)) return -EINVAL; if (!dmb->sba_idx) { @@ -251,10 +251,10 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) } dmb->cpu_addr = folio_address(folio); - dmb->dma_addr = dma_map_page(&ism->pdev->dev, + dmb->dma_addr = dma_map_page(ism->dev.parent, virt_to_page(dmb->cpu_addr), 0, dmb->dmb_len, DMA_FROM_DEVICE); - if (dma_mapping_error(&ism->pdev->dev, dmb->dma_addr)) { + if (dma_mapping_error(ism->dev.parent, dmb->dma_addr)) { rc = -ENOMEM; goto out_free; } @@ -419,10 +419,7 @@ static int ism_supports_v2(void) static u16 ism_get_chid(struct ism_dev *ism) { - if (!ism || !ism->pdev) - return 0; - - return to_zpci(ism->pdev)->pchid; + return to_zpci(to_pci_dev(ism->dev.parent))->pchid; } static void ism_handle_event(struct ism_dev *ism) @@ -499,7 +496,7 @@ static const struct ism_ops ism_vp_ops = { static int ism_dev_init(struct ism_dev *ism) { - struct pci_dev *pdev = ism->pdev; + struct pci_dev *pdev = to_pci_dev(ism->dev.parent); int ret; ret = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_MSI); @@ -565,7 +562,6 @@ static int ism_probe(struct pci_dev *pdev, const struct pci_device_id *id) spin_lock_init(&ism->lock); dev_set_drvdata(&pdev->dev, ism); - ism->pdev = pdev; ism->dev.parent = &pdev->dev; device_initialize(&ism->dev); dev_set_name(&ism->dev, dev_name(&pdev->dev)); @@ -603,14 +599,13 @@ static int ism_probe(struct pci_dev *pdev, const struct pci_device_id *id) device_del(&ism->dev); err_dev: dev_set_drvdata(&pdev->dev, NULL); - kfree(ism); return ret; } static void ism_dev_exit(struct ism_dev *ism) { - struct pci_dev *pdev = ism->pdev; + struct pci_dev *pdev = to_pci_dev(ism->dev.parent); unsigned long flags; int i; diff --git a/include/linux/ism.h b/include/linux/ism.h index bc165d077071..929a1f275419 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -144,6 +144,9 @@ int ism_unregister_client(struct ism_client *client); * identified by dmb_tok and idx. If signal flag (sf) then signal * the remote peer that data has arrived in this dmb. * + * int (*unregister_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); + * Unregister an ism_dmb buffer + * * int (*supports_v2)(void); * * u16 (*get_chid)(struct ism_dev *dev); @@ -218,12 +221,63 @@ struct ism_ops { int (*reset_vlan_required)(struct ism_dev *dev); int (*signal_event)(struct ism_dev *dev, uuid_t *rgid, u32 trigger_irq, u32 event_code, u64 info); +/* no copy option + * -------------- + */ + /** + * support_dmb_nocopy() - does this device provide no-copy option? + * @dev: ism device + * + * In addition to using move_data(), a sender device can provide a + * kernel address + length, that represents a target dmb + * (like MMIO). If a sender writes into such a ghost-send-buffer + * (= at this kernel address) the data will automatically + * immediately appear in the target dmb, even without calling + * move_data(). + * Note that this is NOT related to the MSG_ZEROCOPY socket flag. + * + * Either all 3 function pointers for support_dmb_nocopy(), + * attach_dmb() and detach_dmb() are defined, or all of them must + * be NULL. + * + * Return: non-zero, if no-copy is supported. + */ + int (*support_dmb_nocopy)(struct ism_dev *dev); + /** + * attach_dmb() - attach local memory to a remote dmb + * @dev: Local sending ism device + * @dmb: all other parameters are passed in the form of a + * dmb struct + * TODO: (THIS IS CONFUSING, should be changed) + * dmb_tok: (in) Token of the remote dmb, we want to attach to + * cpu_addr: (out) MMIO address + * dma_addr: (out) MMIO address (if applicable, invalid otherwise) + * dmb_len: (out) length of local MMIO region, + * equal to length of remote DMB. + * sba_idx: (out) index of remote dmb (NOT HELPFUL, should be removed) + * + * Provides a memory address to the sender that can be used to + * directly write into the remote dmb. + * + * Return: Zero upon success, Error code otherwise + */ + int (*attach_dmb)(struct ism_dev *dev, struct ism_dmb *dmb); + /** + * detach_dmb() - Detach the ghost buffer from a remote dmb + * @dev: ism device + * @token: dmb token of the remote dmb + * Return: Zero upon success, Error code otherwise + */ + int (*detach_dmb)(struct ism_dev *dev, u64 token); }; /* Unless we gain unexpected popularity, this limit should hold for a while */ #define MAX_CLIENTS 8 #define NO_CLIENT 0xff /* must be >= MAX_CLIENTS */ #define ISM_NR_DMBS 1920 +/* Defined fabric id / CHID for all loopback devices: */ +#define ISM_LO_RESERVED_CHID 0xFFFF +#define ISM_LO_MAX_DMBS 5000 struct ism_dev { const struct ism_ops *ops; @@ -259,6 +313,11 @@ static inline void ism_set_priv(struct ism_dev *dev, struct ism_client *client, dev->priv[client->id] = priv; } +static inline struct device *ism_get_dev(struct ism_dev *ism) +{ + return &ism->dev; +} + #define ISM_RESERVED_VLANID 0x1FFF #define ISM_ERROR 0xFFFF diff --git a/include/net/smc.h b/include/net/smc.h index 91aab1d44166..7a96ed2ae20c 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -63,8 +63,8 @@ struct smcd_ops { struct smcd_dev { const struct smcd_ops *ops; - void *priv; - void *client; + struct ism_dev *ism; + struct ism_client *client; struct list_head list; spinlock_t lock; struct smc_connection **conn; diff --git a/net/ism/Kconfig b/net/ism/Kconfig index 4329489cc1e9..ac7a9ba7c792 100644 --- a/net/ism/Kconfig +++ b/net/ism/Kconfig @@ -12,3 +12,16 @@ config ISM To compile as a module choose M. The module name is ism. If unsure, choose N. + +config ISM_LO + bool "intra-OS shortcut with loopback-ism" + depends on ISM + default n + help + ISM_LO enables the creation of an Emulated-ISM device named + loopback-ism which can be used for transferring data + when communication occurs within the same OS. This helps in + convenient testing of ISM clients, since loopback-ism is + independent of architecture or hardware. + + if unsure, say N. diff --git a/net/ism/Makefile b/net/ism/Makefile index b752baf72003..5e7c51845862 100644 --- a/net/ism/Makefile +++ b/net/ism/Makefile @@ -5,3 +5,4 @@ ism-y += ism_main.o obj-$(CONFIG_ISM) += ism.o +ism-$(CONFIG_ISM_LO) += ism_loopback.o \ No newline at end of file diff --git a/net/ism/ism_loopback.c b/net/ism/ism_loopback.c new file mode 100644 index 000000000000..47e5ef355dd7 --- /dev/null +++ b/net/ism/ism_loopback.c @@ -0,0 +1,366 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Functions for loopback-ism device. + * + * Copyright (c) 2024, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#include +#include +#include +#include +#include + +#include "ism_loopback.h" + +#define ISM_LO_V2_CAPABLE 0x1 /* loopback-ism acts as ISMv2 */ +#define ISM_LO_SUPPORT_NOCOPY 0x1 +#define ISM_DMA_ADDR_INVALID (~(dma_addr_t)0) + +static const char ism_lo_dev_name[] = "loopback-ism"; +/* global loopback device */ +static struct ism_lo_dev *lo_dev; + +static int ism_lo_query_rgid(struct ism_dev *ism, uuid_t *rgid, + u32 vid_valid, u32 vid) +{ + /* rgid should be the same as lgid; vlan is not supported */ + if (!vid_valid && uuid_equal(rgid, &ism->gid)) + return 0; + return -ENETUNREACH; +} + +static int ism_lo_register_dmb(struct ism_dev *ism, struct ism_dmb *dmb, + struct ism_client *client) +{ + struct ism_lo_dmb_node *dmb_node, *tmp_node; + struct ism_lo_dev *ldev; + unsigned long flags; + int sba_idx, rc; + + ldev = container_of(ism, struct ism_lo_dev, ism); + sba_idx = dmb->sba_idx; + /* check space for new dmb */ + for_each_clear_bit(sba_idx, ldev->sba_idx_mask, ISM_LO_MAX_DMBS) { + if (!test_and_set_bit(sba_idx, ldev->sba_idx_mask)) + break; + } + if (sba_idx == ISM_LO_MAX_DMBS) + return -ENOSPC; + + dmb_node = kzalloc(sizeof(*dmb_node), GFP_KERNEL); + if (!dmb_node) { + rc = -ENOMEM; + goto err_bit; + } + + dmb_node->sba_idx = sba_idx; + dmb_node->len = dmb->dmb_len; + dmb_node->cpu_addr = kzalloc(dmb_node->len, GFP_KERNEL | + __GFP_NOWARN | __GFP_NORETRY | + __GFP_NOMEMALLOC); + if (!dmb_node->cpu_addr) { + rc = -ENOMEM; + goto err_node; + } + dmb_node->dma_addr = ISM_DMA_ADDR_INVALID; + refcount_set(&dmb_node->refcnt, 1); + +again: + /* add new dmb into hash table */ + get_random_bytes(&dmb_node->token, sizeof(dmb_node->token)); + write_lock_bh(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_node->token) { + if (tmp_node->token == dmb_node->token) { + write_unlock_bh(&ldev->dmb_ht_lock); + goto again; + } + } + hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); + write_unlock_bh(&ldev->dmb_ht_lock); + atomic_inc(&ldev->dmb_cnt); + + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; + + spin_lock_irqsave(&ism->lock, flags); + ism->sba_client_arr[sba_idx] = client->id; + spin_unlock_irqrestore(&ism->lock, flags); + + return 0; + +err_node: + kfree(dmb_node); +err_bit: + clear_bit(sba_idx, ldev->sba_idx_mask); + return rc; +} + +static void __ism_lo_unregister_dmb(struct ism_lo_dev *ldev, + struct ism_lo_dmb_node *dmb_node) +{ + /* remove dmb from hash table */ + write_lock_bh(&ldev->dmb_ht_lock); + hash_del(&dmb_node->list); + write_unlock_bh(&ldev->dmb_ht_lock); + + clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); + kvfree(dmb_node->cpu_addr); + kfree(dmb_node); + + if (atomic_dec_and_test(&ldev->dmb_cnt)) + wake_up(&ldev->ldev_release); +} + +static int ism_lo_unregister_dmb(struct ism_dev *ism, struct ism_dmb *dmb) +{ + struct ism_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct ism_lo_dev *ldev; + unsigned long flags; + + ldev = container_of(ism, struct ism_lo_dev, ism); + + /* find dmb from hash table */ + read_lock_bh(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok) { + dmb_node = tmp_node; + break; + } + } + read_unlock_bh(&ldev->dmb_ht_lock); + if (!dmb_node) + return -EINVAL; + + if (refcount_dec_and_test(&dmb_node->refcnt)) { + spin_lock_irqsave(&ism->lock, flags); + ism->sba_client_arr[dmb_node->sba_idx] = NO_CLIENT; + spin_unlock_irqrestore(&ism->lock, flags); + + __ism_lo_unregister_dmb(ldev, dmb_node); + } + return 0; +} + +static int ism_lo_support_dmb_nocopy(struct ism_dev *ism) +{ + return ISM_LO_SUPPORT_NOCOPY; +} + +static int ism_lo_attach_dmb(struct ism_dev *ism, struct ism_dmb *dmb) +{ + struct ism_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct ism_lo_dev *ldev; + + ldev = container_of(ism, struct ism_lo_dev, ism); + + /* find dmb_node according to dmb->dmb_tok */ + read_lock_bh(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { + if (tmp_node->token == dmb->dmb_tok) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock_bh(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock_bh(&ldev->dmb_ht_lock); + + if (!refcount_inc_not_zero(&dmb_node->refcnt)) + /* the dmb is being unregistered, but has + * not been removed from the hash table. + */ + return -EINVAL; + + /* provide dmb information */ + dmb->sba_idx = dmb_node->sba_idx; + dmb->dmb_tok = dmb_node->token; + dmb->cpu_addr = dmb_node->cpu_addr; + dmb->dma_addr = dmb_node->dma_addr; + dmb->dmb_len = dmb_node->len; + return 0; +} + +static int ism_lo_detach_dmb(struct ism_dev *ism, u64 token) +{ + struct ism_lo_dmb_node *dmb_node = NULL, *tmp_node; + struct ism_lo_dev *ldev; + + ldev = container_of(ism, struct ism_lo_dev, ism); + + /* find dmb_node according to dmb->dmb_tok */ + read_lock_bh(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { + if (tmp_node->token == token) { + dmb_node = tmp_node; + break; + } + } + if (!dmb_node) { + read_unlock_bh(&ldev->dmb_ht_lock); + return -EINVAL; + } + read_unlock_bh(&ldev->dmb_ht_lock); + + if (refcount_dec_and_test(&dmb_node->refcnt)) + __ism_lo_unregister_dmb(ldev, dmb_node); + return 0; +} + +static int ism_lo_move_data(struct ism_dev *ism, u64 dmb_tok, + unsigned int idx, bool sf, unsigned int offset, + void *data, unsigned int size) +{ + struct ism_lo_dmb_node *rmb_node = NULL, *tmp_node; + struct ism_lo_dev *ldev; + u16 s_mask; + u8 client_id; + u32 sba_idx; + + ldev = container_of(ism, struct ism_lo_dev, ism); + + if (!sf) + /* since sndbuf is merged with peer DMB, there is + * no need to copy data from sndbuf to peer DMB. + */ + return 0; + + read_lock_bh(&ldev->dmb_ht_lock); + hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { + if (tmp_node->token == dmb_tok) { + rmb_node = tmp_node; + break; + } + } + if (!rmb_node) { + read_unlock_bh(&ldev->dmb_ht_lock); + return -EINVAL; + } + // So why copy the data now?? SMC usecase? Data buffer is attached, + // rw-pointer are not attached? + memcpy((char *)rmb_node->cpu_addr + offset, data, size); + sba_idx = rmb_node->sba_idx; + read_unlock_bh(&ldev->dmb_ht_lock); + + spin_lock(&ism->lock); + client_id = ism->sba_client_arr[sba_idx]; + s_mask = ror16(0x1000, idx); + if (likely(client_id != NO_CLIENT && ism->subs[client_id])) + ism->subs[client_id]->handle_irq(ism, sba_idx, s_mask); + spin_unlock(&ism->lock); + + return 0; +} + +static int ism_lo_supports_v2(void) +{ + return ISM_LO_V2_CAPABLE; +} + +static u16 ism_lo_get_chid(struct ism_dev *ism) +{ + return ISM_LO_RESERVED_CHID; +} + +static const struct ism_ops ism_lo_ops = { + .query_remote_gid = ism_lo_query_rgid, + .register_dmb = ism_lo_register_dmb, + .unregister_dmb = ism_lo_unregister_dmb, + .support_dmb_nocopy = ism_lo_support_dmb_nocopy, + .attach_dmb = ism_lo_attach_dmb, + .detach_dmb = ism_lo_detach_dmb, + .add_vlan_id = NULL, + .del_vlan_id = NULL, + .set_vlan_required = NULL, + .reset_vlan_required = NULL, + .signal_event = NULL, + .move_data = ism_lo_move_data, + .supports_v2 = ism_lo_supports_v2, + .get_chid = ism_lo_get_chid, +}; + +static void ism_lo_dev_init(struct ism_lo_dev *ldev) +{ + rwlock_init(&ldev->dmb_ht_lock); + hash_init(ldev->dmb_ht); + atomic_set(&ldev->dmb_cnt, 0); + init_waitqueue_head(&ldev->ldev_release); +} + +static void ism_lo_dev_exit(struct ism_lo_dev *ldev) +{ + ism_dev_unregister(&ldev->ism); + if (atomic_read(&ldev->dmb_cnt)) + wait_event(ldev->ldev_release, !atomic_read(&ldev->dmb_cnt)); +} + +static void ism_lo_dev_release(struct device *dev) +{ + struct ism_dev *ism; + struct ism_lo_dev *ldev; + + ism = container_of(dev, struct ism_dev, dev); + ldev = container_of(ism, struct ism_lo_dev, ism); + + kfree(ldev); +} + +static int ism_lo_dev_probe(void) +{ + struct ism_lo_dev *ldev; + struct ism_dev *ism; + + ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); + if (!ldev) + return -ENOMEM; + + ism_lo_dev_init(ldev); + ism = &ldev->ism; + uuid_gen(&ism->gid); + ism->ops = &ism_lo_ops; + + ism->sba_client_arr = kzalloc(ISM_LO_MAX_DMBS, GFP_KERNEL); + if (!ism->sba_client_arr) + return -ENOMEM; + memset(ism->sba_client_arr, NO_CLIENT, ISM_LO_MAX_DMBS); + + ism->dev.parent = NULL; + ism->dev.release = ism_lo_dev_release; + device_initialize(&ism->dev); + dev_set_name(&ism->dev, ism_lo_dev_name); + // No device_add() for loopback? + + ism_dev_register(ism); + lo_dev = ldev; + return 0; +} + +static void ism_lo_dev_remove(void) +{ + if (!lo_dev) + return; + + ism_lo_dev_exit(lo_dev); + put_device(&lo_dev->dev); /* device_initialize in ism_lo_dev_probe */ + //Missing anyhow?: + lo_dev = NULL; +} + +int ism_loopback_init(void) +{ + return ism_lo_dev_probe(); +} + +void ism_loopback_exit(void) +{ + ism_lo_dev_remove(); +} diff --git a/net/ism/ism_loopback.h b/net/ism/ism_loopback.h new file mode 100644 index 000000000000..b1484b032d11 --- /dev/null +++ b/net/ism/ism_loopback.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * loopback-ism device structure definitions. + * + * Copyright (c) 2024, Alibaba Inc. + * + * Author: Wen Gu + * Tony Lu + * + */ + +#ifndef _ISM_LOOPBACK_H +#define _ISM_LOOPBACK_H + +#include +#include +#include +#include +#include +#include +#include + +#if IS_ENABLED(CONFIG_ISM_LO) +#define ISM_LO_DMBS_HASH_BITS 12 + +struct ism_lo_dmb_node { + struct hlist_node list; + u64 token; + u32 len; + u32 sba_idx; + void *cpu_addr; + dma_addr_t dma_addr; + refcount_t refcnt; +}; + +struct ism_lo_dev { + struct ism_dev ism; + struct device dev; + atomic_t dmb_cnt; + rwlock_t dmb_ht_lock; + DECLARE_BITMAP(sba_idx_mask, ISM_LO_MAX_DMBS); + DECLARE_HASHTABLE(dmb_ht, ISM_LO_DMBS_HASH_BITS); + wait_queue_head_t ldev_release; +}; + +int ism_loopback_init(void); +void ism_loopback_exit(void); +#else +static inline int ism_loopback_init(void) +{ + return 0; +} + +static inline void ism_loopback_exit(void) +{ +} +#endif + +#endif /* _ISM_LOOPBACK_H */ diff --git a/net/ism/ism_main.c b/net/ism/ism_main.c index 268408dbd691..13edccff45ea 100644 --- a/net/ism/ism_main.c +++ b/net/ism/ism_main.c @@ -14,6 +14,8 @@ #include #include +#include "ism_loopback.h" + MODULE_DESCRIPTION("Internal Shared Memory class"); MODULE_LICENSE("GPL"); @@ -148,14 +150,21 @@ EXPORT_SYMBOL_GPL(ism_dev_unregister); static int __init ism_init(void) { + int rc; + memset(clients, 0, sizeof(clients)); max_client = 0; - return 0; + rc = ism_loopback_init(); + if (rc) + pr_err("%s: ism_loopback_init fails with %d\n", __func__, rc); + + return rc; } static void __exit ism_exit(void) { + ism_loopback_exit(); } module_init(ism_init); diff --git a/net/smc/Kconfig b/net/smc/Kconfig index ba5e6a2dd2fd..746be3996768 100644 --- a/net/smc/Kconfig +++ b/net/smc/Kconfig @@ -20,16 +20,3 @@ config SMC_DIAG smcss. if unsure, say Y. - -config SMC_LO - bool "SMC intra-OS shortcut with loopback-ism" - depends on SMC - default n - help - SMC_LO enables the creation of an Emulated-ISM device named - loopback-ism in SMC and makes use of it for transferring data - when communication occurs within the same OS. This helps in - convenient testing of SMC-D since loopback-ism is independent - of architecture or hardware. - - if unsure, say N. diff --git a/net/smc/Makefile b/net/smc/Makefile index 60f1c87d5212..0e754cbc38f9 100644 --- a/net/smc/Makefile +++ b/net/smc/Makefile @@ -6,4 +6,3 @@ smc-y := af_smc.o smc_pnet.o smc_ib.o smc_clc.o smc_core.o smc_wr.o smc_llc.o smc-y += smc_cdc.o smc_tx.o smc_rx.o smc_close.o smc_ism.o smc_netlink.o smc_stats.o smc-y += smc_tracepoint.o smc_inet.o smc-$(CONFIG_SYSCTL) += smc_sysctl.o -smc-$(CONFIG_SMC_LO) += smc_loopback.o diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index 9e6c69d18581..b80cae1940e1 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -53,7 +53,6 @@ #include "smc_stats.h" #include "smc_tracepoint.h" #include "smc_sysctl.h" -#include "smc_loopback.h" #include "smc_inet.h" static DEFINE_MUTEX(smc_server_lgr_pending); /* serialize link group @@ -3560,16 +3559,10 @@ static int __init smc_init(void) goto out_sock; } - rc = smc_loopback_init(); - if (rc) { - pr_err("%s: smc_loopback_init fails with %d\n", __func__, rc); - goto out_ib; - } - rc = tcp_register_ulp(&smc_ulp_ops); if (rc) { pr_err("%s: tcp_ulp_register fails with %d\n", __func__, rc); - goto out_lo; + goto out_ib; } rc = smc_inet_init(); if (rc) { @@ -3580,8 +3573,6 @@ static int __init smc_init(void) return 0; out_ulp: tcp_unregister_ulp(&smc_ulp_ops); -out_lo: - smc_loopback_exit(); out_ib: smc_ib_unregister_client(); out_sock: @@ -3620,7 +3611,6 @@ static void __exit smc_exit(void) tcp_unregister_ulp(&smc_ulp_ops); sock_unregister(PF_SMC); smc_core_exit(); - smc_loopback_exit(); smc_ib_unregister_client(); smc_ism_exit(); destroy_workqueue(smc_close_wq); diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index a49da16bafd5..22c1cfb2ad09 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -302,7 +302,7 @@ static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, int use_cnt = 0; void *nlh; - ism = smcd->priv; + ism = smcd->ism; nlh = genlmsg_put(skb, NETLINK_CB(cb->skb).portid, cb->nlh->nlmsg_seq, &smc_gen_nl_family, NLM_F_MULTI, SMC_NETLINK_GET_DEV_SMCD); @@ -453,23 +453,24 @@ static void smc_ism_event_work(struct work_struct *work) kfree(wrk); } -static struct smcd_dev *smcd_alloc_dev(struct device *parent, const char *name, - const struct smcd_ops *ops, int max_dmbs) +static struct smcd_dev *smcd_alloc_dev(const char *name, + const struct smcd_ops *ops, + int max_dmbs) { struct smcd_dev *smcd; - smcd = devm_kzalloc(parent, sizeof(*smcd), GFP_KERNEL); + smcd = kzalloc(sizeof(*smcd), GFP_KERNEL); if (!smcd) return NULL; - smcd->conn = devm_kcalloc(parent, max_dmbs, - sizeof(struct smc_connection *), GFP_KERNEL); + smcd->conn = kcalloc(max_dmbs, sizeof(struct smc_connection *), + GFP_KERNEL); if (!smcd->conn) - return NULL; + goto free_smcd; smcd->event_wq = alloc_ordered_workqueue("ism_evt_wq-%s)", WQ_MEM_RECLAIM, name); if (!smcd->event_wq) - return NULL; + goto free_conn; smcd->ops = ops; @@ -479,12 +480,18 @@ static struct smcd_dev *smcd_alloc_dev(struct device *parent, const char *name, INIT_LIST_HEAD(&smcd->lgr_list); init_waitqueue_head(&smcd->lgrs_deleted); return smcd; + +free_conn: + kfree(smcd->conn); +free_smcd: + kfree(smcd); + return NULL; } static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, u32 vid_valid, u32 vid) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; uuid_t ism_rgid; copy_to_ismgid(&ism_rgid, rgid); @@ -494,42 +501,42 @@ static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, static int smcd_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, void *client) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->register_dmb(ism, dmb, (struct ism_client *)client); } static int smcd_unregister_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->unregister_dmb(ism, dmb); } static int smcd_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->add_vlan_id(ism, vlan_id); } static int smcd_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->del_vlan_id(ism, vlan_id); } static int smcd_set_vlan_required(struct smcd_dev *smcd) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->set_vlan_required(ism); } static int smcd_reset_vlan_required(struct smcd_dev *smcd) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->reset_vlan_required(ism); } @@ -537,7 +544,7 @@ static int smcd_reset_vlan_required(struct smcd_dev *smcd) static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, u32 trigger_irq, u32 event_code, u64 info) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; uuid_t ism_rgid; copy_to_ismgid(&ism_rgid, rgid); @@ -549,7 +556,7 @@ static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, bool sf, unsigned int offset, void *data, unsigned int size) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->move_data(ism, dmb_tok, idx, sf, offset, data, size); } @@ -562,23 +569,21 @@ static int smcd_supports_v2(void) static void smcd_get_local_gid(struct smcd_dev *smcd, struct smcd_gid *smcd_gid) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; copy_to_smcdgid(smcd_gid, &ism->gid); } static u16 smcd_get_chid(struct smcd_dev *smcd) { - struct ism_dev *ism = smcd->priv; + struct ism_dev *ism = smcd->ism; return ism->ops->get_chid(ism); } static inline struct device *smcd_get_dev(struct smcd_dev *dev) { - struct ism_dev *ism = dev->priv; - - return &ism->dev; + return ism_get_dev(dev->ism); } static const struct smcd_ops ism_smcd_ops = { @@ -597,22 +602,65 @@ static const struct smcd_ops ism_smcd_ops = { .get_dev = smcd_get_dev, }; +static inline int smcd_support_dmb_nocopy(struct smcd_dev *smcd) +{ + struct ism_dev *ism = smcd->ism; + + return ism->ops->support_dmb_nocopy(ism); +} + +static inline int smcd_attach_dmb(struct smcd_dev *smcd, + struct ism_dmb *dmb) +{ + struct ism_dev *ism = smcd->ism; + + return ism->ops->attach_dmb(ism, dmb); +} + +static inline int smcd_detach_dmb(struct smcd_dev *smcd, u64 token) +{ + struct ism_dev *ism = smcd->ism; + + return ism->ops->detach_dmb(ism, token); +} + +static const struct smcd_ops lo_ops = { + .query_remote_gid = smcd_query_rgid, + .register_dmb = smcd_register_dmb, + .unregister_dmb = smcd_unregister_dmb, + .support_dmb_nocopy = smcd_support_dmb_nocopy, + .attach_dmb = smcd_attach_dmb, + .detach_dmb = smcd_detach_dmb, + .move_data = smcd_move, + .supports_v2 = smcd_supports_v2, + .get_local_gid = smcd_get_local_gid, + .get_chid = smcd_get_chid, + .get_dev = smcd_get_dev, +}; + static void smcd_register_dev(struct ism_dev *ism) { - const struct smcd_ops *ops = &ism_smcd_ops; + const struct smcd_ops *ops; struct smcd_dev *smcd, *fentry; + int max_dmbs; - if (!ops) - return; + if (ism->ops->get_chid(ism) == ISM_LO_RESERVED_CHID) { + max_dmbs = ISM_LO_MAX_DMBS; + ops = &lo_ops; + } else { + max_dmbs = ISM_NR_DMBS; + ops = &ism_smcd_ops; + } - smcd = smcd_alloc_dev(&ism->pdev->dev, dev_name(&ism->pdev->dev), ops, - ISM_NR_DMBS); + smcd = smcd_alloc_dev(dev_name(&ism->dev), ops, max_dmbs); if (!smcd) return; - smcd->priv = ism; + + smcd->ism = ism; smcd->client = &smc_ism_client; ism_set_priv(ism, &smc_ism_client, smcd); - if (smc_pnetid_by_dev_port(&ism->pdev->dev, 0, smcd->pnetid)) + + if (smc_pnetid_by_dev_port(ism->dev.parent, 0, smcd->pnetid)) smc_pnetid_by_table_smcd(smcd); if (ism->ops->supports_v2()) @@ -653,6 +701,8 @@ static void smcd_unregister_dev(struct ism_dev *ism) list_del_init(&smcd->list); mutex_unlock(&smcd_dev_list.mutex); destroy_workqueue(smcd->event_wq); + kfree(smcd->conn); + kfree(smcd); } /* SMCD Device event handler. Called from ISM device interrupt handler. diff --git a/net/smc/smc_loopback.c b/net/smc/smc_loopback.c deleted file mode 100644 index c4020653ae20..000000000000 --- a/net/smc/smc_loopback.c +++ /dev/null @@ -1,427 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0 -/* - * Shared Memory Communications Direct over loopback-ism device. - * - * Functions for loopback-ism device. - * - * Copyright (c) 2024, Alibaba Inc. - * - * Author: Wen Gu - * Tony Lu - * - */ - -#include -#include -#include - -#include "smc_cdc.h" -#include "smc_ism.h" -#include "smc_loopback.h" - -#define SMC_LO_V2_CAPABLE 0x1 /* loopback-ism acts as ISMv2 */ -#define SMC_LO_SUPPORT_NOCOPY 0x1 -#define SMC_DMA_ADDR_INVALID (~(dma_addr_t)0) - -static const char smc_lo_dev_name[] = "loopback-ism"; -static struct smc_lo_dev *lo_dev; - -static void smc_lo_generate_ids(struct smc_lo_dev *ldev) -{ - struct smcd_gid *lgid = &ldev->local_gid; - uuid_t uuid; - - uuid_gen(&uuid); - memcpy(&lgid->gid, &uuid, sizeof(lgid->gid)); - memcpy(&lgid->gid_ext, (u8 *)&uuid + sizeof(lgid->gid), - sizeof(lgid->gid_ext)); - - ldev->chid = SMC_LO_RESERVED_CHID; -} - -static int smc_lo_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, - u32 vid_valid, u32 vid) -{ - struct smc_lo_dev *ldev = smcd->priv; - - /* rgid should be the same as lgid */ - if (!ldev || rgid->gid != ldev->local_gid.gid || - rgid->gid_ext != ldev->local_gid.gid_ext) - return -ENETUNREACH; - return 0; -} - -static int smc_lo_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, - void *client_priv) -{ - struct smc_lo_dmb_node *dmb_node, *tmp_node; - struct smc_lo_dev *ldev = smcd->priv; - int sba_idx, rc; - - /* check space for new dmb */ - for_each_clear_bit(sba_idx, ldev->sba_idx_mask, SMC_LO_MAX_DMBS) { - if (!test_and_set_bit(sba_idx, ldev->sba_idx_mask)) - break; - } - if (sba_idx == SMC_LO_MAX_DMBS) - return -ENOSPC; - - dmb_node = kzalloc(sizeof(*dmb_node), GFP_KERNEL); - if (!dmb_node) { - rc = -ENOMEM; - goto err_bit; - } - - dmb_node->sba_idx = sba_idx; - dmb_node->len = dmb->dmb_len; - dmb_node->cpu_addr = kzalloc(dmb_node->len, GFP_KERNEL | - __GFP_NOWARN | __GFP_NORETRY | - __GFP_NOMEMALLOC); - if (!dmb_node->cpu_addr) { - rc = -ENOMEM; - goto err_node; - } - dmb_node->dma_addr = SMC_DMA_ADDR_INVALID; - refcount_set(&dmb_node->refcnt, 1); - -again: - /* add new dmb into hash table */ - get_random_bytes(&dmb_node->token, sizeof(dmb_node->token)); - write_lock_bh(&ldev->dmb_ht_lock); - hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_node->token) { - if (tmp_node->token == dmb_node->token) { - write_unlock_bh(&ldev->dmb_ht_lock); - goto again; - } - } - hash_add(ldev->dmb_ht, &dmb_node->list, dmb_node->token); - write_unlock_bh(&ldev->dmb_ht_lock); - atomic_inc(&ldev->dmb_cnt); - - dmb->sba_idx = dmb_node->sba_idx; - dmb->dmb_tok = dmb_node->token; - dmb->cpu_addr = dmb_node->cpu_addr; - dmb->dma_addr = dmb_node->dma_addr; - dmb->dmb_len = dmb_node->len; - - return 0; - -err_node: - kfree(dmb_node); -err_bit: - clear_bit(sba_idx, ldev->sba_idx_mask); - return rc; -} - -static void __smc_lo_unregister_dmb(struct smc_lo_dev *ldev, - struct smc_lo_dmb_node *dmb_node) -{ - /* remove dmb from hash table */ - write_lock_bh(&ldev->dmb_ht_lock); - hash_del(&dmb_node->list); - write_unlock_bh(&ldev->dmb_ht_lock); - - clear_bit(dmb_node->sba_idx, ldev->sba_idx_mask); - kvfree(dmb_node->cpu_addr); - kfree(dmb_node); - - if (atomic_dec_and_test(&ldev->dmb_cnt)) - wake_up(&ldev->ldev_release); -} - -static int smc_lo_unregister_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) -{ - struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; - struct smc_lo_dev *ldev = smcd->priv; - - /* find dmb from hash table */ - read_lock_bh(&ldev->dmb_ht_lock); - hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { - if (tmp_node->token == dmb->dmb_tok) { - dmb_node = tmp_node; - break; - } - } - if (!dmb_node) { - read_unlock_bh(&ldev->dmb_ht_lock); - return -EINVAL; - } - read_unlock_bh(&ldev->dmb_ht_lock); - - if (refcount_dec_and_test(&dmb_node->refcnt)) - __smc_lo_unregister_dmb(ldev, dmb_node); - return 0; -} - -static int smc_lo_support_dmb_nocopy(struct smcd_dev *smcd) -{ - return SMC_LO_SUPPORT_NOCOPY; -} - -static int smc_lo_attach_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) -{ - struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; - struct smc_lo_dev *ldev = smcd->priv; - - /* find dmb_node according to dmb->dmb_tok */ - read_lock_bh(&ldev->dmb_ht_lock); - hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb->dmb_tok) { - if (tmp_node->token == dmb->dmb_tok) { - dmb_node = tmp_node; - break; - } - } - if (!dmb_node) { - read_unlock_bh(&ldev->dmb_ht_lock); - return -EINVAL; - } - read_unlock_bh(&ldev->dmb_ht_lock); - - if (!refcount_inc_not_zero(&dmb_node->refcnt)) - /* the dmb is being unregistered, but has - * not been removed from the hash table. - */ - return -EINVAL; - - /* provide dmb information */ - dmb->sba_idx = dmb_node->sba_idx; - dmb->dmb_tok = dmb_node->token; - dmb->cpu_addr = dmb_node->cpu_addr; - dmb->dma_addr = dmb_node->dma_addr; - dmb->dmb_len = dmb_node->len; - return 0; -} - -static int smc_lo_detach_dmb(struct smcd_dev *smcd, u64 token) -{ - struct smc_lo_dmb_node *dmb_node = NULL, *tmp_node; - struct smc_lo_dev *ldev = smcd->priv; - - /* find dmb_node according to dmb->dmb_tok */ - read_lock_bh(&ldev->dmb_ht_lock); - hash_for_each_possible(ldev->dmb_ht, tmp_node, list, token) { - if (tmp_node->token == token) { - dmb_node = tmp_node; - break; - } - } - if (!dmb_node) { - read_unlock_bh(&ldev->dmb_ht_lock); - return -EINVAL; - } - read_unlock_bh(&ldev->dmb_ht_lock); - - if (refcount_dec_and_test(&dmb_node->refcnt)) - __smc_lo_unregister_dmb(ldev, dmb_node); - return 0; -} - -static int smc_lo_move_data(struct smcd_dev *smcd, u64 dmb_tok, - unsigned int idx, bool sf, unsigned int offset, - void *data, unsigned int size) -{ - struct smc_lo_dmb_node *rmb_node = NULL, *tmp_node; - struct smc_lo_dev *ldev = smcd->priv; - struct smc_connection *conn; - - if (!sf) - /* since sndbuf is merged with peer DMB, there is - * no need to copy data from sndbuf to peer DMB. - */ - return 0; - - read_lock_bh(&ldev->dmb_ht_lock); - hash_for_each_possible(ldev->dmb_ht, tmp_node, list, dmb_tok) { - if (tmp_node->token == dmb_tok) { - rmb_node = tmp_node; - break; - } - } - if (!rmb_node) { - read_unlock_bh(&ldev->dmb_ht_lock); - return -EINVAL; - } - memcpy((char *)rmb_node->cpu_addr + offset, data, size); - read_unlock_bh(&ldev->dmb_ht_lock); - - conn = smcd->conn[rmb_node->sba_idx]; - if (!conn || conn->killed) - return -EPIPE; - tasklet_schedule(&conn->rx_tsklet); - return 0; -} - -static int smc_lo_supports_v2(void) -{ - return SMC_LO_V2_CAPABLE; -} - -static void smc_lo_get_local_gid(struct smcd_dev *smcd, - struct smcd_gid *smcd_gid) -{ - struct smc_lo_dev *ldev = smcd->priv; - - smcd_gid->gid = ldev->local_gid.gid; - smcd_gid->gid_ext = ldev->local_gid.gid_ext; -} - -static u16 smc_lo_get_chid(struct smcd_dev *smcd) -{ - return ((struct smc_lo_dev *)smcd->priv)->chid; -} - -static struct device *smc_lo_get_dev(struct smcd_dev *smcd) -{ - return &((struct smc_lo_dev *)smcd->priv)->dev; -} - -static const struct smcd_ops lo_ops = { - .query_remote_gid = smc_lo_query_rgid, - .register_dmb = smc_lo_register_dmb, - .unregister_dmb = smc_lo_unregister_dmb, - .support_dmb_nocopy = smc_lo_support_dmb_nocopy, - .attach_dmb = smc_lo_attach_dmb, - .detach_dmb = smc_lo_detach_dmb, - .add_vlan_id = NULL, - .del_vlan_id = NULL, - .set_vlan_required = NULL, - .reset_vlan_required = NULL, - .signal_event = NULL, - .move_data = smc_lo_move_data, - .supports_v2 = smc_lo_supports_v2, - .get_local_gid = smc_lo_get_local_gid, - .get_chid = smc_lo_get_chid, - .get_dev = smc_lo_get_dev, -}; - -static struct smcd_dev *smcd_lo_alloc_dev(const struct smcd_ops *ops, - int max_dmbs) -{ - struct smcd_dev *smcd; - - smcd = kzalloc(sizeof(*smcd), GFP_KERNEL); - if (!smcd) - return NULL; - - smcd->conn = kcalloc(max_dmbs, sizeof(struct smc_connection *), - GFP_KERNEL); - if (!smcd->conn) - goto out_smcd; - - smcd->ops = ops; - - spin_lock_init(&smcd->lock); - spin_lock_init(&smcd->lgr_lock); - INIT_LIST_HEAD(&smcd->vlan); - INIT_LIST_HEAD(&smcd->lgr_list); - init_waitqueue_head(&smcd->lgrs_deleted); - return smcd; - -out_smcd: - kfree(smcd); - return NULL; -} - -static int smcd_lo_register_dev(struct smc_lo_dev *ldev) -{ - struct smcd_dev *smcd; - - smcd = smcd_lo_alloc_dev(&lo_ops, SMC_LO_MAX_DMBS); - if (!smcd) - return -ENOMEM; - ldev->smcd = smcd; - smcd->priv = ldev; - smc_ism_set_v2_capable(); - mutex_lock(&smcd_dev_list.mutex); - list_add(&smcd->list, &smcd_dev_list.list); - mutex_unlock(&smcd_dev_list.mutex); - pr_warn_ratelimited("smc: adding smcd device %s\n", - dev_name(&ldev->dev)); - return 0; -} - -static void smcd_lo_unregister_dev(struct smc_lo_dev *ldev) -{ - struct smcd_dev *smcd = ldev->smcd; - - pr_warn_ratelimited("smc: removing smcd device %s\n", - dev_name(&ldev->dev)); - smcd->going_away = 1; - smc_smcd_terminate_all(smcd); - mutex_lock(&smcd_dev_list.mutex); - list_del_init(&smcd->list); - mutex_unlock(&smcd_dev_list.mutex); - kfree(smcd->conn); - kfree(smcd); -} - -static int smc_lo_dev_init(struct smc_lo_dev *ldev) -{ - smc_lo_generate_ids(ldev); - rwlock_init(&ldev->dmb_ht_lock); - hash_init(ldev->dmb_ht); - atomic_set(&ldev->dmb_cnt, 0); - init_waitqueue_head(&ldev->ldev_release); - - return smcd_lo_register_dev(ldev); -} - -static void smc_lo_dev_exit(struct smc_lo_dev *ldev) -{ - smcd_lo_unregister_dev(ldev); - if (atomic_read(&ldev->dmb_cnt)) - wait_event(ldev->ldev_release, !atomic_read(&ldev->dmb_cnt)); -} - -static void smc_lo_dev_release(struct device *dev) -{ - struct smc_lo_dev *ldev = - container_of(dev, struct smc_lo_dev, dev); - - kfree(ldev); -} - -static int smc_lo_dev_probe(void) -{ - struct smc_lo_dev *ldev; - int ret; - - ldev = kzalloc(sizeof(*ldev), GFP_KERNEL); - if (!ldev) - return -ENOMEM; - - ldev->dev.parent = NULL; - ldev->dev.release = smc_lo_dev_release; - device_initialize(&ldev->dev); - dev_set_name(&ldev->dev, smc_lo_dev_name); - - ret = smc_lo_dev_init(ldev); - if (ret) - goto free_dev; - - lo_dev = ldev; /* global loopback device */ - return 0; - -free_dev: - put_device(&ldev->dev); - return ret; -} - -static void smc_lo_dev_remove(void) -{ - if (!lo_dev) - return; - - smc_lo_dev_exit(lo_dev); - put_device(&lo_dev->dev); /* device_initialize in smc_lo_dev_probe */ -} - -int smc_loopback_init(void) -{ - return smc_lo_dev_probe(); -} - -void smc_loopback_exit(void) -{ - smc_lo_dev_remove(); -} diff --git a/net/smc/smc_loopback.h b/net/smc/smc_loopback.h deleted file mode 100644 index 04dc6808d2e1..000000000000 --- a/net/smc/smc_loopback.h +++ /dev/null @@ -1,60 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* - * Shared Memory Communications Direct over loopback-ism device. - * - * SMC-D loopback-ism device structure definitions. - * - * Copyright (c) 2024, Alibaba Inc. - * - * Author: Wen Gu - * Tony Lu - * - */ - -#ifndef _SMC_LOOPBACK_H -#define _SMC_LOOPBACK_H - -#include -#include - -#if IS_ENABLED(CONFIG_SMC_LO) -#define SMC_LO_MAX_DMBS 5000 -#define SMC_LO_DMBS_HASH_BITS 12 -#define SMC_LO_RESERVED_CHID 0xFFFF - -struct smc_lo_dmb_node { - struct hlist_node list; - u64 token; - u32 len; - u32 sba_idx; - void *cpu_addr; - dma_addr_t dma_addr; - refcount_t refcnt; -}; - -struct smc_lo_dev { - struct smcd_dev *smcd; - struct device dev; - u16 chid; - struct smcd_gid local_gid; - atomic_t dmb_cnt; - rwlock_t dmb_ht_lock; - DECLARE_BITMAP(sba_idx_mask, SMC_LO_MAX_DMBS); - DECLARE_HASHTABLE(dmb_ht, SMC_LO_DMBS_HASH_BITS); - wait_queue_head_t ldev_release; -}; - -int smc_loopback_init(void); -void smc_loopback_exit(void); -#else -static inline int smc_loopback_init(void) -{ - return 0; -} - -static inline void smc_loopback_exit(void) -{ -} -#endif - -#endif /* _SMC_LOOPBACK_H */ From patchwork Wed Jan 15 19:55:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940863 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67BF61CF28B; Wed, 15 Jan 2025 19:55:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970943; cv=none; b=Xu+9DoVGiNb0b7gSG2JKldR8nLtLLuivedBk7CWZgiSSJPAQUt35khLuTBppAypw1keeelIi58fqJwG1FMB2eiBqKr+pZQ7BBfnXjwUMVAX5xuvhHmNMrjIbV8HiDIK33yo3EkV9EeQqtKnXOD9UcpoHEG8jqQ65Lfi83NsxXBQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970943; c=relaxed/simple; bh=IVIqb1E2G21kFL300bjSUrvLdvaPerISTODAvx9ueQA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dI+N9qAlb663LPDVLh7ovBuZKg/nDB8NR9PZFRoY6CbF9FCXi2RxEQ+Z8sKDLFQbAEkFOllnGEtnklMX5QmtD8KonjNt0USeZOG1obP0l6wRxJtYUk/431RkdOXiWnKk4jwWWhdZM0OfuYQwCHS7VDLNicP9+RE91WuRieKH+Ik= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Rx2pLSO7; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Rx2pLSO7" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX52H022820; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=hQLKBEnKlzYzeuVGJ JMQR/qXty4CyBanDm5QwbDITX8=; b=Rx2pLSO7QNivE+JBhYVgIoYI/j4PH8DFj xyQtWPxZl2WqWQWSp5kxRg917u0mnSHsIqJCCjQcI7ESoHV5ssUrfgacgL/AQZQO PPy55GbS7A9KPL5dX0K9JMZO6XD0DlIaITBXOTwyxKkSkrPX0EP6cDw7gh56vg0M 6GLaWK5w41+4t46hSWdcmTgahTdlm0E6885GKw/WnfrGscrQsKT5IrZGy1fqUsIO KHaSqPAOtZpKlU8Brfa5jB8cFdaJh5ZoNI2JRCuIuZMG1NR5B4Fm7YBY+252PIEL O27Wm0aST/HoEpvJT+dQoq0u5C8f5huhzh6b/Ez/81Rt9C29xCy8g== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy64-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from m0356516.ppops.net (m0356516.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJtW3e006731; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4461rbmy5x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHnm14017359; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4444fka42k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtSWn52887960 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:28 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F3D5E2004B; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D114820040; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 54E95E0D9C; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 6/7] s390/ism: Define ismvp_dev Date: Wed, 15 Jan 2025 20:55:26 +0100 Message-ID: <20250115195527.2094320-7-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: GrVmxLr8UM5Z2DvJFggJ7RckaQZlM8ra X-Proofpoint-GUID: h8YN9VoysU9Cj_4KmwUWr3oh-KbI7SSl X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=846 adultscore=0 bulkscore=0 phishscore=0 priorityscore=1501 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Move the fields that are specific to the s390 ism_vpci driver out of the generic ism_dev into a local ismvp_dev structure. Signed-off-by: Alexandra Winter --- drivers/s390/net/ism.h | 11 +++++ drivers/s390/net/ism_drv.c | 87 +++++++++++++++++++++++--------------- include/linux/ism.h | 20 +++------ 3 files changed, 71 insertions(+), 47 deletions(-) diff --git a/drivers/s390/net/ism.h b/drivers/s390/net/ism.h index 0deca6d0e328..720a783ebf90 100644 --- a/drivers/s390/net/ism.h +++ b/drivers/s390/net/ism.h @@ -196,6 +196,17 @@ struct ism_sba { u16 dmbe_mask[ISM_NR_DMBS]; }; +struct ismvp_dev { + struct ism_dev ism; + struct ism_sba *sba; + dma_addr_t sba_dma_addr; + DECLARE_BITMAP(sba_bitmap, ISM_NR_DMBS); + + struct ism_eq *ieq; + dma_addr_t ieq_dma_addr; + int ieq_idx; +}; + #define ISM_CREATE_REQ(dmb, idx, sf, offset) \ ((dmb) | (idx) << 24 | (sf) << 23 | (offset)) diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index c0954d6dd9f5..c1fb65db504c 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -84,6 +84,7 @@ static int query_info(struct ism_dev *ism) static int register_sba(struct ism_dev *ism) { + struct ismvp_dev *ismvp; union ism_reg_sba cmd; dma_addr_t dma_handle; struct ism_sba *sba; @@ -103,14 +104,16 @@ static int register_sba(struct ism_dev *ism) return -EIO; } - ism->sba = sba; - ism->sba_dma_addr = dma_handle; + ismvp = container_of(ism, struct ismvp_dev, ism); + ismvp->sba = sba; + ismvp->sba_dma_addr = dma_handle; return 0; } static int register_ieq(struct ism_dev *ism) { + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); union ism_reg_ieq cmd; dma_addr_t dma_handle; struct ism_eq *ieq; @@ -131,18 +134,19 @@ static int register_ieq(struct ism_dev *ism) return -EIO; } - ism->ieq = ieq; - ism->ieq_idx = -1; - ism->ieq_dma_addr = dma_handle; + ismvp->ieq = ieq; + ismvp->ieq_idx = -1; + ismvp->ieq_dma_addr = dma_handle; return 0; } static int unregister_sba(struct ism_dev *ism) { + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); int ret; - if (!ism->sba) + if (!ismvp->sba) return 0; ret = ism_cmd_simple(ism, ISM_UNREG_SBA); @@ -150,19 +154,20 @@ static int unregister_sba(struct ism_dev *ism) return -EIO; dma_free_coherent(ism->dev.parent, PAGE_SIZE, - ism->sba, ism->sba_dma_addr); + ismvp->sba, ismvp->sba_dma_addr); - ism->sba = NULL; - ism->sba_dma_addr = 0; + ismvp->sba = NULL; + ismvp->sba_dma_addr = 0; return 0; } static int unregister_ieq(struct ism_dev *ism) { + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); int ret; - if (!ism->ieq) + if (!ismvp->ieq) return 0; ret = ism_cmd_simple(ism, ISM_UNREG_IEQ); @@ -170,10 +175,10 @@ static int unregister_ieq(struct ism_dev *ism) return -EIO; dma_free_coherent(ism->dev.parent, PAGE_SIZE, - ism->ieq, ism->ieq_dma_addr); + ismvp->ieq, ismvp->ieq_dma_addr); - ism->ieq = NULL; - ism->ieq_dma_addr = 0; + ismvp->ieq = NULL; + ismvp->ieq_dma_addr = 0; return 0; } @@ -215,7 +220,9 @@ static int ism_query_rgid(struct ism_dev *ism, uuid_t *rgid, u32 vid_valid, static void ism_free_dmb(struct ism_dev *ism, struct ism_dmb *dmb) { - clear_bit(dmb->sba_idx, ism->sba_bitmap); + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); + + clear_bit(dmb->sba_idx, ismvp->sba_bitmap); dma_unmap_page(ism->dev.parent, dmb->dma_addr, dmb->dmb_len, DMA_FROM_DEVICE); folio_put(virt_to_folio(dmb->cpu_addr)); @@ -223,6 +230,7 @@ static void ism_free_dmb(struct ism_dev *ism, struct ism_dmb *dmb) static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) { + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); struct folio *folio; unsigned long bit; int rc; @@ -231,7 +239,7 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) return -EINVAL; if (!dmb->sba_idx) { - bit = find_next_zero_bit(ism->sba_bitmap, ISM_NR_DMBS, + bit = find_next_zero_bit(ismvp->sba_bitmap, ISM_NR_DMBS, ISM_DMB_BIT_OFFSET); if (bit == ISM_NR_DMBS) return -ENOSPC; @@ -239,7 +247,7 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) dmb->sba_idx = bit; } if (dmb->sba_idx < ISM_DMB_BIT_OFFSET || - test_and_set_bit(dmb->sba_idx, ism->sba_bitmap)) + test_and_set_bit(dmb->sba_idx, ismvp->sba_bitmap)) return -EINVAL; folio = folio_alloc(GFP_KERNEL | __GFP_NOWARN | __GFP_NOMEMALLOC | @@ -264,7 +272,7 @@ static int ism_alloc_dmb(struct ism_dev *ism, struct ism_dmb *dmb) out_free: kfree(dmb->cpu_addr); out_bit: - clear_bit(dmb->sba_idx, ism->sba_bitmap); + clear_bit(dmb->sba_idx, ismvp->sba_bitmap); return rc; } @@ -424,15 +432,16 @@ static u16 ism_get_chid(struct ism_dev *ism) static void ism_handle_event(struct ism_dev *ism) { + struct ismvp_dev *ismvp = container_of(ism, struct ismvp_dev, ism); struct ism_event *entry; struct ism_client *clt; int i; - while ((ism->ieq_idx + 1) != READ_ONCE(ism->ieq->header.idx)) { - if (++(ism->ieq_idx) == ARRAY_SIZE(ism->ieq->entry)) - ism->ieq_idx = 0; + while ((ismvp->ieq_idx + 1) != READ_ONCE(ismvp->ieq->header.idx)) { + if (++ismvp->ieq_idx == ARRAY_SIZE(ismvp->ieq->entry)) + ismvp->ieq_idx = 0; - entry = &ism->ieq->entry[ism->ieq_idx]; + entry = &ismvp->ieq->entry[ismvp->ieq_idx]; debug_event(ism_debug_info, 2, entry, sizeof(*entry)); for (i = 0; i < MAX_CLIENTS; ++i) { clt = ism->subs[i]; @@ -445,16 +454,19 @@ static void ism_handle_event(struct ism_dev *ism) static irqreturn_t ism_handle_irq(int irq, void *data) { struct ism_dev *ism = data; + struct ismvp_dev *ismvp; unsigned long bit, end; unsigned long *bv; u16 dmbemask; u8 client_id; - bv = (void *) &ism->sba->dmb_bits[ISM_DMB_WORD_OFFSET]; - end = sizeof(ism->sba->dmb_bits) * BITS_PER_BYTE - ISM_DMB_BIT_OFFSET; + ismvp = container_of(ism, struct ismvp_dev, ism); + + bv = (void *)&ismvp->sba->dmb_bits[ISM_DMB_WORD_OFFSET]; + end = sizeof(ismvp->sba->dmb_bits) * BITS_PER_BYTE - ISM_DMB_BIT_OFFSET; spin_lock(&ism->lock); - ism->sba->s = 0; + ismvp->sba->s = 0; barrier(); for (bit = 0;;) { bit = find_next_bit_inv(bv, end, bit); @@ -462,8 +474,8 @@ static irqreturn_t ism_handle_irq(int irq, void *data) break; clear_bit_inv(bit, bv); - dmbemask = ism->sba->dmbe_mask[bit + ISM_DMB_BIT_OFFSET]; - ism->sba->dmbe_mask[bit + ISM_DMB_BIT_OFFSET] = 0; + dmbemask = ismvp->sba->dmbe_mask[bit + ISM_DMB_BIT_OFFSET]; + ismvp->sba->dmbe_mask[bit + ISM_DMB_BIT_OFFSET] = 0; barrier(); client_id = ism->sba_client_arr[bit]; if (unlikely(client_id == NO_CLIENT || !ism->subs[client_id])) @@ -471,8 +483,8 @@ static irqreturn_t ism_handle_irq(int irq, void *data) ism->subs[client_id]->handle_irq(ism, bit + ISM_DMB_BIT_OFFSET, dmbemask); } - if (ism->sba->e) { - ism->sba->e = 0; + if (ismvp->sba->e) { + ismvp->sba->e = 0; barrier(); ism_handle_event(ism); } @@ -480,7 +492,7 @@ static irqreturn_t ism_handle_irq(int irq, void *data) return IRQ_HANDLED; } -static const struct ism_ops ism_vp_ops = { +static const struct ism_ops ismvp_ops = { .query_remote_gid = ism_query_rgid, .register_dmb = ism_register_dmb, .unregister_dmb = ism_unregister_dmb, @@ -531,7 +543,7 @@ static int ism_dev_init(struct ism_dev *ism) else ism_v2_capable = false; - ism->ops = &ism_vp_ops; + ism->ops = &ismvp_ops; ism_dev_register(ism); query_info(ism); @@ -553,12 +565,14 @@ static int ism_dev_init(struct ism_dev *ism) static int ism_probe(struct pci_dev *pdev, const struct pci_device_id *id) { + struct ismvp_dev *ismvp; struct ism_dev *ism; int ret; - ism = kzalloc(sizeof(*ism), GFP_KERNEL); - if (!ism) + ismvp = kzalloc(sizeof(*ismvp), GFP_KERNEL); + if (!ismvp) return -ENOMEM; + ism = &ismvp->ism; spin_lock_init(&ism->lock); dev_set_drvdata(&pdev->dev, ism); @@ -599,6 +613,7 @@ static int ism_probe(struct pci_dev *pdev, const struct pci_device_id *id) device_del(&ism->dev); err_dev: dev_set_drvdata(&pdev->dev, NULL); + kfree(ismvp); return ret; } @@ -627,7 +642,11 @@ static void ism_dev_exit(struct ism_dev *ism) static void ism_remove(struct pci_dev *pdev) { - struct ism_dev *ism = dev_get_drvdata(&pdev->dev); + struct ismvp_dev *ismvp; + struct ism_dev *ism; + + ism = dev_get_drvdata(&pdev->dev); + ismvp = container_of(ism, struct ismvp_dev, ism); ism_dev_exit(ism); @@ -635,7 +654,7 @@ static void ism_remove(struct pci_dev *pdev) pci_disable_device(pdev); device_del(&ism->dev); dev_set_drvdata(&pdev->dev, NULL); - kfree(ism); + kfree(ismvp); } static struct pci_driver ism_driver = { diff --git a/include/linux/ism.h b/include/linux/ism.h index 929a1f275419..f28238fb5d74 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -281,24 +281,18 @@ struct ism_ops { struct ism_dev { const struct ism_ops *ops; - spinlock_t lock; /* protects the ism device */ struct list_head list; - struct pci_dev *pdev; - - struct ism_sba *sba; - dma_addr_t sba_dma_addr; - DECLARE_BITMAP(sba_bitmap, ISM_NR_DMBS); - u8 *sba_client_arr; /* entries are indices into 'clients' array */ - void *priv[MAX_CLIENTS]; - - struct ism_eq *ieq; - dma_addr_t ieq_dma_addr; - struct device dev; uuid_t gid; - int ieq_idx; + /* get this lock before accessing any of the fields below */ + spinlock_t lock; + /* indexed by dmb idx; entries are indices into priv and subs arrays: */ + u8 *sba_client_arr; + /* Sparse array of all ISM clients */ struct ism_client *subs[MAX_CLIENTS]; + /* priv pointer per client; for client usage only */ + void *priv[MAX_CLIENTS]; }; int ism_dev_register(struct ism_dev *ism); From patchwork Wed Jan 15 19:55:27 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandra Winter X-Patchwork-Id: 13940866 X-Patchwork-Delegate: kuba@kernel.org Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2F441D8E01; Wed, 15 Jan 2025 19:55:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970946; cv=none; b=OHr8Pr4pbsbgSn8LSNZ29v2ul6DY+UNoLTktndpnTVUgLbct1CiK0MswnTU0qbiKQJxpL5Fb8pwXD55tgQav+/H//PZaa5ZqERJUaaqi9MwU/U9h9FWGZ7BDkB0c+HkQELWW3Mm4Bk0+zIS2wiET7wRurSSGCYhdxTaWeb/W5eg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736970946; c=relaxed/simple; bh=1Qx6dmW2ZxwoHVOgTN1cEe5+zZiPmfPpc44tF6HIa1k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kM+Tut7Rozh1bMv3dHEBbgF2Xa9M2yliBrZ+pg7wq+vDWPLXjmWJ/9zR6E1GMga+9lCAPVuxjGt0FcE9Mfqc9kzkiJJv41npoun0EI8XAyyY7jGzcR9/C6yN3Je+SNgiitq9tpdBysi+u903q0Lay/oZlATzAswenHirfolCaec= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=C6WAAa8K; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="C6WAAa8K" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHX5ri024175; Wed, 15 Jan 2025 19:55:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+mOFwOsB0j3fAlYuj JfsCgnZG9nq6oG0Z5EBWll6+M4=; b=C6WAAa8KPZhfRSsDXdRGwlaDkI9tv6L6Y antppcrPaUQ4K2W68y1j9Qoc4ddQ8PSM/RVO1HJsLIBvyVKFMZYdjz/vb/dOIJKC mcAmTiPx5+2JvT7OeQ9Vzvk4pKnDhfMVbyr1HZxmEaw2WjFyAEWwZZaCfEqms6j7 BJvA6X+xLLXMofxh6mdQlJyFK54JbF23vXs/ujOQcBQ3lOUaDDeig6dlTJSnaUTX o4xJu0ptJIl/6eDvPHLYuYkX4yYfyi3SVd83OLwcxJM7pk9YSa+HjQFNEe3m/E8O w8S6StFJSi0f/FoC9HZ2xAFqFKCciVPQwl9AJAqxKpnxPuxJsOf/A== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4469733c43-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:33 +0000 (GMT) Received: from m0360072.ppops.net (m0360072.ppops.net [127.0.0.1]) by pps.reinject (8.18.0.8/8.18.0.8) with ESMTP id 50FJtW6J005240; Wed, 15 Jan 2025 19:55:32 GMT Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4469733c3x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:32 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 50FHhO1v007371; Wed, 15 Jan 2025 19:55:31 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4443yna6rv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 15 Jan 2025 19:55:31 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 50FJtSrO52363766 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 15 Jan 2025 19:55:28 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 012C120040; Wed, 15 Jan 2025 19:55:28 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CF71220063; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTPS; Wed, 15 Jan 2025 19:55:27 +0000 (GMT) Received: by tuxmaker.boeblingen.de.ibm.com (Postfix, from userid 55271) id 57EB3E0D9D; Wed, 15 Jan 2025 20:55:27 +0100 (CET) From: Alexandra Winter To: Wenjia Zhang , Jan Karcher , Gerd Bayer , Alexandra Winter , Halil Pasic , "D. Wythe" , Tony Lu , Wen Gu , Peter Oberparleiter , David Miller , Jakub Kicinski , Paolo Abeni , Eric Dumazet , Andrew Lunn Cc: Julian Ruess , Niklas Schnelle , Thorsten Winkler , netdev@vger.kernel.org, linux-s390@vger.kernel.org, Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Simon Horman Subject: [RFC net-next 7/7] net/smc: Use only ism_ops Date: Wed, 15 Jan 2025 20:55:27 +0100 Message-ID: <20250115195527.2094320-8-wintera@linux.ibm.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20250115195527.2094320-1-wintera@linux.ibm.com> References: <20250115195527.2094320-1-wintera@linux.ibm.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: f1Pl9saKixFg9Y1_SUTAAa0jQE7yCL3Z X-Proofpoint-GUID: ytb2mwGGoBK4zm63BuNwsGPQ_Wi_Z-td X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-01-15_09,2025-01-15_02,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 bulkscore=0 malwarescore=0 adultscore=0 priorityscore=1501 phishscore=0 clxscore=1015 suspectscore=0 spamscore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2411120000 definitions=main-2501150142 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Replace smcd_ops by using ism_ops directly. Signed-off-by: Alexandra Winter --- include/linux/ism.h | 1 + include/net/smc.h | 30 ------ net/smc/smc_clc.c | 6 +- net/smc/smc_core.c | 6 +- net/smc/smc_diag.c | 2 +- net/smc/smc_ism.c | 222 ++++++++------------------------------------ net/smc/smc_ism.h | 8 +- net/smc/smc_pnet.c | 8 +- 8 files changed, 55 insertions(+), 228 deletions(-) diff --git a/include/linux/ism.h b/include/linux/ism.h index f28238fb5d74..c11de3931722 100644 --- a/include/linux/ism.h +++ b/include/linux/ism.h @@ -30,6 +30,7 @@ struct ism_dmb { */ u64 dmb_tok; /* rgid - GID of designated remote sending device */ + //TODO: Change to uuid_t GID. Ok for now, because loopback ignores it. u64 rgid; u32 dmb_len; /* sba_idx - Index of this DMB on this receiving device */ diff --git a/include/net/smc.h b/include/net/smc.h index 7a96ed2ae20c..a8235de6cf0a 100644 --- a/include/net/smc.h +++ b/include/net/smc.h @@ -28,43 +28,13 @@ struct smc_hashinfo { /* SMCD/ISM device driver interface */ -struct smcd_dev; - struct smcd_gid { u64 gid; u64 gid_ext; }; -struct smcd_ops { - int (*query_remote_gid)(struct smcd_dev *dev, struct smcd_gid *rgid, - u32 vid_valid, u32 vid); - int (*register_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb, - void *client); - int (*unregister_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb); - int (*move_data)(struct smcd_dev *dev, u64 dmb_tok, unsigned int idx, - bool sf, unsigned int offset, void *data, - unsigned int size); - int (*supports_v2)(void); - void (*get_local_gid)(struct smcd_dev *dev, struct smcd_gid *gid); - u16 (*get_chid)(struct smcd_dev *dev); - struct device* (*get_dev)(struct smcd_dev *dev); - - /* optional operations */ - int (*add_vlan_id)(struct smcd_dev *dev, u64 vlan_id); - int (*del_vlan_id)(struct smcd_dev *dev, u64 vlan_id); - int (*set_vlan_required)(struct smcd_dev *dev); - int (*reset_vlan_required)(struct smcd_dev *dev); - int (*signal_event)(struct smcd_dev *dev, struct smcd_gid *rgid, - u32 trigger_irq, u32 event_code, u64 info); - int (*support_dmb_nocopy)(struct smcd_dev *dev); - int (*attach_dmb)(struct smcd_dev *dev, struct ism_dmb *dmb); - int (*detach_dmb)(struct smcd_dev *dev, u64 token); -}; - struct smcd_dev { - const struct smcd_ops *ops; struct ism_dev *ism; - struct ism_client *client; struct list_head list; spinlock_t lock; struct smc_connection **conn; diff --git a/net/smc/smc_clc.c b/net/smc/smc_clc.c index 33fa787c28eb..b546999f83a4 100644 --- a/net/smc/smc_clc.c +++ b/net/smc/smc_clc.c @@ -900,7 +900,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) /* add SMC-D specifics */ if (ini->ism_dev[0]) { smcd = ini->ism_dev[0]; - smcd->ops->get_local_gid(smcd, &smcd_gid); + copy_to_smcdgid(&smcd_gid, &smcd->ism->gid); pclc_smcd->ism.gid = htonll(smcd_gid.gid); pclc_smcd->ism.chid = htons(smc_ism_get_chid(ini->ism_dev[0])); @@ -950,7 +950,7 @@ int smc_clc_send_proposal(struct smc_sock *smc, struct smc_init_info *ini) if (ini->ism_offered_cnt) { for (i = 1; i <= ini->ism_offered_cnt; i++) { smcd = ini->ism_dev[i]; - smcd->ops->get_local_gid(smcd, &smcd_gid); + copy_to_smcdgid(&smcd_gid, &smcd->ism->gid); gidchids[entry].chid = htons(smc_ism_get_chid(ini->ism_dev[i])); gidchids[entry].gid = htonll(smcd_gid.gid); @@ -1043,7 +1043,7 @@ smcd_clc_prep_confirm_accept(struct smc_connection *conn, /* SMC-D specific settings */ memcpy(clc->hdr.eyecatcher, SMCD_EYECATCHER, sizeof(SMCD_EYECATCHER)); - smcd->ops->get_local_gid(smcd, &smcd_gid); + copy_to_smcdgid(&smcd_gid, &smcd->ism->gid); clc->hdr.typev1 = SMC_TYPE_D; clc->d0.gid = htonll(smcd_gid.gid); clc->d0.token = htonll(conn->rmb_desc->token); diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c index d489b80a4503..dca43edfc6be 100644 --- a/net/smc/smc_core.c +++ b/net/smc/smc_core.c @@ -555,7 +555,7 @@ static int smc_nl_fill_smcd_lgr(struct smc_link_group *lgr, if (nla_put_u32(skb, SMC_NLA_LGR_D_ID, *((u32 *)&lgr->id))) goto errattr; - smcd->ops->get_local_gid(smcd, &smcd_gid); + copy_to_smcdgid(&smcd_gid, &smcd->ism->gid); if (nla_put_u64_64bit(skb, SMC_NLA_LGR_D_GID, smcd_gid.gid, SMC_NLA_LGR_D_PAD)) goto errattr; @@ -919,7 +919,7 @@ static int smc_lgr_create(struct smc_sock *smc, struct smc_init_info *ini) if (ini->is_smcd) { /* SMC-D specific settings */ smcd = ini->ism_dev[ini->ism_selected]; - get_device(smcd->ops->get_dev(smcd)); + get_device(ism_get_dev(smcd->ism)); lgr->peer_gid.gid = ini->ism_peer_gid[ini->ism_selected].gid; lgr->peer_gid.gid_ext = @@ -1469,7 +1469,7 @@ static void smc_lgr_free(struct smc_link_group *lgr) destroy_workqueue(lgr->tx_wq); if (lgr->is_smcd) { smc_ism_put_vlan(lgr->smcd, lgr->vlan_id); - put_device(lgr->smcd->ops->get_dev(lgr->smcd)); + put_device(ism_get_dev(lgr->smcd->ism)); } smc_lgr_put(lgr); /* theoretically last lgr_put */ } diff --git a/net/smc/smc_diag.c b/net/smc/smc_diag.c index 6fdb2d96777a..5e79345108d4 100644 --- a/net/smc/smc_diag.c +++ b/net/smc/smc_diag.c @@ -175,7 +175,7 @@ static int __smc_diag_dump(struct sock *sk, struct sk_buff *skb, dinfo.linkid = *((u32 *)conn->lgr->id); dinfo.peer_gid = conn->lgr->peer_gid.gid; dinfo.peer_gid_ext = conn->lgr->peer_gid.gid_ext; - smcd->ops->get_local_gid(smcd, &smcd_gid); + copy_to_smcdgid(&smcd_gid, &smcd->ism->gid); dinfo.my_gid = smcd_gid.gid; dinfo.my_gid_ext = smcd_gid.gid_ext; dinfo.token = conn->rmb_desc->token; diff --git a/net/smc/smc_ism.c b/net/smc/smc_ism.c index 22c1cfb2ad09..9d14aef52283 100644 --- a/net/smc/smc_ism.c +++ b/net/smc/smc_ism.c @@ -68,8 +68,12 @@ static void smc_ism_create_system_eid(void) int smc_ism_cantalk(struct smcd_gid *peer_gid, unsigned short vlan_id, struct smcd_dev *smcd) { - return smcd->ops->query_remote_gid(smcd, peer_gid, vlan_id ? 1 : 0, - vlan_id); + struct ism_dev *ism = smcd->ism; + uuid_t ism_rgid; + + copy_to_ismgid(&ism_rgid, peer_gid); + return ism->ops->query_remote_gid(ism, &ism_rgid, vlan_id ? 1 : 0, + vlan_id); } void smc_ism_get_system_eid(u8 **eid) @@ -82,7 +86,7 @@ void smc_ism_get_system_eid(u8 **eid) u16 smc_ism_get_chid(struct smcd_dev *smcd) { - return smcd->ops->get_chid(smcd); + return smcd->ism->ops->get_chid(smcd->ism); } /* HW supports ISM V2 and thus System EID is defined */ @@ -131,7 +135,7 @@ int smc_ism_get_vlan(struct smcd_dev *smcd, unsigned short vlanid) if (!vlanid) /* No valid vlan id */ return -EINVAL; - if (!smcd->ops->add_vlan_id) + if (!smcd->ism->ops->add_vlan_id) return -EOPNOTSUPP; /* create new vlan entry, in case we need it */ @@ -154,7 +158,7 @@ int smc_ism_get_vlan(struct smcd_dev *smcd, unsigned short vlanid) /* no existing entry found. * add new entry to device; might fail, e.g., if HW limit reached */ - if (smcd->ops->add_vlan_id(smcd, vlanid)) { + if (smcd->ism->ops->add_vlan_id(smcd->ism, vlanid)) { kfree(new_vlan); rc = -EIO; goto out; @@ -178,7 +182,7 @@ int smc_ism_put_vlan(struct smcd_dev *smcd, unsigned short vlanid) if (!vlanid) /* No valid vlan id */ return -EINVAL; - if (!smcd->ops->del_vlan_id) + if (!smcd->ism->ops->del_vlan_id) return -EOPNOTSUPP; spin_lock_irqsave(&smcd->lock, flags); @@ -196,7 +200,7 @@ int smc_ism_put_vlan(struct smcd_dev *smcd, unsigned short vlanid) } /* Found and the last reference just gone */ - if (smcd->ops->del_vlan_id(smcd, vlanid)) + if (smcd->ism->ops->del_vlan_id(smcd->ism, vlanid)) rc = -EIO; list_del(&vlan->list); kfree(vlan); @@ -219,7 +223,8 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) dmb.cpu_addr = dmb_desc->cpu_addr; dmb.dma_addr = dmb_desc->dma_addr; dmb.dmb_len = dmb_desc->len; - rc = smcd->ops->unregister_dmb(smcd, &dmb); + + rc = smcd->ism->ops->unregister_dmb(smcd->ism, &dmb); if (!rc || rc == ISM_ERROR) { dmb_desc->cpu_addr = NULL; dmb_desc->dma_addr = 0; @@ -231,6 +236,7 @@ int smc_ism_unregister_dmb(struct smcd_dev *smcd, struct smc_buf_desc *dmb_desc) int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, struct smc_buf_desc *dmb_desc) { + struct ism_dev *ism; struct ism_dmb dmb; int rc; @@ -239,7 +245,9 @@ int smc_ism_register_dmb(struct smc_link_group *lgr, int dmb_len, dmb.sba_idx = dmb_desc->sba_idx; dmb.vlan_id = lgr->vlan_id; dmb.rgid = lgr->peer_gid.gid; - rc = lgr->smcd->ops->register_dmb(lgr->smcd, &dmb, lgr->smcd->client); + + ism = lgr->smcd->ism; + rc = ism->ops->register_dmb(ism, &dmb, &smc_ism_client); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; dmb_desc->token = dmb.dmb_tok; @@ -256,8 +264,8 @@ bool smc_ism_support_dmb_nocopy(struct smcd_dev *smcd) * merging sndbuf with peer DMB to avoid * data copies between them. */ - return (smcd->ops->support_dmb_nocopy && - smcd->ops->support_dmb_nocopy(smcd)); + return (smcd->ism->ops->support_dmb_nocopy && + smcd->ism->ops->support_dmb_nocopy(smcd->ism)); } int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, @@ -266,12 +274,12 @@ int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, struct ism_dmb dmb; int rc = 0; - if (!dev->ops->attach_dmb) + if (!dev->ism->ops->attach_dmb) return -EINVAL; memset(&dmb, 0, sizeof(dmb)); dmb.dmb_tok = token; - rc = dev->ops->attach_dmb(dev, &dmb); + rc = dev->ism->ops->attach_dmb(dev->ism, &dmb); if (!rc) { dmb_desc->sba_idx = dmb.sba_idx; dmb_desc->token = dmb.dmb_tok; @@ -284,10 +292,10 @@ int smc_ism_attach_dmb(struct smcd_dev *dev, u64 token, int smc_ism_detach_dmb(struct smcd_dev *dev, u64 token) { - if (!dev->ops->detach_dmb) + if (!dev->ism->ops->detach_dmb) return -EINVAL; - return dev->ops->detach_dmb(dev, token); + return dev->ism->ops->detach_dmb(dev->ism, token); } static int smc_nl_handle_smcd_dev(struct smcd_dev *smcd, @@ -412,6 +420,8 @@ static void smcd_handle_sw_event(struct smc_ism_event_work *wrk) struct smcd_gid peer_gid = { .gid = wrk->event.tok, .gid_ext = 0 }; union smcd_sw_event_info ev_info; + struct ism_dev *ism = wrk->smcd->ism; + uuid_t ism_rgid; ev_info.info = wrk->event.info; switch (wrk->event.code) { @@ -420,14 +430,14 @@ static void smcd_handle_sw_event(struct smc_ism_event_work *wrk) break; case ISM_EVENT_CODE_TESTLINK: /* Activity timer */ if (ev_info.code == ISM_EVENT_REQUEST && - wrk->smcd->ops->signal_event) { + ism->ops->signal_event) { ev_info.code = ISM_EVENT_RESPONSE; - wrk->smcd->ops->signal_event(wrk->smcd, - &peer_gid, - ISM_EVENT_REQUEST_IR, - ISM_EVENT_CODE_TESTLINK, - ev_info.info); - } + copy_to_ismgid(&ism_rgid, &peer_gid); + ism->ops->signal_event(ism, &ism_rgid, + ISM_EVENT_REQUEST_IR, + ISM_EVENT_CODE_TESTLINK, + ev_info.info); + } break; } } @@ -453,9 +463,7 @@ static void smc_ism_event_work(struct work_struct *work) kfree(wrk); } -static struct smcd_dev *smcd_alloc_dev(const char *name, - const struct smcd_ops *ops, - int max_dmbs) +static struct smcd_dev *smcd_alloc_dev(const char *name, int max_dmbs) { struct smcd_dev *smcd; @@ -472,8 +480,6 @@ static struct smcd_dev *smcd_alloc_dev(const char *name, if (!smcd->event_wq) goto free_conn; - smcd->ops = ops; - spin_lock_init(&smcd->lock); spin_lock_init(&smcd->lgr_lock); INIT_LIST_HEAD(&smcd->vlan); @@ -488,176 +494,22 @@ static struct smcd_dev *smcd_alloc_dev(const char *name, return NULL; } -static int smcd_query_rgid(struct smcd_dev *smcd, struct smcd_gid *rgid, - u32 vid_valid, u32 vid) -{ - struct ism_dev *ism = smcd->ism; - uuid_t ism_rgid; - - copy_to_ismgid(&ism_rgid, rgid); - return ism->ops->query_remote_gid(ism, &ism_rgid, vid_valid, vid); -} - -static int smcd_register_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb, - void *client) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->register_dmb(ism, dmb, (struct ism_client *)client); -} - -static int smcd_unregister_dmb(struct smcd_dev *smcd, struct ism_dmb *dmb) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->unregister_dmb(ism, dmb); -} - -static int smcd_add_vlan_id(struct smcd_dev *smcd, u64 vlan_id) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->add_vlan_id(ism, vlan_id); -} - -static int smcd_del_vlan_id(struct smcd_dev *smcd, u64 vlan_id) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->del_vlan_id(ism, vlan_id); -} - -static int smcd_set_vlan_required(struct smcd_dev *smcd) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->set_vlan_required(ism); -} - -static int smcd_reset_vlan_required(struct smcd_dev *smcd) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->reset_vlan_required(ism); -} - -static int smcd_signal_ieq(struct smcd_dev *smcd, struct smcd_gid *rgid, - u32 trigger_irq, u32 event_code, u64 info) -{ - struct ism_dev *ism = smcd->ism; - uuid_t ism_rgid; - - copy_to_ismgid(&ism_rgid, rgid); - return ism->ops->signal_event(ism, &ism_rgid, trigger_irq, - event_code, info); -} - -static int smcd_move(struct smcd_dev *smcd, u64 dmb_tok, unsigned int idx, - bool sf, unsigned int offset, void *data, - unsigned int size) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->move_data(ism, dmb_tok, idx, sf, offset, data, size); -} - -static int smcd_supports_v2(void) -{ - return smc_ism_v2_capable; -} - -static void smcd_get_local_gid(struct smcd_dev *smcd, - struct smcd_gid *smcd_gid) -{ - struct ism_dev *ism = smcd->ism; - - copy_to_smcdgid(smcd_gid, &ism->gid); -} - -static u16 smcd_get_chid(struct smcd_dev *smcd) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->get_chid(ism); -} - -static inline struct device *smcd_get_dev(struct smcd_dev *dev) -{ - return ism_get_dev(dev->ism); -} - -static const struct smcd_ops ism_smcd_ops = { - .query_remote_gid = smcd_query_rgid, - .register_dmb = smcd_register_dmb, - .unregister_dmb = smcd_unregister_dmb, - .add_vlan_id = smcd_add_vlan_id, - .del_vlan_id = smcd_del_vlan_id, - .set_vlan_required = smcd_set_vlan_required, - .reset_vlan_required = smcd_reset_vlan_required, - .signal_event = smcd_signal_ieq, - .move_data = smcd_move, - .supports_v2 = smcd_supports_v2, - .get_local_gid = smcd_get_local_gid, - .get_chid = smcd_get_chid, - .get_dev = smcd_get_dev, -}; - -static inline int smcd_support_dmb_nocopy(struct smcd_dev *smcd) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->support_dmb_nocopy(ism); -} - -static inline int smcd_attach_dmb(struct smcd_dev *smcd, - struct ism_dmb *dmb) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->attach_dmb(ism, dmb); -} - -static inline int smcd_detach_dmb(struct smcd_dev *smcd, u64 token) -{ - struct ism_dev *ism = smcd->ism; - - return ism->ops->detach_dmb(ism, token); -} - -static const struct smcd_ops lo_ops = { - .query_remote_gid = smcd_query_rgid, - .register_dmb = smcd_register_dmb, - .unregister_dmb = smcd_unregister_dmb, - .support_dmb_nocopy = smcd_support_dmb_nocopy, - .attach_dmb = smcd_attach_dmb, - .detach_dmb = smcd_detach_dmb, - .move_data = smcd_move, - .supports_v2 = smcd_supports_v2, - .get_local_gid = smcd_get_local_gid, - .get_chid = smcd_get_chid, - .get_dev = smcd_get_dev, -}; - static void smcd_register_dev(struct ism_dev *ism) { - const struct smcd_ops *ops; struct smcd_dev *smcd, *fentry; int max_dmbs; if (ism->ops->get_chid(ism) == ISM_LO_RESERVED_CHID) { max_dmbs = ISM_LO_MAX_DMBS; - ops = &lo_ops; } else { max_dmbs = ISM_NR_DMBS; - ops = &ism_smcd_ops; } - smcd = smcd_alloc_dev(dev_name(&ism->dev), ops, max_dmbs); + smcd = smcd_alloc_dev(dev_name(&ism->dev), max_dmbs); if (!smcd) return; smcd->ism = ism; - smcd->client = &smc_ism_client; ism_set_priv(ism, &smc_ism_client, smcd); if (smc_pnetid_by_dev_port(ism->dev.parent, 0, smcd->pnetid)) @@ -760,16 +612,18 @@ int smc_ism_signal_shutdown(struct smc_link_group *lgr) int rc = 0; #if IS_ENABLED(CONFIG_ISM) union smcd_sw_event_info ev_info; + uuid_t ism_rgid; if (lgr->peer_shutdown) return 0; - if (!lgr->smcd->ops->signal_event) + if (!lgr->smcd->ism->ops->signal_event) return 0; memcpy(ev_info.uid, lgr->id, SMC_LGR_ID_SIZE); ev_info.vlan_id = lgr->vlan_id; ev_info.code = ISM_EVENT_REQUEST; - rc = lgr->smcd->ops->signal_event(lgr->smcd, &lgr->peer_gid, + copy_to_ismgid(&ism_rgid, &lgr->peer_gid); + rc = lgr->smcd->ism->ops->signal_event(lgr->smcd->ism, &ism_rgid, ISM_EVENT_REQUEST_IR, ISM_EVENT_CODE_SHUTDOWN, ev_info.info); diff --git a/net/smc/smc_ism.h b/net/smc/smc_ism.h index d041e5a7c459..e2e8cfba2575 100644 --- a/net/smc/smc_ism.h +++ b/net/smc/smc_ism.h @@ -68,7 +68,9 @@ static inline int smc_ism_write(struct smcd_dev *smcd, u64 dmb_tok, { int rc; - rc = smcd->ops->move_data(smcd, dmb_tok, idx, sf, offset, data, len); + rc = smcd->ism->ops->move_data(smcd->ism, dmb_tok, idx, sf, offset, + data, len); + return rc < 0 ? rc : 0; } @@ -85,14 +87,14 @@ static inline bool __smc_ism_is_emulated(u16 chid) static inline bool smc_ism_is_emulated(struct smcd_dev *smcd) { - u16 chid = smcd->ops->get_chid(smcd); + u16 chid = smcd->ism->ops->get_chid(smcd->ism); return __smc_ism_is_emulated(chid); } static inline bool smc_ism_is_loopback(struct smcd_dev *smcd) { - return (smcd->ops->get_chid(smcd) == 0xFFFF); + return (smcd->ism->ops->get_chid(smcd->ism) == ISM_LO_RESERVED_CHID); } static inline void copy_to_smcdgid(struct smcd_gid *sgid, uuid_t *igid) diff --git a/net/smc/smc_pnet.c b/net/smc/smc_pnet.c index 716808f374a8..397557f4b7d4 100644 --- a/net/smc/smc_pnet.c +++ b/net/smc/smc_pnet.c @@ -169,7 +169,7 @@ static int smc_pnet_remove_by_pnetid(struct net *net, char *pnet_name) pr_warn_ratelimited("smc: smcd device %s " "erased user defined pnetid " "%.16s\n", - dev_name(smcd->ops->get_dev(smcd)), + dev_name(ism_get_dev(smcd->ism)), smcd->pnetid); memset(smcd->pnetid, 0, SMC_MAX_PNETID_LEN); smcd->pnetid_by_user = false; @@ -332,7 +332,7 @@ static struct smcd_dev *smc_pnet_find_smcd(char *smcd_name) mutex_lock(&smcd_dev_list.mutex); list_for_each_entry(smcd_dev, &smcd_dev_list.list, list) { - if (!strncmp(dev_name(smcd_dev->ops->get_dev(smcd_dev)), + if (!strncmp(dev_name(ism_get_dev(smcd_dev->ism)), smcd_name, IB_DEVICE_NAME_MAX - 1)) goto out; } @@ -431,7 +431,7 @@ static int smc_pnet_add_ib(struct smc_pnettable *pnettable, char *ib_name, if (smcd) { smcddev_applied = smc_pnet_apply_smcd(smcd, pnet_name); if (smcddev_applied) { - dev = smcd->ops->get_dev(smcd); + dev = ism_get_dev(smcd->ism); pr_warn_ratelimited("smc: smcd device %s " "applied user defined pnetid " "%.16s\n", dev_name(dev), @@ -1190,7 +1190,7 @@ int smc_pnetid_by_table_ib(struct smc_ib_device *smcibdev, u8 ib_port) */ int smc_pnetid_by_table_smcd(struct smcd_dev *smcddev) { - const char *ib_name = dev_name(smcddev->ops->get_dev(smcddev)); + const char *ib_name = dev_name(ism_get_dev(smcddev->ism)); struct smc_pnettable *pnettable; struct smc_pnetentry *tmp_pe; struct smc_net *sn;