From patchwork Wed Nov 30 04:34:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13059433 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC39EC47088 for ; Wed, 30 Nov 2022 04:34:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233158AbiK3Eet (ORCPT ); Tue, 29 Nov 2022 23:34:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35794 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233176AbiK3Een (ORCPT ); Tue, 29 Nov 2022 23:34:43 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6DC955C84 for ; Tue, 29 Nov 2022 20:34:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669782882; x=1701318882; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5cZfmmuNG8UN8a9/1iXtoDrmN0X5aqXhF850c9jyTDs=; b=POBjChHpGLY6LHh+V+y2ux4b0L8y4vTezIPnPPWd2roqFbFyRifJ+itA AmM4Gb5cl2BklhvQVmlv7CbHdnWGf85V3wHNodDctc8eTVcCxDptVQARN 7+Gg4O7nihfzcYhtnuBsUnD89O4wc7V86/7meSOm6iK1L8ueiueXMyifn 6m8jo09dZNbNG4KtwuhjoXJWwnz58Rs0OqE/jybJKXxXfZMX1UbcuYvDw RnRlnaUfP4/Un9IiLZ5HTwBUkeZ9L5QBlnXH+tzEFCNvZFy1V5THngYDa oSFloQLfXgrRXsPSiJ3chHsexqMgJWwnsOgjNMiWoOY/5LTKX/4Op78d6 w==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="379583230" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="379583230" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:42 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="594525689" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="594525689" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.50.166]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:41 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang Cc: Alison Schofield , linux-cxl@vger.kernel.org Subject: [PATCH 1/5] cxl/memdev: Add support for the Inject Poison mailbox command Date: Tue, 29 Nov 2022 20:34:33 -0800 Message-Id: <3c260749c833f51d5cad9ae3912debcdf8b82753.1669781852.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield CXL devices optionally support the INJECT POISON mailbox command. Add a sysfs attribute and memdev driver support for injecting poison. When a Device Physical Address (DPA) is written to the inject_poison sysfs attribute send an inject poison command to the device for the specified address. Per the CXL Specification (8.2.9.8.4.2), after receiving a valid inject poison request, the device will return poison when the address is accessed through the CXL.mem bus. Injecting poison adds the address to the device's Poison List and the error source is set to injected error. In addition, the device adds a poison creation event to its internal Informational Event log, updates the Event Status register, and if configured, interrupts the host. Also, per the CXL Specification, it is not an error to inject poison into an address that already has poison present and no error is returned from the device. The memdev driver performs basic sanity checking on the address, however, it does not go as far as reading the poison list to see if the address is on the list. That discovery is left to the device. The inject_poison attribute is only visible for devices supporting the capability. Signed-off-by: Alison Schofield Reviewed-by: Jonathan Cameron --- Documentation/ABI/testing/sysfs-bus-cxl | 19 +++++++++ drivers/cxl/core/memdev.c | 53 +++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 3 ++ 3 files changed, 75 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index b715a4609718..20db97f7a1aa 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -416,3 +416,22 @@ Description: if accessed, and the source of the poison. The retrieved errors are logged as kernel trace events with the label 'cxl_poison'. + + +What: /sys/bus/cxl/devices/memX/inject_poison +Date: December, 2022 +KernelVersion: v6.2 +Contact: linux-cxl@vger.kernel.org +Description: + (WO) When a Device Physical Address (DPA) is written to this + attribute the memdev driver sends an inject poison command to + the device for the specified address. If successful, the device + returns poison when the address is accessed through the CXL.mem + bus. Injecting poison adds the address to the device's Poison + List and the error source is set to injected error. In addition, + the device adds a poison creation event to its internal + Informational Event log, updates the Event Status register, and + if configured, interrupts the host. It is not an error to inject + poison into an address that already has poison present and no + error is returned. The inject_poison attribute is only visible + for devices supporting the capability. diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index d08b7295a01c..71130813030f 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -142,6 +142,51 @@ static ssize_t trigger_poison_list_store(struct device *dev, } static DEVICE_ATTR_WO(trigger_poison_list); +static int cxl_validate_poison_dpa(struct cxl_dev_state *cxlds, u64 dpa) +{ + if (!resource_size(&cxlds->dpa_res)) { + dev_dbg(cxlds->dev, "device has no dpa resource\n"); + return -EINVAL; + } + if (dpa < cxlds->dpa_res.start || dpa > cxlds->dpa_res.end) { + dev_dbg(cxlds->dev, "dpa:0x%llx not in resource:%pR\n", + dpa, &cxlds->dpa_res); + return -EINVAL; + } + if ((dpa & CXL_POISON_INJECT_RESERVED) != 0) { + dev_dbg(cxlds->dev, "dpa reserve bit(s) [5:0] set 0x%llx\n", + dpa); + return -EINVAL; + } + return 0; +} + +static ssize_t inject_poison_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + u64 dpa; + int rc; + + rc = kstrtou64(buf, 0, &dpa); + if (rc) + return rc; + rc = cxl_validate_poison_dpa(cxlds, dpa); + if (rc) + return rc; + + dpa = cpu_to_le64(dpa); + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_INJECT_POISON, &dpa, + sizeof(dpa), NULL, cxlds->payload_size); + if (rc) + return rc; + + return len; +} +static DEVICE_ATTR_WO(inject_poison); + static struct attribute *cxl_memdev_attributes[] = { &dev_attr_serial.attr, &dev_attr_firmware_version.attr, @@ -149,6 +194,7 @@ static struct attribute *cxl_memdev_attributes[] = { &dev_attr_label_storage_size.attr, &dev_attr_numa_node.attr, &dev_attr_trigger_poison_list.attr, + &dev_attr_inject_poison.attr, NULL, }; @@ -175,6 +221,13 @@ static umode_t cxl_memdev_visible(struct kobject *kobj, struct attribute *a, to_cxl_memdev(dev)->cxlds->enabled_cmds)) return 0; } + if (a == &dev_attr_inject_poison.attr) { + struct device *dev = kobj_to_dev(kobj); + + if (!test_bit(CXL_MEM_COMMAND_ID_INJECT_POISON, + to_cxl_memdev(dev)->cxlds->enabled_cmds)) + return 0; + } return a->mode; } diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 19a9e545ac19..0d4c34be7335 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -396,6 +396,9 @@ struct cxl_mbox_poison_payload_out { #define CXL_POISON_SOURCE_INJECTED 3 #define CXL_POISON_SOURCE_VENDOR 7 +/* Inject & Clear Poison CXL 3.0 Spec 8.2.9.8.4.2/3 */ +#define CXL_POISON_INJECT_RESERVED GENMASK_ULL(5, 0) + /** * struct cxl_mem_command - Driver representation of a memory device command * @info: Command information as it exists for the UAPI From patchwork Wed Nov 30 04:34:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13059434 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6BFEC4332F for ; Wed, 30 Nov 2022 04:34:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233203AbiK3Ee4 (ORCPT ); Tue, 29 Nov 2022 23:34:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232600AbiK3Eep (ORCPT ); Tue, 29 Nov 2022 23:34:45 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6EF765FD1 for ; Tue, 29 Nov 2022 20:34:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669782884; x=1701318884; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sVMRnx00eGMVfOWgdmYpCvpmA/S83Mh/JeYfCI9dbRs=; b=E/UILkU9pgyozC0+nk2SXo2S9Qppa+WTxGJkQNXyJYkE+kIJZT9JecwT 0lbrt6Aji2+2Hdj9EFy/UqTPXhr7M71umT35hyFc7TRV4DkNKK73K0NjS WOqo6SmhOqcLCZZVSnM6E1+z2B2aX0QxsNO2lw0zu4/5Lbei3CX6AG9QF 4yyfYHJgjnsjprDRmirxUARLEpk7wUbGERa8Q3BwZi7de9G+jUzKyUPmr vZKcXJ0MjxxRqcQCE1S27GLnpcuKktfV4uqFl+J9UxLGj/Lf4LPjapYse +He87UrTp1O/8P7Roem0Hk9RM7ED3cfbKg0ylMmgpe6QlrJcLWupmhHHR Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="379583232" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="379583232" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:43 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="594525696" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="594525696" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.50.166]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:42 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang Cc: Alison Schofield , linux-cxl@vger.kernel.org Subject: [PATCH 2/5] cxl/memdev: Add support for the Clear Poison mailbox command Date: Tue, 29 Nov 2022 20:34:34 -0800 Message-Id: <091f50b2644f220f0607633a4a953184e9c88b53.1669781852.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield CXL devices optionally support the CLEAR POISON mailbox command. Add a sysfs attribute and memdev driver support for clearing poison. When a Device Physical Address (DPA) is written to the clear_poison sysfs attribute send a clear poison command to the device for the specified address. Per the CXL Specification (8.2.9.8.4.3), after receiving a valid clear poison request, the device removes the address from the device's Poison List and writes 0 (zero) for 64 bytes starting at address. If the device cannot clear poison from the address, it returns a permanent media error and ENXIO is returned to the user. Additionally, and per the spec also, it is not an error to clear poison of an address that is not poisoned. No error is returned and the address is not overwritten. The memdev driver performs basic sanity checking on the address, however, it does not go as far as reading the poison list to see if the address is poisoned before clearing. That discovery is left to the device. The device safely handles that case. Implementation note: Although the CXL specification defines the clear command to accept 64 bytes of 'write-data' to be used when clearing the poisoned address, this implementation always uses 0 (zeros) for the write-data. The clear_poison attribute is only visible for devices supporting the capability. Signed-off-by: Alison Schofield Reviewed-by: Jonathan Cameron --- Documentation/ABI/testing/sysfs-bus-cxl | 17 +++++++++ drivers/cxl/core/memdev.c | 47 +++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 6 ++++ 3 files changed, 70 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl index 20db97f7a1aa..9d2b0fa07e17 100644 --- a/Documentation/ABI/testing/sysfs-bus-cxl +++ b/Documentation/ABI/testing/sysfs-bus-cxl @@ -435,3 +435,20 @@ Description: poison into an address that already has poison present and no error is returned. The inject_poison attribute is only visible for devices supporting the capability. + + +What: /sys/bus/cxl/devices/memX/clear_poison +Date: December, 2022 +KernelVersion: v6.2 +Contact: linux-cxl@vger.kernel.org +Description: + (WO) When a Device Physical Address (DPA) is written to this + attribute the memdev driver sends a clear poison command to the + device for the specified address. Clearing poison removes the + address from the device's Poison List and writes 0 (zero) + for 64 bytes starting at address. It is not an error to clear + poison from an address that does not have poison set, and if + poison was not set, the address is not overwritten. If the + device cannot clear poison from the address, ENXIO is returned. + The clear_poison attribute is only visible for devices + supporting the capability. diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index 71130813030f..85caffd5a85c 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -187,6 +187,44 @@ static ssize_t inject_poison_store(struct device *dev, } static DEVICE_ATTR_WO(inject_poison); +static ssize_t clear_poison_store(struct device *dev, + struct device_attribute *attr, + const char *buf, size_t len) +{ + struct cxl_memdev *cxlmd = to_cxl_memdev(dev); + struct cxl_dev_state *cxlds = cxlmd->cxlds; + struct cxl_mbox_clear_poison *pi; + u64 dpa; + int rc; + + rc = kstrtou64(buf, 0, &dpa); + if (rc) + return rc; + rc = cxl_validate_poison_dpa(cxlds, dpa); + if (rc) + return rc; + pi = kzalloc(sizeof(*pi), GFP_KERNEL); + if (!pi) + return -ENOMEM; + /* + * In CXL 3.0 Spec 8.2.9.8.4.3, the Clear Poison mailbox command + * is defined to accept 64 bytes of 'write-data', along with the + * address to clear. The device writes 'write-data' into the DPA, + * atomically, while clearing poison if the location is marked as + * being poisoned. + * + * Always use '0' for the write-data. + */ + pi->address = cpu_to_le64(dpa); + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_CLEAR_POISON, pi, + sizeof(*pi), NULL, cxlds->payload_size); + if (rc) + return rc; + + return len; +} +static DEVICE_ATTR_WO(clear_poison); + static struct attribute *cxl_memdev_attributes[] = { &dev_attr_serial.attr, &dev_attr_firmware_version.attr, @@ -195,6 +233,7 @@ static struct attribute *cxl_memdev_attributes[] = { &dev_attr_numa_node.attr, &dev_attr_trigger_poison_list.attr, &dev_attr_inject_poison.attr, + &dev_attr_clear_poison.attr, NULL, }; @@ -228,6 +267,14 @@ static umode_t cxl_memdev_visible(struct kobject *kobj, struct attribute *a, to_cxl_memdev(dev)->cxlds->enabled_cmds)) return 0; } + if (a == &dev_attr_clear_poison.attr) { + struct device *dev = kobj_to_dev(kobj); + + if (!test_bit(CXL_MEM_COMMAND_ID_CLEAR_POISON, + to_cxl_memdev(dev)->cxlds->enabled_cmds)) { + return 0; + } + } return a->mode; } diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 0d4c34be7335..532adf9c3afd 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -399,6 +399,12 @@ struct cxl_mbox_poison_payload_out { /* Inject & Clear Poison CXL 3.0 Spec 8.2.9.8.4.2/3 */ #define CXL_POISON_INJECT_RESERVED GENMASK_ULL(5, 0) +/* Clear Poison CXL 3.0 Spec 8.2.9.8.4.3 */ +struct cxl_mbox_clear_poison { + __le64 address; + u8 write_data[64]; +} __packed; + /** * struct cxl_mem_command - Driver representation of a memory device command * @info: Command information as it exists for the UAPI From patchwork Wed Nov 30 04:34:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13059435 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6916FC433FE for ; Wed, 30 Nov 2022 04:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233263AbiK3Ee5 (ORCPT ); Tue, 29 Nov 2022 23:34:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233193AbiK3Eep (ORCPT ); Tue, 29 Nov 2022 23:34:45 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACD95A1A8 for ; Tue, 29 Nov 2022 20:34:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669782884; x=1701318884; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZiP41g0dCCWw70B/J6GeqQsICRfRnswtZYGMtJ42xCQ=; b=mQssjcPEO6RHUsYTjHtikNjWSmtJEV+gZVR1LofeGM5w8waPOV0gQ+Qi dtxRiN0naOBaRZ2Z6Toc8sSQMN/iVYQK9794FQWNNOmYWFkvfEciD58bG nLKSVJ1bIDeuSymJOVKjBKS8b487/H/lrKH0VRxweWRw3w5yY0PyPcqqT LvWM7xG6lMvo1I5u6t3Q9xVU6tY2Qpqe7H4EdmiJ5IXOR7ZSjvSkEf2Fp Ye+pAxpOG8TNn/RYflr3o/Z2xMoKeF5b81o0vamOIvOKxO7k60i02tPQe ZGjn5UPZZvR0rL+ZKu34pxui46ab0ByURjyy75PU+MPrOiWlON8sXuao9 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="379583235" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="379583235" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:44 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="594525700" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="594525700" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.50.166]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:43 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang Cc: Alison Schofield , linux-cxl@vger.kernel.org Subject: [PATCH 3/5] tools/testing/cxl: Mock the Inject Poison mailbox command Date: Tue, 29 Nov 2022 20:34:35 -0800 Message-Id: <08eddfad079afbab1f80f17d9670b40b2a7bdea8.1669781852.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield Mock the injection of poison by storing the device:address entry in a cxl_test array: mock_poison[]. Limit the array to 64 entries and fail new inject requests when full. Introducing the mock_poison[] list here, makes it available for use in the mock of Clear Poison, and the mock of Get Poison List. Signed-off-by: Alison Schofield --- tools/testing/cxl/test/mem.c | 53 ++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index a4f81915ec03..98acb9a644df 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -13,6 +13,7 @@ #define LSA_SIZE SZ_128K #define DEV_SIZE SZ_2G #define EFFECT(x) (1U << x) +#define MOCK_INJECT_POISON_MAX 64 static struct cxl_cel_entry mock_cel[] = { { @@ -43,6 +44,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_POISON), .effect = cpu_to_le16(0), }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_INJECT_POISON), + .effect = cpu_to_le16(0), + }, }; /* See CXL 2.0 Table 181 Get Health Info Output Payload */ @@ -210,6 +215,51 @@ static int mock_health_info(struct cxl_dev_state *cxlds, return 0; } +static struct mock_poison { + struct cxl_dev_state *cxlds; + u64 dpa; +} mock_poison[MOCK_INJECT_POISON_MAX]; + +static bool mock_poison_add(struct cxl_dev_state *cxlds, u64 dpa) +{ + for (int i = 0; i < MOCK_INJECT_POISON_MAX; i++) { + if (!mock_poison[i].cxlds) { + mock_poison[i].cxlds = cxlds; + mock_poison[i].dpa = dpa; + return true; + } + } + dev_dbg(cxlds->dev, "Mock poison list full: %d\n", + MOCK_INJECT_POISON_MAX); + return false; +} + +static bool mock_poison_found(struct cxl_dev_state *cxlds, u64 dpa) +{ + for (int i = 0; i < MOCK_INJECT_POISON_MAX; i++) { + if (mock_poison[i].cxlds == cxlds && + mock_poison[i].dpa == dpa) + return true; + } + return false; +} + +static int mock_inject_poison(struct cxl_dev_state *cxlds, + struct cxl_mbox_cmd *cmd) +{ + u64 dpa = le64_to_cpu(*((u64 *)cmd->payload_in)); + + if (mock_poison_found(cxlds, dpa)) { + /* Not an error to inject poison if already poisoned */ + dev_dbg(cxlds->dev, "DPA: 0x%llx already poisoned\n", dpa); + return 0; + } + if (!mock_poison_add(cxlds, dpa)) + return -ENXIO; + + return 0; +} + static int mock_get_poison(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) { struct cxl_mbox_poison_payload_in *pi = cmd->payload_in; @@ -268,6 +318,9 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd * case CXL_MBOX_OP_GET_POISON: rc = mock_get_poison(cxlds, cmd); break; + case CXL_MBOX_OP_INJECT_POISON: + rc = mock_inject_poison(cxlds, cmd); + break; default: break; } From patchwork Wed Nov 30 04:34:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13059436 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EF82FC352A1 for ; Wed, 30 Nov 2022 04:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231192AbiK3Ee6 (ORCPT ); Tue, 29 Nov 2022 23:34:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35824 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233166AbiK3Eeq (ORCPT ); Tue, 29 Nov 2022 23:34:46 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B322C55C84 for ; Tue, 29 Nov 2022 20:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669782885; x=1701318885; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m9sxd4/9lr3pk29+PfPF2PfG2nkRO2UYx9gMEYwpIdc=; b=PkBRLdm+DF6v1MTccOPwd1P7eDZVaRIr+s3iDckysIGu0JlFAhaBuHde T2aao+lIzXpAZ9am18Cg4gEVqHDXzhEVVJknGgZAeE4H7yR1I8BgjnjBR 9WPOB/kang+TL8sKxxlwsBF2EnzUp6VI3FaAAUTz4jv+K/CCK81snF15A 9OW/thUtbXNqpx2aIZIDpZr6RYfLfLhRlOsM3wIfHWKDpLMpclCasE/Q4 6BW4NBxgT250cHMjzTncNRfrx/+d1mdXvMACMkHsEPxWxv28+JEjcg2W2 j9N6+4jzOVDOks576VjlcumLp6cTuXVLnazbOpC2ltb1uatmFUCMcxwVY g==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="379583240" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="379583240" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:44 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="594525703" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="594525703" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.50.166]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:44 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang Cc: Alison Schofield , linux-cxl@vger.kernel.org Subject: [PATCH 4/5] tools/testing/cxl: Mock the Clear Poison mailbox command Date: Tue, 29 Nov 2022 20:34:36 -0800 Message-Id: X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield Mock the clear of poison by deleting the device:address entry from the cxl_test array: mock_poison[]. Behave like a real CXL device and do not fail if the address is not in the poison list, but offer a dev_dbg() message. Unlike a real CXL device, no data is written to the address being cleared. Signed-off-by: Alison Schofield Reviewed-by: Jonathan Cameron --- tools/testing/cxl/test/mem.c | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 98acb9a644df..9d794fbe5ee1 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -48,6 +48,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_INJECT_POISON), .effect = cpu_to_le16(0), }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_CLEAR_POISON), + .effect = cpu_to_le16(0), + }, }; /* See CXL 2.0 Table 181 Get Health Info Output Payload */ @@ -244,6 +248,35 @@ static bool mock_poison_found(struct cxl_dev_state *cxlds, u64 dpa) return false; } +static bool mock_poison_del(struct cxl_dev_state *cxlds, u64 dpa) +{ + for (int i = 0; i < MOCK_INJECT_POISON_MAX; i++) { + if (mock_poison[i].cxlds == cxlds && + mock_poison[i].dpa == dpa) { + mock_poison[i].cxlds = NULL; + return true; + } + } + return false; +} + +static int mock_clear_poison(struct cxl_dev_state *cxlds, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_clear_poison *pi = cmd->payload_in; + + u64 dpa = le64_to_cpu(pi->address); + /* + * A real CXL device will write pi->write_data to the address + * being cleared. In this mock, just delete this address from + * the mock poison list. + */ + if (!mock_poison_del(cxlds, dpa)) + dev_dbg(cxlds->dev, "DPA: 0x%llx not in poison list\n", dpa); + + return 0; +} + static int mock_inject_poison(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) { @@ -321,6 +354,9 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd * case CXL_MBOX_OP_INJECT_POISON: rc = mock_inject_poison(cxlds, cmd); break; + case CXL_MBOX_OP_CLEAR_POISON: + rc = mock_clear_poison(cxlds, cmd); + break; default: break; } From patchwork Wed Nov 30 04:34:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 13059437 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A51DC433FE for ; Wed, 30 Nov 2022 04:35:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233160AbiK3Ee7 (ORCPT ); Tue, 29 Nov 2022 23:34:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35826 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233202AbiK3Eeq (ORCPT ); Tue, 29 Nov 2022 23:34:46 -0500 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA34C59851 for ; Tue, 29 Nov 2022 20:34:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669782885; x=1701318885; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=5vXuv/Unb1cvTGr4KjdHtgI9+2CY7iEnNOQJ9BHWdSA=; b=nLlWSGQauaBx5bWhr7/KP3bRQegjBO3FFZp8G8dOdb77UApMCJTHiICk syl1Sv/3w3ukH+4HMTFOzUqUFG0ml4BRmNhPt5HsFSiuL5ZJ0G9sykYmj NGk23LJrXTWFsN3b8nchlu4/fzYAF3hGBpRXBzpzchjmAW2LOihY/RlGZ uvu/fzbq+faxhY0wUH13bgVoKVg75lGu40DmsBG7633y8L2TpHc46yI82 amI9EZ9hqyU/6izQxc45wFbXHDW9YkB2AQNZpL2yFZu7QwRKqN6DGxRmR xMOOzWST/RviLplIf9AxXl4RJdhKdPLb0fn5t9uj+iMBDyuGK2uTOEuY8 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="379583245" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="379583245" Received: from orsmga003.jf.intel.com ([10.7.209.27]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:45 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10546"; a="594525710" X-IronPort-AV: E=Sophos;i="5.96,205,1665471600"; d="scan'208";a="594525710" Received: from aschofie-mobl2.amr.corp.intel.com (HELO localhost) ([10.209.50.166]) by orsmga003-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Nov 2022 20:34:45 -0800 From: alison.schofield@intel.com To: Dan Williams , Ira Weiny , Vishal Verma , Ben Widawsky , Dave Jiang Cc: Alison Schofield , linux-cxl@vger.kernel.org Subject: [PATCH 5/5] tools/testing/cxl: Use injected poison for Get Poison List Date: Tue, 29 Nov 2022 20:34:37 -0800 Message-Id: <2c068669b1b31a94b319f60c413c2169bb854111.1669781852.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.37.3 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield Prior to poison inject & clear support, the mock of 'Get Poison List' returned a list containing a single mocked error record. Now that the mock of Inject and Clear poison maintains a mock_poison[] list, use that when available for the device. Otherwise, return the existing default mocked error record. This enables roundtrip userspace testing of the inject, clear, and poison list support. Signed-off-by: Alison Schofield --- tools/testing/cxl/test/mem.c | 97 +++++++++++++++++++++++++----------- 1 file changed, 69 insertions(+), 28 deletions(-) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 9d794fbe5ee1..31c147cf2d5b 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -224,6 +224,75 @@ static struct mock_poison { u64 dpa; } mock_poison[MOCK_INJECT_POISON_MAX]; +struct mock_poison_po { + struct cxl_mbox_poison_payload_out poison_out; + struct cxl_poison_record record[MOCK_INJECT_POISON_MAX]; +}; + +/* + * Use the default poison payload when no injected poison is currently + * mocked for this device. Mock one poison record with length 64 bytes. + */ +static struct mock_poison_po default_poison_po = { + .poison_out.count = cpu_to_le16(1), + .record[0].length = cpu_to_le32(1), +}; + +static struct mock_poison_po *cxl_get_injected_po(struct cxl_dev_state *cxlds, + u64 offset, u64 length) +{ + struct mock_poison_po *po; + int nr_records = 0; + u64 dpa; + + po = kzalloc(sizeof(*po), GFP_KERNEL); + if (!po) + return NULL; + + for (int i = 0; i < MOCK_INJECT_POISON_MAX; i++) { + if (mock_poison[i].cxlds != cxlds) + continue; + if (mock_poison[i].dpa < offset || + mock_poison[i].dpa > offset + length - 1) + continue; + dpa = cpu_to_le64(mock_poison[i].dpa | + CXL_POISON_SOURCE_INJECTED); + po->record[nr_records].address = dpa; + po->record[nr_records].length = cpu_to_le32(1); + nr_records++; + } + if (!nr_records) { + kfree(po); + return NULL; + } + po->poison_out.count = cpu_to_le16(nr_records); + return po; +} + +static int mock_get_poison(struct cxl_dev_state *cxlds, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_poison_payload_in *pi = cmd->payload_in; + struct mock_poison_po *po; + + po = cxl_get_injected_po(cxlds, pi->offset, pi->length); + if (!po) { + po = &default_poison_po; + po->record[0].address = cpu_to_le64(pi->offset | + CXL_POISON_SOURCE_INTERNAL); + } + if (cmd->size_out < sizeof(po)) + return -EINVAL; + + memcpy(cmd->payload_out, po, sizeof(*po)); + cmd->size_out = sizeof(po); + + if (po != &default_poison_po) + kfree(po); + + return 0; +} + static bool mock_poison_add(struct cxl_dev_state *cxlds, u64 dpa) { for (int i = 0; i < MOCK_INJECT_POISON_MAX; i++) { @@ -293,34 +362,6 @@ static int mock_inject_poison(struct cxl_dev_state *cxlds, return 0; } -static int mock_get_poison(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) -{ - struct cxl_mbox_poison_payload_in *pi = cmd->payload_in; - - struct { - struct cxl_mbox_poison_payload_out poison_out; - struct cxl_poison_record record; - } mock_poison_list = { - .poison_out = { - .count = cpu_to_le16(1), - }, - /* Mock one poison record at pi.offset for 64 bytes */ - .record = { - /* .address encodes DPA and poison source bits */ - .address = cpu_to_le64(pi->offset | - CXL_POISON_SOURCE_INTERNAL), - .length = cpu_to_le32(1), - }, - }; - - if (cmd->size_out < sizeof(mock_poison_list)) - return -EINVAL; - - memcpy(cmd->payload_out, &mock_poison_list, sizeof(mock_poison_list)); - cmd->size_out = sizeof(mock_poison_list); - return 0; -} - static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) { struct device *dev = cxlds->dev;