From patchwork Tue Oct 29 20:34:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13855421 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.9]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D13D120CCD0 for ; Tue, 29 Oct 2024 20:36:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.9 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730234163; cv=none; b=cANuwsP6AKhk+QIwsheJuInBkmvl1qB5A6bKgR7crmJa+VxnOsZ6J1zB1o0+erSZ7txtIRVnP0DQTcAjveEGBHk0AIXY+h+7ErUNmP6cqH3uWNIpWq/kzhM8hzKSpp0nfCrSsge5dy2hYt9trZ+mjplb3OW1mKwmucd8YmrKZpM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730234163; c=relaxed/simple; bh=IhUG+FoWvxdWkzWKmYBbP+Z6o6aWX0Dw5pujHfj+ELE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=YtZkp6nMoGVEoM1TPqsnr+mzuTB5wTiH2kakWf/b36cHL/OalZS1IBTFSMH+2PAeAPmfWHSdhsoPvU5OPwQYuBfPeZO34+l9cpJABlsFS7uSM7RFDMDP2YMrMWTdmp2Ong4Fi0B4AjS/xgY1vHo8U8SRcA8hWFFB458BO1i3cBA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JvxY9QEY; arc=none smtp.client-ip=198.175.65.9 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JvxY9QEY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730234161; x=1761770161; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=IhUG+FoWvxdWkzWKmYBbP+Z6o6aWX0Dw5pujHfj+ELE=; b=JvxY9QEYZt3mRTzpxV/QeHtsC07ea09LFe0Mx8HoVxfFZnwbJM3Aa03X 92YIO3WsfYrNdqAAexZfQet6edD9RWdjcl4V4TS0SB0rxunf4imeCDzU/ 05OWi/MpQ12KDJHRIrlUMTmAdBupcwkRfnTlnNrsUtExcpfm+sobKWgOo nrAXRybWP+SjGyU0hFkA3hKL3BOO0NrBYj13B2IaPuyhjGDk+rlJE4T9q Y1BpHP3srczcMBbwMQLLoMAKrvdX9AKW9tNJBa/RvdsnhYsco+QdKv1ps 6o9DzXJYt+KCiMxjT26FzZISAV6kM2Pdy3xNY4edMQ4I44LyAdicOpgMy g==; X-CSE-ConnectionGUID: lPqkGzJrT7iXCGKqtv4e5g== X-CSE-MsgGUID: F5NYdF/TRceVaofhm5D6mQ== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="52457608" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="52457608" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2024 13:36:00 -0700 X-CSE-ConnectionGUID: uKWJhEdjShWVWXMdqM/RHQ== X-CSE-MsgGUID: W/HzwBUXRUyqxA6539hYKA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,243,1725346800"; d="scan'208";a="119561300" Received: from ldmartin-desk2.corp.intel.com (HELO localhost) ([10.125.108.77]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 29 Oct 2024 13:35:59 -0700 From: ira.weiny@intel.com Date: Tue, 29 Oct 2024 15:34:53 -0500 Subject: [PATCH v5 18/27] cxl/mem: Configure dynamic capacity interrupts Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241029-dcd-type2-upstream-v5-18-8739cb67c374@intel.com> References: <20241029-dcd-type2-upstream-v5-0-8739cb67c374@intel.com> In-Reply-To: <20241029-dcd-type2-upstream-v5-0-8739cb67c374@intel.com> To: Dave Jiang , Fan Ni , Jonathan Cameron , Navneet Singh , Jonathan Corbet , Andrew Morton Cc: Dan Williams , Davidlohr Bueso , Alison Schofield , Vishal Verma , Ira Weiny , linux-cxl@vger.kernel.org, linux-doc@vger.kernel.org, nvdimm@lists.linux.dev, linux-kernel@vger.kernel.org, Li Ming X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1730234086; l=5825; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=27mVzj+f0vEplzqSDDuuXfPyNBQDUcgpFxeNLjIxFAg=; b=zpMquEIItJQY5+sVzG/bipnf5cIPeFcipaH82+wxd0a3XuprcOdGKsUWxio7FOkGcKmvZdXGD uJJilgKoazZApmepWOcRvOXYfHzvbpEoK/Nups5ebbgnQeFAwu6T2R0 X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= From: Navneet Singh Dynamic Capacity Devices (DCD) support extent change notifications through the event log mechanism. The interrupt mailbox commands were extended in CXL 3.1 to support these notifications. Firmware can't configure DCD events to be FW controlled but can retain control of memory events. Configure DCD event log interrupts on devices supporting dynamic capacity. Disable DCD if interrupts are not supported. Care is taken to preserve the interrupt policy set by the FW if FW first has been selected by the BIOS. Signed-off-by: Navneet Singh Reviewed-by: Jonathan Cameron Reviewed-by: Dave Jiang Reviewed-by: Li Ming Co-developed-by: Ira Weiny Signed-off-by: Ira Weiny --- Changes: [Fan: Don't fail probe on DCD irq failure, just disable dcd and print error] [Jonathan: move zero'ing of policy to this patch] --- drivers/cxl/cxlmem.h | 2 ++ drivers/cxl/pci.c | 73 ++++++++++++++++++++++++++++++++++++++++++---------- 2 files changed, 62 insertions(+), 13 deletions(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 204f7bd9197bd1a02de44ef56a345811d2107ab4..16e06b59d7f04762ca73a81740b0d6b2487301af 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -226,7 +226,9 @@ struct cxl_event_interrupt_policy { u8 warn_settings; u8 failure_settings; u8 fatal_settings; + u8 dcd_settings; } __packed; +#define CXL_EVENT_INT_POLICY_BASE_SIZE 4 /* info, warn, failure, fatal */ /** * struct cxl_event_state - Event log driver state diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index ac085a0b4881fc4f074d23f3606f9a3b7e70d05f..13672b8cad5be4b5a955a91e9faaba0a0acd345a 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -672,23 +672,34 @@ static int cxl_event_get_int_policy(struct cxl_memdev_state *mds, } static int cxl_event_config_msgnums(struct cxl_memdev_state *mds, - struct cxl_event_interrupt_policy *policy) + struct cxl_event_interrupt_policy *policy, + bool native_cxl) { struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + size_t size_in = CXL_EVENT_INT_POLICY_BASE_SIZE; struct cxl_mbox_cmd mbox_cmd; int rc; - *policy = (struct cxl_event_interrupt_policy) { - .info_settings = CXL_INT_MSI_MSIX, - .warn_settings = CXL_INT_MSI_MSIX, - .failure_settings = CXL_INT_MSI_MSIX, - .fatal_settings = CXL_INT_MSI_MSIX, - }; + /* memory event policy is left if FW has control */ + if (native_cxl) { + *policy = (struct cxl_event_interrupt_policy) { + .info_settings = CXL_INT_MSI_MSIX, + .warn_settings = CXL_INT_MSI_MSIX, + .failure_settings = CXL_INT_MSI_MSIX, + .fatal_settings = CXL_INT_MSI_MSIX, + .dcd_settings = 0, + }; + } + + if (cxl_dcd_supported(mds)) { + policy->dcd_settings = CXL_INT_MSI_MSIX; + size_in += sizeof(policy->dcd_settings); + } mbox_cmd = (struct cxl_mbox_cmd) { .opcode = CXL_MBOX_OP_SET_EVT_INT_POLICY, .payload_in = policy, - .size_in = sizeof(*policy), + .size_in = size_in, }; rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); @@ -735,6 +746,30 @@ static int cxl_event_irqsetup(struct cxl_memdev_state *mds, return 0; } +static int cxl_irqsetup(struct cxl_memdev_state *mds, + struct cxl_event_interrupt_policy *policy, + bool native_cxl) +{ + struct cxl_dev_state *cxlds = &mds->cxlds; + int rc; + + if (native_cxl) { + rc = cxl_event_irqsetup(mds, policy); + if (rc) + return rc; + } + + if (cxl_dcd_supported(mds)) { + rc = cxl_event_req_irq(cxlds, policy->dcd_settings); + if (rc) { + dev_err(cxlds->dev, "Failed to get interrupt for DCD event log\n"); + cxl_disable_dcd(mds); + } + } + + return 0; +} + static bool cxl_event_int_is_fw(u8 setting) { u8 mode = FIELD_GET(CXLDEV_EVENT_INT_MODE_MASK, setting); @@ -760,18 +795,26 @@ static bool cxl_event_validate_mem_policy(struct cxl_memdev_state *mds, static int cxl_event_config(struct pci_host_bridge *host_bridge, struct cxl_memdev_state *mds, bool irq_avail) { - struct cxl_event_interrupt_policy policy; + struct cxl_event_interrupt_policy policy = { 0 }; + bool native_cxl = host_bridge->native_cxl_error; int rc; /* * When BIOS maintains CXL error reporting control, it will process * event records. Only one agent can do so. + * + * If BIOS has control of events and DCD is not supported skip event + * configuration. */ - if (!host_bridge->native_cxl_error) + if (!native_cxl && !cxl_dcd_supported(mds)) return 0; if (!irq_avail) { dev_info(mds->cxlds.dev, "No interrupt support, disable event processing.\n"); + if (cxl_dcd_supported(mds)) { + dev_info(mds->cxlds.dev, "DCD requires interrupts, disable DCD\n"); + cxl_disable_dcd(mds); + } return 0; } @@ -779,10 +822,10 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, if (rc) return rc; - if (!cxl_event_validate_mem_policy(mds, &policy)) + if (native_cxl && !cxl_event_validate_mem_policy(mds, &policy)) return -EBUSY; - rc = cxl_event_config_msgnums(mds, &policy); + rc = cxl_event_config_msgnums(mds, &policy, native_cxl); if (rc) return rc; @@ -790,12 +833,16 @@ static int cxl_event_config(struct pci_host_bridge *host_bridge, if (rc) return rc; - rc = cxl_event_irqsetup(mds, &policy); + rc = cxl_irqsetup(mds, &policy, native_cxl); if (rc) return rc; cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); + dev_dbg(mds->cxlds.dev, "Event config : %s DCD %s\n", + native_cxl ? "OS" : "BIOS", + cxl_dcd_supported(mds) ? "supported" : "not supported"); + return 0; }