From patchwork Mon Dec 9 23:33:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13900489 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 911D519B3EE for ; Mon, 9 Dec 2024 23:33:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733787192; cv=none; b=NFfWR7+kkp6gdKlDfuTr3waQNDC9eXoY6vP9mhO149XROGIYRiEPpN2Ob5G8ZjCAgAlpn0h/94JE+pOF1gyc92tX1vmTgzsGt6AtsN81Ne3imhu2OvtGtxZCD8G38BR8Ph29/a4o6VUZ3I0mMULROH4c0F5J8ndi2ibFud9rIHY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733787192; c=relaxed/simple; bh=8F86IZ34mf3V4v566o3e/6t4tcXDheRg0PxHZgJj0bQ=; h=Subject:From:To:Cc:Date:Message-ID:MIME-Version:Content-Type; b=UNg1Wz+zGF3soXK1Bg4uDt7CsuImmoQC7F8kPKLswmZE0bTnJMXW2bSql7m+O3VIkwpygfFChWxtbP4LLGQOHpY2yBY+2Fq+yQI1G2h4XiPsZgNOCG53mLejxO9IPXgIgwURI04RPZszO2rpWWPopzQX3hnGTa29OSRZJsVUa+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=EEn/erCL; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EEn/erCL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1733787190; x=1765323190; h=subject:from:to:cc:date:message-id:mime-version: content-transfer-encoding; bh=8F86IZ34mf3V4v566o3e/6t4tcXDheRg0PxHZgJj0bQ=; b=EEn/erCLELx/2NyiC0FdpQgE/GULNIe7r/SRK8su/ts8bD757k8/9bM3 yP71rbveoKlP+Jn+InPHH0PKBh282nEIZCRwZ1kxWSnmmKmRadRBr9O9J sl/6Y1LTiiuTk1pBVJ2lT5FIGvl7ip9w3/bTUhGqC5ygI3fuAUKcEovJK Ckccre7RtwO42tF1avLNa3QAZwPu5qgGJeSfDFVSFutMQqgcQOx5mxYP2 +UNipD+D5iaeE3DlQ0zB+7brqUhYcLwfNaCWmRLMypp9iqk0VEFbc/XDK /7YY/rP9+UwTwTMIZc0oeGmdYPDGp/Z2jMZHlFtu8iGBJ8FcgV2iIkSBn A==; X-CSE-ConnectionGUID: 0dFHJlY0QLCuLX6JKxv6Sw== X-CSE-MsgGUID: w+RXb4SoTF+EjKUxxvd8jA== X-IronPort-AV: E=McAfee;i="6700,10204,11281"; a="51523221" X-IronPort-AV: E=Sophos;i="6.12,220,1728975600"; d="scan'208";a="51523221" Received: from fmviesa010.fm.intel.com ([10.60.135.150]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Dec 2024 15:33:10 -0800 X-CSE-ConnectionGUID: Y8p3jMPSRwq3MH2QDDsf2w== X-CSE-MsgGUID: OHKNdm34QlKh9n2Lpy0UAg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,220,1728975600"; d="scan'208";a="95565956" Received: from kinlongk-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.109.118]) by fmviesa010-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Dec 2024 15:33:03 -0800 Subject: [PATCH v2] cxl/region: Fix region creation for greater than x2 switches From: Dan Williams To: dave.jiang@intel.com Cc: Huaisheng Ye , Li Zhijian , linux-cxl@vger.kernel.org Date: Mon, 09 Dec 2024 15:33:02 -0800 Message-ID: <173378716722.1270362.9546805175813426729.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Huaisheng Ye The cxl_port_setup_targets() algorithm fails to identify valid target list ordering in the presence of 4-way and above switches resulting in 'cxl create-region' failures of the form: # cxl create-region -d decoder0.0 -g 1024 -s 2G -t ram -w 8 -m mem4 mem1 mem6 mem3 mem2 mem5 mem7 mem0 cxl region: create_region: region0: failed to set target7 to mem0 cxl region: cmd_create_region: created 0 regions [kernel debug message] check_last_peer:1213: cxl region0: pci0000:0c:port1: cannot host mem6:decoder7.0 at 2 bus_remove_device:574: bus: 'cxl': remove device region0 QEMU can create this failing topology: ACPI0017:00 [root0] | HB_0 [port1] / \ RP_0 RP_1 | | USP [port2] USP [port3] / / \ \ / / \ \ DSP DSP DSP DSP DSP DSP DSP DSP | | | | | | | | mem4 mem6 mem2 mem7 mem1 mem3 mem5 mem0 Pos: 0 2 4 6 1 3 5 7 HB: Host Bridge RP: Root Port USP: Upstream Port DSP: Downstream Port ...with the following command steps: $ qemu-system-x86_64 -machine q35,cxl=on,accel=tcg \ -smp cpus=8 \ -m 8G \ -hda /home/work/vm-images/centos-stream8-02.qcow2 \ -object memory-backend-ram,size=4G,id=m0 \ -object memory-backend-ram,size=4G,id=m1 \ -object memory-backend-ram,size=2G,id=cxl-mem0 \ -object memory-backend-ram,size=2G,id=cxl-mem1 \ -object memory-backend-ram,size=2G,id=cxl-mem2 \ -object memory-backend-ram,size=2G,id=cxl-mem3 \ -object memory-backend-ram,size=2G,id=cxl-mem4 \ -object memory-backend-ram,size=2G,id=cxl-mem5 \ -object memory-backend-ram,size=2G,id=cxl-mem6 \ -object memory-backend-ram,size=2G,id=cxl-mem7 \ -numa node,memdev=m0,cpus=0-3,nodeid=0 \ -numa node,memdev=m1,cpus=4-7,nodeid=1 \ -netdev user,id=net0,hostfwd=tcp::2222-:22 \ -device virtio-net-pci,netdev=net0 \ -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \ -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \ -device cxl-upstream,bus=root_port0,id=us0 \ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ -device cxl-type3,bus=swport0,volatile-memdev=cxl-mem0,id=cxl-vmem0 \ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ -device cxl-type3,bus=swport1,volatile-memdev=cxl-mem1,id=cxl-vmem1 \ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \ -device cxl-type3,bus=swport2,volatile-memdev=cxl-mem2,id=cxl-vmem2 \ -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \ -device cxl-type3,bus=swport3,volatile-memdev=cxl-mem3,id=cxl-vmem3 \ -device cxl-upstream,bus=root_port1,id=us1 \ -device cxl-downstream,port=4,bus=us1,id=swport4,chassis=0,slot=8 \ -device cxl-type3,bus=swport4,volatile-memdev=cxl-mem4,id=cxl-vmem4 \ -device cxl-downstream,port=5,bus=us1,id=swport5,chassis=0,slot=9 \ -device cxl-type3,bus=swport5,volatile-memdev=cxl-mem5,id=cxl-vmem5 \ -device cxl-downstream,port=6,bus=us1,id=swport6,chassis=0,slot=10 \ -device cxl-type3,bus=swport6,volatile-memdev=cxl-mem6,id=cxl-vmem6 \ -device cxl-downstream,port=7,bus=us1,id=swport7,chassis=0,slot=11 \ -device cxl-type3,bus=swport7,volatile-memdev=cxl-mem7,id=cxl-vmem7 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=32G & In Guest OS: # cxl create-region -d decoder0.0 -g 1024 -s 2G -t ram -w 8 -m mem4 mem1 mem6 mem3 mem2 mem5 mem7 mem0 Fix the method to calculate @distance by iterativeley multiplying the number of targets per switch port. This also follows the algorithm recommended here [1]. Fixes: 27b3f8d13830 ("cxl/region: Program target lists") Link: http://lore.kernel.org/6538824b52349_7258329466@dwillia2-xfh.jf.intel.com.notmuch [1] Signed-off-by: Huaisheng Ye Tested-by: Li Zhijian [djbw: add a comment explaining 'distance'] Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- Changes since the RFC [1]: * add a comment to clarify what 'distance' represents (Li Zhijian) [1]: http://lore.kernel.org/20241027075717.3714821-1-huaisheng.ye@intel.com drivers/cxl/core/region.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 70d0a017e99c..30ac76df9cba 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1295,6 +1295,7 @@ static int cxl_port_setup_targets(struct cxl_port *port, struct cxl_region_params *p = &cxlr->params; struct cxl_decoder *cxld = cxl_rr->decoder; struct cxl_switch_decoder *cxlsd; + struct cxl_port *iter = port; u16 eig, peig; u8 eiw, peiw; @@ -1311,16 +1312,26 @@ static int cxl_port_setup_targets(struct cxl_port *port, cxlsd = to_cxl_switch_decoder(&cxld->dev); if (cxl_rr->nr_targets_set) { - int i, distance; + int i, distance = 1; + struct cxl_region_ref *cxl_rr_iter; /* - * Passthrough decoders impose no distance requirements between - * peers + * The "distance" between peer downstream ports represents which + * endpoint positions in the region interleave a given port can + * host. + * + * For example, at the root of a hierarchy the distance is + * always 1 as every index targets a different host-bridge. At + * each subsequent switch level those ports map every Nth region + * position where N is the width of the switch == distance. */ - if (cxl_rr->nr_targets == 1) - distance = 0; - else - distance = p->nr_targets / cxl_rr->nr_targets; + do { + cxl_rr_iter = cxl_rr_load(iter, cxlr); + distance *= cxl_rr_iter->nr_targets; + iter = to_cxl_port(iter->dev.parent); + } while (!is_cxl_root(iter)); + distance *= cxlrd->cxlsd.cxld.interleave_ways; + for (i = 0; i < cxl_rr->nr_targets_set; i++) if (ep->dport == cxlsd->target[i]) { rc = check_last_peer(cxled, ep, cxl_rr, From patchwork Tue Mar 19 18:15:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13596991 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1BD42D046 for ; Tue, 19 Mar 2024 18:15:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710872110; cv=none; b=oM12ocFNLh+AtwUrJZpeWsiOiwbGedge5awODVPugIY7Ms00IYljtfXn0LP7huucy+Kpusln0zgBfmg+GQRAWYTfDYJl0U72jfmar931FAGblerqCaFdUjBYpWub2QRJqh3KYDeb/mcUUWDczQ7eAKBJQT1MAVVkhAIeYYK92pg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710872110; c=relaxed/simple; bh=/ACNik5rZNLC07RjhItowJReAKcAw8khPkP90t/tx4U=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=X6PRudtKD1I/739Ca+FpzvuKrI5oQXxBVKjrMH36H9Kdny9ZN95EFuSGseUX8C7TOqHQt27Z+bqyP1YuZmPu3R08XAi5Z7v+2xiYYn500XZiFLKFt4eTXKda3V673mETNx1rJd0OlZDjQ09m5IlbpjPw96K/UJTS1tkt10O1HwM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3E2EBC433C7; Tue, 19 Mar 2024 18:15:09 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Subject: [PATCH] cxl/core/regs: Fix usage of map->reg_type in cxl_decode_regblock() before assigned Date: Tue, 19 Mar 2024 11:15:08 -0700 Message-ID: <20240319181508.690837-1-dave.jiang@intel.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In the error path, map->reg_type is being used for kernel warning before its value is setup. Found by code inspection. Exposure to user is wrong reg_type being emitted via kernel log. Use a local var for reg_type and retrieve value for usage. Fixes: 6c7f4f1e51c2 ("cxl/core/regs: Make cxl_map_{component, device}_regs() device generic") Signed-off-by: Dave Jiang Reviewed-by: Dan Williams Reviewed-by: Davidlohr Bueso --- drivers/cxl/core/regs.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/regs.c b/drivers/cxl/core/regs.c index 372786f80955..3c42f984eeaf 100644 --- a/drivers/cxl/core/regs.c +++ b/drivers/cxl/core/regs.c @@ -271,6 +271,7 @@ EXPORT_SYMBOL_NS_GPL(cxl_map_device_regs, CXL); static bool cxl_decode_regblock(struct pci_dev *pdev, u32 reg_lo, u32 reg_hi, struct cxl_register_map *map) { + u8 reg_type = FIELD_GET(CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK, reg_lo); int bar = FIELD_GET(CXL_DVSEC_REG_LOCATOR_BIR_MASK, reg_lo); u64 offset = ((u64)reg_hi << 32) | (reg_lo & CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK); @@ -278,11 +279,11 @@ static bool cxl_decode_regblock(struct pci_dev *pdev, u32 reg_lo, u32 reg_hi, if (offset > pci_resource_len(pdev, bar)) { dev_warn(&pdev->dev, "BAR%d: %pr: too small (offset: %pa, type: %d)\n", bar, - &pdev->resource[bar], &offset, map->reg_type); + &pdev->resource[bar], &offset, reg_type); return false; } - map->reg_type = FIELD_GET(CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK, reg_lo); + map->reg_type = reg_type; map->resource = pci_resource_start(pdev, bar) + offset; map->max_size = pci_resource_len(pdev, bar) - offset; return true; From patchwork Tue Apr 2 08:14:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kwangjin Ko X-Patchwork-Id: 13613542 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 401933D552; Tue, 2 Apr 2024 08:14:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=166.125.252.92 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712045662; cv=none; b=QYHEDwfiXnWDizkqKiNA15cwckAgEI3WfINj2KyK/unw50WSRrq1xCKcmqx87QzD7lzhurafKeiJ7AkkPGd+IFDAbn/AbSbbgzVZItbnJBE+Q/PPu1JYRshKTKBNHtAA/qrX5Y4GaGuTgRhCswzaoJHSifCiJ+KAacYFeTJKv4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712045662; c=relaxed/simple; bh=zFoKGVF+3N6sg6qvMpvexaatvVtPYRghID21GGzQsC0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u5TVEjVTW95nuobeJZrimN5SCzIbsm+R67g+Gt1QGn6IJOaOJkWql7TO+sHSKY/h3qISSfHH/cUl19O9fKcFEvq1KD+HDi/tI6WTtKKr2FjWrNVzVKVvLcqfhXIhMEYljR7A8r8iSKmP/bc5+FgOF48GhlpJUshaPQbc375K770= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com; spf=pass smtp.mailfrom=sk.com; arc=none smtp.client-ip=166.125.252.92 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=sk.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sk.com X-AuditID: a67dfc5b-d6dff70000001748-62-660bbe500238 From: Kwangjin Ko To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, kernel_team@skhynix.com Subject: [PATCH v2 1/1] cxl/core: Fix initialization of mbox_cmd.size_out in get event Date: Tue, 2 Apr 2024 17:14:03 +0900 Message-ID: <20240402081404.1106-2-kwangjin.ko@sk.com> X-Mailer: git-send-email 2.44.0.windows.1 In-Reply-To: <20240402081404.1106-1-kwangjin.ko@sk.com> References: <20240402081404.1106-1-kwangjin.ko@sk.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrCLMWRmVeSWpSXmKPExsXC9ZZnkW7APu40g9b1fBZ3H19gs5g+9QKj xYmbjWwWq2+uYbTY//Q5i8WqhdfYLM7POsVicXnXHDaLWxOOMTlwerQcecvqsXjPSyaPqbPr PT5vkgtgieKySUnNySxLLdK3S+DKOP55HlvBTu6K3Ru/MDcw9nJ2MXJySAiYSPz9dZcZxt70 6CaYzSagJjHp2mG2LkYuDhGBnYwSt5etAUswC0RIbGnqZwKxhQXCJO7vnMMGYrMIqEosvXQf yObg4BUwkzi6zRhipqbE+1nfwVo5Bcwl2hd+ZQexhYBK+u8uYASxeQUEJU7OfMICMV5eonnr bGaQvRICC9gkdi+9ywoxSFLi4IobLBMY+Wch6ZmFpGcBI9MqRqHMvLLcxMwcE72MyrzMCr3k /NxNjMCwXVb7J3oH46cLwYcYBTgYlXh4b2zjShNiTSwrrsw9xCjBwawkwvvTmzNNiDclsbIq tSg/vqg0J7X4EKM0B4uSOK/Rt/IUIYH0xJLU7NTUgtQimCwTB6dUA+PqTyXneJivST/pTVv9 Vli/1pnp9J8XLov6zDM2sHhnpywQSlr/d1FG8Vum19XHpTNfNAZZO/zSFP300TW9ST9cfWam 0PSwqRNu+2gKvo0p+PK+p4Kb/67TYVbe/49/PrTefj0+3tPOuzb2fYhAiADX3IAauSln5BZe 9rdd93HhuY618nW1p5VYijMSDbWYi4oTAVWkrKxXAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNLMWRmVeSWpSXmKPExsXCNUPhiW7APu40g95+Jou7jy+wWUyfeoHR 4sTNRjaL1TfXMFrsf/qcxWLVwmtsFofnnmS1OD/rFIvF5V1z2CxuTTjG5MDl0XLkLavH4j0v mTwWv/jA5DF1dr3H501yAaxRXDYpqTmZZalF+nYJXBnHP89jK9jJXbF74xfmBsZezi5GTg4J AROJTY9uMoPYbAJqEpOuHWbrYuTiEBHYyShxe9kasASzQITElqZ+JhBbWCBM4v7OOWwgNouA qsTSS/eBbA4OXgEziaPbjCFmakq8n/UdrJVTwFyifeFXdhBbCKik/+4CRhCbV0BQ4uTMJywQ 4+UlmrfOZp7AyDMLSWoWktQCRqZVjCKZeWW5iZk5pnrF2RmVeZkVesn5uZsYgUG4rPbPxB2M Xy67H2IU4GBU4uG9sJ4rTYg1say4MvcQowQHs5II709vzjQh3pTEyqrUovz4otKc1OJDjNIc LErivF7hqQlCAumJJanZqakFqUUwWSYOTqkGxuQ7G69yunt0m1z4c2hFwdG6z162M2KU+m5K a/PvddF/5b+pVfz8ZZfwwp598TXfLns1ZD+b+kj/0oaLJ6YUz/b0iT9p7tF/13erfN/HTofl 21zMfSxP+9du3cB/a6dZ+elUp6wO7Xcv0pS6wi1fzKvcLHJzpRiDRE2ChFvVn1fqTxnfTe1j V2Ipzkg01GIuKk4EAAgrpqQ+AgAA X-CFilter-Loop: Reflected Since mbox_cmd.size_out is overwritten with the actual output size in the function below, it needs to be initialized every time. cxl_internal_send_cmd -> __cxl_pci_mbox_send_cmd Problem scenario: 1) The size_out variable is initially set to the size of the mailbox. 2) Read an event. - size_out is set to 160 bytes(header 32B + one event 128B). - Two event are created while reading. 3) Read the new *two* events. - size_out is still set to 160 bytes. - Although the value of out_len is 288 bytes, only 160 bytes are copied from the mailbox register to the local variable. - record_count is set to 2. - Accessing records[1] will result in reading incorrect data. Signed-off-by: Kwangjin Ko Reviewed-by: Jonathan Cameron Tested-by: Ira Weiny Reviewed-by: Ira Weiny --- drivers/cxl/core/mbox.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 9adda4795eb7..a38531a055c8 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -958,13 +958,14 @@ static void cxl_mem_get_records_log(struct cxl_memdev_state *mds, .payload_in = &log_type, .size_in = sizeof(log_type), .payload_out = payload, - .size_out = mds->payload_size, .min_out = struct_size(payload, records, 0), }; do { int rc, i; + mbox_cmd.size_out = mds->payload_size; + rc = cxl_internal_send_cmd(mds, &mbox_cmd); if (rc) { dev_err_ratelimited(dev, From patchwork Wed Apr 3 15:47:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13616399 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5606414A0A2 for ; Wed, 3 Apr 2024 15:48:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712159332; cv=none; b=KBKeqVGjVxo6bAk+3ijU6ruhpYUxF6aqt45d1TVfcOvW0QD7wrq2rCErnTRFApPXNKiNsAagAiPkPhg0WThWZYN373onTSHnmuQp8HkNcRs4WVjin0NmaasDNORnpmQFGs2/sgy3/oaZz+poZ1OyICQapbScDpCl4V88xpLinqQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712159332; c=relaxed/simple; bh=Pv+FyXjpfAetiPjStiEOtJb7AsfOAqggK6o+5etYUp8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BRnygbanSIq0KInlieeVsLD18bBDW2f29n/VYw0197gTo7Hn3i6te3ABpUmcjC+E9JhHebItS9W8Xth3qUUnYGnMQSIRJpkKagQ8mXO4m8Dr70YVSC8gTGlU2rBZJNkdtgqQxTyXdclciWmfwCF74OdFGEuUEdqMkK3ibpluMdM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id D7868C433C7; Wed, 3 Apr 2024 15:48:51 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net Subject: [PATCH v7 1/5] cxl: Remove checking of iter in cxl_endpoint_get_perf_coordinates() Date: Wed, 3 Apr 2024 08:47:12 -0700 Message-ID: <20240403154844.3403859-2-dave.jiang@intel.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240403154844.3403859-1-dave.jiang@intel.com> References: <20240403154844.3403859-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The while() loop in cxl_endpoint_get_perf_coordinates() checks to see if 'iter' is valid as part of the condition breaking out of the loop. is_cxl_root() will stop the loop before the next iteration could go NULL. Remove the iter check. The presence of the iter or removing the iter does not impact the behavior of the code. This is a code clean up and not a bug fix. Reviewed-by: Jonathan Cameron Reviewed-by: Davidlohr Bueso Reviewed-by: Dan Williams Signed-off-by: Dave Jiang --- drivers/cxl/core/port.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 2b0cab556072..6cbde50a742b 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -2197,7 +2197,7 @@ int cxl_endpoint_get_perf_coordinates(struct cxl_port *port, * port each iteration. If the parent is cxl root then there is * nothing to gather. */ - while (iter && !is_cxl_root(to_cxl_port(iter->dev.parent))) { + while (!is_cxl_root(to_cxl_port(iter->dev.parent))) { cxl_coordinates_combine(&c, &c, &dport->sw_coord); c.write_latency += dport->link_latency; c.read_latency += dport->link_latency; From patchwork Mon Jul 8 15:29:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: peng guo X-Patchwork-Id: 13726725 Received: from zg8tmja2lje4os4yms4ymjma.icoremail.net (zg8tmja2lje4os4yms4ymjma.icoremail.net [206.189.21.223]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 0ED49142E80; Mon, 8 Jul 2024 15:29:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=206.189.21.223 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720452571; cv=none; b=rF3B97KaFtUIuGqyhhe5JkKEr897wXl2BMiAD0DTg9/JqSDRdTmVks6udxka1f7ZvEfnl/fZ+jSwIToz21OboFR7G+Uq6eb1lc+mW4JXntQx9udiidEAFJTUM1iFoOsDSKjlyPdLGcZdCpMzpb95CpAS696WPfLPKdp9ePhdWX8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720452571; c=relaxed/simple; bh=pHGrU06dDgIohfOZNyTOXRGFIfPJGLhM05zDRXQwlIA=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=ABwqTDGga2U8aIBMS9VoUC2/DsSYHqHK7ucekrAHld9XZiTL2vhWZlmCaWADTSJ2ZylFh+IQYlqZzlcXqOG7viunLqrb32k4STw9n/PJfXvGWeb4OUXXuW1teKjYpDhXUP6Apr6vXQUkg1VsUaDzgc/Cv16B3b6CbTLFQZbh71I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=buaa.edu.cn; spf=pass smtp.mailfrom=buaa.edu.cn; dkim=pass (1024-bit key) header.d=buaa.edu.cn header.i=@buaa.edu.cn header.b=KU5FpEh8; arc=none smtp.client-ip=206.189.21.223 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=buaa.edu.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=buaa.edu.cn Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=buaa.edu.cn header.i=@buaa.edu.cn header.b="KU5FpEh8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=buaa.edu.cn; s=buaa; h=Received:From:To:Cc:Subject:Date: Message-ID:MIME-Version:Content-Transfer-Encoding; bh=jpG9/lUJIA huGAbBRSauqWsUisuORJ+1OpZD4Zt02hY=; b=KU5FpEh8pSwVYDeS0VgumKDCGA FxgPTXY0PVxaILalUASm/BRhtyLBT/H7Jwc30mK0MXkoJeHeueR4TqYf4H3WXTKd P5/nlgik4La5rxsQP9xidv3mtSamvCZe1b/tFnvJnbbtu6ZwEYxVgtqnsKmXkDTT W1jc71x0WrPQh4MJw= Received: from gp-VMware-Virtual-Platform.localdomain (unknown [139.227.253.190]) by coremail-app2 (Coremail) with SMTP id Nyz+CgD3UfXABYxmOvOYAA--.32315S2; Mon, 08 Jul 2024 23:29:18 +0800 (CST) From: peng guo To: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, wyguopeng@163.com, peng guo Subject: [PATCH] cxl core:wrong value of macro definition Date: Mon, 8 Jul 2024 23:29:02 +0800 Message-ID: <20240708152902.5853-1-engguopeng@buaa.edu.cn> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-CM-TRANSID: Nyz+CgD3UfXABYxmOvOYAA--.32315S2 X-Coremail-Antispam: 1UD129KBjvdXoW7Xw4DZFWxZr4kKF4fury8Zrb_yoW3Wrg_Gr 1rWrZxZa1FvF9rKFnIgr4rC3yS9a1kXr1FvFn5Kr4akayakrs8Wry09r47ZrW2vrW8tr1D J34DJr18tr47tjkaLaAFLSUrUUUUUb8apTn2vfkv8UJUUUU8Yxn0WfASr-VFAUDa7-sFnT 9fnUUIcSsGvfJTRUUUbakFc2x0x2IEx4CE42xK8VAvwI8IcIk0rVWrJVCq3wAFIxvE14AK wVWUJVWUGwA2ocxC64kIII0Yj41l84x0c7CEw4AK67xGY2AK021l84ACjcxK6xIIjxv20x vE14v26F1j6w1UM28EF7xvwVC0I7IYx2IY6xkF7I0E14v26F4j6r4UJwA2z4x0Y4vEx4A2 jsIE14v26rxl6s0DM28EF7xvwVC2z280aVCY1x0267AKxVW0oVCq3wAS0I0E0xvYzxvE52 x082IY62kv0487Mc02F40EFcxC0VAKzVAqx4xG6I80ewAv7VC0I7IYx2IY67AKxVWUGVWU XwAv7VC2z280aVAFwI0_Jr0_Gr1lOx8S6xCaFVCjc4AY6r1j6r4UM4x0Y48IcxkI7VAKI4 8JM4x0x7Aq67IIx4CEVc8vx2IErcIFxwACI402YVCY1x02628vn2kIc2xKxwCY02Avz4vE 14v_XrWl42xK82IYc2Ij64vIr41l42xK82IY6x8ErcxFaVAv8VW7tr1UJr1l4I8I3I0E4I kC6x0Yz7v_Jr0_Gr1lx2IqxVAqx4xG67AKxVWUJVWUGwC20s026x8GjcxK67AKxVWUGVWU WwC2zVAF1VAY17CE14v26r1q6r43MIIYrxkI7VAKI48JMIIF0xvE2Ix0cI8IcVAFwI0_Jr 0_JF4lIxAIcVC0I7IYx2IY6xkF7I0E14v26r4j6F4UMIIF0xvE42xK8VAvwI8IcIk0rVWU JVWUCwCI42IY6I8E87Iv67AKxVWUJVW8JwCI42IY6I8E87Iv6xkF7I0E14v26r4j6r4UJb IYCTnIWIevJa73UjIFyTuYvjfU5Q6JUUUUU X-CM-SenderInfo: d2isijirrujqpexdthxhgxhubq/ The first value of the macro definition DEFINE_CXL_VENDOR_DEBUG_UUID does not match the definition in the CXL 2.0 specification table 170. Signed-off-by: peng guo --- drivers/cxl/cxlmem.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index af8169ccdbc0..feb1106559d2 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -563,7 +563,7 @@ enum cxl_opcode { 0x3b, 0x3f, 0x17) #define DEFINE_CXL_VENDOR_DEBUG_UUID \ - UUID_INIT(0xe1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, \ + UUID_INIT(0x5e1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, \ 0x40, 0x3d, 0x86) struct cxl_mbox_get_supported_logs { From patchwork Tue Aug 6 04:15:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Ming4" X-Patchwork-Id: 13754380 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4ACC0A35 for ; Tue, 6 Aug 2024 04:46:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722919589; cv=none; b=bqeHhZdivlBnhVG1UsM6C5MItPHGWsq4brZALUD6WW7oEj6g+R45ThqHVGQumNd4KPVQmnw5wwXTFwil/3EI8H2xLiOYnsAkDXvqJ6l1n5uazwvCRchzFFpYoU8QnjCvEgWhbYQJRZMqvFWjoafIrtx+9FsVBOYnnsmEH0Yo+bQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1722919589; c=relaxed/simple; bh=Du61TcVfSZXRbwLaBLaRpgqhH49Z7t69PBOS37m4o5k=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=VfQRysHrWrkjVCDj/qJyiN7hxFj4UnXq7MuURDYFKCXmgiUI6wZxPRvEjeU5+dpswRlldDPnaB+WQlGeDCDQWRxxrdrhzdMvoHZfdexFbqrVo4LoOy8UVpseHMi8c/VZXum4TEZ48dGkt/dSL2/UCt25YfbUrjEDuucpmD4Mciw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=cWTqw+x6; arc=none smtp.client-ip=192.198.163.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="cWTqw+x6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1722919587; x=1754455587; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=Du61TcVfSZXRbwLaBLaRpgqhH49Z7t69PBOS37m4o5k=; b=cWTqw+x66Q6MiWZIqnnqQu9GNPfmyTnVamXS2K1H97rfeqkLsROAnTP/ /BobWJUyDPNzbJy+AjtiwheC2XZmStFLIBBdeWOivZaiXbCz3VrZxVTfN ZWMBK8xZ2FdkO5RpnfBqINNhQAuaZ6lQmpMMBobgQItSNduPwlJdtn4Wi dB+kQ1nM8cM2vybHrHqKgpNvRiG8XH/MxvaA5AOzvCgL6q2qafAebEVsV u6SNNHVzSLu3snmug7S7XCUz34jtLEM6614mJ4UsS3DOJwomPM2uYmQF7 wzihq+/ni9/Jqg3RfriHj2OGdTQY5WwoZV0T+sIMKL4oUXvjylD/3AQJg A==; X-CSE-ConnectionGUID: 9oQDrEpfTZmWpDdHS7hmPw== X-CSE-MsgGUID: b7mmi6MWRZ62k7wTtKynww== X-IronPort-AV: E=McAfee;i="6700,10204,11155"; a="12879304" X-IronPort-AV: E=Sophos;i="6.09,266,1716274800"; d="scan'208";a="12879304" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa110.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2024 21:46:26 -0700 X-CSE-ConnectionGUID: 4NzoHd4dSt+vzdgdz/bzMg== X-CSE-MsgGUID: 3kbF7Z1LR3CvIh1b7OhNmQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,266,1716274800"; d="scan'208";a="61010769" Received: from s2600wttr.bj.intel.com ([10.240.192.138]) by fmviesa004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Aug 2024 21:46:24 -0700 From: Li Ming To: linux-cxl@vger.kernel.org, rrichter@amd.com, terry.bowman@amd.com, dan.j.williams@intel.com Cc: Li Ming Subject: [PATCH 1/1] cxl/pci: Get AER capability address from RCRB only for RCH dport Date: Tue, 6 Aug 2024 04:15:47 +0000 Message-Id: <20240806041547.1958787-1-ming4.li@intel.com> X-Mailer: git-send-email 2.40.1 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 cxl_setup_parent_dport() needs to get RCH dport AER capability address from RCRB to disable AER interrupt. The function does not check if dport is RCH dport, it will get a wrong pci_host_bridge structure by dport_dev in VH case because dport_dev points to a pci device(RP or switch DSP) rather than a pci host bridge device. Besides, cxl_setup_parent_dport() can exit simply if a RCH dport in the RCH topology created by cxl-test, in the case, the RCH dport's dport_dev is a platform device, and RCH dport with an available RCRB and AER Capability is not supported yet. Fixes: f05fd10d138d ("cxl/pci: Add RCH downstream port AER register discovery") Signed-off-by: Li Ming Reported-by: Pengfei Xu Tested-by: Pengfei Xu --- drivers/cxl/core/pci.c | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a663e7566c48..f8a3188f5b17 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -4,6 +4,7 @@ #include #include #include +#include #include #include #include @@ -834,11 +835,16 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) { struct device *dport_dev = dport->dport_dev; - struct pci_host_bridge *host_bridge; - host_bridge = to_pci_host_bridge(dport_dev); - if (host_bridge->native_aer) - dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); + if (dev_is_platform(dport_dev)) + return; + + if (dport->rch) { + struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport_dev); + + if (host_bridge->native_aer) + dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); + } dport->reg_map.host = host; cxl_dport_map_regs(dport); From patchwork Wed Aug 7 13:19:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13756252 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60CA01BDAB0 for ; Wed, 7 Aug 2024 13:26:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723037206; cv=none; b=YihsyqkGkqp5MWq07Hsq1mesMsKrjKtx8NJVjx0g2/Xnturr3S+8SxyfBgCXoxQcD0ZSccmwlz55YtyYaz7XMlwSChMq2hxpTYN68JeECbaTMro42Zc2zi94Fu2WwaDWvVhXmPJWWm82TY6XajdN6kmM7pTDJtlZQx/wfm+cnnE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723037206; c=relaxed/simple; bh=3o4URN7i0Bg17bUtTjZuHlQc2rZcZLzIyP9TBSAlQKM=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=AoWEC1sCCyRFFwBzl821j4N9nCzCkv5ZysrneCCn9KQp8xY7I+jVaISUdVWwOa5x83wrgTxHmyJAWmweiaL78zqSIfARsTwExc/qTFmo+/S9EnLwo2CQ5hsKoWGHiVnDU1/oaLGRj0YeB2DkxxrLklD7vw1ZrGyzQKJefRinydg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Z7E7mM6U; arc=none smtp.client-ip=198.175.65.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Z7E7mM6U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723037204; x=1754573204; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=3o4URN7i0Bg17bUtTjZuHlQc2rZcZLzIyP9TBSAlQKM=; b=Z7E7mM6UNMePE1lHDyB//pgn+U1BEHy4Hb4QANxpKRF6u4l1djJZMmMm DhlqxjpameX4iVSGxNacyqVHZErdSDn4b3BxDyOKiB77vtM6Vx4h5GC+X n/w+lu98///iYRu71AWM365Bc/GGePH3wgWteS234MVxf5D1AoyLDXRxD I9EzuQj6VZ4kMlnnmq1leSNvilkR+FE1iC1Dqv7rndY26x9ZM7RbvnjuW GHQxoi9fFz/xUubzWaL1aEtU+QHkUYSW4+Y0Z4RK3ED8Nrkc2giZc/e21 uWuZP6gj9CKFzgMStPQF6OzwWwgWUCNEjyHXM568/y1Z4V5WGtxXL49Ar g==; X-CSE-ConnectionGUID: YZlFG0VYRCSYBbC8tx0DEA== X-CSE-MsgGUID: kLX+f0wgRW2lCmihf2t3SQ== X-IronPort-AV: E=McAfee;i="6700,10204,11157"; a="20961029" X-IronPort-AV: E=Sophos;i="6.09,270,1716274800"; d="scan'208";a="20961029" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa111.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 06:26:40 -0700 X-CSE-ConnectionGUID: DJpdkA9RSuqrwi1KD86BlA== X-CSE-MsgGUID: Y7IaP8mVRWuK5Js0mf4WxQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,270,1716274800"; d="scan'208";a="56799025" Received: from tower.bj.intel.com ([10.238.157.70]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Aug 2024 06:26:37 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [PATCH] cxl/pci: Fix DVSEC ranges validation to cover all ranges Date: Wed, 7 Aug 2024 21:19:08 +0800 Message-Id: <20240807131908.303600-1-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 cxl_endpoint_dvsec_info.ranges means the number of non-zero DVSEC range, and it will be less than the value of HDM_count when occuring zero DVSEC range. Hence using it for looping validation of DVSEC ranges in cxl_hdm_decode_init() and looping DVSEC decoder initialization in devm_cxl_enumerate_decoders could miss non-zero DVSEC ranges. And we should only create decoder for the allowed ranges. Initializing content of all dvsec_range[] to invalid range and moving the check of dvsec_range_allowed() in advance to cxl_dvsec_rr_decode() could address that. Other non-functional changes, refectoring cxl_dvsec_rr_decode to improve its readability and droping wait_for_valid() to use cxl_dvsec_mem_range_valid(). Fixes: 560f78559006 ("cxl/pci: Retrieve CXL DVSEC memory info") Signed-off-by: Yanfei Xu --- Background: Found this issue when reading the CXL code. I didn't encounter the discribed issue in real environment. drivers/cxl/core/hdm.c | 2 +- drivers/cxl/core/pci.c | 121 +++++++++++++--------------------- drivers/cxl/cxl.h | 5 +- drivers/cxl/port.c | 2 +- tools/testing/cxl/test/mock.c | 4 +- 5 files changed, 53 insertions(+), 81 deletions(-) diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..65f5fd2e4189 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -768,7 +768,7 @@ static int cxl_setup_hdm_decoder_from_dvsec( cxled = to_cxl_endpoint_decoder(&cxld->dev); len = range_len(&info->dvsec_range[which]); - if (!len) + if (WARN_ON(len == 0 || len == CXL_RESOURCE_NONE)) return -ENOENT; cxld->target_type = CXL_DECODER_HOSTONLYMEM; diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 8567dd11eaac..c8420a7995f1 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -211,37 +211,6 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) } EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, CXL); -static int wait_for_valid(struct pci_dev *pdev, int d) -{ - u32 val; - int rc; - - /* - * Memory_Info_Valid: When set, indicates that the CXL Range 1 Size high - * and Size Low registers are valid. Must be set within 1 second of - * deassertion of reset to CXL device. Likely it is already set by the - * time this runs, but otherwise give a 1.5 second timeout in case of - * clock skew. - */ - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - msleep(1500); - - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - return -ETIMEDOUT; -} - static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val) { struct pci_dev *pdev = to_pci_dev(cxlds->dev); @@ -322,11 +291,14 @@ static int devm_cxl_enable_hdm(struct device *host, struct cxl_hdm *cxlhdm) return devm_add_action_or_reset(host, disable_hdm, cxlhdm); } -int cxl_dvsec_rr_decode(struct device *dev, int d, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { struct pci_dev *pdev = to_pci_dev(dev); + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); int hdm_count, rc, i, ranges = 0; + int d = cxlds->cxl_dvsec; + struct cxl_port *root; u16 cap, ctrl; if (!d) { @@ -357,10 +329,19 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = wait_for_valid(pdev, d); - if (rc) { - dev_dbg(dev, "Failure awaiting MEM_INFO_VALID (%d)\n", rc); - return rc; + root = to_cxl_port(port->dev.parent); + while (!is_cxl_root(root) && is_cxl_port(root->dev.parent)) + root = to_cxl_port(root->dev.parent); + if (!is_cxl_root(root)) { + dev_err(dev, "Failed to acquire root port for HDM enable\n"); + return -ENODEV; + } + + for (i = 0; i < CXL_DVSEC_RANGE_MAX; i++) { + info->dvsec_range[i] = (struct range) { + .start = 0, + .end = CXL_RESOURCE_NONE, + }; } /* @@ -373,9 +354,15 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, return 0; for (i = 0; i < hdm_count; i++) { + struct device *cxld_dev; + struct range dvsec_range; u64 base, size; u32 temp; + rc = cxl_dvsec_mem_range_valid(cxlds, i); + if (rc) + return rc; + rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_SIZE_HIGH(i), &temp); if (rc) @@ -389,13 +376,8 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, return rc; size |= temp & CXL_DVSEC_MEM_SIZE_LOW_MASK; - if (!size) { - info->dvsec_range[i] = (struct range) { - .start = 0, - .end = CXL_RESOURCE_NONE, - }; + if (!size) continue; - } rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_BASE_HIGH(i), &temp); @@ -411,11 +393,22 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, base |= temp & CXL_DVSEC_MEM_BASE_LOW_MASK; - info->dvsec_range[i] = (struct range) { + dvsec_range = (struct range) { .start = base, - .end = base + size - 1 + .end = base + size - 1, }; + cxld_dev = device_find_child(&root->dev, &dvsec_range, + dvsec_range_allowed); + if (!cxld_dev) { + dev_dbg(dev, "DVSEC Range%d denied by platform\n", i); + continue; + } + dev_dbg(dev, "DVSEC Range%d allowed by platform\n", i); + put_device(cxld_dev); + + info->dvsec_range[ranges] = dvsec_range; + ranges++; } @@ -439,9 +432,8 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, void __iomem *hdm = cxlhdm->regs.hdm_decoder; struct cxl_port *port = cxlhdm->port; struct device *dev = cxlds->dev; - struct cxl_port *root; - int i, rc, allowed; u32 global_ctrl = 0; + int rc; if (hdm) global_ctrl = readl(hdm + CXL_HDM_DECODER_CTRL_OFFSET); @@ -455,30 +447,16 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, else if (!hdm) return -ENODEV; - root = to_cxl_port(port->dev.parent); - while (!is_cxl_root(root) && is_cxl_port(root->dev.parent)) - root = to_cxl_port(root->dev.parent); - if (!is_cxl_root(root)) { - dev_err(dev, "Failed to acquire root port for HDM enable\n"); - return -ENODEV; - } - - for (i = 0, allowed = 0; info->mem_enabled && i < info->ranges; i++) { - struct device *cxld_dev; + if (!info->mem_enabled) { + rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); + if (rc) + return rc; - cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i], - dvsec_range_allowed); - if (!cxld_dev) { - dev_dbg(dev, "DVSEC Range%d denied by platform\n", i); - continue; - } - dev_dbg(dev, "DVSEC Range%d allowed by platform\n", i); - put_device(cxld_dev); - allowed++; + return devm_cxl_enable_mem(&port->dev, cxlds); } - if (!allowed && info->mem_enabled) { - dev_err(dev, "Range register decodes outside platform defined CXL ranges.\n"); + if (!info->ranges && info->mem_enabled) { + dev_err(dev, "No available DVSEC register ranges.\n"); return -ENXIO; } @@ -491,14 +469,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, * match. If at least one DVSEC range is enabled and allowed, skip HDM * Decoder Capability Enable. */ - if (info->mem_enabled) - return 0; - - rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); - if (rc) - return rc; - - return devm_cxl_enable_mem(&port->dev, cxlds); + return 0; } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, CXL); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index a6613a6f8923..6d9126d5ee56 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -790,6 +790,7 @@ static inline int cxl_root_decoder_autoremove(struct device *host, } int cxl_endpoint_autoremove(struct cxl_memdev *cxlmd, struct cxl_port *endpoint); +#define CXL_DVSEC_RANGE_MAX 2 /** * struct cxl_endpoint_dvsec_info - Cached DVSEC info * @mem_enabled: cached value of mem_enabled in the DVSEC at init time @@ -801,7 +802,7 @@ struct cxl_endpoint_dvsec_info { bool mem_enabled; int ranges; struct cxl_port *port; - struct range dvsec_range[2]; + struct range dvsec_range[CXL_DVSEC_RANGE_MAX]; }; struct cxl_hdm; @@ -810,7 +811,7 @@ struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port, int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); -int cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info); bool is_cxl_region(struct device *dev); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index 97c21566677a..a8c241cb4ce2 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -98,7 +98,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_port *root; int rc; - rc = cxl_dvsec_rr_decode(cxlds->dev, cxlds->cxl_dvsec, &info); + rc = cxl_dvsec_rr_decode(cxlds->dev, port, &info); if (rc < 0) return rc; diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index 6f737941dc0e..79fdfaad49e8 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -228,7 +228,7 @@ int __wrap_cxl_hdm_decode_init(struct cxl_dev_state *cxlds, } EXPORT_SYMBOL_NS_GPL(__wrap_cxl_hdm_decode_init, CXL); -int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int __wrap_cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { int rc = 0, index; @@ -237,7 +237,7 @@ int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, if (ops && ops->is_mock_dev(dev)) rc = 0; else - rc = cxl_dvsec_rr_decode(dev, dvsec, info); + rc = cxl_dvsec_rr_decode(dev, port, info); put_cxl_mock_ops(index); return rc; From patchwork Fri Aug 9 08:27:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Ming4" X-Patchwork-Id: 13758562 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8A3318F2F2 for ; Fri, 9 Aug 2024 08:58:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723193908; cv=none; b=FF0+oaGae2cCIdK7fLznM+z/LX4/ZFyivzJ0KkyjvViYdG8dPfqoF79R01CTWKuWqQpKa12+j6xSGRZpD8KDu6QQXOqaHQJgqvB9yuG2hSXu0Nqxwqc0jEcRw4QL7SCjcodNSvemBu7k5tpgBQcUENanDTchjjnDx4zmkALcPVk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723193908; c=relaxed/simple; bh=aaBjJhGZi5IEPHxQZlvkqNzzr/hnUolzpoaE2tqhXf0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HtgQSSZt1KCAywFKruDsHtlP4wiCOaiBRFsJGN6jVzt41xoNKP2tH+IMEDFRhc0CiKUpD+hIyhIDOM26Li9k+aRFOmQ0HvGb3wy96SL4qjI1zMJKx5dK/zpnaa2c17d6gjJHv68nxAu3SntXDqn5G0RchRjSf/LmP0Etu8ORoUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=XXpT5X+8; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="XXpT5X+8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723193907; x=1754729907; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aaBjJhGZi5IEPHxQZlvkqNzzr/hnUolzpoaE2tqhXf0=; b=XXpT5X+8K9GNDEPSdltTT0GJeosuSLgR4G7CUDuClAQAMnmK1dwHFvVr W19UMPhZ0G3ruqDCxp8KIZmTN1rqmIyvHmC+R499I7EAGMpzcuJ6rf6Qm HR5jpfOA05mX/3r7CYBiDFL+76/IkUEm2wE6SIjwBKXDOhBk4IxWlwN0V MKsH2p/XoEoi+wRuXxPznMrKSCMe/zNL4sd/3+ylmV/6/TWPTWnIwLS26 y7r1xWIoPVBWrT4n5ja3eATJocVa55BwXa2Rd0ngQN7TbTbGchbtuUqiD 75ufd9iw7DlTPc5yaN2W2/oA4E1pYdZqlMKJbfufHI+r1mRilhvKERyIN A==; X-CSE-ConnectionGUID: PD1mGgqORBKuoGDvJFuh9Q== X-CSE-MsgGUID: hTBLiaEcR7ywIxz6p/2hvA== X-IronPort-AV: E=McAfee;i="6700,10204,11158"; a="32507236" X-IronPort-AV: E=Sophos;i="6.09,275,1716274800"; d="scan'208";a="32507236" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2024 01:58:19 -0700 X-CSE-ConnectionGUID: RYhkaLcGQtmXUVERnYfi5Q== X-CSE-MsgGUID: /KAh7UX8QBSlEWAlhfb7SQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,275,1716274800"; d="scan'208";a="57164171" Received: from s2600wttr.bj.intel.com ([10.240.192.138]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2024 01:58:16 -0700 From: Li Ming To: linux-cxl@vger.kernel.org, rrichter@amd.com, terry.bowman@amd.com, dan.j.williams@intel.com, alison.schofield@intel.com, pengfei.xu@intel.com Cc: Li Ming Subject: [PATCH v2 1/2] cxl/pci: Get AER capability address from RCRB only for RCH dport Date: Fri, 9 Aug 2024 08:27:49 +0000 Message-Id: <20240809082750.3015641-2-ming4.li@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240809082750.3015641-1-ming4.li@intel.com> References: <20240809082750.3015641-1-ming4.li@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 cxl_setup_parent_dport() needs to get RCH dport AER capability address from RCRB to disable AER interrupt. The function does not check if dport is RCH dport, it will get a wrong pci_host_bridge structure by dport_dev in VH case because dport_dev points to a pci device(RP or switch DSP) rather than a pci host bridge device. Fixes: f05fd10d138d ("cxl/pci: Add RCH downstream port AER register discovery") Signed-off-by: Li Ming --- drivers/cxl/core/pci.c | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a663e7566c48..51132a575b27 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -834,11 +834,13 @@ static void cxl_disable_rch_root_ints(struct cxl_dport *dport) void cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) { struct device *dport_dev = dport->dport_dev; - struct pci_host_bridge *host_bridge; - host_bridge = to_pci_host_bridge(dport_dev); - if (host_bridge->native_aer) - dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); + if (dport->rch) { + struct pci_host_bridge *host_bridge = to_pci_host_bridge(dport_dev); + + if (host_bridge->native_aer) + dport->rcrb.aer_cap = cxl_rcrb_to_aer(dport_dev, dport->rcrb.base); + } dport->reg_map.host = host; cxl_dport_map_regs(dport); From patchwork Fri Aug 9 08:27:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Li, Ming4" X-Patchwork-Id: 13758563 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B3DA18FDD0 for ; Fri, 9 Aug 2024 08:58:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723193910; cv=none; b=Eo+HsikeINaEV2ED1TjvI53OdeYGvvmfUgo929s0JvyRpghv+n7RMjlotETQdJHfhyIgIYeBYi0iB8i7muukZDyfOcVXwiULzRLykG5A7UV7ovxXJ3YBXaZjS8z5pd0A9qklRRov1BqcUcSqoXqPZ/FgG9/vkmA70aZQSjHrAf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723193910; c=relaxed/simple; bh=6pEZquy8orhHZvOHNSun1kxpAHpdMw7xLJAtNs3rRuo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Xu2YWZ0liBIxVyCUVvsdmtcTBYK4HE32NwvPFlUR8hO4xOFtEMiDqdtwZf9+I2++n8BpLNrfiN/Ge99f1eh1+FIJGTex+4GRKCnWJ+cdnK0/2RsEMtp+qbN80rQreu0pCB4tYhZAAIOe+0Ridljt4g6Yd2NlPBhp/DZUPsImJdY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=KCt/YUSp; arc=none smtp.client-ip=198.175.65.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="KCt/YUSp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723193909; x=1754729909; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6pEZquy8orhHZvOHNSun1kxpAHpdMw7xLJAtNs3rRuo=; b=KCt/YUSpoGIQYZ13h2WZEPVvkhZ7xHZjlfLwnqkHSOp5bbjJy08s62jk Q7a57Z/Z+2G7dNlX6htezL9A3LnIMM5y01A7U+WxpZD9WvrkIHNYVDgD3 XB60o0he39Ov1s5KOCzzjukFtH4N4YoT8W4DS/0cWexhd2G3p3vL3rQII BGFSetevfvxvOnQoOpPIMqIS/t34YcJfQsawjzHToln55V2bDl3uRxe2J JtLhdv7oF4H1ZnK0gAogXg8NGl2+1i9I1RidLgo1/v/6YmdqBu7nzvzWB cz+JBpjXcA8M9hd9YqEg5pSZcpx0buRzNumgEfMmix21OTbJ4W1FsUU8q A==; X-CSE-ConnectionGUID: pmVbE8yFRyGoFIvvdHaH3g== X-CSE-MsgGUID: TV8WUHJhR1S/9u9Qdm+3GQ== X-IronPort-AV: E=McAfee;i="6700,10204,11158"; a="32507240" X-IronPort-AV: E=Sophos;i="6.09,275,1716274800"; d="scan'208";a="32507240" Received: from fmviesa007.fm.intel.com ([10.60.135.147]) by orvoesa105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2024 01:58:21 -0700 X-CSE-ConnectionGUID: 9T73IJDgQheA8gSHsdvZvQ== X-CSE-MsgGUID: 0z8tthZTSr2WzOp/Epi8tQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,275,1716274800"; d="scan'208";a="57164175" Received: from s2600wttr.bj.intel.com ([10.240.192.138]) by fmviesa007-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 09 Aug 2024 01:58:18 -0700 From: Li Ming To: linux-cxl@vger.kernel.org, rrichter@amd.com, terry.bowman@amd.com, dan.j.williams@intel.com, alison.schofield@intel.com, pengfei.xu@intel.com Cc: Li Ming Subject: [PATCH v2 2/2] cxl/test: Skip cxl_setup_parent_dport() for emulated dports Date: Fri, 9 Aug 2024 08:27:50 +0000 Message-Id: <20240809082750.3015641-3-ming4.li@intel.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: <20240809082750.3015641-1-ming4.li@intel.com> References: <20240809082750.3015641-1-ming4.li@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The cxl_test unit test environment on qemu always hits below call trace with KASAN enabled: BUG: KASAN: slab-out-of-bounds in cxl_setup_parent_dport+0x480/0x530 [cxl_core] Read of size 1 at addr ff110000676014f8 by task (udev-worker)/676[ 24.424403] CPU: 2 PID: 676 Comm: (udev-worker) Tainted: G O N 6.10.0-qemucxl #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS edk2-20240214-2.el9 02/14/2024 Call Trace: dump_stack_lvl+0xea/0x150 print_report+0xce/0x610 ? kasan_complete_mode_report_info+0x40/0x200 kasan_report+0xcc/0x110 __asan_report_load1_noabort+0x18/0x20 cxl_setup_parent_dport+0x480/0x530 [cxl_core] cxl_mem_probe+0x49b/0xaa0 [cxl_mem] cxl_test module models a CXL topology for testing, it creates some emulated dports with platform devices in the CXL topology, so the dport_dev of an emulated dport points to a platform device rather than a pci device or a pci host bridge in the case. Currently, cxl_setup_parent_dport() is used to set up RAS and AER capability on the dport connected to the CXL memory device, but cxl_test does not support RAS or AER functionality yet, so the fix is implementing a __wrap_cxl_setup_parent_dport() to filter out all emulated dports, guarantees only real dports can be handled by cxl_setup_parent_dport(). Fixes: f05fd10d138d ("cxl/pci: Add RCH downstream port AER register discovery") Reported-by: Pengfei Xu Closes: https://lore.kernel.org/linux-cxl/ZrHTBp2O+HtUe6kt@xpf.sh.intel.com/T/#t Signed-off-by: Li Ming --- tools/testing/cxl/Kbuild | 1 + tools/testing/cxl/test/mock.c | 12 ++++++++++++ 2 files changed, 13 insertions(+) diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index 030b388800f0..3d1ca9e38b1f 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -14,6 +14,7 @@ ldflags-y += --wrap=cxl_dvsec_rr_decode ldflags-y += --wrap=devm_cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat +ldflags-y += --wrap=cxl_setup_parent_dport DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxl diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index 6f737941dc0e..d619672faa49 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -299,6 +299,18 @@ void __wrap_cxl_endpoint_parse_cdat(struct cxl_port *port) } EXPORT_SYMBOL_NS_GPL(__wrap_cxl_endpoint_parse_cdat, CXL); +void __wrap_cxl_setup_parent_dport(struct device *host, struct cxl_dport *dport) +{ + int index; + struct cxl_mock_ops *ops = get_cxl_mock_ops(&index); + + if (!ops || !ops->is_mock_port(dport->dport_dev)) + cxl_setup_parent_dport(host, dport); + + put_cxl_mock_ops(index); +} +EXPORT_SYMBOL_NS_GPL(__wrap_cxl_setup_parent_dport, CXL); + MODULE_LICENSE("GPL v2"); MODULE_IMPORT_NS(ACPI); MODULE_IMPORT_NS(CXL); From patchwork Tue Aug 13 11:05:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13761816 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8917E60EC4 for ; Tue, 13 Aug 2024 11:13:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547588; cv=none; b=e8+WrK6ub026WrAfQ9P3eohqq3vRyPovNFRlmY//dl5pMyQDV3PyV+wDIByAiK5alJiu96tPXRDuTTVkE6jdi4FvyH5GqXmPQ6/OdzPYBTzMC8SWeD6Eny193xgrW+Kv+BH9CTn1Nr2E8DahLuQUbr9EExtuStsPnTbuKR0wggw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547588; c=relaxed/simple; bh=gZwhe97f+wILNfWjc23vBK22qTvjp4eHrxhYf9C7Gj8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=p78NgkxNr6z8Un7laTnSKG4C4jWAPvYMC4HVCDcpiFAziluZUO2s1R/qwKD4A4ZfsohhfKPexP1zjfylFBTvDbfnMXf8q6o8+C7YZDqZQXETwHvl4i/Rsc/vVU81FdePRFUj80yELOeov4rtoCgZcl3CaXqXa3VoXmBnM4n8tek= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FF/DGmb1; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FF/DGmb1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723547586; x=1755083586; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gZwhe97f+wILNfWjc23vBK22qTvjp4eHrxhYf9C7Gj8=; b=FF/DGmb1J1YwOcxcRPT6v252YoTzxtl44ymgmYj5NMWCukXxXiczDdUR MwZY10qgi1c6k15RV+jAKzTQhGL4QTBPagt15aXfktKcZLiF/T3Q5Den7 vS4G7M517pB1pjWqvfSDFvx2F/MO09TygQkQk08hR1i08lkIwAMzLKiJh uHGpJTEw/5DcTibT6QWO9xn4Mf+/E6tZExiV40BHQ+yYZitL2Mr0nkmMo 3VlwAsKXoIg3e7oISgFaREuQZmPabb/SytIuk+oXiNrTgB9Slsci77Rmn LY+Zi0yjSznM8SjFHJaMn/FAIVqRCNvvkKtbVPMQrIDyq3TVKpWuwXCWw w==; X-CSE-ConnectionGUID: iYbGQHkIQJiBAbzUeEmfkQ== X-CSE-MsgGUID: Jec62hZ/RQyTlT1n7C8B+g== X-IronPort-AV: E=McAfee;i="6700,10204,11162"; a="21262277" X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="21262277" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:06 -0700 X-CSE-ConnectionGUID: NNgYINwTQWuEMf0DlnorpA== X-CSE-MsgGUID: mQ63OAixSz2oPScArVQ5JQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="58568752" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:04 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v3 1/4] cxl/pci: Fix to record only non-zero ranges Date: Tue, 13 Aug 2024 19:05:29 +0800 Message-Id: <20240813110532.870869-2-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240813110532.870869-1-yanfei.xu@intel.com> References: <20240813110532.870869-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The function cxl_dvsec_rr_decode() retrieves and records DVSEC ranges into info->dvsec_range[], regardless of whether it is non-zero range, and the variable info->ranges indicates the number of non-zero ranges. However, in cxl_hdm_decode_init(), the validation for info->dvsec_range[] occurs in a for loop that iterates based on info->ranges. It may result in zero range to be validated but non-zero range not be validated, in turn, the number of allowed ranges is to be 0. Address it by only record non-zero ranges. This fix is not urgent as it requires a configuration that zeroes out the first dvsec range while populating the second. This has not been observed, but it is theoretically possible. If this gets picked up for -stable, no harm done, but there is no urgency to backport. Fixes: 560f78559006 ("cxl/pci: Retrieve CXL DVSEC memory info") Signed-off-by: Yanfei Xu Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index a663e7566c48..2d69340134da 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -390,10 +390,6 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, size |= temp & CXL_DVSEC_MEM_SIZE_LOW_MASK; if (!size) { - info->dvsec_range[i] = (struct range) { - .start = 0, - .end = CXL_RESOURCE_NONE, - }; continue; } @@ -411,12 +407,10 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, base |= temp & CXL_DVSEC_MEM_BASE_LOW_MASK; - info->dvsec_range[i] = (struct range) { + info->dvsec_range[ranges++] = (struct range) { .start = base, .end = base + size - 1 }; - - ranges++; } info->ranges = ranges; From patchwork Tue Aug 13 11:05:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13761817 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B191960EC4 for ; Tue, 13 Aug 2024 11:13:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547591; cv=none; b=h6LaBU9L95Ye5SujnEFN7DysWN7vUk//Q/U8vgIrhK/IWPKFWjtIfwzJc9UhiNeixT5zWkkIWGfQU4mOT+xUMBTw47JWxB3QKv7CQoTKeb11T0P39tUDdawqJfGltXiiAefvvMQdR6SaJOs2ch/OKQpnjVmmeiyD0HE7a/qM6VY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547591; c=relaxed/simple; bh=cjyg3QVdjFxExYESW6PikqbPYX5Rl03jMOJ/Yn+E064=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=OIsT2Vm9tnwPliVEDfR2+9wzHYMA43gnzxrOtyknHLIah+HfoTehrhqWRNVcbm+kRI0h4qIAJSTtaGR08U8TDSOZkw+4OmJDAPX5NIujoS5BnfpePASwIxzf108YU3O/WPyhND3CkwWXwqNYydyx86fsTj42EHfIUuPpw+O1mB4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=C3VcF3Ao; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="C3VcF3Ao" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723547589; x=1755083589; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=cjyg3QVdjFxExYESW6PikqbPYX5Rl03jMOJ/Yn+E064=; b=C3VcF3AoUQmYG7paw+pTd36s7nNLmY1/U9FsGSnCX6HlMed8XzPzR2cJ XJV90x/pNTbMVRDfC9SqHSrf19wm1IxM2IJ612hhYDN1Hh9BUMM2+WC+7 t1/EnPIkaJQxdPWTL0zAFLbpWE6iGghE6rJ9ngnE3FtHGE2UEXyRne9Kb KtvwvanQv7PZ4dfBGojp88u9HJUg3lpXdiXGxl24GWJp/cZzMz68KFcAs OCBKG5q1vF9qezV4wKwAMvCp/n9tGVR5G892+dn3MEUxJPc1Qio29wPGL HBshE3QHCJUA+0F+3UKCAGoPAgDMzltPdTRz5AisFnLrsfBMn6Z/0shlT g==; X-CSE-ConnectionGUID: 8RMR8SH+TiymmI8yqB+jrw== X-CSE-MsgGUID: iL+ss7IoSBqOmfHTMz6+og== X-IronPort-AV: E=McAfee;i="6700,10204,11162"; a="21262318" X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="21262318" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:09 -0700 X-CSE-ConnectionGUID: moZFYLtfQ0GLwM5gpyMMow== X-CSE-MsgGUID: OTrRLb4sR2Gu6k7D1KgEpA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="58568783" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:07 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v3 2/4] cxl/pci: Remove duplicated implementation of waiting for memory_info_valid Date: Tue, 13 Aug 2024 19:05:30 +0800 Message-Id: <20240813110532.870869-3-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240813110532.870869-1-yanfei.xu@intel.com> References: <20240813110532.870869-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 commit ce17ad0d5498 ("cxl: Wait Memory_Info_Valid before access memory related info") added another implementation of waiting for memory_info_valid without realizing it duplicated wait_for_valid() Suggested-by: Dan Williams Signed-off-by: Yanfei Xu Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 41 +++++------------------------------ drivers/cxl/cxl.h | 2 +- drivers/cxl/port.c | 2 +- tools/testing/cxl/test/mock.c | 4 ++-- 4 files changed, 9 insertions(+), 40 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 2d69340134da..38c567727dbb 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -211,37 +211,6 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) } EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, CXL); -static int wait_for_valid(struct pci_dev *pdev, int d) -{ - u32 val; - int rc; - - /* - * Memory_Info_Valid: When set, indicates that the CXL Range 1 Size high - * and Size Low registers are valid. Must be set within 1 second of - * deassertion of reset to CXL device. Likely it is already set by the - * time this runs, but otherwise give a 1.5 second timeout in case of - * clock skew. - */ - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - msleep(1500); - - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - return -ETIMEDOUT; -} - static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val) { struct pci_dev *pdev = to_pci_dev(cxlds->dev); @@ -322,11 +291,13 @@ static int devm_cxl_enable_hdm(struct device *host, struct cxl_hdm *cxlhdm) return devm_add_action_or_reset(host, disable_hdm, cxlhdm); } -int cxl_dvsec_rr_decode(struct device *dev, int d, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { struct pci_dev *pdev = to_pci_dev(dev); + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); int hdm_count, rc, i, ranges = 0; + int d = cxlds->cxl_dvsec; u16 cap, ctrl; if (!d) { @@ -353,11 +324,9 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = wait_for_valid(pdev, d); - if (rc) { - dev_dbg(dev, "Failure awaiting MEM_INFO_VALID (%d)\n", rc); + rc = cxl_dvsec_mem_range_valid(cxlds, 0); + if (rc) return rc; - } /* * The current DVSEC values are moot if the memory capability is diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9afb407d438f..e2e277463794 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -809,7 +809,7 @@ struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port, int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); -int cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info); bool is_cxl_region(struct device *dev); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d7d5d982ce69..861dde65768f 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -98,7 +98,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_port *root; int rc; - rc = cxl_dvsec_rr_decode(cxlds->dev, cxlds->cxl_dvsec, &info); + rc = cxl_dvsec_rr_decode(cxlds->dev, port, &info); if (rc < 0) return rc; diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index 6f737941dc0e..79fdfaad49e8 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -228,7 +228,7 @@ int __wrap_cxl_hdm_decode_init(struct cxl_dev_state *cxlds, } EXPORT_SYMBOL_NS_GPL(__wrap_cxl_hdm_decode_init, CXL); -int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int __wrap_cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { int rc = 0, index; @@ -237,7 +237,7 @@ int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, if (ops && ops->is_mock_dev(dev)) rc = 0; else - rc = cxl_dvsec_rr_decode(dev, dvsec, info); + rc = cxl_dvsec_rr_decode(dev, port, info); put_cxl_mock_ops(index); return rc; From patchwork Tue Aug 13 11:05:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13761818 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C17E60EC4 for ; Tue, 13 Aug 2024 11:13:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547593; cv=none; b=W560KP7/WiDdXMhnlQ0E5DzFOqa1KKRd8AmLBRdxYNB2EXCo4Nq8P6oLmLB9zndqEDSAKszHaDLhm+qwdzOQtL+kqwQhELqRQZ0854MGDcTIO3cj6z2s11e7ElxtRhpiEdMLU6cv9GpXkA9SEqk1GMwCW0Ob4qB+OMwVks7WJRw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547593; c=relaxed/simple; bh=HtbhDkCJFzG9yz3rILklI4MVQo2jGhPyNbk4lIsj02U=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=BW+6Ajo6elulhdA/aZGHmEOQ/ZUCobSVeURuSP9Dbjp2kEB0Qp2CkcsxY9I7+FnfG8xoKx2fSp+ito88YhHztSZFvKWhdHLOR4L1u6PJkHST00+CPBmYtGRQ2ycqymLHRrQZhpHBqgxmuee9x1AWEnELIFZwTILsgP4GJSz3F/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PkCuA4Eb; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PkCuA4Eb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723547592; x=1755083592; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HtbhDkCJFzG9yz3rILklI4MVQo2jGhPyNbk4lIsj02U=; b=PkCuA4Eb+sWy45TiNAIBiFI1mwBI5f0hsLn1vqrWIQwZxoeH1RzEjQcE +3iZlIG/So3ZnG69q80zhub4gdfNp8TonaEfswTL0YiRQXl7ptXO+fOSX 5Otnm9nrREJ/jI/CJR6vzu0Qu1GQm8IC5NNx/OjLk2euwPRgpkLPXUYvM Xvo2WFYy/uhTjyN0AR3a9Yt4D/0ggB5H3HTowTXgnQmQrp8pjQ7t4FQOG pI9JCIVnzIKF7/nOJLkvC/S5pFjNh+ZmGn/3YWBrLzpU+qtfroV+Q3eny ZPtg2SCMN1mBeRSxN2D8LlUXKfJxS4pbqEbf7M0TM6+vW+oFlosR5W2+7 g==; X-CSE-ConnectionGUID: khRhn2dZQL+l8c2+b3BGZw== X-CSE-MsgGUID: 52ZBBNcdQoaG1FaJS4Sigg== X-IronPort-AV: E=McAfee;i="6700,10204,11162"; a="21262341" X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="21262341" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:12 -0700 X-CSE-ConnectionGUID: f3H6uXsGRY2WycN3Ix924g== X-CSE-MsgGUID: C5MnOlrDS3iHmnWF1+HgTA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="58568797" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:10 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v3 3/4] cxl/pci: Check Mem_info_valid bit for each applicable DVSEC Date: Tue, 13 Aug 2024 19:05:31 +0800 Message-Id: <20240813110532.870869-4-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240813110532.870869-1-yanfei.xu@intel.com> References: <20240813110532.870869-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The right way is to checking Mem_info_valid bit for each applicable DVSEC range against HDM_COUNT, not only for the DVSEC range 1, hence let's move the check into the "for loop" of handling each DVSEC range. Signed-off-by: Yanfei Xu Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 38c567727dbb..519989ada48e 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -324,10 +324,6 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = cxl_dvsec_mem_range_valid(cxlds, 0); - if (rc) - return rc; - /* * The current DVSEC values are moot if the memory capability is * disabled, and they will remain moot after the HDM Decoder @@ -345,6 +341,10 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, u64 base, size; u32 temp; + rc = cxl_dvsec_mem_range_valid(cxlds, i); + if (rc) + return rc; + rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_SIZE_HIGH(i), &temp); if (rc) From patchwork Tue Aug 13 11:05:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13761819 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0617D60EC4 for ; Tue, 13 Aug 2024 11:13:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547596; cv=none; b=EJGFVnI7eFZxBAGj28HkePsRSe8L+Eg6ePmRfovlVwXxUVAkaQJPhuF3FQWKVAFSJwNrTF7TlmJPig7xW3Vl4gCptgGHcKmwGcqQ7o9szXdcXStnYt8cOenjVeykKp9L+jvukAwuoQPH5NcL4QkmE3wJjh9VWCTrMDBeoW/QTqs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723547596; c=relaxed/simple; bh=kMjqxcvsJPTaZkKSSakL6w6VOlcnSVu80kSsAT0yGoY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=bwFOD5DIB2aR6wiE26bazfeA13wobz/4ss5uaTKDxljXE5Yx9Qt6DHBeLcISTBSkge8jhTXHSm5ZR0nhPOCTVojNbXKMcrXK8sz/SzRFZABrlgKURbCQy3kuFUj//y9240KeWdXLFDOmA9V1PqOITjHVd6iaYwAWVz56uNblvEA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=a7894Puk; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="a7894Puk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723547595; x=1755083595; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kMjqxcvsJPTaZkKSSakL6w6VOlcnSVu80kSsAT0yGoY=; b=a7894PukFtkZdiG5e0UWKFalvrZlVdWpgWgYwJ8VQ/Gk/aXNcRn3Tt2m BWSJXLXewqXU5+sZW37kyxbNYI6ija4rHZlIf7HMH/yGgXh4PqucBE/Zv VNDG6EhtfFoXZaZsBI2vrQToGxGC2Et6+GuwjTVWw4ywBkYnDc7M0WZGk xz/uexeby9yKKgO8OAuuxWprXVFQsU5j/OL3FfcZ6lWSVfST7wl+kd5he ZuSvAY7oXXfB0kRDIcBWBcnoPvCxHJ+oztxvTlUZ5qknghvADssq/Rm2L gMKtx38VZbKTYZAadO7GMHEsDUn8GLEVDyPjRBotek+XqjDwHhCASLmy0 w==; X-CSE-ConnectionGUID: WwygHzvMTu6DrGanDCCD5A== X-CSE-MsgGUID: jtYZoDPBSJSzTrmn1bIIxQ== X-IronPort-AV: E=McAfee;i="6700,10204,11162"; a="21262357" X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="21262357" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:14 -0700 X-CSE-ConnectionGUID: ymDDzEDESySXB8R4SqK64w== X-CSE-MsgGUID: Hz0H4BaqSEeu+LJy+xQm5A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,285,1716274800"; d="scan'208";a="58568812" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 04:13:13 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v3 4/4] cxl/pci: simplify the check of mem_enabled in cxl_hdm_decode_init() Date: Tue, 13 Aug 2024 19:05:32 +0800 Message-Id: <20240813110532.870869-5-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240813110532.870869-1-yanfei.xu@intel.com> References: <20240813110532.870869-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Cases can be divided into two categories which are DVSEC range enabled and not enabled when HDM decoders exist but is not enabled. To avoid checking info->mem_enabled, which indicates the enablement of DVSEC range, every time, we can check !info->mem_enabled once in advance. This simplification can make the code clearer. No functional change intended. Signed-off-by: Yanfei Xu Reviewed-by: Jonathan Cameron --- drivers/cxl/core/pci.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 519989ada48e..be00266c8907 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -426,7 +426,15 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, return -ENODEV; } - for (i = 0, allowed = 0; info->mem_enabled && i < info->ranges; i++) { + if (!info->mem_enabled) { + rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); + if (rc) + return rc; + + return devm_cxl_enable_mem(&port->dev, cxlds); + } + + for (i = 0, allowed = 0; i < info->ranges; i++) { struct device *cxld_dev; cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i], @@ -440,7 +448,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, allowed++; } - if (!allowed && info->mem_enabled) { + if (!allowed) { dev_err(dev, "Range register decodes outside platform defined CXL ranges.\n"); return -ENXIO; } @@ -454,14 +462,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, * match. If at least one DVSEC range is enabled and allowed, skip HDM * Decoder Capability Enable. */ - if (info->mem_enabled) - return 0; - - rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); - if (rc) - return rc; - - return devm_cxl_enable_mem(&port->dev, cxlds); + return 0; } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, CXL); From patchwork Tue Aug 13 18:25:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13762402 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4E7935894; Tue, 13 Aug 2024 18:26:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723573563; cv=none; b=dmKWvE4irQig/00E5enjqgIJgtBTSReg77zFXttRn0xJAb88rHRKC5ebZhyQlAn5nHKOuRvRyxLD+/ZMQgY6gdHGUU5eQFai+XzmNja3/Eu4s4V99npAPKJR/XT9EWJl9P9OvBzP/R95zpbjhbXrjuSRBQPSpreQNWIsTpKjjJc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723573563; c=relaxed/simple; bh=X/RhGHI5I+eLq6MIzW8PFCsIaPH0gvXDQbvnvsDnvcA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=RAJgQl5oa5q1n62WGc3ps9KpTXoMeQwmu6PRNl0kQiSYX3xEb6UXYLAk6yfJP43B1wPmjleEygqNqWmccI/X5Sl7rR2+dV4GfYox/fNJC7eW1niC3abwHOewWA+JTuwy8Ebw9nANUJ9BN3kbklcPjaNXcQOORGebJLsY3ET0stU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Pa4Li6Uk; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Pa4Li6Uk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723573561; x=1755109561; h=from:date:subject:mime-version:content-transfer-encoding: message-id:to:cc; bh=X/RhGHI5I+eLq6MIzW8PFCsIaPH0gvXDQbvnvsDnvcA=; b=Pa4Li6UkzBy4SZVMcbnC6qoKCUzh0AfrPdgZwavUwLCN6fsrqc6MHfJ3 j42IiizXf3JT+rObmSVG1+Phlhzuhl1dJDugHMBCx8LzjkgVAR2JqNYKm o1+KaUnZKrrUT3FnVYoKNhFvDQqLQJg/PTWXhuupH5vG0chkkyC33pHUK tmkq5pO0VTaVHzoTT42dhWWzR2PFKVNHrPuhci22BZEae31ZFHq3BAomg OnFGpaGlTqiHwAIGlw/m1Ge+Ama+XAcrs54h2H9b2jmBIvpRrgyA+7nWx WPNEV0tKkO7VaJorV7FbKdF2MkiRB4kGsyFKhPwUSki1T6npGK8O72Fls Q==; X-CSE-ConnectionGUID: qkPbto2lQRKk6Ow1qQywUA== X-CSE-MsgGUID: 5ScsTmddRoKCA1jL9uKAtQ== X-IronPort-AV: E=McAfee;i="6700,10204,11163"; a="24667184" X-IronPort-AV: E=Sophos;i="6.09,286,1716274800"; d="scan'208";a="24667184" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 11:26:00 -0700 X-CSE-ConnectionGUID: 2lInno0gQNyovCmc8izrWg== X-CSE-MsgGUID: M7AtvTALT/2ewlxSO+v4vA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.09,286,1716274800"; d="scan'208";a="62911307" Received: from cmdeoliv-mobl.amr.corp.intel.com (HELO localhost) ([10.125.109.241]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Aug 2024 11:25:59 -0700 From: Ira Weiny Date: Tue, 13 Aug 2024 13:25:57 -0500 Subject: [PATCH] cxl/region: Remove lock from memory notifier callback Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240813-fix-notifiers-v1-1-efd23a18688d@intel.com> X-B4-Tracking: v=1; b=H4sIADSlu2YC/x2MSwqAIBQAryJvneAvyK4SLcye9TYaGhGId09aD sxMhYKZsMDMKmR8qFCKHeTAwJ8uHshp7wxKKCMmqXmgl8d0UyDMhVvr9Sg2YYxT0JsrYxf+37K 29gGjqmAJXwAAAA== To: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Dan Williams Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Ira Weiny X-Mailer: b4 0.13-dev-2d940 X-Developer-Signature: v=1; a=ed25519-sha256; t=1723573557; l=3584; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=X/RhGHI5I+eLq6MIzW8PFCsIaPH0gvXDQbvnvsDnvcA=; b=Yxcp83IkiNMm99+GDSIQ1nNY/qPVzEw6VjRRKYBrVO2ZPhu++AfM5J2N1Hq4Vx6WQrUSaECGs H1h/l6a9n/uD8UdGZfN0x8KqMzMZMmCcMM4NBXVDY9soSZCzrDYhmus X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= In testing Dynamic Capacity Device (DCD) support, a lockdep splat revealed an ABBA issue between the memory notifiers and the DCD extent processing code.[0] Changing the lock ordering within DCD proved difficult because regions must be stable while searching for the proper region and then the device lock must be held to properly notify the DAX region driver of memory changes. Dan points out in the thread that notifiers should be able to trust that it is safe to access static data. Region data is static once the device is realized and until it's destruction. Thus it is better to manage the notifiers within the region driver. Remove the need for a lock by ensuring the notifiers are active only during the region's lifetime. Link: https://lore.kernel.org/all/66b4cf539a79b_a36e829416@iweiny-mobl.notmuch/ [0] Cc: Huang, Ying Suggested-by: Dan Williams Signed-off-by: Ira Weiny --- drivers/cxl/core/region.c | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) --- base-commit: afdab700f65e14070d8ab92175544b1c62b8bf03 change-id: 20240813-fix-notifiers-99c350b044a2 Best regards, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21ad5f242875..971a314b6b0e 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2313,8 +2313,6 @@ static void unregister_region(void *_cxlr) struct cxl_region_params *p = &cxlr->params; int i; - unregister_memory_notifier(&cxlr->memory_notifier); - unregister_mt_adistance_algorithm(&cxlr->adist_notifier); device_del(&cxlr->dev); /* @@ -2396,7 +2394,6 @@ static int cxl_region_nid(struct cxl_region *cxlr) struct cxl_region_params *p = &cxlr->params; struct resource *res; - guard(rwsem_read)(&cxl_region_rwsem); res = p->res; if (!res) return NUMA_NO_NODE; @@ -2484,14 +2481,6 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, if (rc) goto err; - cxlr->memory_notifier.notifier_call = cxl_region_perf_attrs_callback; - cxlr->memory_notifier.priority = CXL_CALLBACK_PRI; - register_memory_notifier(&cxlr->memory_notifier); - - cxlr->adist_notifier.notifier_call = cxl_region_calculate_adistance; - cxlr->adist_notifier.priority = 100; - register_mt_adistance_algorithm(&cxlr->adist_notifier); - rc = devm_add_action_or_reset(port->uport_dev, unregister_region, cxlr); if (rc) return ERR_PTR(rc); @@ -3386,6 +3375,14 @@ static int is_system_ram(struct resource *res, void *arg) return 1; } +static void shutdown_notifiers(void *_cxlr) +{ + struct cxl_region *cxlr = _cxlr; + + unregister_memory_notifier(&cxlr->memory_notifier); + unregister_mt_adistance_algorithm(&cxlr->adist_notifier); +} + static int cxl_region_probe(struct device *dev) { struct cxl_region *cxlr = to_cxl_region(dev); @@ -3418,6 +3415,18 @@ static int cxl_region_probe(struct device *dev) out: up_read(&cxl_region_rwsem); + if (rc) + return rc; + + cxlr->memory_notifier.notifier_call = cxl_region_perf_attrs_callback; + cxlr->memory_notifier.priority = CXL_CALLBACK_PRI; + register_memory_notifier(&cxlr->memory_notifier); + + cxlr->adist_notifier.notifier_call = cxl_region_calculate_adistance; + cxlr->adist_notifier.priority = 100; + register_mt_adistance_algorithm(&cxlr->adist_notifier); + + rc = devm_add_action_or_reset(&cxlr->dev, shutdown_notifiers, cxlr); if (rc) return rc; From patchwork Wed Aug 14 17:49:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13763790 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11F06566A; Wed, 14 Aug 2024 17:49:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723657795; cv=none; b=FeVFA+zGOEDTglhtbVl+viPAvXY50FJYjks0Apmbik1D7DGrdGMnvHB5frChcZifKg33vYvdniInXN2DR/HBC+YaYjT1QqAzDlRfRlG4s2Gsc9tREiOy5UrPsZyoA1XQ+Fqh2GTxdeZZPnxpE0TiirqxaINCbKQJ4sOXGjCGrOQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1723657795; c=relaxed/simple; bh=7Kig8+LDSPEdSjGC360OezgR5t+AxhfIthSl98qftO8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=eVQQCRmbNZBDKGPf+VKB6z1kppOQ7clFqmGtIkobaK0LLZDZL5NOc6Ez/7EQiozq3t2qf6sKdjEwHk9n/S9/DWFNSOZP5y8lgHrMnZ09c30ZEiXrAGe78Mdz5u0/AjyOwlmSUWRNpX1HFhtE4Ze4EVDYlBIumIzBBWjr7ox1I6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=lPlioNc5; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="lPlioNc5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1723657793; x=1755193793; h=from:date:subject:mime-version:content-transfer-encoding: message-id:to:cc; bh=7Kig8+LDSPEdSjGC360OezgR5t+AxhfIthSl98qftO8=; b=lPlioNc5eRtlAMBzOxRKI/0gma8pmsW/65TViEEnLZul60h+VCnzZocI drRSxp8JLAfESzzcVcFidtPYuqmnvdpfrBdAgkOb3/7vVw4604673/sm4 bCr6qM5oUE+56EkR+g8pykcyZT3pCKS8udwm4421mshklnL481Qc5hZ1m +wkfDWdE678z0gZZ4evVfmCKhJCa4SRPTZzfqPvqlzC/iLm0fu1wC3is4 5GBBEVFCLbR6ghTkCRTD38dTQv9Dt1M4NVVM4jZUonIRWbp+3HN8/7+0g cpOIBnEdZ75zNOCosNZ4F+wR5/bEmx4YQmwnqXOZA08E6qP258dHIap7M A==; X-CSE-ConnectionGUID: GRw1Yzp1RVqAujo/JjHZMg== X-CSE-MsgGUID: 8JXfHUOXRWi5hcHFlJjMEg== X-IronPort-AV: E=McAfee;i="6700,10204,11164"; a="25757438" X-IronPort-AV: E=Sophos;i="6.10,146,1719903600"; d="scan'208";a="25757438" Received: from fmviesa004.fm.intel.com ([10.60.135.144]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2024 10:49:53 -0700 X-CSE-ConnectionGUID: 9ylW5xX1RsmNSa41dolpDg== X-CSE-MsgGUID: g0e0J6bTSVyuM0LjU4Qtgg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,146,1719903600"; d="scan'208";a="63750436" Received: from iweiny-mobl.amr.corp.intel.com (HELO localhost) ([10.125.110.97]) by fmviesa004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Aug 2024 10:49:51 -0700 From: Ira Weiny Date: Wed, 14 Aug 2024 12:49:39 -0500 Subject: [PATCH v2] cxl/region: Remove lock from memory notifier callback Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240814-fix-notifiers-v2-1-6bab38192c7c@intel.com> X-B4-Tracking: v=1; b=H4sIADLuvGYC/3WMyw7CIBBFf6WZtRheGurK/zBdIAx2EoUGSKNp+ Hexe5fn5p6zQcFMWOAybJBxpUIpdpCHAdxs4wMZ+c4gudTcCMUCvVlMlQJhLmwcnTrxO9faSuj OkrEf9t5t6jxTqSl/9vwqfuu/0iqYYBi8VFaYszH+SrHi8+jSC6bW2hdijUOPqgAAAA== To: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Dan Williams Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Ira Weiny X-Mailer: b4 0.13-dev-2d940 X-Developer-Signature: v=1; a=ed25519-sha256; t=1723657790; l=4730; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=7Kig8+LDSPEdSjGC360OezgR5t+AxhfIthSl98qftO8=; b=0cC1vjRF+p44Si7lE8qwrFJJNFTMTeEgm+lFR3O/TH69vKj/fvY0pvJYiVqWFbS0PPi+Vep+O 4vb2D0FalpyCF+3hsMUOyBHmdDQQ1+2ILsxyIgf6Krh2eQSUlnPUZjl X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= In testing Dynamic Capacity Device (DCD) support, a lockdep splat revealed an ABBA issue between the memory notifiers and the DCD extent processing code.[0] Changing the lock ordering within DCD proved difficult because regions must be stable while searching for the proper region and then the device lock must be held to properly notify the DAX region driver of memory changes. Dan points out in the thread that notifiers should be able to trust that it is safe to access static data. Region data is static once the device is realized and until it's destruction. Thus it is better to manage the notifiers within the region driver. Remove the need for a lock by ensuring the notifiers are active only during the region's lifetime. Furthermore, remove cxl_region_nid() because resource can't be NULL while the region is stable. Link: https://lore.kernel.org/all/66b4cf539a79b_a36e829416@iweiny-mobl.notmuch/ [0] Cc: Huang, Ying Suggested-by: Dan Williams Signed-off-by: Ira Weiny Reviewed-by: Davidlohr Bueso Reviewed-by: Jonathan Cameron Reviewed-by: "Huang, Ying" --- Changes in v2: - [djbw: remove cxl_region_nid()] - Link to v1: https://patch.msgid.link/20240813-fix-notifiers-v1-1-efd23a18688d@intel.com --- drivers/cxl/core/region.c | 46 ++++++++++++++++++++++------------------------ 1 file changed, 22 insertions(+), 24 deletions(-) --- base-commit: 6b0f8db921abf0520081d779876d3a41069dab95 change-id: 20240813-fix-notifiers-99c350b044a2 Best regards, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21ad5f242875..588add3536c3 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -2313,8 +2313,6 @@ static void unregister_region(void *_cxlr) struct cxl_region_params *p = &cxlr->params; int i; - unregister_memory_notifier(&cxlr->memory_notifier); - unregister_mt_adistance_algorithm(&cxlr->adist_notifier); device_del(&cxlr->dev); /* @@ -2391,18 +2389,6 @@ static bool cxl_region_update_coordinates(struct cxl_region *cxlr, int nid) return true; } -static int cxl_region_nid(struct cxl_region *cxlr) -{ - struct cxl_region_params *p = &cxlr->params; - struct resource *res; - - guard(rwsem_read)(&cxl_region_rwsem); - res = p->res; - if (!res) - return NUMA_NO_NODE; - return phys_to_target_node(res->start); -} - static int cxl_region_perf_attrs_callback(struct notifier_block *nb, unsigned long action, void *arg) { @@ -2415,7 +2401,7 @@ static int cxl_region_perf_attrs_callback(struct notifier_block *nb, if (nid == NUMA_NO_NODE || action != MEM_ONLINE) return NOTIFY_DONE; - region_nid = cxl_region_nid(cxlr); + region_nid = phys_to_target_node(cxlr->params.res->start); if (nid != region_nid) return NOTIFY_DONE; @@ -2434,7 +2420,7 @@ static int cxl_region_calculate_adistance(struct notifier_block *nb, int *adist = data; int region_nid; - region_nid = cxl_region_nid(cxlr); + region_nid = phys_to_target_node(cxlr->params.res->start); if (nid != region_nid) return NOTIFY_OK; @@ -2484,14 +2470,6 @@ static struct cxl_region *devm_cxl_add_region(struct cxl_root_decoder *cxlrd, if (rc) goto err; - cxlr->memory_notifier.notifier_call = cxl_region_perf_attrs_callback; - cxlr->memory_notifier.priority = CXL_CALLBACK_PRI; - register_memory_notifier(&cxlr->memory_notifier); - - cxlr->adist_notifier.notifier_call = cxl_region_calculate_adistance; - cxlr->adist_notifier.priority = 100; - register_mt_adistance_algorithm(&cxlr->adist_notifier); - rc = devm_add_action_or_reset(port->uport_dev, unregister_region, cxlr); if (rc) return ERR_PTR(rc); @@ -3386,6 +3364,14 @@ static int is_system_ram(struct resource *res, void *arg) return 1; } +static void shutdown_notifiers(void *_cxlr) +{ + struct cxl_region *cxlr = _cxlr; + + unregister_memory_notifier(&cxlr->memory_notifier); + unregister_mt_adistance_algorithm(&cxlr->adist_notifier); +} + static int cxl_region_probe(struct device *dev) { struct cxl_region *cxlr = to_cxl_region(dev); @@ -3418,6 +3404,18 @@ static int cxl_region_probe(struct device *dev) out: up_read(&cxl_region_rwsem); + if (rc) + return rc; + + cxlr->memory_notifier.notifier_call = cxl_region_perf_attrs_callback; + cxlr->memory_notifier.priority = CXL_CALLBACK_PRI; + register_memory_notifier(&cxlr->memory_notifier); + + cxlr->adist_notifier.notifier_call = cxl_region_calculate_adistance; + cxlr->adist_notifier.priority = 100; + register_mt_adistance_algorithm(&cxlr->adist_notifier); + + rc = devm_add_action_or_reset(&cxlr->dev, shutdown_notifiers, cxlr); if (rc) return rc; From patchwork Wed Aug 28 08:42:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13780921 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DBC8E15ECDF for ; Wed, 28 Aug 2024 08:50:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835017; cv=none; b=LghWrKTHYOYQV/+c4G8y0LQL+X+zE6Kwn/YZx/RZhxQbNACaBv35JOPT3UkGVCx0CUMEydB191ddUgnOeb4OBdSABgSTNjJTr5NnfIU3aDGFPmH00r5mHKgqcnPFESZSzUz1myreLoAYZdhm91N4OQwCgHGz6B79bfaBfyTsNkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835017; c=relaxed/simple; bh=FyWxiWn0CsNsynvJoAaJPwfJyM4demAlzCWNG/a+21Q=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Cfa5L4Qv2R2kDKzXmDg27N1TNQLgYcCG5/Jbz06dTjZAgT3Bn2Yjaa1TFSTtOT/wbGBIhyNAZcZ6A2x8/ikmKjdLCJ0Ry2hzxAnjum18kwT/lH1KS7StngH0FnygsRBWcr7WIkWKJ4+vmegB+o5JvS/XUYuHglZI9bt/akrWHqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nd96Fyie; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nd96Fyie" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724835016; x=1756371016; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=FyWxiWn0CsNsynvJoAaJPwfJyM4demAlzCWNG/a+21Q=; b=nd96FyieWY+mUN4N/XmgVLCb50IfF7kmyR2Mi8PdSv8DyBRpyj+EwFLw pawcf/QD/VjwJxcWwvxTH9zhgQnqgPpx1HjveWgHesY6ZEIleM+CQum8i xiYJCPLMvpr3/LhIjxeyT4AttmJi0hJ162ujaAJWGI3g5rnWKWvCF3wZO +73yy3cOfC+K3xDVrskG2XY4eyVJaBrz6DLj9mPIL4E96Hwnm0vaYzYET ZGjkYQtCqvo8LmzJkntcVvoIhXd2xcPBux//85uxds7b6SVTiouc/d+JC gNsjpWhxUMO+OAgCCarWYkUKiicP8JZpganHCerKA2BFhPOGDvA9/79pj A==; X-CSE-ConnectionGUID: ZExlWzX7QuOlu3o1YGOhwg== X-CSE-MsgGUID: QUrenFLSQBO4d1/UjxhWUg== X-IronPort-AV: E=McAfee;i="6700,10204,11177"; a="22874621" X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="22874621" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:16 -0700 X-CSE-ConnectionGUID: yG5npigPTD+1TbrN8mWjVA== X-CSE-MsgGUID: UW4cIURLSGibgHYQ3oGf7w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="62998986" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:08 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v4 1/4] cxl/pci: Fix to record only non-zero ranges Date: Wed, 28 Aug 2024 16:42:28 +0800 Message-Id: <20240828084231.1378789-2-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828084231.1378789-1-yanfei.xu@intel.com> References: <20240828084231.1378789-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The function cxl_dvsec_rr_decode() retrieves and records DVSEC ranges into info->dvsec_range[], regardless of whether it is non-zero range, and the variable info->ranges indicates the number of non-zero ranges. However, in cxl_hdm_decode_init(), the validation for info->dvsec_range[] occurs in a for loop that iterates based on info->ranges. It may result in zero range to be validated but non-zero range not be validated, in turn, the number of allowed ranges is to be 0. Address it by only record non-zero ranges. This fix is not urgent as it requires a configuration that zeroes out the first dvsec range while populating the second. This has not been observed, but it is theoretically possible. If this gets picked up for -stable, no harm done, but there is no urgency to backport. Fixes: 560f78559006 ("cxl/pci: Retrieve CXL DVSEC memory info") Reviewed-by: Jonathan Cameron Signed-off-by: Yanfei Xu Reviewed-by: Alison Schofield --- drivers/cxl/core/pci.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 51132a575b27..73b6498d5e5c 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -390,10 +390,6 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, size |= temp & CXL_DVSEC_MEM_SIZE_LOW_MASK; if (!size) { - info->dvsec_range[i] = (struct range) { - .start = 0, - .end = CXL_RESOURCE_NONE, - }; continue; } @@ -411,12 +407,10 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, base |= temp & CXL_DVSEC_MEM_BASE_LOW_MASK; - info->dvsec_range[i] = (struct range) { + info->dvsec_range[ranges++] = (struct range) { .start = base, .end = base + size - 1 }; - - ranges++; } info->ranges = ranges; From patchwork Wed Aug 28 08:42:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13780922 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB91D15DBDD for ; Wed, 28 Aug 2024 08:50:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835018; cv=none; b=iQuS2C9088P/4U5iVRUbYWRpHUk70RCbaaSOvzjivvlK9i/HtDHkr4s2U47yOkL9xdm8x3an7fmdT+ygQL/LqaLUHEKetpzZnQEt1oLLrpUgrJeC3NMbEei/PS6B0phcOlr8aN3th2uLehPcamrNj0BQG88k2Yw+Sjh/uuaQQOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835018; c=relaxed/simple; bh=43nIiQjTzmSP5bh6pAfz1We2jpQJ+M1wLHmyKywU0+s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EjSlygMYNpaMeukLLJ8YipqCQtekB6mowoGWeAf4nfE7nPH748mxDYoqvvNY8PbikxpRHMQJ0Cb1pE7Yjxz+KvC4S3gRVm6gRcBEhBvUignUIUtVwCQuyYcV3aVjyMdiWW5uJCrvg54NlkdSKGUudW7j/J1NIuIrOd5d1h3HPjk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bqC+k6wb; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bqC+k6wb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724835017; x=1756371017; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=43nIiQjTzmSP5bh6pAfz1We2jpQJ+M1wLHmyKywU0+s=; b=bqC+k6wbmJIU6jDCuQWtIoimctFc2cox4RqtWmsvN5KhQafnOi4wOlK8 w9r06elqFB0n8HGUKBjwUcrItEQkRRsa5mty3OEOEEmy2m428wPBV4d+c 0fhSPPN/gU6EulZ221HiIhgzOoQ6JKVnReNYIwPzAbqLy7GIxSeJRghEo VWL5kRj1zIBoyo95S+SwfRBiExeCa1lC/sIoQx/q8HHdcskbPG6cYhZIY lOH+BfYjCsPmtANWI7nj/OViDHrzx6piQpCryx+NyOyz/lc6djj8mDv5O NO2BD58mvDoVkADi7+7icSx8/efE2jx5gXceDWKuyiWHXxfKnWKfuLK/5 g==; X-CSE-ConnectionGUID: eCe+fg3tRBW43qgOWmgPkA== X-CSE-MsgGUID: UsqSpu1kQh+IgZbFRFDA9w== X-IronPort-AV: E=McAfee;i="6700,10204,11177"; a="22874626" X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="22874626" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:16 -0700 X-CSE-ConnectionGUID: PKbQTGY7SAiB/JpEIryfZg== X-CSE-MsgGUID: aohfmdeFQIaGyEzl6YVdsA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="62999040" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:11 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v4 2/4] cxl/pci: Remove duplicated implementation of waiting for memory_info_valid Date: Wed, 28 Aug 2024 16:42:29 +0800 Message-Id: <20240828084231.1378789-3-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828084231.1378789-1-yanfei.xu@intel.com> References: <20240828084231.1378789-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 commit ce17ad0d5498 ("cxl: Wait Memory_Info_Valid before access memory related info") added another implementation, which is cxl_dvsec_mem_range_valid(), of waiting for memory_info_valid without realizing it duplicated wait_for_valid(). Remove wait_for_valid() and retain cxl_dvsec_mem_range_valid() as the former is hardcoded to check only the Memory_Info_Valid bit of DVSEC range 1, while the latter allows for selection between DVSEC range 1 or 2 via parameter. Suggested-by: Dan Williams Reviewed-by: Jonathan Cameron Signed-off-by: Yanfei Xu Reviewed-by: Alison Schofield --- drivers/cxl/core/pci.c | 41 +++++------------------------------ drivers/cxl/cxl.h | 2 +- drivers/cxl/port.c | 2 +- tools/testing/cxl/test/mock.c | 4 ++-- 4 files changed, 9 insertions(+), 40 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index 73b6498d5e5c..f29af0b788d9 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -211,37 +211,6 @@ int cxl_await_media_ready(struct cxl_dev_state *cxlds) } EXPORT_SYMBOL_NS_GPL(cxl_await_media_ready, CXL); -static int wait_for_valid(struct pci_dev *pdev, int d) -{ - u32 val; - int rc; - - /* - * Memory_Info_Valid: When set, indicates that the CXL Range 1 Size high - * and Size Low registers are valid. Must be set within 1 second of - * deassertion of reset to CXL device. Likely it is already set by the - * time this runs, but otherwise give a 1.5 second timeout in case of - * clock skew. - */ - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - msleep(1500); - - rc = pci_read_config_dword(pdev, d + CXL_DVSEC_RANGE_SIZE_LOW(0), &val); - if (rc) - return rc; - - if (val & CXL_DVSEC_MEM_INFO_VALID) - return 0; - - return -ETIMEDOUT; -} - static int cxl_set_mem_enable(struct cxl_dev_state *cxlds, u16 val) { struct pci_dev *pdev = to_pci_dev(cxlds->dev); @@ -322,11 +291,13 @@ static int devm_cxl_enable_hdm(struct device *host, struct cxl_hdm *cxlhdm) return devm_add_action_or_reset(host, disable_hdm, cxlhdm); } -int cxl_dvsec_rr_decode(struct device *dev, int d, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { struct pci_dev *pdev = to_pci_dev(dev); + struct cxl_dev_state *cxlds = pci_get_drvdata(pdev); int hdm_count, rc, i, ranges = 0; + int d = cxlds->cxl_dvsec; u16 cap, ctrl; if (!d) { @@ -353,11 +324,9 @@ int cxl_dvsec_rr_decode(struct device *dev, int d, if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = wait_for_valid(pdev, d); - if (rc) { - dev_dbg(dev, "Failure awaiting MEM_INFO_VALID (%d)\n", rc); + rc = cxl_dvsec_mem_range_valid(cxlds, 0); + if (rc) return rc; - } /* * The current DVSEC values are moot if the memory capability is diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 9afb407d438f..e2e277463794 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -809,7 +809,7 @@ struct cxl_hdm *devm_cxl_setup_hdm(struct cxl_port *port, int devm_cxl_enumerate_decoders(struct cxl_hdm *cxlhdm, struct cxl_endpoint_dvsec_info *info); int devm_cxl_add_passthrough_decoder(struct cxl_port *port); -int cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info); bool is_cxl_region(struct device *dev); diff --git a/drivers/cxl/port.c b/drivers/cxl/port.c index d7d5d982ce69..861dde65768f 100644 --- a/drivers/cxl/port.c +++ b/drivers/cxl/port.c @@ -98,7 +98,7 @@ static int cxl_endpoint_port_probe(struct cxl_port *port) struct cxl_port *root; int rc; - rc = cxl_dvsec_rr_decode(cxlds->dev, cxlds->cxl_dvsec, &info); + rc = cxl_dvsec_rr_decode(cxlds->dev, port, &info); if (rc < 0) return rc; diff --git a/tools/testing/cxl/test/mock.c b/tools/testing/cxl/test/mock.c index d619672faa49..63a404e05ced 100644 --- a/tools/testing/cxl/test/mock.c +++ b/tools/testing/cxl/test/mock.c @@ -228,7 +228,7 @@ int __wrap_cxl_hdm_decode_init(struct cxl_dev_state *cxlds, } EXPORT_SYMBOL_NS_GPL(__wrap_cxl_hdm_decode_init, CXL); -int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, +int __wrap_cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, struct cxl_endpoint_dvsec_info *info) { int rc = 0, index; @@ -237,7 +237,7 @@ int __wrap_cxl_dvsec_rr_decode(struct device *dev, int dvsec, if (ops && ops->is_mock_dev(dev)) rc = 0; else - rc = cxl_dvsec_rr_decode(dev, dvsec, info); + rc = cxl_dvsec_rr_decode(dev, port, info); put_cxl_mock_ops(index); return rc; From patchwork Wed Aug 28 08:42:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13780923 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC2DB15ECDF for ; Wed, 28 Aug 2024 08:50:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835019; cv=none; b=LQXO9rZ4RSyQK5YBV3yJEvvlemZ1zjxla0RtNUatP1QRtaKYSzq3ZXA03l2iry/+0BTS4wnqH39L1ocTPzVx/eYYOHGVJePJusoa/fdBGYMjVb1hahKZxLytUM8hNe2dhET0ceGIuh/2kColiLwC1JJtvG+Q0xF9N28OwtECo3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835019; c=relaxed/simple; bh=PvEWhAEI/tCdHo06VsYqKztrs2pLHWpr3bJc4P6bHBQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=lCqaQVMDAh3mrn4B+znE0QdVJI9EL6LdZiFlBjVB2Ox0Zov9/CLg0WUsU660++8jQi4NPAZXEzfuo6+Lgp7H67LrCKgtzPatJh8ufXvJtN/Mo11HV0hHVdwi6gG8GzekviFsBNQjbkmwsLPK4VudV+bWRATeilhqoe9tJWAFpw8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=DQrbWULq; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="DQrbWULq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724835018; x=1756371018; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PvEWhAEI/tCdHo06VsYqKztrs2pLHWpr3bJc4P6bHBQ=; b=DQrbWULq/pfu/SDh6qfNgkEdIu5vTVvtIMf/x26LlQf2LXf0Ad9f6TW/ b2lf4pl0kOJ+2ObF853386HYV+mE/FTlAq5ug1VDonYxaT6LjC9glQwEC 2sgBh5Ad6RkNY7kWBSL8JEPMABWpduMifRmZ/lGFKU6KHseopU9FFqjpS c4+/gt6bfCcA0/BiXA7XsA6vlsvyJVSLgeFb27+oyOrdY9zyUkj2zeQPe F5rsJ6AstTrqgV57sMHBxbuDf29xVrmbDm9D2qdKMNzKXrPaZXMy8t1Yx QTXMQwW64q6KwruZLTFxIWBPpBTYhg3CI3PBmyHUyYPJhcX06k/Gn23r8 w==; X-CSE-ConnectionGUID: Pyu7oofyTgKksdE8f/tdKg== X-CSE-MsgGUID: TAMvk1nDTyyiLltTOxh0cQ== X-IronPort-AV: E=McAfee;i="6700,10204,11177"; a="22874632" X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="22874632" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:18 -0700 X-CSE-ConnectionGUID: 4s4WUgIxSESiSXbP6Z1ZIA== X-CSE-MsgGUID: fQKVm0bNQdu2YnfgN4GehA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="62999078" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:14 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v4 3/4] cxl/pci: Check Mem_info_valid bit for each applicable DVSEC Date: Wed, 28 Aug 2024 16:42:30 +0800 Message-Id: <20240828084231.1378789-4-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828084231.1378789-1-yanfei.xu@intel.com> References: <20240828084231.1378789-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In theory a device might set the mem_info_valid bit for a first range after it is ready but before as second range has reached that state. Therefore, the correct approach is to check the Mem_info_valid bit for each applicable DVSEC range against HDM_COUNT, rather than only for the DVSEC range 1. Consequently, let's move the check into the "for loop" that handles each DVSEC range. Reviewed-by: Jonathan Cameron Signed-off-by: Yanfei Xu Reviewed-by: Alison Schofield --- drivers/cxl/core/pci.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index f29af0b788d9..cda22feadbd3 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -324,10 +324,6 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, if (!hdm_count || hdm_count > 2) return -EINVAL; - rc = cxl_dvsec_mem_range_valid(cxlds, 0); - if (rc) - return rc; - /* * The current DVSEC values are moot if the memory capability is * disabled, and they will remain moot after the HDM Decoder @@ -345,6 +341,10 @@ int cxl_dvsec_rr_decode(struct device *dev, struct cxl_port *port, u64 base, size; u32 temp; + rc = cxl_dvsec_mem_range_valid(cxlds, i); + if (rc) + return rc; + rc = pci_read_config_dword( pdev, d + CXL_DVSEC_RANGE_SIZE_HIGH(i), &temp); if (rc) From patchwork Wed Aug 28 08:42:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yanfei Xu X-Patchwork-Id: 13780924 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7F60615AD9C for ; Wed, 28 Aug 2024 08:50:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835027; cv=none; b=Sh/nN4/k6omXBFmved/HxQIRWOEREaBhk25l1aNr3Z1bxY0K5jLso/W1mYyYGihtxtAJ3kJaZGS5985cg1rCkLK8bdeQHD2Kc77QQIdUUGzvJDJ/HKSsp4tGOndBUobc0HS9jeiT+C3Eiw2WIs++V4Rc9jSiCoUQYbjsmj7/OVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724835027; c=relaxed/simple; bh=KK5Gj3jhOLfFJrDZozwHot44Z64Z9TXr0NckbSv0uDM=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=G7lnhRbVCUldLKI1JhGHiStKfHqy3FwLWLfxnEiqSVT3xA1IibPqwOH5yaI1SKIqfJ/l5MwJiDktutlH/n2UoP6l+GRa1DWxDz6QGtvpkgQApHLcdznhr7glcSwK0y1gFEEEtgJLmaReFWyEMj/kx2bQlu77imdOEovD4xALxbE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=TwucQj++; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="TwucQj++" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1724835026; x=1756371026; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KK5Gj3jhOLfFJrDZozwHot44Z64Z9TXr0NckbSv0uDM=; b=TwucQj++Wv9ZCKRWnN9XPHTitwTRITNptZFsq5Ht0OOdZAuO6QFDXHns qreOJA4d3B8k/e6vEgFfUAfV87oUAiIA/Pc9Xn4+OWG38Ot6NKjSah6oM tVp3gibe72VKSeZrZ720NvnKn48hFi2jWzSynpLRBDw9HKDG76RYEgQEk 23YSPDROYwUn6ZIJWkF8mAkgiLmaXFRTq8OX0YWQZbuKYm+/2ldeGW7V6 vcGRQVRfkk7FTNHLiyvh6NZAu2wDTNrAQNHk4ZfkiMCMbYKN9oBjeeaAe p6YqhzWjfluS4z85uS7fCCniJrEuLJJqFMGbNoxfN+67m3hxjdzKAMddu g==; X-CSE-ConnectionGUID: GCHkl5pMRae1taQhOdFXOQ== X-CSE-MsgGUID: FhdBga9fRPepjiutSk4iKQ== X-IronPort-AV: E=McAfee;i="6700,10204,11177"; a="22874638" X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="22874638" Received: from orviesa010.jf.intel.com ([10.64.159.150]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:25 -0700 X-CSE-ConnectionGUID: xZXJ1FP4QuGGbobyLFFUUw== X-CSE-MsgGUID: D8sGPEsISEO0KKB45167eg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.10,182,1719903600"; d="scan'208";a="62999089" Received: from tower.bj.intel.com ([10.238.157.70]) by orviesa010-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Aug 2024 01:50:18 -0700 From: Yanfei Xu To: linux-cxl@vger.kernel.org Cc: dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com, ming4.li@intel.com, yanfei.xu@intel.com Subject: [v4 4/4] cxl/pci: simplify the check of mem_enabled in cxl_hdm_decode_init() Date: Wed, 28 Aug 2024 16:42:31 +0800 Message-Id: <20240828084231.1378789-5-yanfei.xu@intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240828084231.1378789-1-yanfei.xu@intel.com> References: <20240828084231.1378789-1-yanfei.xu@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Cases can be divided into two categories which are DVSEC range enabled and not enabled when HDM decoders exist but is not enabled. To avoid checking info->mem_enabled, which indicates the enablement of DVSEC range, every time, we can check !info->mem_enabled once in advance. This simplification can make the code clearer. No functional change intended. Reviewed-by: Jonathan Cameron Signed-off-by: Yanfei Xu Reviewed-by: Alison Schofield --- drivers/cxl/core/pci.c | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/pci.c b/drivers/cxl/core/pci.c index cda22feadbd3..a3f0e907d08c 100644 --- a/drivers/cxl/core/pci.c +++ b/drivers/cxl/core/pci.c @@ -426,7 +426,15 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, return -ENODEV; } - for (i = 0, allowed = 0; info->mem_enabled && i < info->ranges; i++) { + if (!info->mem_enabled) { + rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); + if (rc) + return rc; + + return devm_cxl_enable_mem(&port->dev, cxlds); + } + + for (i = 0, allowed = 0; i < info->ranges; i++) { struct device *cxld_dev; cxld_dev = device_find_child(&root->dev, &info->dvsec_range[i], @@ -440,7 +448,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, allowed++; } - if (!allowed && info->mem_enabled) { + if (!allowed) { dev_err(dev, "Range register decodes outside platform defined CXL ranges.\n"); return -ENXIO; } @@ -454,14 +462,7 @@ int cxl_hdm_decode_init(struct cxl_dev_state *cxlds, struct cxl_hdm *cxlhdm, * match. If at least one DVSEC range is enabled and allowed, skip HDM * Decoder Capability Enable. */ - if (info->mem_enabled) - return 0; - - rc = devm_cxl_enable_hdm(&port->dev, cxlhdm); - if (rc) - return rc; - - return devm_cxl_enable_mem(&port->dev, cxlds); + return 0; } EXPORT_SYMBOL_NS_GPL(cxl_hdm_decode_init, CXL); From patchwork Tue Sep 3 12:41:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zijun Hu X-Patchwork-Id: 13788663 Received: from pv50p00im-ztdg10021801.me.com (pv50p00im-ztdg10021801.me.com [17.58.6.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A29111D0965 for ; Tue, 3 Sep 2024 12:44:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=17.58.6.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725367448; cv=none; b=X+F8SEPvACKuflUQ0lzeJjdJZy10IzrDvPcgMft+StIkCHywZIMS30Uyh7k1Z1YplmikgxJygRrjl6fIHy8RXy0GmyN4ghRO1754r0B9zTNcteF/v4flpvztLejZGBrO8Kr4aqVjHd0PzDTlFvXqV0pJUZO3Kxvu46xyGSOn9XU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725367448; c=relaxed/simple; bh=NCxLoS7pJBA/A6sZejri9a3QzwC1XvakjDQEyT5H520=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=h0Ac5AnKVIOubbxQf8MklkUf/AB3n6ShTjbbD39K2UswFOOJ85nnxyqaQtu+oNvS1IGYBMzsQdz8I/zOSfxN2DmcVEvdq16dRRQLigYjIAJlrfqU7qP6LoN+X1qaW9PijnDlGlz+2P0cfBZHy4NRx3vyIhgR021P6T3HiVyZicU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=RpQsetTt; arc=none smtp.client-ip=17.58.6.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="RpQsetTt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1725367446; bh=tEfTAfNjlVglE6qtj9e61J5uaCFS+zpjP4F6IfurRvs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To; b=RpQsetTtpYAcAZDJdo5WBAxLsuubUbh2IDwdNMXXN88kD0LbyDCyEwOdSEFZlRZE+ 8vUdzYeglDnnmaBRdYIBj+eGfeDBOe951MCx+SZbJwIoR7/H/VihtKIvqg3NZIAh9G Ok8iXN4hck/ImVBlr3YnRNuxOln/Kcaw+egEtGwC4WpA0jJpc3T/iQtwlq1WlQjqBm jRTmAcOLoSq76MXcH5CwkvG2Y1+NsZ1e+QB6iHbV8fGjVlBuBd2XDJ2ipusSKtL0yT I1stCPSCs2HhWa3CZIezd6R1UIknc45Y1nGIxsSsXnvxPMxx0onU6R6sG+iRqpE2ge kuPtzMPCwWUEw== Received: from [192.168.1.26] (pv50p00im-dlb-asmtp-mailmevip.me.com [17.56.9.10]) by pv50p00im-ztdg10021801.me.com (Postfix) with ESMTPSA id 2DDC220102BD; Tue, 3 Sep 2024 12:43:59 +0000 (UTC) From: Zijun Hu Date: Tue, 03 Sep 2024 20:41:44 +0800 Subject: [PATCH] cxl/region: Fix logic for finding a free cxl decoder Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240903-fix_cxld-v1-1-61acba7198ae@quicinc.com> X-B4-Tracking: v=1; b=H4sIAAcE12YC/6tWKk4tykwtVrJSqFYqSi3LLM7MzwNyDHUUlJIzE vPSU3UzU4B8JSMDIxMDSwNj3bTMivjkipwUXZM0M1Nz00RLAzNDSyWg8oKiVKAc2Kjo2NpaAOd 52OZaAAAA To: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Ira Weiny , Dan Williams , Ben Widawsky Cc: Greg Kroah-Hartman , Zijun Hu , Jonathan Cameron , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Zijun Hu , stable@vger.kernel.org X-Mailer: b4 0.14.1 X-Proofpoint-GUID: hjGKpDrPx5TerylKFjNZHA9BMEHdaUxu X-Proofpoint-ORIG-GUID: hjGKpDrPx5TerylKFjNZHA9BMEHdaUxu X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-02_06,2024-09-03_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 malwarescore=0 clxscore=1011 mlxlogscore=999 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2409030103 X-Apple-Remote-Links: v=1;h=KCk=;charset=UTF-8 From: Zijun Hu match_free_decoder()'s logic for finding a free cxl decoder depends on a prerequisite that all child decoders are sorted by ID in ascending order but the prerequisite may not be guaranteed, fix by finding a free cxl decoder with minimal ID. Fixes: 384e624bb211 ("cxl/region: Attach endpoint decoders") Closes: https://lore.kernel.org/all/cdfc6f98-1aa0-4cb5-bd7d-93256552c39b@icloud.com/ Cc: stable@vger.kernel.org Signed-off-by: Zijun Hu --- drivers/cxl/core/region.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) --- base-commit: 67784a74e258a467225f0e68335df77acd67b7ab change-id: 20240903-fix_cxld-4f6575a90619 Best regards, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21ad5f242875..b9607b4fc40b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -797,21 +797,26 @@ static size_t show_targetN(struct cxl_region *cxlr, char *buf, int pos) static int match_free_decoder(struct device *dev, void *data) { struct cxl_decoder *cxld; - int *id = data; + struct cxl_decoder *target_cxld; + struct device **target_device = data; if (!is_switch_decoder(dev)) return 0; cxld = to_cxl_decoder(dev); - - /* enforce ordered allocation */ - if (cxld->id != *id) + if (cxld->region) return 0; - if (!cxld->region) - return 1; - - (*id)++; + if (!*target_device) { + *target_device = get_device(dev); + return 0; + } + /* enforce ordered allocation */ + target_cxld = to_cxl_decoder(*target_device); + if (cxld->id < target_cxld->id) { + put_device(*target_device); + *target_device = get_device(dev); + } return 0; } @@ -839,8 +844,7 @@ cxl_region_find_decoder(struct cxl_port *port, struct cxl_endpoint_decoder *cxled, struct cxl_region *cxlr) { - struct device *dev; - int id = 0; + struct device *dev = NULL; if (port == cxled_to_port(cxled)) return &cxled->cxld; @@ -849,7 +853,8 @@ cxl_region_find_decoder(struct cxl_port *port, dev = device_find_child(&port->dev, &cxlr->params, match_auto_decoder); else - dev = device_find_child(&port->dev, &id, match_free_decoder); + /* Need to put_device(@dev) after use */ + device_for_each_child(&port->dev, &dev, match_free_decoder); if (!dev) return NULL; /* From patchwork Thu Sep 5 13:02:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zijun Hu X-Patchwork-Id: 13792345 Received: from ci74p00im-qukt09090102.me.com (ci74p00im-qukt09090102.me.com [17.57.156.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5D8FE1993BB for ; Thu, 5 Sep 2024 13:03:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=17.57.156.19 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725541390; cv=none; b=JXT4l48pRCcI//S/CzwsO1rz9Y3Bb7VM6TFMo4guu3iE+dludz3Ssk/RfTwY+ZLf4SCQY0TFluKJrG7ceI4Ytv07K+1Yk83/TiStkBetteOUEMOTs6NOq02EhroitorM81/x4K73bK1k4h39bczcfC5CGEuAIKlvFrcYScuExBI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1725541390; c=relaxed/simple; bh=Gy5mWK6LxwI/0NuTLhsqYHhodN9ViqHz2JElQa0aLvw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=stv7lpe1xG4Qx802bYtwG7qRRyEL3FJZ0kwnNEnngbNStJzM9DBGq0jhTtwGk/N8pCjQGlkc2ztypnoqVxQC8lsbQoe9tVCs83cp+Y1RCtMp5lpWEKCuUXBfSmxA16dJ5jVjdMsaiLpZiOdBNUn4WEdi1BEexLLpSIkGnYdbpqs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com; spf=pass smtp.mailfrom=icloud.com; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b=EdBfqLfJ; arc=none smtp.client-ip=17.57.156.19 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=icloud.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=icloud.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=icloud.com header.i=@icloud.com header.b="EdBfqLfJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=icloud.com; s=1a1hai; t=1725541388; bh=bnoOmXI6avdPPfQGkv4d4OEelImbpwyUa/01maOEGrM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To; b=EdBfqLfJDL/kB4k/t/HptlQdVZBgOM3oGW1YVVvvLhgkxGhtj1REsdr0NRaSUAKXd GxWLulyMYM1q+Mtajp8WgNTczyJzBqG6OO7CYezoBg+OX4ELROWQIv6PkSu6epVwTa cunfnb4J8ArtJgVUpQoXj7mMfcYK91RR9s5spzMSC4H3PGcbkQUgseIpQabyrxuvlg +rQnNUPGsQxahFSo+EEPYTaUKmYHGjrBwn8oCmKfg9AbWQECNy9XHjbuCgNmkTM6it JT2X3ybQ9GNg0+qGoxmSZdZnNKr/+JMX+X83cmJz9ppY3g1f1/c3sKdgKdnL0uHL3f vfRt2RhJZHK3w== Received: from [192.168.1.26] (ci77p00im-dlb-asmtp-mailmevip.me.com [17.57.156.26]) by ci74p00im-qukt09090102.me.com (Postfix) with ESMTPSA id AFAE13C00487; Thu, 5 Sep 2024 13:03:01 +0000 (UTC) From: Zijun Hu Date: Thu, 05 Sep 2024 21:02:27 +0800 Subject: [PATCH v2] cxl/region: Fix bad logic for finding a free switch cxl decoder Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240905-fix_cxld-v2-1-51a520a709e4@quicinc.com> X-B4-Tracking: v=1; b=H4sIAOKr2WYC/22MQQ7CIBBFr9LMWgxgS8WV9zCNwenUTqJUQUlNw 93Frl2+/1/eApECU4RDtUCgxJEnX0BvKsDR+SsJ7guDlrqWVu7EwPMZ51sv6sE0beOsNMpC0R+ ByremTl3hkeNrCp+1nNRv/RNJSihhlMOLa5XdOzo+34zscYvTHbqc8xdaVL+WogAAAA== To: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Ira Weiny , Dan Williams , Ben Widawsky Cc: Greg Kroah-Hartman , Zijun Hu , Jonathan Cameron , linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Zijun Hu , stable@vger.kernel.org X-Mailer: b4 0.14.1 X-Proofpoint-ORIG-GUID: 9Lw1bpLcfH68NP7i-aWjwUNHGk-gs2Gr X-Proofpoint-GUID: 9Lw1bpLcfH68NP7i-aWjwUNHGk-gs2Gr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-05_08,2024-09-04_01,2024-09-02_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxlogscore=999 suspectscore=0 clxscore=1015 mlxscore=0 bulkscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2308100000 definitions=main-2409050096 X-Apple-Remote-Links: v=1;h=KCk=;charset=UTF-8 From: Zijun Hu It is bad for current match_free_decoder()'s logic to find a free switch cxl decoder as explained below: - If all child decoders are sorted by ID in ascending order, then current logic can be simplified as below one: static int match_free_decoder(struct device *dev, void *data) { struct cxl_decoder *cxld; if (!is_switch_decoder(dev)) return 0; cxld = to_cxl_decoder(dev); return cxld->region ? 0 : 1; } dev = device_find_child(&port->dev, NULL, match_free_decoder); which does not also need to modify device_find_child()'s match data. - If all child decoders are NOT sorted by ID in ascending order, then current logic are wrong as explained below: F: free, (cxld->region == NULL) B: busy, (cxld->region != NULL) S(n)F : State of switch cxl_decoder with ID n is Free S(n)B : State of switch cxl_decoder with ID n is Busy Provided there are 2 child decoders: S(1)F -> S(0)B, then current logic will fail to find a free decoder even if there are a free one with ID 1 Anyway, current logic is not good, fixed by finding a free switch cxl decoder with minimal ID. Fixes: 384e624bb211 ("cxl/region: Attach endpoint decoders") Closes: https://lore.kernel.org/all/cdfc6f98-1aa0-4cb5-bd7d-93256552c39b@icloud.com/ Cc: stable@vger.kernel.org Signed-off-by: Zijun Hu --- Changes in v2: - Correct title and commit message - Link to v1: https://lore.kernel.org/r/20240903-fix_cxld-v1-1-61acba7198ae@quicinc.com --- drivers/cxl/core/region.c | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) --- base-commit: 67784a74e258a467225f0e68335df77acd67b7ab change-id: 20240903-fix_cxld-4f6575a90619 Best regards, diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 21ad5f242875..b9607b4fc40b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -797,21 +797,26 @@ static size_t show_targetN(struct cxl_region *cxlr, char *buf, int pos) static int match_free_decoder(struct device *dev, void *data) { struct cxl_decoder *cxld; - int *id = data; + struct cxl_decoder *target_cxld; + struct device **target_device = data; if (!is_switch_decoder(dev)) return 0; cxld = to_cxl_decoder(dev); - - /* enforce ordered allocation */ - if (cxld->id != *id) + if (cxld->region) return 0; - if (!cxld->region) - return 1; - - (*id)++; + if (!*target_device) { + *target_device = get_device(dev); + return 0; + } + /* enforce ordered allocation */ + target_cxld = to_cxl_decoder(*target_device); + if (cxld->id < target_cxld->id) { + put_device(*target_device); + *target_device = get_device(dev); + } return 0; } @@ -839,8 +844,7 @@ cxl_region_find_decoder(struct cxl_port *port, struct cxl_endpoint_decoder *cxled, struct cxl_region *cxlr) { - struct device *dev; - int id = 0; + struct device *dev = NULL; if (port == cxled_to_port(cxled)) return &cxled->cxld; @@ -849,7 +853,8 @@ cxl_region_find_decoder(struct cxl_port *port, dev = device_find_child(&port->dev, &cxlr->params, match_auto_decoder); else - dev = device_find_child(&port->dev, &id, match_free_decoder); + /* Need to put_device(@dev) after use */ + device_for_each_child(&port->dev, &dev, match_free_decoder); if (!dev) return NULL; /* From patchwork Fri Sep 27 16:34:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Cheatham X-Patchwork-Id: 13814467 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2072.outbound.protection.outlook.com [40.107.94.72]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B30F13EFFB; Fri, 27 Sep 2024 16:35:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.72 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727454911; cv=fail; b=RvuaY84nz6B7XtyPCfLModA87KyEOEw4YozQ4vR8YwuvLqPNvDcGkmP6HxbSK6L0A4WOL0U9Lez3xvkAbZBLBIs2hT1FZvDeoyZqpcyDI9eKMKFCfPqZjG0aPTbGKFmB4C9ycHuTSpXYZlVXmMj8BfGDEs/aMyCvoJ5kbx5Wcwo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727454911; c=relaxed/simple; bh=SUo3VDZ1YDV1H4CstGDL/UgKT14H43BbgkqP5DZh7Xw=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=a4mOXagJWnncib8Zubv/RlVoHXWYB1kZHUyvEivqkBu3Xk8k+uZesslG5rNT8150/MTmjQz4+k17PVaHCVM2i8PYvREAEoO4ixTTMAryUY1KPXIQRB5K83lJYqnXhOCcCiRu+wRDzxju7T7EvKDyN+OadZMjo5naB8EhGwSiFgg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=UBoBeA67; arc=fail smtp.client-ip=40.107.94.72 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="UBoBeA67" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vUfe9GqL/9Va4OfzmRA4nkpHowzFy/Sw/vZdm+OgdaZ6VvzWJluhv1DaYppS+4RNvCJMptDLTBxmIJdujnG8uS+4QwN8qAPf/j7SzGFzocOOdF0RImQPqC6k++RDKpWaaSQ2nPNimwxHpwEz98xEKZ2bq1/TgK8H9DWaQbzgvspeUz0N8cHFIzdVw3C8wUgD7oCVNSzTzLDPcoDwMN7HsPQNFyv+TED2xm2SA1rCOL1enuURkxx3nMkw1SEAkPZO3L+n+7KT/JFx9Re7Czh30i4zG4Fz0epIx4dnGpkdZ3R91sD5KvvPs6xMccZ8wX+RD+zPm8MIQe9cf8p+G+byWA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W8uD+WIjRxmP8l7R6rweIawgIgxLgnwyW7Xey0zWdfI=; b=QTqj3fp2GRZZ6IBV2iwfsrKQPTuWR+FZzCXbAMVuHL6Sj2y6927w7MVudyyuTcZPXoKvPbJADiETUV0x3R7ptrFL3DyEjkgqrtBwSjdGoxuWBfD0FmAqlDg/KK4p7chwz8h0UYbSlB8qC0l1WxpXkbvIToEOiURNsoZuX4ujCh52wfWM/fpLF+ythkgwXv6EeA0NWSoQt75lvOu6lzlnWcdk51HSUZ/75gB69XUu2mzHM2wCViRwG9uKdUPyDKFLB1seMt/0Nm3rGwcz8blKdm1WLlXkb8uEk1iXtyAcgGriEuzZPpUkVhBApSSI62JROzqHZyhdV/IWTGbUWu2vag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W8uD+WIjRxmP8l7R6rweIawgIgxLgnwyW7Xey0zWdfI=; b=UBoBeA676z1a9mA3wG1fH4nmHIR4HNlad3mrHh7Pr/8dUEygTWWb7Y1SlTClWUsfmm+JrLUYcAZVkYv4NnkYJRFpmc/jVxyP6S8L6swy28idINfRLJ8wPp+HIw3nJ8u3JArGsF7Oc0Khl8/usZqEvOZ23RskO9VBJNb+bC9z0Ak= Received: from PH7P220CA0153.NAMP220.PROD.OUTLOOK.COM (2603:10b6:510:33b::15) by CY8PR12MB7241.namprd12.prod.outlook.com (2603:10b6:930:5a::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7982.28; Fri, 27 Sep 2024 16:35:05 +0000 Received: from CY4PEPF0000EE3E.namprd03.prod.outlook.com (2603:10b6:510:33b:cafe::3c) by PH7P220CA0153.outlook.office365.com (2603:10b6:510:33b::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8005.24 via Frontend Transport; Fri, 27 Sep 2024 16:35:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by CY4PEPF0000EE3E.mail.protection.outlook.com (10.167.242.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8005.15 via Frontend Transport; Fri, 27 Sep 2024 16:35:04 +0000 Received: from bcheatha-HP-EliteBook-845-G8-Notebook-PC.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Fri, 27 Sep 2024 11:35:01 -0500 From: Ben Cheatham To: , , , , CC: , , , , , Ben Cheatham Subject: [RESEND][PATCH] EINJ, CXL: Fix CXL device SBDF calculation Date: Fri, 27 Sep 2024 11:34:28 -0500 Message-ID: <20240927163428.366557-1-Benjamin.Cheatham@amd.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB04.amd.com (10.181.40.145) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CY4PEPF0000EE3E:EE_|CY8PR12MB7241:EE_ X-MS-Office365-Filtering-Correlation-Id: ad796d7f-e064-4483-6a09-08dcdf12570a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|376014|7416014|82310400026|36860700013; X-Microsoft-Antispam-Message-Info: 6vP9p8mnm+HZxyLW8IX1xXOfKz4rL5F5SIRengd41baeAL7OM7eQrsumjcUTVX7PYvyt+FlDI3ULn89RtYGRNVpSNQl8cBQ4ZXqWH9SlynMoB+y3WLut5zySo2HikcL0ZUkh9Z8AGpBVPgjDXMGAoqXxAFdQQxWiIIQfO7FY+jAFNN96gMfjA0p+gYz+6B6nwaKHH6gaSSC1aN5sLW/FES4fdIgxiKqukZhlyvCE8tzTnpLxK7HMMCWIxkInCMNEvSWK54j73IP8uM138A3ZhhhpSy+mtbUKygzshj2HZxYwXABWKS05jqudlDMIzu5gGhbJ9dpnWmMo9FIDs+njMUqKqHmH6oVQ+8N5Sf5yftYHz/ozBPPlqorDH0HiXbOiJmmKXAzewmD+LzwreFJd4iPGWzXO+Nx6AQhvOIscYr0iG1t2cFXzB6s7sF6PFsCvFLutQzDPfKCpcJMfDkct0h+zX5BhX9Y29uICSN3gKQm7CvfDw4531qu64ebsdPAqjh/JJ9umt1FIe61hDbEeev/+Co3VWneyA76FgJ4hAkrbAW3fKWFh4dWn1j7GPzjiHQoHY8B66TZ3Z/Au/yHYGXmUuGJsnonbV2ANODhA7iEiTWrGr3NnZHkF7bKKN/qjjJK+IhVXaiWxRU0QanrXcsqkuA2ZrPiSz3oAH8aafsaVj6TjUaGbBHO+pnkASYRZCEnPYRBn1M0zNtteoMC80haI9FUJi5lV92mLbwPUJfuRujU1ISXxtPPq2UGUwPvBJo4EXPZ9OtcPaNQ43779G29PKghmfcLmVLKMa/PP59YUx2+q8ZMPV+/V4X6Zt7VPaFdkcDjdE3AGxJLKOTZWypqsaYb7CKgvevNMqymWWaiyuUrmDu6RBozoEetgwL/Duauxx5/z6vnSLjwsPPQWxERDoUGY8VbcXLGYZpiXmlvSHNKkO4H+L1WL66VLtjI7keSUBxR0M7A60f5XUQnv67gnSIEzoQvz2bbU4woEH5KOx1BtPzE/cfoKVMz0FQT+aOX5ta5jXW03owW5u701XxEFG/e9F37Bzr+2Itbtv3BuKY/qar8061snCJAdNv326ww6LG6SAne+RgUsUZNMSrJ5/HacT2FQR0NewwYV/qDmuIc7vwvn0no2igMj35MJ8rIRzF2ev0mR+g49MXwgCUVgQzgFuEYfVYtnawi9LTlULy6GG9Lu8iQm7sMr4/mPXge38KzbxXQTfMfl33TlaIa1iaMauYbPQ0G3TGkpppBPBSu7BlBStmZ1n9z5yEN4WxZBX+XsK1GpEfAd6L2rC26EKLAOEkwktoRo8Z2X51/O4/Mg5Sy7TZxWOwrEoYOIi1exYjOMw761uT7Pna31IzIuP/cC8fi+98lAF+aTQ1BkfLdXNlwil6rP+ei1P9vCU/Yy5OtTQ1qG/AAHRU10RwtGjR4odJrSNZ3+LtkFfKoznUS19jfUA4ToPN5JL8HZ X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(1800799024)(376014)(7416014)(82310400026)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Sep 2024 16:35:04.0598 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: ad796d7f-e064-4483-6a09-08dcdf12570a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: CY4PEPF0000EE3E.namprd03.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7241 The SBDF of the target CXL 2.0 compliant root port is required to inject a CXL protocol error as per ACPI 6.5. The SBDF given has to be in the following format: 31 24 23 16 15 11 10 8 7 0 +-------------------------------------------------+ | segment | bus | device | function | reserved | +-------------------------------------------------+ The SBDF calculated in cxl_dport_get_sbdf() doesn't account for the reserved bits currently, causing the wrong SBDF to be used. Fix said calculation to properly shift the SBDF. Without this fix, error injection into CXL 2.0 root ports through the CXL debugfs interface (/cxl) is broken. Injection through the legacy interface (/apei/einj/) will still work because the SBDF is manually provided by the user. Fixes: 12fb28ea6b1cf ("EINJ: Add CXL error type support") Signed-off-by: Ben Cheatham Reviewed-by: Dan Williams Tested-by: Srinivasulu Thanneeru Reviewed-by: Srinivasulu Thanneeru --- Resent to add linux-cxl list and Srinivasulu to CC --- drivers/acpi/apei/einj-cxl.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/acpi/apei/einj-cxl.c b/drivers/acpi/apei/einj-cxl.c index 8b8be0c90709..d64e2713aae4 100644 --- a/drivers/acpi/apei/einj-cxl.c +++ b/drivers/acpi/apei/einj-cxl.c @@ -63,7 +63,7 @@ static int cxl_dport_get_sbdf(struct pci_dev *dport_dev, u64 *sbdf) seg = bridge->domain_nr; bus = pbus->number; - *sbdf = (seg << 24) | (bus << 16) | dport_dev->devfn; + *sbdf = (seg << 24) | (bus << 16) | (dport_dev->devfn << 8); return 0; } From patchwork Fri Oct 4 21:25:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Gregory Price X-Patchwork-Id: 13823013 X-Patchwork-Delegate: iweiny@gmail.com Received: from mail-qt1-f174.google.com (mail-qt1-f174.google.com [209.85.160.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 994B91AE015 for ; Fri, 4 Oct 2024 21:25:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.160.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728077113; cv=none; b=RcU/sj7lFld/sTa4+QEvQA9kycZFH+Bq6C8eiCl6yvC7by1c8ZBN6EfcQ7sW8JmQaJTeGA7fhUBestHzgmiHpFoFJB/DN6JiR/XC+r+8h99Jplwz07u54ilg1UTuYEdhG0jL7f4nCd4hD7x8JolIui0dsenP7k53a1CRWrC5myo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728077113; c=relaxed/simple; bh=xWcayRwGq99qmIJOCaLa8kUZscX03I82nm2+VL9TA4k=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=goeGWRKCOvV3mObUWdG02AJtibX79baUdBMVSp4tBZS8sYveUTDAzcEcWyriK6FVTflEhhb5wWTZxWKj83smAksWteUJPXEQW8G1Df3xzLg8/58O9roVCH/tR6Q1H08W1NvekpCVj9xVU+hRsjQEphh5e4Tf8DKQL02YDAlbZuE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=IDfNB8Cn; arc=none smtp.client-ip=209.85.160.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="IDfNB8Cn" Received: by mail-qt1-f174.google.com with SMTP id d75a77b69052e-4582c62ee33so24172251cf.3 for ; Fri, 04 Oct 2024 14:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1728077110; x=1728681910; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=c38DjANBRpYnTCiRKoh3ao5Dt3VewIRXoROl0i/M6LY=; b=IDfNB8CnXupAkFroRP2q1wzwSTjZnGwOxs5OJ2CMFPei+0LK60UK5xQl5DeJ28qB53 G8tWDUfs/wiWxaqMFTO3P+aYuCp9a70uH/jZf8mhhvWekBwVKMXmB5yREEtPhbSE/en6 1dmPLkEYqLtGcYy5CLzENqWxmq9ySbtpwXXQazqw5QKEc3Wz59iO0/mqG/Fttl6V2K1p igW4Evr8dMaEJS1MiUzR3V7LkFLEcofOcLQhlUr7Ix9M+Wi2JrckyRSr224rjJfPk33i AABVzB29U31iuGIriZNnYur+J4GxBfIGRsnMTyE87EVrvH/HMb6zickseshsmJUCLOzU TulQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1728077110; x=1728681910; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=c38DjANBRpYnTCiRKoh3ao5Dt3VewIRXoROl0i/M6LY=; b=oO1l9ufYzbbTT7IoUqBowBPYvaFRuO5egIChyvx1IuqtgBGQsbxz7z9A/HsojpxGaG SNdM73Tg951vEZ/BqijLw+jpROWlwADGR6Xb+zBC3R102t//eIeXcCnJAnZZhzjsAp9B Rg50BKSvPH5u0mBmdk0Mfawj7JjwX987ZWTymt+BgU7/iLS5tPzkFE+p05I/YypP4ola FrT2gAYrr1G73y1itOMsg2L1O8h7++RWH+YTqNo5dl8tw0uTihLrLiSBy0+IHI9qHuKq sgZibShA4CUxCQpLWNjNCtlIz2nbmp1+QvGy+U0qVyhlq3vSjy8jPhYt+j2WJDT89D4y ScjA== X-Gm-Message-State: AOJu0YxEMg6nymjAr0BG3+dAiiQw4/GlQVtzAjvZexHbSchaa03fScaq uA+b3IsLeMS3WZhUVNulH3nsWg2dBob16jn2EAMjLgRKhLqEd2lIn6CnT8Hi63dW9l23/0OuD1J R X-Google-Smtp-Source: AGHT+IHiEiOEDSUe+eNwT3Ozk4QbVjqwugZuiizvLvWkgugVkAkEIZArS9gYMMA4Xs2+CzRDpN8KWw== X-Received: by 2002:ac8:7fcb:0:b0:458:40e9:8d0d with SMTP id d75a77b69052e-45d9bb3a6d9mr71503881cf.58.1728077110093; Fri, 04 Oct 2024 14:25:10 -0700 (PDT) Received: from PC2K9PVX.TheFacebook.com (pool-173-79-56-208.washdc.fios.verizon.net. [173.79.56.208]) by smtp.gmail.com with ESMTPSA id d75a77b69052e-45da74ec504sm2660471cf.23.2024.10.04.14.25.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Oct 2024 14:25:09 -0700 (PDT) From: Gregory Price To: linux-cxl@vger.kernel.org Cc: linux-kernel@vger.kernel.org, dan.j.williams@intel.com, Jonathan.Cameron@Huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, rrichter@amd.com, terry.bowman@amd.com, dave@stgolabs.net Subject: [PATCH v2] cxl/core/port: defer endpoint probes when ACPI likely hasn't finished Date: Fri, 4 Oct 2024 17:25:04 -0400 Message-ID: <20241004212504.1246-1-gourry@gourry.net> X-Mailer: git-send-email 2.43.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In cxl_acpi_probe, we add dports and uports to host bridges iteratively: - bus_for_each_dev(adev->dev.bus, NULL, root_port, add_host_bridge_dport); - bus_for_each_dev(adev->dev.bus, NULL, root_port, add_host_bridge_uport); Simultaneously, as ports are probed, memdev endpoints can also be probed. This creates a race condition, where an endpoint can perceive its path to the root being broken in devm_cxl_enumerate_ports. The memdev/endpoint probe will see a heirarchy that may look like: mem1 parent => 0000:c1:00.0 parent => 0000:c0:01.1 parent->parent => NULL This results in find_cxl_port() returning NULL (since the port hasn't been associated with the host bridge yet), and add_port_attach_ep fails because the grandparent's grandparent is NULL. When the latter condition is detected, the comments suggest: /* * The iteration reached the topology root without finding the * CXL-root 'cxl_port' on a previous iteration, fail for now to * be re-probed after platform driver attaches. */ This case results in an -ENXIO; however, a re-probe never occurs. Change this return condition to -EPROBE_DEFER to explicitly cause a reprobe. (v2: additional debug information) Signed-off-by: Gregory Price --- drivers/cxl/core/port.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 1d5007e3795a..d6bebf70d142 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -1553,7 +1553,7 @@ static int add_port_attach_ep(struct cxl_memdev *cxlmd, */ dev_dbg(&cxlmd->dev, "%s is a root dport\n", dev_name(dport_dev)); - return -ENXIO; + return -EPROBE_DEFER; } parent_port = find_cxl_port(dparent, &parent_dport); From patchwork Fri Oct 11 05:34:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13832082 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2802E2F26; Fri, 11 Oct 2024 05:34:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624847; cv=none; b=E7FlMzDEhQtxKtQ+UDVX8fOaODpQcoPnGNiIiOYlZ8Jpnx+pAusvsahH94h5F9uOqtxdEG1mcSnJRyWaf5V68XHi96df08fjH+qEFR3GF1l6kmJrL+73la1egG6j30q2VivPYzJkZxFPJYgJpYQwYjLVBLldsGmNanwSI5BDMoc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624847; c=relaxed/simple; bh=ULtgtOIlkVwYdScG5guVFKkOrJW54pUZLqgsTvXscJU=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=B6X7FHB4r+WaKtIXacXMiYBdWFVc2nIRb125BjNk0k4pzblC5cni1/a2zn6Xi/ND2Rx/gR0956TaUMqlKBaZ5nJf3/5gBRdH7ROziPfbtFqGbOKoovllNKwD0QMzqkFwAeU7a4M6aqGv263FwDQdU6vC4Cb0tfqvOw2rp/xEzRo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=htPjWPNe; arc=none smtp.client-ip=198.175.65.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="htPjWPNe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728624845; x=1760160845; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ULtgtOIlkVwYdScG5guVFKkOrJW54pUZLqgsTvXscJU=; b=htPjWPNe8paVFDcFlbJDXH7PJBuA+HvPLtblQaulMcCn/QeJvJtuZziQ cNh6K4VYVbnQKYdxkqmVD8UnIGVXK2zSFTy2tITMeI4L+8NMIxWfyuL4E qm7ZNjGqZvXhPgY54sx9yavvQCDloJPTTy/zXm74JYZZCMsBEB7ZcfZuQ bJPhHvf3ZoeY5u39Cc8inkobGYiWD5vQbtgTdSZkVov3np0qW4QcxwGlh WMhHEfJAEutwlHag6V0XHDwcg9m/7fZ4Rq/De7N0gRcbCeOZgRpGLM5ty 8PL8bZV3CQFiHDLUiPLn/KeWWlqnd2Bp8O5TqXKR1a8KA7470GzsvJGjx g==; X-CSE-ConnectionGUID: 3vGeTBW3SEKP5qPE/4W8mA== X-CSE-MsgGUID: WZTP8ivQRBSFchGSbpSSLQ== X-IronPort-AV: E=McAfee;i="6700,10204,11221"; a="28124543" X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="28124543" Received: from fmviesa003.fm.intel.com ([10.60.135.143]) by orvoesa109.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:04 -0700 X-CSE-ConnectionGUID: AW9v4LBzTHqtfECgjm8KwA== X-CSE-MsgGUID: LC0t25+gTYybLvJaHATJOg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="80814144" Received: from inaky-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.111.110]) by fmviesa003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:03 -0700 Subject: [PATCH 1/5] cxl/port: Fix CXL port initialization order when the subsystem is built-in From: Dan Williams To: dave.jiang@intel.com, ira.weiny@intel.com Cc: Gregory Price , Gregory Price , stable@vger.kernel.org, Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , linux-cxl@vger.kernel.org Date: Thu, 10 Oct 2024 22:34:02 -0700 Message-ID: <172862484072.2150669.9910214123827630595.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> References: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When the CXL subsystem is built-in the module init order is determined by Makefile order. That order violates expectations. The expectation is that cxl_acpi and cxl_mem can race to attach and that if cxl_acpi wins the race cxl_mem will find the enabled CXL root ports it needs and if cxl_acpi loses the race it will retrigger cxl_mem to attach via cxl_bus_rescan(). That only works if cxl_acpi can assume ports are enabled immediately upone cxl_acpi_probe() return. That in turn can only happen in the CONFIG_CXL_ACPI=y case if the cxl_port object appears before the cxl_acpi object in the Makefile. Fix up the order to prevent initialization failures, and make sure that cxl_port is built-in if cxl_acpi is also built-in. As for what contributed to this not being found earlier, the CXL regression environment, cxl_test, builds all CXL functionality as a module to allow to symbol mocking and other dynamic reload tests. As a result there is no regression coverage for the built-in case. Reported-by: Gregory Price Closes: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net Tested-by: Gregory Price Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Cc: Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Signed-off-by: Dan Williams --- drivers/cxl/Kconfig | 1 + drivers/cxl/Makefile | 12 ++++++------ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 29c192f20082..876469e23f7a 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -60,6 +60,7 @@ config CXL_ACPI default CXL_BUS select ACPI_TABLE_LIB select ACPI_HMAT + select CXL_PORT help Enable support for host managed device memory (HDM) resources published by a platform's ACPI CXL memory layout description. See diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile index db321f48ba52..374829359275 100644 --- a/drivers/cxl/Makefile +++ b/drivers/cxl/Makefile @@ -1,13 +1,13 @@ # SPDX-License-Identifier: GPL-2.0 obj-y += core/ -obj-$(CONFIG_CXL_PCI) += cxl_pci.o -obj-$(CONFIG_CXL_MEM) += cxl_mem.o +obj-$(CONFIG_CXL_PORT) += cxl_port.o obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o +obj-$(CONFIG_CXL_PCI) += cxl_pci.o obj-$(CONFIG_CXL_PMEM) += cxl_pmem.o -obj-$(CONFIG_CXL_PORT) += cxl_port.o +obj-$(CONFIG_CXL_MEM) += cxl_mem.o -cxl_mem-y := mem.o -cxl_pci-y := pci.o +cxl_port-y := port.o cxl_acpi-y := acpi.o +cxl_pci-y := pci.o cxl_pmem-y := pmem.o security.o -cxl_port-y := port.o +cxl_mem-y := mem.o From patchwork Fri Oct 11 05:34:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13832083 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E19E519258C for ; Fri, 11 Oct 2024 05:34:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624854; cv=none; b=f8WgqWT+L8bqNsfFjt2w19Lc1+/kxHgu7pCgQ3LPEOFjfMfjGC6c8l7OGQgwYXM/bBdPKMtx1zaoqB5urAHD1i0v8T3pLlxTcDpUe6+KOjpCvOVJKbuoYvtZb23sfW4T/CkbejJ+xOYleNdI4ZP7mIjIayBAYs3x1I+XD4AiTlo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624854; c=relaxed/simple; bh=6XKBmBurizIDsKL/ClGY3T4qE+YKUGlekxva2IKnRIc=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OUZVCRlz+H1RdJLSQMQlhI3Cr1Q8e2L2s6nBs/gW8Sqsgdfy2Eboe2YVp6pmQxRLaI9Ans5o7ec84oZULSg53QzTeH+R6rSlpLCUoeP++orwRGlKCBcSK+wbfrkoFk3EyGyjUocLAkHupjUjrtUe0dn2Ez9C8sSgMMWxjD817iU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=CD0N1VLx; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="CD0N1VLx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728624853; x=1760160853; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6XKBmBurizIDsKL/ClGY3T4qE+YKUGlekxva2IKnRIc=; b=CD0N1VLxcAKrXa0pFvyrZSii06VFxbBagUrw1L81P7GhXpkQj5niYf1O HfdA9F+DOexs6sqBZXQz9kWA8sjyw5MpU6XFIFrDC1CgEsFaEPozQDa5K uG/Y2O/M++RwaUaLSRDD7wSvARXpMUzrIAFcPJR5LVxgoKK0R9QdSPaAp Hev3RC8mqb0lZwqxH+8FBMVSDqj2d2Gh4htb5t6cn+nOayWqnN8v4ybb5 6TNlqmUD53BwKe08HFyQtWYhZqPacTtZLHXFnj/F54mxEyTA/xw8guoKs OyvxtUe4K8muW65LFtmv5Ka56IIcBxi/43gy8lUWxPEopW3+cBHScU0Bn A==; X-CSE-ConnectionGUID: U5jSKwd5RmOYeUd+g0/QqQ== X-CSE-MsgGUID: xwgAVo1BSoe1+QxIq/78Ww== X-IronPort-AV: E=McAfee;i="6700,10204,11221"; a="53416711" X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="53416711" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:12 -0700 X-CSE-ConnectionGUID: VP0D2cswR+CiBaMxuGznnw== X-CSE-MsgGUID: 9ZdE4PJOSROCERjgd7HXAQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="114262259" Received: from inaky-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.111.110]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:12 -0700 Subject: [PATCH 2/5] cxl/port: Fix cxl_bus_rescan() vs bus_rescan_devices() From: Dan Williams To: dave.jiang@intel.com, ira.weiny@intel.com Cc: vishal.l.verma@intel.com, alison.schofield@intel.com, linux-cxl@vger.kernel.org Date: Thu, 10 Oct 2024 22:34:10 -0700 Message-ID: <172862484920.2150669.7306809902566347902.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> References: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 It turns out since its original introduction, pre-2.6.12, bus_rescan_devices() has skipped devices that might be in the process of attaching or detaching from their driver. For CXL this behavior is unwanted and expects that cxl_bus_rescan() is a probe barrier. That behavior is simple enough to achieve with bus_for_each_dev() paired with call to device_attach(), and it is unclear why bus_rescan_devices() took the positition of lockless consumption of dev->driver which is racy. The "Fixes:" but no "Cc: stable" on this patch reflects that the issue is merely by inspection since the bug that triggered the discovery of this potential problem [1] is fixed by other means. However, a stable backport should do no harm. Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Signed-off-by: Dan Williams --- drivers/cxl/core/port.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e666ec6a9085..b7828b6c7826 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -2084,11 +2084,18 @@ static void cxl_bus_remove(struct device *dev) static struct workqueue_struct *cxl_bus_wq; -static void cxl_bus_rescan_queue(struct work_struct *w) +static int attach_device(struct device *dev, void *data) { - int rc = bus_rescan_devices(&cxl_bus_type); + int rc = device_attach(dev); + + dev_vdbg(dev, "rescan: %s\n", rc ? "attach" : "detached"); - pr_debug("CXL bus rescan result: %d\n", rc); + return 0; +} + +static void cxl_bus_rescan_queue(struct work_struct *w) +{ + bus_for_each_dev(&cxl_bus_type, NULL, NULL, attach_device); } void cxl_bus_rescan(void) From patchwork Fri Oct 11 05:34:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13832084 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33A3C2F26 for ; Fri, 11 Oct 2024 05:34:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624862; cv=none; b=Kc/aXdwIQoegvtnRCjkjeTJrPQ86ByU94xmEz7FbiOnEauFHj+/DxatEV9wRFvoKDWfLaReM1E/KdXQbEGY9hm6NsxMTyKK1RbQ0FgCBemIDQWQLmnA1ezDqXIMMAYWrG0dneexSpm0t+f0YpXsmmoNBm6c69YSi/+/gF/bKMW0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624862; c=relaxed/simple; bh=juOGAtdjznPc0RKrnj5+K0hX4qicmjHRC5I+vDohYYI=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=o7cMBqfwv+KA/DNlkSNrVz+v0Uk26irPxS6T730NCEVbs8jLCYSojrmRPW1U2hvjEpDQO8w/A7f/2XfQ7xFyLKORQVWAhPMJRLn3Lux0C4XDBdBecs8pMmheJqEbB0bQEbbpXwvlU8xJttPyEW4qAj/UJuo8JgVExjlablVWzNE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=lA7sqUve; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="lA7sqUve" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728624861; x=1760160861; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=juOGAtdjznPc0RKrnj5+K0hX4qicmjHRC5I+vDohYYI=; b=lA7sqUveSuBOIGaap59SpLhhD6hCW8t9Zt4yfAI+H1PBhTYa6LVKRkN4 76WE5LLXXhviZFBZLQLLYpe4Fuoy57CO3SZcVqqAWOovnjE3eFvC+jD31 VlqawPaUptITu/XOPsjgbNIXodv0KwrQRZ5YMX6sr2rzUs/IXOAwN3FkF ZkPWgOshc2QtCxla0lhmGVl/QVsnvjipNIu4gATZivVZE9B+W8Mb35AmM PC2Tmn4AG/kyOFzwMwrFUWyVStSRCyVuEhzsLYz92LGxpvbvSc7xVPlDO uRU+J3XyRE1sP0Xxe86ydetMt0PG1ouB5qCJkLrEDwUddAbUkLec7omhR Q==; X-CSE-ConnectionGUID: S1yuStNKQCu+xqZCMAgSPw== X-CSE-MsgGUID: +r5ZGhBmRTiturWSqosMsg== X-IronPort-AV: E=McAfee;i="6700,10204,11221"; a="53416730" X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="53416730" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:20 -0700 X-CSE-ConnectionGUID: GgI7DopJQW2ojZuB3k69Qw== X-CSE-MsgGUID: H1BfrLXMQMGFXnTjujJ+lw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="114262344" Received: from inaky-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.111.110]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:20 -0700 Subject: [PATCH 3/5] cxl/acpi: Ensure ports ready at cxl_acpi_probe() return From: Dan Williams To: dave.jiang@intel.com, ira.weiny@intel.com Cc: vishal.l.verma@intel.com, alison.schofield@intel.com, linux-cxl@vger.kernel.org Date: Thu, 10 Oct 2024 22:34:18 -0700 Message-ID: <172862485724.2150669.12670806990969973843.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> References: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In order to ensure root CXL ports are enabled upon cxl_acpi_probe() when the 'cxl_port' driver is built as a module, arrange for the module to be pre-loaded or built-in. The "Fixes:" but no "Cc: stable" on this patch reflects that the issue is merely by inspection since the bug that triggered the discovery of this potential problem [1] is fixed by other means. However, a stable backport should do no harm. Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Signed-off-by: Dan Williams --- drivers/cxl/acpi.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 82b78e331d8e..432b7cfd12a8 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -924,6 +924,13 @@ static void __exit cxl_acpi_exit(void) /* load before dax_hmem sees 'Soft Reserved' CXL ranges */ subsys_initcall(cxl_acpi_init); + +/* + * Arrange for host-bridge ports to be active synchronous with + * cxl_acpi_probe() exit. + */ +MODULE_SOFTDEP("pre: cxl_port"); + module_exit(cxl_acpi_exit); MODULE_DESCRIPTION("CXL ACPI: Platform Support"); MODULE_LICENSE("GPL v2"); From patchwork Fri Oct 11 05:34:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13832085 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAEEF2F26; Fri, 11 Oct 2024 05:34:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.7 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624871; cv=none; b=OxIF59WEe7MO4ieBfGi4XVVl/XFjmB/td41fyRLdbkvUyzNcJBh4ZehEjP9wmO5M5H8PQZqEDkgnoe+o1NLV6lmZg8LL4ElcpOEPUV9nZm4+ISeTwy7z7eq/fgneLgW9cL+vhR5ocY4WMw6X1KQb4es7ybtQcIRzzsCm4ctKp28= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624871; c=relaxed/simple; bh=obtioSTRwwRcBv5j9xguOwlGpsr+Bb/BJZ8PlRGyjvI=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=nVgX/9z/cLlacH/McveSMZdwN7NHRz1QeRmOyAjuFzKWQqc+EiR6z8Bca4i05yzN1QMi/sYh/eOPb1/0EhW5N9MOKkmcQBk/pzJFg3Gm1FSJ2jsGbHNkOR4fyz2EDtdym+ZYFNS48csv8LctdnVoj9KdM9+nhfGOWFXPZOE5SeU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=clI8qarG; arc=none smtp.client-ip=192.198.163.7 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="clI8qarG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728624869; x=1760160869; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=obtioSTRwwRcBv5j9xguOwlGpsr+Bb/BJZ8PlRGyjvI=; b=clI8qarGzdkGAVF8sK7+r9KEeTL1P1Ew6GWZEjzFJCYAGk3NBBDpEqBG 5R9fazGja54Elc0eldG59sfRY70b2XAQD2q1QT7xHwngL8ceWbQScrEVN XS1CKxzL67+D2gjQ3ty7/LnZy6jK/dfUzypz5gYx1ELQkLj5qO5wfdWQO pGz9lKlENjgOvxbtZBocTGgMHZOpmYQzxdzaRb9odxaE1fEn18FREiYRX xY7D+ZKS1JL1eiz+lP3ubxOnc1kOSSylaVHKFRZl7E4s8HHv7Su797ZA6 m3eMiB0AY4lq7eJSpdGd4Gayxh5s1GkA0jkdZYGU4CEcNvDnvXKui/R1g A==; X-CSE-ConnectionGUID: zlXdb/kCSlqzX/qV+dOXpg== X-CSE-MsgGUID: 7XO47e79SvCo6WoSST2aqA== X-IronPort-AV: E=McAfee;i="6700,10204,11221"; a="53416739" X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="53416739" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:29 -0700 X-CSE-ConnectionGUID: sR/lD/9QThOYEuzw1k7Pfw== X-CSE-MsgGUID: JsrE8DygRrei8KcvRSSP4g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="114262380" Received: from inaky-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.111.110]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:29 -0700 Subject: [PATCH 4/5] cxl/port: Fix use-after-free, permit out-of-order decoder shutdown From: Dan Williams To: dave.jiang@intel.com, ira.weiny@intel.com Cc: stable@vger.kernel.org, Greg Kroah-Hartman , Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Zijun Hu , vishal.l.verma@intel.com, linux-cxl@vger.kernel.org Date: Thu, 10 Oct 2024 22:34:26 -0700 Message-ID: <172862486548.2150669.3548553804904171839.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> References: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In support of investigating an initialization failure report [1], cxl_test was updated to register mock memory-devices after the mock root-port/bus device had been registered. That led to cxl_test crashing with a use-after-free bug with the following signature: cxl_port_attach_region: cxl region3: cxl_host_bridge.0:port3 decoder3.0 add: mem0:decoder7.0 @ 0 next: cxl_switch_uport.0 nr_eps: 1 nr_targets: 1 cxl_port_attach_region: cxl region3: cxl_host_bridge.0:port3 decoder3.0 add: mem4:decoder14.0 @ 1 next: cxl_switch_uport.0 nr_eps: 2 nr_targets: 1 cxl_port_setup_targets: cxl region3: cxl_switch_uport.0:port6 target[0] = cxl_switch_dport.0 for mem0:decoder7.0 @ 0 1) cxl_port_setup_targets: cxl region3: cxl_switch_uport.0:port6 target[1] = cxl_switch_dport.4 for mem4:decoder14.0 @ 1 [..] cxld_unregister: cxl decoder14.0: cxl_region_decode_reset: cxl_region region3: mock_decoder_reset: cxl_port port3: decoder3.0 reset 2) mock_decoder_reset: cxl_port port3: decoder3.0: out of order reset, expected decoder3.1 cxl_endpoint_decoder_release: cxl decoder14.0: [..] cxld_unregister: cxl decoder7.0: 3) cxl_region_decode_reset: cxl_region region3: Oops: general protection fault, probably for non-canonical address 0x6b6b6b6b6b6b6bc3: 0000 [#1] PREEMPT SMP PTI [..] RIP: 0010:to_cxl_port+0x8/0x60 [cxl_core] [..] Call Trace: cxl_region_decode_reset+0x69/0x190 [cxl_core] cxl_region_detach+0xe8/0x210 [cxl_core] cxl_decoder_kill_region+0x27/0x40 [cxl_core] cxld_unregister+0x5d/0x60 [cxl_core] At 1) a region has been established with 2 endpoint decoders (7.0 and 14.0). Those endpoints share a common switch-decoder in the topology (3.0). At teardown, 2), decoder14.0 is the first to be removed and hits the "out of order reset case" in the switch decoder. The effect though is that region3 cleanup is aborted leaving it in-tact and referencing decoder14.0. At 3) the second attempt to teardown region3 trips over the stale decoder14.0 object which has long since been deleted. The fix here is to recognize that the CXL specification places no mandate on in-order shutdown of switch-decoders, the driver enforces in-order allocation, and hardware enforces in-order commit. So, rather than fail and leave objects dangling, always remove them. In support of making cxl_region_decode_reset() always succeed, cxl_region_invalidate_memregion() failures are turned into warnings. Crashing the kernel is ok there since system integrity is at risk if caches cannot be managed around physical address mutation events like CXL region destruction. A new device_for_each_child_reverse_from() is added to cleanup port->commit_end after all dependent decoders have been disabled. In other words if decoders are allocated 0->1->2 and disabled 1->2->0 then port->commit_end only decrements from 2 after 2 has been disabled, and it decrements all the way to zero since 1 was disabled previously. Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Cc: Fixes: 176baefb2eb5 ("cxl/hdm: Commit decoder state to hardware") Cc: Greg Kroah-Hartman Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Ira Weiny Cc: Zijun Hu Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron --- drivers/base/core.c | 35 +++++++++++++++++++++++++++++ drivers/cxl/core/hdm.c | 50 +++++++++++++++++++++++++++++++++++------- drivers/cxl/core/region.c | 48 +++++++++++----------------------------- drivers/cxl/cxl.h | 3 ++- include/linux/device.h | 3 +++ tools/testing/cxl/test/cxl.c | 14 ++++-------- 6 files changed, 100 insertions(+), 53 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index a4c853411a6b..e42f1ad73078 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4037,6 +4037,41 @@ int device_for_each_child_reverse(struct device *parent, void *data, } EXPORT_SYMBOL_GPL(device_for_each_child_reverse); +/** + * device_for_each_child_reverse_from - device child iterator in reversed order. + * @parent: parent struct device. + * @from: optional starting point in child list + * @fn: function to be called for each device. + * @data: data for the callback. + * + * Iterate over @parent's child devices, starting at @from, and call @fn + * for each, passing it @data. This helper is identical to + * device_for_each_child_reverse() when @from is NULL. + * + * @fn is checked each iteration. If it returns anything other than 0, + * iteration stop and that value is returned to the caller of + * device_for_each_child_reverse_from(); + */ +int device_for_each_child_reverse_from(struct device *parent, + struct device *from, const void *data, + int (*fn)(struct device *, const void *)) +{ + struct klist_iter i; + struct device *child; + int error = 0; + + if (!parent->p) + return 0; + + klist_iter_init_node(&parent->p->klist_children, &i, + (from ? &from->p->knode_parent : NULL)); + while ((child = prev_device(&i)) && !error) + error = fn(child, data); + klist_iter_exit(&i); + return error; +} +EXPORT_SYMBOL_GPL(device_for_each_child_reverse_from); + /** * device_find_child - device iterator for locating a particular device. * @parent: parent struct device diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..223c273c0cd1 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -712,7 +712,44 @@ static int cxl_decoder_commit(struct cxl_decoder *cxld) return 0; } -static int cxl_decoder_reset(struct cxl_decoder *cxld) +static int commit_reap(struct device *dev, const void *data) +{ + struct cxl_port *port = to_cxl_port(dev->parent); + struct cxl_decoder *cxld; + + if (!is_switch_decoder(dev) && !is_endpoint_decoder(dev)) + return 0; + + cxld = to_cxl_decoder(dev); + if (port->commit_end == cxld->id && + ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) { + port->commit_end--; + dev_dbg(&port->dev, "reap: %s commit_end: %d\n", + dev_name(&cxld->dev), port->commit_end); + } + + return 0; +} + +void cxl_port_commit_reap(struct cxl_decoder *cxld) +{ + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + + lockdep_assert_held_write(&cxl_region_rwsem); + + /* + * Once the highest committed decoder is disabled, free any other + * decoders that were pinned allocated by out-of-order release. + */ + port->commit_end--; + dev_dbg(&port->dev, "reap: %s commit_end: %d\n", dev_name(&cxld->dev), + port->commit_end); + device_for_each_child_reverse_from(&port->dev, &cxld->dev, NULL, + commit_reap); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_commit_reap, CXL); + +static void cxl_decoder_reset(struct cxl_decoder *cxld) { struct cxl_port *port = to_cxl_port(cxld->dev.parent); struct cxl_hdm *cxlhdm = dev_get_drvdata(&port->dev); @@ -721,14 +758,14 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) u32 ctrl; if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) - return 0; + return; - if (port->commit_end != id) { + if (port->commit_end == id) + cxl_port_commit_reap(cxld); + else dev_dbg(&port->dev, "%s: out of order reset, expected decoder%d.%d\n", dev_name(&cxld->dev), port->id, port->commit_end); - return -EBUSY; - } down_read(&cxl_dpa_rwsem); ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(id)); @@ -741,7 +778,6 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) writel(0, hdm + CXL_HDM_DECODER0_BASE_LOW_OFFSET(id)); up_read(&cxl_dpa_rwsem); - port->commit_end--; cxld->flags &= ~CXL_DECODER_F_ENABLE; /* Userspace is now responsible for reconfiguring this decoder */ @@ -751,8 +787,6 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) cxled = to_cxl_endpoint_decoder(&cxld->dev); cxled->state = CXL_DECODER_STATE_MANUAL; } - - return 0; } static int cxl_setup_hdm_decoder_from_dvsec( diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e701e4b04032..3478d2058303 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -232,8 +232,8 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) "Bypassing cpu_cache_invalidate_memregion() for testing!\n"); return 0; } else { - dev_err(&cxlr->dev, - "Failed to synchronize CPU cache state\n"); + dev_WARN(&cxlr->dev, + "Failed to synchronize CPU cache state\n"); return -ENXIO; } } @@ -242,19 +242,17 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) return 0; } -static int cxl_region_decode_reset(struct cxl_region *cxlr, int count) +static void cxl_region_decode_reset(struct cxl_region *cxlr, int count) { struct cxl_region_params *p = &cxlr->params; - int i, rc = 0; + int i; /* - * Before region teardown attempt to flush, and if the flush - * fails cancel the region teardown for data consistency - * concerns + * Before region teardown attempt to flush, evict any data cached for + * this region, or scream loudly about missing arch / platform support + * for CXL teardown. */ - rc = cxl_region_invalidate_memregion(cxlr); - if (rc) - return rc; + cxl_region_invalidate_memregion(cxlr); for (i = count - 1; i >= 0; i--) { struct cxl_endpoint_decoder *cxled = p->targets[i]; @@ -277,23 +275,17 @@ static int cxl_region_decode_reset(struct cxl_region *cxlr, int count) cxl_rr = cxl_rr_load(iter, cxlr); cxld = cxl_rr->decoder; if (cxld->reset) - rc = cxld->reset(cxld); - if (rc) - return rc; + cxld->reset(cxld); set_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); } endpoint_reset: - rc = cxled->cxld.reset(&cxled->cxld); - if (rc) - return rc; + cxled->cxld.reset(&cxled->cxld); set_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); } /* all decoders associated with this region have been torn down */ clear_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); - - return 0; } static int commit_decoder(struct cxl_decoder *cxld) @@ -409,16 +401,8 @@ static ssize_t commit_store(struct device *dev, struct device_attribute *attr, * still pending. */ if (p->state == CXL_CONFIG_RESET_PENDING) { - rc = cxl_region_decode_reset(cxlr, p->interleave_ways); - /* - * Revert to committed since there may still be active - * decoders associated with this region, or move forward - * to active to mark the reset successful - */ - if (rc) - p->state = CXL_CONFIG_COMMIT; - else - p->state = CXL_CONFIG_ACTIVE; + cxl_region_decode_reset(cxlr, p->interleave_ways); + p->state = CXL_CONFIG_ACTIVE; } } @@ -2054,13 +2038,7 @@ static int cxl_region_detach(struct cxl_endpoint_decoder *cxled) get_device(&cxlr->dev); if (p->state > CXL_CONFIG_ACTIVE) { - /* - * TODO: tear down all impacted regions if a device is - * removed out of order - */ - rc = cxl_region_decode_reset(cxlr, p->interleave_ways); - if (rc) - goto out; + cxl_region_decode_reset(cxlr, p->interleave_ways); p->state = CXL_CONFIG_ACTIVE; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0d8b810a51f0..5406e3ab3d4a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -359,7 +359,7 @@ struct cxl_decoder { struct cxl_region *region; unsigned long flags; int (*commit)(struct cxl_decoder *cxld); - int (*reset)(struct cxl_decoder *cxld); + void (*reset)(struct cxl_decoder *cxld); }; /* @@ -730,6 +730,7 @@ static inline bool is_cxl_root(struct cxl_port *port) int cxl_num_decoders_committed(struct cxl_port *port); bool is_cxl_port(const struct device *dev); struct cxl_port *to_cxl_port(const struct device *dev); +void cxl_port_commit_reap(struct cxl_decoder *cxld); struct pci_bus; int devm_cxl_register_pci_bus(struct device *host, struct device *uport_dev, struct pci_bus *bus); diff --git a/include/linux/device.h b/include/linux/device.h index b4bde8d22697..667cb6db9019 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -1078,6 +1078,9 @@ int device_for_each_child(struct device *dev, void *data, int (*fn)(struct device *dev, void *data)); int device_for_each_child_reverse(struct device *dev, void *data, int (*fn)(struct device *dev, void *data)); +int device_for_each_child_reverse_from(struct device *parent, + struct device *from, const void *data, + int (*fn)(struct device *, const void *)); struct device *device_find_child(struct device *dev, void *data, int (*match)(struct device *dev, void *data)); struct device *device_find_child_by_name(struct device *parent, diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 90d5afd52dd0..c5bbd89b3192 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -693,26 +693,22 @@ static int mock_decoder_commit(struct cxl_decoder *cxld) return 0; } -static int mock_decoder_reset(struct cxl_decoder *cxld) +static void mock_decoder_reset(struct cxl_decoder *cxld) { struct cxl_port *port = to_cxl_port(cxld->dev.parent); int id = cxld->id; if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) - return 0; + return; dev_dbg(&port->dev, "%s reset\n", dev_name(&cxld->dev)); - if (port->commit_end != id) { + if (port->commit_end == id) + cxl_port_commit_reap(cxld); + else dev_dbg(&port->dev, "%s: out of order reset, expected decoder%d.%d\n", dev_name(&cxld->dev), port->id, port->commit_end); - return -EBUSY; - } - - port->commit_end--; cxld->flags &= ~CXL_DECODER_F_ENABLE; - - return 0; } static void default_mock_decoder(struct cxl_decoder *cxld) From patchwork Fri Oct 11 05:34:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13832086 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91B4F2F26 for ; Fri, 11 Oct 2024 05:34:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624879; cv=none; b=rdYm8BVxzQ5X6EPs3xLSk4FSluF/mmRKizWX9U6WRHwodh9Hgt1tIwlRB3aZjECGH7IOcMUgRSehr09ZCxuSCPfAbWcyhPHot8Eq7ULjx4YOM12YL0L2LRJkdt/XbyFEkYHKCAtH7+A6D+RYiTyptIRM0/2uSU2sUNBx/K8Knds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728624879; c=relaxed/simple; bh=8+1Oxg2KD3Wc6rz5kN6jZmUPo8IGhAIV8pkcpyE4JhM=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bqkBNc3HGcXsNXZg+g/Q3/siEseM6eLkE8TKe4N1UN2fXoHGCHkGBuUTsdic4h7hZr3NiHIVfbxnkRTCB8NsLdJOFgKFhkRJbJvtM6QWV00mWi1FfIo8ZZpupYKjvxae1OSzOPbjyhEDVi2MNjfipzblyMjk9eO9meNKUJ3K+xQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=iKXJoD3V; arc=none smtp.client-ip=198.175.65.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="iKXJoD3V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1728624878; x=1760160878; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8+1Oxg2KD3Wc6rz5kN6jZmUPo8IGhAIV8pkcpyE4JhM=; b=iKXJoD3VYO8p+7Uhn6EEXsVwSaJ0tv1hszkupU1UYWc7fAOs23ITim7y 39Hu2MnoRSInAqExh1g4IyjX9Awl9INPHH9iof2NeHmz6fm7IyTOf295o +NJzE0wR++UeHkhcNXHPU3vovVdq32P9Ad84yR+98lXMWcg+suFubmOAy iw3Z10rqK5UZKL6bQw+GBcNNa//mg4xR8oZe27e8rSFrIUO/ba1lBfOM1 ZU0aQSIDOYFhDI1M562bbKQWV/yWJV7Lm2zHLXe9se844R5UdmWaywt0e Uut8WP0zNs3NoKzp/qupDpphdVoL+7EQKYWZ5+RpOFasgZgCVJipzxbXQ A==; X-CSE-ConnectionGUID: rEjsM0YdTeKoGkATdw196A== X-CSE-MsgGUID: g38dApfETDKI8atNiEqH7w== X-IronPort-AV: E=McAfee;i="6700,10204,11221"; a="28145550" X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="28145550" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa110.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:37 -0700 X-CSE-ConnectionGUID: l8LqQW78SdCQhamZTN7F+A== X-CSE-MsgGUID: uK8SRVV0Rs+t7V6zKqDdQw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,195,1725346800"; d="scan'208";a="77018399" Received: from inaky-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.111.110]) by orviesa006-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Oct 2024 22:34:37 -0700 Subject: [PATCH 5/5] cxl/test: Improve init-order fidelity relative to real-world systems From: Dan Williams To: dave.jiang@intel.com, ira.weiny@intel.com Cc: Davidlohr Bueso , Jonathan Cameron , Alison Schofield , Vishal Verma , linux-cxl@vger.kernel.org Date: Thu, 10 Oct 2024 22:34:35 -0700 Message-ID: <172862487394.2150669.13859971851815282473.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> References: <172862483180.2150669.5564474284074502692.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The investigation of an initialization failure [1] highlighted that cxl_test does not reflect the init-order of real world systems. The expected order is root/bus first then async probing of the memory devices. Fix up cxl_test to reflect that order. While it did not reproduce the initial bug report (since that is dependent on built-in vs modular builds), it did reveal a separate latent bug in the subsystem's decoder shutdown flow. Fix for that sent separately. Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Signed-off-by: Dan Williams --- tools/testing/cxl/test/cxl.c | 186 +++++++++++++++++++++++------------------- tools/testing/cxl/test/mem.c | 1 2 files changed, 104 insertions(+), 83 deletions(-) diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index c5bbd89b3192..050725afa45d 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -1058,7 +1058,7 @@ static void mock_companion(struct acpi_device *adev, struct device *dev) #define SZ_64G (SZ_32G * 2) #endif -static __init int cxl_rch_init(void) +static __init int cxl_rch_topo_init(void) { int rc, i; @@ -1086,30 +1086,8 @@ static __init int cxl_rch_init(void) goto err_bridge; } - for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) { - int idx = NR_MEM_MULTI + NR_MEM_SINGLE + i; - struct platform_device *rch = cxl_rch[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_rcd", idx); - if (!pdev) - goto err_mem; - pdev->dev.parent = &rch->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_rcd[i] = pdev; - } - return 0; -err_mem: - for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) - platform_device_unregister(cxl_rcd[i]); err_bridge: for (i = ARRAY_SIZE(cxl_rch) - 1; i >= 0; i--) { struct platform_device *pdev = cxl_rch[i]; @@ -1123,12 +1101,10 @@ static __init int cxl_rch_init(void) return rc; } -static void cxl_rch_exit(void) +static void cxl_rch_topo_exit(void) { int i; - for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) - platform_device_unregister(cxl_rcd[i]); for (i = ARRAY_SIZE(cxl_rch) - 1; i >= 0; i--) { struct platform_device *pdev = cxl_rch[i]; @@ -1139,7 +1115,7 @@ static void cxl_rch_exit(void) } } -static __init int cxl_single_init(void) +static __init int cxl_single_topo_init(void) { int i, rc; @@ -1224,29 +1200,8 @@ static __init int cxl_single_init(void) cxl_swd_single[i] = pdev; } - for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) { - struct platform_device *dport = cxl_swd_single[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_mem", NR_MEM_MULTI + i); - if (!pdev) - goto err_mem; - pdev->dev.parent = &dport->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_mem_single[i] = pdev; - } - return 0; -err_mem: - for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem_single[i]); err_dport: for (i = ARRAY_SIZE(cxl_swd_single) - 1; i >= 0; i--) platform_device_unregister(cxl_swd_single[i]); @@ -1269,12 +1224,10 @@ static __init int cxl_single_init(void) return rc; } -static void cxl_single_exit(void) +static void cxl_single_topo_exit(void) { int i; - for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem_single[i]); for (i = ARRAY_SIZE(cxl_swd_single) - 1; i >= 0; i--) platform_device_unregister(cxl_swd_single[i]); for (i = ARRAY_SIZE(cxl_swu_single) - 1; i >= 0; i--) @@ -1291,6 +1244,91 @@ static void cxl_single_exit(void) } } +static void cxl_mem_exit(void) +{ + int i; + + for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) + platform_device_unregister(cxl_rcd[i]); + for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem_single[i]); + for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem[i]); +} + +static int cxl_mem_init(void) +{ + int i, rc; + + for (i = 0; i < ARRAY_SIZE(cxl_mem); i++) { + struct platform_device *dport = cxl_switch_dport[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_mem", i); + if (!pdev) + goto err_mem; + pdev->dev.parent = &dport->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_mem; + } + cxl_mem[i] = pdev; + } + + for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) { + struct platform_device *dport = cxl_swd_single[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_mem", NR_MEM_MULTI + i); + if (!pdev) + goto err_single; + pdev->dev.parent = &dport->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_single; + } + cxl_mem_single[i] = pdev; + } + + for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) { + int idx = NR_MEM_MULTI + NR_MEM_SINGLE + i; + struct platform_device *rch = cxl_rch[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_rcd", idx); + if (!pdev) + goto err_rcd; + pdev->dev.parent = &rch->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_rcd; + } + cxl_rcd[i] = pdev; + } + + return 0; + +err_rcd: + for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) + platform_device_unregister(cxl_rcd[i]); +err_single: + for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem_single[i]); +err_mem: + for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem[i]); + return rc; +} + static __init int cxl_test_init(void) { int rc, i; @@ -1403,29 +1441,11 @@ static __init int cxl_test_init(void) cxl_switch_dport[i] = pdev; } - for (i = 0; i < ARRAY_SIZE(cxl_mem); i++) { - struct platform_device *dport = cxl_switch_dport[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_mem", i); - if (!pdev) - goto err_mem; - pdev->dev.parent = &dport->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_mem[i] = pdev; - } - - rc = cxl_single_init(); + rc = cxl_single_topo_init(); if (rc) - goto err_mem; + goto err_dport; - rc = cxl_rch_init(); + rc = cxl_rch_topo_init(); if (rc) goto err_single; @@ -1438,19 +1458,20 @@ static __init int cxl_test_init(void) rc = platform_device_add(cxl_acpi); if (rc) - goto err_add; + goto err_root; + + rc = cxl_mem_init(); + if (rc) + goto err_root; return 0; -err_add: +err_root: platform_device_put(cxl_acpi); err_rch: - cxl_rch_exit(); + cxl_rch_topo_exit(); err_single: - cxl_single_exit(); -err_mem: - for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem[i]); + cxl_single_topo_exit(); err_dport: for (i = ARRAY_SIZE(cxl_switch_dport) - 1; i >= 0; i--) platform_device_unregister(cxl_switch_dport[i]); @@ -1482,11 +1503,10 @@ static __exit void cxl_test_exit(void) { int i; + cxl_mem_exit(); platform_device_unregister(cxl_acpi); - cxl_rch_exit(); - cxl_single_exit(); - for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem[i]); + cxl_rch_topo_exit(); + cxl_single_topo_exit(); for (i = ARRAY_SIZE(cxl_switch_dport) - 1; i >= 0; i--) platform_device_unregister(cxl_switch_dport[i]); for (i = ARRAY_SIZE(cxl_switch_uport) - 1; i >= 0; i--) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index ad5c4c18c5c6..71916e0e1546 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1673,6 +1673,7 @@ static struct platform_driver cxl_mock_mem_driver = { .name = KBUILD_MODNAME, .dev_groups = cxl_mock_mem_groups, .groups = cxl_mock_mem_core_groups, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, }; From patchwork Mon Oct 14 14:30:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shiju Jose X-Patchwork-Id: 13835122 Received: from frasgout.his.huawei.com (frasgout.his.huawei.com [185.176.79.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA7AF39FCE for ; Mon, 14 Oct 2024 14:30:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=185.176.79.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728916226; cv=none; b=KZngorXnT4rMiQbjT9ONcaspYamFYORGHNwTaCfy6h6fIQ4ogjYGy6aeKEkODK9nIbyEAFk0c/JhDZqBopKZ0XtjbGiCSldSGYsvP67oPNcJS1pTEUZwJq7RnJHC/JOzF+ywTUXWTr1v0UeLw2h2bElv1VGyVCpRWHagiTa1mzE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728916226; c=relaxed/simple; bh=sHXb0A2rz2Q0pap06AsBCI5FpEJMOHD0sCY3CLZKQJk=; h=From:To:CC:Subject:Date:Message-ID:MIME-Version:Content-Type; b=dyvCzw6U34vjO7tPMP6BJTu+5Kl8Aes5jf0c+WREyjn55gUVXDoAthiEkR8AbxagrQMMKJ7PMHH3vlZPi3U3M71IS9QAkVzFJo0Pkz0jold7C9mEb25C814JFzUZk3ANbBj9djdd6HJTy2QIyqQQbxv9EvaWqd1lLeT/5UwBsxU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com; spf=pass smtp.mailfrom=huawei.com; arc=none smtp.client-ip=185.176.79.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=huawei.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=huawei.com Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4XRzzj2kfbz6LC0x; Mon, 14 Oct 2024 22:25:53 +0800 (CST) Received: from frapeml500007.china.huawei.com (unknown [7.182.85.172]) by mail.maildlp.com (Postfix) with ESMTPS id 9D721140AB8; Mon, 14 Oct 2024 22:30:20 +0800 (CST) Received: from P_UKIT01-A7bmah.china.huawei.com (10.126.169.100) by frapeml500007.china.huawei.com (7.182.85.172) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.39; Mon, 14 Oct 2024 16:30:19 +0200 From: To: , , , , , , , CC: , , , Subject: [PATCH 1/1] cxl/events: Fix Trace DRAM Event Record Date: Mon, 14 Oct 2024 15:30:03 +0100 Message-ID: <20241014143003.1170-1-shiju.jose@huawei.com> X-Mailer: git-send-email 2.43.0.windows.1 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: lhrpeml500005.china.huawei.com (7.191.163.240) To frapeml500007.china.huawei.com (7.182.85.172) From: Shiju Jose CXL spec rev 3.0 section 8.2.9.2.1.2 defines the DRAM Event Record. Fix decode memory event type field of DRAM Event Record. For e.g. if value is 0x1 it will be reported as an Invalid Address (General Media Event Record - Memory Event Type) instead of Scrub Media ECC Error (DRAM Event Record - Memory Event Type) and so on. Fixes: 2d6c1e6d60ba ("cxl/mem: Trace DRAM Event Record") Signed-off-by: Shiju Jose Reviewed-by: Ira Weiny --- Updates for event records in CXL spec r3.1 will follow shortly. drivers/cxl/core/trace.h | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/core/trace.h b/drivers/cxl/core/trace.h index 9167cfba7f59..cdffebcf20a4 100644 --- a/drivers/cxl/core/trace.h +++ b/drivers/cxl/core/trace.h @@ -279,7 +279,7 @@ TRACE_EVENT(cxl_generic_event, #define CXL_GMER_MEM_EVT_TYPE_ECC_ERROR 0x00 #define CXL_GMER_MEM_EVT_TYPE_INV_ADDR 0x01 #define CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR 0x02 -#define show_mem_event_type(type) __print_symbolic(type, \ +#define show_gmer_mem_event_type(type) __print_symbolic(type, \ { CXL_GMER_MEM_EVT_TYPE_ECC_ERROR, "ECC Error" }, \ { CXL_GMER_MEM_EVT_TYPE_INV_ADDR, "Invalid Address" }, \ { CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR, "Data Path Error" } \ @@ -373,7 +373,7 @@ TRACE_EVENT(cxl_general_media, "hpa=%llx region=%s region_uuid=%pUb", __entry->dpa, show_dpa_flags(__entry->dpa_flags), show_event_desc_flags(__entry->descriptor), - show_mem_event_type(__entry->type), + show_gmer_mem_event_type(__entry->type), show_trans_type(__entry->transaction_type), __entry->channel, __entry->rank, __entry->device, __print_hex(__entry->comp_id, CXL_EVENT_GEN_MED_COMP_ID_SIZE), @@ -391,6 +391,17 @@ TRACE_EVENT(cxl_general_media, * DRAM Event Record defines many fields the same as the General Media Event * Record. Reuse those definitions as appropriate. */ +#define CXL_DER_MEM_EVT_TYPE_ECC_ERROR 0x00 +#define CXL_DER_MEM_EVT_TYPE_SCRUB_MEDIA_ECC_ERROR 0x01 +#define CXL_DER_MEM_EVT_TYPE_INV_ADDR 0x02 +#define CXL_DER_MEM_EVT_TYPE_DATA_PATH_ERROR 0x03 +#define show_dram_mem_event_type(type) __print_symbolic(type, \ + { CXL_DER_MEM_EVT_TYPE_ECC_ERROR, "ECC Error" }, \ + { CXL_DER_MEM_EVT_TYPE_SCRUB_MEDIA_ECC_ERROR, "Scrub Media ECC Error" }, \ + { CXL_DER_MEM_EVT_TYPE_INV_ADDR, "Invalid Address" }, \ + { CXL_DER_MEM_EVT_TYPE_DATA_PATH_ERROR, "Data Path Error" } \ +) + #define CXL_DER_VALID_CHANNEL BIT(0) #define CXL_DER_VALID_RANK BIT(1) #define CXL_DER_VALID_NIBBLE BIT(2) @@ -477,7 +488,7 @@ TRACE_EVENT(cxl_dram, "hpa=%llx region=%s region_uuid=%pUb", __entry->dpa, show_dpa_flags(__entry->dpa_flags), show_event_desc_flags(__entry->descriptor), - show_mem_event_type(__entry->type), + show_dram_mem_event_type(__entry->type), show_trans_type(__entry->transaction_type), __entry->channel, __entry->rank, __entry->nibble_mask, __entry->bank_group, __entry->bank, From patchwork Wed Oct 16 01:52:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhijian Li (Fujitsu)" X-Patchwork-Id: 13837611 Received: from esa7.hc1455-7.c3s2.iphmx.com (esa7.hc1455-7.c3s2.iphmx.com [139.138.61.252]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A152769D2B; Wed, 16 Oct 2024 01:51:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=139.138.61.252 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729043508; cv=none; b=LlA3gmb1e97IDN73mMmyXMXnU14Eb8IsIqz/UDzzhdQLzSpFjBsBd0FdsHBmz5pF64ZcrW+fyvCy4w5emjtzRtAmTigGgAy/YO0wWrTd1w7wZWav2zHBVU82si2rd5BoyXdqql9ToBcs8m+TqUZdFhspu2zsyE0SiiAD2XuZG30= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729043508; c=relaxed/simple; bh=j7qOHC4J4CEpVnKIhfVxPI4BkVnPiOW+nm/eX7OdYm0=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=KA23yGc2v2zDrtTPNh0UyC+tvph8HH2rT19uBIe//AlodY202/DcDeSLlRYj0C37Yi2wqIqIz/Oo1wTRP9nj78yR/jLsn7PXhrAWy/co4W5WapNxDCl21BS8o+42fb6A0SzXkg10zsJIILUafOnjy40MP8Ijv1QHtAd7kF3swqE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fujitsu.com; spf=pass smtp.mailfrom=fujitsu.com; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b=KTddlpuV; arc=none smtp.client-ip=139.138.61.252 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=fujitsu.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=fujitsu.com header.i=@fujitsu.com header.b="KTddlpuV" DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj2; t=1729043505; x=1760579505; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=j7qOHC4J4CEpVnKIhfVxPI4BkVnPiOW+nm/eX7OdYm0=; b=KTddlpuV/siw5xGxuzTUoOql3XGK/yyxGO0p7Kr/MEmUZcFzy1PlmUXZ J4OkITbIv8x9MA2xXIS1fhX8wK0EuGilom6PGJO5YJ95Rhc0oU3va4UQm 23cYxDcBqgM7Wvdb9aUqOZ/+BkGM7HUPs8+86Fj6xOEg4CUOhGzaHqK1e YiWLy3TDvW4sNObbPKfKFTPDR9WFDyoFM0JXv1hKi02hdNEU8KNsX4GrT Y8bVwfXdUJvAu8TNvZjukiPzhI9dVjaFhz8Yfa3wqtOHAIgxV0YE3Sg53 BADpY3fm4Ymc3eWDgBxGz95+qgRNZAzJ7MX7XsfA10RxkAjfhyXJRFw19 w==; X-CSE-ConnectionGUID: 3dIjEyeQRSaq3ES0LbST+g== X-CSE-MsgGUID: fCigllwgQ16FCJpFk2qUNQ== X-IronPort-AV: E=McAfee;i="6700,10204,11225"; a="155759722" X-IronPort-AV: E=Sophos;i="6.11,206,1725289200"; d="scan'208";a="155759722" Received: from unknown (HELO oym-r4.gw.nic.fujitsu.com) ([210.162.30.92]) by esa7.hc1455-7.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Oct 2024 10:51:34 +0900 Received: from oym-m1.gw.nic.fujitsu.com (oym-nat-oym-m1.gw.nic.fujitsu.com [192.168.87.58]) by oym-r4.gw.nic.fujitsu.com (Postfix) with ESMTP id E792DD800A; Wed, 16 Oct 2024 10:51:32 +0900 (JST) Received: from kws-ab3.gw.nic.fujitsu.com (kws-ab3.gw.nic.fujitsu.com [192.51.206.21]) by oym-m1.gw.nic.fujitsu.com (Postfix) with ESMTP id BBC5ED8CB4; Wed, 16 Oct 2024 10:51:31 +0900 (JST) Received: from edo.cn.fujitsu.com (edo.cn.fujitsu.com [10.167.33.5]) by kws-ab3.gw.nic.fujitsu.com (Postfix) with ESMTP id 5320B20071A2B; Wed, 16 Oct 2024 10:51:31 +0900 (JST) Received: from iaas-rdma.. (unknown [10.167.135.44]) by edo.cn.fujitsu.com (Postfix) with ESMTP id 729841A000B; Wed, 16 Oct 2024 09:51:30 +0800 (CST) From: Li Zhijian To: linux-cxl@vger.kernel.org Cc: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , Ira Weiny , Dan Williams , linux-kernel@vger.kernel.org, Li Zhijian , "Huang, Ying" Subject: [PATCH] testing/cxl: Fix abused pci_bus_read_config_word() on platform device Date: Wed, 16 Oct 2024 09:52:13 +0800 Message-ID: <20241016015213.900985-1-lizhijian@fujitsu.com> X-Mailer: git-send-email 2.44.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-TM-AS-Product-Ver: IMSS-9.1.0.1417-9.0.0.1002-28734.003 X-TM-AS-User-Approved-Sender: Yes X-TMASE-Version: IMSS-9.1.0.1417-9.0.1002-28734.003 X-TMASE-Result: 10--9.738800-10.000000 X-TMASE-MatchedRID: 5tAkVIfDYR6/CtrsiqyIUhFbgtHjUWLya9qiaDSLgo3j7F/ald2q5Nxp athSJH+2A9H5R5f2ce3Xcf4B0rHCc7+wzU3ImWPrkCThXPqsqivSde/CNbaZJdALOWoT1h/Un+n rvjaMDYE07flFHfXJs3UofCUICGvnPetSk5bB6twCOoDG5aR1MW+twLxyosqhuSp7W+BawKAfg4 t/uEEkXM+7+sxizLzN8e8JBtC0n8uzVFZ6osYjDoL5ja7E+OhyV9/n5IZng2Q6FHRWx2FGsI3ce RXYSJoDIvrftAIhWmLy9zcRSkKatQ719kpOO37PCtzGvPCy/m5SuvtBzlaEqFH3rkvzbWPM4Ibz 3CH3vCeZL0XMB8MktZ2oLZ8u2T3EHxPMjOKY7A9t1O49r1VEa8RB0bsfrpPIfiAqrjYtFiTnZwI PZIlPj4+nYgHbS6oneIvBCuZpVZJyl8yGHuEJkH7cGd19dSFd X-TMASE-SNAP-Result: 1.821001.0001-0-1-22:0,33:0,34:0-0 The cxl_region_shared_upstream_bandwidth_update() in clx_core works on PCI/PCIe CXL device only while cxl_test was implemeneted by platform device. Mock a cxl_region_shared_upstream_bandwidth_update() which does nothing for cxl_core so that the cxl_test goes well. Abuse cxl_region_shared_upstream_bandwidth_update() on platform device will cause a kernel panic with calltrace: platform cxl_host_bridge.3: host supports CXL (restricted) Oops: general protection fault, probably for non-canonical address 0x3ef17856fcae4fbd: 0000 [#1] PREEMPT SMP PTI CPU: 1 UID: 0 PID: 9167 Comm: cxl Kdump: loaded Tainted: G OE 6.12.0-rc3-master+ #66 Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE Hardware name: LENOVO 90CXCTO1WW/, BIOS FCKT70AUS 04/23/2015 RIP: 0010:pci_bus_read_config_word+0x1c/0x60 Code: 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 53 b8 87 00 00 00 48 83 ec 08 c7 44 24 04 00 00 00 00 f6 c2 01 75 29 <48> 8b 87 c0 00 00 00 48 89 cb 4c 8d 44 24 04 b9 02 00 00 00 48 8b RSP: 0018:ffffa115034dfbb8 EFLAGS: 00010246 RAX: 0000000000000087 RBX: 0000000000000012 RCX: ffffa115034dfbfe RDX: 0000000000000016 RSI: 000000006f4e2f4e RDI: 3ef17856fcae4efd RBP: ffff8cc229121b48 R08: 0000000000000010 R09: 0000000000000000 R10: 0000000000000001 R11: ffff8cc225434360 R12: ffffa115034dfbfe R13: 0000000000000000 R14: ffff8cc2f119a080 R15: ffffa115034dfc50 FS: 00007f31d93537c0(0000) GS:ffff8cc510a80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 00007f31d95f3370 CR3: 00000001163ea001 CR4: 00000000001726f0 Call Trace: ? __die_body.cold+0x19/0x27 ? die_addr+0x38/0x60 ? exc_general_protection+0x1f5/0x4b0 ? asm_exc_general_protection+0x22/0x30 ? pci_bus_read_config_word+0x1c/0x60 pcie_capability_read_word+0x93/0xb0 pcie_link_speed_mbps+0x18/0x50 cxl_pci_get_bandwidth+0x18/0x60 [cxl_core] cxl_endpoint_gather_bandwidth.constprop.0+0xf4/0x230 [cxl_core] ? xas_store+0x54/0x660 ? preempt_count_add+0x69/0xa0 ? _raw_spin_lock+0x13/0x40 ? __kmalloc_cache_noprof+0xe7/0x270 cxl_region_shared_upstream_bandwidth_update+0x9c/0x790 [cxl_core] cxl_region_attach+0x520/0x7e0 [cxl_core] store_targetN+0xf2/0x120 [cxl_core] kernfs_fop_write_iter+0x13a/0x1f0 vfs_write+0x23b/0x410 ksys_write+0x53/0xd0 do_syscall_64+0x62/0x180 entry_SYSCALL_64_after_hwframe+0x76/0x7e And Ying also reported a KASAN error with similar calltrace. Reported-by: "Huang, Ying" Closes: https://lore.kernel.org/linux-cxl/87y12w9vp5.fsf@yhuang6-desk2.ccr.corp.intel.com/ Fixes: a5ab0de0ebaa ("cxl: Calculate region bandwidth of targets with shared upstream link") Signed-off-by: Li Zhijian Tested-by: "Huang, Ying" Reviewed-by: Alison Schofield --- tools/testing/cxl/Kbuild | 2 ++ tools/testing/cxl/mock_cdat.c | 8 ++++++++ 2 files changed, 10 insertions(+) create mode 100644 tools/testing/cxl/mock_cdat.c diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index b1256fee3567..ed9f50dee3f5 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -15,6 +15,7 @@ ldflags-y += --wrap=devm_cxl_add_rch_dport ldflags-y += --wrap=cxl_rcd_component_reg_phys ldflags-y += --wrap=cxl_endpoint_parse_cdat ldflags-y += --wrap=cxl_dport_init_ras_reporting +ldflags-y += --wrap=cxl_region_shared_upstream_bandwidth_update DRIVERS := ../../../drivers CXL_SRC := $(DRIVERS)/cxl @@ -61,6 +62,7 @@ cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o +cxl_core-y += mock_cdat.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o cxl_core-y += config_check.o diff --git a/tools/testing/cxl/mock_cdat.c b/tools/testing/cxl/mock_cdat.c new file mode 100644 index 000000000000..99974153b3f6 --- /dev/null +++ b/tools/testing/cxl/mock_cdat.c @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright (c) 2024 FUJITSU LIMITED. All rights reserved. */ + +#include + +void __wrap_cxl_region_shared_upstream_bandwidth_update(struct cxl_region *cxlr) +{ +} From patchwork Wed Oct 23 01:43:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846300 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C79EA20323; Wed, 23 Oct 2024 01:43:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647808; cv=none; b=HTnl4AWwJ0qs+KzjbOZPdVtLqSa3xg61Fch5+IYJjC8n1USJ5oBwJZWM+zr3KzdIuWU5vyrMzizsG9kui9VqobUvOY5IGzi2wqSfKN9e09h6p7t1nVMQrJN7fCoALWmVfzGEMbYgVo92cZguTJHzfCIi7hd2NLUBfxCadMSNAhQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647808; c=relaxed/simple; bh=hRlt6zWXN+JO1BCxg9TJx3kZIWg+qY5u+t8CECAw79I=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=mUwWZgEq6eGZLCAnRmoA3vzqTCgCOCmU+Dm7E0t7UwqKh5OBY422Aa15BIWGle1IO55Tg0koJQJVxWXwEtin2IYSN0ZjVGoOZ6l7nNQtLGbGQcD0n+c8bShflYE6eS4yvG5gcO/JYxO/7eX40xh0Ffa+FAUrcBIrYHYx2awMtAE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ejr6m//b; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ejr6m//b" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647807; x=1761183807; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hRlt6zWXN+JO1BCxg9TJx3kZIWg+qY5u+t8CECAw79I=; b=ejr6m//bC9VdH/UX3jy62fotKCym8DXrzOnQ9xRN5X+8MvpMimCJJNY7 ogp+Di7jnz9TdZMn2jcLDoDYEQpl6pRwUbujkdrjcmUcjnRVjJkAKrWj0 y7zBIy465obwiyGZemcQr+bkLXCxWGBs1A8go7uLufQeieTYO1Pu6O8E2 a7L/ckav8IojmiyE5dcz1YJ/W39JJsiaWZpJJDWYz5kokqPBy+q2vmcaR kzO/KBrTDuJvya/36LziPEcMPiPDw4tWuf18qQ5sa+hyky7EMshEHAGho B/9Yg2OqjmAb7UliyV0isjQ9xA+Z1/xuvx6hwLIZENJDlWUaIRedLE+q+ A==; X-CSE-ConnectionGUID: lz3YJNCcTxq9Nq2SgTMJZw== X-CSE-MsgGUID: NU8jjOWvS7WtHQmKIGFJyA== X-IronPort-AV: E=McAfee;i="6700,10204,11233"; a="28658048" X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="28658048" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:26 -0700 X-CSE-ConnectionGUID: i2Z/HCKzQLOZxygwkIZ+3Q== X-CSE-MsgGUID: mWDpW2H9QVCnl0wFzaZ8ew== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="84630701" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:25 -0700 Subject: [PATCH v2 1/6] cxl/port: Fix CXL port initialization order when the subsystem is built-in From: Dan Williams To: ira.weiny@intel.com Cc: Gregory Price , Gregory Price , stable@vger.kernel.org, Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:43:24 -0700 Message-ID: <172964780249.81806.11601867702278939388.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When the CXL subsystem is built-in the module init order is determined by Makefile order. That order violates expectations. The expectation is that cxl_acpi and cxl_mem can race to attach and that if cxl_acpi wins the race cxl_mem will find the enabled CXL root ports it needs and if cxl_acpi loses the race it will retrigger cxl_mem to attach via cxl_bus_rescan(). That only works if cxl_acpi can assume ports are enabled immediately upon cxl_acpi_probe() return. That in turn can only happen in the CONFIG_CXL_ACPI=y case if the cxl_port object appears before the cxl_acpi object in the Makefile. Fix up the order to prevent initialization failures, and make sure that cxl_port is built-in if cxl_acpi is also built-in. As for what contributed to this not being found earlier, the CXL regression environment, cxl_test, builds all CXL functionality as a module to allow to symbol mocking and other dynamic reload tests. As a result there is no regression coverage for the built-in case. Reported-by: Gregory Price Closes: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net Tested-by: Gregory Price Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Cc: Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny Tested-by: Alejandro Lucero Reviewed-by: Alejandro Lucero --- drivers/cxl/Kconfig | 1 + drivers/cxl/Makefile | 20 ++++++++++++++------ 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 29c192f20082..876469e23f7a 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -60,6 +60,7 @@ config CXL_ACPI default CXL_BUS select ACPI_TABLE_LIB select ACPI_HMAT + select CXL_PORT help Enable support for host managed device memory (HDM) resources published by a platform's ACPI CXL memory layout description. See diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile index db321f48ba52..2caa90fa4bf2 100644 --- a/drivers/cxl/Makefile +++ b/drivers/cxl/Makefile @@ -1,13 +1,21 @@ # SPDX-License-Identifier: GPL-2.0 + +# Order is important here for the built-in case: +# - 'core' first for fundamental init +# - 'port' before platform root drivers like 'acpi' so that CXL-root ports +# are immediately enabled +# - 'mem' and 'pmem' before endpoint drivers so that memdevs are +# immediately enabled +# - 'pci' last, also mirrors the hardware enumeration hierarchy obj-y += core/ -obj-$(CONFIG_CXL_PCI) += cxl_pci.o -obj-$(CONFIG_CXL_MEM) += cxl_mem.o +obj-$(CONFIG_CXL_PORT) += cxl_port.o obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o obj-$(CONFIG_CXL_PMEM) += cxl_pmem.o -obj-$(CONFIG_CXL_PORT) += cxl_port.o +obj-$(CONFIG_CXL_MEM) += cxl_mem.o +obj-$(CONFIG_CXL_PCI) += cxl_pci.o -cxl_mem-y := mem.o -cxl_pci-y := pci.o +cxl_port-y := port.o cxl_acpi-y := acpi.o cxl_pmem-y := pmem.o security.o -cxl_port-y := port.o +cxl_mem-y := mem.o +cxl_pci-y := pci.o From patchwork Wed Oct 23 01:43:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846301 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84FE820323 for ; Wed, 23 Oct 2024 01:43:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647817; cv=none; b=Wi0u8RSNDDcl9l36RYSkXXr2xWRbWIXUwptcZ/OQmwNkkvxkjiOsIC1O81+hwaQb7ztUhN2vCYVI+3RzV2+pat2MoeNCF0uIfwKaz15Fe5gb3IxMaN7hFrDGMvQ7fb8i8GVnN8yi1nrAzLyivFBPQKJi9zoVCQWoee4Q4JJp/Zo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647817; c=relaxed/simple; bh=dwNJEPj6jHE8CJs/VGrofccHyDyGMrVYwMK3E8SxFxY=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=j9gCHEKagMlB22LiRBnrDr4RK82X6J7n/HLAVC2091A+bRx6XkdWthdEW+OemazdZ/mQyEYtUTOoslFjpJHczBsi+LCvXTz3ptOgxrBw2CB/QrZeOmecAEgKEyxdQvlK1nGJQ6lHvauHfukVHiDxYLjNsun2YP5H/8N9f8ORakc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Sb6UDXEk; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Sb6UDXEk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647815; x=1761183815; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dwNJEPj6jHE8CJs/VGrofccHyDyGMrVYwMK3E8SxFxY=; b=Sb6UDXEkxHZNmva6JK0IUhHb9U6ZmImNJ1nD4Q5nhp1/bBrCP4h8q9IK DjbDyw70ERUCJThuChXcf2xSAYPc9tH2V3U0zoY3QlKmQN0/up1Tmi9Kh 8wo7RvY3tmY1H9X1bldr0X/i8zdFcHMem4Q2PczysSVL5Gtoexv9Z06f3 6StnArzhlkzjW39Oa8NdRIhLDYLmLgfZ8ZJqJWNWfywGOci9OfuieXEG/ S6NAfcH9Z7P3tFmkF8W8M8+/7XRvmVANzjp9QrI0SEl+novIrDDYq1mkF USSKX1YXATOjhYG+zjXLXA4t0x2zzSySfllelOXbMmfobvtZ16AaQyXMg w==; X-CSE-ConnectionGUID: 3tLA65WnQHudjTrBz1rEuQ== X-CSE-MsgGUID: PZkkgdrsS1S7Vgy0oxS3rw== X-IronPort-AV: E=McAfee;i="6700,10204,11233"; a="28658053" X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="28658053" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:34 -0700 X-CSE-ConnectionGUID: A5sJ9CMKQKaDfFLJv4akpg== X-CSE-MsgGUID: XKNHBFwMQs2mMbQSOZKAag== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="84630734" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:34 -0700 Subject: [PATCH v2 2/6] cxl/port: Fix cxl_bus_rescan() vs bus_rescan_devices() From: Dan Williams To: ira.weiny@intel.com Cc: vishal.l.verma@intel.com, alison.schofield@intel.com, dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:43:32 -0700 Message-ID: <172964781104.81806.4277549800082443769.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 It turns out since its original introduction, pre-2.6.12, bus_rescan_devices() has skipped devices that might be in the process of attaching or detaching from their driver. For CXL this behavior is unwanted and expects that cxl_bus_rescan() is a probe barrier. That behavior is simple enough to achieve with bus_for_each_dev() paired with call to device_attach(), and it is unclear why bus_rescan_devices() took the position of lockless consumption of dev->driver which is racy. The "Fixes:" but no "Cc: stable" on this patch reflects that the issue is merely by inspection since the bug that triggered the discovery of this potential problem [1] is fixed by other means. However, a stable backport should do no harm. Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Signed-off-by: Dan Williams Tested-by: Gregory Price Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny --- drivers/cxl/core/port.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index e666ec6a9085..af92c67bc954 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -2084,11 +2084,18 @@ static void cxl_bus_remove(struct device *dev) static struct workqueue_struct *cxl_bus_wq; -static void cxl_bus_rescan_queue(struct work_struct *w) +static int cxl_rescan_attach(struct device *dev, void *data) { - int rc = bus_rescan_devices(&cxl_bus_type); + int rc = device_attach(dev); + + dev_vdbg(dev, "rescan: %s\n", rc ? "attach" : "detached"); - pr_debug("CXL bus rescan result: %d\n", rc); + return 0; +} + +static void cxl_bus_rescan_queue(struct work_struct *w) +{ + bus_for_each_dev(&cxl_bus_type, NULL, NULL, cxl_rescan_attach); } void cxl_bus_rescan(void) From patchwork Wed Oct 23 01:43:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846302 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E18520323 for ; Wed, 23 Oct 2024 01:43:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647825; cv=none; b=GGmnfoz16SisvFg2bHpWeS03bkEh9bbvqycsnjgwFZ7PLb98YaSTYtd6pstRPYumRlcqH+88XBr+yMU75J8ocmzMNdsS/tBcxjIc4LjUkTj11ZTO5Ai5qKLr/C4B4tr4Sb+247UaOuv9mz3O5KZAgaTBydfkKVQr596eHk4nnCY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647825; c=relaxed/simple; bh=juOGAtdjznPc0RKrnj5+K0hX4qicmjHRC5I+vDohYYI=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=itc/TKVLtJj1SgHnCRz7rL/zD7Np/BdO/8YnSb4SS29Z80JEY6xCdbpmlB5acz/lRlGBVkepLOdcEexWcx0Gct9XKiYXQDbhQ/HHQdVIzQff0qe6E6RehGQGw1T7GijsW4eFNhi8TE1kRVY5CszGg0KE1hkq5Sel09m+Pkkxg0U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YaKOaXKk; arc=none smtp.client-ip=192.198.163.18 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YaKOaXKk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647824; x=1761183824; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=juOGAtdjznPc0RKrnj5+K0hX4qicmjHRC5I+vDohYYI=; b=YaKOaXKkL4FCh74strGq9dr+KGCQgsM+zBJCzh2ozC8HzpLxYQ221gee pGI9Ncbi8kcWxhnuhkLWGj8wvhQIa2tPiOQQ2B6J/4/T5LMAwkbS25pGN GzmzKYO3xFSzYg/eLrlYfNAIgU5DjgKCZa3SfEeruZY48nKYP+iaTurDT ue3ldbLlYO47MECWGTih02dHUEozEU1RAzUKtfBZ870rlTW0vahtOaPpv hK73HMDBsgJ8yAREW4ywXmP6iCYv1i/cX4IrSQMuiKQSj5DMo2wgmkct7 jaqKkfep/yGfvZO7gAuaxUN4+85yDlnEQjY2+VHR5T/LYYrtCBQS/az+d g==; X-CSE-ConnectionGUID: o8LpFVF9Sn+fwxqEVBnehQ== X-CSE-MsgGUID: IiqqZbeXSR+qhiEXxpy3lQ== X-IronPort-AV: E=McAfee;i="6700,10204,11233"; a="28658057" X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="28658057" Received: from fmviesa005.fm.intel.com ([10.60.135.145]) by fmvoesa112.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:43 -0700 X-CSE-ConnectionGUID: sh9FYzkmT2Gp3DCdZhSqFg== X-CSE-MsgGUID: DM8MvJYvS++44cn4ZY8GWA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="84630744" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by fmviesa005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:42 -0700 Subject: [PATCH v2 3/6] cxl/acpi: Ensure ports ready at cxl_acpi_probe() return From: Dan Williams To: ira.weiny@intel.com Cc: vishal.l.verma@intel.com, alison.schofield@intel.com, dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:43:40 -0700 Message-ID: <172964781969.81806.17276352414854540808.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In order to ensure root CXL ports are enabled upon cxl_acpi_probe() when the 'cxl_port' driver is built as a module, arrange for the module to be pre-loaded or built-in. The "Fixes:" but no "Cc: stable" on this patch reflects that the issue is merely by inspection since the bug that triggered the discovery of this potential problem [1] is fixed by other means. However, a stable backport should do no harm. Fixes: 8dd2bc0f8e02 ("cxl/mem: Add the cxl_mem driver") Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Signed-off-by: Dan Williams Tested-by: Gregory Price Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny --- drivers/cxl/acpi.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index 82b78e331d8e..432b7cfd12a8 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -924,6 +924,13 @@ static void __exit cxl_acpi_exit(void) /* load before dax_hmem sees 'Soft Reserved' CXL ranges */ subsys_initcall(cxl_acpi_init); + +/* + * Arrange for host-bridge ports to be active synchronous with + * cxl_acpi_probe() exit. + */ +MODULE_SOFTDEP("pre: cxl_port"); + module_exit(cxl_acpi_exit); MODULE_DESCRIPTION("CXL ACPI: Platform Support"); MODULE_LICENSE("GPL v2"); From patchwork Wed Oct 23 01:43:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846303 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A03EC20323; Wed, 23 Oct 2024 01:43:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647834; cv=none; b=UoPQgO1xhG1oVSQ/VvPjpaZFc9P/R3frWKj5WnvY9TGiKaIzViOQpN+1HnDC9kagqBYO3FMvt4K65DQZa4W9T+Xw8oVyjSVijiPxXKVNnjXhyXRSPmN+JtaiYx7oYnPMr7WiSlhz3jzW2LWvhmLrv+fe/SLW1zyAjFR75FGHSkk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647834; c=relaxed/simple; bh=S5SBBD34sDQ+NONT3hUfjKBSzcn8qXTm64swiCBSqr0=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=klRe8SLvKG0+OJc//6q0cdy/pyPTfKNf1tCB+zei6eATLCnxc3gvrvRpqOpUho5sU/gY2VCYjv8JIog2NC3rpy0FfV4sa+gGnPfYCsEB1Hm1g/YtDtPXaffuCbzKSIivrWKnItWNuz3aSd0MvnwNciLbDzPVr6fhgpRSUzR4pSY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BJyVL+O2; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BJyVL+O2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647832; x=1761183832; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=S5SBBD34sDQ+NONT3hUfjKBSzcn8qXTm64swiCBSqr0=; b=BJyVL+O2783aKByymKxVauZpPr/rlChm1u926ek7aqzNIZwDR//SJKuR 8Iso/WUnIdQkVcFH457BYkFJQvX9E90lNMvYERDURCrFPCChHZjszYkT2 j4CzPs+rWtB5BO2l1CoyWMSECmhJZBR0CssuCfYWxuwolPHuYkL0BrONc dicmh9ABi2XBfpHC2OWIWGvz9Ob1TiuxeswHfCqXlPdH4zTwuRjnvl76h Mna7PZ5ruyz1hdbx47rpQsYJPj7XtAZQXmv0xwBPqOYUhIRrihhRoQmDS gAFGateY/EXKX5LHNtHSZi7lkvZI245jKFvMcoUc/J3mqaovKwsV57Ol+ A==; X-CSE-ConnectionGUID: hPdAX1+jR++W+zrU1O1rFQ== X-CSE-MsgGUID: ftKi00aXQDq0pwl+vswHng== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46675945" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46675945" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:52 -0700 X-CSE-ConnectionGUID: DJemh+FDTI2zxZuGYJwvuw== X-CSE-MsgGUID: /AB1vowvR7WHTPh/BgX3sg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="110844040" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:43:51 -0700 Subject: [PATCH v2 4/6] cxl/port: Fix use-after-free, permit out-of-order decoder shutdown From: Dan Williams To: ira.weiny@intel.com Cc: stable@vger.kernel.org, Jonathan Cameron , Greg Kroah-Hartman , Davidlohr Bueso , Dave Jiang , Alison Schofield , Zijun Hu , vishal.l.verma@intel.com, dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:43:49 -0700 Message-ID: <172964782781.81806.17902885593105284330.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In support of investigating an initialization failure report [1], cxl_test was updated to register mock memory-devices after the mock root-port/bus device had been registered. That led to cxl_test crashing with a use-after-free bug with the following signature: cxl_port_attach_region: cxl region3: cxl_host_bridge.0:port3 decoder3.0 add: mem0:decoder7.0 @ 0 next: cxl_switch_uport.0 nr_eps: 1 nr_targets: 1 cxl_port_attach_region: cxl region3: cxl_host_bridge.0:port3 decoder3.0 add: mem4:decoder14.0 @ 1 next: cxl_switch_uport.0 nr_eps: 2 nr_targets: 1 cxl_port_setup_targets: cxl region3: cxl_switch_uport.0:port6 target[0] = cxl_switch_dport.0 for mem0:decoder7.0 @ 0 1) cxl_port_setup_targets: cxl region3: cxl_switch_uport.0:port6 target[1] = cxl_switch_dport.4 for mem4:decoder14.0 @ 1 [..] cxld_unregister: cxl decoder14.0: cxl_region_decode_reset: cxl_region region3: mock_decoder_reset: cxl_port port3: decoder3.0 reset 2) mock_decoder_reset: cxl_port port3: decoder3.0: out of order reset, expected decoder3.1 cxl_endpoint_decoder_release: cxl decoder14.0: [..] cxld_unregister: cxl decoder7.0: 3) cxl_region_decode_reset: cxl_region region3: Oops: general protection fault, probably for non-canonical address 0x6b6b6b6b6b6b6bc3: 0000 [#1] PREEMPT SMP PTI [..] RIP: 0010:to_cxl_port+0x8/0x60 [cxl_core] [..] Call Trace: cxl_region_decode_reset+0x69/0x190 [cxl_core] cxl_region_detach+0xe8/0x210 [cxl_core] cxl_decoder_kill_region+0x27/0x40 [cxl_core] cxld_unregister+0x5d/0x60 [cxl_core] At 1) a region has been established with 2 endpoint decoders (7.0 and 14.0). Those endpoints share a common switch-decoder in the topology (3.0). At teardown, 2), decoder14.0 is the first to be removed and hits the "out of order reset case" in the switch decoder. The effect though is that region3 cleanup is aborted leaving it in-tact and referencing decoder14.0. At 3) the second attempt to teardown region3 trips over the stale decoder14.0 object which has long since been deleted. The fix here is to recognize that the CXL specification places no mandate on in-order shutdown of switch-decoders, the driver enforces in-order allocation, and hardware enforces in-order commit. So, rather than fail and leave objects dangling, always remove them. In support of making cxl_region_decode_reset() always succeed, cxl_region_invalidate_memregion() failures are turned into warnings. Crashing the kernel is ok there since system integrity is at risk if caches cannot be managed around physical address mutation events like CXL region destruction. A new device_for_each_child_reverse_from() is added to cleanup port->commit_end after all dependent decoders have been disabled. In other words if decoders are allocated 0->1->2 and disabled 1->2->0 then port->commit_end only decrements from 2 after 2 has been disabled, and it decrements all the way to zero since 1 was disabled previously. Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Cc: Fixes: 176baefb2eb5 ("cxl/hdm: Commit decoder state to hardware") Reviewed-by: Jonathan Cameron Cc: Greg Kroah-Hartman Cc: Davidlohr Bueso Cc: Dave Jiang Cc: Alison Schofield Cc: Ira Weiny Cc: Zijun Hu Signed-off-by: Dan Williams Reviewed-by: Ira Weiny --- drivers/base/core.c | 35 +++++++++++++++++++++++++++++ drivers/cxl/core/hdm.c | 50 +++++++++++++++++++++++++++++++++++------- drivers/cxl/core/region.c | 48 +++++++++++----------------------------- drivers/cxl/cxl.h | 3 ++- include/linux/device.h | 3 +++ tools/testing/cxl/test/cxl.c | 14 ++++-------- 6 files changed, 100 insertions(+), 53 deletions(-) diff --git a/drivers/base/core.c b/drivers/base/core.c index a4c853411a6b..e42f1ad73078 100644 --- a/drivers/base/core.c +++ b/drivers/base/core.c @@ -4037,6 +4037,41 @@ int device_for_each_child_reverse(struct device *parent, void *data, } EXPORT_SYMBOL_GPL(device_for_each_child_reverse); +/** + * device_for_each_child_reverse_from - device child iterator in reversed order. + * @parent: parent struct device. + * @from: optional starting point in child list + * @fn: function to be called for each device. + * @data: data for the callback. + * + * Iterate over @parent's child devices, starting at @from, and call @fn + * for each, passing it @data. This helper is identical to + * device_for_each_child_reverse() when @from is NULL. + * + * @fn is checked each iteration. If it returns anything other than 0, + * iteration stop and that value is returned to the caller of + * device_for_each_child_reverse_from(); + */ +int device_for_each_child_reverse_from(struct device *parent, + struct device *from, const void *data, + int (*fn)(struct device *, const void *)) +{ + struct klist_iter i; + struct device *child; + int error = 0; + + if (!parent->p) + return 0; + + klist_iter_init_node(&parent->p->klist_children, &i, + (from ? &from->p->knode_parent : NULL)); + while ((child = prev_device(&i)) && !error) + error = fn(child, data); + klist_iter_exit(&i); + return error; +} +EXPORT_SYMBOL_GPL(device_for_each_child_reverse_from); + /** * device_find_child - device iterator for locating a particular device. * @parent: parent struct device diff --git a/drivers/cxl/core/hdm.c b/drivers/cxl/core/hdm.c index 3df10517a327..223c273c0cd1 100644 --- a/drivers/cxl/core/hdm.c +++ b/drivers/cxl/core/hdm.c @@ -712,7 +712,44 @@ static int cxl_decoder_commit(struct cxl_decoder *cxld) return 0; } -static int cxl_decoder_reset(struct cxl_decoder *cxld) +static int commit_reap(struct device *dev, const void *data) +{ + struct cxl_port *port = to_cxl_port(dev->parent); + struct cxl_decoder *cxld; + + if (!is_switch_decoder(dev) && !is_endpoint_decoder(dev)) + return 0; + + cxld = to_cxl_decoder(dev); + if (port->commit_end == cxld->id && + ((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) { + port->commit_end--; + dev_dbg(&port->dev, "reap: %s commit_end: %d\n", + dev_name(&cxld->dev), port->commit_end); + } + + return 0; +} + +void cxl_port_commit_reap(struct cxl_decoder *cxld) +{ + struct cxl_port *port = to_cxl_port(cxld->dev.parent); + + lockdep_assert_held_write(&cxl_region_rwsem); + + /* + * Once the highest committed decoder is disabled, free any other + * decoders that were pinned allocated by out-of-order release. + */ + port->commit_end--; + dev_dbg(&port->dev, "reap: %s commit_end: %d\n", dev_name(&cxld->dev), + port->commit_end); + device_for_each_child_reverse_from(&port->dev, &cxld->dev, NULL, + commit_reap); +} +EXPORT_SYMBOL_NS_GPL(cxl_port_commit_reap, CXL); + +static void cxl_decoder_reset(struct cxl_decoder *cxld) { struct cxl_port *port = to_cxl_port(cxld->dev.parent); struct cxl_hdm *cxlhdm = dev_get_drvdata(&port->dev); @@ -721,14 +758,14 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) u32 ctrl; if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) - return 0; + return; - if (port->commit_end != id) { + if (port->commit_end == id) + cxl_port_commit_reap(cxld); + else dev_dbg(&port->dev, "%s: out of order reset, expected decoder%d.%d\n", dev_name(&cxld->dev), port->id, port->commit_end); - return -EBUSY; - } down_read(&cxl_dpa_rwsem); ctrl = readl(hdm + CXL_HDM_DECODER0_CTRL_OFFSET(id)); @@ -741,7 +778,6 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) writel(0, hdm + CXL_HDM_DECODER0_BASE_LOW_OFFSET(id)); up_read(&cxl_dpa_rwsem); - port->commit_end--; cxld->flags &= ~CXL_DECODER_F_ENABLE; /* Userspace is now responsible for reconfiguring this decoder */ @@ -751,8 +787,6 @@ static int cxl_decoder_reset(struct cxl_decoder *cxld) cxled = to_cxl_endpoint_decoder(&cxld->dev); cxled->state = CXL_DECODER_STATE_MANUAL; } - - return 0; } static int cxl_setup_hdm_decoder_from_dvsec( diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e701e4b04032..3478d2058303 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -232,8 +232,8 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) "Bypassing cpu_cache_invalidate_memregion() for testing!\n"); return 0; } else { - dev_err(&cxlr->dev, - "Failed to synchronize CPU cache state\n"); + dev_WARN(&cxlr->dev, + "Failed to synchronize CPU cache state\n"); return -ENXIO; } } @@ -242,19 +242,17 @@ static int cxl_region_invalidate_memregion(struct cxl_region *cxlr) return 0; } -static int cxl_region_decode_reset(struct cxl_region *cxlr, int count) +static void cxl_region_decode_reset(struct cxl_region *cxlr, int count) { struct cxl_region_params *p = &cxlr->params; - int i, rc = 0; + int i; /* - * Before region teardown attempt to flush, and if the flush - * fails cancel the region teardown for data consistency - * concerns + * Before region teardown attempt to flush, evict any data cached for + * this region, or scream loudly about missing arch / platform support + * for CXL teardown. */ - rc = cxl_region_invalidate_memregion(cxlr); - if (rc) - return rc; + cxl_region_invalidate_memregion(cxlr); for (i = count - 1; i >= 0; i--) { struct cxl_endpoint_decoder *cxled = p->targets[i]; @@ -277,23 +275,17 @@ static int cxl_region_decode_reset(struct cxl_region *cxlr, int count) cxl_rr = cxl_rr_load(iter, cxlr); cxld = cxl_rr->decoder; if (cxld->reset) - rc = cxld->reset(cxld); - if (rc) - return rc; + cxld->reset(cxld); set_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); } endpoint_reset: - rc = cxled->cxld.reset(&cxled->cxld); - if (rc) - return rc; + cxled->cxld.reset(&cxled->cxld); set_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); } /* all decoders associated with this region have been torn down */ clear_bit(CXL_REGION_F_NEEDS_RESET, &cxlr->flags); - - return 0; } static int commit_decoder(struct cxl_decoder *cxld) @@ -409,16 +401,8 @@ static ssize_t commit_store(struct device *dev, struct device_attribute *attr, * still pending. */ if (p->state == CXL_CONFIG_RESET_PENDING) { - rc = cxl_region_decode_reset(cxlr, p->interleave_ways); - /* - * Revert to committed since there may still be active - * decoders associated with this region, or move forward - * to active to mark the reset successful - */ - if (rc) - p->state = CXL_CONFIG_COMMIT; - else - p->state = CXL_CONFIG_ACTIVE; + cxl_region_decode_reset(cxlr, p->interleave_ways); + p->state = CXL_CONFIG_ACTIVE; } } @@ -2054,13 +2038,7 @@ static int cxl_region_detach(struct cxl_endpoint_decoder *cxled) get_device(&cxlr->dev); if (p->state > CXL_CONFIG_ACTIVE) { - /* - * TODO: tear down all impacted regions if a device is - * removed out of order - */ - rc = cxl_region_decode_reset(cxlr, p->interleave_ways); - if (rc) - goto out; + cxl_region_decode_reset(cxlr, p->interleave_ways); p->state = CXL_CONFIG_ACTIVE; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index 0d8b810a51f0..5406e3ab3d4a 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -359,7 +359,7 @@ struct cxl_decoder { struct cxl_region *region; unsigned long flags; int (*commit)(struct cxl_decoder *cxld); - int (*reset)(struct cxl_decoder *cxld); + void (*reset)(struct cxl_decoder *cxld); }; /* @@ -730,6 +730,7 @@ static inline bool is_cxl_root(struct cxl_port *port) int cxl_num_decoders_committed(struct cxl_port *port); bool is_cxl_port(const struct device *dev); struct cxl_port *to_cxl_port(const struct device *dev); +void cxl_port_commit_reap(struct cxl_decoder *cxld); struct pci_bus; int devm_cxl_register_pci_bus(struct device *host, struct device *uport_dev, struct pci_bus *bus); diff --git a/include/linux/device.h b/include/linux/device.h index b4bde8d22697..667cb6db9019 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -1078,6 +1078,9 @@ int device_for_each_child(struct device *dev, void *data, int (*fn)(struct device *dev, void *data)); int device_for_each_child_reverse(struct device *dev, void *data, int (*fn)(struct device *dev, void *data)); +int device_for_each_child_reverse_from(struct device *parent, + struct device *from, const void *data, + int (*fn)(struct device *, const void *)); struct device *device_find_child(struct device *dev, void *data, int (*match)(struct device *dev, void *data)); struct device *device_find_child_by_name(struct device *parent, diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index 90d5afd52dd0..c5bbd89b3192 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -693,26 +693,22 @@ static int mock_decoder_commit(struct cxl_decoder *cxld) return 0; } -static int mock_decoder_reset(struct cxl_decoder *cxld) +static void mock_decoder_reset(struct cxl_decoder *cxld) { struct cxl_port *port = to_cxl_port(cxld->dev.parent); int id = cxld->id; if ((cxld->flags & CXL_DECODER_F_ENABLE) == 0) - return 0; + return; dev_dbg(&port->dev, "%s reset\n", dev_name(&cxld->dev)); - if (port->commit_end != id) { + if (port->commit_end == id) + cxl_port_commit_reap(cxld); + else dev_dbg(&port->dev, "%s: out of order reset, expected decoder%d.%d\n", dev_name(&cxld->dev), port->id, port->commit_end); - return -EBUSY; - } - - port->commit_end--; cxld->flags &= ~CXL_DECODER_F_ENABLE; - - return 0; } static void default_mock_decoder(struct cxl_decoder *cxld) From patchwork Wed Oct 23 01:43:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846304 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0098D20323 for ; Wed, 23 Oct 2024 01:44:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647843; cv=none; b=NXQkhkxqHXGNJLjVVvyurv8/ho+eph2twAiDgxNYXi/UTRe1zGIrhPNMchCYmU0EgvSDWqZnKoqEXrJq1568Oq0B033vbRQjfa/JSAW32M8PRiHBx1MkFD8dnQCEb6a3RNDKw9213BqNqbxIzPUnXsnov6hlh8nHKYlvB9p68RY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647843; c=relaxed/simple; bh=dnQwbUB68OkA+2eKZ7BUN1fY2s+v4Sxk//VQ3Ml6v1I=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=eu0/u1QNsow5NPAvWtYjxx/ecPk9fB3LhPHX7OEuOw6nGvAmgBwc1PJQcT3SAideP9DlBsL4R4mDffD95Jee/TKcPzbf0fbj85qTYICTQAwqRZvnzPFQOyo2/XOBeNq8SdQF8FGZV1PttGnuC9Apk0DcULeyEZ2OKvTWe+K5PCk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=R58uUQj+; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="R58uUQj+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647842; x=1761183842; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=dnQwbUB68OkA+2eKZ7BUN1fY2s+v4Sxk//VQ3Ml6v1I=; b=R58uUQj+8ht4WNx9FpUx8cckINUGxnnD+XSxJ3tcj7h7aW1kyY5LW+CH iwAjZ3C793bFQschko/GKQGhy4P7iykZHDujAU6BgXcxa0Z/BVy8joiL7 r+xsMlFxjsBklVrIyZyqQSXiYQmCpnDwa3vkFco3v73mtebaxQhRqSXfl T1C807gcfU5B3WtbSXLTCEzQOuoixBZQkkp8lpHMqfZ5vi9KEU+5S8mqX 8sqSKzyNWOSF5M8oE6Bs6bpPKqnY47l8Co2i+x/hiUfLJQt51jH5WxfMH o2U1M78gTyVEildd5Ek/fR6JOp+tXxm6vs+Jf1n6YjbrM8jOCleSv0GOD g==; X-CSE-ConnectionGUID: aL+blriDQiuJbmbM2vtYIA== X-CSE-MsgGUID: D4/iOe9BSnytcGWQ0UWY4A== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46675957" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46675957" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:44:01 -0700 X-CSE-ConnectionGUID: 0gf9+tbsQgmb/GWw+EUyWw== X-CSE-MsgGUID: F+6dFQgtRa2k5GKPgKSwUg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="110844073" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:44:00 -0700 Subject: [PATCH v2 5/6] cxl/port: Prevent out-of-order decoder allocation From: Dan Williams To: ira.weiny@intel.com Cc: Zijun Hu , Davidlohr Bueso , Vishal Verma , Alison Schofield , Jonathan Cameron , dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:43:57 -0700 Message-ID: <172964783668.81806.14962699553881333486.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With the recent change to allow out-of-order decoder de-commit it highlights a need to strengthen the in-order decoder commit guarantees. As it stands match_free_decoder() ensures that if 2 regions are racing decoder allocations the one that wins the race will get the lower id decoder, but that still leaves the race to *commit* the decoder. Rather than have this complicated case of "reserved in-order, but may still commit out-of-order", just arrange for the reservation order to match the commit-order. In other words, prevent subsequent allocations until the last reservation is committed. This precludes overlapping region creation events and requires the previous regionN to either move forward to the decoder commit stage or drop its reservation before regionN+1 can move forward. That is, provided that regionN and regionN+1 decode through the same switch port. As a side effect this allows match_free_decoder() to drop its dependency on needing write access to the device_find_child() @data parameter [1]. Reported-by: Zijun Hu Closes: http://lore.kernel.org/20240905-const_dfc_prepare-v4-0-4180e1d5a244@quicinc.com Cc: Davidlohr Bueso Cc: Vishal Verma Cc: Alison Schofield Cc: Jonathan Cameron Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny --- drivers/cxl/core/region.c | 43 +++++++++++++++++++++++++++++++++---------- 1 file changed, 33 insertions(+), 10 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 3478d2058303..dff618c708dc 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -778,26 +778,50 @@ static size_t show_targetN(struct cxl_region *cxlr, char *buf, int pos) return rc; } +static int check_commit_order(struct device *dev, const void *data) +{ + struct cxl_decoder *cxld = to_cxl_decoder(dev); + + /* + * if port->commit_end is not the only free decoder, then out of + * order shutdown has occurred, block further allocations until + * that is resolved + */ + if (((cxld->flags & CXL_DECODER_F_ENABLE) == 0)) + return -EBUSY; + return 0; +} + static int match_free_decoder(struct device *dev, void *data) { + struct cxl_port *port = to_cxl_port(dev->parent); struct cxl_decoder *cxld; - int *id = data; + int rc; if (!is_switch_decoder(dev)) return 0; cxld = to_cxl_decoder(dev); - /* enforce ordered allocation */ - if (cxld->id != *id) + if (cxld->id != port->commit_end + 1) return 0; - if (!cxld->region) - return 1; - - (*id)++; + if (cxld->region) { + dev_dbg(dev->parent, + "next decoder to commit (%s) is already reserved (%s)\n", + dev_name(dev), dev_name(&cxld->region->dev)); + return 0; + } - return 0; + rc = device_for_each_child_reverse_from(dev->parent, dev, NULL, + check_commit_order); + if (rc) { + dev_dbg(dev->parent, + "unable to allocate %s due to out of order shutdown\n", + dev_name(dev)); + return 0; + } + return 1; } static int match_auto_decoder(struct device *dev, void *data) @@ -824,7 +848,6 @@ cxl_region_find_decoder(struct cxl_port *port, struct cxl_region *cxlr) { struct device *dev; - int id = 0; if (port == cxled_to_port(cxled)) return &cxled->cxld; @@ -833,7 +856,7 @@ cxl_region_find_decoder(struct cxl_port *port, dev = device_find_child(&port->dev, &cxlr->params, match_auto_decoder); else - dev = device_find_child(&port->dev, &id, match_free_decoder); + dev = device_find_child(&port->dev, NULL, match_free_decoder); if (!dev) return NULL; /* From patchwork Wed Oct 23 01:44:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 13846305 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 75D8220323 for ; Wed, 23 Oct 2024 01:44:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647851; cv=none; b=n4DPxTCPf/Un46TMPm+oIDB81pZQ9Yh1lc9izhIFVmSXRuuIGmVcwdh9oj9jSahSKGmtTyZ4Q0X/p1eHEGoQBERxQ0o95zRY6OLBrXF9vslFdHUBqlzgy6gQCLugGxWi1wAOjh/m46ekok5ShIWuX/9DltYi5CgS1a4O4si0sqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729647851; c=relaxed/simple; bh=8+1Oxg2KD3Wc6rz5kN6jZmUPo8IGhAIV8pkcpyE4JhM=; h=Subject:From:To:Cc:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Tevb8shG468siUYekbtOWU/oMoVJ176r/SO6nuA3mQ7XI4BLT4fmNqO2XcuZCtcDluu+jI9IgGE2zEYCtbN0v0GxoTyzlouYJuCqrsnO5nzpEeV4df9JJ4TRZitlEA10uIGlaJCAUHV3IIaVontyMR3lZiv7kZEJ5uZVtaQKUIY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ncbiu9w8; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ncbiu9w8" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1729647849; x=1761183849; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8+1Oxg2KD3Wc6rz5kN6jZmUPo8IGhAIV8pkcpyE4JhM=; b=ncbiu9w88SNqV6VBhGatP1BrdrcI5vIwVwEvjwSnN2pbS7Q0sbXWvNO7 NxhROOFvTTL3tG+LZ4q5fUfJiTHuBS6pDPEZN+It5ZFUNPsFQs0o89+fy CrU9i0pGHSDZS+wLVa/Sdpb+RRfSSOeYstLcw+IIPzM2zALLmiacOVSdK bCncEfIzjFYx88uFFPcDKGbok96igdAUQjLLrePZQ9cgeZjnEqWl/qTWF lEv1+5ATqMK8/+Y03XTbdpVOnLh+l/aMwb4raSzdQeSt/3a3hMdOKVTso JTEKJFN546VLttfUcBOxxkXYY26/N+oiWu3C1P6y4/D2ojbTuSiUOduYF A==; X-CSE-ConnectionGUID: IA7DMlk4QLusni/iktDdNA== X-CSE-MsgGUID: mQLnjKeGQFaPy/VaPS8G4A== X-IronPort-AV: E=McAfee;i="6700,10204,11222"; a="46675968" X-IronPort-AV: E=Sophos;i="6.11,199,1725346800"; d="scan'208";a="46675968" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:44:09 -0700 X-CSE-ConnectionGUID: AudK1LvdTQucC9r4+nZ97g== X-CSE-MsgGUID: KeRn4tcrTO6ckgcvoN6Zvw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,223,1725346800"; d="scan'208";a="110844108" Received: from cmdeoliv-mobl4.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.125.110.222]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Oct 2024 18:44:08 -0700 Subject: [PATCH v2 6/6] cxl/test: Improve init-order fidelity relative to real-world systems From: Dan Williams To: ira.weiny@intel.com Cc: Davidlohr Bueso , Jonathan Cameron , Dave Jiang , Alison Schofield , Vishal Verma , dave.jiang@intel.com, linux-cxl@vger.kernel.org Date: Tue, 22 Oct 2024 18:44:06 -0700 Message-ID: <172964784521.81806.15791069994065969243.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> References: <172964779333.81806.8852577918216421011.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The investigation of an initialization failure [1] highlighted that cxl_test does not reflect the init-order of real world systems. The expected order is root/bus first then async probing of the memory devices. Fix up cxl_test to reflect that order. While it did not reproduce the initial bug report (since that is dependent on built-in vs modular builds), it did reveal a separate latent bug in the subsystem's decoder shutdown flow. Fix for that sent separately. Link: http://lore.kernel.org/20241004212504.1246-1-gourry@gourry.net [1] Cc: Davidlohr Bueso Cc: Jonathan Cameron Cc: Dave Jiang Cc: Alison Schofield Cc: Vishal Verma Cc: Ira Weiny Signed-off-by: Dan Williams Reviewed-by: Jonathan Cameron Reviewed-by: Ira Weiny --- tools/testing/cxl/test/cxl.c | 186 +++++++++++++++++++++++------------------- tools/testing/cxl/test/mem.c | 1 2 files changed, 104 insertions(+), 83 deletions(-) diff --git a/tools/testing/cxl/test/cxl.c b/tools/testing/cxl/test/cxl.c index c5bbd89b3192..050725afa45d 100644 --- a/tools/testing/cxl/test/cxl.c +++ b/tools/testing/cxl/test/cxl.c @@ -1058,7 +1058,7 @@ static void mock_companion(struct acpi_device *adev, struct device *dev) #define SZ_64G (SZ_32G * 2) #endif -static __init int cxl_rch_init(void) +static __init int cxl_rch_topo_init(void) { int rc, i; @@ -1086,30 +1086,8 @@ static __init int cxl_rch_init(void) goto err_bridge; } - for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) { - int idx = NR_MEM_MULTI + NR_MEM_SINGLE + i; - struct platform_device *rch = cxl_rch[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_rcd", idx); - if (!pdev) - goto err_mem; - pdev->dev.parent = &rch->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_rcd[i] = pdev; - } - return 0; -err_mem: - for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) - platform_device_unregister(cxl_rcd[i]); err_bridge: for (i = ARRAY_SIZE(cxl_rch) - 1; i >= 0; i--) { struct platform_device *pdev = cxl_rch[i]; @@ -1123,12 +1101,10 @@ static __init int cxl_rch_init(void) return rc; } -static void cxl_rch_exit(void) +static void cxl_rch_topo_exit(void) { int i; - for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) - platform_device_unregister(cxl_rcd[i]); for (i = ARRAY_SIZE(cxl_rch) - 1; i >= 0; i--) { struct platform_device *pdev = cxl_rch[i]; @@ -1139,7 +1115,7 @@ static void cxl_rch_exit(void) } } -static __init int cxl_single_init(void) +static __init int cxl_single_topo_init(void) { int i, rc; @@ -1224,29 +1200,8 @@ static __init int cxl_single_init(void) cxl_swd_single[i] = pdev; } - for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) { - struct platform_device *dport = cxl_swd_single[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_mem", NR_MEM_MULTI + i); - if (!pdev) - goto err_mem; - pdev->dev.parent = &dport->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_mem_single[i] = pdev; - } - return 0; -err_mem: - for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem_single[i]); err_dport: for (i = ARRAY_SIZE(cxl_swd_single) - 1; i >= 0; i--) platform_device_unregister(cxl_swd_single[i]); @@ -1269,12 +1224,10 @@ static __init int cxl_single_init(void) return rc; } -static void cxl_single_exit(void) +static void cxl_single_topo_exit(void) { int i; - for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem_single[i]); for (i = ARRAY_SIZE(cxl_swd_single) - 1; i >= 0; i--) platform_device_unregister(cxl_swd_single[i]); for (i = ARRAY_SIZE(cxl_swu_single) - 1; i >= 0; i--) @@ -1291,6 +1244,91 @@ static void cxl_single_exit(void) } } +static void cxl_mem_exit(void) +{ + int i; + + for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) + platform_device_unregister(cxl_rcd[i]); + for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem_single[i]); + for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem[i]); +} + +static int cxl_mem_init(void) +{ + int i, rc; + + for (i = 0; i < ARRAY_SIZE(cxl_mem); i++) { + struct platform_device *dport = cxl_switch_dport[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_mem", i); + if (!pdev) + goto err_mem; + pdev->dev.parent = &dport->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_mem; + } + cxl_mem[i] = pdev; + } + + for (i = 0; i < ARRAY_SIZE(cxl_mem_single); i++) { + struct platform_device *dport = cxl_swd_single[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_mem", NR_MEM_MULTI + i); + if (!pdev) + goto err_single; + pdev->dev.parent = &dport->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_single; + } + cxl_mem_single[i] = pdev; + } + + for (i = 0; i < ARRAY_SIZE(cxl_rcd); i++) { + int idx = NR_MEM_MULTI + NR_MEM_SINGLE + i; + struct platform_device *rch = cxl_rch[i]; + struct platform_device *pdev; + + pdev = platform_device_alloc("cxl_rcd", idx); + if (!pdev) + goto err_rcd; + pdev->dev.parent = &rch->dev; + set_dev_node(&pdev->dev, i % 2); + + rc = platform_device_add(pdev); + if (rc) { + platform_device_put(pdev); + goto err_rcd; + } + cxl_rcd[i] = pdev; + } + + return 0; + +err_rcd: + for (i = ARRAY_SIZE(cxl_rcd) - 1; i >= 0; i--) + platform_device_unregister(cxl_rcd[i]); +err_single: + for (i = ARRAY_SIZE(cxl_mem_single) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem_single[i]); +err_mem: + for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) + platform_device_unregister(cxl_mem[i]); + return rc; +} + static __init int cxl_test_init(void) { int rc, i; @@ -1403,29 +1441,11 @@ static __init int cxl_test_init(void) cxl_switch_dport[i] = pdev; } - for (i = 0; i < ARRAY_SIZE(cxl_mem); i++) { - struct platform_device *dport = cxl_switch_dport[i]; - struct platform_device *pdev; - - pdev = platform_device_alloc("cxl_mem", i); - if (!pdev) - goto err_mem; - pdev->dev.parent = &dport->dev; - set_dev_node(&pdev->dev, i % 2); - - rc = platform_device_add(pdev); - if (rc) { - platform_device_put(pdev); - goto err_mem; - } - cxl_mem[i] = pdev; - } - - rc = cxl_single_init(); + rc = cxl_single_topo_init(); if (rc) - goto err_mem; + goto err_dport; - rc = cxl_rch_init(); + rc = cxl_rch_topo_init(); if (rc) goto err_single; @@ -1438,19 +1458,20 @@ static __init int cxl_test_init(void) rc = platform_device_add(cxl_acpi); if (rc) - goto err_add; + goto err_root; + + rc = cxl_mem_init(); + if (rc) + goto err_root; return 0; -err_add: +err_root: platform_device_put(cxl_acpi); err_rch: - cxl_rch_exit(); + cxl_rch_topo_exit(); err_single: - cxl_single_exit(); -err_mem: - for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem[i]); + cxl_single_topo_exit(); err_dport: for (i = ARRAY_SIZE(cxl_switch_dport) - 1; i >= 0; i--) platform_device_unregister(cxl_switch_dport[i]); @@ -1482,11 +1503,10 @@ static __exit void cxl_test_exit(void) { int i; + cxl_mem_exit(); platform_device_unregister(cxl_acpi); - cxl_rch_exit(); - cxl_single_exit(); - for (i = ARRAY_SIZE(cxl_mem) - 1; i >= 0; i--) - platform_device_unregister(cxl_mem[i]); + cxl_rch_topo_exit(); + cxl_single_topo_exit(); for (i = ARRAY_SIZE(cxl_switch_dport) - 1; i >= 0; i--) platform_device_unregister(cxl_switch_dport[i]); for (i = ARRAY_SIZE(cxl_switch_uport) - 1; i >= 0; i--) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index ad5c4c18c5c6..71916e0e1546 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -1673,6 +1673,7 @@ static struct platform_driver cxl_mock_mem_driver = { .name = KBUILD_MODNAME, .dev_groups = cxl_mock_mem_groups, .groups = cxl_mock_mem_core_groups, + .probe_type = PROBE_PREFER_ASYNCHRONOUS, }, }; From patchwork Sun Oct 27 07:57:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ye, Huaisheng" X-Patchwork-Id: 13852432 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DAE239FD9; Sun, 27 Oct 2024 07:58:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730015889; cv=none; b=ov1VHicrNJ6Fr1QQ81+WKLu9j215vsIJSdQdlx+D8m8T2jACMXF428LT7AfnpyoI0sQle2ovIIdMlkhZOfjrqAW9Cpe8MxjszMER2lcufy0lvMCetbNNhKVTqtbvsWkl4y8kp4JE8QqD9tDfoBSivuAUAEzUI9VbOWKvzNE0ilw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730015889; c=relaxed/simple; bh=D8heilxrwBNBqtWGOdlYvysv+6a/+zwkQs22gXXTAaI=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=DGEGvJFVi5zpa6mqORp6E7bMnEnzanQwUqITpwyD4/CZphkRRzkwAxA3/RckSMt7yHD63ow8Wt0wSJwfJCJYP1qrlGWXdM8c+xFvrj+QJIgw+Dy02zqgpY+BRlWxtBdKdekrmE2V9cBv2jhG6TzxUYgLb/1ZJGuQf1qEy40cnqY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=V15x4ptw; arc=none smtp.client-ip=198.175.65.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="V15x4ptw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1730015886; x=1761551886; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=D8heilxrwBNBqtWGOdlYvysv+6a/+zwkQs22gXXTAaI=; b=V15x4ptwowmscYFwKnQ0Uzyv5pa1oT7KjiOT9XqY9/WyQNfLswaNPqO4 jmIY97IX85uNGSZRbTqpaMQUp7OvyuV9Vu/n7/M8RVWei5uLYmKZoWciy IbbjuMhmxmusqJCMVLOq/JIqJMC3Ho5WcwCasFte8U1anOjPdxjauOJci sEWiFWjUeh5VJkQCu3vTeyMPuLTGKmQplc8HYL3rHF71Wsuvw1MmU9CBH yXJgX3GDag57PqlTw7JUgIz40hC3TyFjRaEP/gS8u/fQr//embKYAQij8 CW+WYBF9hUgG213qcEJstL0D0KwjRMGIXyWq9Sw5q8HZK7ipmVoh3I90m A==; X-CSE-ConnectionGUID: 0v6iC9G2T8mTpjkvjL1s8A== X-CSE-MsgGUID: Fou5him9STyD2l0xa4rwRg== X-IronPort-AV: E=McAfee;i="6700,10204,11237"; a="41033994" X-IronPort-AV: E=Sophos;i="6.11,236,1725346800"; d="scan'208";a="41033994" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by orvoesa104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 27 Oct 2024 00:58:06 -0700 X-CSE-ConnectionGUID: c7eGueviT3uRDiIjBFG3xA== X-CSE-MsgGUID: JbuLpqg2TfOAfZSNiJRAaA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.11,236,1725346800"; d="scan'208";a="81477242" Received: from unknown (HELO ixc04.bj.intel.com) ([10.238.153.130]) by orviesa006.jf.intel.com with ESMTP; 27 Oct 2024 00:58:04 -0700 From: Huaisheng Ye To: linux-cxl@vger.kernel.org, dan.j.williams@intel.com Cc: linux-kernel@vger.kernel.org, pei.p.jia@intel.com, Huaisheng Ye Subject: [PATCH] [RFC] cxl/region: Fix region creation for greater than x2 switches Date: Sun, 27 Oct 2024 15:57:17 +0800 Message-Id: <20241027075717.3714821-1-huaisheng.ye@intel.com> X-Mailer: git-send-email 2.39.3 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The cxl_port_setup_targets() algorithm fails to identify valid target list ordering in the presence of 4-way and above switches resulting in 'cxl create-region' failures of the form: # cxl create-region -d decoder0.0 -g 1024 -s 2G -t ram -w 8 -m mem4 mem1 mem6 mem3 mem2 mem5 mem7 mem0 cxl region: create_region: region0: failed to set target7 to mem0 cxl region: cmd_create_region: created 0 regions [kernel debug message] check_last_peer:1213: cxl region0: pci0000:0c:port1: cannot host mem6:decoder7.0 at 2 bus_remove_device:574: bus: 'cxl': remove device region0 QEMU can create this failing topology: ACPI0017:00 [root0] | HB_0 [port1] / \ RP_0 RP_1 | | USP [port2] USP [port3] / / \ \ / / \ \ DSP DSP DSP DSP DSP DSP DSP DSP | | | | | | | | mem4 mem6 mem2 mem7 mem1 mem3 mem5 mem0 Pos: 0 2 4 6 1 3 5 7 HB: Host Bridge RP: Root Port USP: Upstream Port DSP: Downstream Port ...with the following command steps: $ qemu-system-x86_64 -machine q35,cxl=on,accel=tcg \ -smp cpus=8 \ -m 8G \ -hda /home/work/vm-images/centos-stream8-02.qcow2 \ -object memory-backend-ram,size=4G,id=m0 \ -object memory-backend-ram,size=4G,id=m1 \ -object memory-backend-ram,size=2G,id=cxl-mem0 \ -object memory-backend-ram,size=2G,id=cxl-mem1 \ -object memory-backend-ram,size=2G,id=cxl-mem2 \ -object memory-backend-ram,size=2G,id=cxl-mem3 \ -object memory-backend-ram,size=2G,id=cxl-mem4 \ -object memory-backend-ram,size=2G,id=cxl-mem5 \ -object memory-backend-ram,size=2G,id=cxl-mem6 \ -object memory-backend-ram,size=2G,id=cxl-mem7 \ -numa node,memdev=m0,cpus=0-3,nodeid=0 \ -numa node,memdev=m1,cpus=4-7,nodeid=1 \ -netdev user,id=net0,hostfwd=tcp::2222-:22 \ -device virtio-net-pci,netdev=net0 \ -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \ -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \ -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \ -device cxl-upstream,bus=root_port0,id=us0 \ -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \ -device cxl-type3,bus=swport0,volatile-memdev=cxl-mem0,id=cxl-vmem0 \ -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \ -device cxl-type3,bus=swport1,volatile-memdev=cxl-mem1,id=cxl-vmem1 \ -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \ -device cxl-type3,bus=swport2,volatile-memdev=cxl-mem2,id=cxl-vmem2 \ -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \ -device cxl-type3,bus=swport3,volatile-memdev=cxl-mem3,id=cxl-vmem3 \ -device cxl-upstream,bus=root_port1,id=us1 \ -device cxl-downstream,port=4,bus=us1,id=swport4,chassis=0,slot=8 \ -device cxl-type3,bus=swport4,volatile-memdev=cxl-mem4,id=cxl-vmem4 \ -device cxl-downstream,port=5,bus=us1,id=swport5,chassis=0,slot=9 \ -device cxl-type3,bus=swport5,volatile-memdev=cxl-mem5,id=cxl-vmem5 \ -device cxl-downstream,port=6,bus=us1,id=swport6,chassis=0,slot=10 \ -device cxl-type3,bus=swport6,volatile-memdev=cxl-mem6,id=cxl-vmem6 \ -device cxl-downstream,port=7,bus=us1,id=swport7,chassis=0,slot=11 \ -device cxl-type3,bus=swport7,volatile-memdev=cxl-mem7,id=cxl-vmem7 \ -M cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=32G & In Guest OS: # cxl create-region -d decoder0.0 -g 1024 -s 2G -t ram -w 8 -m mem4 mem1 mem6 mem3 mem2 mem5 mem7 mem0 Fix the method to calculate @distance by iterativeley multiplying the number of targets per switch port. This also follows the algorithm recommended here [1]. Fixes: 27b3f8d13830 ("cxl/region: Program target lists") Link: http://lore.kernel.org/6538824b52349_7258329466@dwillia2-xfh.jf.intel.com.notmuch [1] Signed-off-by: Huaisheng Ye Tested-by: Li Zhijian --- drivers/cxl/core/region.c | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index e701e4b04032..9e226a293f45 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -1288,6 +1288,7 @@ static int cxl_port_setup_targets(struct cxl_port *port, struct cxl_region_params *p = &cxlr->params; struct cxl_decoder *cxld = cxl_rr->decoder; struct cxl_switch_decoder *cxlsd; + struct cxl_port *iter = port; u16 eig, peig; u8 eiw, peiw; @@ -1304,16 +1305,20 @@ static int cxl_port_setup_targets(struct cxl_port *port, cxlsd = to_cxl_switch_decoder(&cxld->dev); if (cxl_rr->nr_targets_set) { - int i, distance; + int i, distance = 1; + struct cxl_region_ref *cxl_rr_iter; /* - * Passthrough decoders impose no distance requirements between - * peers + * Get distance from the number of distinct targets in region's + * interest and the ancestral nr_targets. */ - if (cxl_rr->nr_targets == 1) - distance = 0; - else - distance = p->nr_targets / cxl_rr->nr_targets; + do { + cxl_rr_iter = cxl_rr_load(iter, cxlr); + distance *= cxl_rr_iter->nr_targets; + iter = to_cxl_port(iter->dev.parent); + } while (!is_cxl_root(iter)); + distance *= cxlrd->cxlsd.cxld.interleave_ways; + for (i = 0; i < cxl_rr->nr_targets_set; i++) if (ep->dport == cxlsd->target[i]) { rc = check_last_peer(cxled, ep, cxl_rr, From patchwork Thu Nov 14 21:34:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Davidlohr Bueso X-Patchwork-Id: 13875698 Received: from poodle.tulip.relay.mailchannels.net (poodle.tulip.relay.mailchannels.net [23.83.218.249]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4708B1AC45F for ; Thu, 14 Nov 2024 22:17:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=23.83.218.249 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731622681; cv=pass; b=ZNYOKqMUTnkx6ka7TfC3VVpMmEXBh6B2GWASIG9aoiuMaCMyA9bSHCfhNGbI5V25NolltAi9Jbfh3nCp/5dn151NR2th9phRbT79VeoKz1RFxEOeH6xJ/4ABMJjxIEbFCOB9CzZsmJZZnFe8t8DiGnXEZB1uD8159s0QiqR5hNY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731622681; c=relaxed/simple; bh=LgUA0KNx718wujUsUwBSefRoTiz4FfCM/26pxTXo15E=; h=From:To:Cc:Subject:Date:Message-ID:MIME-Version; b=PHff/X3KJVJcWqWKKHpUicGwQKC+BertgxdjxaOo4BCn0guKVSGNHFO1H9uqGsvNvUJ7iIkYq95SfWkJbEwgMXZ9JZT/a8G5k8R56InsGfNKPVJ8I+WzK3qF/aqjWIW0+VdpOpoprsb7Aubzr0WmFX1O4M3A8WpQA6m63g3jKeY= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=stgolabs.net; spf=pass smtp.mailfrom=stgolabs.net; dkim=pass (2048-bit key) header.d=stgolabs.net header.i=@stgolabs.net header.b=WGITTmlo; arc=pass smtp.client-ip=23.83.218.249 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=stgolabs.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=stgolabs.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=stgolabs.net header.i=@stgolabs.net header.b="WGITTmlo" X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 973EEC1D79; Thu, 14 Nov 2024 21:38:43 +0000 (UTC) Received: from pdx1-sub0-mail-a233.dreamhost.com (trex-8.trex.outbound.svc.cluster.local [100.111.109.236]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 1F899C3CF7; Thu, 14 Nov 2024 21:38:43 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1731620323; a=rsa-sha256; cv=none; b=4JHIE7SF9Qnx2eSfg+LPvdvt7AERAkOXH+QrLOi3juuCYKD02bA2r+pvKnDVllNDjOtXuw 6HhUcnJwndf9+r0BZ6BPQ9dQB+K2YpxlPQL+4YffuCFIjaBm0RzL6cGA9eUdCAspve0JNe F3L3tWnV9NJR5uiHJ11KGbbO8cb5nHvECxF4T25DxyDCkrVw4xSwvM1mg1xjmCYjzfAVc8 8ZSvR1ipZTjBxCofiWiSylkHePmTfBoWj7MfRdRpP0P6fbQh8phX1hJI3Q0OY+q0TxQmmo 42Bg/FVTxtTc4WUQ+fyVyC6rGPpShmpGdAYlDM7gpqDnL1NKTmzasTd8it3Eug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1731620323; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding:dkim-signature; bh=OYm0g8k3fBxj7KfPxV6h2Ev+gtxx8fg5hi3oMeVyzxs=; b=Cs13CmS4XfG5YIDmDpID2Pfo6pI8pXa1KjACUpRcowCFXLy+lQ3/kPg+FQf07Y8CRJpm1f kpCE0+tYTSl50mkKQeh2LfMwXd5g9kmPZ6uK+1ym411d2qcF53fQsw9z18fyOX3+DzlMI1 2hMITPaIF3QC0lTotCEKm2eLpt5cMhV5H9GeintOdVqz+mNl/3XPQ94Nmlup1woAwEplL3 grO4Uo9YSb7tKrA8knMADBMbO80qkGNHx10GmLmkDpq7iGR/eIR4KidJ157xuMS4AXla1x lb2t4MjpvbkTwDEO+pt2pXJBkKvuMRIizhNCUj/R+zFfjJc5rmSX5S+IqGvoUw== ARC-Authentication-Results: i=1; rspamd-645676964-kdvg4; auth=pass smtp.auth=dreamhost smtp.mailfrom=dave@stgolabs.net X-Sender-Id: dreamhost|x-authsender|dave@stgolabs.net X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|dave@stgolabs.net X-MailChannels-Auth-Id: dreamhost X-Reign-Glossy: 2ae9610077860b20_1731620323429_58666898 X-MC-Loop-Signature: 1731620323429:2798751667 X-MC-Ingress-Time: 1731620323428 Received: from pdx1-sub0-mail-a233.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.111.109.236 (trex/7.0.2); Thu, 14 Nov 2024 21:38:43 +0000 Received: from localhost.localdomain (unknown [108.175.208.146]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: dave@stgolabs.net) by pdx1-sub0-mail-a233.dreamhost.com (Postfix) with ESMTPSA id 4XqD6p2QDhz7d; Thu, 14 Nov 2024 13:38:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=stgolabs.net; s=dreamhost; t=1731620322; bh=OYm0g8k3fBxj7KfPxV6h2Ev+gtxx8fg5hi3oMeVyzxs=; h=From:To:Cc:Subject:Date:Content-Transfer-Encoding; b=WGITTmloaS5QLfK1q9SZY4QfLTOaXiwk3D6iQvxwsOTVE/ROp9TBx91GEMfb9dSHP 6SsbN/qgX8gG6a5Vay0r2nHFh3SGq+D7/cyM5/N2ny7ia6rnuVSDmAMJE/BFvFFrDC PXGygj6KUPRqHDG9iG5xg1fXdTGdBhKd2oM9T/FYbocmGiq8LluBr6TDLcMdQ1KcEE Ov2T5LTe6i6Ld+bjranXzMc/lNyLEJYhVmP7fuydfX3sOAT0gKwWf3/Uxn9HRdnFmx ETFXJmg4Ve13WmGnWqlcWNtmpCBBaRsqdwJdeAuBOXIe1sy6xvdVrb9GVh6RuXmG9+ YeXxhzJj6Td5A== From: Davidlohr Bueso To: dave.jiang@intel.com Cc: dan.j.williams@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, linux-cxl@vger.kernel.org, dave@stgolabs.net Subject: [PATCH] cxl/pci: Fix potential bogus return value upon successful probing Date: Thu, 14 Nov 2024 13:34:39 -0800 Message-ID: <20241114213439.6022-1-dave@stgolabs.net> X-Mailer: git-send-email 2.47.0 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 If cxl_pci_ras_unmask() returns non-zero, cxl_pci_probe() will end up returning that value, instead of zero. Found by code inspeaction. Fixes: 248529edc86 (cxl: add RAS status unmasking for CXL) Signed-off-by: Davidlohr Bueso Reviewed-by: Fan Ni --- drivers/cxl/pci.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 188412d45e0d..01092e5b3b46 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -948,7 +948,7 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) pci_save_state(pdev); - return rc; + return 0; } static const struct pci_device_id cxl_mem_pci_tbl[] = { From patchwork Fri Nov 29 13:28:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Li Ming X-Patchwork-Id: 13888686 Received: from sender4-pp-o95.zoho.com (sender4-pp-o95.zoho.com [136.143.188.95]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05FBC1A38C2; Fri, 29 Nov 2024 13:28:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=pass smtp.client-ip=136.143.188.95 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732886935; cv=pass; b=p/PIgzhJ7UXacVGeb4SIzy4jAgSzexNrzkEj9uR+bZyij+c9ruNz8wDuWY1gzK0Etm4RRpCpWMkNqXN0B1JHuQc9LYs3/Zs6vPrbY7RlXMAXSYDh5hsO3JSpDpVYtA45yVy/rB/UUdu3S9MO6A2Ciojkc5Ihr7pTpFeu3RHCJuI= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732886935; c=relaxed/simple; bh=wy6gqQcFEFaVjfqLlyCe9evK3Ai+1LDLdq5P0+hNfXU=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=D/SHTuxNwr4H/gjjYknqE8SKulL5wvTEOHCmbtD8yfVayip37YoFCDKnFSawbFWRRq22dR7Sx6AlaOYLYC4o/Lo/5TLfziiTDF+z9+o0Ns4GGn89etYvt0EK2gqJxFv6gUgyQhEWAr/0tZS80cvC3nZOV37XsWa6jpjW6I3vjZE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com; spf=pass smtp.mailfrom=zohomail.com; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b=Ck2M9hJ9; arc=pass smtp.client-ip=136.143.188.95 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=zohomail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=zohomail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=zohomail.com header.i=ming.li@zohomail.com header.b="Ck2M9hJ9" ARC-Seal: i=1; a=rsa-sha256; t=1732886917; cv=none; d=zohomail.com; s=zohoarc; b=MV7waX7Feug8VctlviINNLAXbACXy9WLqylSe7mo2MO41rY65LyhTqv8mLynGUxn63NaNSMIAy5ML3tbJ+3Aoann6n2uALQ5H+cYPSKqwivF+N4kLvFIdZFL3GarYoeAJ3RII3KwxMZ4BrgvKT5CLJrH1wpK/aNcggW5u9Rsn2w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1732886917; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=AlThrPchPL0Jpq0oUlqE/avwqc2LgQC4NFI+hCPhZao=; b=KUA0EhADhIDpcGGSCrQQkyMHkHyeqqTh0HCVBaFcp5HaPUVVawkPI2MvGU4x2KxiEPrbcVj6ru5K5fZRvuKR2dlTR3FHNrYBdBFZu1NsRVAUkTcMQBF6j2VIjV5LmEMsRYwWIrgdKlYZMbrA9bE1fhvoxkd+X5O/CpGMPThRGPg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=zohomail.com; spf=pass smtp.mailfrom=ming.li@zohomail.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1732886917; s=zm2022; d=zohomail.com; i=ming.li@zohomail.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-Id:Message-Id:MIME-Version:Content-Transfer-Encoding:Feedback-ID:Reply-To; bh=AlThrPchPL0Jpq0oUlqE/avwqc2LgQC4NFI+hCPhZao=; b=Ck2M9hJ9ZWUnwOhay36K0/8r+Uan/CmG1TZyiQUekx84M95uOvLpJf0to+gkOsFy LndYRiKiZfJSkK6C1KYjYffuLll6o7XEwX8tTbHZs+TcGR/UdSYmzv58h80ZEzLQViO RCTHrcs1sAtn+BcfEBGk1qXYSLTUkXTNL0o8CK4A= Received: by mx.zohomail.com with SMTPS id 1732886915948639.0421116615139; Fri, 29 Nov 2024 05:28:35 -0800 (PST) From: Li Ming To: kobayashi.da-06@fujitsu.com, dave@stgolabs.net, jonathan.cameron@huawei.com, dave.jiang@intel.com, alison.schofield@intel.com, vishal.l.verma@intel.com, ira.weiny@intel.com, dan.j.williams@intel.com Cc: linux-cxl@vger.kernel.org, linux-kernel@vger.kernel.org, Li Ming Subject: [PATCH 1/1] cxl/pci: Check dport->regs.rcd_pcie_cap availability before accessing Date: Fri, 29 Nov 2024 21:28:25 +0800 Message-Id: <20241129132825.569237-1-ming.li@zohomail.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Feedback-ID: rr080112271204b56560940fc1fb4815dc00005b0ba9c547ac05fcf8edf5d86c3b5608c27b4fe2934730b37d:zu08011227f38e96589e94b8178385bb7700003290854c15527369e4d62d22afaa7dc5d268e6607a61167b15:rf0801122683064ef6b684090f8f9c5da900006d5da07f2b1217700b4ad7b2de7aaff1798b3b7563e8862b:ZohoMail X-ZohoMailClient: External RCD Upstream Port's PCI Express Capability is a component registers block stored in RCD Upstream Port RCRB. CXL PCI driver helps to map it during the RCD probing, but mapping failure is allowed for component registers blocks in CXL PCI driver. dport->regs.rcd_pcie_cap is used to store the virtual address of the RCD Upstream Port's PCI Express Capability, add a dport->regs.rcd_pcie_cap checking in rcd_pcie_cap_emit() just in case user accesses a invalid address via RCD sysfs. Fixes: c5eaec79fa43 ("cxl/pci: Add sysfs attribute for CXL 1.1 device link status") Signed-off-by: Li Ming Reviewed-by: Alison Schofield Reviewed-by: Dan Williams --- drivers/cxl/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index b2cb81f6d9e7..e53b1c95a248 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -836,6 +836,9 @@ static ssize_t rcd_pcie_cap_emit(struct device *dev, u16 offset, char *buf, size if (!root_dev) return -ENXIO; + if (!dport->regs.rcd_pcie_cap) + return -ENXIO; + guard(device)(root_dev); if (!root_dev->driver) return -ENXIO;