From patchwork Thu May 2 04:54:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651279 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 325D71C2AF for ; Thu, 2 May 2024 04:54:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625659; cv=none; b=YdM3zhji2q3i2XBdALM+KWaVA5MFmKD+kNrrvjdixXCi9kBebql9TaWK3aADXld8Zi90sgJ1iw+/FbElMez4yO9jHPC/vz7vssaTeLoCRO8BxJalEnyKDasDxc6zzDikk4+aL3l6QfjwmENm6cupHl/rvMpCViSax671FgjpU64= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625659; c=relaxed/simple; bh=65/iGdiJmESR6Y+bbIicl/WBzYNXyKpyKTLjRzyDAyk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RSlx4bUpWGa9dXJBQHhsS7lmpYUhP5k6z54I5u40Pim9Y7I1sV7kPj7UqLkQrBypPGq4oldBLyW/dk0PaEQIajiabhTQLpPABrfbqM8nyurjhafLdqXqk+bh8BFvulxmUYu6hSoDlmvuzPPz9uVmrFtQoJNF6iF8AgJsdvwEG7M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=KZPeEJzH; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="KZPeEJzH" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-1e65a1370b7so69865165ad.3 for ; Wed, 01 May 2024 21:54:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625657; x=1715230457; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FSsbakNpU8SVJbOYsOnVPvnpJ6qrM0BD4y0o4BP6NEs=; b=KZPeEJzHIjqZ3NGW04I6MiSgs26noLKna2jusqDcwyVc4W7dHSamHDeng9Ald4+Ghv Spu1mj8Pub3G2MqEnpJDpLrytGfpw3qpe+JPyCT31vWlRmCUIKttr7b3Cp0hIR0s+tcr CjlRF2tzQG77c+fnyzghjVyKk2ljx1h1IA81PR6TJfmruSJ8iPT4njBX1LZho/LpqUYi ZJmRPczTeXEbwq0GFuhfuluFA/146cwZfO//EfGRSGCIx7mKNSlKKPHeoezyDO/y7lkW WGUspfriQ1fWdyYtMRP2wk3YSMwrrZG7L/Sj9L/S5RUd/gyMVbsw1xAAmlbIkYAdg3oC G0bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625657; x=1715230457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FSsbakNpU8SVJbOYsOnVPvnpJ6qrM0BD4y0o4BP6NEs=; b=kc44aOQPhxX9p9jc6kDnMaCgOnepQsHrpmS9adeRW2C06VWj6uDHPdqGZwgpokUB+F LGpGLnf/6/xYv9uU3yQJZz8e0y5NzeEeh922ozBjrv8OMudTxO2itvBzxoGO1iK0yQfA 3/qF4jSpYkj0UHKf5Xx4bwc3Qq+cc5IT9SGGhph4piSesxqprbFtJOKuOYtVW/7XmKbd gJu5fG4ml+OuJ+5o6t3gN1h5RGDEfWcLth2xcSMYY3qHwYkAC+sGsp+CSaYU2HliQkxn /7pWujMVuAjhjXr9izziMF0iC7E/b+s6gXcj1oxmz44iYl9liR+VME+scnBMtILyPeMa 2lWQ== X-Gm-Message-State: AOJu0YxflXtNfIwasZHf7UHyzWzqqLYrt8LIudZimMDMSx7NZzFNCSfH Eod07ChdTMs9DdhmYZHHMGsAVgzA3CBhdVZLAdv8gTftPJOAd4JrJl35BQugKCatMYHroBcv9FF v X-Google-Smtp-Source: AGHT+IF1VuzemZitQ3fVfKfdvXvo06YjOrksixJiB3+ocq60Am01w4GNNdtkGH9gCNDSDRVwf2xPhg== X-Received: by 2002:a17:902:e9d5:b0:1e0:b689:950b with SMTP id 21-20020a170902e9d500b001e0b689950bmr964242plk.16.1714625657223; Wed, 01 May 2024 21:54:17 -0700 (PDT) Received: from localhost (fwdproxy-prn-013.fbsv.net. [2a03:2880:ff:d::face:b00c]) by smtp.gmail.com with ESMTPSA id kw4-20020a170902f90400b001e47972a2casm248949plb.96.2024.05.01.21.54.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:16 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 1/9] queue_api: define queue api Date: Wed, 1 May 2024 21:54:02 -0700 Message-ID: <20240502045410.3524155-2-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Mina Almasry This API enables the net stack to reset the queues used for devmem TCP. Signed-off-by: Mina Almasry Signed-off-by: David Wei --- include/linux/netdevice.h | 3 +++ include/net/netdev_queues.h | 27 +++++++++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index f849e7d110ed..6a58ec73c5e8 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -1957,6 +1957,7 @@ enum netdev_reg_state { * @sysfs_rx_queue_group: Space for optional per-rx queue attributes * @rtnl_link_ops: Rtnl_link_ops * @stat_ops: Optional ops for queue-aware statistics + * @queue_mgmt_ops: Optional ops for queue management * * @gso_max_size: Maximum size of generic segmentation offload * @tso_max_size: Device (as in HW) limit on the max TSO request size @@ -2340,6 +2341,8 @@ struct net_device { const struct netdev_stat_ops *stat_ops; + const struct netdev_queue_mgmt_ops *queue_mgmt_ops; + /* for setting kernel sock attribute on TCP connection setup */ #define GSO_MAX_SEGS 65535u #define GSO_LEGACY_MAX_SIZE 65536u diff --git a/include/net/netdev_queues.h b/include/net/netdev_queues.h index 1ec408585373..58042957c39f 100644 --- a/include/net/netdev_queues.h +++ b/include/net/netdev_queues.h @@ -60,6 +60,33 @@ struct netdev_stat_ops { struct netdev_queue_stats_tx *tx); }; +/** + * struct netdev_queue_mgmt_ops - netdev ops for queue management + * + * @ndo_queue_mem_alloc: Allocate memory for an RX queue. The memory returned + * in the form of a void* can be passed to + * ndo_queue_mem_free() for freeing or to ndo_queue_start + * to create an RX queue with this memory. + * + * @ndo_queue_mem_free: Free memory from an RX queue. + * + * @ndo_queue_start: Start an RX queue at the specified index. + * + * @ndo_queue_stop: Stop the RX queue at the specified index. + */ +struct netdev_queue_mgmt_ops { + void * (*ndo_queue_mem_alloc)(struct net_device *dev, + int idx); + void (*ndo_queue_mem_free)(struct net_device *dev, + void *queue_mem); + int (*ndo_queue_start)(struct net_device *dev, + int idx, + void *queue_mem); + int (*ndo_queue_stop)(struct net_device *dev, + int idx, + void **out_queue_mem); +}; + /** * DOC: Lockless queue stopping / waking helpers. * From patchwork Thu May 2 04:54:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651280 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com [209.85.167.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 686E51CAB3 for ; Thu, 2 May 2024 04:54:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625660; cv=none; b=OFQOjMqq590kJybgb4+DoHQ+DeaHBYHO5Mcx3cLIejocY3Qxk+sfMbTM4L2dzNaUPbHjYfdaJbnTDUZLlVVsgrKMX/bQQk/xgebmbQGgf/BxvoG2GPxsNHhkPP31ZBObIqhCRUhSEbkCLNvI7oOI0YVwmNcCviIyhf60YDUMDtE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625660; c=relaxed/simple; bh=ulnp01jdTxgIRCr18Tn2JZfsVZNTyq1zlColj8gCddE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fQJJQ5gWMCsnKoYVSdL9RcDvy4Uc63qnGnDC7hgg08DuLQX0LOKCI/sIPNygzSrGwruG0pc/wtdgOgEUYLZktI/+LVkOJHprlnXwsuMVw4VknNxqA1dOiO9Kp29IYlOfSZykvvPYDiA2HzDQfHfP8f1CsnQmjqwFuW3cPa5cuYY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=gG4R3hbj; arc=none smtp.client-ip=209.85.167.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="gG4R3hbj" Received: by mail-oi1-f172.google.com with SMTP id 5614622812f47-3c6f6c5bc37so3741348b6e.1 for ; Wed, 01 May 2024 21:54:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625658; x=1715230458; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=k3TV5L/JsIHMDJfGRMClntZErKHDDCQVMuxFiLMd8sM=; b=gG4R3hbju0xCAHkyP3aYpxq/b7XtpJYg94GwI6PQnEeBRySVsli1BU6o/US8wVH4XB lB5Ee9ml3etj3xD9nruApCYsX24Av0P4J1W1fwmdsE3wdUhZHk2I0dcZvy/GbC5RLY5o XwrHJdCUTzyRiU0WddVyFGDC95pqBJlT93QtpsuzpNrRy0FvnP+KLJrB2ia8CJ8l/rGF 74TErpg0RfWUA0l2/99QQW3g0j5C2NthwSydMVrt5s3yNfYsA+3e05Pa4qboMY2d3d4V K6FS36d/70sCoP9RhZGZ/mTYt2RXGhJk73eOS9WGB9Le1umAa2vVx9B2OqOke4o4QDGZ 47dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625658; x=1715230458; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k3TV5L/JsIHMDJfGRMClntZErKHDDCQVMuxFiLMd8sM=; b=fuNOTrIhza/FR7uIN4E411kubLlIxfJxLnG3azwvRdDlqRYBGfpJQu+ifr41v8bE2L lRr5H2FiKaoJiBlhXP1nxs0p94Xf1N5yyHQgxTttkwuARagnlsyM2xLEAH6yvodZ1EPj pyTm1vRNWx9x4w7eNWqg5o46DLk6xtbJQoUDhGzlhm0fSD16xKfGkLX1gmjLeRzUoWgq rfhsmvb0QcrxTeFi9xBm3U8IvVhDl0tMDArQBPYf5J2BbeAF+Hbk/5Yu/kH/svJ/5TYt u49YWMB36Vuj4Okg/7SB8BhpKlp9swQB6yxAiaD6VGND2Q+VO2N9sjgPULVuU9drNbsK 0kPw== X-Gm-Message-State: AOJu0YyuyaBR/QIe5bZ/uklmxernna8x1LWmj+1Ajmdci8Gnp5FQdiwj lfFeW59w4ZBGoDprzVClyFW79HmBTvicjkpc8rjYZ4PsgwOQD0lz3Aew8pOIkEOXRmX+HEsF7Aq j X-Google-Smtp-Source: AGHT+IEvCeEbgqLTRLowRumhGOsWHRrx57Trlg/djFD+IqTr68v83s/RuCuSKv9ehirZTergCFAOYg== X-Received: by 2002:a05:6808:9a5:b0:3c8:47a3:3d06 with SMTP id e5-20020a05680809a500b003c847a33d06mr4907774oig.33.1714625658226; Wed, 01 May 2024 21:54:18 -0700 (PDT) Received: from localhost (fwdproxy-prn-005.fbsv.net. [2a03:2880:ff:5::face:b00c]) by smtp.gmail.com with ESMTPSA id f9-20020a056a001ac900b006f09d5807ebsm269541pfv.82.2024.05.01.21.54.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:17 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 2/9] bnxt: implement queue api Date: Wed, 1 May 2024 21:54:03 -0700 Message-ID: <20240502045410.3524155-3-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Implement the bare minimum queue API for bnxt. I'm essentially breaking up the existing bnxt_rx_ring_reset() into two steps: 1. bnxt_queue_stop() 2. bnxt_queue_start() The other two ndos are left as no-ops for now, so the queue mem is allocated after the queue has been stopped. Doing this before stopping the queue is a lot more work, so I'm looking for some feedback first. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 62 +++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 06b7a963bbbd..788c87271eb1 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -14810,6 +14810,67 @@ static const struct netdev_stat_ops bnxt_stat_ops = { .get_base_stats = bnxt_get_base_stats, }; +static void *bnxt_queue_mem_alloc(struct net_device *dev, int idx) +{ + struct bnxt *bp = netdev_priv(dev); + + return &bp->rx_ring[idx]; +} + +static void bnxt_queue_mem_free(struct net_device *dev, void *qmem) +{ +} + +static int bnxt_queue_start(struct net_device *dev, int idx, void *qmem) +{ + struct bnxt_rx_ring_info *rxr = qmem; + struct bnxt *bp = netdev_priv(dev); + + bnxt_alloc_one_rx_ring(bp, idx); + + if (bp->flags & BNXT_FLAG_AGG_RINGS) + bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); + bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); + + if (bp->flags & BNXT_FLAG_TPA) + bnxt_set_tpa(bp, true); + + return 0; +} + +static int bnxt_queue_stop(struct net_device *dev, int idx, void **out_qmem) +{ + struct bnxt *bp = netdev_priv(dev); + struct bnxt_rx_ring_info *rxr; + struct bnxt_cp_ring_info *cpr; + int rc; + + rc = bnxt_hwrm_rx_ring_reset(bp, idx); + if (rc) + return rc; + + bnxt_free_one_rx_ring_skbs(bp, idx); + rxr = &bp->rx_ring[idx]; + rxr->rx_prod = 0; + rxr->rx_agg_prod = 0; + rxr->rx_sw_agg_prod = 0; + rxr->rx_next_cons = 0; + + cpr = &rxr->bnapi->cp_ring; + cpr->sw_stats.rx.rx_resets++; + + *out_qmem = rxr; + + return 0; +} + +static const struct netdev_queue_mgmt_ops bnxt_queue_mgmt_ops = { + .ndo_queue_mem_alloc = bnxt_queue_mem_alloc, + .ndo_queue_mem_free = bnxt_queue_mem_free, + .ndo_queue_start = bnxt_queue_start, + .ndo_queue_stop = bnxt_queue_stop, +}; + static void bnxt_remove_one(struct pci_dev *pdev) { struct net_device *dev = pci_get_drvdata(pdev); @@ -15275,6 +15336,7 @@ static int bnxt_init_one(struct pci_dev *pdev, const struct pci_device_id *ent) dev->stat_ops = &bnxt_stat_ops; dev->watchdog_timeo = BNXT_TX_TIMEOUT; dev->ethtool_ops = &bnxt_ethtool_ops; + dev->queue_mgmt_ops = &bnxt_queue_mgmt_ops; pci_set_drvdata(pdev, dev); rc = bnxt_alloc_hwrm_resources(bp); From patchwork Thu May 2 04:54:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651281 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 478FA1DDD1 for ; Thu, 2 May 2024 04:54:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625661; cv=none; b=Rlqhl3WlGb7cz+syTOb+wauA8RKxWc50LRzb052UqSZsrSsDrCujbPTXGZ/lJmMQpBwZP4E5W3M6Bg39sb34I7Gd3b1ZiJZfdUfc8vtCgFaYVKV5LLMAeiwBRW+HU+B12aQ9H88PybAAOek3VHgbS5SjNzzMJ1anxYiXQiKRZlY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625661; c=relaxed/simple; bh=OYY5KT41NM5uN1HZKt+u1zzFIxSC++IQ3CLaeU7YVME=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ovbUcaSvYsIDprb3+3JNNqqKhMRPcMHMiP952/oKVtN+fubQ7OtrWqT4YFB0WXXuZ22A0L1+KTfiLVMCM1ixRb46ZW+tfaoAitXdu0X1SAex5ryY8dofYhLz9WTWR1tyQ0PMrgsE+nSBFpdGByyqEOHKdb2BGjik69L0WeLiIoA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=dmGMAOLE; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="dmGMAOLE" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-1ec486198b6so20625405ad.1 for ; Wed, 01 May 2024 21:54:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625659; x=1715230459; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=V/ZJcAuW84ecF6scZWO7f0oLtlZZw7KnY0k3vPst4TE=; b=dmGMAOLERUUOaSsdIHxM8ligMMrHJHN1qRzGLD5/uZOcnYzZsE1HjKlWrK+rYkA06v F26BixrqdJP8hJwVS3iRtPcl2ajGy44YbN9v0mCrIz6jegce97syZ9JNkz7GTNZDg9Yv jMaiJ3dwOYDqDhVKpPdlANYTYBCOd7Hg7RMQbF+D4IYLc6hH5HuQk0LlP7jLWL7x+DeX ZjsSdIM20I4cUdG84jNw2QEjpKSVr8ajWjVqzW0Js3GwRzkgaG+5McoA2GO/MWVgvG3d D1gs4zPk8BufI8SsL8gFaWfQqUb8J9MB0zOASConfzWt7Jdngk3JwS+5XA8uhJpOzeor XdNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625659; x=1715230459; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=V/ZJcAuW84ecF6scZWO7f0oLtlZZw7KnY0k3vPst4TE=; b=Q1JXdZuTVjy+XvMa0+UpMZgvURVcpm/6x6LrqzxfkUn5rBonsWSqxWTZX9IY29WQP2 dBGtHOh4zm88FuW58TXCp8WgyicY6J57PLj4AXmssoM7YOEh9ME0SYM2hSl6GhAmZa5d zB9aEr0EzxBr3/1olZ+ccYicvNf6/OlDQAAgd18dV8pNHOylOWhI2Ixn6hyK3fc1ahH0 namrP/k4rNEUdO6IqmjrdQIi5J28qAn3dDyLAh6pUcagGcUuP3ioNY6FIbJ3XxRNDS26 T6GP+QeGDQKDSe94C+rNyJCy3aXuYeEtLCaKk2VBgsUoFhv51c08VgHaP88ia0qlc/3D +BOA== X-Gm-Message-State: AOJu0YyAH427u8FSvTfgn7NhnKnlj+Y6S8cySEeBpAZP2yPyvvwueN/n 98UhLUJIsQgK5pBqTvGbe/Jwv7GH16P1hJi/vu7Q2w7+cnfWplKG993IrtQ7o9WVicyqfkxZn+E I X-Google-Smtp-Source: AGHT+IE4GAL9EY+w1IH4d5B9UMs5riu6zXx+a518zfYASI+lqYqd9Q2tCMYOfIjs0kSNp0+SD5Nx+g== X-Received: by 2002:a17:902:ed46:b0:1eb:86d:70cb with SMTP id y6-20020a170902ed4600b001eb086d70cbmr5014933plb.45.1714625659255; Wed, 01 May 2024 21:54:19 -0700 (PDT) Received: from localhost (fwdproxy-prn-006.fbsv.net. [2a03:2880:ff:6::face:b00c]) by smtp.gmail.com with ESMTPSA id n8-20020a170903110800b001e22860c32asm243800plh.143.2024.05.01.21.54.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:19 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 3/9] netdev: add netdev_rx_queue_restart() Date: Wed, 1 May 2024 21:54:04 -0700 Message-ID: <20240502045410.3524155-4-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC From: Mina Almasry Add netdev_rx_queue_restart() function to netdev_rx_queue.h. This is taken from Mina's work in [1] with a slight modification of taking rtnl_lock() during the queue stop and start ops. For bnxt specifically, if the firmware doesn't support BNXT_RST_RING_SP_EVENT, then ndo_queue_stop() returns -EOPNOTSUPP and the whole restart fails. Unlike bnxt_rx_ring_reset(), there is no attempt to reset the whole device. [1]: https://lore.kernel.org/linux-kernel/20240403002053.2376017-6-almasrymina@google.com/#t Signed-off-by: David Wei --- include/net/netdev_rx_queue.h | 3 ++ net/core/Makefile | 1 + net/core/netdev_rx_queue.c | 58 +++++++++++++++++++++++++++++++++++ 3 files changed, 62 insertions(+) create mode 100644 net/core/netdev_rx_queue.c diff --git a/include/net/netdev_rx_queue.h b/include/net/netdev_rx_queue.h index aa1716fb0e53..e78ca52d67fb 100644 --- a/include/net/netdev_rx_queue.h +++ b/include/net/netdev_rx_queue.h @@ -54,4 +54,7 @@ get_netdev_rx_queue_index(struct netdev_rx_queue *queue) return index; } #endif + +int netdev_rx_queue_restart(struct net_device *dev, unsigned int rxq); + #endif diff --git a/net/core/Makefile b/net/core/Makefile index 21d6fbc7e884..f2aa63c167a3 100644 --- a/net/core/Makefile +++ b/net/core/Makefile @@ -19,6 +19,7 @@ obj-$(CONFIG_NETDEV_ADDR_LIST_TEST) += dev_addr_lists_test.o obj-y += net-sysfs.o obj-y += hotdata.o +obj-y += netdev_rx_queue.o obj-$(CONFIG_PAGE_POOL) += page_pool.o page_pool_user.o obj-$(CONFIG_PROC_FS) += net-procfs.o obj-$(CONFIG_NET_PKTGEN) += pktgen.o diff --git a/net/core/netdev_rx_queue.c b/net/core/netdev_rx_queue.c new file mode 100644 index 000000000000..9633fb36f6d1 --- /dev/null +++ b/net/core/netdev_rx_queue.c @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include +#include +#include + +int netdev_rx_queue_restart(struct net_device *dev, unsigned int rxq) +{ + void *new_mem; + void *old_mem; + int err; + + if (!dev->queue_mgmt_ops->ndo_queue_stop || + !dev->queue_mgmt_ops->ndo_queue_mem_free || + !dev->queue_mgmt_ops->ndo_queue_mem_alloc || + !dev->queue_mgmt_ops->ndo_queue_start) + return -EOPNOTSUPP; + + new_mem = dev->queue_mgmt_ops->ndo_queue_mem_alloc(dev, rxq); + if (!new_mem) + return -ENOMEM; + + rtnl_lock(); + err = dev->queue_mgmt_ops->ndo_queue_stop(dev, rxq, &old_mem); + if (err) + goto err_free_new_mem; + + err = dev->queue_mgmt_ops->ndo_queue_start(dev, rxq, new_mem); + if (err) + goto err_start_queue; + rtnl_unlock(); + + dev->queue_mgmt_ops->ndo_queue_mem_free(dev, old_mem); + + return 0; + +err_start_queue: + /* Restarting the queue with old_mem should be successful as we haven't + * changed any of the queue configuration, and there is not much we can + * do to recover from a failure here. + * + * WARN if the we fail to recover the old rx queue, and at least free + * old_mem so we don't also leak that. + */ + if (dev->queue_mgmt_ops->ndo_queue_start(dev, rxq, old_mem)) { + WARN(1, + "Failed to restart old queue in error path. RX queue %d may be unhealthy.", + rxq); + dev->queue_mgmt_ops->ndo_queue_mem_free(dev, &old_mem); + } + +err_free_new_mem: + dev->queue_mgmt_ops->ndo_queue_mem_free(dev, new_mem); + rtnl_unlock(); + + return err; +} +EXPORT_SYMBOL_GPL(netdev_rx_queue_restart); From patchwork Thu May 2 04:54:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651282 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pf1-f179.google.com (mail-pf1-f179.google.com [209.85.210.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1D08C1F95E for ; Thu, 2 May 2024 04:54:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625662; cv=none; b=WiC/0v88WMnooxq/Jl9fXotIuCiwxZIz5i1X8jL1yXYeZYd16E76xsZORjyK72zemFUIx6p0pUpGUxCGSvQ3awQ/bfNfYdiA/V3qisr1NDgmCRm6Dj+3ON4xFNKk7hrupboa80yVK4S64NhUmwr4c24dUCnRlb+Wx+eKMMD2DPU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625662; c=relaxed/simple; bh=suisxc1RtAB4hm7382CkzuinVaWy4Upjpyn+tyuukDI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TCy2en3fzmdYfP69cEtjOz+VBULsLG08s8u16vHhoHmmsMwjq8CAGl1SSsy+r6D9FX7r1MKrqYRNpL6dUIamQkqUfC2euTB42ud9jvkZqU2gX238ppvbVV9tTsue65MsaE/KqB8Lz/fl3otESUsvbwN9uiJVU41C6KIFDJ3voFU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=BbEB2JOG; arc=none smtp.client-ip=209.85.210.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="BbEB2JOG" Received: by mail-pf1-f179.google.com with SMTP id d2e1a72fcca58-6ee13f19e7eso6898106b3a.1 for ; Wed, 01 May 2024 21:54:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625660; x=1715230460; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=JbEkwCacQh5bYWcqD2XE9ClJ1unjel5HXLhJbkE5ieU=; b=BbEB2JOGWPq0c0NkGgKgVTHuQ1SBHIipX2koOXH6aPceKuthhZNG98TxM80IQ8lIlS fHZbmlLf2vtJtIgXqp+fjB5cMO00XjxGxZsgCPhzrJG4Bgttqie3+5WdOCTdSEr8LSxy 2Z87EG/kDnpHuDAWTyL/DbBtf9DPKbcSpiWiV9flArxFgqW+2kP8AbDj1JUi6gonfca8 TTY3MAdkVE68Uz7xdmcNwP33dMwIcJyprXZwUSBhS4zIW77gQ7NmTyM98rLy0yfhFvgc wYUM7EkrEv88hXSKwAAlEnxap2qs31oiOFVxPMsV0fYa/C+hUUX/9N/s7svngBwuJEbR Ivjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625660; x=1715230460; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JbEkwCacQh5bYWcqD2XE9ClJ1unjel5HXLhJbkE5ieU=; b=fcL/iKc0SVN2kLeKUp2DFeImn5XxCUD9ltuBFW9l3/MqjAK58tSSqXF4kJOsM5bwLv ++aCYXx63yn5E/kOoED7dd+1CDYB8wmDFpNzauHKFUjiXjj0hiXZhEnuVB4OmUl/aXol Q6bQrHTfdFxQtPDdjNHgXxrpvJVAFiJSOBXjz/EWl6U2O0QUDIoiUYh/uHtw5uEsevzx HTyaUSZNt3jZDjtkH3Nyq/Lzl7LTUjabDxmmJvYxYMQQ9tUVxDHQICoHHHMKQIYiizNM 07z9xTEYgI7NPL3h291DLPfZbUbcK/RBn5JxeWB8P6Di50KqKSizRJWUGIO5UONPJrRx pR6Q== X-Gm-Message-State: AOJu0Yw7eH8AlZ+VvLQ+HdnWeCxQNwDX8ENatnO1S2ndAp9kToWggmY5 YtkkvEBNtDToto7e2LkHoXPTQUQLaJRg3trh85VYrS9ED9JGaj5fXqMVIjQ9AarU6bVoxkYIFzn l X-Google-Smtp-Source: AGHT+IHV6c9y1O7oo6L+70KpzjKtc7PSx2yLx6mRf58gYHFSJlu0e3IirI36S42BgFoyBYt67mF4rQ== X-Received: by 2002:a05:6a20:f3aa:b0:1af:667c:38cc with SMTP id qr42-20020a056a20f3aa00b001af667c38ccmr4690960pzb.5.1714625660221; Wed, 01 May 2024 21:54:20 -0700 (PDT) Received: from localhost (fwdproxy-prn-017.fbsv.net. [2a03:2880:ff:11::face:b00c]) by smtp.gmail.com with ESMTPSA id iw14-20020a170903044e00b001ec80ed863asm244526plb.152.2024.05.01.21.54.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:20 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 4/9] bnxt: refactor bnxt_{alloc,free}_one_rx_ring() Date: Wed, 1 May 2024 21:54:05 -0700 Message-ID: <20240502045410.3524155-5-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Refactor bnxt_free_one_rx_ring_skbs() and bnxt_alloc_one_rx_ring() to take the rx ring directly as a parameter instead of an index. This makes the functions usable with an rx ring that is allocated outside of bp->rx_ring. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 788c87271eb1..7b20303f3b7d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3307,9 +3307,8 @@ static void bnxt_free_tx_skbs(struct bnxt *bp) } } -static void bnxt_free_one_rx_ring_skbs(struct bnxt *bp, int ring_nr) +static void bnxt_free_one_rx_ring_skbs(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) { - struct bnxt_rx_ring_info *rxr = &bp->rx_ring[ring_nr]; struct pci_dev *pdev = bp->pdev; struct bnxt_tpa_idx_map *map; int i, max_idx, max_agg_idx; @@ -3389,7 +3388,7 @@ static void bnxt_free_rx_skbs(struct bnxt *bp) return; for (i = 0; i < bp->rx_nr_rings; i++) - bnxt_free_one_rx_ring_skbs(bp, i); + bnxt_free_one_rx_ring_skbs(bp, &bp->rx_ring[i]); } static void bnxt_free_skbs(struct bnxt *bp) @@ -4051,9 +4050,8 @@ static void bnxt_init_rxbd_pages(struct bnxt_ring_struct *ring, u32 type) } } -static int bnxt_alloc_one_rx_ring(struct bnxt *bp, int ring_nr) +static int bnxt_alloc_one_rx_ring(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) { - struct bnxt_rx_ring_info *rxr = &bp->rx_ring[ring_nr]; struct net_device *dev = bp->dev; u32 prod; int i; @@ -4062,7 +4060,7 @@ static int bnxt_alloc_one_rx_ring(struct bnxt *bp, int ring_nr) for (i = 0; i < bp->rx_ring_size; i++) { if (bnxt_alloc_rx_data(bp, rxr, prod, GFP_KERNEL)) { netdev_warn(dev, "init'ed rx ring %d with %d/%d skbs only\n", - ring_nr, i, bp->rx_ring_size); + rxr->bnapi->index, i, bp->rx_ring_size); break; } prod = NEXT_RX(prod); @@ -4076,7 +4074,7 @@ static int bnxt_alloc_one_rx_ring(struct bnxt *bp, int ring_nr) for (i = 0; i < bp->rx_agg_ring_size; i++) { if (bnxt_alloc_rx_page(bp, rxr, prod, GFP_KERNEL)) { netdev_warn(dev, "init'ed rx ring %d with %d/%d pages only\n", - ring_nr, i, bp->rx_ring_size); + rxr->bnapi->index, i, bp->rx_ring_size); break; } prod = NEXT_RX_AGG(prod); @@ -4135,7 +4133,7 @@ static int bnxt_init_one_rx_ring(struct bnxt *bp, int ring_nr) bnxt_init_rxbd_pages(ring, type); } - return bnxt_alloc_one_rx_ring(bp, ring_nr); + return bnxt_alloc_one_rx_ring(bp, rxr); } static void bnxt_init_cp_rings(struct bnxt *bp) @@ -13238,13 +13236,13 @@ static void bnxt_rx_ring_reset(struct bnxt *bp) bnxt_reset_task(bp, true); break; } - bnxt_free_one_rx_ring_skbs(bp, i); + bnxt_free_one_rx_ring_skbs(bp, rxr); rxr->rx_prod = 0; rxr->rx_agg_prod = 0; rxr->rx_sw_agg_prod = 0; rxr->rx_next_cons = 0; rxr->bnapi->in_reset = false; - bnxt_alloc_one_rx_ring(bp, i); + bnxt_alloc_one_rx_ring(bp, rxr); cpr = &rxr->bnapi->cp_ring; cpr->sw_stats.rx.rx_resets++; if (bp->flags & BNXT_FLAG_AGG_RINGS) @@ -14826,7 +14824,7 @@ static int bnxt_queue_start(struct net_device *dev, int idx, void *qmem) struct bnxt_rx_ring_info *rxr = qmem; struct bnxt *bp = netdev_priv(dev); - bnxt_alloc_one_rx_ring(bp, idx); + bnxt_alloc_one_rx_ring(bp, rxr); if (bp->flags & BNXT_FLAG_AGG_RINGS) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); @@ -14849,8 +14847,8 @@ static int bnxt_queue_stop(struct net_device *dev, int idx, void **out_qmem) if (rc) return rc; - bnxt_free_one_rx_ring_skbs(bp, idx); rxr = &bp->rx_ring[idx]; + bnxt_free_one_rx_ring_skbs(bp, rxr); rxr->rx_prod = 0; rxr->rx_agg_prod = 0; rxr->rx_sw_agg_prod = 0; From patchwork Thu May 2 04:54:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651283 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1134F208A9 for ; Thu, 2 May 2024 04:54:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625663; cv=none; b=gByKhb8HGLOXi5QpzjXqCoIkXbL+Ted0KvhanwOlSfpdRRLWnnm3c7cozLQhpAl90q11jzry1bO+77zrXrJXNM4LnI6bY8rtCZ25M8vL55A9eqGsnNGdXyg0oqgw6oHL/fEj14qyFOl7IS33KpuPx0gKkf8SLr0GC+4Vjxno4eQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625663; c=relaxed/simple; bh=ZHF4Go+nOkdq39sKD/sqLIpOLW/meubx55Acw8RoEYg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H0L+QdWFhP1tKGfbJ3Qi+eWhaXuFxGuWLBv9yuWK3XLVYcGBbs0QjBxGnnSloGLVTIxuFGEOKqfF/Wf2p6odTkzJseiI/D2B4FnsFO98PXCoj2JZ4iwXrasjunF8MDPOlJracY8GANG0JK6/+nOWpZC7g9oFAsnT6YV8NP4jJEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=lB4tP0Gk; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="lB4tP0Gk" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-1e4f341330fso69475805ad.0 for ; Wed, 01 May 2024 21:54:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625661; x=1715230461; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2zYfD4DNKBLYCTzuc2jEB4CxeXC1l1DUWPGVBNCSW3Y=; b=lB4tP0Gkzh8X5wjVVrRpfn/9K2YCQmYP1mMcEv6b6rTfeeVPAiuLAkVWYkSjaejNgT T8Foec8f3O5AFX0RHIwv7nMTWlDypEaagNUBjkDxFyhskK6xP0MsCepWNL37wooGIN0T j54egwYXZ7XvXfLAoicDiljwcyiZvEu4pm6N/7twX7vsz083mybjXP2LEXmeaRmyBaPI v0OFbg5AYkD7UlwIlI2fkDiHgP8cGBgmKK2Z7pbOJXMvxgDmGBZzDAhZbeasnZ5ns/WF Ixp3Uze+pbpSd/W5VZb5DFi9pbQavJEQ8ySHtSzC55fJOs3L3xQtJzkPBaQnkVN2A4uz 0XJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625661; x=1715230461; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2zYfD4DNKBLYCTzuc2jEB4CxeXC1l1DUWPGVBNCSW3Y=; b=ZExGmYoaX8Y+c4TYkwAOd+SlgZ+hzk7+xoPHC2XeuAEMYy9vMuFtbYpLtjz3HEgW4J V5PE+3GBTiGodz+4YajZSf1fnAE+mdVKR54vrV/GW7qGPE8xoaDCeu6XWUKiNHnLln9V GitJb9oWZkk/2Rl0ZNeopYl2hfHl9HDGriqZwyG6OTlyv9n51FinfrmTLB4/be82ehrR z8sqQlV76/gL0ElqT6pXLGR5nlodHtJw75GpQqqjQ1mBhJR+o7sVy9BSBUdNctnN8jwl UllKsnb8cH2LRqMCdBuIHGzKgq4l50hcjzTLAaYGxdlRX/jhwIYubDup4YRn6mZQBca6 KTNw== X-Gm-Message-State: AOJu0Yx3e0C99qdbTaWevLMxviUiNo9FVvL0mQh4kslamAxSE3ZuixGZ hg6VQNN1z89PCM2k80FUbV9Qmh/EL1DhEmF8Sct3I3QXliXj/cvI21h4aOzUS3PmlTV8v0e9MS+ m X-Google-Smtp-Source: AGHT+IE6kZEWK8zvBs8txByjfCvMeugXv3X8x2uCxAwhSB1KLWc1he+GaOUgqw32xHYQLtGaj0O4mA== X-Received: by 2002:a17:902:6bca:b0:1e4:df0c:a570 with SMTP id m10-20020a1709026bca00b001e4df0ca570mr4421353plt.8.1714625661186; Wed, 01 May 2024 21:54:21 -0700 (PDT) Received: from localhost (fwdproxy-prn-014.fbsv.net. [2a03:2880:ff:e::face:b00c]) by smtp.gmail.com with ESMTPSA id x10-20020a170902ec8a00b001eb3dae7ef1sm248395plg.16.2024.05.01.21.54.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:20 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 5/9] bnxt: refactor bnxt_{alloc,free}_one_tpa_info() Date: Wed, 1 May 2024 21:54:06 -0700 Message-ID: <20240502045410.3524155-6-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Refactor the allocation of each rx ring's tpa_info in bnxt_alloc_tpa_info() out into a standalone function __bnxt_alloc_one_tpa_info(). In case of allocation failures during bnxt_alloc_tpa_info(), clean up in-place. Change bnxt_free_tpa_info() to free a single rx ring passed in as a parameter. This makes bnxt_free_rx_rings() more symmetrical. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 95 +++++++++++++++-------- 1 file changed, 62 insertions(+), 33 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7b20303f3b7d..bda49e7f6c3d 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3500,29 +3500,66 @@ static int bnxt_alloc_ring(struct bnxt *bp, struct bnxt_ring_mem_info *rmem) return 0; } -static void bnxt_free_tpa_info(struct bnxt *bp) +static void bnxt_free_tpa_info(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) { - int i, j; + int i; - for (i = 0; i < bp->rx_nr_rings; i++) { - struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i]; + kfree(rxr->rx_tpa_idx_map); + rxr->rx_tpa_idx_map = NULL; + if (rxr->rx_tpa) { + for (i = 0; i < bp->max_tpa; i++) { + kfree(rxr->rx_tpa[i].agg_arr); + rxr->rx_tpa[i].agg_arr = NULL; + } + } + kfree(rxr->rx_tpa); + rxr->rx_tpa = NULL; +} - kfree(rxr->rx_tpa_idx_map); - rxr->rx_tpa_idx_map = NULL; - if (rxr->rx_tpa) { - for (j = 0; j < bp->max_tpa; j++) { - kfree(rxr->rx_tpa[j].agg_arr); - rxr->rx_tpa[j].agg_arr = NULL; - } +static int __bnxt_alloc_one_tpa_info(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) +{ + struct rx_agg_cmp *agg; + int i, rc; + + rxr->rx_tpa = kcalloc(bp->max_tpa, sizeof(struct bnxt_tpa_info), + GFP_KERNEL); + if (!rxr->rx_tpa) + return -ENOMEM; + + if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)) + return 0; + + for (i = 0; i < bp->max_tpa; i++) { + agg = kcalloc(MAX_SKB_FRAGS, sizeof(*agg), GFP_KERNEL); + if (!agg) { + rc = -ENOMEM; + goto err_free; } - kfree(rxr->rx_tpa); - rxr->rx_tpa = NULL; + rxr->rx_tpa[i].agg_arr = agg; + } + rxr->rx_tpa_idx_map = kzalloc(sizeof(*rxr->rx_tpa_idx_map), + GFP_KERNEL); + if (!rxr->rx_tpa_idx_map) { + rc = -ENOMEM; + goto err_free; } + + return 0; + +err_free: + while(i--) { + kfree(rxr->rx_tpa[i].agg_arr); + rxr->rx_tpa[i].agg_arr = NULL; + } + kfree(rxr->rx_tpa); + rxr->rx_tpa = NULL; + + return rc; } static int bnxt_alloc_tpa_info(struct bnxt *bp) { - int i, j; + int i, rc; bp->max_tpa = MAX_TPA; if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { @@ -3533,27 +3570,18 @@ static int bnxt_alloc_tpa_info(struct bnxt *bp) for (i = 0; i < bp->rx_nr_rings; i++) { struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i]; - struct rx_agg_cmp *agg; - - rxr->rx_tpa = kcalloc(bp->max_tpa, sizeof(struct bnxt_tpa_info), - GFP_KERNEL); - if (!rxr->rx_tpa) - return -ENOMEM; - if (!(bp->flags & BNXT_FLAG_CHIP_P5_PLUS)) - continue; - for (j = 0; j < bp->max_tpa; j++) { - agg = kcalloc(MAX_SKB_FRAGS, sizeof(*agg), GFP_KERNEL); - if (!agg) - return -ENOMEM; - rxr->rx_tpa[j].agg_arr = agg; - } - rxr->rx_tpa_idx_map = kzalloc(sizeof(*rxr->rx_tpa_idx_map), - GFP_KERNEL); - if (!rxr->rx_tpa_idx_map) - return -ENOMEM; + rc = __bnxt_alloc_one_tpa_info(bp, rxr); + if (rc) + goto err_free; } return 0; + +err_free: + while (i--) + bnxt_free_tpa_info(bp, &bp->rx_ring[i]); + + return rc; } static void bnxt_free_rx_rings(struct bnxt *bp) @@ -3563,11 +3591,12 @@ static void bnxt_free_rx_rings(struct bnxt *bp) if (!bp->rx_ring) return; - bnxt_free_tpa_info(bp); for (i = 0; i < bp->rx_nr_rings; i++) { struct bnxt_rx_ring_info *rxr = &bp->rx_ring[i]; struct bnxt_ring_struct *ring; + bnxt_free_tpa_info(bp, rxr); + if (rxr->xdp_prog) bpf_prog_put(rxr->xdp_prog); From patchwork Thu May 2 04:54:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651284 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F128620DD2 for ; Thu, 2 May 2024 04:54:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625664; cv=none; b=Yf+Q5BShNPAgIfi4qRpqQulXP2QLAUwCFEbGAlW3xspdWNUgQHq6Fe2JtR8GHmTmosUSodb2na1Xit2aTkCgv7knzskVDGqUCSScpDhZ0GzkghcL6yEKgWat/09s7xEzTXGtxoR9Ra/nDxEHFibekadJUNaCtLWIg38WP47JbSI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625664; c=relaxed/simple; bh=oDHaNd5AziFDac6uLmqQVbFbY+RtD94LtAULAsjEU+M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=THIyfuW0w2dstrPabCGU9Yl6lImS9Qc3x2brqmBVEmwJEkawKkhRJ3/Pq2dN3TEDZnOvGmxe7bLvgT/qAVmj9HjJ2fWkc8iz/KoqCjr2j/b7T0OBASheiBugapD4ydm3oD+pFXTQBylavkDLy++YbWtDpH5qld+Sax5+So5u4Nk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=FBebrXrn; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="FBebrXrn" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-6f07de6ab93so7024556b3a.2 for ; Wed, 01 May 2024 21:54:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625662; x=1715230462; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HXMMmEd8a24CLHsrwT2GRQHruMFZl07cdN6A9cDx06E=; b=FBebrXrnSh14XwahL41mHiHeczzbZbLc/Kpd/0RzK1XipCRHmafw7gFprIjlbue+dV jQL2uWPq+Ic/4fbd2ZTvvAplVhlpVHZiHBi6ZMNGohynrjjuuxGzUWaFAbs6pA4yrjCG n4fYMWJd4x41gueU1ayAAXtH3xLTdnnXjnXbeh0yW9jMMoGi5Sqo10R+dIbkeVVHWBsE ecOJ5SUGjWq1EECYCHsfu3eyXbgePlCgbfg1vc3gu2TwqBiI/Cdw81gdw04NT3QEYoZA Keg5kOBF/6x49OM3m+O6Ng+A4narKQQb6GrwLoXKmpRd9cA1WjPbZieFxX/IM+a7Q77x Ds1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625662; x=1715230462; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HXMMmEd8a24CLHsrwT2GRQHruMFZl07cdN6A9cDx06E=; b=HTaPkgvSXbFErQ8QR5W8YhWTBT3eNfCiuDnHbcmZ1K4d/wUJjJT58chwKBTsnhEBXe Q0m4Nnr4CHpwJ1nLoqFUd3VY5cwtvgZWZg6ZwY0Iv9fIy/P7dpHC9a9qjpzjs5jx53nc NP2DGPJ3oFBavHKdh7bHA51GVvWXHWWc9Ei2IwRjOWp6t6ENUyDLobzQmn5oI1vIVQL+ CjLgugRwZzGicFBp09E5nhaS+mmXEKxzOM/MIz/sn4a8qCgoYg+sjGueCgT8tR2Cpnaq Gr648NK7Lr7aSDLj6hkENmeq3Zmzx3GqL+7inx8IS5Amfrq+obyAfsh9kp11AnzU+wLG kZbg== X-Gm-Message-State: AOJu0YzXwp0OD0UGbDVEKxPZ66+m1bNrHh8kILcrmAtCP2iO3eT/W3b1 NjME+epOv88xBproqMIuY8+P9QtXfva7C3Py3OpOgRqXIxLV4F0vtcb5R4Pr+bhYl2iihu63bYP r X-Google-Smtp-Source: AGHT+IFlnXKVDgwn3h0Bd3lCrRslrqGeOtQb0Lct/5wKDoKZviSWE3n9QcbSokVvpdAhUUfm+qjaMA== X-Received: by 2002:a05:6a20:9150:b0:1a9:c843:6f37 with SMTP id x16-20020a056a20915000b001a9c8436f37mr5455756pzc.19.1714625662196; Wed, 01 May 2024 21:54:22 -0700 (PDT) Received: from localhost (fwdproxy-prn-002.fbsv.net. [2a03:2880:ff:2::face:b00c]) by smtp.gmail.com with ESMTPSA id o18-20020a170902e29200b001e99fdbc515sm253476plc.3.2024.05.01.21.54.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:21 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 6/9] bnxt: add __bnxt_init_rx_ring_struct() helper Date: Wed, 1 May 2024 21:54:07 -0700 Message-ID: <20240502045410.3524155-7-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Move the initialisation of rx ring and rx agg ring structs into a helper function. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 44 +++++++++++++---------- 1 file changed, 26 insertions(+), 18 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index bda49e7f6c3d..b0a8d14b7319 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -3997,6 +3997,31 @@ static int bnxt_alloc_cp_rings(struct bnxt *bp) return 0; } +static void __bnxt_init_rx_ring_struct(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr) +{ + struct bnxt_ring_mem_info *rmem; + struct bnxt_ring_struct *ring; + + ring = &rxr->rx_ring_struct; + rmem = &ring->ring_mem; + rmem->nr_pages = bp->rx_nr_pages; + rmem->page_size = HW_RXBD_RING_SIZE; + rmem->pg_arr = (void **)rxr->rx_desc_ring; + rmem->dma_arr = rxr->rx_desc_mapping; + rmem->vmem_size = SW_RXBD_RING_SIZE * bp->rx_nr_pages; + rmem->vmem = (void **)&rxr->rx_buf_ring; + + ring = &rxr->rx_agg_ring_struct; + rmem = &ring->ring_mem; + rmem->nr_pages = bp->rx_agg_nr_pages; + rmem->page_size = HW_RXBD_RING_SIZE; + rmem->pg_arr = (void **)rxr->rx_agg_desc_ring; + rmem->dma_arr = rxr->rx_agg_desc_mapping; + rmem->vmem_size = SW_RXBD_AGG_RING_SIZE * bp->rx_agg_nr_pages; + rmem->vmem = (void **)&rxr->rx_agg_ring; +} + static void bnxt_init_ring_struct(struct bnxt *bp) { int i, j; @@ -4024,24 +4049,7 @@ static void bnxt_init_ring_struct(struct bnxt *bp) rxr = bnapi->rx_ring; if (!rxr) goto skip_rx; - - ring = &rxr->rx_ring_struct; - rmem = &ring->ring_mem; - rmem->nr_pages = bp->rx_nr_pages; - rmem->page_size = HW_RXBD_RING_SIZE; - rmem->pg_arr = (void **)rxr->rx_desc_ring; - rmem->dma_arr = rxr->rx_desc_mapping; - rmem->vmem_size = SW_RXBD_RING_SIZE * bp->rx_nr_pages; - rmem->vmem = (void **)&rxr->rx_buf_ring; - - ring = &rxr->rx_agg_ring_struct; - rmem = &ring->ring_mem; - rmem->nr_pages = bp->rx_agg_nr_pages; - rmem->page_size = HW_RXBD_RING_SIZE; - rmem->pg_arr = (void **)rxr->rx_agg_desc_ring; - rmem->dma_arr = rxr->rx_agg_desc_mapping; - rmem->vmem_size = SW_RXBD_AGG_RING_SIZE * bp->rx_agg_nr_pages; - rmem->vmem = (void **)&rxr->rx_agg_ring; + __bnxt_init_rx_ring_struct(bp, rxr); skip_rx: bnxt_for_each_napi_tx(j, bnapi, txr) { From patchwork Thu May 2 04:54:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651285 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pg1-f171.google.com (mail-pg1-f171.google.com [209.85.215.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D608D2135B for ; Thu, 2 May 2024 04:54:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625665; cv=none; b=scmPHWTgrAgBRGr00ajCwqUpEhCy5/AGdYu27qXZ8T9+CrTxqGDvjfrEezXWCYN/5/5d6cW/LqX1fCMqUWjymKTS6vUAEnKSaVF9XZeJEktf8T6hPXNiGp3/6z11rPDCFchocvB13AAiAMp5rReMZlW1MNSiXjfoo+d9e9zWlMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625665; c=relaxed/simple; bh=jthI8hK9MrxYg3JgyHZimpUnG3c5aYxNuigyuiMuFuY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TWbSTTV6XO4wnN5Xj61EuuDbiI/BsSiGKMJ9VtAPCabDrlwDheZOWaoI+C74WAWtjxgFlyd0PxWZNA+P5zDwmrWPe3y3IyfXK93mpFRBlRRCwjdOqNxQTLZU6H9zbQX3rggXfqWg8CMzIgsWvvBPeclkwK/B2+s3o0uHjWpUaMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=bUH8QL1S; arc=none smtp.client-ip=209.85.215.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="bUH8QL1S" Received: by mail-pg1-f171.google.com with SMTP id 41be03b00d2f7-5ce07cf1e5dso5361234a12.2 for ; Wed, 01 May 2024 21:54:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625663; x=1715230463; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g/D9eyfvnwvo/JJPBJPqo7cc98rn6LXpR69ql3JkEKM=; b=bUH8QL1So81BzduQ+xExIOx5s7j2n/iavhPvPiFkEmr9kMF/MT0Fj7cEbkKExkSJ+E 1/fcUFdUcqoE2pYf+S6IpcnRRi7Snhigv7/e2HPK62vhziDlw/gMKze9KPVYLy9iMBfd /8wCn7IrhBgsvkZr7EqxyD5Rcsj4RvWOh1jgzfWbEE4pUvrtsCgpJSy4+k0qysOXMXdy PUmLTCsc44A47lw9gQsluuf0n4V9t3UpBQiOp5iSJuokWPEeMIC12tqlXCl8OBFU903t H8FLGL8fWba5zzAokekgZfSEXWnzG5KAIUJD0p6MriTGLfTiSEKuJGRbuRp+P1jHK7jA QaCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625663; x=1715230463; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g/D9eyfvnwvo/JJPBJPqo7cc98rn6LXpR69ql3JkEKM=; b=J64zCpKOu1uXy4UkRmhd0U13i6hJaEgmea0WH4VSQwQzm2WFMuKvs9T2632RaErsrw jNrOanYfEoYEhw+wJjMT1EY8mKzTF/hYK3MmuG0+CUHIueeWS+l6Uccgdsw5E+XRwAci U9/w3rioIxJXVPrjAPzTeKKeDrv2SlZy/NG0IL6FW4TvCDsW7Hj4zik78YhDG+GlHE1h 29GX7rtMfJYu9+gpltkTVMjt2Qzt0FNxVfPOHBZlcf4bMRYEOkG0oCewIFicWBDzo3GT tjczsRlSJh22NjlsZriPSppzT2qcXrx5MUvzhEHNZtn/B104sNpw0bwc+wUKNSlJJwbP M1tA== X-Gm-Message-State: AOJu0YymQoX7IVpp4LuggytUDPz6eTjzU6tVIPdV+IiLn+fm7Cnt571G SreXt6hTpub+O9UxJqXtp0FgkAlJV7G4DRCXPoynlebwGpzo9OliPQDyQer2fyoFXbYuzE9Tie0 c X-Google-Smtp-Source: AGHT+IEgM3zUkxqfLKYbfGfbBQrKH3frJNv5Zo/G+Df7Kc7l2SwM4GtWkqK9VrQdf5CClRiX53hacA== X-Received: by 2002:a05:6a20:3ca2:b0:1af:73a5:238f with SMTP id b34-20020a056a203ca200b001af73a5238fmr1194774pzj.44.1714625663091; Wed, 01 May 2024 21:54:23 -0700 (PDT) Received: from localhost (fwdproxy-prn-018.fbsv.net. [2a03:2880:ff:12::face:b00c]) by smtp.gmail.com with ESMTPSA id q8-20020a17090a178800b002b24c3fce2esm296405pja.33.2024.05.01.21.54.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:22 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 7/9] bnxt: add helpers for allocating rx ring mem Date: Wed, 1 May 2024 21:54:08 -0700 Message-ID: <20240502045410.3524155-8-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Add several helper functions for allocating rx ring memory. These are mostly taken from existing functions, but with unnecessary bits stripped out such that only allocations are done. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 87 +++++++++++++++++++++++ 1 file changed, 87 insertions(+) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index b0a8d14b7319..21c1a7cb70ab 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -14845,6 +14845,93 @@ static const struct netdev_stat_ops bnxt_stat_ops = { .get_base_stats = bnxt_get_base_stats, }; +static int __bnxt_alloc_rx_desc_ring(struct pci_dev *pdev, struct bnxt_ring_mem_info *rmem) +{ + int i, rc; + + for (i = 0; i < rmem->nr_pages; i++) { + rmem->pg_arr[i] = dma_alloc_coherent(&pdev->dev, + rmem->page_size, + &rmem->dma_arr[i], + GFP_KERNEL); + if (!rmem->pg_arr[i]) { + rc = -ENOMEM; + goto err_free; + } + } + + return 0; + +err_free: + while (i--) { + dma_free_coherent(&pdev->dev, rmem->page_size, + rmem->pg_arr[i], rmem->dma_arr[i]); + rmem->pg_arr[i] = NULL; + } + return rc; +} + +static int bnxt_alloc_rx_ring_struct(struct bnxt *bp, struct bnxt_ring_struct *ring) +{ + struct bnxt_ring_mem_info *rmem; + int rc; + + rmem = &ring->ring_mem; + rc = __bnxt_alloc_rx_desc_ring(bp->pdev, rmem); + if (rc) + return rc; + + *rmem->vmem = vzalloc(rmem->vmem_size); + if (!(*rmem->vmem)) { + rc = -ENOMEM; + goto err_free; + } + + return 0; + +err_free: + bnxt_free_ring(bp, rmem); + return rc; +} + +static int bnxt_alloc_rx_agg_bmap(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) +{ + u16 mem_size; + + rxr->rx_agg_bmap_size = bp->rx_agg_ring_mask + 1; + mem_size = rxr->rx_agg_bmap_size / 8; + rxr->rx_agg_bmap = kzalloc(mem_size, GFP_KERNEL); + if (!rxr->rx_agg_bmap) + return -ENOMEM; + + return 0; +} + +static void bnxt_init_rx_ring_rxbd_pages(struct bnxt *bp, struct bnxt_rx_ring_info *rxr) +{ + struct bnxt_ring_struct *ring; + u32 type; + + type = (bp->rx_buf_use_size << RX_BD_LEN_SHIFT) | + RX_BD_TYPE_RX_PACKET_BD | RX_BD_FLAGS_EOP; + + if (NET_IP_ALIGN == 2) + type |= RX_BD_FLAGS_SOP; + + ring = &rxr->rx_ring_struct; + ring->fw_ring_id = INVALID_HW_RING_ID; + bnxt_init_rxbd_pages(ring, type); + + ring = &rxr->rx_agg_ring_struct; + ring->fw_ring_id = INVALID_HW_RING_ID; + if ((bp->flags & BNXT_FLAG_AGG_RINGS)) { + type = ((u32)BNXT_RX_PAGE_SIZE << RX_BD_LEN_SHIFT) | + RX_BD_TYPE_RX_AGG_BD | RX_BD_FLAGS_SOP; + + bnxt_init_rxbd_pages(ring, type); + } +} + static void *bnxt_queue_mem_alloc(struct net_device *dev, int idx) { struct bnxt *bp = netdev_priv(dev); From patchwork Thu May 2 04:54:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651286 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f177.google.com (mail-pl1-f177.google.com [209.85.214.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBCCF224D7 for ; Thu, 2 May 2024 04:54:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625666; cv=none; b=Fa3M3kMk8HmqOZt722m0hPVNtsymIhvBaeb1FtHmi4Vv8Lt3PEYbjQD8y3FVHTJZyKovmz1mphIk2om/CyaIeYYttRqfqTSvZY5PSfq4UgJCucXsJ+lbxZR+pNaw4wwKVQVvFMrYzNhDwBjUzh8lflaWnUkBK629e6J1lBLqY+Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625666; c=relaxed/simple; bh=S/1yZF1TI+dwRZfaETByTwmVELoDx+n/rqzs9Rcapms=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gD1fRt/a5/56GSrGy5PGQnkkfq5mow8TrxgxMkICC3sjrX67DvhPwaUC6GSo73AJ78YBBPQ0TJZaJU7hvrBwGM/R/9j9QVArgqbeUJGbCzUy+y18a/NERqzrE9tqz6dB4QdrFOziaVGoM4lH8s2Kn6WA7I9eaXP0d5vlA0LDAhQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=icSe6gwQ; arc=none smtp.client-ip=209.85.214.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="icSe6gwQ" Received: by mail-pl1-f177.google.com with SMTP id d9443c01a7336-1ecd9a81966so2968495ad.0 for ; Wed, 01 May 2024 21:54:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625664; x=1715230464; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wGXdpI60jd3JHXKmbQ4WSfHFellxb6CxbrnH7cNwkdY=; b=icSe6gwQ3SJy1GvhB2VX8wvU/dJJnBCRkdmzARcyk/8bdgOWPqpnh+8B/MG9bAzDQn lRSm9yg8Svc2dvtcy+SBUtr4kSIonDevv7enBWWuWgcU0+m/ut/+sGCPTCTWfWJ+5BzG 97makQITL//JiVhRvOuzETTdHOKAEQfnq1xpCtgz1GnVtJhsh7VCwxWO7JszwmgiSxmK pOS0heOza8aUKBwciSJhTV9fBrmbQvn5bI6G4p0SdCqCpD1ThNi/ka4nHlAdB1UKO0eo YJW0O1+VYiQqiwQdgh+i0hCDUElAoL3NNASH16viPhtZ5Ll91VzG8GyQKSm7TkHVWM9R hNFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625664; x=1715230464; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wGXdpI60jd3JHXKmbQ4WSfHFellxb6CxbrnH7cNwkdY=; b=PRxpJa+eM6FI6YWNyJJzxW75im++gXzQu/FAPGm9MdYFqfJ7lo4luNZ4CHc3jKlKJA 2Ct0p6yPaPTaFxBRwsUklF2TKPAs3eEI1N3uH8xc7ntZzTq8xqr+YuDgZxjMG6m9ng1R RLrDYgF63peETMk5iy3LhGFORXMiABigr99CJWDlTtrz4qLkHEeJbSk5GiET6/HOk1BD yRIUv0uFB8WzYjLUttYGn3E48i6XhAFlhK+VIZB0F1lQGfvB0cCzcsnye/pkUUI+g6pj tkpfI1ahKhph1FInKGaOqBzmtH/BRf0LWc38o+yflBwgf6nj6JCf+MIvTOoxcKmi7sXe 6/7Q== X-Gm-Message-State: AOJu0YyGBzRSLZWR3pqVhpYiol8l9ZaTejIeUcQqSdQeGQ9Zt7XUKm4d SHpp805mVrvllBjwAHijd7q5VLDgZ8fWeaaN5hZl3xkykKoBJI6oOkr0IgKfXlEbDeLyG/lBn1W O X-Google-Smtp-Source: AGHT+IHrFDiYM14G+XgK11d/REEkHO0+n0A7BRJgiMa7hnGzkXf9KGF6StfCQRn2P0N8OPZFkAHCbQ== X-Received: by 2002:a17:902:f950:b0:1e4:7bf1:521 with SMTP id kx16-20020a170902f95000b001e47bf10521mr1957773plb.19.1714625664008; Wed, 01 May 2024 21:54:24 -0700 (PDT) Received: from localhost (fwdproxy-prn-016.fbsv.net. [2a03:2880:ff:10::face:b00c]) by smtp.gmail.com with ESMTPSA id lf4-20020a170902fb4400b001e8d180766dsm237844plb.278.2024.05.01.21.54.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:23 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 8/9] bnxt: alloc rx ring mem first before reset Date: Wed, 1 May 2024 21:54:09 -0700 Message-ID: <20240502045410.3524155-9-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC Implement allocating rx ring mem in bnxt_queue_mem_alloc(). This is done by duplicating the existing rx ring entirely, then allocating new memory into this clone as most functions take rx ring as an argument. I've identified the following memory that gets allocated: * rx_desc_ring * separate allocation per hw page * rx_buf_ring * rx_agg_desc_ring * rx_agg_ring * rx_agg_bmap * rx_tpa * rx_tpa_idx_map So, zero the ring heads, alloc the rings, then call bnxt_alloc_one_rx_ring() to fills in the descriptors. It's interesting that struct bnxt_ring_mem_info points to addresses of stack allocated arrays within struct bnxt_rx_ring_info, instead of the heap allocated queue memory. __bnxt_init_rx_ring_struct() is first called to reconfigure these ptrs for the clone. The hardware is only aware of the pg_tbl, which we do not touch. In the coming patches after an rx ring has been quiesced, we'll swap the bits that are dynamically allocated then update the dma mappings in pg_tbl. Then the hardware is none the wiser! Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 78 ++++++++++++++++++++++- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 + 2 files changed, 79 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 21c1a7cb70ab..d848b9ceabf0 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -14934,13 +14934,89 @@ static void bnxt_init_rx_ring_rxbd_pages(struct bnxt *bp, struct bnxt_rx_ring_in static void *bnxt_queue_mem_alloc(struct net_device *dev, int idx) { + struct bnxt_rx_ring_info *rxr, *clone; struct bnxt *bp = netdev_priv(dev); + int rc; + + rxr = &bp->rx_ring[idx]; + clone = kmemdup(rxr, sizeof(*rxr), GFP_KERNEL); + if (!clone) + return ERR_PTR(-ENOMEM); + + clone->rx_prod = 0; + clone->rx_agg_prod = 0; + clone->rx_sw_agg_prod = 0; + clone->rx_next_cons = 0; + + __bnxt_init_rx_ring_struct(bp, clone); + + rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid); + if (rc) + goto err_free_clone; + + rc = bnxt_alloc_rx_ring_struct(bp, &clone->rx_ring_struct); + if (rc) + goto err_free_page_pool; - return &bp->rx_ring[idx]; + if (bp->flags & BNXT_FLAG_AGG_RINGS) { + rc = bnxt_alloc_rx_ring_struct(bp, &clone->rx_agg_ring_struct); + if (rc) + goto err_free_rx_ring; + + rc = bnxt_alloc_rx_agg_bmap(bp, clone); + if (rc) + goto err_free_rx_agg_ring; + } + + if (bp->flags & BNXT_FLAG_TPA) { + rc = __bnxt_alloc_one_tpa_info(bp, clone); + if (rc) + goto err_free_rx_agg_bmap; + } + + bnxt_init_rx_ring_rxbd_pages(bp, clone); + bnxt_alloc_one_rx_ring(bp, clone); + + rxr->rplc = clone; + + return clone; + +err_free_rx_agg_bmap: + kfree(clone->rx_agg_bmap); +err_free_rx_agg_ring: + bnxt_free_ring(bp, &clone->rx_agg_ring_struct.ring_mem); +err_free_rx_ring: + bnxt_free_ring(bp, &clone->rx_ring_struct.ring_mem); +err_free_page_pool: + page_pool_destroy(clone->page_pool); + rxr->page_pool = NULL; +err_free_clone: + kfree(clone); + + return ERR_PTR(rc); } static void bnxt_queue_mem_free(struct net_device *dev, void *qmem) { + struct bnxt_rx_ring_info *rxr = qmem; + struct bnxt *bp = netdev_priv(dev); + struct bnxt_ring_struct *ring; + + bnxt_free_tpa_info(bp, rxr); + + page_pool_destroy(rxr->page_pool); + rxr->page_pool = NULL; + + kfree(rxr->rx_agg_bmap); + rxr->rx_agg_bmap = NULL; + + ring = &rxr->rx_ring_struct; + bnxt_free_ring(bp, &ring->ring_mem); + + ring = &rxr->rx_agg_ring_struct; + bnxt_free_ring(bp, &ring->ring_mem); + + kfree(rxr); } static int bnxt_queue_start(struct net_device *dev, int idx, void *qmem) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index ad57ef051798..ce2aa48911bb 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1076,6 +1076,8 @@ struct bnxt_rx_ring_info { struct bnxt_ring_struct rx_agg_ring_struct; struct xdp_rxq_info xdp_rxq; struct page_pool *page_pool; + + struct bnxt_rx_ring_info *rplc; }; struct bnxt_rx_sw_stats { From patchwork Thu May 2 04:54:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Wei X-Patchwork-Id: 13651287 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CBBE7225D6 for ; Thu, 2 May 2024 04:54:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625667; cv=none; b=EJwjEnXpWJqiQoDET8NRcpPwh4weVf78Wkpk4GriAkB9iLplLRkQKUZ9gR0q+NbDSPbpMQb2K4XgKmDT0IjCniIxEJ+SiOxA+P5egPAanqq2v883mEi0NEPUjo63Lq4MU4VpLhQHCv10CDjTCRF23rl2zpGjPhFDSFEK/u9pAbA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1714625667; c=relaxed/simple; bh=3ZH+1bvU4Ejw/DcxJIqAYAKts80FqvGAFvHgraw+zvU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lEulGpGBdLONhRTXjELgfwvIlKssfr0UcDVkoQPxHXWlRdiUIjKIs2TzM+6b6Vv9Lk+uG/sRPZfuZl0gLg4VeWKLSNN8KOLxCgdJm7l0vVlitHabks+sor8KwdmRHZ8EjOWxS+umCMw/jBFYqXoiAqT3Fonx0SbtGJK4FlALv2g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk; spf=none smtp.mailfrom=davidwei.uk; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b=O30DF8JQ; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=davidwei.uk Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=davidwei-uk.20230601.gappssmtp.com header.i=@davidwei-uk.20230601.gappssmtp.com header.b="O30DF8JQ" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-1ecff927a45so282905ad.0 for ; Wed, 01 May 2024 21:54:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=davidwei-uk.20230601.gappssmtp.com; s=20230601; t=1714625665; x=1715230465; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yk6i6LPlwsXNVuX1b+f4RX3mSLS5KKbdRNMKlofqpos=; b=O30DF8JQC7f9m0rqvCo8L7vfLQ20VIzBCh87n81RTsyGJphMG2E+75xa0mfFOI2fdS 1wf8JY494hpeS0oP1DcYqguDHbp6ZKK5zuxODMQ+zxMrvFXO9G3yFEg4avlsPxsxrsBB qtz4gTNpi6Qi9dsykXjld4KJG+A1EhsWqH0gOYrPY1VrnBHXSA8CEYVEoPgY2IdKcBL6 SS7XCnf0laCt+p1/MTqLycE1Zq9xJ1iIpc/Q2i3bBLN1Uvda7VxXI/IRowiL3FeCP2C9 uzujE0NaJ0JMVz09hFf+TwSZ/aMjSNeS2ItK60BXOxrSHixxgCdlkSYZaIyVHOxJC/EY pORw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714625665; x=1715230465; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yk6i6LPlwsXNVuX1b+f4RX3mSLS5KKbdRNMKlofqpos=; b=fn3xBElU4EtEIulUVG+y7aAjRYTFKM+MW7Q/BAOOWleYCmirba5Oa86NXrCD1JMyp2 JY9yyKKJCest5Ka8cOvybHt0n/JhipGarx3UnFqMJSrO74Z7/3DNUhcc2l5iYMXwWlgX BtadAz8c08iVZ70U2IdwxqwGS8W4K1WBo9HVs562dQ+3G3wpnTiPyX8Bbm6vfmLEHlqH sN+R5CiyQOglGpcQzHv/yQZBY0a9rkbMsx8p+JPKsvvwtzNI8tATuw/Yje7cjdYlkBQW yt+5BukxUIJ9e3w1I+L/CXTLJFcqiJ6hkAZfYBkNFp42/6Be/q8vay6ZsoYrikL7UQcd Pe0A== X-Gm-Message-State: AOJu0YyAtsZ7AMdwL626niKnNfBtkmLYbFoDWimWXYN6v/yf2MOXvI9r 2Bsfd0UYiIuS7LaGxCFHTRQbID6QPOqw8i4O2n4v9zPjq15mmltafAGiY8ry9OVH3S6qGFd0rsO 7 X-Google-Smtp-Source: AGHT+IEELsdkM3ZmqnIAyGj8Gt3RcV+MdiYXWYmIhP3o4u+cglEz8i5uuTLB8JY2oGA7OyGRBLxK4A== X-Received: by 2002:a17:902:e5d2:b0:1e5:1867:d9fa with SMTP id u18-20020a170902e5d200b001e51867d9famr5810955plf.44.1714625664918; Wed, 01 May 2024 21:54:24 -0700 (PDT) Received: from localhost (fwdproxy-prn-120.fbsv.net. [2a03:2880:ff:78::face:b00c]) by smtp.gmail.com with ESMTPSA id lf14-20020a170902fb4e00b001e556734814sm242297plb.134.2024.05.01.21.54.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 May 2024 21:54:24 -0700 (PDT) From: David Wei To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andy Gospodarek , Adrian Alvarado , Mina Almasry , Shailend Chand Cc: Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni Subject: [RFC PATCH net-next v2 9/9] bnxt: swap rx ring mem during queue_stop() Date: Wed, 1 May 2024 21:54:10 -0700 Message-ID: <20240502045410.3524155-10-dw@davidwei.uk> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240502045410.3524155-1-dw@davidwei.uk> References: <20240502045410.3524155-1-dw@davidwei.uk> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org X-Patchwork-State: RFC After stopping an rx ring in bnxt_queue_stop(), swap the preallocated ring memory into the existing rx ring that the hardware is aware of. As discussed in the last patch, the hardware ring is associated with the address of static arrays in struct bnxt_rx_ring_info. For example: struct bnxt_rx_ring_info struct bnxt_ring_mem_info struct rx_bd *rx_desc_ring[MAX] <-> void **pg_arr struct bnxt_sw_rx_bd *rx_buf_ring <-> void **vmem The pg_tbl that is registered w/ the hardware via HWRM contains an array of dma mappings to the pg_arr above. We can't touch this association during reset, so can't simply swap the ring and its clone directly. Instead, swap the ring memory only then update the pg_tbl. Functionally it should be the same as the existing bnxt_rx_ring_reset(), except the allocations happen before resetting the ring using a clone struct bnxt_rx_ring_info. Signed-off-by: David Wei --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 54 +++++++++++++++++------ 1 file changed, 41 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index d848b9ceabf0..4dd4aa0911b1 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -15024,8 +15024,6 @@ static int bnxt_queue_start(struct net_device *dev, int idx, void *qmem) struct bnxt_rx_ring_info *rxr = qmem; struct bnxt *bp = netdev_priv(dev); - bnxt_alloc_one_rx_ring(bp, rxr); - if (bp->flags & BNXT_FLAG_AGG_RINGS) bnxt_db_write(bp, &rxr->rx_agg_db, rxr->rx_agg_prod); bnxt_db_write(bp, &rxr->rx_db, rxr->rx_prod); @@ -15038,26 +15036,56 @@ static int bnxt_queue_start(struct net_device *dev, int idx, void *qmem) static int bnxt_queue_stop(struct net_device *dev, int idx, void **out_qmem) { + struct bnxt_rx_ring_info *orig, *rplc; struct bnxt *bp = netdev_priv(dev); - struct bnxt_rx_ring_info *rxr; + struct bnxt_ring_mem_info *rmem; struct bnxt_cp_ring_info *cpr; - int rc; + int i, rc; rc = bnxt_hwrm_rx_ring_reset(bp, idx); if (rc) return rc; - rxr = &bp->rx_ring[idx]; - bnxt_free_one_rx_ring_skbs(bp, rxr); - rxr->rx_prod = 0; - rxr->rx_agg_prod = 0; - rxr->rx_sw_agg_prod = 0; - rxr->rx_next_cons = 0; - - cpr = &rxr->bnapi->cp_ring; + /* HW ring is registered w/ the original bnxt_rx_ring_info so we cannot + * do a direct swap between orig and rplc. Instead, swap the + * dynamically allocated queue memory and then update pg_tbl. + */ + orig = &bp->rx_ring[idx]; + rplc = orig->rplc; + + swap(orig->rx_prod, rplc->rx_prod); + swap(orig->rx_agg_prod, rplc->rx_agg_prod); + swap(orig->rx_sw_agg_prod, rplc->rx_sw_agg_prod); + swap(orig->rx_next_cons, rplc->rx_next_cons); + + for (i = 0; i < MAX_RX_PAGES; i++) { + swap(orig->rx_desc_ring[i], rplc->rx_desc_ring[i]); + swap(orig->rx_desc_mapping[i], rplc->rx_desc_mapping[i]); + + swap(orig->rx_agg_desc_ring[i], rplc->rx_agg_desc_ring[i]); + swap(orig->rx_agg_desc_mapping[i], rplc->rx_agg_desc_mapping[i]); + } + swap(orig->rx_buf_ring, rplc->rx_buf_ring); + swap(orig->rx_agg_ring, rplc->rx_agg_ring); + swap(orig->rx_agg_bmap, rplc->rx_agg_bmap); + swap(orig->rx_agg_bmap_size, rplc->rx_agg_bmap_size); + swap(orig->rx_tpa, rplc->rx_tpa); + swap(orig->rx_tpa_idx_map, rplc->rx_tpa_idx_map); + swap(orig->page_pool, rplc->page_pool); + + rmem = &orig->rx_ring_struct.ring_mem; + for (i = 0; i < rmem->nr_pages; i++) + rmem->pg_tbl[i] = cpu_to_le64(rmem->dma_arr[i]); + + rmem = &rplc->rx_ring_struct.ring_mem; + for (i = 0; i < rmem->nr_pages; i++) + rmem->pg_tbl[i] = cpu_to_le64(rmem->dma_arr[i]); + + cpr = &orig->bnapi->cp_ring; cpr->sw_stats.rx.rx_resets++; - *out_qmem = rxr; + *out_qmem = rplc; + orig->rplc = NULL; return 0; }