From patchwork Mon Sep 30 20:13:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michal Schmidt X-Patchwork-Id: 13816988 X-Patchwork-Delegate: kuba@kernel.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A71301A0BD1 for ; Mon, 30 Sep 2024 20:13:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727221; cv=none; b=ckFJfmOk/3TE3P7As1xqVzG+nuII5c3OEVCwgr9cElGF1NyQzTZdvtgYe7Pigd39/Cm9CQKxAiB//NDF4cxUrv9TLx+ZwDWauHMSdYcF3h5EPTSTeYIXkBl71fDYcBoSxzf92/z9Vd4A7H6lSNb1HUxRSO8OLD0RENZgZ8OGKv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727221; c=relaxed/simple; bh=QCabOFyjz/9789nssv0DMhLQlRo6ZtJa+RMYl3QA2aA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=WTbjmcU37Hh7A3HQGdFEBcQBCgN847fnnt6WjK7KPyj9HLrOg9bsDxhnW0wC+PuFNoVqpUuJATO53tmoiI1FAynBAlWH+j+JTwQwrqKRv66EpizUU5QVkMtvnydJjkdskTQJKXLZ/J8l9gRYn6fTqk+jV7V+AiCPE5vw1RpnisM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UYnhkpsV; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UYnhkpsV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727727218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=S6ce+k6Atn+qrW3kCUH4ga1Kn2zJml2TCYfASckzxJw=; b=UYnhkpsVBPmxmuIN6jk03+1xk0aAXfHTwkcfoRiFHeqe7t1lUvcnd3OXISfhqrZLkBrL2b DeFOa9gptXXgcuKgKSjQ+ZX0yvQWZIX+tw3qkOdrdL8HtoYH33+4+lXJ2EXGz38u2PldYi EfrptOqAEO7zxVziYJ5BZ+dYZVhK1uM= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-180-MY8GH7vAOva5gVx0LqYGMw-1; Mon, 30 Sep 2024 16:13:33 -0400 X-MC-Unique: MY8GH7vAOva5gVx0LqYGMw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CA89D1955D56; Mon, 30 Sep 2024 20:13:31 +0000 (UTC) Received: from rhel-developer-toolbox-latest.redhat.com (unknown [10.45.224.53]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C8EDB19560AA; Mon, 30 Sep 2024 20:13:28 +0000 (UTC) From: Michal Schmidt To: Manish Chopra , netdev@vger.kernel.org Cc: Caleb Sander , Alok Prasad , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH net-next 1/4] qed: make 'ethtool -d' 10 times faster Date: Mon, 30 Sep 2024 22:13:04 +0200 Message-ID: <20240930201307.330692-2-mschmidt@redhat.com> In-Reply-To: <20240930201307.330692-1-mschmidt@redhat.com> References: <20240930201307.330692-1-mschmidt@redhat.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: kuba@kernel.org As a side effect of commit 5401c3e09928 ("qed: allow sleep in qed_mcp_trace_dump()"), 'ethtool -d' became much slower. Almost all the time is spent collecting the "mcp_trace". It is caused by sleeping too long in _qed_mcp_cmd_and_union. When called with sleeping not allowed, the function delays for 10 µs between firmware polls. But if sleeping is allowed, it sleeps for 10 ms instead. The sleeps in _qed_mcp_cmd_and_union are unnecessarily long. Replace msleep with usleep_range, which allows to achieve a similar polling interval like in the no-sleeping mode (10 - 20 µs). The only caller, qed_mcp_cmd_and_union, can stop doing the multiplication/division of the usecs/max_retries. The polling interval and the number of retries do not need to be parameters at all. On my test system, 'ethtool -d' now takes 4 seconds instead of 44. Signed-off-by: Michal Schmidt --- drivers/net/ethernet/qlogic/qed/qed_mcp.c | 36 ++++++++++------------- 1 file changed, 15 insertions(+), 21 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c index 16e6bd466143..00f0abc1c2d2 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c +++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c @@ -459,12 +459,11 @@ static void qed_mcp_print_cpu_info(struct qed_hwfn *p_hwfn, static int _qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, struct qed_ptt *p_ptt, - struct qed_mcp_mb_params *p_mb_params, - u32 max_retries, u32 usecs) + struct qed_mcp_mb_params *p_mb_params) { - u32 cnt = 0, msecs = DIV_ROUND_UP(usecs, 1000); struct qed_mcp_cmd_elem *p_cmd_elem; u16 seq_num; + u32 cnt = 0; int rc = 0; /* Wait until the mailbox is non-occupied */ @@ -488,12 +487,13 @@ _qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, spin_unlock_bh(&p_hwfn->mcp_info->cmd_lock); if (QED_MB_FLAGS_IS_SET(p_mb_params, CAN_SLEEP)) - msleep(msecs); + usleep_range(QED_MCP_RESP_ITER_US, + QED_MCP_RESP_ITER_US * 2); else - udelay(usecs); - } while (++cnt < max_retries); + udelay(QED_MCP_RESP_ITER_US); + } while (++cnt < QED_DRV_MB_MAX_RETRIES); - if (cnt >= max_retries) { + if (cnt >= QED_DRV_MB_MAX_RETRIES) { DP_NOTICE(p_hwfn, "The MFW mailbox is occupied by an uncompleted command. Failed to send command 0x%08x [param 0x%08x].\n", p_mb_params->cmd, p_mb_params->param); @@ -520,9 +520,10 @@ _qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, */ if (QED_MB_FLAGS_IS_SET(p_mb_params, CAN_SLEEP)) - msleep(msecs); + usleep_range(QED_MCP_RESP_ITER_US, + QED_MCP_RESP_ITER_US * 2); else - udelay(usecs); + udelay(QED_MCP_RESP_ITER_US); spin_lock_bh(&p_hwfn->mcp_info->cmd_lock); @@ -536,9 +537,9 @@ _qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, goto err; spin_unlock_bh(&p_hwfn->mcp_info->cmd_lock); - } while (++cnt < max_retries); + } while (++cnt < QED_DRV_MB_MAX_RETRIES); - if (cnt >= max_retries) { + if (cnt >= QED_DRV_MB_MAX_RETRIES) { DP_NOTICE(p_hwfn, "The MFW failed to respond to command 0x%08x [param 0x%08x].\n", p_mb_params->cmd, p_mb_params->param); @@ -564,7 +565,8 @@ _qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, "MFW mailbox: response 0x%08x param 0x%08x [after %d.%03d ms]\n", p_mb_params->mcp_resp, p_mb_params->mcp_param, - (cnt * usecs) / 1000, (cnt * usecs) % 1000); + (cnt * QED_MCP_RESP_ITER_US) / 1000, + (cnt * QED_MCP_RESP_ITER_US) % 1000); /* Clear the sequence number from the MFW response */ p_mb_params->mcp_resp &= FW_MSG_CODE_MASK; @@ -581,8 +583,6 @@ static int qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, struct qed_mcp_mb_params *p_mb_params) { size_t union_data_size = sizeof(union drv_union_data); - u32 max_retries = QED_DRV_MB_MAX_RETRIES; - u32 usecs = QED_MCP_RESP_ITER_US; /* MCP not initialized */ if (!qed_mcp_is_init(p_hwfn)) { @@ -606,13 +606,7 @@ static int qed_mcp_cmd_and_union(struct qed_hwfn *p_hwfn, return -EINVAL; } - if (QED_MB_FLAGS_IS_SET(p_mb_params, CAN_SLEEP)) { - max_retries = DIV_ROUND_UP(max_retries, 1000); - usecs *= 1000; - } - - return _qed_mcp_cmd_and_union(p_hwfn, p_ptt, p_mb_params, max_retries, - usecs); + return _qed_mcp_cmd_and_union(p_hwfn, p_ptt, p_mb_params); } static int _qed_mcp_cmd(struct qed_hwfn *p_hwfn, From patchwork Mon Sep 30 20:13:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Schmidt X-Patchwork-Id: 13816987 X-Patchwork-Delegate: kuba@kernel.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0466B1A0BDE for ; Mon, 30 Sep 2024 20:13:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727220; cv=none; b=AwehiPqFzR0w8dhSLSvkiIcTPXbHhw7kpfyQXA/LR+HeTA5JQ0VMNHBpCNIDu5A2BAnzmcYm9vg6WZ3D2SHlv4UU0fJ/wgW+p754vdG3S0nkyVVGRragPFbfITLPc40MboMo38zWi6drGKnyxqd7eKP/Pp4yIUxeFAIgAiT464Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727220; c=relaxed/simple; bh=qRH5YwDbM1DAHggYroIrUSh7Z56KGEOACzrObi2p8iE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QzrcbyfAXSud2YSmLjOIa57qurDfhUVudn7y4ZOWtJ9O3GuOLrhud/P8piqmvktMfqk1EvXAZ2rBjzPsx0XjmQ1rzbm4HTfjWKnLWogs+B10rSexzE02ld5yHc454sTH/yzpSxZvS1/NcUVVCXhph2GBS8AkGb5mJzovhw9pkDI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cBR9XpCj; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cBR9XpCj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727727218; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GRgJGBAJFn3EFrLh4/BAMAfyDsSerAQKWxwR8Z7g+7c=; b=cBR9XpCjtWocJSvS42qINSY7uXrg/pu+J3I7oDBcJJ/KhY85ZKbMkbUSWuCErO8srI+uXm lT1xVb4vANial3L0ArLnNZRhHURAPw/CF2o1xDZuzuxlCiY0K7pCI6+HQhfbk+8KANGY/Q veM4XhyAfCxTg2c2BkSUtYfY07kDtQE= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-YfAdo8n-PhSvBZT5WFCWNg-1; Mon, 30 Sep 2024 16:13:36 -0400 X-MC-Unique: YfAdo8n-PhSvBZT5WFCWNg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5761A1944DD9; Mon, 30 Sep 2024 20:13:35 +0000 (UTC) Received: from rhel-developer-toolbox-latest.redhat.com (unknown [10.45.224.53]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4ED5E19560AA; Mon, 30 Sep 2024 20:13:32 +0000 (UTC) From: Michal Schmidt To: Manish Chopra , netdev@vger.kernel.org Cc: Caleb Sander , Alok Prasad , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH net-next 2/4] qed: put cond_resched() in qed_grc_dump_ctx_data() Date: Mon, 30 Sep 2024 22:13:05 +0200 Message-ID: <20240930201307.330692-3-mschmidt@redhat.com> In-Reply-To: <20240930201307.330692-1-mschmidt@redhat.com> References: <20240930201307.330692-1-mschmidt@redhat.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: kuba@kernel.org On a kernel with preemption none or voluntary, 'ethtool -d' on a qede network device can cause a big latency spike. The biggest part of it is the loop in qed_grc_dump_ctx_data. The function is called only from the .get_size and .perform_dump callbacks for the "grc" feature defined in qed_features_lookup[]. As far as I can see, they are used in: - qed's devlink healh reporter .dump op - qede's ethtool get_regs/get_regs_len/get_dump_data ops - qedf's qedf_get_grc_dump, called from: - qedf_sysfs_write_grcdump - "grcdump" sysfs attribute write - qedf_wq_grcdump - a workqueue It is safe to sleep in all of them. Let's insert a cond_resched() in the outer loop to let other tasks run. Measured using this script: #!/bin/bash DEV=ens3f1 echo wakeup_rt > /sys/kernel/tracing/current_tracer echo 0 > /sys/kernel/tracing/tracing_max_latency echo 1 > /sys/kernel/tracing/tracing_on echo "Setting the task CPU affinity" taskset -p 1 $$ > /dev/null echo "Starting the real-time task" chrt -f 50 bash -c 'while sleep 0.01; do :; done' & sleep 1 echo "Running: ethtool -d $DEV" time ethtool -d $DEV > /dev/null kill %1 echo 0 > /sys/kernel/tracing/tracing_on echo "Measured latency: $( --- drivers/net/ethernet/qlogic/qed/qed_debug.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/qlogic/qed/qed_debug.c b/drivers/net/ethernet/qlogic/qed/qed_debug.c index f67be4b8ad43..464a72afb758 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_debug.c +++ b/drivers/net/ethernet/qlogic/qed/qed_debug.c @@ -2873,6 +2873,7 @@ static u32 qed_grc_dump_ctx_data(struct qed_hwfn *p_hwfn, false, SPLIT_TYPE_NONE, 0); } + cond_resched(); } return offset; From patchwork Mon Sep 30 20:13:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Schmidt X-Patchwork-Id: 13816989 X-Patchwork-Delegate: kuba@kernel.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91A7519994A for ; Mon, 30 Sep 2024 20:13:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727226; cv=none; b=bscbDuUqoArZropWoj4ERU1eyikkNFCbhn/b0h/8tc99WjrmzkPS1U2XozBtJKBcw1hDYgDptI3v6/T4bJcZcAdLwyUBMCK43ixI9UYkiWNi1gIknLABlsmVd1qKJ7Bex0YGtZiDWPFLMogctex4QV09pB47YropgyM8E/Z5UD8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727226; c=relaxed/simple; bh=NLsBTy+bcv4pI2HiJjd7u/CFuPoasgF91UA3EXm64Nc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=evK+g6VdJUx7LmWL66aLtnoD/QVJGUAEZ/4EcoUkOiQrwcY1bXOVT88A9TLJWiLTzuo9Lv5w4iWNDo4l6X/GyaCF8zAsnWuU4uDvpG3GSPUqF14ijJrAMKBAflJ++cud3BCEuX0Z5inToR9r7sddW6b47pjCA1XGhE6dp2/d3C0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RMeak+gm; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RMeak+gm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727727223; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=MqrvHKCp/tNNkz57D8OOtSU9teDeybtDBbPcG0l7M2w=; b=RMeak+gm2WIY0oJs5cop060Uyr3qUeC1LAbVjHjH756az5WAW4yiqUPJ6xKwEyss14Ljbf st400YOti0zI0ImFul/JSh0I7YzKymZxgSfQ+9BjpR1VWZs7lglwHILA6tYDOhZxYuwKDX ey0h3omxfihcFTTCMD4QaYG2TcfihfI= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-j6L5Ks4APkSLL_-vB7Matg-1; Mon, 30 Sep 2024 16:13:40 -0400 X-MC-Unique: j6L5Ks4APkSLL_-vB7Matg-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CFF4F1953940; Mon, 30 Sep 2024 20:13:38 +0000 (UTC) Received: from rhel-developer-toolbox-latest.redhat.com (unknown [10.45.224.53]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D7C4F19560AD; Mon, 30 Sep 2024 20:13:35 +0000 (UTC) From: Michal Schmidt To: Manish Chopra , netdev@vger.kernel.org Cc: Caleb Sander , Alok Prasad , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH net-next 3/4] qed: allow the callee of qed_mcp_nvm_read() to sleep Date: Mon, 30 Sep 2024 22:13:06 +0200 Message-ID: <20240930201307.330692-4-mschmidt@redhat.com> In-Reply-To: <20240930201307.330692-1-mschmidt@redhat.com> References: <20240930201307.330692-1-mschmidt@redhat.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: kuba@kernel.org qed_mcp_nvm_read has a loop where it calls qed_mcp_nvm_rd_cmd with the argument b_can_sleep=false. And it sleeps once every 0x1000 bytes read. Simplify this by letting qed_mcp_nvm_rd_cmd itself sleep (b_can_sleep=true). It will have slept at least once when successful (in the "Wait for the MFW response" loop). So the extra sleep once every 0x1000 bytes becomes superfluous. Delete it. On my test system with voluntary preemption, this lowers the latency caused by 'ethtool -d' from 53 ms to 10 ms. Signed-off-by: Michal Schmidt --- drivers/net/ethernet/qlogic/qed/qed_mcp.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_mcp.c b/drivers/net/ethernet/qlogic/qed/qed_mcp.c index 00f0abc1c2d2..26a714bfad4e 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_mcp.c +++ b/drivers/net/ethernet/qlogic/qed/qed_mcp.c @@ -3079,20 +3079,13 @@ int qed_mcp_nvm_read(struct qed_dev *cdev, u32 addr, u8 *p_buf, u32 len) DRV_MB_PARAM_NVM_LEN_OFFSET), &resp, &resp_param, &read_len, - (u32 *)(p_buf + offset), false); + (u32 *)(p_buf + offset), true); if (rc || (resp != FW_MSG_CODE_NVM_OK)) { DP_NOTICE(cdev, "MCP command rc = %d\n", rc); break; } - /* This can be a lengthy process, and it's possible scheduler - * isn't preemptible. Sleep a bit to prevent CPU hogging. - */ - if (bytes_left % 0x1000 < - (bytes_left - read_len) % 0x1000) - usleep_range(1000, 2000); - offset += read_len; bytes_left -= read_len; } From patchwork Mon Sep 30 20:13:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michal Schmidt X-Patchwork-Id: 13816990 X-Patchwork-Delegate: kuba@kernel.org Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DCED71A0BFB for ; Mon, 30 Sep 2024 20:13:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727228; cv=none; b=ZKaYmT4ixF1A7AqjPZaaGg11UA1/DXM8whwiZ6VV08iE3nSlbgraAGNhhSVXp+I8DKOGSLJ++Mxpj/gZrAAcTyIruFW+AQHt5EolVDaUp5owz8j10ouhW9ES9kkLPaKmoqaKZPjkOXzAwhlH5Wkac7uzNJzIpRXY/+9MJVZRE4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727727228; c=relaxed/simple; bh=XQ71XnM5RENP1ZeK3uEijRqSKcCIm14ZzuO6RwVvoA0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dVGF7pe9nV3hKKZGNoY1P9ui/d1VJDVUN75LI990ok3hWM+qscqVd7K2x53cnKTFDyjBu+i/Ctn/9OPQ46QrzvI5gVV/ZBZLzjjf4Xn3edKMopM302tGcRoRAxD4Gryj2Q27cgsd9RzDkiq1CJtHYc0nNLwRJbawq9whd1+RWic= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XCyFHdZG; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XCyFHdZG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727727225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aaUDeqsjhVwNR0DztDYhNSLJtJvLC/H3mGOW8DMWbCE=; b=XCyFHdZGsVBZpT1E23DDVyJw6quFr055YHjVcJEX3WWGZDXBsWC5PJu1kia/dAwZV2V/8g 4l+k0cU0ac7cNe0TFy1P86S0AuShQse5oqKn0lJCgKOLsF4Ho8nf3WpqYpwxWRUUbQY8vH CQK4tU04/OcZfD6mPWgVRhr1HWv1Mqc= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-513-twh71N5PM2eVM2t9dXSiXw-1; Mon, 30 Sep 2024 16:13:44 -0400 X-MC-Unique: twh71N5PM2eVM2t9dXSiXw-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 47353196A133; Mon, 30 Sep 2024 20:13:43 +0000 (UTC) Received: from rhel-developer-toolbox-latest.redhat.com (unknown [10.45.224.53]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E87B419560AA; Mon, 30 Sep 2024 20:13:39 +0000 (UTC) From: Michal Schmidt To: Manish Chopra , netdev@vger.kernel.org Cc: Caleb Sander , Alok Prasad , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni Subject: [PATCH net-next 4/4] qed: put cond_resched() in qed_dmae_operation_wait() Date: Mon, 30 Sep 2024 22:13:07 +0200 Message-ID: <20240930201307.330692-5-mschmidt@redhat.com> In-Reply-To: <20240930201307.330692-1-mschmidt@redhat.com> References: <20240930201307.330692-1-mschmidt@redhat.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Patchwork-Delegate: kuba@kernel.org It is OK to sleep in qed_dmae_operation_wait, because it is called only in process context, while holding p_hwfn->dmae_info.mutex from one of the qed_dmae_{host,grc}2{host,grc} functions. The udelay(DMAE_MIN_WAIT_TIME=2) in the function is too short to replace with usleep_range, but at least it's a suitable point for checking if we should give up the CPU with cond_resched(). This lowers the latency caused by 'ethtool -d' from 10 ms to less than 2 ms on my test system with voluntary preemption. Signed-off-by: Michal Schmidt --- drivers/net/ethernet/qlogic/qed/qed_hw.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/qlogic/qed/qed_hw.c b/drivers/net/ethernet/qlogic/qed/qed_hw.c index 6263f847b6b9..9e5f0dbc8a07 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_hw.c +++ b/drivers/net/ethernet/qlogic/qed/qed_hw.c @@ -596,6 +596,7 @@ static int qed_dmae_operation_wait(struct qed_hwfn *p_hwfn) barrier(); while (*p_hwfn->dmae_info.p_completion_word != DMAE_COMPLETION_VAL) { udelay(DMAE_MIN_WAIT_TIME); + cond_resched(); if (++wait_cnt > wait_cnt_limit) { DP_NOTICE(p_hwfn->cdev, "Timed-out waiting for operation to complete. Completion word is 0x%08x expected 0x%08x.\n",