From patchwork Fri Jan 24 21:32:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tony Nguyen X-Patchwork-Id: 13949939 X-Patchwork-Delegate: kuba@kernel.org Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 405B61E7C27 for ; Fri, 24 Jan 2025 21:32:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.13 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737754343; cv=none; b=hHy6mw8kwjtIYOAYEVZS1iElXAsscRdCOBXTs/q/rZ1k8uHEEGTJPDIwXOQJx65bRlgKdOoAReEAGuOsA51EdTxgYq3r3m6ODuzCOKGSGY1xs1Oq7IIf7U/WfvcddGkBZ1sEGO8RtbSubk1jxQikVOTiBax4aFzRif7AFFpL24Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737754343; c=relaxed/simple; bh=Iv3SD0WsKeyqpIbRwKokjasKKgiPm8GebeHbkz3lK3w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YFCLxfZ9W/vJ12aPIoOzxmNUDeTdE1UKbuRjVbqNIfO4FnW6o5RFtbzflZ3+S1uIbzTYUuNcy9bEiq7rHM5+nOb8ef01ltD/DyGXnxutv7aIo4frtTE5FvsN5LUnB46DgpjRYitXKmIc6YSJJ8foNmZSY2HfjK4seC+0azv9puY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=bUfjYBqB; arc=none smtp.client-ip=192.198.163.13 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="bUfjYBqB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1737754342; x=1769290342; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Iv3SD0WsKeyqpIbRwKokjasKKgiPm8GebeHbkz3lK3w=; b=bUfjYBqB92lxcvFYWO2EFGRpx9rclYnt3jpkzsfD574DTGH7FPzE3VXU SDfsc7Sbe1erc0QdpGKkTEUEWaLLYk4qC12BnmJzoVfqUX62jbZeu5ona qwjNejL3yuQMIHb4AgzSEsoRx39u046kj6fVFNjnHqMJut5HxxOOI5vo9 O3BTSOkj1NSzlWj4pyTWBxqleQmG/8I2+iSe0dT7XLgd9ayqopFCPqkjN G0bbu0s3G7CIhgl0XhTixklQ9HqbJK65P1WSqXBXwnoEPIivlATg8tJR8 tzZm4UWAjLWFtNLYdl0hOUhA6rzGRrCvm/LFIMYxGhHTjNNkDCGoUZ4Zd g==; X-CSE-ConnectionGUID: rX0JYwUqQHCcy6wWeJ8vOg== X-CSE-MsgGUID: R88Gy0TyR82Fv3NzExhcGw== X-IronPort-AV: E=McAfee;i="6700,10204,11325"; a="41140402" X-IronPort-AV: E=Sophos;i="6.13,232,1732608000"; d="scan'208";a="41140402" Received: from orviesa006.jf.intel.com ([10.64.159.146]) by fmvoesa107.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 24 Jan 2025 13:32:17 -0800 X-CSE-ConnectionGUID: qpYqG4HuTUuvWlaUfD8JAw== X-CSE-MsgGUID: i0Y+RkvQRtGDgtsGhW+ZeA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.13,232,1732608000"; d="scan'208";a="107861086" Received: from anguy11-upstream.jf.intel.com ([10.166.9.133]) by orviesa006.jf.intel.com with ESMTP; 24 Jan 2025 13:32:17 -0800 From: Tony Nguyen To: davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com, edumazet@google.com, andrew+netdev@lunn.ch, netdev@vger.kernel.org Cc: Marco Leogrande , anthony.l.nguyen@intel.com, emil.s.tantilov@intel.com, anjali.singhai@intel.com, przemyslaw.kitszel@intel.com, sridhar.samudrala@intel.com, jacob.e.keller@intel.com, brianvv.kernel@gmail.com, decot@google.com, manojvishy@google.com, vivekmr@google.com, willemb@google.com, Brian Vazquez , Pavan Kumar Linga , Krishneil Singh Subject: [PATCH net 4/8] idpf: convert workqueues to unbound Date: Fri, 24 Jan 2025 13:32:06 -0800 Message-ID: <20250124213213.1328775-5-anthony.l.nguyen@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250124213213.1328775-1-anthony.l.nguyen@intel.com> References: <20250124213213.1328775-1-anthony.l.nguyen@intel.com> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Marco Leogrande When a workqueue is created with `WQ_UNBOUND`, its work items are served by special worker-pools, whose host workers are not bound to any specific CPU. In the default configuration (i.e. when `queue_delayed_work` and friends do not specify which CPU to run the work item on), `WQ_UNBOUND` allows the work item to be executed on any CPU in the same node of the CPU it was enqueued on. While this solution potentially sacrifices locality, it avoids contention with other processes that might dominate the CPU time of the processor the work item was scheduled on. This is not just a theoretical problem: in a particular scenario misconfigured process was hogging most of the time from CPU0, leaving less than 0.5% of its CPU time to the kworker. The IDPF workqueues that were using the kworker on CPU0 suffered large completion delays as a result, causing performance degradation, timeouts and eventual system crash. Tested: * I have also run a manual test to gauge the performance improvement. The test consists of an antagonist process (`./stress --cpu 2`) consuming as much of CPU 0 as possible. This process is run under `taskset 01` to bind it to CPU0, and its priority is changed with `chrt -pQ 9900 10000 ${pid}` and `renice -n -20 ${pid}` after start. Then, the IDPF driver is forced to prefer CPU0 by editing all calls to `queue_delayed_work`, `mod_delayed_work`, etc... to use CPU 0. Finally, `ktraces` for the workqueue events are collected. Without the current patch, the antagonist process can force arbitrary delays between `workqueue_queue_work` and `workqueue_execute_start`, that in my tests were as high as `30ms`. With the current patch applied, the workqueue can be migrated to another unloaded CPU in the same node, and, keeping everything else equal, the maximum delay I could see was `6us`. Fixes: 0fe45467a104 ("idpf: add create vport and netdev configuration") Signed-off-by: Marco Leogrande Signed-off-by: Manoj Vishwanathan Signed-off-by: Brian Vazquez Reviewed-by: Jacob Keller Reviewed-by: Pavan Kumar Linga Tested-by: Krishneil Singh Signed-off-by: Tony Nguyen --- drivers/net/ethernet/intel/idpf/idpf_main.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/intel/idpf/idpf_main.c b/drivers/net/ethernet/intel/idpf/idpf_main.c index f71d3182580b..b6c515d14cbf 100644 --- a/drivers/net/ethernet/intel/idpf/idpf_main.c +++ b/drivers/net/ethernet/intel/idpf/idpf_main.c @@ -174,7 +174,8 @@ static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) pci_set_master(pdev); pci_set_drvdata(pdev, adapter); - adapter->init_wq = alloc_workqueue("%s-%s-init", 0, 0, + adapter->init_wq = alloc_workqueue("%s-%s-init", + WQ_UNBOUND | WQ_MEM_RECLAIM, 0, dev_driver_string(dev), dev_name(dev)); if (!adapter->init_wq) { @@ -183,7 +184,8 @@ static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_free; } - adapter->serv_wq = alloc_workqueue("%s-%s-service", 0, 0, + adapter->serv_wq = alloc_workqueue("%s-%s-service", + WQ_UNBOUND | WQ_MEM_RECLAIM, 0, dev_driver_string(dev), dev_name(dev)); if (!adapter->serv_wq) { @@ -192,7 +194,8 @@ static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_serv_wq_alloc; } - adapter->mbx_wq = alloc_workqueue("%s-%s-mbx", 0, 0, + adapter->mbx_wq = alloc_workqueue("%s-%s-mbx", + WQ_UNBOUND | WQ_MEM_RECLAIM, 0, dev_driver_string(dev), dev_name(dev)); if (!adapter->mbx_wq) { @@ -201,7 +204,8 @@ static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_mbx_wq_alloc; } - adapter->stats_wq = alloc_workqueue("%s-%s-stats", 0, 0, + adapter->stats_wq = alloc_workqueue("%s-%s-stats", + WQ_UNBOUND | WQ_MEM_RECLAIM, 0, dev_driver_string(dev), dev_name(dev)); if (!adapter->stats_wq) { @@ -210,7 +214,8 @@ static int idpf_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_stats_wq_alloc; } - adapter->vc_event_wq = alloc_workqueue("%s-%s-vc_event", 0, 0, + adapter->vc_event_wq = alloc_workqueue("%s-%s-vc_event", + WQ_UNBOUND | WQ_MEM_RECLAIM, 0, dev_driver_string(dev), dev_name(dev)); if (!adapter->vc_event_wq) {