From patchwork Fri Mar 21 02:15:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samiullah Khawaja X-Patchwork-Id: 14024782 Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7816A1E3793 for ; Fri, 21 Mar 2025 02:15:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742523326; cv=none; b=P9C/wC2o+8BGvbgy9VsEmT/HN/owxBd9kzX7tOr1qDax9g21TWL+fItdoEYgsW5TkGlupuxt1N2VgpwVcPvbB/sPYZ+4Wxf7v89pi/R8vazIAjH/t9etL98IDW20nYkEtzMypQKvQtxV2zrRUMqUUOHjSBJkickxhaNh/hCC7rw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1742523326; c=relaxed/simple; bh=pkOdm3LVE5bfGbqLu+M0r5NzvqaZDjVURBnY9zGQhhI=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=Jx8CKOxI4xDQH2G496fQeA8u44yjrdEICEpoV12wQ7NTLeUBa7Wn6+ELyASCcjlICu+AUxdQ4dpq2D3rDBOySSjOMzteP4xxsQw/zlEfveemlRMMEe3XwEMJU4aHWbU8WdSYxl7UWrcOTFMuHnecCpINSOKtj4eUXnd7STk86is= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=Mgi9lnCQ; arc=none smtp.client-ip=209.85.216.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--skhawaja.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="Mgi9lnCQ" Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-301bbe9e084so4160954a91.0 for ; Thu, 20 Mar 2025 19:15:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1742523323; x=1743128123; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=8LyBAErTL0Hl4/3apcGMDR43mJBGBGGCQlORuO+rrUw=; b=Mgi9lnCQYxelcvTznJl8ilmoROVXLiu7S5s/3K+hFfnxrtKwkzGfKo9n2g1LYchKG2 SORoDI+IdX+Blsi2AWRhl2xLYvJ5GCqKOEeKT4AXWi3i1jox97QhOY+t7vHvVPwTWT5N s2STzXI3M/qY7qdBjQAM7u+gpRk4sD6Dr1YKN/FxRvmgTjWEdBmJRR4HiT4SMeUpEQvl 2KW3ki2K4kbtmYZlv3tdoVgLfvYaPbM8Vh4Va/Fa5jmfxUQU0EYZY6OO4EeSIA74AKZm KO/3NG2Vge2ZmvNJUzvVjeXZzlcSz5XdeWHhrd81VOo+BkAnIBq/u2RVQXJRBZEPUG31 ERyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1742523323; x=1743128123; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=8LyBAErTL0Hl4/3apcGMDR43mJBGBGGCQlORuO+rrUw=; b=L+4BR/MyMjYTwqe0hGZo1Vl88qAXVbxqyOVn/aWjxhRy8YVQHsqec4S76QFBSSRX82 8fHTxK+Llk3F2k5T8ghRfkjDofQR5R8pnKRR6PXMAFrVmnQXWsH470aSNOneTs0EPXpZ vyEkmcxWHtJqFlX9V2UCVK7Fr1b7euExcs2SPQP5a70eKCP5wkgzPdS30v+FtyYBG/RI mZIIZlAeNTk2olIkOMaY64FQwaMskC+RRVbKaoIj5XQ0WJDqMH69tAz6Nb/jIqUmkacf VBL0OYIxjoBX8AICVBwwit0Gg8IeMjM+u1+CeF0D2wnSBvkG1hYcqO5GH9o5/nZ7bOLj Ltkw== X-Gm-Message-State: AOJu0YwWFlPI6EVs6e9Pre3Nes3t38VXtsBy3rFSd2c/jSUPOA/7Nm3Y 8ixZuN4J0DJIvPrmS1oYJpGSuCAV5cZu06U9gA78HgGOj27hAGYdDxlkppzAZV1Kq1/8Bh57d6p IRiNqGxeutg== X-Google-Smtp-Source: AGHT+IFsIrdlVf8V7Ie1vnWrS+zIzsvFIAuzsBf/iNjXjnF0TuldTSwMuv4Hdsd5vpon/1lbT4VOAoSE+Rp8sg== X-Received: from pjbpq18.prod.google.com ([2002:a17:90b:3d92:b0:2fc:1158:9fe5]) (user=skhawaja job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:4c4a:b0:2f4:4003:f3ea with SMTP id 98e67ed59e1d1-3030ff21efdmr2733328a91.33.1742523322682; Thu, 20 Mar 2025 19:15:22 -0700 (PDT) Date: Fri, 21 Mar 2025 02:15:17 +0000 Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.49.0.395.g12beb8f557-goog Message-ID: <20250321021521.849856-1-skhawaja@google.com> Subject: [PATCH net-next v4 0/4] Add support to do threaded napi busy poll From: Samiullah Khawaja To: Jakub Kicinski , "David S . Miller " , Eric Dumazet , Paolo Abeni , almasrymina@google.com, willemb@google.com, jdamato@fastly.com, mkarsten@uwaterloo.ca Cc: netdev@vger.kernel.org, skhawaja@google.com X-Patchwork-Delegate: kuba@kernel.org Extend the already existing support of threaded napi poll to do continuous busy polling. This is used for doing continuous polling of napi to fetch descriptors from backing RX/TX queues for low latency applications. Allow enabling of threaded busypoll using netlink so this can be enabled on a set of dedicated napis for low latency applications. Once enabled user can fetch the PID of the kthread doing NAPI polling and set affinity, priority and scheduler for it depending on the low-latency requirements. Currently threaded napi is only enabled at device level using sysfs. Add support to enable/disable threaded mode for a napi individually. This can be done using the netlink interface. Extend `napi-set` op in netlink spec that allows setting the `threaded` attribute of a napi. Extend the threaded attribute in napi struct to add an option to enable continuous busy polling. Extend the netlink and sysfs interface to allow enabling/disabling threaded busypolling at device or individual napi level. We use this for our AF_XDP based hard low-latency usecase with usecs level latency requirement. For our usecase we want low jitter and stable latency at P99. Following is an analysis and comparison of available (and compatible) busy poll interfaces for a low latency usecase with stable P99. Please note that the throughput and cpu efficiency is a non-goal. For analysis we use an AF_XDP based benchmarking tool `xdp_rr`. The description of the tool and how it tries to simulate the real workload is following, - It sends UDP packets between 2 machines. - The client machine sends packets at a fixed frequency. To maintain the frequency of the packet being sent, we use open-loop sampling. That is the packets are sent in a separate thread. - The server replies to the packet inline by reading the pkt from the recv ring and replies using the tx ring. - To simulate the application processing time, we use a configurable delay in usecs on the client side after a reply is received from the server. The xdp_rr tool is posted separately as an RFC for tools/testing/selftest. We use this tool with following napi polling configurations, - Interrupts only - SO_BUSYPOLL (inline in the same thread where the client receives the packet). - SO_BUSYPOLL (separate thread and separate core) - Threaded NAPI busypoll System is configured using following script in all 4 cases, ``` echo 0 | sudo tee /sys/class/net/eth0/threaded echo 0 | sudo tee /proc/sys/kernel/timer_migration echo off | sudo tee /sys/devices/system/cpu/smt/control sudo ethtool -L eth0 rx 1 tx 1 sudo ethtool -G eth0 rx 1024 echo 0 | sudo tee /proc/sys/net/core/rps_sock_flow_entries echo 0 | sudo tee /sys/class/net/eth0/queues/rx-0/rps_cpus # pin IRQs on CPU 2 IRQS="$(gawk '/eth0-(TxRx-)?1/ {match($1, /([0-9]+)/, arr); \ print arr[0]}' < /proc/interrupts)" for irq in "${IRQS}"; \ do echo 2 | sudo tee /proc/irq/$irq/smp_affinity_list; done echo -1 | sudo tee /proc/sys/kernel/sched_rt_runtime_us for i in /sys/devices/virtual/workqueue/*/cpumask; \ do echo $i; echo 1,2,3,4,5,6 > $i; done if [[ -z "$1" ]]; then echo 400 | sudo tee /proc/sys/net/core/busy_read echo 100 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout fi sudo ethtool -C eth0 adaptive-rx off adaptive-tx off rx-usecs 0 tx-usecs 0 if [[ "$1" == "enable_threaded" ]]; then echo 0 | sudo tee /proc/sys/net/core/busy_poll echo 0 | sudo tee /proc/sys/net/core/busy_read echo 100 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout echo 2 | sudo tee /sys/class/net/eth0/threaded NAPI_T=$(ps -ef | grep napi | grep -v grep | awk '{ print $2 }') sudo chrt -f -p 50 $NAPI_T # pin threaded poll thread to CPU 2 sudo taskset -pc 2 $NAPI_T fi if [[ "$1" == "enable_interrupt" ]]; then echo 0 | sudo tee /proc/sys/net/core/busy_read echo 0 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs echo 15000 | sudo tee /sys/class/net/eth0/gro_flush_timeout fi ``` To enable various configurations, script can be run as following, - Interrupt Only ```