From patchwork Fri Mar 28 23:10:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032486 Received: from mail-ej1-f48.google.com (mail-ej1-f48.google.com [209.85.218.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8C1B1F4170 for ; Fri, 28 Mar 2025 23:10:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203425; cv=none; b=JqoLZYnRlyxEI9MVjxG34YyonIMrGCmU5K32E4jGLYNKWFU3L0iNormm0iAJo1EYh6AOxZxIMgyPGtuOwc6Ue1p/TWw5THhrf4tvy2bYomjx3tQKAMbha4o4soVcVpCaugklJWE6qYvQZd7/LNLHXvyePO8HEDX1QXq+ZHNVKdM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203425; c=relaxed/simple; bh=Y/fiP1v+FsUuVLuzHz7CU2h58F31zxI8C1H002mhp64=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PAnsOCg/zQq/+/4qBkvGInUXuEaJy9acttMtp5pTqTVOMU480QJ0jQ7JERo8Si6GKuhCmwK+SIE/BYS7vQasa6IhOIXy70Omo96J1B2Htc3kQ33PPAQp4QVLCS0gVpKV8SSA5fJn8P7NNeJwUNP9B6OjxvbDSOsT69ZYRl8bi+U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eOenYfFn; arc=none smtp.client-ip=209.85.218.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eOenYfFn" Received: by mail-ej1-f48.google.com with SMTP id a640c23a62f3a-ac2bb7ca40bso520250466b.3 for ; Fri, 28 Mar 2025 16:10:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203421; x=1743808221; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=56KfdtGJFtkzbDr6PNEAPYr+/QVEqxl0tvfDOuXU6/w=; b=eOenYfFnXWsD/0BAfuJ+d/Bk1HHg5cjVDizqnAm2EEMMGNRC/WFn7EX5I96/6nct+S xAzbIkCEWMV/i7BQFqEgUAVWhLDKuBBM0Vl27UyZXcnPpPDkw1kmcdMFnEKLUj2xPQ47 hI9fNp0MIFRv1Ku1q//szaiR6WspapM7MeP+UBgJaY9yRfB/6caugi3lH+Gu2tdd+sqI ebthdV7dSvF6vt4cvRJRLp1z/rm1tNdmmxy4wDlRYNwJ4rjCeEs22mnRVdg5tMyLoc4Y HxFzxYr4CacKCtHagRJLspF0zltHD6zVdqoCFvkJHY279dNhHQqqCAFhVjW+fid3Ep1R ofHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203421; x=1743808221; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=56KfdtGJFtkzbDr6PNEAPYr+/QVEqxl0tvfDOuXU6/w=; b=E4QOMmXPYTOq4zA+/W3+CtADL9Z/NJTOiSW3wUKOIqaMX3oVnfyzw/Iq0funHBF6Hc UJvBheNfSOXEsBjCy8rgUr+XyazChblEsohCjH7t+5ZYQiwzno1d8EKovTQgclY9Vkn+ McweD+z/wes61dI9AgiM7jPBBQEQ55oAM+NTG5xcYH/SC6qOdukAYu3vrdCTWI4VCcxs uw+YXeXnQ1LOhhBBlklbu7x/06sHqHAy5GgASRa8uwY9EW5BdsHJ01VjavX2iR7NV+NH 5B09lmWwLm68ZbYXzaxZz7PLikGwe380CuqgdQBnA804AQwK4LR30NypcCZE9tJgCmzL W22Q== X-Gm-Message-State: AOJu0YxSowsuKL8BZ0vqbVP8FQrgF2YEHON5Mjc5pwTRqMG45Lkm7v73 I9hxTDiKtlRtSwtfIb2jDaXjQLt50B7QECBwzpWIc4RlU0z+GnFHRTuYFQ== X-Gm-Gg: ASbGnct2uV9Dns6Fmc3iXkJdUyKVZg5MM+dMiSndydRCuqagIJPyrJEqYLgW/CTmvVx exY1Tf5iC+sjSb+PL7F1Gf7MKQ6uIoyUazHeBFJy9cU+luOLVtcSEjk3CwZzvifbzygsHNSV6b7 HqxI5vdpjJxpHXQ2TXaL6amLSo3oGs8SPHUB7BU5ebcF+NgbvzA/aPVZt4wvASDEmYttO7ET/+V wZnFBAOlEhSoOVEpMcG8CTNqN3tUF/1L9gg11hXW2cJOxASdHY0r0cq6pi21VUCqTPcuo/Vky1q eLIRKecnmbMp5bKxVJ7isMx4OEEaXWHBzVVitOETScLagjn3ZKMuPWTB9B4= X-Google-Smtp-Source: AGHT+IGPo1ceVUMl81lftza+0DqjxWdR1paJSOBvzhzxMVnL9jQa4ixuev85lZ1QAej92QvQDRJlzA== X-Received: by 2002:a17:907:9484:b0:abf:7a26:c47b with SMTP id a640c23a62f3a-ac738c1c598mr84169366b.39.1743203421225; Fri, 28 Mar 2025 16:10:21 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:20 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 1/7] io_uring/net: open code io_sendmsg_copy_hdr() Date: Fri, 28 Mar 2025 23:10:54 +0000 Message-ID: <565318ce585665e88053663eeee5178d2c15692f.1743202294.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_sendmsg_setup() is trivial and io_sendmsg_copy_hdr() doesn't add any good abstraction, open code one into another. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 30 ++++++++++-------------------- 1 file changed, 10 insertions(+), 20 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index a1d32555fe6a..fefe66c2f029 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -325,25 +325,6 @@ static int io_msg_copy_hdr(struct io_kiocb *req, struct io_async_msghdr *iomsg, return 0; } -static int io_sendmsg_copy_hdr(struct io_kiocb *req, - struct io_async_msghdr *iomsg) -{ - struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg); - struct user_msghdr msg; - int ret; - - ret = io_msg_copy_hdr(req, iomsg, &msg, ITER_SOURCE, NULL); - if (unlikely(ret)) - return ret; - - if (!(req->flags & REQ_F_BUFFER_SELECT)) - ret = io_net_import_vec(req, iomsg, msg.msg_iov, msg.msg_iovlen, - ITER_SOURCE); - /* save msg_control as sys_sendmsg() overwrites it */ - sr->msg_control = iomsg->msg.msg_control_user; - return ret; -} - void io_sendmsg_recvmsg_cleanup(struct io_kiocb *req) { struct io_async_msghdr *io = req->async_data; @@ -392,10 +373,19 @@ static int io_sendmsg_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe { struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg); struct io_async_msghdr *kmsg = req->async_data; + struct user_msghdr msg; + int ret; sr->umsg = u64_to_user_ptr(READ_ONCE(sqe->addr)); + ret = io_msg_copy_hdr(req, kmsg, &msg, ITER_SOURCE, NULL); + if (unlikely(ret)) + return ret; + /* save msg_control as sys_sendmsg() overwrites it */ + sr->msg_control = kmsg->msg.msg_control_user; - return io_sendmsg_copy_hdr(req, kmsg); + if (req->flags & REQ_F_BUFFER_SELECT) + return 0; + return io_net_import_vec(req, kmsg, msg.msg_iov, msg.msg_iovlen, ITER_SOURCE); } static int io_sendmsg_zc_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe) From patchwork Fri Mar 28 23:10:55 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032487 Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E69461F180C for ; Fri, 28 Mar 2025 23:10:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203426; cv=none; b=Iozd+ctUDpnrn+UPezPUKYYBi+nbdEYORt0anvNN3+TMFzcatXAstAoMWoh49g5ospyxLu5AuLKLpyAKt6RCuoLzI5JHFOKI5p+gVzBY25BZX3Qa5uMQArarjkGRliDJZcvxtnTzWl0KJuql7bacfcsIBqHriuSuqS6SN/yQh5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203426; c=relaxed/simple; bh=pfohiY3gLhmEF6mm//S5JjS//jtAAN9DLtVTTe1wM1I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r7vMPoKwkJFAAjJ7lXhsrjE3LCBvvSLGGniyVhevpTAaHeyz0k1YlpHMfIK85yGtFy/jTndwgXLYfeeHlq+ZVc0oomED48NTwjMibCD/wnJdTu40xNK9qGKmlYKBJkExUTDqktRVAfIztAFxEJ4Q8lCep8vaucWKFv+SubozihU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Af4tsUHL; arc=none smtp.client-ip=209.85.208.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Af4tsUHL" Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-5e61d91a087so4354418a12.0 for ; Fri, 28 Mar 2025 16:10:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203423; x=1743808223; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d2/Vq2XFEWyYZ9UT//dodrp0lMZ5Xmzy/2U4EiSVBGM=; b=Af4tsUHLNVKUCzdvchQWISmRBhkpdrjVqEPf804bli9uH/R130IrE78/kXuVIW1am5 ywsICObMa680XlAIbX2rSoYW4NvMt2APAIbzPBu4+7AHmnI/fP/kGkqitTUhOKYUmFit gWQkfZKliwN+OJVr6kltuwRzNxDTuiVjsGZbVzcDdN5IcUNSazCKjethQwKuFKPKbwb8 ejpkqVy1pf9rBIZoP5pzdZUS1GTke9tAD4ja/wn95SbDsDNQJxjenDlfklQV3Ie8xlwZ i4LrScNa60Yj0ac3H1lgAz+tXkW2UlQMrbSjZ2jWqyV6T/eE+2/Dui8PHvmcGhh0aSuD H0Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203423; x=1743808223; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d2/Vq2XFEWyYZ9UT//dodrp0lMZ5Xmzy/2U4EiSVBGM=; b=VwLzemJSE47biB3/iSwDBeJIUBcpQULgpYs2V6rFkdxVLYJjte1LNHWvtYpo6sU7di jMDjvh0a2CI8w0B3jHeymxpkDUJ/ZznIyHdlHOv7aQ17sG21EE0KelEifONBodH9pr3h R48gK1+V8dv5XQUbeaPEvIc/nFGSxoXLtc+p8LmX2ushQs0lKuiCVGcnaNNYOXFo1eg0 Owr0jvQoRP0AtlE8NpjBrOQ3vQASu56n4rYb3Brk+SKYzf9C1eMJY7hqSMQnpe+Wowew rQI2sfhdwjMCn4RXfTCLpshn7CvmbVsBmBm1EGyobx4gQLzfsVUkRpxGxN8EKNlvbbHq Q6jg== X-Gm-Message-State: AOJu0YwAXYxou/m5u2cTE95ctxfHxkXa+fFNEH73ALYwEvek1me4kdaU 8EmKI3/s3L7a6ctjAF/GglGe7z3rI8/MvcStiiLuMPwG6HngB103OmEgSA== X-Gm-Gg: ASbGncsSym4nZivEmdzXmtj+hjWLP8fMVdDCj8amOYkFVmYiJV2AIQLk9bWGbpcRLVN 77vXIQ/8s9UHzo+ujypdnD7G5JJ6wtdvSWPOVYAqYBTRKPnjvPS+cksi5gDIckAbHbHRPFG0He2 2efkscerJ9irbfUnCH9dV1vVQ6QJqKQXxKKdKeaOLS2YnJuXnDR0vyPKF0acEVeC368qk2M49H+ WLF1knbr0PjTojNdG/xVWKT2ZMXY0T8xWPlsl0p0X+fvU21uYSpLVs7VuBCk7ZYU2ArgI02BluH 2a7LnThP2jNL5ccr63KrbBAYk2ejaHmZ9JF9mVIJtQ9jUnjA768yYRJ0Ph/Z3lpdGt8D8Q== X-Google-Smtp-Source: AGHT+IFUu3d3A0oLU7SFZWIJGMCKOXnJ1dsT7gYa+nV+FHNz7sQ7XhSDXfh3daBWa+z4b9A8uEpMMg== X-Received: by 2002:a17:907:6d27:b0:abf:e7c1:b3bf with SMTP id a640c23a62f3a-ac738932c5bmr75341066b.11.1743203422609; Fri, 28 Mar 2025 16:10:22 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:21 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 2/7] io_uring/net: open code io_net_vec_assign() Date: Fri, 28 Mar 2025 23:10:55 +0000 Message-ID: <19191c34b5cfe1161f7eeefa6e785418ea9ad56d.1743202294.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Get rid of io_net_vec_assign() by open coding it into its only caller. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index fefe66c2f029..78c72806d697 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -176,16 +176,6 @@ static struct io_async_msghdr *io_msg_alloc_async(struct io_kiocb *req) return hdr; } -/* assign new iovec to kmsg, if we need to */ -static void io_net_vec_assign(struct io_kiocb *req, struct io_async_msghdr *kmsg, - struct iovec *iov) -{ - if (iov) { - req->flags |= REQ_F_NEED_CLEANUP; - io_vec_reset_iovec(&kmsg->vec, iov, kmsg->msg.msg_iter.nr_segs); - } -} - static inline void io_mshot_prep_retry(struct io_kiocb *req, struct io_async_msghdr *kmsg) { @@ -217,7 +207,11 @@ static int io_net_import_vec(struct io_kiocb *req, struct io_async_msghdr *iomsg &iomsg->msg.msg_iter, io_is_compat(req->ctx)); if (unlikely(ret < 0)) return ret; - io_net_vec_assign(req, iomsg, iov); + + if (iov) { + req->flags |= REQ_F_NEED_CLEANUP; + io_vec_reset_iovec(&iomsg->vec, iov, iomsg->msg.msg_iter.nr_segs); + } return 0; } From patchwork Fri Mar 28 23:10:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032488 Received: from mail-ej1-f42.google.com (mail-ej1-f42.google.com [209.85.218.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99DD41F4170 for ; Fri, 28 Mar 2025 23:10:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203428; cv=none; b=R9UAiPbgnIXRwk6qiv7G/oXe6diIwlQrXfTcQUFeqRejlwzPWqFPnd9etwvwm2gOGNFohuZa4+S6XPAoVBkgkKgjrvVjclfhhqfJ61kuSPuFfToDohI481a35/pK+/junpZvxAZXpCULySsk/o2sVGrXerb7/MyoLpKyvsr7wG4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203428; c=relaxed/simple; bh=rqsDQJ7bq4hI0AISnnf+b1Yl4yyvzyHX9OPc6L8E5UY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FAr4mI0BMnKPFcgY+VKsruA410d2syPX8YTbwaZTR877iUoksvaVg95h6J+hYoT3OBH2Rz+xLw5IXfmClgdnGgIMLmuQIp+hqmVfmFHB0t11SRh2549RprGdIepPZY8bPEjzS/LJ3UR1YhQ6WaJm/dPG22ma5CPA3V1Hh0ky3C0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aiIOsU0j; arc=none smtp.client-ip=209.85.218.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aiIOsU0j" Received: by mail-ej1-f42.google.com with SMTP id a640c23a62f3a-ac2902f7c2aso456656766b.1 for ; Fri, 28 Mar 2025 16:10:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203424; x=1743808224; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y8joMM0e9MKFzlH1siqj7abLLw9+811LBtPm8cuQc14=; b=aiIOsU0j9WPpCNPEN5QcjjhiPBt9A/glerl7HntS3JPGrzg/EpZaaPHbODPbmhDGzo crctu9n0AEkc1LUCJ3GOet/wP/ngNKUPXXPp4YvXyVsXUPrqcyKXHoAwnikD4m3ligPb ATIX9jknAfL2M33y3EgOzVc55RoZNZUOq6w8schK0xVdD9oya7DKDcdNhY52JYbWbAVd WlrJlqpRw16GQSAeqEmyZz+SnAupyjTGyWx8pDYqkfL86ztrYMsH0BS1nUE157LG5v9a Xai+yf6hijdo2qCVCBjVqdErkk5MAwI10S10SipqP8XVcSzWAxd1Uzx4qYdscIZASiML eEkg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203424; x=1743808224; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y8joMM0e9MKFzlH1siqj7abLLw9+811LBtPm8cuQc14=; b=cCBU5PaUH/+0PEp7LnKmg3Pinxz5JBPNjB/as/RX6UhoDVzedUlHk+mGuDfudzKfvv y/fWnWR6+ERNzwhtXQKOlCu4l9iPVfjRkmdSHWmsVtGWO+D0yX7Pgn7A7n7nd/xzcPmX mL7geIIBbYR+2densUZ2QrINVBkMpbHE5w1Ps9tqyJfxFfZePxKdTO+ASrl1R3Yz3GTM 2nx/btBl0Av80Y7dQ2KJM3H8WzzlI6IDLTJgVIOEE26srauA1H9BwABR4pS82KWQ8LG0 fn/O/1jYZxKxVtqn8ru7Wo69RETrfoczOfaSlZLPMls4HH8Kqcpl+CG+rtt1c95Tb0rq dZxg== X-Gm-Message-State: AOJu0YxJnX7q7vkR7aiHZtP9/nxFu6yrcPe4irT9ZXdgfzArHQ1rx4ao 3COehW9w+/xqRFdWqVYPZC2xGOzwBtSh3Qi2D7pj/bvZgTcgIQlj4pUqbQ== X-Gm-Gg: ASbGncsKZzvPhTKocSpuJK5357CA1so4hprqY9Yr9tXrxbXLiidqlqPJDpqlQe/TFB4 69JHwpcUELTPh5rz6C6rW3Imp4ugJaQXxENVOf6phS2dHPjFYKnuQBxU3BQmm9L2LxgrOfSkXce vjLS4/Kr88ID1jweB8woM1cSAtt6mLz/bD6K2LCt3KtIyQbHZ4KDd5fbBFk4SFcb6sk2TnESFM7 wQCZntO1oPPbkDvKnuIwBrmzeinFvfoZNnI2wjQuTWz2qkiu+vqEZe+eula2i3w1nFvfEw5XvjO b39s/s7PAf6qMHy1KXFWx6IraVZfXyVJNCSxxfC4sTTC+jTnj+pP1JvH908= X-Google-Smtp-Source: AGHT+IHWjrTvolVyc7Y3ks//A6mvyjb1nSyxy37hJLH/IjC8dNNFV3NNeElVwcIQ4Rj7L4yOE6JGLw== X-Received: by 2002:a17:907:3f07:b0:ac2:9093:6856 with SMTP id a640c23a62f3a-ac738bf65d7mr78242066b.54.1743203424157; Fri, 28 Mar 2025 16:10:24 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:23 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 3/7] io_uring/net: combine sendzc flags writes Date: Fri, 28 Mar 2025 23:10:56 +0000 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Save an instruction / trip to the cache and assign some of sendzc flags together. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 78c72806d697..e293f848a686 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1292,7 +1292,6 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) zc->done_io = 0; zc->retry = false; - req->flags |= REQ_F_POLL_NO_LAZY; if (unlikely(READ_ONCE(sqe->__pad2[0]) || READ_ONCE(sqe->addr3))) return -EINVAL; @@ -1306,7 +1305,7 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) notif->cqe.user_data = req->cqe.user_data; notif->cqe.res = 0; notif->cqe.flags = IORING_CQE_F_NOTIF; - req->flags |= REQ_F_NEED_CLEANUP; + req->flags |= REQ_F_NEED_CLEANUP | REQ_F_POLL_NO_LAZY; zc->flags = READ_ONCE(sqe->ioprio); if (unlikely(zc->flags & ~IO_ZC_FLAGS_COMMON)) { From patchwork Fri Mar 28 23:10:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032489 Received: from mail-ej1-f54.google.com (mail-ej1-f54.google.com [209.85.218.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10A2D1F09BB for ; Fri, 28 Mar 2025 23:10:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203429; cv=none; b=t26rUJ7Jj6SGV9CqUdpARTPPNPEtJHv5fC7B74zIhlFsyoCGDRKgdcb2OgOjIXzVxgmA4DqPOSZvrhlDxpQcOXcvmlLDL5Ioq0H+KQ7c2naQCLinJ2/TtRz/2WKkxDapYeah0xypmaXUQv2EqwgUA0rMwAHZjyXw3aJR+ADRy4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203429; c=relaxed/simple; bh=MRwPtSTtoUKe8z6mtJ6YexyKPQcc9oq2ThB6nlAw6hg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gKRcgsxRGQs5jJ3GN5LmX0xDXZLZ+QcCMVyojYwXsJhK1kYUCFY5Xs3htGgOIEtDWkj/zKtB90R+ElhrMxSydPA9k1JleS6wspJfID23qxBhsW2kv3fbccrnV1Be1WiV6l9uQgGSy6GY10SNJnlv8W3OSMaVLiIK/Q4QWzQu4SI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hcA0eHde; arc=none smtp.client-ip=209.85.218.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hcA0eHde" Received: by mail-ej1-f54.google.com with SMTP id a640c23a62f3a-ac2af2f15d1so351065966b.1 for ; Fri, 28 Mar 2025 16:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203426; x=1743808226; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QpLC+APiMQo6GwLi+pSxmIAR52vwZUqDPZ+ebbatanQ=; b=hcA0eHdeA6z5YINto1S5owNAY8SSlGiDUWhpLX7ck/PQEGwSSSecjw/vlw4Xc5rmM5 kOMO7pOYaIPFC9WvtTxsUWxzo1oUeaCiAUUF2N37DdcEqg3hdG0fUqNdk1WQjdy7DtR2 ZcWEDWn8QSGJSVK9bIdkQFjjkvosUgONsJEwUJxat2sPNcgh8UXCiFewV9493NCf/BUm bUzd7uQD4dN8V96EMj8gq4Rpu8MPru2wCvDoMD6Av8oXd0Hy+ArGqYBIxDCAgwBg2NBK VZmff1HSUc+7dfrvhjTOeTZ8R1WtzLKpNOx9Z4IYGB+mEwuLeG9rS25KuMAUDC1haBbn T/Ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203426; x=1743808226; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QpLC+APiMQo6GwLi+pSxmIAR52vwZUqDPZ+ebbatanQ=; b=wOxA+iVvUjkMEIoGdEeOvxhlIPMT4jKFv86HvkL0ZGBns85pzvl09h24H9d14TcDQM 1oXECfniLuGwh4Gi3aXRGJ09ZfEJe8+fB8zLf+8TiPRhCfM2OgzyANMCWEhdZJb0CG82 fDhXOSLTFIrfwatz8edf19dbdOMRgDrCL7cPlo+gEBZrvfmfVh7DnQXXGD2wQWmhNCGk 5bb/qnVMj1boFBphmPaashG6lJnTz9SDh63zUCypBQiODLcHg12E7urpUQgyc5xRfl8r 87G818lidyIIK+Y009uQFpDjE0p6qEdnGFCGQLYx9ZKcIYI7FXUtuoAHRBOur4LcCh4r 0wCA== X-Gm-Message-State: AOJu0YwHJLLoexVtY3t4YZT9qsuMYNfGZsNPeogn75Er3K90VYkBPwlG qdwUozkFKMaWG72imZP6R4SLRTVO9Rzo8/lS/JrTS7yl0GeiMHmjYjamOw== X-Gm-Gg: ASbGnctjS2IOnHb0gCakTPmedqL70qLJLUwkG1Wi2XJ11y+QmbwGVWhSWhVPd8q7ulE Aq0ToShXY6FBs+rq5xl/G8OqWUSTqbQpOMcUJf/yI80zrDYOsEMGKel/1rYOGY/ccwtHGAwXD/q 2IoyzIdb7QGbQHaJXR1QZ1OybVsWTp0T2MfpsE2zph6eDq7gkTjYgvb4kg3mFpn8NOENCYz+Gkc QAQKRbesN0uAD+oXggZjMdJvnZNcZHYvyLqM6vWZjkrelPhzDZObw04u44spGhPIlaJvaQC1/i+ qhBsUqrqx4Jn7HOE5PmTH/sc01UyBk6ecV56QliYXW9VcFUspMt6ZjlZBnU= X-Google-Smtp-Source: AGHT+IGoglNTmlquoSng+mSKPAfJwhQz4D7rop41nljG8h6cv0GkXzitrPJWRdqol3IN0CXUgX57Yg== X-Received: by 2002:a17:907:2d87:b0:ac3:cff:80e1 with SMTP id a640c23a62f3a-ac738baeab8mr73516966b.56.1743203425532; Fri, 28 Mar 2025 16:10:25 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:24 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 4/7] io_uring/net: unify sendmsg setup with zc Date: Fri, 28 Mar 2025 23:10:57 +0000 Message-ID: <7e5ec40f9dc93355399dc6fa0cbc8b31f0b20ac5.1743202294.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_sendmsg_zc_setup() duplicates parts of io_sendmsg_setup(), and the only difference between them is that the former support vectored registered buffers with nothing zerocopy specific. Merge them together, we want regular sendmsg to eventually support fixed buffers either way. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 28 ++++++---------------------- 1 file changed, 6 insertions(+), 22 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index e293f848a686..6514323f4c60 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -377,32 +377,16 @@ static int io_sendmsg_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe /* save msg_control as sys_sendmsg() overwrites it */ sr->msg_control = kmsg->msg.msg_control_user; + if (sr->flags & IORING_RECVSEND_FIXED_BUF) { + kmsg->msg.msg_iter.nr_segs = msg.msg_iovlen; + return io_prep_reg_iovec(req, &kmsg->vec, msg.msg_iov, + msg.msg_iovlen); + } if (req->flags & REQ_F_BUFFER_SELECT) return 0; return io_net_import_vec(req, kmsg, msg.msg_iov, msg.msg_iovlen, ITER_SOURCE); } -static int io_sendmsg_zc_setup(struct io_kiocb *req, const struct io_uring_sqe *sqe) -{ - struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg); - struct io_async_msghdr *kmsg = req->async_data; - struct user_msghdr msg; - int ret; - - if (!(sr->flags & IORING_RECVSEND_FIXED_BUF)) - return io_sendmsg_setup(req, sqe); - - sr->umsg = u64_to_user_ptr(READ_ONCE(sqe->addr)); - - ret = io_msg_copy_hdr(req, kmsg, &msg, ITER_SOURCE, NULL); - if (unlikely(ret)) - return ret; - sr->msg_control = kmsg->msg.msg_control_user; - kmsg->msg.msg_iter.nr_segs = msg.msg_iovlen; - - return io_prep_reg_iovec(req, &kmsg->vec, msg.msg_iov, msg.msg_iovlen); -} - #define SENDMSG_FLAGS (IORING_RECVSEND_POLL_FIRST | IORING_RECVSEND_BUNDLE) int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) @@ -1340,7 +1324,7 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) req->flags |= REQ_F_IMPORT_BUFFER; return io_send_setup(req, sqe); } - ret = io_sendmsg_zc_setup(req, sqe); + ret = io_sendmsg_setup(req, sqe); if (unlikely(ret)) return ret; From patchwork Fri Mar 28 23:10:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032490 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 646F61F4167 for ; Fri, 28 Mar 2025 23:10:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203431; cv=none; b=D/HYcijGPvVVGFErNfY907OTZIGtvQBRoxYOW7zvrVHkypM7xBp9Y5/xK0EHE9ClQRU98ZXfXZrm7LPlA75M9zt4vSdBm/NFsz5bpHkT8fFC7utp8cjVfvBDk22ep2oJ8rl2Xs/DgWzgQfSqicXWQEkpT1V01hdEL6sJAsjaqDI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203431; c=relaxed/simple; bh=Sh4H/2tYm/gesORQ7UXULXbYqsk4VSd2hXxSsadCTvg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pt06ERWrHrLrOdnLfKLjp6y6dS5udGULDBwJmJW+zI9I4MSvS4RfoPoFLurvbP3qCoJ3ZFSqZbDusuYP4rh6poXMfrnq0C40nqv/1rouD+zrkr9Vqzr+Tx0pDkZclzzaCjgZwYzNSNTCY5pGQWCwqA+D1KJGmKyuElu3unJkm5A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QFxV/liv; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QFxV/liv" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-ac3eb3fdd2eso529302366b.0 for ; Fri, 28 Mar 2025 16:10:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203427; x=1743808227; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=utAzfAKpwciWumQY5hOPviH80Nc7+i4uRCtsmEiZtiU=; b=QFxV/livYistNlfSq0KCXoiKFBOckzXVE0JDMT/lxavYw+mT19JEdYaDtPbP3jlE2a iejxnYNPzyzzEZz9XdG83aLRoiX7kyUGVFyCLgD8CIibHD/MBbwdWIbxz6opzVrrDnie 6Pg3PNj59L9UJ5jRAlkk3Gag3H6nMaugMX9OJFU5lttscD94Hg28+izyAfzWgzveOo2g FVXgg5lIKmF+W3/DxatbE3YnTUXojn9k0acsde/9BMrnHUYbhG/UlTQTa7Shv3I3QCK7 6+9CuuEZz5eivvW6T4PZg7MkL/jgNeBH15oeEtitTUnLwxsvOStTtQKH6x0fijRNvdsJ MUyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203427; x=1743808227; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=utAzfAKpwciWumQY5hOPviH80Nc7+i4uRCtsmEiZtiU=; b=PQ4Uc5DCYyj5oYYCyuQBNw61/0qDedSvVhLM5iEVH0smUi7CKQG6sWnReKsorNDaxz 29STAkBfAdVBNVhrryXVeySZCRD4WA0reHs75DMoh4t/IMZcOJUC52Hsl5GKxxV09Gqi cKAtdi4gtyG9uXzFEZLrLSFenaL/Z7Ck/ZIz59+jsFG+U0Gok9o1wsD5Fc+Un1LYMA0L S9MBsDaPvr6V/+IRx8BNt2a2Be9gK4qQ1CnUnEhPmymPoOYTnWV9N2crGruwiawGgMdq sro3/1tzbBBYPbj8qQ1zbgX8L8up1trXpB8dLjcJmG9ShKMubQh5EBQ1vslbwS/uKJyA VSbg== X-Gm-Message-State: AOJu0YwFggeAQDNW2csW9Pu5oQmCUYd0WWJMIFpVN8edeUH0ikei5PUa gMwoDerTbBrYeCRkCQ9QGEBjSSJTVOYzvFqtItG1HfQ1jHuD9J8DcVUDsw== X-Gm-Gg: ASbGnctCRN4OPyMwTHdm/wNGD7vlrhFvIIa+dIs6hZyP9N7dhzfy3Q7qExUXB1eaV6J eMr2XaD4E6rtCg8xEygJDHoFTe5RahlBYahBFbH56YfbpF9XtnORRN0H1aSBgR0uZNAMUq4fj4f 9v5AFGriCg+8zrUs6+ZKkAuuyGpcxWgKf1jYYbV1IY6ce954inoCEV2yenogADWRsO/kKHEllqA gH6OA51qgsDGOcPMcw71ti2PtiXYmAm6d893ewPSXWWUrgXExTwpKHcrHGQod49znuTFVgn8Wsc cE59cML78w+Blj2VQMOLMIVzQmOKdrzwQxXTqgfGHgJ5UOc/i4COJVft7bA= X-Google-Smtp-Source: AGHT+IEsaY3nHB/nkyXEEHZDqC7azJjvlNLox7t345cDzLL748VHTfQQoA+bHXTklmFTWfU+Vb/bkg== X-Received: by 2002:a17:907:72c2:b0:ac3:3e43:3b2b with SMTP id a640c23a62f3a-ac738aaa69amr74071166b.18.1743203427053; Fri, 28 Mar 2025 16:10:27 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:26 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 5/7] io_uring/net: clusterise send vs msghdr branches Date: Fri, 28 Mar 2025 23:10:58 +0000 Message-ID: <33abf666d9ded74cba4da2f0d9fe58e88520dffe.1743202294.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 We have multiple branches at prep for send vs sendmsg handling, put them together so that the variant handling is more localised. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 6514323f4c60..9162dc6ac5e9 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -395,12 +395,6 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) sr->done_io = 0; sr->retry = false; - - if (req->opcode != IORING_OP_SEND) { - if (sqe->addr2 || sqe->file_index) - return -EINVAL; - } - sr->len = READ_ONCE(sqe->len); sr->flags = READ_ONCE(sqe->ioprio); if (sr->flags & ~SENDMSG_FLAGS) @@ -426,6 +420,8 @@ int io_sendmsg_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) return -ENOMEM; if (req->opcode != IORING_OP_SENDMSG) return io_send_setup(req, sqe); + if (unlikely(sqe->addr2 || sqe->file_index)) + return -EINVAL; return io_sendmsg_setup(req, sqe); } @@ -1304,11 +1300,6 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) } } - if (req->opcode != IORING_OP_SEND_ZC) { - if (unlikely(sqe->addr2 || sqe->file_index)) - return -EINVAL; - } - zc->len = READ_ONCE(sqe->len); zc->msg_flags = READ_ONCE(sqe->msg_flags) | MSG_NOSIGNAL | MSG_ZEROCOPY; req->buf_index = READ_ONCE(sqe->buf_index); @@ -1324,6 +1315,8 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) req->flags |= REQ_F_IMPORT_BUFFER; return io_send_setup(req, sqe); } + if (unlikely(sqe->addr2 || sqe->file_index)) + return -EINVAL; ret = io_sendmsg_setup(req, sqe); if (unlikely(ret)) return ret; From patchwork Fri Mar 28 23:10:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032491 Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1EA81F4C8B for ; Fri, 28 Mar 2025 23:10:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203432; cv=none; b=LhJKk10NLNoh8oO3FPkrImwSWP5H07W5ckCVzPdbx8oh4XnPTcDtoym/mA9qbV2BY+6REEHCTbpm6fOBssZAoqjBGinoyheIufsDsirtfnOVueAguRvrsg7WwuY0HUt01hKi57BI4oZ58SS16J4KWcSQ5UYVPAdgl8D4zXF1rYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203432; c=relaxed/simple; bh=iIvhi2AVRhhXlh7mBMZZJms52m2vl3jzgThUmvX4jNA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lNMa+GsoKWSkrDdvYePnhx+DWCo7yYahXdMP0eynmoe/Yx4x0wxH+yli4gpfvk3oDvaeGPVsvCxw0ToU/6K2kECSH6QjSkveULLcl5iuIP4EBwxT7svJ8Cm1q7A69bEqKK9h+/Z9/6/lggvJd6SOEvuIayLZk+PH6iNCfb86qdo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=AWqo43mA; arc=none smtp.client-ip=209.85.208.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="AWqo43mA" Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5e66407963fso5109153a12.2 for ; Fri, 28 Mar 2025 16:10:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203429; x=1743808229; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/cy7Sop//9jk/fDE3IuJ+GB3m0VFXoWlKNpr+QpyH+4=; b=AWqo43mAhk/tfHTglMH2Mf2i7XmJnopdySZr+bgcgI0X98+2X7ZxVsLhDtGrSr6j7S /qkQIYYrEVOvwgJWaIUCpLE7ak9CRrcb1ubgwZVh66O+bNZeWWIQX03rH3rDBWq7G/Qf ortglQSPBHyhb36P2/61j0OCmrY91mU8/Ros06z1iIiQNOTUShM/XU7w8Bf9MavNs1lG c81vLwGdL+cWU9VzXYLINyF8BGMPqZmQuEOeO7XQllMkuVepOO0KNqU48nUJ+GlQ+GmR FKSQ5a6828xCV8l6TRF3Wu8iXh05cgqweM4u5DCCsKazEewTUHIlqOiYC61eM1Ofx8W3 XtVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203429; x=1743808229; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/cy7Sop//9jk/fDE3IuJ+GB3m0VFXoWlKNpr+QpyH+4=; b=Ap62Xe7kq9EWt3sB/hAvxkX+hD2xsV/7++2LBH5R8Odaqhlh/vQZKpGVre0gZy0UQY wJ4gmoKkz75+zz7CQ+TZjI+Um30wZttnOFMTbjtDXMjwRiEac0I9pmqojUov5jKJal9T nbTFvXZv/AjAu3/5rjcG2CqQCGCJYV4rfJEIb2or04DbzmsYMOEdBok0n2jcqQjQ9Ks6 IjKIByGnhkfycf5NyJlV22owqU/sIeNr7uOPufb9BMdEYwPvXnUtH8HBO0VUyGA/5HbK Y63qHjJY7d7z3uiC49NHV+YFVQM9IoJjKGgW7M2Uku421DVHSNj7OnSvaIZkcLe73/T0 JkVQ== X-Gm-Message-State: AOJu0Yz9aHzw2hLJpZkRiDmN1uFVoESfoDmheHGEaf4r9l5pG1wn0UHm WIwU80Kj8Uy/yyj9BDneljHin7AdYppXH9hsTsoISQ5D5OIjrT94DpYDag== X-Gm-Gg: ASbGncvE/jj1XD8YwGtXqT3GMMXaEZAlIZj0JXZooikWaLMAvvH3o7pHZ7TNskU4/WH Z+azM5Zzl0UAE5keZHbfdwh0PqX3mmfKdFzgexmHp7l4cSoXF14vpYHsjNi+fR74yrqUt0yJSNL CmolSWLmIHIL9LWcrBR0prLdm7xP1MnYLMTQrrI2K2COhJo0ALc1yBKu6pEnVD9a6gMU8+Uf9Br uMToF+702X8Mpbr0QOBAGhyUHGqlrOyoUqwP6B3vgTGoawa2g96rre3CG2VStNrXddKHl6kSZrg tY7bEewtDNKO+f+qzfa+u/AOlqF5w/mBSIqHUkovpqGFAmFz/nB6yHPbteo= X-Google-Smtp-Source: AGHT+IH35on6M8Eb3A9vdOlK7HMDaI0fT9f6pSQGkjgkhpS9eYgJGIjKZ0TWDnNYs8mOOb71zLKABQ== X-Received: by 2002:a17:906:6a10:b0:ac2:6bb5:413c with SMTP id a640c23a62f3a-ac738a9700amr101345166b.31.1743203428563; Fri, 28 Mar 2025 16:10:28 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:27 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 6/7] io_uring/net: set sg_from_iter in advance Date: Fri, 28 Mar 2025 23:10:59 +0000 Message-ID: <5fe2972701df3bacdb3d760bce195fa640bee201.1743202294.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation to the next patch, set ->sg_from_iter callback at request prep time. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 9162dc6ac5e9..f3eaa35d9de3 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -97,6 +97,11 @@ struct io_recvzc { struct io_zcrx_ifq *ifq; }; +static int io_sg_from_iter_iovec(struct sk_buff *skb, + struct iov_iter *from, size_t length); +static int io_sg_from_iter(struct sk_buff *skb, + struct iov_iter *from, size_t length); + int io_shutdown_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_shutdown *shutdown = io_kiocb_to_cmd(req, struct io_shutdown); @@ -1267,6 +1272,7 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_sr_msg *zc = io_kiocb_to_cmd(req, struct io_sr_msg); struct io_ring_ctx *ctx = req->ctx; + struct io_async_msghdr *iomsg; struct io_kiocb *notif; int ret; @@ -1309,8 +1315,15 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (io_is_compat(req->ctx)) zc->msg_flags |= MSG_CMSG_COMPAT; - if (unlikely(!io_msg_alloc_async(req))) + iomsg = io_msg_alloc_async(req); + if (unlikely(!iomsg)) return -ENOMEM; + + if (zc->flags & IORING_RECVSEND_FIXED_BUF) + iomsg->msg.sg_from_iter = io_sg_from_iter; + else + iomsg->msg.sg_from_iter = io_sg_from_iter_iovec; + if (req->opcode == IORING_OP_SEND_ZC) { req->flags |= REQ_F_IMPORT_BUFFER; return io_send_setup(req, sqe); @@ -1321,11 +1334,8 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (unlikely(ret)) return ret; - if (!(zc->flags & IORING_RECVSEND_FIXED_BUF)) { - struct io_async_msghdr *iomsg = req->async_data; - + if (!(zc->flags & IORING_RECVSEND_FIXED_BUF)) return io_notif_account_mem(zc->notif, iomsg->msg.msg_iter.count); - } return 0; } @@ -1392,7 +1402,6 @@ static int io_send_zc_import(struct io_kiocb *req, unsigned int issue_flags) ITER_SOURCE, issue_flags); if (unlikely(ret)) return ret; - kmsg->msg.sg_from_iter = io_sg_from_iter; } else { ret = import_ubuf(ITER_SOURCE, sr->buf, sr->len, &kmsg->msg.msg_iter); if (unlikely(ret)) @@ -1400,7 +1409,6 @@ static int io_send_zc_import(struct io_kiocb *req, unsigned int issue_flags) ret = io_notif_account_mem(sr->notif, sr->len); if (unlikely(ret)) return ret; - kmsg->msg.sg_from_iter = io_sg_from_iter_iovec; } return ret; @@ -1483,8 +1491,6 @@ int io_sendmsg_zc(struct io_kiocb *req, unsigned int issue_flags) unsigned flags; int ret, min_ret = 0; - kmsg->msg.sg_from_iter = io_sg_from_iter_iovec; - if (req->flags & REQ_F_IMPORT_BUFFER) { unsigned uvec_segs = kmsg->msg.msg_iter.nr_segs; int ret; @@ -1493,7 +1499,6 @@ int io_sendmsg_zc(struct io_kiocb *req, unsigned int issue_flags) &kmsg->vec, uvec_segs, issue_flags); if (unlikely(ret)) return ret; - kmsg->msg.sg_from_iter = io_sg_from_iter; req->flags &= ~REQ_F_IMPORT_BUFFER; } From patchwork Fri Mar 28 23:11:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14032492 Received: from mail-ej1-f52.google.com (mail-ej1-f52.google.com [209.85.218.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54D081F4C9E for ; Fri, 28 Mar 2025 23:10:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203434; cv=none; b=WlHf22q86Kuj0kEs8PzqTBYh6mbLbDXjxQZvPX3RKjchG4p+L2vskOA/d6OAUoKPhZsNYIE6QbSKpyOUzOaZXRM+8AFG03CEkxhGqSQl2o9WEbpkdadddT7oaC4ySSI7/3r3gaa9XYbkSGrPYyPvy/0u6RTZKyrnSCfSfQ7irLo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743203434; c=relaxed/simple; bh=ekDm7thqp/NdUIke9Zh/E7lpnAedleXFaIYXqLJfp0Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f5e0ihpkDzPmBgMg4nqV2d/MiR4XEgc6u5Np8pKRQzQXYnsuJYPYs3TOV0Lpaz5ygrGPy4ypWrZUVXSMPNuX+x3+PACHbkn0jMPQygeJ9hH7VNJi8mN9psyh4YQfst6IYpIYqz+IYjpH9Owk9hm4IPXrrw7axRe9I0prpzTOSR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=c67AX/76; arc=none smtp.client-ip=209.85.218.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="c67AX/76" Received: by mail-ej1-f52.google.com with SMTP id a640c23a62f3a-ac2902f7c2aso456663766b.1 for ; Fri, 28 Mar 2025 16:10:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743203430; x=1743808230; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6zjwoAz4Y23GOKZIoKGgRu48Sf407dlcgecALR70baY=; b=c67AX/76FjcjwYA6WtGMgDplPwlZF1+lidIIoSs93jmX41ntdR81vJqj+1zirK7kvM 2XA4AThBSXGjdkLBM+yJ6gMVM8igh+vTdC15JcbWqpgL/bDvRhlyEibatrq+byUFmULf IHIiyQ68TYNkKqrONzPjbOrPaUbOVFKcFhrDLaTYlkihCiy6QzCgZRm6w7dGtr+lJ4zL gwkxJXF0FUPrnBx6Qq7BzYT1fBjTXEcLoCsQx5zEqJiWHRl3jeWPyzuhDLDmRkJ4stTO cwevzKE6r4A6tpiPpJOvp1pI2T16hxFUSZEIuqRyyLuPPmjRz5vHKlp7tyjumcPTAsKH FClA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743203430; x=1743808230; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6zjwoAz4Y23GOKZIoKGgRu48Sf407dlcgecALR70baY=; b=ewhNtF08wmvGFdhsMlxaVJY09Dgnz6rAGOIVZ9pr4aDUpjysYx7evzbmcf9tGDVf00 UPr/nXzXOuZH3OfmOLTNcpohBDPbOrBqasK21EEkgHsBFR8zBKjkwHKkfPXkNnTzZ+sf 5hMZMWrxs2rR+UjMEGLQ/qXySCAbRGlLoSbrhdtLizwE47r2gX26Wo5KbkVX/zj5vxdS fK3Eii35PFK1S+2R3ZjsKaj7nbaZeQAoWCm2qxRcnikDSubHyBdXW/ooW/acHAziwR/T D0SbS9QqEyUuxbywFTi4WiR5Gw8tFEyUryNYmhQ4CEvH5Vo/s/cTW5+0HYpTd7Mez1bi XniQ== X-Gm-Message-State: AOJu0YzSUEmt2JnBDan2HPzBF54iF8EfrblIVAjRUGs62FJpTxh3yoBE UunGiYPwOVQB7aoYgmAnoAkLr53meFV3jjj7BBuslKxb+KuUVlhAc/uE/g== X-Gm-Gg: ASbGnctO2UoFJ4RiDf0UDvFVEz4+AhjKijz0Oyp+K5hVEZVpPAkp14OTViOsDm0MDFn +bXXjzSa+Me2cMt0M+MjNivbzdApk6mc8mZGAAUYKOFMxPbCwauhSTi136gBwMC7ciPrW5xY88h 7ShJfsxpfMY/QWAGs4lpUnKgZWWXD+NG1d7zMVMX6EYIsw2Zi7oANLfm+Nt4yKAdVDet7s5YaW5 UDgtLvwz9y1YRsUo/ABD0K1GKNDtDKV/e3xXk/HGIfBS+FizivGm5YQt/Q0tpnM66eVz3HsnxMW H9vYL2zn89gBiFJdRZVPg9KHlMHjXX4gJSFw9lBbsb/yz0+vYDMt2RXZiaA= X-Google-Smtp-Source: AGHT+IHDevLnVAQqp3mGTfpMeQJ7mqGvXsMZKj1iAB0+kMkcFvTxu5+WgSOibRT91rHF58kPCCBKag== X-Received: by 2002:a17:907:7e82:b0:ac3:cc9b:5b6d with SMTP id a640c23a62f3a-ac7389edcf6mr102597166b.16.1743203429994; Fri, 28 Mar 2025 16:10:29 -0700 (PDT) Received: from 127.0.0.1localhost ([148.252.129.232]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac71961f04dsm228915966b.91.2025.03.28.16.10.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Mar 2025 16:10:29 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 7/7] io_uring/net: import zc ubuf earlier Date: Fri, 28 Mar 2025 23:11:00 +0000 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 io_send_setup() already sets up the iterator for IORING_OP_SEND_ZC, we don't need repeating that at issue time. Move it all together with mem accounting at prep time, which is more consistent with how the non-zc version does that. Signed-off-by: Pavel Begunkov --- io_uring/net.c | 44 ++++++++++++++++---------------------------- 1 file changed, 16 insertions(+), 28 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index f3eaa35d9de3..f8dfa6166e3c 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1319,23 +1319,23 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (unlikely(!iomsg)) return -ENOMEM; - if (zc->flags & IORING_RECVSEND_FIXED_BUF) - iomsg->msg.sg_from_iter = io_sg_from_iter; - else - iomsg->msg.sg_from_iter = io_sg_from_iter_iovec; - if (req->opcode == IORING_OP_SEND_ZC) { - req->flags |= REQ_F_IMPORT_BUFFER; - return io_send_setup(req, sqe); + if (zc->flags & IORING_RECVSEND_FIXED_BUF) + req->flags |= REQ_F_IMPORT_BUFFER; + ret = io_send_setup(req, sqe); + } else { + if (unlikely(sqe->addr2 || sqe->file_index)) + return -EINVAL; + ret = io_sendmsg_setup(req, sqe); } - if (unlikely(sqe->addr2 || sqe->file_index)) - return -EINVAL; - ret = io_sendmsg_setup(req, sqe); if (unlikely(ret)) return ret; - if (!(zc->flags & IORING_RECVSEND_FIXED_BUF)) + if (!(zc->flags & IORING_RECVSEND_FIXED_BUF)) { + iomsg->msg.sg_from_iter = io_sg_from_iter_iovec; return io_notif_account_mem(zc->notif, iomsg->msg.msg_iter.count); + } + iomsg->msg.sg_from_iter = io_sg_from_iter; return 0; } @@ -1393,25 +1393,13 @@ static int io_send_zc_import(struct io_kiocb *req, unsigned int issue_flags) { struct io_sr_msg *sr = io_kiocb_to_cmd(req, struct io_sr_msg); struct io_async_msghdr *kmsg = req->async_data; - int ret; - if (sr->flags & IORING_RECVSEND_FIXED_BUF) { - sr->notif->buf_index = req->buf_index; - ret = io_import_reg_buf(sr->notif, &kmsg->msg.msg_iter, - (u64)(uintptr_t)sr->buf, sr->len, - ITER_SOURCE, issue_flags); - if (unlikely(ret)) - return ret; - } else { - ret = import_ubuf(ITER_SOURCE, sr->buf, sr->len, &kmsg->msg.msg_iter); - if (unlikely(ret)) - return ret; - ret = io_notif_account_mem(sr->notif, sr->len); - if (unlikely(ret)) - return ret; - } + WARN_ON_ONCE(!(sr->flags & IORING_RECVSEND_FIXED_BUF)); - return ret; + sr->notif->buf_index = req->buf_index; + return io_import_reg_buf(sr->notif, &kmsg->msg.msg_iter, + (u64)(uintptr_t)sr->buf, sr->len, + ITER_SOURCE, issue_flags); } int io_send_zc(struct io_kiocb *req, unsigned int issue_flags)