From patchwork Thu Apr 4 22:17:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Crumrine X-Patchwork-Id: 13618375 Received: from mail-qk1-f172.google.com (mail-qk1-f172.google.com [209.85.222.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6674179C4; Fri, 5 Apr 2024 02:17:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283464; cv=none; b=sEj4o0T9jQ3Lu8UadfKHn2zU8jDFl67NVIgGRLfX+EP0KNylcXA/jAkcNPipyhD/R1rFEHKizKzwU+Da+52ulYubl9f0UKd63a+hx49zPilRNRLi6sQhi8A5BaLTS45EUNi6t7H2sg00BPek4sZFidcUqa7kie5Td+umAIvjsVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283464; c=relaxed/simple; bh=95Xk9x3INlTNfIL8CNkINyPj5/oz86DuWfwmqEH2ZBg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tMLYzIMRkhv0WW7nqyCAHlC4obLCZKcNc1p7q6uzdCVhotadM62HIeZ87uzQzcTDLLd0Eg2NNGhAfFtTfPnLRGOPh74025NmFpblPphb5NL0GSy/RHXp1OBdkH0WH2jPudqkO1j+TXVOBJP/DP8J0DIENTWCG6cmVWeQbVEcRYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=HhiJGwBj; arc=none smtp.client-ip=209.85.222.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="HhiJGwBj" Received: by mail-qk1-f172.google.com with SMTP id af79cd13be357-78d496add91so13120385a.2; Thu, 04 Apr 2024 19:17:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712283461; x=1712888261; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yQIiTD5tCLcRtpuvftDkyv/yaTGrweTC1c2Ho0HIRxc=; b=HhiJGwBjLKi/CoLWHc7fpM/Phc3LGgqtWgJ8rclW7+Sy+x1D7A195Ib5ZaPNHMDLwx brWzsET8VjQt6lzosrpAONOJzQ0d6FaVznWjsZ2iomgnDb6z+8r7noN1L84y6xNnKgXG KmsCJ0c0xa+eajzGofIHdOm+6cbr8ODIhJSpUUdmfW6Jel1Rs8/uB7TZiruRThVgCdRP tQMMuPjljCHTsX23kEYyITzvj9eMEL58HAEjY03avN2lNp+2aeIJGZoWNpKxYipSB0FM 9KdnNmjmVEcbSRB43MLhCrLqDy9huaaQAAf+vsOZmntMSC9vIzVRNNT8r2wv23aVSZWS HK4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712283461; x=1712888261; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=yQIiTD5tCLcRtpuvftDkyv/yaTGrweTC1c2Ho0HIRxc=; b=G8Uz/k1XPosqnmVLsHbfLN3BkqrahAAe8zG3tX+wJ2lc20A+ItL/D4SBqhFq810Gyb Uw9IFuvx5Ut6xYL5AK0CCvUZg3MPVEsW2mLHIdWiBLwQ4I4UF4UAO9DWFNVG80koY5kA WjEiDoeg1X9QbO/pWCb6a51LadPNvPrDGKqhi3C6oz/kqMc827QC+qu0sxHT+eZKKLgt d+r6iR5KMFJiJE/atbSFpXoKQ42otkMuVh+yWCAjQVRKsJxR1EOTH+ildiUDtKpAt+lw 0nKNr1nDe4h4kb6A7aOj+GYFWcXuHNgOKxbZgqFxpNmx8khA4mdEmwv6jrhu5rtZbEFf nS+A== X-Forwarded-Encrypted: i=1; AJvYcCVUQOUHFhLd44KYAHhjJTu/PCpBMJRmQGn8D0zuUO9XteHkNO2uaCuwr9Id3gVmupMiDzgBxRU+VQ/pi+SMS9JEzqNz+o0awqESsUyGxsSWg/RVFh34P4FZLHGTujlfVDwVfp0IB7v+ZNCmzMIn+UaKnnrLgEydaOxudiZ/fyUNVFV3tvvo X-Gm-Message-State: AOJu0YykrKCqCsaZMImb4G1shzEnKP41Rm1hmNfZ5QkfwLHsI8aWelzb sWf3SrCRX3Q0GuT3szV6c07vEqK/XOgr0p1vNgQRyZFVWOmqO84L X-Google-Smtp-Source: AGHT+IF3ow1B9J0lbr1hpY+cfkZGU0Oc9xKWN/nUeeSP5PSzEOm/O0vWfinWMNAf1a82ldP7ao1kFw== X-Received: by 2002:a05:620a:4f3:b0:78b:e190:1f2a with SMTP id b19-20020a05620a04f300b0078be1901f2amr181875qkh.18.1712283461231; Thu, 04 Apr 2024 19:17:41 -0700 (PDT) Received: from localhost ([2601:8c:502:14f0:acdd:1182:de4a:7f88]) by smtp.gmail.com with ESMTPSA id bk32-20020a05620a1a2000b00789ee7c9d2fsm260333qkb.109.2024.04.04.19.17.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Apr 2024 19:17:40 -0700 (PDT) Date: Thu, 4 Apr 2024 18:17:41 -0400 From: Oliver Crumrine To: axboe@kernel.dk, asml.silence@gmail.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, leitao@debian.org Cc: io-uring@vger.kernel.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/3] io_uring: Add REQ_F_CQE_SKIP support for io_uring zerocopy Message-ID: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: In his patch to enable zerocopy networking for io_uring, Pavel Begunkov specifically disabled REQ_F_CQE_SKIP, as (at least from my understanding) the userspace program wouldn't receive the IORING_CQE_F_MORE flag in the result value. To fix this, instead of keeping track of how many CQEs have been received, and subtracting notifs from that, programs can keep track of how many SQEs they have issued, and if a CQE is returned with an error, they can simply subtract from how many notifs they expect to receive. Signed-off-by: Oliver Crumrine --- io_uring/net.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/io_uring/net.c b/io_uring/net.c index 1e7665ff6ef7..822f49809b68 100644 --- a/io_uring/net.c +++ b/io_uring/net.c @@ -1044,9 +1044,6 @@ int io_send_zc_prep(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (unlikely(READ_ONCE(sqe->__pad2[0]) || READ_ONCE(sqe->addr3))) return -EINVAL; - /* we don't support IOSQE_CQE_SKIP_SUCCESS just yet */ - if (req->flags & REQ_F_CQE_SKIP) - return -EINVAL; notif = zc->notif = io_alloc_notif(ctx); if (!notif) @@ -1342,7 +1339,8 @@ void io_sendrecv_fail(struct io_kiocb *req) req->cqe.res = sr->done_io; if ((req->flags & REQ_F_NEED_CLEANUP) && - (req->opcode == IORING_OP_SEND_ZC || req->opcode == IORING_OP_SENDMSG_ZC)) + (req->opcode == IORING_OP_SEND_ZC || req->opcode == IORING_OP_SENDMSG_ZC) && + !(req->flags & REQ_F_CQE_SKIP)) req->cqe.flags |= IORING_CQE_F_MORE; } From patchwork Thu Apr 4 22:19:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Crumrine X-Patchwork-Id: 13618376 Received: from mail-qk1-f174.google.com (mail-qk1-f174.google.com [209.85.222.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C484AB67E; Fri, 5 Apr 2024 02:19:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.222.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283543; cv=none; b=BaD66s75Nob7oIYa8lKsurw/XJJ8NOpnpm3adGU6b50ElkaEHcLFHsZ3njidVI8WZMuc8yKLR8MigplL5CvldcQnWSgjjNVJJnzCzBUpRROeGtWwzFxyP/MrqzOOVwYEhLVCeo0gkeaLhHkEglgmtkFNFJTb7JqpJ+KWBPUKJV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283543; c=relaxed/simple; bh=MdKz9HO/MEnCu98IZ9g57PWWEkBZ4g/hdCyy2WmbaM4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TyidWuzMWMcqzppu3srfQms317Hu8RPQhc5rvrRu0aZ1wV74SL3b3ckooKXiXFMizmvPW6WVeLqya2+tyN9DRpsXrknrpabtEIIeTd76StEkIHGjuUCFa+Rsv2NQjMFKlyToiNlGTzuqKM2rKDvjwik5qbVWZ4710+tf0W8i1+U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=MOMbDrJo; arc=none smtp.client-ip=209.85.222.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="MOMbDrJo" Received: by mail-qk1-f174.google.com with SMTP id af79cd13be357-78a5580324eso94374585a.3; Thu, 04 Apr 2024 19:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712283540; x=1712888340; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QrBxNOgOhwOoTM6st8k20/ILsbMyEoFne61a12r+YLw=; b=MOMbDrJozYvIsrCpBYWt/IjKLvkLr1WSz1EfGGMzRjRi0Px74J7mnaKCPxZSSoVSsX rexR7EdRZbsi0Wk3H9WzKlE2Ai5rXiAcOlqpFUzM2Eip2r3zfchSc/wzHF9atek6Mkj8 EhbcdjbOk2U06/QDXN5PRqMh7FVyJq9FgbFf/ub5ETNaOBZQmnC1765Ihnn1+JRz+CTO 0q6bOfok11nSUaRpbosF6fHigBjgds9iYPGSLO0IlCxxv10NElMo7lFpKRWQX/oPZ8Cy F57T5jH8n5TpWeEpO/yNtUqssrMuRohGfHqre7/a62/+ixOzDURaJtwm5Wef61W347mJ RMJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712283540; x=1712888340; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=QrBxNOgOhwOoTM6st8k20/ILsbMyEoFne61a12r+YLw=; b=IacSuZGO8rLIwKQyGJqonntGi+r0FRfWckpNsDasfKTGBTIN2uoQv32YhVyHXFiKnY aYr/20OAq8HAWi5UfCq/IijS4rkvomy0VrJVAEw1qn+ARNrf7FXbuTknalfIXuLA4BeQ +RAbdtIT4YY0iZ5GhCKZSBTOfCP6cj23Im5QglRmz8TeU7Fd13nfyeIjhM9zGODzy3r9 KIe7lTYl6nQ5yKGCDeHpZA1ALuB7ucpnfV2PbHsEVTFejnpDgALfFeMZ/s/pLCslwVzW 38v52+wpkSQF5NP9izLCagSzllSrrdGCPbPq5FuzwqfPWpZV2rp3OOi4iKLh8WV+xGjm 283Q== X-Forwarded-Encrypted: i=1; AJvYcCX0mwazYVSDPV4fA5TYRmFRvaSqtRGkIPRzpZh2M5zdrRgGmxNy15TGsaT77q0PUjoqBi3IEh2los9r/Vx1loyDXbekE4MOFPKg/Cmjs3QgvoD+0Qjqd90carpWs04SgbndIAtmizvboYQNlak1iYMP1I4sHv03/tX/cxtusz1KgUUNGh8l X-Gm-Message-State: AOJu0YzCxBCrxrdtWzSjh3S8/QFDqeG+s03r/NC8Ni5iEOQFQlGTAjjL BqtzpcfhiFKkS9v4E939ltQ9Y78E76Q2RHEWvRq6P5qgacy0MQ1M X-Google-Smtp-Source: AGHT+IHkPd5B8LovJ0cyWAC2USRF0wUN7jGuq6K4BDdMroZ053FUDaivvQrXBMPx03zxifKFr/r7Og== X-Received: by 2002:a05:620a:46a2:b0:78d:4337:a291 with SMTP id bq34-20020a05620a46a200b0078d4337a291mr240658qkb.12.1712283540710; Thu, 04 Apr 2024 19:19:00 -0700 (PDT) Received: from localhost ([2601:8c:502:14f0:acdd:1182:de4a:7f88]) by smtp.gmail.com with ESMTPSA id bk32-20020a05620a1a2000b00789ee7c9d2fsm260939qkb.109.2024.04.04.19.19.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Apr 2024 19:19:00 -0700 (PDT) Date: Thu, 4 Apr 2024 18:19:01 -0400 From: Oliver Crumrine To: axboe@kernel.dk, asml.silence@gmail.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, leitao@debian.org Cc: io-uring@vger.kernel.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/3] io_uring: Add io_uring_peek_cqe to mini_liburing Message-ID: <44916a6a3f94bfcff78da2c602c916c856d7cfcf.1712268605.git.ozlinuxc@gmail.com> References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: mini_liburing.h is used by the io_uring tests as a stand-in for liburing. The next patch in this series needs the io_uring_peek_cqe function, and I speculate that future tests will too, so add that here. Please scrutinize this code, I am somewhat new to io_uring, and whilst I have tested this and know it to work, I very well could've made a mistake somewhere without noticing. Signed-off-by: Oliver Crumrine --- tools/include/io_uring/mini_liburing.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tools/include/io_uring/mini_liburing.h b/tools/include/io_uring/mini_liburing.h index 9ccb16074eb5..94dcc7f9aa33 100644 --- a/tools/include/io_uring/mini_liburing.h +++ b/tools/include/io_uring/mini_liburing.h @@ -182,6 +182,24 @@ static inline int io_uring_wait_cqe(struct io_uring *ring, return 0; } +static inline int io_uring_peek_cqe(struct io_uring *ring, + struct io_uring_cqe **cqe_ptr) +{ + struct io_uring_cq *cq = &ring->cq; + const unsigned int mask = *cq->kring_mask; + unsigned int head = *cq->khead; + unsigned int tail = *cq->ktail; + + *cqe_ptr = NULL; + + read_barrier(); + if (head-tail) { + *cqe_ptr = &cq->cqes[head & mask]; + return 0; + } + return -EAGAIN; +} + static inline int io_uring_submit(struct io_uring *ring) { struct io_uring_sq *sq = &ring->sq; From patchwork Thu Apr 4 22:19:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Oliver Crumrine X-Patchwork-Id: 13618377 X-Patchwork-Delegate: kuba@kernel.org Received: from mail-qv1-f48.google.com (mail-qv1-f48.google.com [209.85.219.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A0A47101F2; Fri, 5 Apr 2024 02:19:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283601; cv=none; b=RbZ/IlJ+uD1DED0J/06gpkFj9ZWLsHiyou9WY5f5A+t48ff/apxL8mX6KlZaCzWD7UV6oL0yDyvbu30/s01hqUBysDT9rKlKZ2RktsjSMK7zsdlamj+wHKIV53lEArmbz2qp9l3SH0FbXtmT640svmQMYynpkAoey/DH7ZIBO2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712283601; c=relaxed/simple; bh=2DYTKFCxaDTpCQ12eT+9jVBI7+leHm0ooeFmGLJgzTE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pGU6XwOm+8KNf1p3lKBwriT78e2TYBlOmeZZ5eklH+3J827uO9Wgo4j8vk5flnC+8aDBtddkjNIbS8ctWI7ApfVtHBv2BWLWNuYtek26Gy0GdGOzbD+d1jgcPDe4mZuH3s5gO5glv4IKNm2XLRKYDcJUBNGVKULdcfwpG8u5Qkc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=T1Zds3kD; arc=none smtp.client-ip=209.85.219.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T1Zds3kD" Received: by mail-qv1-f48.google.com with SMTP id 6a1803df08f44-6963c0c507eso18855456d6.1; Thu, 04 Apr 2024 19:19:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1712283598; x=1712888398; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cizTluUusU/8LespqdwSJr+fCAo0OEMHrmoen45RruY=; b=T1Zds3kDow7K5FbKBAv0aPSgTMl8PXKGPdnlkSH0jxqcM8OgiQyTnzAk0BwjjTmirO gu2SIY6+TlDsgMRmqRWnxjEPHFfsJj6nZ0GfJNIC2+US9P0vvJ0s5mskVvUKa0yDPWkp +LtDgo4nZyTr69LoPUA3Mt2j2mBE0meZXsMzwTewMGDSuZKiLJzglF814+Mf0uZ9UnMC Ky8SGaB5BUe8UobcYy2IMQl1OHbmaofZasbjQRPzbltA5dIQQ5BDvD00AEY9qvPddqcQ oxPbZSy/tsDLXvml/pvDkBVHV7OUVRv11RQ50DbO18AQ2es+qNr/dJoWsgD3OYc5OKF0 ziWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1712283598; x=1712888398; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=cizTluUusU/8LespqdwSJr+fCAo0OEMHrmoen45RruY=; b=R6c/7XauUD78X/Iq0U36RXwpuEQaDPbpdW1mGylfFXemT2yR/VHPP2HGSfikFExLcg oiV1Dx3FS/x7QxN/FMyhvm7GKO3kJirZGTrY1Sg0tjQ5sChN8Q03fZSFdgqZ0ELdrYzD S8fYxeq1y/BBJ74es+BgF1WmSTQCMWafIlR8ZnMle5aW7LBV5RkKXkuAKPoF9GjZfFCb ku7CLEWg0msP9V4X4Rg1+MPvtSK5nE7YqQixNDO1hYo1fiXnzbo+nav5ZSaFH6KzDbOM +6q1fUkklIb6I5sUmFMJgUZ/FO9owXvWxdPjJzTt7Wf95kr08kH5mp/+InEzgzflXoVw Elpg== X-Forwarded-Encrypted: i=1; AJvYcCUdE4GU2EXjEnTlO+BV8ecezAvEMm7Ivl/jpdGQ3SUG4VudcBKStvyXWYCziCqwb2lhj/WVp7xRkKUrjJHJFBvMDKGSlOGXIRLU9E1+BGL5FLMkFgz72ZlqxujSaF02mx4R9rcI6yMsVq8jJd5cx/Gb3gdMdCBSFCgQ3ks381rdg2y6hZ5B X-Gm-Message-State: AOJu0Yx3ZW77wN6ObPx4pXTRSNw8mhNUJlKH8BPPtx8FX5/6Pin+6cL1 VyXYlwu8EWM5AF6HOcgep6buhuTUR3o8CkqAaFbuUd7I2QaLnvyL X-Google-Smtp-Source: AGHT+IGHqrNkxNFn83ZsabNt/OGQdpSNkJMryp83nT7fa8Roho4klsxduoA+FlIxnJaOonFD9Lskow== X-Received: by 2002:a0c:fa8e:0:b0:696:8e65:b6a4 with SMTP id o14-20020a0cfa8e000000b006968e65b6a4mr2037023qvn.31.1712283598544; Thu, 04 Apr 2024 19:19:58 -0700 (PDT) Received: from localhost ([2601:8c:502:14f0:acdd:1182:de4a:7f88]) by smtp.gmail.com with ESMTPSA id ek1-20020ad45981000000b00699032e555bsm250035qvb.127.2024.04.04.19.19.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Apr 2024 19:19:58 -0700 (PDT) Date: Thu, 4 Apr 2024 18:19:59 -0400 From: Oliver Crumrine To: axboe@kernel.dk, asml.silence@gmail.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, shuah@kernel.org, leitao@debian.org Cc: io-uring@vger.kernel.org, netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/3] io_uring: Support IOSQE_CQE_SKIP_SUCCESS in io_uring zerocopy test Message-ID: References: Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Patchwork-Delegate: kuba@kernel.org Add support for the IOSQE_CQE_SKIP_SUCCESS flag in the io_uring_zerocopy_tx test, using the "-a" option. Instead of incrementing when IORING_CQE_F_MORE is set, remember how many SQEs are sent and simply wait on notifs instead of regular completions. For non-zc stuff, there won't be notifs or completions, so don't wait on either of those, but check the completion queue for errors at the end to make sure none have popped up. The changes to the shell script run the tests both with and without the "-a" option. Signed-off-by: Oliver Crumrine Acked-by: Muhammad Usama Anjum --- .../selftests/net/io_uring_zerocopy_tx.c | 38 +++++++++++++++++-- .../selftests/net/io_uring_zerocopy_tx.sh | 7 +++- 2 files changed, 39 insertions(+), 6 deletions(-) diff --git a/tools/testing/selftests/net/io_uring_zerocopy_tx.c b/tools/testing/selftests/net/io_uring_zerocopy_tx.c index 76e604e4810e..11a43594935f 100644 --- a/tools/testing/selftests/net/io_uring_zerocopy_tx.c +++ b/tools/testing/selftests/net/io_uring_zerocopy_tx.c @@ -50,8 +50,10 @@ enum { }; static bool cfg_cork = false; +static bool cfg_nocqe = false; static int cfg_mode = MODE_ZC_FIXED; static int cfg_nr_reqs = 8; +static int cfg_nr_completions = 8; static int cfg_family = PF_UNSPEC; static int cfg_payload_len; static int cfg_port = 8000; @@ -134,11 +136,21 @@ static void do_tx(int domain, int type, int protocol) if (mode == MODE_NONZC) { io_uring_prep_send(sqe, fd, payload, cfg_payload_len, msg_flags); + if (cfg_nocqe) { + sqe->flags |= IOSQE_CQE_SKIP_SUCCESS; + cfg_nr_completions--; + } sqe->user_data = NONZC_TAG; } else { io_uring_prep_sendzc(sqe, fd, payload, cfg_payload_len, msg_flags, zc_flags); + if (cfg_nocqe) { + sqe->flags |= IOSQE_CQE_SKIP_SUCCESS; + packets++; + compl_cqes++; + bytes += cfg_payload_len; + } if (mode == MODE_ZC_FIXED) { sqe->ioprio |= IORING_RECVSEND_FIXED_BUF; sqe->buf_index = buf_idx; @@ -153,7 +165,7 @@ static void do_tx(int domain, int type, int protocol) if (cfg_cork) do_setsockopt(fd, IPPROTO_UDP, UDP_CORK, 0); - for (i = 0; i < cfg_nr_reqs; i++) { + for (i = 0; i < cfg_nr_completions; i++) { ret = io_uring_wait_cqe(&ring, &cqe); if (ret) error(1, ret, "wait cqe"); @@ -168,7 +180,9 @@ static void do_tx(int domain, int type, int protocol) if (compl_cqes <= 0) error(1, -EINVAL, "notification mismatch"); compl_cqes--; - i--; + if (!cfg_nocqe) + i--; io_uring_cqe_seen(&ring); continue; } @@ -200,6 +214,17 @@ static void do_tx(int domain, int type, int protocol) compl_cqes--; } + /* The above code does not account for a send error when + * IOSQE_CQE_SKIP_SUCCESS is set. This is operating under the + * assumption that an error CQE will get put on the ring before + * the above code completes: + */ + while (!io_uring_peek_cqe(&ring, &cqe)) { + if (cqe->res == -EAGAIN) + continue; + error(1, -EINVAL, "send failed"); + } + fprintf(stderr, "tx=%lu (MB=%lu), tx/s=%lu (MB/s=%lu)\n", packets, bytes >> 20, packets / (cfg_runtime_ms / 1000), @@ -221,7 +246,7 @@ static void do_test(int domain, int type, int protocol) static void usage(const char *filepath) { error(1, 0, "Usage: %s (-4|-6) (udp|tcp) -D [-s] " - "[-t