From patchwork Fri Apr 4 16:22:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14038676 Received: from mail-ed1-f44.google.com (mail-ed1-f44.google.com [209.85.208.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D89B20ADD5 for ; Fri, 4 Apr 2025 16:21:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783670; cv=none; b=IxqcbTf+l8xsbV6LI7TW+e1myUMeIPs5wkcdpQx4ZeuMSCebn3pytD06OQNdGeTKD/x0Vj98IAxDbFR2OwfmqEskk2Cy8JsowJUWFSd2d/XvUuUAFfJRMvpyHadhIjw87VpRA3O0PLjrn1LCx3Z432dgXzkRXGl8lKfU9ThnxF4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783670; c=relaxed/simple; bh=/eaRuqUBlgqkRoy26AaBK/Rg6uR+UHhdwO5PjPGQrtk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Eo2iF5rFKFic5lMJXsDd4Klk9ZkN5KbHR7999ytti16w1pHRIP2PXyly97AWoUbGarbnDBeYWimH73cUoB7S+VHWW+dQDmL4fm69GvfegnzN1mIOinQanu139TbZZYyoTlXBwhQfeh6KtxnNni8mE0pxTdmnrR0DEA35ug4vHxA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=V1QkGM/d; arc=none smtp.client-ip=209.85.208.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="V1QkGM/d" Received: by mail-ed1-f44.google.com with SMTP id 4fb4d7f45d1cf-5e61375c108so3080889a12.1 for ; Fri, 04 Apr 2025 09:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743783667; x=1744388467; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6OhjmtmCDME3DRXsETFG1/DtMblO4RfJlQAZuTIgAyo=; b=V1QkGM/dF7KE82Wn2+XgF/2wFBXUppfm4yOfn5PDd3RunrE+KwOptKXAeKIL7Gw9G2 FTi7E9l3726Jdv7wKm3KlHN2pkUbLBmpS4tCLLRi8HwCrKOcC8IIAciKAbG8sp1+z++o PZtweJEuoOaEXrVu2WuhX969qiQNUf3cPWLzKH9kUBivE7o88m9JctI4ZNcYkeLIaYZv H+NX0g5IAfoUU3uAyVfV0fIj4WGFqdGH9iVp4aivV2/6YDkaMuS553I72oap3qsIOci3 786kcBCs5KvYcSvyPRoYZpzjhWDhEEYiMJFlf8tM8U2BVvsKjsTe49aEN51qAaSaXO9Y aDyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743783667; x=1744388467; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6OhjmtmCDME3DRXsETFG1/DtMblO4RfJlQAZuTIgAyo=; b=gREFJJBeV+jVJkSzazl01CifchgfmDyKAHUIMsWRvmyIajcd2LsAwh4sNW7jnaK2oY 1am3ujw8fSUpCj8gbKIhpmyVxFOVi8UhjW1gX6qceWWM/CSOAMK8qMTtClKTyn4LlXUt qhO+GHDUXxTME9rXqpj1tokk/XvxsZPgPLD0Rw13tomzHB5wqt4dvmwN6tDstjLVVCvt RO6G65RlFuMoMVml1t6WEQm2X5g4SDygFJHODSJ1zLIyFdJPhgZCZ1lZjDTYZuwZdS6g kSMhcBqkqP4FnzWIPaxWNKfzaEOOyUHbu1CVSM13u+CxvFjw+hKF2svHTns8wZcneQ9f PwLQ== X-Gm-Message-State: AOJu0YwkEcZ1Zt+GKw0/xxBbTgBQnj3lcfRqGljK7DI0gyOC+gWo9wl/ P/jPSJlsxIH7+GTc2Dky9/GCuSKNZxIqdkLKkS5OhYDMpKVlJPEZ6REy+A== X-Gm-Gg: ASbGncu+oG/P4iW7NgFLycmkTTWYP1e19vS3onskdJstEfO+5GwfdCxj/vaNHya7SJ3 41Zg+rFyEI2JpI4zSboHXfV3puQJ6AbcwCft8iArBh1tproHMn9ZdSJpxaqeWo+GOG+2KWmZn5D 3K+1LqL84xBGRcFAC7szuVzwbb1FFmqA7P1iogqposuJ9wFTGZPXBHnHNntfFTZ8dS9BK4xqpUE JVJOF17rAeGYDyIp6MRVt2aB+tIe3ewesLE8T1jU5Jx5r0Ei6C7sUamvDJe9l4iChlIEmRyxK76 86+Jm/QqSETGrtk1r5yjxFtJVaIl X-Google-Smtp-Source: AGHT+IE1TaHj5fhfRK6gJwLdxmTO9BoVYnDeaS7FmMkOKvxNa5OkYMHch8GWBO3uPArji1GvmoND6A== X-Received: by 2002:a17:907:3f89:b0:ac6:e42b:7556 with SMTP id a640c23a62f3a-ac7d6cbd8acmr261904066b.11.1743783666881; Fri, 04 Apr 2025 09:21:06 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::1:ce28]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac7c0184865sm273316066b.124.2025.04.04.09.21.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Apr 2025 09:21:06 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 1/4] io_uring/rsrc: avoid assigning buf table on failure Date: Fri, 4 Apr 2025 17:22:14 +0100 Message-ID: <668bcc8f0590e9b8b8a51599a5832b48e387e396.1743783348.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The fail path of io_sqe_buffers_register() assigns ->buf_table just to implicitly pass it into io_sqe_buffers_unregister(). Be more explicit by using io_rsrc_data_free() for destruction and passing the table to it directly. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index b36c8825550e..3c6e6e396052 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -914,11 +914,12 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg, data.nodes[i] = node; } - ctx->buf_table = data; if (ret) { - io_clear_table_tags(&ctx->buf_table); - io_sqe_buffers_unregister(ctx); + io_clear_table_tags(&data); + io_rsrc_data_free(ctx, &data); + return ret; } + ctx->buf_table = data; return ret; } From patchwork Fri Apr 4 16:22:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14038677 Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5154920ADE9 for ; Fri, 4 Apr 2025 16:21:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.45 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783673; cv=none; b=gNLslEB4LlN5nEVdgLSET6JUFZDabHE10FDUqZt1X5urYS9+JjnAUpK4H+4hrFgmBHvffYjn3alW9iRmixHne+Xourm/Q43/QTLGaFe1MUZJAxqglTv2gjy2O5ePMGxT8esO+9hKyjRHr13Vxlrzg3SyKDZA8cgoL882/Dhtkdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783673; c=relaxed/simple; bh=X6XkgO/2J6cWEumjWggsd7LobOoRYSrXua06ZrAMVUw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QwQrdoleo/lRlN4QaV/oqEam9ZmuhQupB2a2kd8/ESK78aSwZ+kvGsmiEeb4mlZe4abMyQkOoPIbWZJR0+IoJP3INbCu3gNR9oUIjdpEbxAuLgm+uqj/kFKGliaHj0RLvxxcnt1DvAqfS+crgOXlQdvSPoHuRzjMB/yolbvbskc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=l9Fen5Yi; arc=none smtp.client-ip=209.85.218.45 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="l9Fen5Yi" Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-ac3eb3fdd2eso45277766b.0 for ; Fri, 04 Apr 2025 09:21:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743783668; x=1744388468; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zOfK4mLxM92OSI9jXfUtr+7+kCu9rBgviQpi8ZslCPY=; b=l9Fen5YiCEbqYgx9sLONJoe/5YkpgqgEJ1uYw8P5uwBdP7X1HIUac/t0u60J/zMtae +KI0/4ZTCmL9m7D7pZWUixvPoSBIyW1xEO2LJy7SPRGix60a4cdS1og/VQflCnQ1Wrxh 1Q99R3SMcKVozFxbZrdMZ0QgRFIizt8D+qqSOU90sTXIv+2ZqLajgR0NR8wHNgeWe3jd 3+Nu4rYnNRgwjcm8PvyxQtRhmVYVr1RqPyvpZOo/ujAzZvpArEwDoms0BhwdhSn5eKdv wUxbeiuzs/98EgB97Qpp8vh3fEBujWd2pGnAoA7Z8uWLJhqwSVC2qxe2dekTUeL7mdyq I/0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743783668; x=1744388468; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zOfK4mLxM92OSI9jXfUtr+7+kCu9rBgviQpi8ZslCPY=; b=VRxmnUhuVLpGz3KklLJ+uJG0fQ27sb+5lSidVbbFyJHbnotCPYSeeOov6xHOdzEEj6 b8Y4IkipN3acoI1CwcRnz+cbs/5fubixIj8FfXqh20YD4WUeViaDZ168mBIvaPFT/HlU Z5w1f2UseCuo2vOPpZIfLTzdCaz2SKtH9Vs07WkZDDJnAolQbEX86n+ohcrA/ZXeNvAY CovIfYQNaqwgcndPActNQnp6zXME2RrIA3uv4WY63bOomQpzRQOiY/jhdV0wyxCr954G Y1JJIM+euBkAVhmJQiyEpoixTGkZA4gb0s8/VuEK/znoMuUFzdWrUxe7jL1DMuNKFFa/ Kacw== X-Gm-Message-State: AOJu0YwLxbDvh7f8wrKoshxRxDGCnTPTcEuAut+j1yOCWaSvN35DnJ8c AHvF8a6b2QmBi1s9OLvPrmONe51m3W50uTbR+bKSZDORA7SKw0GqoGOrEA== X-Gm-Gg: ASbGncs50C7rQ73608N3oQwVCm61OKz3DMvJca3son/38x7wLdCH7M1fK3jLvC6TDIH +uMtq65v7UWMwjcgcAP2bk/vwMV88L5mhr9j95Q9XgDbSnjEvKohslsWU8HMJ0y/wqKD4GsayyT 4cQIYjxFP2PW3+Wy2hDHQ43ryCKFQiX4c7D7FoolUKtSYVyULjYAf67yUIO1NlMeUPqIEU+d1Q9 cMBASx/bGVLf8J3dWurMQSy+0OmhJHDeWT8ltApgAPB/w3jdsquddWC6Drclu/dsZazqqbP/152 UgmFVsC7XkRsEhgrHtJdYa5ZvIOG X-Google-Smtp-Source: AGHT+IEaW/3jOskcp+/e5PMX8Xhqyajz2GI07Z1haR29Ir32GD0x8tuL3XdOPJAYY95x215vyETFmg== X-Received: by 2002:a17:907:94c9:b0:ac7:18e3:1b16 with SMTP id a640c23a62f3a-ac7d1877e2fmr418351966b.20.1743783668054; Fri, 04 Apr 2025 09:21:08 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::1:ce28]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac7c0184865sm273316066b.124.2025.04.04.09.21.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Apr 2025 09:21:07 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 2/4] io_uring: deduplicate node tagging Date: Fri, 4 Apr 2025 17:22:15 +0100 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Move tag handling into io_sqe_buffer_register() so that we don't need to duplicate it. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 29 ++++++++++------------------- 1 file changed, 10 insertions(+), 19 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 3c6e6e396052..d5e29536466c 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -27,7 +27,7 @@ struct io_rsrc_update { }; static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, - struct iovec *iov, struct page **last_hpage); + struct iovec *iov, struct page **last_hpage, u64 tag); /* only define max */ #define IORING_MAX_FIXED_FILES (1U << 20) @@ -311,18 +311,11 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, err = io_buffer_validate(iov); if (err) break; - node = io_sqe_buffer_register(ctx, iov, &last_hpage); + node = io_sqe_buffer_register(ctx, iov, &last_hpage, tag); if (IS_ERR(node)) { err = PTR_ERR(node); break; } - if (tag) { - if (!node) { - err = -EINVAL; - break; - } - node->tag = tag; - } i = array_index_nospec(up->offset + done, ctx->buf_table.nr); io_reset_rsrc_node(ctx, &ctx->buf_table, i); ctx->buf_table.nodes[i] = node; @@ -771,7 +764,8 @@ bool io_check_coalesce_buffer(struct page **page_array, int nr_pages, static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, struct iovec *iov, - struct page **last_hpage) + struct page **last_hpage, + u64 tag) { struct io_mapped_ubuf *imu = NULL; struct page **pages = NULL; @@ -782,12 +776,16 @@ static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, struct io_imu_folio_data data; bool coalesced = false; - if (!iov->iov_base) + if (!iov->iov_base) { + if (tag) + return ERR_PTR(-EINVAL); return NULL; + } node = io_rsrc_node_alloc(ctx, IORING_RSRC_BUFFER); if (!node) return ERR_PTR(-ENOMEM); + node->tag = tag; ret = -ENOMEM; pages = io_pin_pages((unsigned long) iov->iov_base, iov->iov_len, @@ -899,18 +897,11 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg, } } - node = io_sqe_buffer_register(ctx, iov, &last_hpage); + node = io_sqe_buffer_register(ctx, iov, &last_hpage, tag); if (IS_ERR(node)) { ret = PTR_ERR(node); break; } - if (tag) { - if (!node) { - ret = -EINVAL; - break; - } - node->tag = tag; - } data.nodes[i] = node; } From patchwork Fri Apr 4 16:22:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14038678 Received: from mail-ed1-f43.google.com (mail-ed1-f43.google.com [209.85.208.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC20120ADD5 for ; Fri, 4 Apr 2025 16:21:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783673; cv=none; b=OPSQAv2YpBLIW8EwR6SI7MtNl9+29/9NnyG3PGSMi1f2g0hrsUuLYi6dkPu5T+ZP6fzm/pLhgGXZefcd2sJrFWGNP5ZIMDJi+2q8nqdFzOocJC53lGCo7e3JU/yEmN98pjxGaqVtb9WMAwFnS4fUzRc0nAe2UVL+1hy78pIMhUw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783673; c=relaxed/simple; bh=5y+2BsX8Un6kP+AA8e0DO1Wa27hBqYXKRko/IGKIUOg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tp9tCiJ6r0SyNjfKw0Cfu2wTJZnIfpfDpJdLODRIyPWoxcdL2e6TftOVx/HUoCPe8EAaznYmQt5b9P6LgloiPoYjU+YmoUsTCuKTV36j+jiec9S3s48/ZvOeSccUVC86sB/RV2sYyR7un3sQFu5oHHQNHTBu5JSoA7M2Q4RsmMI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=bFXHnCNU; arc=none smtp.client-ip=209.85.208.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bFXHnCNU" Received: by mail-ed1-f43.google.com with SMTP id 4fb4d7f45d1cf-5e677f59438so3516214a12.2 for ; Fri, 04 Apr 2025 09:21:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743783670; x=1744388470; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GMM5qhvjL8gczQPyq62vkmefgWGt7CIYLJ+nMX+wd/w=; b=bFXHnCNU562uKSvbcR69eeUV3l7dCgsSIdNnCRNT/5AGBDJedOb7lHRaBghWpIn6Xd 9Ql03rnX7bTThi+iBv0w9Bu1vu2PTg2sGaQ+6DLQBH+ethTR5OEcRtM46ypJ5EiPqIDO HBDSyHLNvojB0CKLF0BkD+V9EAwE8is7kbSM45RSx4xv24df8AfQZdIEtbboP1Wg6DN3 cbONXeGrlqp+I8t6ZjyF7Q+957G5xIXZU7xRKyHeNv4lnH1Q2n8y9w0Hsrr5loGTKN9L 7vVFUnt3vq32ur+uqpXPBo19FPLuhlXqhEmtQsMsn/caqHPeMLXSBgVw18fbEfFNKEZa UD5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743783670; x=1744388470; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GMM5qhvjL8gczQPyq62vkmefgWGt7CIYLJ+nMX+wd/w=; b=Fc++upwv8AR6QUJF+Khe+80Fo0TUNL36k9xDPm7uZHJ0pvIIbwZJSQBg6kRWpMdeuS u4CIf2CwUrf8+MWmAq4VqmwLJBnFQFqrbSCFcuaSjOGxMU4gD0MTC+0NpvdwK1Y+xRYh I9aGgogvl8H5hjTMI1yrawvJcBndhB7BCPqNzkgHsjFtwqQ6lRhxvad6aseXau0f1uVk /3g52gZ9Oy476h9OwhQgEmJCTDonGgKcceAeyADrjuO+6QsAFnwnUDwLiCnVgzNT3W5Y Ut4FiMifVO1wndI15xubBCvlziop79GosjmIRtD0KJhCcbT/7dsX4RdFI1aW9DDjevAq xBBA== X-Gm-Message-State: AOJu0YwWZaxCyditG9Bry+ySOhSIR1wDk7piHj/1iU/llY+ELb/41U5j 8nv9iapE09YiNT2hrvsp7VD0GB0pMQVk0biJozfDpXBHfmPmLcb281K+mw== X-Gm-Gg: ASbGncssvtTedC9i7PIE6OhYLJFk/+jt1wOoUPt4ki4y+mfJgkZYPvsx9htbxX+2cth Po/e6X025djLCouNiXvvjDGLptrZmNwDFikZCoy5JD6MJ76O+SxPvmPw8zh1uABLcp4KtgxyAf0 XhA7T9aOZ4TUtYOYl+MDXk/vj0RaTDbcOCefXliQNNUzoTs6zIKlVGxySUtkYH+gSM+VZL+s8P5 FpykAX1qAiFx7qo3OWSE7l4VJibq0q1k1lvM4h6KlocnBG0IqkZ1ER6+VNSZNMaLZVmP9jvpu33 JQ4DPrGG4fIWjFGgs6MbNCMOhe7Q X-Google-Smtp-Source: AGHT+IG74HDo6IN0gy3Pz7rnvR8lm361TUACaEWj38MMdiWE8oDRbhhA7YilrOJDd/O5x9vSetqgug== X-Received: by 2002:a17:906:6a1e:b0:ac6:f6ea:cc21 with SMTP id a640c23a62f3a-ac7d6e9fe32mr277373766b.55.1743783669526; Fri, 04 Apr 2025 09:21:09 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::1:ce28]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac7c0184865sm273316066b.124.2025.04.04.09.21.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Apr 2025 09:21:08 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 3/4] io_uring: early return for sparse buf table registration Date: Fri, 4 Apr 2025 17:22:16 +0100 Message-ID: <11655e617b90ee11bf95bf59b2cdc433d9448056.1743783348.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 We don't need to do anything after io_rsrc_data_alloc() if the goal is to create an empty buffer table, so just return earlier and clean up the rest of io_sqe_buffers_register(). It should be particularly nice with IORING_RSRC_REGISTER_SPARSE. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index d5e29536466c..958eee7b4a47 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -867,28 +867,28 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg, if (ret) return ret; - if (!arg) - memset(iov, 0, sizeof(*iov)); + if (!arg) { + ctx->buf_table = data; + return 0; + } for (i = 0; i < nr_args; i++) { struct io_rsrc_node *node; u64 tag = 0; - if (arg) { - uvec = (struct iovec __user *) arg; - iov = iovec_from_user(uvec, 1, 1, &fast_iov, ctx->compat); - if (IS_ERR(iov)) { - ret = PTR_ERR(iov); - break; - } - ret = io_buffer_validate(iov); - if (ret) - break; - if (ctx->compat) - arg += sizeof(struct compat_iovec); - else - arg += sizeof(struct iovec); + uvec = (struct iovec __user *) arg; + iov = iovec_from_user(uvec, 1, 1, &fast_iov, ctx->compat); + if (IS_ERR(iov)) { + ret = PTR_ERR(iov); + break; } + ret = io_buffer_validate(iov); + if (ret) + break; + if (ctx->compat) + arg += sizeof(struct compat_iovec); + else + arg += sizeof(struct iovec); if (tags) { if (copy_from_user(&tag, &tags[i], sizeof(tag))) { From patchwork Fri Apr 4 16:22:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 14038679 Received: from mail-ed1-f50.google.com (mail-ed1-f50.google.com [209.85.208.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F395F20ADEF for ; Fri, 4 Apr 2025 16:21:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.50 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783674; cv=none; b=DYxU6E+idDwLoMzTUN9MsZkJFFoF3QC7mlZkQbsFB1MwWndE5v1b/xjFbonXulfXdOFZqv8qdXTHxX3ePW12ZB5EHd5c5lOpz8OaghPiCnO6B7L2mMj4zW8VPthddB37zSFD97VWk7KNLIm0tRWaPkF2LoMJS2B/GgXkcVm5pv4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743783674; c=relaxed/simple; bh=RcRPgE6C713ahI/WHS1q1AUMAiALW8eR5rkNPODwygU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VajmmP/u2qjAFDbhIwXNh9SHR24sFpTWDo2eQ11CbZa3XOqGetMDXe4UnwnIb884jyvmFyhTCItJo+q10RsaIt1EA17lM9pMO4wITlmNIg3rojmDKcgGBaxsaTh6aqo5FW66WE1NTCxW/73dHeLQN0fuPlIU3EeVPvGj+/3u16I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=K0xZxZ7f; arc=none smtp.client-ip=209.85.208.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="K0xZxZ7f" Received: by mail-ed1-f50.google.com with SMTP id 4fb4d7f45d1cf-5ec9d24acfbso6069951a12.0 for ; Fri, 04 Apr 2025 09:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1743783671; x=1744388471; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8ArU/1UqGYXmE5dL0jOxUulm9rILJXC1pvO/5ZPhgLk=; b=K0xZxZ7fAV4xtUhc5/UOSsCjqQRpWgYmeDxTqhaL6xNRt6Hyt3JanftgEDTfyP+XH/ 6BIEs2YnUg0hmS02iH+yAFzZ+khzj/AlbpEq6qaWzVyvHDe51VQ/tG+6WvZ6vI7FBfyP Bec+/+kfjMNrUFG1eqg+4qkzaLNMM6JZdw8Yw+NQB/IJDR7OasBmMAs5LesZazwhhqaR 34oUE7RxM30zHOx2cRm52eslGNosA8wiGR+0phhAQwOw5pd1vFLR114euwn4IEjqEid+ 1hHiT7Y+OdtB/E5ddkwtHy6MNRzKp5NaQOfjDV8A4rhrOvxF/RQNRgVpdb+Ai4+SoEsZ VdfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1743783671; x=1744388471; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8ArU/1UqGYXmE5dL0jOxUulm9rILJXC1pvO/5ZPhgLk=; b=FXEk0iwYmYC9Jxtu2xcs3LToxGqSvVv/CAnTpqTjMYLHx7pB/BDjYHbw4dQBc7PsVw cduTa9VEeT5kZYPJOK7w16COtFpzKN7+OQnxFZcrxaNGm+jnuTjeTQNIWMxrM9ATIIYy 66Mvjt56+2jfZqJ0T/pd0aXQpgb4aUoARNsxEZvBaMu5UFH3fLYWKnHwngy0E1UJftt7 M7ZbOmW5uLWveJ8P3bUowKg4TWjbGRCHZ4nPZAYUPZRIFQGsjz2EDKSZMywKqjcz+ekU +fMQn4Y+kVIDJOL+CowsKkUtg8lnIONkdqLN5f88NyYuANX5Whd5mkulju+TrRni8ZrO MZwg== X-Gm-Message-State: AOJu0YyFvGEwtpvqlazpe1ayFy2L1/kHxgq8Ci6bW/tODDM10yfIfPVb D1uFIMDqL/d2bcjgPxjzOA5H3wF5VPUXvqfQuhyATeOpGjMoqAcdCRPNjQ== X-Gm-Gg: ASbGnctzoxc2cEJuFMTsEfUsXCDzAy27cRFnCtT40LSai5gHIEThUa60Titd5QZoiuA xfKEdXAMCQYQL7VU3sDAkBH1kTFU6AjEbgzIJ18Lk+fkWSLUeZGgEKbNxDRLIc7G/WxkvLgundu 3Wjuebhew7QhvLtWG/w8pTLzDO7WkNyxKIWXxidBlSDL7M8i0UNBdmmKYVgkow+BS4XbJ5Ug9KX zn7xCg1eynIBX+GZ9y0FXodTK3K4t8a/CiZhf4fur3dtYjkk12fqhkHaz2C25QuU96sVEWwsM8g kr/9yVwWDKWUNKS9GQ+GfX4Jg2x5 X-Google-Smtp-Source: AGHT+IG2l6hIYRJvgPkMiIPAzbzVr9pOEr09iiyjUWYg2xV9DbvWJXYWNaF/UtMpSLVeMSFYlaiXAQ== X-Received: by 2002:a17:907:6e8e:b0:ac2:d5d3:2b77 with SMTP id a640c23a62f3a-ac7d2e1b9c5mr418107066b.8.1743783670715; Fri, 04 Apr 2025 09:21:10 -0700 (PDT) Received: from 127.com ([2620:10d:c092:600::1:ce28]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-ac7c0184865sm273316066b.124.2025.04.04.09.21.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Apr 2025 09:21:09 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: asml.silence@gmail.com Subject: [PATCH 4/4] io_uring: reuse buffer updates for registration Date: Fri, 4 Apr 2025 17:22:17 +0100 Message-ID: <8996ffd533db8bd12c84cdc2ccef1fddbbb3da27.1743783348.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: io-uring@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 After registering an empty buffer table, we can reuse the code for buffer updates to actually register buffers. Signed-off-by: Pavel Begunkov --- io_uring/rsrc.c | 85 +++++++++++++++++++------------------------------ 1 file changed, 32 insertions(+), 53 deletions(-) diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 958eee7b4a47..6b5ec1504dcd 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -277,9 +277,11 @@ static int __io_sqe_files_update(struct io_ring_ctx *ctx, return done ? done : err; } -static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, - struct io_uring_rsrc_update2 *up, - unsigned int nr_args) +static int io_buffer_table_update(struct io_ring_ctx *ctx, + struct io_rsrc_data *buf_table, + struct io_uring_rsrc_update2 *up, + unsigned int nr_args, + unsigned *last_error) { u64 __user *tags = u64_to_user_ptr(up->tags); struct iovec fast_iov, *iov; @@ -289,9 +291,9 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, __u32 done; int i, err; - if (!ctx->buf_table.nr) + if (!buf_table->nr) return -ENXIO; - if (up->offset + nr_args > ctx->buf_table.nr) + if (up->offset + nr_args > buf_table->nr) return -EINVAL; for (done = 0; done < nr_args; done++) { @@ -316,17 +318,26 @@ static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, err = PTR_ERR(node); break; } - i = array_index_nospec(up->offset + done, ctx->buf_table.nr); - io_reset_rsrc_node(ctx, &ctx->buf_table, i); - ctx->buf_table.nodes[i] = node; + i = array_index_nospec(up->offset + done, buf_table->nr); + io_reset_rsrc_node(ctx, buf_table, i); + buf_table->nodes[i] = node; if (ctx->compat) user_data += sizeof(struct compat_iovec); else user_data += sizeof(struct iovec); } + if (last_error) + *last_error = err; return done ? done : err; } +static int __io_sqe_buffers_update(struct io_ring_ctx *ctx, + struct io_uring_rsrc_update2 *up, + unsigned int nr_args) +{ + return io_buffer_table_update(ctx, &ctx->buf_table, up, nr_args, NULL); +} + static int __io_register_rsrc_update(struct io_ring_ctx *ctx, unsigned type, struct io_uring_rsrc_update2 *up, unsigned nr_args) @@ -851,11 +862,8 @@ static struct io_rsrc_node *io_sqe_buffer_register(struct io_ring_ctx *ctx, int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg, unsigned int nr_args, u64 __user *tags) { - struct page *last_hpage = NULL; struct io_rsrc_data data; - struct iovec fast_iov, *iov = &fast_iov; - const struct iovec __user *uvec; - int i, ret; + int ret, err; BUILD_BUG_ON(IORING_MAX_REG_BUFFERS >= (1u << 16)); @@ -867,51 +875,22 @@ int io_sqe_buffers_register(struct io_ring_ctx *ctx, void __user *arg, if (ret) return ret; - if (!arg) { - ctx->buf_table = data; - return 0; - } - - for (i = 0; i < nr_args; i++) { - struct io_rsrc_node *node; - u64 tag = 0; - - uvec = (struct iovec __user *) arg; - iov = iovec_from_user(uvec, 1, 1, &fast_iov, ctx->compat); - if (IS_ERR(iov)) { - ret = PTR_ERR(iov); - break; - } - ret = io_buffer_validate(iov); - if (ret) - break; - if (ctx->compat) - arg += sizeof(struct compat_iovec); - else - arg += sizeof(struct iovec); - - if (tags) { - if (copy_from_user(&tag, &tags[i], sizeof(tag))) { - ret = -EFAULT; - break; - } - } + if (arg) { + struct io_uring_rsrc_update2 update_arg = { + .tags = (u64)(unsigned long)tags, + .data = (u64)(unsigned long)arg, + .offset = 0, + }; - node = io_sqe_buffer_register(ctx, iov, &last_hpage, tag); - if (IS_ERR(node)) { - ret = PTR_ERR(node); - break; + ret = io_buffer_table_update(ctx, &data, &update_arg, nr_args, &err); + if (ret != nr_args) { + io_clear_table_tags(&data); + io_rsrc_data_free(ctx, &data); + return err; } - data.nodes[i] = node; - } - - if (ret) { - io_clear_table_tags(&data); - io_rsrc_data_free(ctx, &data); - return ret; } ctx->buf_table = data; - return ret; + return 0; } int io_buffer_register_bvec(struct io_uring_cmd *cmd, struct request *rq,