From patchwork Sun Oct 16 20:30:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13008037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 984ACC4332F for ; Sun, 16 Oct 2022 20:33:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229633AbiJPUdH (ORCPT ); Sun, 16 Oct 2022 16:33:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42258 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229486AbiJPUdG (ORCPT ); Sun, 16 Oct 2022 16:33:06 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D76030F43 for ; Sun, 16 Oct 2022 13:33:05 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id bj12so20785120ejb.13 for ; Sun, 16 Oct 2022 13:33:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PYx2685bsWJJw39MW3yJCcH+2KP08OPq5xO0rYWVOLA=; b=THETVeKMXqOaA9K3j0TV7fM9xjDnZBu4C/ATYX6HMKAWd4D8k11Qb5aPIh+UjMwyXe RqHzi9Sim8DwrtEtUMe1nDq3hDTrFlPkrYkewT1JutHQxNN/YoBqk/xvN1HuizsBCeDO eKlVyJuwYhySLyBvbQjInF9HaqsHXq10vWI/J2KQE8q07KREj5uMSiw5nbKmF3TYHLfQ oedESvYfKsJRn1FSgyOe/weoLQYozb3uRVmJLmq5qYs8k9xPryBcZ+JMl4KcatqmQ0ob ta0o4px9pESBktI8GY/EePsECEJjCMchHI0IAKF3vv4KiNGpelZiswvKCok0gnreD6+C 2ipg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PYx2685bsWJJw39MW3yJCcH+2KP08OPq5xO0rYWVOLA=; b=YjH+cCeEtvbIlYEiFU1NLaVm0DKeMWf8Emvfx16/bWg5R8pUTwBx0UzVQYKw/3vCQq FYYlmQw5OVWA8hGIpU596d0mq6Khwl5wGV/Oh+qXvDXwWiXuTomzY5Op455d4lqBYsFm +DA6bPs5gCUS17M+t4hJij2egU4mUzxUcaK5ADLwH6loN62COUEX5/jn/ObBksBNpyLM 2yIqJcH2bzkl+/cmpthZS1MR/vYAylY15Vj+3szHCtsDdnSJ9Mp+vA7eiLs0lw03yb6k xHC+6bbvCvC/SpO8dJdU21+FD4c9365NeFU2vrkPVb0Kt4lVcE7BgZZ2edWfPMu3P995 fYGQ== X-Gm-Message-State: ACrzQf1hS7AqnotMfF/LPIJzxc0ql6/I0MbKniQ8yhCBaAHFLl24xzKQ +qjNlgqPoXwy6D3u+Lxkrl+CC7vkaG8= X-Google-Smtp-Source: AMsMyM62cRKVGerlVRLxgNeBCsANZyULs2Z09Cq11yIJfzGQxwjGFfE5PS9N22XRQSE8E3vruuRV8Q== X-Received: by 2002:a17:907:d22:b0:78e:2788:51ae with SMTP id gn34-20020a1709070d2200b0078e278851aemr6038480ejc.689.1665952383271; Sun, 16 Oct 2022 13:33:03 -0700 (PDT) Received: from 127.0.0.1localhost (94.196.234.149.threembb.co.uk. [94.196.234.149]) by smtp.gmail.com with ESMTPSA id y11-20020a1709060a8b00b00788c622fa2csm5069345ejf.135.2022.10.16.13.33.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Oct 2022 13:33:02 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 1/4] io_uring: remove FFS_SCM Date: Sun, 16 Oct 2022 21:30:48 +0100 Message-Id: <984226a1045adf42dc35d8bd7fb5a8bbfa472ce1.1665891182.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org scm'ed files lifetime is bound to ring_sock, which is destroyed strictly after we're done with registered file tables, so no need for the FFS_SCM hack. Signed-off-by: Pavel Begunkov --- io_uring/filetable.h | 15 +-------------- io_uring/io_uring.c | 2 -- io_uring/rsrc.c | 7 ++----- io_uring/rsrc.h | 4 ---- 4 files changed, 3 insertions(+), 25 deletions(-) diff --git a/io_uring/filetable.h b/io_uring/filetable.h index ff3a712e11bf..19d2aed66c72 100644 --- a/io_uring/filetable.h +++ b/io_uring/filetable.h @@ -5,22 +5,9 @@ #include #include -/* - * FFS_SCM is only available on 64-bit archs, for 32-bit we just define it as 0 - * and define IO_URING_SCM_ALL. For this case, we use SCM for all files as we - * can't safely always dereference the file when the task has exited and ring - * cleanup is done. If a file is tracked and part of SCM, then unix gc on - * process exit may reap it before __io_sqe_files_unregister() is run. - */ #define FFS_NOWAIT 0x1UL #define FFS_ISREG 0x2UL -#if defined(CONFIG_64BIT) -#define FFS_SCM 0x4UL -#else -#define IO_URING_SCM_ALL -#define FFS_SCM 0x0UL -#endif -#define FFS_MASK ~(FFS_NOWAIT|FFS_ISREG|FFS_SCM) +#define FFS_MASK ~(FFS_NOWAIT|FFS_ISREG) bool io_alloc_file_tables(struct io_file_table *table, unsigned nr_files); void io_free_file_tables(struct io_file_table *table); diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index de08d9902b30..18aa39709fae 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1587,8 +1587,6 @@ unsigned int io_file_get_flags(struct file *file) res |= FFS_ISREG; if (__io_file_supports_nowait(file, mode)) res |= FFS_NOWAIT; - if (io_file_need_scm(file)) - res |= FFS_SCM; return res; } diff --git a/io_uring/rsrc.c b/io_uring/rsrc.c index 012fdb04ec23..55d4ab96fb92 100644 --- a/io_uring/rsrc.c +++ b/io_uring/rsrc.c @@ -757,20 +757,17 @@ int io_queue_rsrc_removal(struct io_rsrc_data *data, unsigned idx, void __io_sqe_files_unregister(struct io_ring_ctx *ctx) { -#if !defined(IO_URING_SCM_ALL) int i; for (i = 0; i < ctx->nr_user_files; i++) { struct file *file = io_file_from_index(&ctx->file_table, i); - if (!file) - continue; - if (io_fixed_file_slot(&ctx->file_table, i)->file_ptr & FFS_SCM) + /* skip scm accounted files, they'll be freed by ->ring_sock */ + if (!file || io_file_need_scm(file)) continue; io_file_bitmap_clear(&ctx->file_table, i); fput(file); } -#endif #if defined(CONFIG_UNIX) if (ctx->ring_sock) { diff --git a/io_uring/rsrc.h b/io_uring/rsrc.h index 9bce15665444..81445a477622 100644 --- a/io_uring/rsrc.h +++ b/io_uring/rsrc.h @@ -82,11 +82,7 @@ int __io_scm_file_account(struct io_ring_ctx *ctx, struct file *file); #if defined(CONFIG_UNIX) static inline bool io_file_need_scm(struct file *filp) { -#if defined(IO_URING_SCM_ALL) - return true; -#else return !!unix_get_socket(filp); -#endif } #else static inline bool io_file_need_scm(struct file *filp) From patchwork Sun Oct 16 20:30:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13008039 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11135C43219 for ; Sun, 16 Oct 2022 20:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229646AbiJPUdI (ORCPT ); Sun, 16 Oct 2022 16:33:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229597AbiJPUdH (ORCPT ); Sun, 16 Oct 2022 16:33:07 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2D8553057D for ; Sun, 16 Oct 2022 13:33:06 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id a26so20862736ejc.4 for ; Sun, 16 Oct 2022 13:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZjPK20Gpm06cfu/aaNkEBhOYqkMQWlW3+Fjwk493YA0=; b=IZtW6RZRokKotnndtoGqITMYU1czP/veWNGRhbk2d5t1HKW8Z7vu/KBU5UM6UhXmLV PaxPc8kujcfYXdHMlFG3m0Nc8Wgh5HSqtlX/Lu66ePiwYe+B4Rtj4GDEEWihz5576slr BcNqHsT/bsyOl91/7qTonqTvAOzmSUUZv+zuSmSPy+mqpyVzBsml47nQwyYJ1a/Ccyoo Xnfmghmv00QpojkK5aKs7QGbD4YUqa4EC/KGQ5gAeJibmtz3FzNBEeka4paj3bC7BtOM 6S2N39VB+JVqmlqIEXTr+MpTW6+CafFwe96a30GvVpsQyjUuX0+tE+1kZjcQAOmCMZrc 1Aeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZjPK20Gpm06cfu/aaNkEBhOYqkMQWlW3+Fjwk493YA0=; b=daXi/VMfFyUZ4/5CGm1uVOi2c+E8H6pXo/wFJynchfJhKnR0z0nt1PQhklt9QuyqdH sQkWLXdj2TqheJ9mCbixSyBQemMs758gVlvDUa9zon7My9EldpFq2AHEgmSglovfV2JE A47/nfQqw+uo2fLMqlNBhtSkvPN5WTG+TU58alUnpory+iJJ+60X4lVH1kWD1Xncu3bF fK5/XPaiMMVoyz/l4qV0hm4g7OpS/mZq3ybQyjiUi/ANehAy9DwA8YElUmQlqSvWoh0r djMBQdzjARKrLegLMEMAKCtsQ880juOofrX0eygJqnbrcC2TzWI5quoo6efZx77a5LYe UrAg== X-Gm-Message-State: ACrzQf0+exf0X0Xj6JNXV+KpnFhHv1hln8VL1yVrDM5iHxrv5B61iMre QlrJ8xpbw3sD0tC024wNgy33EZp4/3Y= X-Google-Smtp-Source: AMsMyM5OYnPGbwv2+n0FMmU9dazc7UG5KwOtjr77EZije99ZoPDCyC80u6NZV65Zl7292ivF0g2Shw== X-Received: by 2002:a17:907:7635:b0:78d:c5e9:3e57 with SMTP id jy21-20020a170907763500b0078dc5e93e57mr6257074ejc.204.1665952384318; Sun, 16 Oct 2022 13:33:04 -0700 (PDT) Received: from 127.0.0.1localhost (94.196.234.149.threembb.co.uk. [94.196.234.149]) by smtp.gmail.com with ESMTPSA id y11-20020a1709060a8b00b00788c622fa2csm5069345ejf.135.2022.10.16.13.33.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Oct 2022 13:33:03 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 2/4] io_uring: kill hot path fixed file bitmap debug checks Date: Sun, 16 Oct 2022 21:30:49 +0100 Message-Id: X-Mailer: git-send-email 2.38.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org We test file_table.bitmap in io_file_get_fixed() to check invariants, don't do it, it's expensive and was showing up in profiles. Signed-off-by: Pavel Begunkov --- io_uring/filetable.h | 1 + io_uring/io_uring.c | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/io_uring/filetable.h b/io_uring/filetable.h index 19d2aed66c72..351111ff8882 100644 --- a/io_uring/filetable.h +++ b/io_uring/filetable.h @@ -25,6 +25,7 @@ unsigned int io_file_get_flags(struct file *file); static inline void io_file_bitmap_clear(struct io_file_table *table, int bit) { + WARN_ON_ONCE(!test_bit(bit, table->bitmap)); __clear_bit(bit, table->bitmap); table->alloc_hint = bit; } diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 18aa39709fae..6e50f548de1a 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -1858,7 +1858,6 @@ inline struct file *io_file_get_fixed(struct io_kiocb *req, int fd, /* mask in overlapping REQ_F and FFS bits */ req->flags |= (file_ptr << REQ_F_SUPPORT_NOWAIT_BIT); io_req_set_rsrc_node(req, ctx, 0); - WARN_ON_ONCE(file && !test_bit(fd, ctx->file_table.bitmap)); out: io_ring_submit_unlock(ctx, issue_flags); return file; From patchwork Sun Oct 16 20:30:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13008038 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85CA8C43217 for ; Sun, 16 Oct 2022 20:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229597AbiJPUdI (ORCPT ); Sun, 16 Oct 2022 16:33:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbiJPUdH (ORCPT ); Sun, 16 Oct 2022 16:33:07 -0400 Received: from mail-ej1-x631.google.com (mail-ej1-x631.google.com [IPv6:2a00:1450:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1938230F5F for ; Sun, 16 Oct 2022 13:33:06 -0700 (PDT) Received: by mail-ej1-x631.google.com with SMTP id bj12so20785202ejb.13 for ; Sun, 16 Oct 2022 13:33:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7kJ/8KrJtwWrQtL3HJvPRSeBswfmXX7pgLh8LlFHkeQ=; b=MJNd//DhY0B1B5RJxLzgCAoHw+3v3pyAxsGF76IoSXh/SXEZ06yhQcB00VCCsMNfAY l57EeWVAtu4lBstcGIAEja/yoEznOfDc/JDiIz0btRI+IuUc9vbkkreZN9UMAfvbNl6x wat8bwO2YA5O+cX2ub3wlXfezkqhB+GjA6XkCxWiG98ibC4yuOoC6zmIOdnb2P5ta2ro PG98DfBbiBR3k/nh4fmxfTcd4EyChfCkZVoypHuwFDDzy00THIw+d086LP8hVFirFX0V csShOfo1brR/Iy3y9ImM9MxFng86Vkt1BHIgBrtI6/6RYlCf0zLY4yWjQ3w2m6/UoqoV os8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7kJ/8KrJtwWrQtL3HJvPRSeBswfmXX7pgLh8LlFHkeQ=; b=LrHZTfFTno2nNYSJ1oM425+mZianyzpwsaZIXcWKC0mFX2eVK+3fRLJwdBWzjSIh1Y QS2GApgPaagYfTnktPkHzqjr7Tf3urhL2aGN/oiP0Afuqjx+96EKArZ6Y1QyPBBvi42u mOUrSlUPVDG+QS5+K+aX+FDwyHQhnfaYG9YpUIQWRFRfo/UzSr3AERDR7d+7kM08EBM8 +D7Do9HvJXp1NIO0ByGthM6DZqqXMIRdFPySu2HG5difxmqHbizrXlUuqiBdRvt5MYvV A+FwQwi9mwHpCvbJ97Bm2Qan7kFR2wLmN2O3+sJRR4CTSh/LnmXLjoaX9p0MF2Fa5Ykb ykRw== X-Gm-Message-State: ACrzQf3uEsqUjgZs3e67m98298kjH2S+axf2fY3f51bDB8PROrBWtyMH MofkZPn3C8l8OwtMcjsF2TuDZ444fqM= X-Google-Smtp-Source: AMsMyM4ihH7Z7+omEL5JzqYhtw4lNT9IJcgnKY8b/TdcxfMm4o4YboG/ppCLSDMOAzstzFXWScebew== X-Received: by 2002:a17:907:7f07:b0:779:7f94:d259 with SMTP id qf7-20020a1709077f0700b007797f94d259mr6204310ejc.525.1665952385312; Sun, 16 Oct 2022 13:33:05 -0700 (PDT) Received: from 127.0.0.1localhost (94.196.234.149.threembb.co.uk. [94.196.234.149]) by smtp.gmail.com with ESMTPSA id y11-20020a1709060a8b00b00788c622fa2csm5069345ejf.135.2022.10.16.13.33.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Oct 2022 13:33:04 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 3/4] io_uring: reuse io_alloc_req() Date: Sun, 16 Oct 2022 21:30:50 +0100 Message-Id: <6005fc88274864a49fc3096c22d8bdd605cf8576.1665891182.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org Don't duplicate io_alloc_req() in io_req_caches_free() but reuse the helper. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 6e50f548de1a..62be51fbf39c 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2560,18 +2560,14 @@ static int io_eventfd_unregister(struct io_ring_ctx *ctx) static void io_req_caches_free(struct io_ring_ctx *ctx) { - struct io_submit_state *state = &ctx->submit_state; int nr = 0; mutex_lock(&ctx->uring_lock); - io_flush_cached_locked_reqs(ctx, state); + io_flush_cached_locked_reqs(ctx, &ctx->submit_state); while (!io_req_cache_empty(ctx)) { - struct io_wq_work_node *node; - struct io_kiocb *req; + struct io_kiocb *req = io_alloc_req(ctx); - node = wq_stack_extract(&state->free_list); - req = container_of(node, struct io_kiocb, comp_list); kmem_cache_free(req_cachep, req); nr++; } From patchwork Sun Oct 16 20:30:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pavel Begunkov X-Patchwork-Id: 13008040 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CEB4BC4332F for ; Sun, 16 Oct 2022 20:33:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229622AbiJPUdK (ORCPT ); Sun, 16 Oct 2022 16:33:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42294 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229577AbiJPUdJ (ORCPT ); Sun, 16 Oct 2022 16:33:09 -0400 Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3003B3057D for ; Sun, 16 Oct 2022 13:33:08 -0700 (PDT) Received: by mail-ej1-x62c.google.com with SMTP id b2so20856216eja.6 for ; Sun, 16 Oct 2022 13:33:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tUHvLcKLFL36tYPs8CH/YI7G5vNcC4S7LNovyR3sUgc=; b=ovv705iLe0E+UXCWEah8EIgQ2/ufU/yy4BhLzRIe2SVT1vBd/il4wPPJlbuYOo49MR aCx9hSn45FXuc54eJj9oVNIStE+9D/hUtVbEc7PmxnVeTARgGdyfSPUVdgWpN4uLNnvz 2RdQzQ8qAb41h8INVG2KnxJkUX3u8xYxVQ8lzxTpjsAAprXLahMHzDncOWqe8l/i8d+a 6kbFE7gurVgwnIDaB5PJckvfEjZIYEHHSx3ZuKpb0VEROaEqrBsaT4VcR4WU8xsdMVIP rX1pdCOIACVNny4kZtVA+4kK4AFxXfokfL6GooMIdJHp7j/0mm9A0m9f2fQwUhRZWubZ owMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tUHvLcKLFL36tYPs8CH/YI7G5vNcC4S7LNovyR3sUgc=; b=WN4rghDmmp8SnzBvKtkHFy5LdOY1yFfhrzciL2DtHw2zehaA9TSFmGyQyfYqqHEtd+ W8lMtJAj7ClOirv56pHwcHwS6Yxdj8kPAfKzN+H0dtBFUoyTJ+zGTduxCz1X0KhCWWwh D3BMsb/4nL9EqLjJhlZQk2wMgEr+xu0RdFalORGNp//tIqbpPXfzoz0z6EwjbBWz8LcN gGEvKOmB7cw2+/F7hK4gjTIWAeX5CZ1oQuUdViCCg9zW0jMmveCJo5HNJLqn7xOmdQdk 85Mm/xh65kzaL+i9Tc6B3hNnZ/GpFH9nEy5274QW+VcQphqdOfchLpJ1zDkPs5ZxqcI/ MDRQ== X-Gm-Message-State: ACrzQf0+4AISWj5saJR2PSWrYktsRDgKcwij+CbrHKAUUe2fsz80dDjV lcSniO349J9vaGaWVLOdkVF3l1sVjJE= X-Google-Smtp-Source: AMsMyM4AJoTfU0xZCQG6xrkIZphYHuHMHT7WAKWQIK5KLL65H0rRwh3+p2U3HT4dLPp1Yio61OhUrg== X-Received: by 2002:a17:907:162a:b0:78e:2859:76be with SMTP id hb42-20020a170907162a00b0078e285976bemr6152436ejc.768.1665952386449; Sun, 16 Oct 2022 13:33:06 -0700 (PDT) Received: from 127.0.0.1localhost (94.196.234.149.threembb.co.uk. [94.196.234.149]) by smtp.gmail.com with ESMTPSA id y11-20020a1709060a8b00b00788c622fa2csm5069345ejf.135.2022.10.16.13.33.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 16 Oct 2022 13:33:06 -0700 (PDT) From: Pavel Begunkov To: io-uring@vger.kernel.org Cc: Jens Axboe , asml.silence@gmail.com Subject: [PATCH 4/4] io_uring: don't iopoll from io_ring_ctx_wait_and_kill() Date: Sun, 16 Oct 2022 21:30:51 +0100 Message-Id: <7c03cc91455c4a1af49c6b9cbda4e57ea467aa11.1665891182.git.asml.silence@gmail.com> X-Mailer: git-send-email 2.38.0 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: io-uring@vger.kernel.org We should not be completing requests from a task context that already undergone io_uring cancellations, i.e. __io_uring_cancel(), as there are some assumptions, e.g. aroundd cached task refs draining. Remove iopolling from io_ring_ctx_wait_and_kill() as it can be called later after PF_EXITING is set with the last task_work run. Signed-off-by: Pavel Begunkov --- io_uring/io_uring.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c index 62be51fbf39c..6cc16e39b27f 100644 --- a/io_uring/io_uring.c +++ b/io_uring/io_uring.c @@ -2804,15 +2804,12 @@ static __cold void io_ring_ctx_wait_and_kill(struct io_ring_ctx *ctx) io_poll_remove_all(ctx, NULL, true); mutex_unlock(&ctx->uring_lock); - /* failed during ring init, it couldn't have issued any requests */ - if (ctx->rings) { + /* + * If we failed setting up the ctx, we might not have any rings + * and therefore did not submit any requests + */ + if (ctx->rings) io_kill_timeouts(ctx, NULL, true); - /* if we failed setting up the ctx, we might not have any rings */ - io_iopoll_try_reap_events(ctx); - /* drop cached put refs after potentially doing completions */ - if (current->io_uring) - io_uring_drop_tctx_refs(current); - } INIT_WORK(&ctx->exit_work, io_ring_exit_work); /*