From patchwork Fri Mar 15 14:54:36 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854909 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A9AE86C2 for ; Fri, 15 Mar 2019 14:54:52 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 989D92A85F for ; Fri, 15 Mar 2019 14:54:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8CF372A8BA; Fri, 15 Mar 2019 14:54:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1F43B2A8A7 for ; Fri, 15 Mar 2019 14:54:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728592AbfCOOyv (ORCPT ); Fri, 15 Mar 2019 10:54:51 -0400 Received: from mail-pf1-f196.google.com ([209.85.210.196]:33784 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727518AbfCOOyv (ORCPT ); Fri, 15 Mar 2019 10:54:51 -0400 Received: by mail-pf1-f196.google.com with SMTP id i19so6540979pfd.0 for ; Fri, 15 Mar 2019 07:54:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=975aIYGsayGhYwnFheH5tzZGlPlxWAP+69eiKl725fc=; b=QY5YqW+AVRpk0VH2egU18qARYZ1kuUI/1JRMjcRQh2zMEzeZefQzhlaTJ1jS6qStZn jEmPalCvQf+tIb8KGOtnbVYVFwlp4Hm3p82/WZQdS9R1GJjW9dvuvEWJx5cknp4J0bGs xVaH4KFZWhx4cBIcc77+du7YkdiU+eC0WX3IENhXMmE7V4NdOBbfmtraGkXY+5z1nboQ NY86FXCeO4brCurkTt95xRAkDnB/L0pGLpaDrCdHd5Hf3Hph3M2nQUH9h4LCuLujAcxx 6oVsU58KVBNGum8S5tjYaEG382JtzamVvbtI+rfq0/6/wz5K9Ulk/zRIHggDzTKayqXn WeiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=975aIYGsayGhYwnFheH5tzZGlPlxWAP+69eiKl725fc=; b=PvR05IAL+3+AvOHN8F3puU3c01uYSNUNo9uUA4A4tHXAQFO2psMWQ2XRnueOmRWXcx Pl4e5E+2MSB3ZQtSceAgIWOpMzMy645m8w48iDfTGEE66kgRmZc+D45yGPLugs/Kr1p2 remZ9neVQt/xKQpk2XpnUjJcXA1fhMAxdebpatbNSpPfXE7JTm8KIUV6w2XEAsO58bFA 59YeKSQRvJlgpRU7GdtCsVHEqaM72F7sPJY/vyFPIqdRcfIb90KzSO+Ghe135BWx01Uw A9/OD6WT+DDfAirLurnWK3PFRqTdbSmZObOocuZa0sT7RD6Ygn6wt8F/ZZuni+I15nvS Rh5Q== X-Gm-Message-State: APjAAAVzMm6nAwizIVFLaJ8SR5j48gLgHrLgjYjvo95ZH2uZDZa67UuN 3tdxRRO0IMOHHXVjHSRlYUY495LZhf5Khg== X-Google-Smtp-Source: APXvYqwb3VQdeu5FjHc3UYDajBCDdERyhsFpzgPZgtcmQlrP27xdSmLjRHCzryGcM4O3thJ7CxsUcA== X-Received: by 2002:a17:902:24:: with SMTP id 33mr4678757pla.259.1552661689647; Fri, 15 Mar 2019 07:54:49 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:48 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 1/7] io_uring: use regular request ref counts Date: Fri, 15 Mar 2019 08:54:36 -0600 Message-Id: <20190315145442.21127-2-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Get rid of the special casing of "normal" requests not having any references to the io_kiocb. We initialize the ref count to 2, one for the submission side, and one or the completion side. Signed-off-by: Jens Axboe --- fs/io_uring.c | 54 +++++++++++++++++++++++++++++++++------------------ 1 file changed, 35 insertions(+), 19 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 5d99376d2369..9071fca118a4 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -411,7 +411,8 @@ static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx, req->ctx = ctx; req->flags = 0; - refcount_set(&req->refs, 0); + /* one is dropped after submission, the other at completion */ + refcount_set(&req->refs, 2); return req; out: io_ring_drop_ctx_refs(ctx, 1); @@ -429,10 +430,14 @@ static void io_free_req_many(struct io_ring_ctx *ctx, void **reqs, int *nr) static void io_free_req(struct io_kiocb *req) { - if (!refcount_read(&req->refs) || refcount_dec_and_test(&req->refs)) { - io_ring_drop_ctx_refs(req->ctx, 1); - kmem_cache_free(req_cachep, req); - } + io_ring_drop_ctx_refs(req->ctx, 1); + kmem_cache_free(req_cachep, req); +} + +static void io_put_req(struct io_kiocb *req) +{ + if (refcount_dec_and_test(&req->refs)) + io_free_req(req); } /* @@ -453,7 +458,8 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, io_cqring_fill_event(ctx, req->user_data, req->error, 0); - reqs[to_free++] = req; + if (refcount_dec_and_test(&req->refs)) + reqs[to_free++] = req; (*nr_events)++; /* @@ -616,7 +622,7 @@ static void io_complete_rw(struct kiocb *kiocb, long res, long res2) io_fput(req); io_cqring_add_event(req->ctx, req->user_data, res, 0); - io_free_req(req); + io_put_req(req); } static void io_complete_rw_iopoll(struct kiocb *kiocb, long res, long res2) @@ -1083,7 +1089,7 @@ static int io_nop(struct io_kiocb *req, u64 user_data) io_fput(req); } io_cqring_add_event(ctx, user_data, err, 0); - io_free_req(req); + io_put_req(req); return 0; } @@ -1146,7 +1152,7 @@ static int io_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe, io_fput(req); io_cqring_add_event(req->ctx, sqe->user_data, ret, 0); - io_free_req(req); + io_put_req(req); return 0; } @@ -1204,7 +1210,7 @@ static int io_poll_remove(struct io_kiocb *req, const struct io_uring_sqe *sqe) spin_unlock_irq(&ctx->completion_lock); io_cqring_add_event(req->ctx, sqe->user_data, ret, 0); - io_free_req(req); + io_put_req(req); return 0; } @@ -1212,7 +1218,7 @@ static void io_poll_complete(struct io_kiocb *req, __poll_t mask) { io_cqring_add_event(req->ctx, req->user_data, mangle_poll(mask), 0); io_fput(req); - io_free_req(req); + io_put_req(req); } static void io_poll_complete_work(struct work_struct *work) @@ -1346,9 +1352,6 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) INIT_LIST_HEAD(&poll->wait.entry); init_waitqueue_func_entry(&poll->wait, io_poll_wake); - /* one for removal from waitqueue, one for this function */ - refcount_set(&req->refs, 2); - mask = vfs_poll(poll->file, &ipt.pt) & poll->events; if (unlikely(!poll->head)) { /* we did not manage to set up a waitqueue, done */ @@ -1380,13 +1383,12 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) * Drop one of our refs to this req, __io_submit_sqe() will * drop the other one since we're returning an error. */ - io_free_req(req); + io_put_req(req); return ipt.error; } if (mask) io_poll_complete(req, mask); - io_free_req(req); return 0; } @@ -1524,10 +1526,13 @@ static void io_sq_wq_submit_work(struct work_struct *work) break; cond_resched(); } while (1); + + /* drop submission reference */ + io_put_req(req); } if (ret) { io_cqring_add_event(ctx, sqe->user_data, ret, 0); - io_free_req(req); + io_put_req(req); } /* async context always use a copy of the sqe */ @@ -1649,11 +1654,22 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, INIT_WORK(&req->work, io_sq_wq_submit_work); queue_work(ctx->sqo_wq, &req->work); } - ret = 0; + + /* + * Queued up for async execution, worker will release + * submit reference when the iocb is actually + * submitted. + */ + return 0; } } + + /* drop submission reference */ + io_put_req(req); + + /* and drop final reference, if we failed */ if (ret) - io_free_req(req); + io_put_req(req); return ret; } From patchwork Fri Mar 15 14:54:37 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854913 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E96C715AC for ; Fri, 15 Mar 2019 14:54:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D7DAA2A85F for ; Fri, 15 Mar 2019 14:54:53 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CC8142A8A7; Fri, 15 Mar 2019 14:54:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79A242A8C4 for ; Fri, 15 Mar 2019 14:54:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729136AbfCOOyw (ORCPT ); Fri, 15 Mar 2019 10:54:52 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:37592 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728663AbfCOOyw (ORCPT ); Fri, 15 Mar 2019 10:54:52 -0400 Received: by mail-pf1-f195.google.com with SMTP id 8so3112085pfr.4 for ; Fri, 15 Mar 2019 07:54:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=rpzM7j8edKfPQoYwMy8De353jI/mm9uBUGKHdkCij4k=; b=EBGAojY8gQkPlBFoCp+GEVtLSh73fOgI/UkeISjjMmEGI0noFDAhGuibMwE7XIZTvm zijPppvT63twHH9bOruK1US+uJoy05WwkrW7xK/2tRC4xB0zKhw08RorNdKOHmXBwQSD z1tdt+SpAgpUPM+BIjgzqQLvaxCbE9abP4fKkSmn0ibSHV76gJJwukhZiyTesbKO5xD7 uoum+PMzMVLWiqKR11VG80xO4iW7gtMUZWNWRMnr9g+Z0RTv3jzmLDygkTUl5lLRdM03 HwrEnuXPW76qA4CJ4ZAVwZ12cK3gZrxdmkEvHpD6ZmJZBOqk3xDGDQYmlujmLWXaesy5 eIJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=rpzM7j8edKfPQoYwMy8De353jI/mm9uBUGKHdkCij4k=; b=BpeNluJyqyMyOXtpSkgLepNtvYeYvRbv4l0+zxu4G80V1/ZPpe7ob8uOPi6G9O832j VvWFhLrTald1bDEPEUr6kn0IQUf+HwfHeo6Naq6M4vEkITEFruXEnEOZjvy27dZ7EN3X 33UZ+UFX50o0l2jVgNUrwHk4kT7ymy1ABYAjn4pLVB1YNVr4NHwfAZncphD2czjmjXs7 ZWxzmYPQR5k6BcX/0CW2akd3kTEFysXn42QJLOnrEkQBsDIp2KqofUQQDxgW4I72vhBa swzp1lyO0P73p7vxIm2sSKmLIj70agbJrngiWhGs+H3iYlOBOPQk3qBxWLO9df5Avzum M1lQ== X-Gm-Message-State: APjAAAWbRTMK39dKLyxmqW80pILwb+uT0fJxUtVgTmAhq5WlrMnYbDLY JJO7BUV32UlqxSRTy3H3orTaTXTLFWCJxQ== X-Google-Smtp-Source: APXvYqzRWKc6cv+iKNZtSVqtXA9NrU0vCGl9GqAAgPtf0G1pUBxydJm0tBOC68DoUM90n1Wur0aOHw== X-Received: by 2002:a17:902:7242:: with SMTP id c2mr4551994pll.245.1552661691585; Fri, 15 Mar 2019 07:54:51 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:50 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 2/7] io_uring: make io_read/write return an integer Date: Fri, 15 Mar 2019 08:54:37 -0600 Message-Id: <20190315145442.21127-3-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The callers all convert to an integer, and we only return 0/-ERROR anyway. Signed-off-by: Jens Axboe --- fs/io_uring.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 9071fca118a4..caf39663466f 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -893,7 +893,7 @@ static int io_import_iovec(struct io_ring_ctx *ctx, int rw, opcode = READ_ONCE(sqe->opcode); if (opcode == IORING_OP_READ_FIXED || opcode == IORING_OP_WRITE_FIXED) { - ssize_t ret = io_import_fixed(ctx, rw, sqe, iter); + int ret = io_import_fixed(ctx, rw, sqe, iter); *iovec = NULL; return ret; } @@ -951,15 +951,15 @@ static void io_async_list_note(int rw, struct io_kiocb *req, size_t len) async_list->io_end = io_end; } -static ssize_t io_read(struct io_kiocb *req, const struct sqe_submit *s, - bool force_nonblock, struct io_submit_state *state) +static int io_read(struct io_kiocb *req, const struct sqe_submit *s, + bool force_nonblock, struct io_submit_state *state) { struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs; struct kiocb *kiocb = &req->rw; struct iov_iter iter; struct file *file; size_t iov_count; - ssize_t ret; + int ret; ret = io_prep_rw(req, s, force_nonblock, state); if (ret) @@ -1004,15 +1004,15 @@ static ssize_t io_read(struct io_kiocb *req, const struct sqe_submit *s, return ret; } -static ssize_t io_write(struct io_kiocb *req, const struct sqe_submit *s, - bool force_nonblock, struct io_submit_state *state) +static int io_write(struct io_kiocb *req, const struct sqe_submit *s, + bool force_nonblock, struct io_submit_state *state) { struct iovec inline_vecs[UIO_FASTIOV], *iovec = inline_vecs; struct kiocb *kiocb = &req->rw; struct iov_iter iter; struct file *file; size_t iov_count; - ssize_t ret; + int ret; ret = io_prep_rw(req, s, force_nonblock, state); if (ret) @@ -1396,8 +1396,7 @@ static int __io_submit_sqe(struct io_ring_ctx *ctx, struct io_kiocb *req, const struct sqe_submit *s, bool force_nonblock, struct io_submit_state *state) { - ssize_t ret; - int opcode; + int ret, opcode; if (unlikely(s->index >= ctx->sq_entries)) return -EINVAL; @@ -1623,7 +1622,7 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, struct io_submit_state *state) { struct io_kiocb *req; - ssize_t ret; + int ret; /* enforce forwards compatibility on users */ if (unlikely(s->sqe->flags & ~IOSQE_FIXED_FILE)) From patchwork Fri Mar 15 14:54:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854917 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C47FC15AC for ; Fri, 15 Mar 2019 14:54:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B48022A85F for ; Fri, 15 Mar 2019 14:54:55 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A90A72A8A7; Fri, 15 Mar 2019 14:54:55 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 594BF2A8BB for ; Fri, 15 Mar 2019 14:54:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729151AbfCOOyy (ORCPT ); Fri, 15 Mar 2019 10:54:54 -0400 Received: from mail-pf1-f195.google.com ([209.85.210.195]:35532 "EHLO mail-pf1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728663AbfCOOyy (ORCPT ); Fri, 15 Mar 2019 10:54:54 -0400 Received: by mail-pf1-f195.google.com with SMTP id j5so6540586pfa.2 for ; Fri, 15 Mar 2019 07:54:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JIi4VzMT1X1u9pm4Nir4DVemESOOMMWl9QaKO0EzHRw=; b=S+vB3OAh6VkVMR2ruPyXezbjsJrdML4whtAmfDrA9iH+Pxdx2+VsFscRqyzrswElzY euB7jviOFgI91Gn7U3oqDLwHKOBURBYCrySbn3I2ex1Zi51YuKUD/M3OF3XDhZQLtgP/ V1UBJh929GumUi+3RkDXLUZ+/rDmriIwCcwOwtFLSJcUdS8T/djQnHsDm9QcaXDA4Yse H9Y+LCYnrXmdfccy7D8IZZ3WtFCeAdUC1KZnByzLI+daOYrksEBS71V0eCiBqNOtMqFt 3RPD0qDQkZd5M4Qxwgbv41um/iXsDZ5FQjAxASG998G6E7n/yaDNwEM8OrddCCURtl81 0eJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JIi4VzMT1X1u9pm4Nir4DVemESOOMMWl9QaKO0EzHRw=; b=AYDgQ2mDO06PCxgk+vtqOjuC0UnjvtQ/t9NZgCFDj7odg+bUPgkVz/qweTluPlvC75 eGj+tG9knk9ljBdkgBPtc7Nv2pOcyNodO4zK0M8E4c6s0JnQ1IET96ZShN37TEqXeFH6 YkAaJ0rW8AZDkLxjBEIwBYc99BSNqMrRNeFYaGQW195a3/ediX1hrI+Xz/3g8ybSsjdN d8z2Ci8SuUSsc/EqhcXVJI0WD1zVUuAG+ozcmwUcqS8n5zeIM08JeRqBvdNubTYKqswk fjjsH9Eq8BLIJU/XqxK0NE1FuRAjTX7ex2wGFuJp1AxtC853ejSM5ZTpnnIKtIxa4rD+ o8pA== X-Gm-Message-State: APjAAAVDSpuaWz1yDVUn4PfaMz7kv9aIRhGlXLNW6OOh0HIkq8Uo/2wf WO/9iBX5mu37NJVRR9XMreAVJlW89SZYGA== X-Google-Smtp-Source: APXvYqzFctntzhP71QP4nVH82f+PLKqdUVBkejjagHJjUaQ7uiFmNkZy8zRE+lf9r0Q5WHn9EAteiw== X-Received: by 2002:a17:902:266:: with SMTP id 93mr4726151plc.161.1552661693264; Fri, 15 Mar 2019 07:54:53 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:52 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 3/7] io_uring: add prepped flag Date: Fri, 15 Mar 2019 08:54:38 -0600 Message-Id: <20190315145442.21127-4-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP We currently use the fact that if ->ki_filp is already set, then we've done the prep. In preparation for moving the file assignment earlier, use a separate flag to tell whether the request has been prepped for IO or not. Signed-off-by: Jens Axboe --- fs/io_uring.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index caf39663466f..d259e8a6cb2e 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -214,6 +214,7 @@ struct io_kiocb { #define REQ_F_IOPOLL_COMPLETED 2 /* polled IO has completed */ #define REQ_F_FIXED_FILE 4 /* ctx owns file */ #define REQ_F_SEQ_PREV 8 /* sequential with previous */ +#define REQ_F_PREPPED 16 /* prep already done */ u64 user_data; u64 error; @@ -741,7 +742,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct sqe_submit *s, int fd, ret; /* For -EAGAIN retry, everything is already prepped */ - if (kiocb->ki_filp) + if (req->flags & REQ_F_PREPPED) return 0; flags = READ_ONCE(sqe->flags); @@ -799,6 +800,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct sqe_submit *s, } kiocb->ki_complete = io_complete_rw; } + req->flags |= REQ_F_PREPPED; return 0; out_fput: if (!(flags & IOSQE_FIXED_FILE)) { @@ -1099,8 +1101,8 @@ static int io_prep_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe) unsigned flags; int fd; - /* Prep already done */ - if (req->rw.ki_filp) + /* Prep already done (EAGAIN retry) */ + if (req->flags & REQ_F_PREPPED) return 0; if (unlikely(ctx->flags & IORING_SETUP_IOPOLL)) @@ -1122,6 +1124,7 @@ static int io_prep_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe) return -EBADF; } + req->flags |= REQ_F_PREPPED; return 0; } @@ -1632,8 +1635,6 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, if (unlikely(!req)) return -EAGAIN; - req->rw.ki_filp = NULL; - ret = __io_submit_sqe(ctx, req, s, true, state); if (ret == -EAGAIN) { struct io_uring_sqe *sqe_copy; From patchwork Fri Mar 15 14:54:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854921 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 612A41515 for ; Fri, 15 Mar 2019 14:54:58 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4E9C82A8A7 for ; Fri, 15 Mar 2019 14:54:58 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 42F6B2A8BB; Fri, 15 Mar 2019 14:54:58 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 608822A8A7 for ; Fri, 15 Mar 2019 14:54:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729216AbfCOOy4 (ORCPT ); Fri, 15 Mar 2019 10:54:56 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:41203 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729157AbfCOOy4 (ORCPT ); Fri, 15 Mar 2019 10:54:56 -0400 Received: by mail-pg1-f195.google.com with SMTP id k11so6633183pgb.8 for ; Fri, 15 Mar 2019 07:54:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VfwoNYgRi6g5zaEp6znu37VlEfaXVzN7EGevuZSfpuQ=; b=QFDl2oB5hUOAz6PfaT3jhMsbD/yJNdYExpn9bjREr2B1+g6/E4+szyWdVsWCiWf+q1 Xx1pyGjB5EIkDfwGyGEd2xQE2U93HdVunJ7CeRSstcZlMceQNVnctel+2mAioNO630uY +1zD/q9Irc2OSZbPRGmvF9t4cIgTSOOra1JD8HqadxJVFV/DOmR5KcimOGwEULZhmr0O XyAZbrSoDvbKwDB5CZWx+1goGIALhm6qebI5/mOlHD4um0h3CekUUQqvnMALoVe/II3A bYWMNl7GceBieaBkntIkLBa1FBcX/KxdYjJXEIlo1tGrhrlOmG5/tR3gvMU7mHDDiSxq Ndhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VfwoNYgRi6g5zaEp6znu37VlEfaXVzN7EGevuZSfpuQ=; b=Frv4HXbMjWWqimT7vcTlTb05Tzb6ocpuOeI76+0KZKWBkdHkgpMsq7EIajnph3b3Z0 x7d8K3jL67oppI8bOiTsfmkT6TIkpZ+r3J7fiYsYAPVoNPMe1Vad0lCp0SzZdsPTKjK1 QxoSRYzsz//M2Lg4eitHhz3D3t1wk8IwhnmOXAkMva5AEooxn8QM03PnZ8y2RzY3tHlj oLYmU4ACF636PA4F5CFcGZfXGsJBWtis5NYLzCyu5Q4k1C2wCASMAlb5UXE1cDAvnH+Z PMZvuHQAk+ipDDcQmlDvV2swYIHCjdMUM6aOBW9yIkYWeuCScmI6BC7y+xwzRg51+nIE 58Pw== X-Gm-Message-State: APjAAAXcrUjVbxCdRZEFKNk4YR24yHLtyKkvEAE9lf+hIysSCCLJc+7k 4T8r/C2WFSPzIh3Qmaj7CQE5tc0rIdX8qg== X-Google-Smtp-Source: APXvYqxhY5ilR9a+6qGIDDn3zO2ftpxBaOfsonDwbdflLU1Kb2GaLc2Y890NB43hcvl78ld5fXVZTQ== X-Received: by 2002:a62:1346:: with SMTP id b67mr4570545pfj.195.1552661694927; Fri, 15 Mar 2019 07:54:54 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:54 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 4/7] io_uring: fix fget/fput handling Date: Fri, 15 Mar 2019 08:54:39 -0600 Message-Id: <20190315145442.21127-5-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This isn't a straight port of commit 84c4e1f89fef for aio.c, since io_uring doesn't use files in exactly the same way. But it's pretty close. See the commit message for that commit. This essentially fixes a use-after-free with the poll command handling, but it takes cue from Linus's approach to just simplifying the file handling. We move the setup of the file into a higher level location, so the individual commands don't have to deal with it. And then we release the reference when we free the associated io_kiocb. Fixes: 221c5eb23382 ("io_uring: add support for IORING_OP_POLL") Signed-off-by: Jens Axboe --- fs/io_uring.c | 236 ++++++++++++++++++++++---------------------------- 1 file changed, 103 insertions(+), 133 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index d259e8a6cb2e..6c12f2e5dfe6 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -189,6 +189,10 @@ struct sqe_submit { bool needs_fixed_file; }; +/* + * First field must be the file pointer in all the + * iocb unions! See also 'struct kiocb' in + */ struct io_poll_iocb { struct file *file; struct wait_queue_head *head; @@ -198,8 +202,15 @@ struct io_poll_iocb { struct wait_queue_entry wait; }; +/* + * NOTE! Each of the iocb union members has the file pointer + * as the first entry in their struct definition. So you can + * access the file pointer through any of the sub-structs, + * or directly as just 'ki_filp' in this struct. + */ struct io_kiocb { union { + struct file *file; struct kiocb rw; struct io_poll_iocb poll; }; @@ -429,8 +440,16 @@ static void io_free_req_many(struct io_ring_ctx *ctx, void **reqs, int *nr) } } +static void io_fput(struct io_kiocb *req) +{ + if (!(req->flags & REQ_F_FIXED_FILE)) + fput(req->file); +} + static void io_free_req(struct io_kiocb *req) { + if (req->file) + io_fput(req); io_ring_drop_ctx_refs(req->ctx, 1); kmem_cache_free(req_cachep, req); } @@ -448,45 +467,34 @@ static void io_iopoll_complete(struct io_ring_ctx *ctx, unsigned int *nr_events, struct list_head *done) { void *reqs[IO_IOPOLL_BATCH]; - int file_count, to_free; - struct file *file = NULL; struct io_kiocb *req; + int to_free; - file_count = to_free = 0; + to_free = 0; while (!list_empty(done)) { req = list_first_entry(done, struct io_kiocb, list); list_del(&req->list); io_cqring_fill_event(ctx, req->user_data, req->error, 0); - - if (refcount_dec_and_test(&req->refs)) - reqs[to_free++] = req; (*nr_events)++; - /* - * Batched puts of the same file, to avoid dirtying the - * file usage count multiple times, if avoidable. - */ - if (!(req->flags & REQ_F_FIXED_FILE)) { - if (!file) { - file = req->rw.ki_filp; - file_count = 1; - } else if (file == req->rw.ki_filp) { - file_count++; + if (refcount_dec_and_test(&req->refs)) { + /* If we're not using fixed files, we have to pair the + * completion part with the file put. Use regular + * completions for those, only batch free for fixed + * file. + */ + if (req->flags & REQ_F_FIXED_FILE) { + reqs[to_free++] = req; + if (to_free == ARRAY_SIZE(reqs)) + io_free_req_many(ctx, reqs, &to_free); } else { - fput_many(file, file_count); - file = req->rw.ki_filp; - file_count = 1; + io_free_req(req); } } - - if (to_free == ARRAY_SIZE(reqs)) - io_free_req_many(ctx, reqs, &to_free); } - io_commit_cqring(ctx); - if (file) - fput_many(file, file_count); + io_commit_cqring(ctx); io_free_req_many(ctx, reqs, &to_free); } @@ -609,19 +617,12 @@ static void kiocb_end_write(struct kiocb *kiocb) } } -static void io_fput(struct io_kiocb *req) -{ - if (!(req->flags & REQ_F_FIXED_FILE)) - fput(req->rw.ki_filp); -} - static void io_complete_rw(struct kiocb *kiocb, long res, long res2) { struct io_kiocb *req = container_of(kiocb, struct io_kiocb, rw); kiocb_end_write(kiocb); - io_fput(req); io_cqring_add_event(req->ctx, req->user_data, res, 0); io_put_req(req); } @@ -738,31 +739,18 @@ static int io_prep_rw(struct io_kiocb *req, const struct sqe_submit *s, const struct io_uring_sqe *sqe = s->sqe; struct io_ring_ctx *ctx = req->ctx; struct kiocb *kiocb = &req->rw; - unsigned ioprio, flags; - int fd, ret; + unsigned ioprio; + int ret; + if (!req->file) + return -EBADF; /* For -EAGAIN retry, everything is already prepped */ if (req->flags & REQ_F_PREPPED) return 0; - flags = READ_ONCE(sqe->flags); - fd = READ_ONCE(sqe->fd); + if (force_nonblock && !io_file_supports_async(req->file)) + force_nonblock = false; - if (flags & IOSQE_FIXED_FILE) { - if (unlikely(!ctx->user_files || - (unsigned) fd >= ctx->nr_user_files)) - return -EBADF; - kiocb->ki_filp = ctx->user_files[fd]; - req->flags |= REQ_F_FIXED_FILE; - } else { - if (s->needs_fixed_file) - return -EBADF; - kiocb->ki_filp = io_file_get(state, fd); - if (unlikely(!kiocb->ki_filp)) - return -EBADF; - if (force_nonblock && !io_file_supports_async(kiocb->ki_filp)) - force_nonblock = false; - } kiocb->ki_pos = READ_ONCE(sqe->off); kiocb->ki_flags = iocb_flags(kiocb->ki_filp); kiocb->ki_hint = ki_hint_validate(file_write_hint(kiocb->ki_filp)); @@ -771,7 +759,7 @@ static int io_prep_rw(struct io_kiocb *req, const struct sqe_submit *s, if (ioprio) { ret = ioprio_check_cap(ioprio); if (ret) - goto out_fput; + return ret; kiocb->ki_ioprio = ioprio; } else @@ -779,39 +767,26 @@ static int io_prep_rw(struct io_kiocb *req, const struct sqe_submit *s, ret = kiocb_set_rw_flags(kiocb, READ_ONCE(sqe->rw_flags)); if (unlikely(ret)) - goto out_fput; + return ret; if (force_nonblock) { kiocb->ki_flags |= IOCB_NOWAIT; req->flags |= REQ_F_FORCE_NONBLOCK; } if (ctx->flags & IORING_SETUP_IOPOLL) { - ret = -EOPNOTSUPP; if (!(kiocb->ki_flags & IOCB_DIRECT) || !kiocb->ki_filp->f_op->iopoll) - goto out_fput; + return -EOPNOTSUPP; req->error = 0; kiocb->ki_flags |= IOCB_HIPRI; kiocb->ki_complete = io_complete_rw_iopoll; } else { - if (kiocb->ki_flags & IOCB_HIPRI) { - ret = -EINVAL; - goto out_fput; - } + if (kiocb->ki_flags & IOCB_HIPRI) + return -EINVAL; kiocb->ki_complete = io_complete_rw; } req->flags |= REQ_F_PREPPED; return 0; -out_fput: - if (!(flags & IOSQE_FIXED_FILE)) { - /* - * in case of error, we didn't use this file reference. drop it. - */ - if (state) - state->used_refs--; - io_file_put(state, kiocb->ki_filp); - } - return ret; } static inline void io_rw_done(struct kiocb *kiocb, ssize_t ret) @@ -968,16 +943,14 @@ static int io_read(struct io_kiocb *req, const struct sqe_submit *s, return ret; file = kiocb->ki_filp; - ret = -EBADF; if (unlikely(!(file->f_mode & FMODE_READ))) - goto out_fput; - ret = -EINVAL; + return -EBADF; if (unlikely(!file->f_op->read_iter)) - goto out_fput; + return -EINVAL; ret = io_import_iovec(req->ctx, READ, s, &iovec, &iter); if (ret) - goto out_fput; + return ret; iov_count = iov_iter_count(&iter); ret = rw_verify_area(READ, file, &kiocb->ki_pos, iov_count); @@ -999,10 +972,6 @@ static int io_read(struct io_kiocb *req, const struct sqe_submit *s, } } kfree(iovec); -out_fput: - /* Hold on to the file for -EAGAIN */ - if (unlikely(ret && ret != -EAGAIN)) - io_fput(req); return ret; } @@ -1020,17 +989,15 @@ static int io_write(struct io_kiocb *req, const struct sqe_submit *s, if (ret) return ret; - ret = -EBADF; file = kiocb->ki_filp; if (unlikely(!(file->f_mode & FMODE_WRITE))) - goto out_fput; - ret = -EINVAL; + return -EBADF; if (unlikely(!file->f_op->write_iter)) - goto out_fput; + return -EINVAL; ret = io_import_iovec(req->ctx, WRITE, s, &iovec, &iter); if (ret) - goto out_fput; + return ret; iov_count = iov_iter_count(&iter); @@ -1062,10 +1029,6 @@ static int io_write(struct io_kiocb *req, const struct sqe_submit *s, } out_free: kfree(iovec); -out_fput: - /* Hold on to the file for -EAGAIN */ - if (unlikely(ret && ret != -EAGAIN)) - io_fput(req); return ret; } @@ -1080,16 +1043,6 @@ static int io_nop(struct io_kiocb *req, u64 user_data) if (unlikely(ctx->flags & IORING_SETUP_IOPOLL)) return -EINVAL; - /* - * Twilight zone - it's possible that someone issued an opcode that - * has a file attached, then got -EAGAIN on submission, and changed - * the sqe before we retried it from async context. Avoid dropping - * a file reference for this malicious case, and flag the error. - */ - if (req->rw.ki_filp) { - err = -EBADF; - io_fput(req); - } io_cqring_add_event(ctx, user_data, err, 0); io_put_req(req); return 0; @@ -1098,9 +1051,9 @@ static int io_nop(struct io_kiocb *req, u64 user_data) static int io_prep_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe) { struct io_ring_ctx *ctx = req->ctx; - unsigned flags; - int fd; + if (!req->file) + return -EBADF; /* Prep already done (EAGAIN retry) */ if (req->flags & REQ_F_PREPPED) return 0; @@ -1110,20 +1063,6 @@ static int io_prep_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe) if (unlikely(sqe->addr || sqe->ioprio || sqe->buf_index)) return -EINVAL; - fd = READ_ONCE(sqe->fd); - flags = READ_ONCE(sqe->flags); - - if (flags & IOSQE_FIXED_FILE) { - if (unlikely(!ctx->user_files || fd >= ctx->nr_user_files)) - return -EBADF; - req->rw.ki_filp = ctx->user_files[fd]; - req->flags |= REQ_F_FIXED_FILE; - } else { - req->rw.ki_filp = fget(fd); - if (unlikely(!req->rw.ki_filp)) - return -EBADF; - } - req->flags |= REQ_F_PREPPED; return 0; } @@ -1153,7 +1092,6 @@ static int io_fsync(struct io_kiocb *req, const struct io_uring_sqe *sqe, end > 0 ? end : LLONG_MAX, fsync_flags & IORING_FSYNC_DATASYNC); - io_fput(req); io_cqring_add_event(req->ctx, sqe->user_data, ret, 0); io_put_req(req); return 0; @@ -1220,7 +1158,6 @@ static int io_poll_remove(struct io_kiocb *req, const struct io_uring_sqe *sqe) static void io_poll_complete(struct io_kiocb *req, __poll_t mask) { io_cqring_add_event(req->ctx, req->user_data, mangle_poll(mask), 0); - io_fput(req); io_put_req(req); } @@ -1314,34 +1251,20 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) struct io_poll_iocb *poll = &req->poll; struct io_ring_ctx *ctx = req->ctx; struct io_poll_table ipt; - unsigned flags; __poll_t mask; u16 events; - int fd; if (unlikely(req->ctx->flags & IORING_SETUP_IOPOLL)) return -EINVAL; if (sqe->addr || sqe->ioprio || sqe->off || sqe->len || sqe->buf_index) return -EINVAL; + if (!poll->file) + return -EBADF; INIT_WORK(&req->work, io_poll_complete_work); events = READ_ONCE(sqe->poll_events); poll->events = demangle_poll(events) | EPOLLERR | EPOLLHUP; - flags = READ_ONCE(sqe->flags); - fd = READ_ONCE(sqe->fd); - - if (flags & IOSQE_FIXED_FILE) { - if (unlikely(!ctx->user_files || fd >= ctx->nr_user_files)) - return -EBADF; - poll->file = ctx->user_files[fd]; - req->flags |= REQ_F_FIXED_FILE; - } else { - poll->file = fget(fd); - } - if (unlikely(!poll->file)) - return -EBADF; - poll->head = NULL; poll->woken = false; poll->canceled = false; @@ -1380,8 +1303,6 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) out: if (unlikely(ipt.error)) { - if (!(flags & IOSQE_FIXED_FILE)) - fput(poll->file); /* * Drop one of our refs to this req, __io_submit_sqe() will * drop the other one since we're returning an error. @@ -1621,6 +1542,50 @@ static bool io_add_to_prev_work(struct async_list *list, struct io_kiocb *req) return ret; } +static bool io_op_needs_file(const struct io_uring_sqe *sqe) +{ + int op = READ_ONCE(sqe->opcode); + + switch (op) { + case IORING_OP_NOP: + case IORING_OP_POLL_REMOVE: + return false; + default: + return true; + } +} + +static int io_req_set_file(struct io_ring_ctx *ctx, const struct sqe_submit *s, + struct io_submit_state *state, struct io_kiocb *req) +{ + unsigned flags; + int fd; + + flags = READ_ONCE(s->sqe->flags); + fd = READ_ONCE(s->sqe->fd); + + if (!io_op_needs_file(s->sqe)) { + req->file = NULL; + return 0; + } + + if (flags & IOSQE_FIXED_FILE) { + if (unlikely(!ctx->user_files || + (unsigned) fd >= ctx->nr_user_files)) + return -EBADF; + req->file = ctx->user_files[fd]; + req->flags |= REQ_F_FIXED_FILE; + } else { + if (s->needs_fixed_file) + return -EBADF; + req->file = io_file_get(state, fd); + if (unlikely(!req->file)) + return -EBADF; + } + + return 0; +} + static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, struct io_submit_state *state) { @@ -1635,6 +1600,10 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, if (unlikely(!req)) return -EAGAIN; + ret = io_req_set_file(ctx, s, state, req); + if (unlikely(ret)) + goto out; + ret = __io_submit_sqe(ctx, req, s, true, state); if (ret == -EAGAIN) { struct io_uring_sqe *sqe_copy; @@ -1664,6 +1633,7 @@ static int io_submit_sqe(struct io_ring_ctx *ctx, struct sqe_submit *s, } } +out: /* drop submission reference */ io_put_req(req); From patchwork Fri Mar 15 14:54:40 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854925 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 96CB06C2 for ; Fri, 15 Mar 2019 14:55:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 818242A8D1 for ; Fri, 15 Mar 2019 14:55:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7255C2A8D5; Fri, 15 Mar 2019 14:55:00 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 620992A85F for ; Fri, 15 Mar 2019 14:54:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729285AbfCOOy6 (ORCPT ); Fri, 15 Mar 2019 10:54:58 -0400 Received: from mail-pf1-f193.google.com ([209.85.210.193]:43413 "EHLO mail-pf1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729221AbfCOOy6 (ORCPT ); Fri, 15 Mar 2019 10:54:58 -0400 Received: by mail-pf1-f193.google.com with SMTP id q17so6512626pfh.10 for ; Fri, 15 Mar 2019 07:54:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=oEj8RboTvTc2Nc/I/jt66K7OFIKQLXjtkhf4/fIsNF4=; b=QTwlwNoXI2u497UwzmPtpvbR/oafjZlas8UckstomsCrtk/p5fr0HjnxD6AYJJYAzN po3XcZSCVLYW5HxzLbFoMjIVpeEdFivIzKAd7T/3ygRtk3CV7RZjsF5oyLUMmyZCHMlU mqDHnffwooCKjcyCVGsHLN2nTFRQVDc2OGetwS8BlXziUzVgS6dmhFsuRWztwBp9IGRI DkqlfEP2wnK24wBq9wNgP70xoIcgXLJh8e4ubUo88Auvf645PrPGDhca7iOFyKcrjluI yV31vYvWH+c6gyqjhl6EjSYlWk1tMcO3SFUwYALj4SPTosQyMfoSoLkR/bqCieVVgJSq 7lHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=oEj8RboTvTc2Nc/I/jt66K7OFIKQLXjtkhf4/fIsNF4=; b=RjugNblezaHmTVCAXtw40DovyFNPM2tlDDi5ngIPR8hFDqFyI2viKgYDxuUertMe3i bzuKnabHimCRkWeKup238gHU6b5zuRIccps84PmnZ2OAqmoXx4THQHxSN+EMHHqLyI2U 7+LoutbelZN5QO8pBNXAa/jnb2TKagdDoPaTZQX0psBi10GCQP5vCAjibWPIUKnnpSW7 7tASycrnbl56bnnYIkqReOYCUJJTkVSPs0y6J9bDsdWMI2uAbUjDIx15MR+HAyoKFIDB WsdxPrAk4J2wxzCf11oPS0yoRGOSxxcWutN3huQy1y9wlSmbJaaXKl9++PnfsBOt9lgv LjcQ== X-Gm-Message-State: APjAAAWHONe30HfnH7smEyFTtrHQMo2yA7MJZha98mLSevQWRkRDkrtK /6TQnDaEx/68spFPxkNW8p8faCHxh028wg== X-Google-Smtp-Source: APXvYqzBCw2r/VppMgJVdhiecJXXC62LaHudpuLXcygqZ8zp0hAqCsKyGaEpA3EQKM6jnWf2Kyg/Mg== X-Received: by 2002:a62:2ad1:: with SMTP id q200mr4435683pfq.34.1552661696664; Fri, 15 Mar 2019 07:54:56 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:55 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 5/7] io_uring: fix poll races Date: Fri, 15 Mar 2019 08:54:40 -0600 Message-Id: <20190315145442.21127-6-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This is a straight port of Al's fix for the aio poll implementation, since the io_uring version is heavily based on that. The below description is almost straight from that patch, just modified to fit the io_uring situation. io_poll() has to cope with several unpleasant problems: * requests that might stay around indefinitely need to be made visible for io_cancel(2); that must not be done to a request already completed, though. * in cases when ->poll() has placed us on a waitqueue, wakeup might have happened (and request completed) before ->poll() returns. * worse, in some early wakeup cases request might end up re-added into the queue later - we can't treat "woken up and currently not in the queue" as "it's not going to stick around indefinitely" * ... moreover, ->poll() might have decided not to put it on any queues to start with, and that needs to be distinguished from the previous case * ->poll() might have tried to put us on more than one queue. Only the first will succeed for io poll, so we might end up missing wakeups. OTOH, we might very well notice that only after the wakeup hits and request gets completed (all before ->poll() gets around to the second poll_wait()). In that case it's too late to decide that we have an error. req->woken was an attempt to deal with that. Unfortunately, it was broken. What we need to keep track of is not that wakeup has happened - the thing might come back after that. It's that async reference is already gone and won't come back, so we can't (and needn't) put the request on the list of cancellables. The easiest case is "request hadn't been put on any waitqueues"; we can tell by seeing NULL apt.head, and in that case there won't be anything async. We should either complete the request ourselves (if vfs_poll() reports anything of interest) or return an error. In all other cases we get exclusion with wakeups by grabbing the queue lock. If request is currently on queue and we have something interesting from vfs_poll(), we can steal it and complete the request ourselves. If it's on queue and vfs_poll() has not reported anything interesting, we either put it on the cancellable list, or, if we know that it hadn't been put on all queues ->poll() wanted it on, we steal it and return an error. If it's _not_ on queue, it's either been already dealt with (in which case we do nothing), or there's io_poll_complete_work() about to be executed. In that case we either put it on the cancellable list, or, if we know it hadn't been put on all queues ->poll() wanted it on, simulate what cancel would've done. Fixes: 221c5eb23382 ("io_uring: add support for IORING_OP_POLL") Signed-off-by: Jens Axboe --- fs/io_uring.c | 110 +++++++++++++++++++++++++------------------------- 1 file changed, 54 insertions(+), 56 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 6c12f2e5dfe6..443ca5615554 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -197,7 +197,7 @@ struct io_poll_iocb { struct file *file; struct wait_queue_head *head; __poll_t events; - bool woken; + bool done; bool canceled; struct wait_queue_entry wait; }; @@ -367,20 +367,25 @@ static void io_cqring_fill_event(struct io_ring_ctx *ctx, u64 ki_user_data, } } -static void io_cqring_add_event(struct io_ring_ctx *ctx, u64 ki_user_data, +static void io_cqring_ev_posted(struct io_ring_ctx *ctx) +{ + if (waitqueue_active(&ctx->wait)) + wake_up(&ctx->wait); + if (waitqueue_active(&ctx->sqo_wait)) + wake_up(&ctx->sqo_wait); +} + +static void io_cqring_add_event(struct io_ring_ctx *ctx, u64 user_data, long res, unsigned ev_flags) { unsigned long flags; spin_lock_irqsave(&ctx->completion_lock, flags); - io_cqring_fill_event(ctx, ki_user_data, res, ev_flags); + io_cqring_fill_event(ctx, user_data, res, ev_flags); io_commit_cqring(ctx); spin_unlock_irqrestore(&ctx->completion_lock, flags); - if (waitqueue_active(&ctx->wait)) - wake_up(&ctx->wait); - if (waitqueue_active(&ctx->sqo_wait)) - wake_up(&ctx->sqo_wait); + io_cqring_ev_posted(ctx); } static void io_ring_drop_ctx_refs(struct io_ring_ctx *ctx, unsigned refs) @@ -1155,10 +1160,13 @@ static int io_poll_remove(struct io_kiocb *req, const struct io_uring_sqe *sqe) return 0; } -static void io_poll_complete(struct io_kiocb *req, __poll_t mask) +static void io_poll_complete(struct io_ring_ctx *ctx, struct io_kiocb *req, + __poll_t mask) { - io_cqring_add_event(req->ctx, req->user_data, mangle_poll(mask), 0); - io_put_req(req); + req->poll.done = true; + io_cqring_fill_event(ctx, req->user_data, mangle_poll(mask), 0); + io_commit_cqring(ctx); + io_cqring_ev_posted(ctx); } static void io_poll_complete_work(struct work_struct *work) @@ -1186,9 +1194,10 @@ static void io_poll_complete_work(struct work_struct *work) return; } list_del_init(&req->list); + io_poll_complete(ctx, req, mask); spin_unlock_irq(&ctx->completion_lock); - io_poll_complete(req, mask); + io_put_req(req); } static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync, @@ -1199,29 +1208,23 @@ static int io_poll_wake(struct wait_queue_entry *wait, unsigned mode, int sync, struct io_kiocb *req = container_of(poll, struct io_kiocb, poll); struct io_ring_ctx *ctx = req->ctx; __poll_t mask = key_to_poll(key); - - poll->woken = true; + unsigned long flags; /* for instances that support it check for an event match first: */ - if (mask) { - unsigned long flags; - - if (!(mask & poll->events)) - return 0; + if (mask && !(mask & poll->events)) + return 0; - /* try to complete the iocb inline if we can: */ - if (spin_trylock_irqsave(&ctx->completion_lock, flags)) { - list_del(&req->list); - spin_unlock_irqrestore(&ctx->completion_lock, flags); + list_del_init(&poll->wait.entry); - list_del_init(&poll->wait.entry); - io_poll_complete(req, mask); - return 1; - } + if (mask && spin_trylock_irqsave(&ctx->completion_lock, flags)) { + list_del(&req->list); + io_poll_complete(ctx, req, mask); + spin_unlock_irqrestore(&ctx->completion_lock, flags); + io_put_req(req); + } else { + queue_work(ctx->sqo_wq, &req->work); } - list_del_init(&poll->wait.entry); - queue_work(ctx->sqo_wq, &req->work); return 1; } @@ -1251,6 +1254,7 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) struct io_poll_iocb *poll = &req->poll; struct io_ring_ctx *ctx = req->ctx; struct io_poll_table ipt; + bool cancel = false; __poll_t mask; u16 events; @@ -1266,7 +1270,7 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) poll->events = demangle_poll(events) | EPOLLERR | EPOLLHUP; poll->head = NULL; - poll->woken = false; + poll->done = false; poll->canceled = false; ipt.pt._qproc = io_poll_queue_proc; @@ -1279,41 +1283,35 @@ static int io_poll_add(struct io_kiocb *req, const struct io_uring_sqe *sqe) init_waitqueue_func_entry(&poll->wait, io_poll_wake); mask = vfs_poll(poll->file, &ipt.pt) & poll->events; - if (unlikely(!poll->head)) { - /* we did not manage to set up a waitqueue, done */ - goto out; - } spin_lock_irq(&ctx->completion_lock); - spin_lock(&poll->head->lock); - if (poll->woken) { - /* wake_up context handles the rest */ - mask = 0; + if (likely(poll->head)) { + spin_lock(&poll->head->lock); + if (unlikely(list_empty(&poll->wait.entry))) { + if (ipt.error) + cancel = true; + ipt.error = 0; + mask = 0; + } + if (mask || ipt.error) + list_del_init(&poll->wait.entry); + else if (cancel) + WRITE_ONCE(poll->canceled, true); + else if (!poll->done) /* actually waiting for an event */ + list_add_tail(&req->list, &ctx->cancel_list); + spin_unlock(&poll->head->lock); + } + if (mask) { /* no async, we'd stolen it */ + req->error = mangle_poll(mask); ipt.error = 0; - } else if (mask || ipt.error) { - /* if we get an error or a mask we are done */ - WARN_ON_ONCE(list_empty(&poll->wait.entry)); - list_del_init(&poll->wait.entry); - } else { - /* actually waiting for an event */ - list_add_tail(&req->list, &ctx->cancel_list); } - spin_unlock(&poll->head->lock); spin_unlock_irq(&ctx->completion_lock); -out: - if (unlikely(ipt.error)) { - /* - * Drop one of our refs to this req, __io_submit_sqe() will - * drop the other one since we're returning an error. - */ + if (mask) { + io_poll_complete(ctx, req, mask); io_put_req(req); - return ipt.error; } - - if (mask) - io_poll_complete(req, mask); - return 0; + return ipt.error; } static int __io_submit_sqe(struct io_ring_ctx *ctx, struct io_kiocb *req, From patchwork Fri Mar 15 14:54:41 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854929 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E3BE71515 for ; Fri, 15 Mar 2019 14:55:01 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D2F542A8BA for ; Fri, 15 Mar 2019 14:55:01 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C765C2A8C2; Fri, 15 Mar 2019 14:55:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BE9AB2A8C4 for ; Fri, 15 Mar 2019 14:55:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729288AbfCOOzA (ORCPT ); Fri, 15 Mar 2019 10:55:00 -0400 Received: from mail-pf1-f194.google.com ([209.85.210.194]:46181 "EHLO mail-pf1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729221AbfCOOy7 (ORCPT ); Fri, 15 Mar 2019 10:54:59 -0400 Received: by mail-pf1-f194.google.com with SMTP id s23so6502069pfe.13 for ; Fri, 15 Mar 2019 07:54:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VMMrvenrcLVFXCMF4rg9hpdiO7zJtE9scP5M1jwx6b8=; b=PtuT3Z8TUppeq3xH3Z04/gx2P25PdU4MJoAaePhh+59uWj+9rO1/NAkiXk7aIuCQ71 UGSez+3E2GAWwchKq2EKJE8c6sYhcdh6CHgtzGqdZr+R1D4+cKyHduvgQ+Seln9hVqX7 QrbvvePmVNO9JOgzZSk3ZawHMXwyfCWJQysrOqUKGjhZONBuMSqaqDnB2jva52k3MTqN PRHoK66Y3AAm8tD6tL+odDrygb59wLY8ga3/zrUWkGNCtxM1dKGoP/VQ00sJ7ZT9pC2g QLjK+VJqab4pFVWHlx9TltT8Ru9T5mPicV5aMg9G//FLZx1GWR3t+G/jDzY5vGkegGuA vGBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VMMrvenrcLVFXCMF4rg9hpdiO7zJtE9scP5M1jwx6b8=; b=krGiG5gBksPFJfW6/Lp9Wv6lX6pefUI7KDcxsgAChvBI/R3ZhFWSkYsO/7obP5XAug cC4EGmVk2v1g733Tvb/wyfYsCkzsKF9TA9AVz7lcvbpgHyYaHdo+wRQ1iXWDo+p6eTs0 bxTla5liMBfLJlF0NzCzxiuNS8jGzJHBkBt85hjxS2Rplwq1k0/CmgZmer6M7uOR9rGQ 91Z/iisOMoOP2rPZb43QmZwBZeZEKqnZ/ziu9P9D0QmWVvXrLsdPDH1svZQp/KKKzezK eCobWOTVD0nT8JYvNIHjwGAmeAwvUWRNH99OaxvmQWZM1G5kYXsJEFeB1uO/kCypJFwJ CaAA== X-Gm-Message-State: APjAAAVvv7zxPjVnnKsl95+Ip10OVfhFuyiH7Jz94r6DGXuVnz6k97wA 5X3Hp9xkIR6v/KggdA73SVJsG3AirOd3XQ== X-Google-Smtp-Source: APXvYqz8WINpv+zU3JcKw7p9D3/Ms0Z5L073GBmQWAfySgTVQAH3Fc3bhnWRbsLvdE3bD6gxyrL//w== X-Received: by 2002:a63:ee55:: with SMTP id n21mr3795224pgk.211.1552661698404; Fri, 15 Mar 2019 07:54:58 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:57 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 6/7] io_uring: retry bulk slab allocs as single allocs Date: Fri, 15 Mar 2019 08:54:41 -0600 Message-Id: <20190315145442.21127-7-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP I've seen cases where bulk alloc fails, since the bulk alloc API is all-or-nothing - either we get the number we ask for, or it returns 0 as number of entries. If we fail a batch bulk alloc, retry a "normal" kmem_cache_alloc() and just use that instead of failing with -EAGAIN. While in there, ensure we use GFP_KERNEL. That was an oversight in the original code, when we switched away from GFP_ATOMIC. Signed-off-by: Jens Axboe --- fs/io_uring.c | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index 443ca5615554..5be6e4f99a9e 100644 --- a/fs/io_uring.c +++ b/fs/io_uring.c @@ -399,13 +399,14 @@ static void io_ring_drop_ctx_refs(struct io_ring_ctx *ctx, unsigned refs) static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx, struct io_submit_state *state) { + gfp_t gfp = GFP_KERNEL | __GFP_NOWARN; struct io_kiocb *req; if (!percpu_ref_tryget(&ctx->refs)) return NULL; if (!state) { - req = kmem_cache_alloc(req_cachep, __GFP_NOWARN); + req = kmem_cache_alloc(req_cachep, gfp); if (unlikely(!req)) goto out; } else if (!state->free_reqs) { @@ -413,13 +414,22 @@ static struct io_kiocb *io_get_req(struct io_ring_ctx *ctx, int ret; sz = min_t(size_t, state->ios_left, ARRAY_SIZE(state->reqs)); - ret = kmem_cache_alloc_bulk(req_cachep, __GFP_NOWARN, sz, - state->reqs); - if (unlikely(ret <= 0)) - goto out; + ret = kmem_cache_alloc_bulk(req_cachep, gfp, sz, state->reqs); + + /* + * Bulk alloc is all-or-nothing. If we fail to get a batch, + * retry single alloc to be on the safe side. + */ + if (ret <= 0) { + req = kmem_cache_alloc(req_cachep, gfp); + if (unlikely(!req)) + goto out; + ret = 1; + } else { + req = state->reqs[0]; + } state->free_reqs = ret - 1; state->cur_req = 1; - req = state->reqs[0]; } else { req = state->reqs[state->cur_req]; state->free_reqs--; From patchwork Fri Mar 15 14:54:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 10854933 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2090A6C2 for ; Fri, 15 Mar 2019 14:55:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0F6772A85F for ; Fri, 15 Mar 2019 14:55:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 037FB2A8BB; Fri, 15 Mar 2019 14:55:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 29BC92A8BA for ; Fri, 15 Mar 2019 14:55:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729299AbfCOOzB (ORCPT ); Fri, 15 Mar 2019 10:55:01 -0400 Received: from mail-pg1-f196.google.com ([209.85.215.196]:38944 "EHLO mail-pg1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729293AbfCOOzB (ORCPT ); Fri, 15 Mar 2019 10:55:01 -0400 Received: by mail-pg1-f196.google.com with SMTP id h8so6642003pgp.6 for ; Fri, 15 Mar 2019 07:55:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PsoNd4sOAZ3OGvoiEniWe7TzVWmF8MVo9Y93Xa4kZvU=; b=Ql5zrRR2FgjjDkjfMLlEH1Z7V3LsZ+jB6ocFhoLlZWhv4w40FQUd0hBURJSCWPsS1C Xyb1qUdaiSt5LKrFtTcUN6vCOeJOP9tgKihDGV7sLpkEOxNMUhbGq0im5GCA1hiWL3nT OSVK6x2piqSboEBOG6HbRVaqWdYM7Hyq42PuaIIDcWUtFfWcdAHTySkkqTs/SueQVKGe eM0ld7JBceO/Dy3U/CVVbUdsc+s8kSbtTqNKiTNrqmdO1KujN5E8cPg6Ebj+z3kq1j3V /CAkcIyuDJgM/pCv84CgEnfHb4HbdhWOTpSYUaGoWjhydxf1iMIx7QLmVk4r3Pp8+eZp AEvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PsoNd4sOAZ3OGvoiEniWe7TzVWmF8MVo9Y93Xa4kZvU=; b=BA+GJ86dhSdYGk0LnKi147koSE6FbEV/qgBhJtqClwLCA9As2w1VIMPFy5oKMiQfQy oBvBDeYzWYEC89i6qumYWs4bIEc26OvhIfCjIGJRWHc2I7XAu7Y5UJy1qwQ+FThNaN+p XHYVoZA6vlkvlfa9MB35iX3UlIiFnCdHjJPcXztt+BYmjpuj2oVbtFAjff9TZbIyi982 9FFfzloVMH6XOdegZsYSA5bEShYU4SVuwuCuZzgtmqhSexjAKWTgDrZ+aMmMJlAXqluq 8rak0vCUz/0Rmx7ciB2jfGh433v587cXE7bYH53c0lK7qoSyuUI9oQ2hTsQ4ZubG9HYS Fv7w== X-Gm-Message-State: APjAAAU/3fdYiiozU7kzhvIw+r6CHpPpwWa36o1cJMPI2mH697qpPfpq M5UHvbYkJKHvBFsBlD9K6DxHUsSjz4IKug== X-Google-Smtp-Source: APXvYqzIRSCyAzhwwzW8n5vrnZpJ8MgzO9AEiwQNaubTrxdyzC2D5/KKS1Xt6aoj/gaQCQM92jMdsg== X-Received: by 2002:a17:902:aa0c:: with SMTP id be12mr4650665plb.63.1552661700028; Fri, 15 Mar 2019 07:55:00 -0700 (PDT) Received: from x1.localdomain (66.29.188.166.static.utbb.net. [66.29.188.166]) by smtp.gmail.com with ESMTPSA id t10sm3192272pgo.27.2019.03.15.07.54.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 15 Mar 2019 07:54:59 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: viro@ZenIV.linux.org.uk, Jens Axboe Subject: [PATCH 7/7] io_uring: mark me as the maintainer Date: Fri, 15 Mar 2019 08:54:42 -0600 Message-Id: <20190315145442.21127-8-axboe@kernel.dk> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190315145442.21127-1-axboe@kernel.dk> References: <20190315145442.21127-1-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP And io_uring as maintained in general. Signed-off-by: Jens Axboe --- MAINTAINERS | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 17b59b66474b..a90137af48c2 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -8091,6 +8091,16 @@ F: include/linux/iommu.h F: include/linux/of_iommu.h F: include/linux/iova.h +IO_URING +M: Jens Axboe +L: linux-block@vger.kernel.org +L: linux-fsdevel@vger.kernel.org +T: git git://git.kernel.dk/linux-block +T: git git://git.kernel.dk/liburing +S: Maintained +F: fs/io_uring.c +F: include/uapi/linux/io_uring.h + IP MASQUERADING M: Juanjo Ciarlante S: Maintained