From patchwork Mon Jul 4 23:13:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC7AEC43334 for ; Mon, 4 Jul 2022 23:14:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E4D996B0071; Mon, 4 Jul 2022 19:14:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DFD8D6B0073; Mon, 4 Jul 2022 19:14:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CECD16B0074; Mon, 4 Jul 2022 19:14:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BE1926B0071 for ; Mon, 4 Jul 2022 19:14:24 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 8616480201 for ; Mon, 4 Jul 2022 23:14:24 +0000 (UTC) X-FDA: 79650973248.20.FB5D4A6 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf16.hostedemail.com (Postfix) with ESMTP id C8C30180007 for ; Mon, 4 Jul 2022 23:14:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=QDUaP7j+G+WPLXwk1QBlWXofxxUo5BaNc7z5+9RX4yU=; b=icrM6bJz3wbQfxMbdA8qlU+e39 fPiSRE5Z8oNpGGsPoLLeaRckneZd3+jP8s8GS7Rwh1qA+2QDR+wqf/HPjTt/fxh42hy6hOkmFhvOX I2DfeLa6S+HKC5+xOPOdPLzM1nJ0X6oAFQ71HahrNq+mfWfIZh8mydyB7YEhw1EPjMxtA4hrg+uEw Ggpnnv2AppA2y4gBlibMSjI9ppUPUlji59C4TBGLOa431ipFR83PbleNOfUSFOfkV8B/RWdoHpNK7 zU0QiG/DFxZWebcck592vkGHauKyBzk+qu1h4p7OtyBjDZjqAWIvM1c3rTEIGgzzOOd/QgDkwKMTo YUaAHrTQ==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VFx-008AVO-KU; Mon, 04 Jul 2022 23:13:29 +0000 Date: Tue, 5 Jul 2022 00:13:29 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 1/7] __follow_mount_rcu(): verify that mount_lock remains unchanged Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976464; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QDUaP7j+G+WPLXwk1QBlWXofxxUo5BaNc7z5+9RX4yU=; b=jCVY1X3TFJfsQ6NB90UBfu1otKfNkblErh90c6QEdSBVAy75LMbi+pUvPTaf1woXq3UBdD L3+waGCU47C9jTty2UEC1FwLDafT/71w+z7uUb+z9WKIR1WOrYZZxj+yhp0EDRTyKulu5c vA7iSNF2qjYSa/a0zOixoybKGyMvTn4= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=icrM6bJz; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf16.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976464; a=rsa-sha256; cv=none; b=wEetG3meQc+udjCg6CByFitTdAm36rpdBMU/VifsET54OX5WVc90xqGFMJKgE+fcnNd2Be 8E9nWyMpdJOvAZQv8DJgn6P+HV93EojS/TTLvg7E9TC8/lsj1LuJ9OlADc5QBObwgHsPFp J4EFIoy7TzZxN5jj/8SMaFgIIA9IO20= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=icrM6bJz; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf16.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: C8C30180007 X-Stat-Signature: ysrtx9tqnenubpend8rceuqoosyixqxw X-Rspam-User: X-HE-Tag: 1656976461-309717 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Validate mount_lock seqcount as soon as we cross into mount in RCU mode. Sure, ->mnt_root is pinned and will remain so until we do rcu_read_unlock() anyway, and we will eventually fail to unlazy if the mount_lock had been touched, but we might run into a hard error (e.g. -ENOENT) before trying to unlazy. And it's possible to end up with RCU pathwalk racing with rename() and umount() in a way that would fail with -ENOENT while non-RCU pathwalk would've succeeded with any timings. Once upon a time we hadn't needed that, but analysis had been subtle, brittle and went out of window as soon as RENAME_EXCHANGE had been added. It's narrow, hard to hit and won't get you anything other than stray -ENOENT that could be arranged in much easier way with the same priveleges, but it's a bug all the same. Cc: stable@kernel.org X-sky-is-falling: unlikely Fixes: da1ce0670c14 "vfs: add cross-rename" Signed-off-by: Al Viro --- fs/namei.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/namei.c b/fs/namei.c index 1f28d3f463c3..4dbf55b37ec6 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1505,6 +1505,8 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, * becoming unpinned. */ flags = dentry->d_flags; + if (read_seqretry(&mount_lock, nd->m_seq)) + return false; continue; } if (read_seqretry(&mount_lock, nd->m_seq)) From patchwork Mon Jul 4 23:14:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C50BC43334 for ; Mon, 4 Jul 2022 23:14:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C04796B0073; Mon, 4 Jul 2022 19:14:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BB3EC6B0074; Mon, 4 Jul 2022 19:14:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7BF06B0075; Mon, 4 Jul 2022 19:14:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9B4076B0073 for ; Mon, 4 Jul 2022 19:14:45 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6FD4D6056B for ; Mon, 4 Jul 2022 23:14:45 +0000 (UTC) X-FDA: 79650974130.30.9C110E7 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf09.hostedemail.com (Postfix) with ESMTP id 18B7014001E for ; Mon, 4 Jul 2022 23:14:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=yVde6OT5Ehv5tuk0crezw61+e6oS2AMOyyAHAkO3zT4=; b=sWOTLQXmpJ5PfJPX0D6RfCVV8H y3xvhrfTU2W/13c3jKVExU64lREaPCNI3kLc87vy8xbN53d6RWOm7HLmvowNV6yO9umwHHEAxLltY g2fm+8ci7srGfVRTbgBwK/yZQXOgy3zEFEDvsGtjfdWGQoLQwIOshkKMw/1Kl9PU/RVxAYHUiWxNZ Mr9lfga8Boj5cohoAiN5fnfAx/unPr6CJUFNRlggKGbOWN085O604eoaLlqaIXnbM5inZ36arEOnz hbuh9jANU1BzNfmrKbsuK7nALZC7ZsGGbtXshIOeCy/+I8XqbB9DmkUwjfoHo6GSSZPD/NgZip9rM leAOLVMw==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VGd-008AW9-Ka; Mon, 04 Jul 2022 23:14:11 +0000 Date: Tue, 5 Jul 2022 00:14:11 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 2/7] follow_dotdot{,_rcu}(): change calling conventions Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976485; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yVde6OT5Ehv5tuk0crezw61+e6oS2AMOyyAHAkO3zT4=; b=bi7YN8Jx5BoiHP5ZMU5TcOtTtttzguKMMoG6dNXILfjlMwp/fKJBB5Y81k0tDYlY+TgCJm RxFW32Oitr1oKesmNq3hF3iuhwko+0zRN1S8yLxuoIBkRXrYZ27lzKlPT2H0k4w7ph6yOm dLbhZsfL+bRgXF/VNNIzyw6BOa3wpYI= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=sWOTLQXm; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf09.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976485; a=rsa-sha256; cv=none; b=IYCYxQZuZp4R3GpIIpuMF58bFtEs/8huSQdoBOYpnyROCvdBjXytFPZ7WcnF3AfPCgOVFP fpijE7KhYWURoUQR1tyZjTbNRAj0fhjk3pfZBMdsQ3agB6eNZmj7/DwBuhBHKESv1dUzYs 1asSixAHJFeq2uRPLVxknQoUIIYnM0E= Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=sWOTLQXm; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf09.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 18B7014001E X-Stat-Signature: aopcwxccnau6hek59p7ksqr5u9qf956b X-Rspam-User: X-HE-Tag: 1656976484-830361 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Instead of returning NULL when we are in root, just make it return the current position (and set *seqp and *inodep accordingly). That collapses the calls of step_into() in handle_dots() Signed-off-by: Al Viro --- fs/namei.c | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index 4dbf55b37ec6..ecdb9ac21ece 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1909,7 +1909,9 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, return ERR_PTR(-ECHILD); if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-ECHILD); - return NULL; + *seqp = nd->seq; + *inodep = nd->path.dentry->d_inode; + return nd->path.dentry; } static struct dentry *follow_dotdot(struct nameidata *nd, @@ -1945,8 +1947,9 @@ static struct dentry *follow_dotdot(struct nameidata *nd, in_root: if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-EXDEV); - dget(nd->path.dentry); - return NULL; + *seqp = 0; + *inodep = nd->path.dentry->d_inode; + return dget(nd->path.dentry); } static const char *handle_dots(struct nameidata *nd, int type) @@ -1968,12 +1971,7 @@ static const char *handle_dots(struct nameidata *nd, int type) parent = follow_dotdot(nd, &inode, &seq); if (IS_ERR(parent)) return ERR_CAST(parent); - if (unlikely(!parent)) - error = step_into(nd, WALK_NOFOLLOW, - nd->path.dentry, nd->inode, nd->seq); - else - error = step_into(nd, WALK_NOFOLLOW, - parent, inode, seq); + error = step_into(nd, WALK_NOFOLLOW, parent, inode, seq); if (unlikely(error)) return error; From patchwork Mon Jul 4 23:14:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6FBAFC43334 for ; Mon, 4 Jul 2022 23:15:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E9C6B6B0074; Mon, 4 Jul 2022 19:15:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E4C1A6B0075; Mon, 4 Jul 2022 19:15:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CEC4F6B0078; Mon, 4 Jul 2022 19:15:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id BCD4A6B0074 for ; Mon, 4 Jul 2022 19:15:09 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 9074734B97 for ; Mon, 4 Jul 2022 23:15:09 +0000 (UTC) X-FDA: 79650975138.15.6CEE592 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf05.hostedemail.com (Postfix) with ESMTP id 0F5EC100008 for ; Mon, 4 Jul 2022 23:15:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=Br5WlNNviFIy/GyDPGw5BoV/8TxCR0XRwVG74vstYTg=; b=OJehKGWkeFwhw7w7SONgVoHhwM q4v+a9IonIgCxktIHz+I5e1T0+Rty5oZloBZrWZhicGetV9rjlw4KtxEFm2kxvr6SGgK++c8jwkn/ hF3/9M4I3yb8yLR6XmrtFrFCylmyH+0yKV9Ikg+AzO5HAx78bm1q/PdFqqzVrHHhxDkuirPCholkR KQ3X33t4oEK7AMHFrYvJ3YFBiCygmU8ppCpEoydfQITd2UCJsIf9NvMeMC8lWRBaN1LsRU9s44mGs iAyJvaVptepQ63q9HrVgBgnFLLBUA43sIB57KYqm0nIRlqkrqjiegK0YEDNyCEiQmY4lPXY4sCc8H KrNgW4Pg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VH6-008AX3-3B; Mon, 04 Jul 2022 23:14:40 +0000 Date: Tue, 5 Jul 2022 00:14:40 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 3/7] namei: stash the sampled ->d_seq into nameidata Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976509; a=rsa-sha256; cv=none; b=wRLGwnayqH5VmfVFiW+sNk6ByBYbchLUTFYXQ2RFKoyp5YeGAmFoHoJLD6Xnzzq+2A90QT 9mhzdWV7QAKNVYcsz6ThMgCtjaVuq27C3TI71A/qGqVU6grqRMPA80rakqExxUOy4D7iKt An715aKLa08aIvRJWLnx4CGGIySeSbQ= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=OJehKGWk; spf=none (imf05.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976509; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Br5WlNNviFIy/GyDPGw5BoV/8TxCR0XRwVG74vstYTg=; b=MgYZFXxFymgzy/XWYUqxCLyltyyO5ZVosBXLmY12DirvPwEHiehmpdgtCCA1aDu6bfpsxk KoQb6YwhTj/8ZcLt26adcwAqeyrsJoKbjRFzULnsFPf4TXh3fIMJ4SC/nI7e5iwJutywOg 4rzKEit+tZYpCUool7M5WoEbOe6kbSs= X-Stat-Signature: 4swigmiqtfbw9fqx1rt9y44s7ab15k3u X-Rspamd-Queue-Id: 0F5EC100008 Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=OJehKGWk; spf=none (imf05.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1656976508-84532 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: New field: nd->next_seq. Set to 0 outside of RCU mode, holds the sampled value for the next dentry to be considered. Used instead of an arseload of local variables, arguments, etc. step_into() has lost seq argument; nd->next_seq is used, so dentry passed to it must be the one ->next_seq is about. There are two requirements for RCU pathwalk: 1) it should not give a hard failure (other than -ECHILD) unless non-RCU pathwalk might fail that way given suitable timings. 2) it should not succeed unless non-RCU pathwalk might succeed with the same end location given suitable timings. The use of seq numbers is the way we achieve that. Invariant we want to maintain is: if RCU pathwalk can reach the state with given nd->path, nd->inode and nd->seq after having traversed some part of pathname, it must be possible for non-RCU pathwalk to reach the same nd->path and nd->inode after having traversed the same part of pathname, and observe the nd->path.dentry->d_seq equal to what RCU pathwalk has in nd->seq For transition from parent to child, we sample child's ->d_seq and verify that parent's ->d_seq remains unchanged. For transitions from child to parent we sample parent's ->d_seq and verify that child's ->d_seq has not changed. For transition from mountpoint to root of mounted we sample the ->d_seq of root and verify that nobody has touched mount_lock since the beginning of pathwalk. That guarantees that mount we'd found had been there all along, with these mountpoint and root of mounted. It would be possible for a non-RCU pathwalk to reach the previous state, find the same mount and observe its root at the moment we'd sampled ->d_seq of that For transitions from root of mounted to mountpoint we sample ->d_seq of mountpoint and verify that mount_lock had not been touched since the beginning of pathwalk. The same reasoning as in the previous case applies. Signed-off-by: Al Viro --- fs/namei.c | 104 ++++++++++++++++++++++++++--------------------------- 1 file changed, 52 insertions(+), 52 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index ecdb9ac21ece..c7c9e88add85 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -567,7 +567,7 @@ struct nameidata { struct path root; struct inode *inode; /* path.dentry.d_inode */ unsigned int flags, state; - unsigned seq, m_seq, r_seq; + unsigned seq, next_seq, m_seq, r_seq; int last_type; unsigned depth; int total_link_count; @@ -772,6 +772,7 @@ static bool try_to_unlazy(struct nameidata *nd) goto out; if (unlikely(!legitimize_root(nd))) goto out; + nd->seq = nd->next_seq = 0; rcu_read_unlock(); BUG_ON(nd->inode != parent->d_inode); return true; @@ -780,6 +781,7 @@ static bool try_to_unlazy(struct nameidata *nd) nd->path.mnt = NULL; nd->path.dentry = NULL; out: + nd->seq = nd->next_seq = 0; rcu_read_unlock(); return false; } @@ -788,7 +790,6 @@ static bool try_to_unlazy(struct nameidata *nd) * try_to_unlazy_next - try to switch to ref-walk mode. * @nd: nameidata pathwalk data * @dentry: next dentry to step into - * @seq: seq number to check @dentry against * Returns: true on success, false on failure * * Similar to try_to_unlazy(), but here we have the next dentry already @@ -797,7 +798,7 @@ static bool try_to_unlazy(struct nameidata *nd) * Nothing should touch nameidata between try_to_unlazy_next() failure and * terminate_walk(). */ -static bool try_to_unlazy_next(struct nameidata *nd, struct dentry *dentry, unsigned seq) +static bool try_to_unlazy_next(struct nameidata *nd, struct dentry *dentry) { BUG_ON(!(nd->flags & LOOKUP_RCU)); @@ -818,7 +819,7 @@ static bool try_to_unlazy_next(struct nameidata *nd, struct dentry *dentry, unsi */ if (unlikely(!lockref_get_not_dead(&dentry->d_lockref))) goto out; - if (unlikely(read_seqcount_retry(&dentry->d_seq, seq))) + if (unlikely(read_seqcount_retry(&dentry->d_seq, nd->next_seq))) goto out_dput; /* * Sequence counts matched. Now make sure that the root is @@ -826,6 +827,7 @@ static bool try_to_unlazy_next(struct nameidata *nd, struct dentry *dentry, unsi */ if (unlikely(!legitimize_root(nd))) goto out_dput; + nd->seq = nd->next_seq = 0; rcu_read_unlock(); return true; @@ -834,9 +836,11 @@ static bool try_to_unlazy_next(struct nameidata *nd, struct dentry *dentry, unsi out1: nd->path.dentry = NULL; out: + nd->seq = nd->next_seq = 0; rcu_read_unlock(); return false; out_dput: + nd->seq = nd->next_seq = 0; rcu_read_unlock(); dput(dentry); return false; @@ -1467,7 +1471,7 @@ EXPORT_SYMBOL(follow_down); * we meet a managed dentry that would need blocking. */ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, - struct inode **inode, unsigned *seqp) + struct inode **inode) { struct dentry *dentry = path->dentry; unsigned int flags = dentry->d_flags; @@ -1496,7 +1500,7 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, path->mnt = &mounted->mnt; dentry = path->dentry = mounted->mnt.mnt_root; nd->state |= ND_JUMPED; - *seqp = read_seqcount_begin(&dentry->d_seq); + nd->next_seq = read_seqcount_begin(&dentry->d_seq); *inode = dentry->d_inode; /* * We don't need to re-check ->d_seq after this @@ -1505,6 +1509,8 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, * becoming unpinned. */ flags = dentry->d_flags; + // makes sure that non-RCU pathwalk could reach + // this state. if (read_seqretry(&mount_lock, nd->m_seq)) return false; continue; @@ -1517,8 +1523,7 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, } static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, - struct path *path, struct inode **inode, - unsigned int *seqp) + struct path *path, struct inode **inode) { bool jumped; int ret; @@ -1526,16 +1531,15 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, path->mnt = nd->path.mnt; path->dentry = dentry; if (nd->flags & LOOKUP_RCU) { - unsigned int seq = *seqp; - if (unlikely(!*inode)) - return -ENOENT; - if (likely(__follow_mount_rcu(nd, path, inode, seqp))) + unsigned int seq = nd->next_seq; + if (likely(__follow_mount_rcu(nd, path, inode))) return 0; - if (!try_to_unlazy_next(nd, dentry, seq)) - return -ECHILD; - // *path might've been clobbered by __follow_mount_rcu() + // *path and nd->next_seq might've been clobbered path->mnt = nd->path.mnt; path->dentry = dentry; + nd->next_seq = seq; + if (!try_to_unlazy_next(nd, dentry)) + return -ECHILD; } ret = traverse_mounts(path, &jumped, &nd->total_link_count, nd->flags); if (jumped) { @@ -1550,7 +1554,6 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, mntput(path->mnt); } else { *inode = d_backing_inode(path->dentry); - *seqp = 0; /* out of RCU mode, so the value doesn't matter */ } return ret; } @@ -1610,8 +1613,7 @@ static struct dentry *__lookup_hash(const struct qstr *name, } static struct dentry *lookup_fast(struct nameidata *nd, - struct inode **inode, - unsigned *seqp) + struct inode **inode) { struct dentry *dentry, *parent = nd->path.dentry; int status = 1; @@ -1622,8 +1624,7 @@ static struct dentry *lookup_fast(struct nameidata *nd, * going to fall back to non-racy lookup. */ if (nd->flags & LOOKUP_RCU) { - unsigned seq; - dentry = __d_lookup_rcu(parent, &nd->last, &seq); + dentry = __d_lookup_rcu(parent, &nd->last, &nd->next_seq); if (unlikely(!dentry)) { if (!try_to_unlazy(nd)) return ERR_PTR(-ECHILD); @@ -1635,7 +1636,7 @@ static struct dentry *lookup_fast(struct nameidata *nd, * the dentry name information from lookup. */ *inode = d_backing_inode(dentry); - if (unlikely(read_seqcount_retry(&dentry->d_seq, seq))) + if (unlikely(read_seqcount_retry(&dentry->d_seq, nd->next_seq))) return ERR_PTR(-ECHILD); /* @@ -1648,11 +1649,10 @@ static struct dentry *lookup_fast(struct nameidata *nd, if (unlikely(__read_seqcount_retry(&parent->d_seq, nd->seq))) return ERR_PTR(-ECHILD); - *seqp = seq; status = d_revalidate(dentry, nd->flags); if (likely(status > 0)) return dentry; - if (!try_to_unlazy_next(nd, dentry, seq)) + if (!try_to_unlazy_next(nd, dentry)) return ERR_PTR(-ECHILD); if (status == -ECHILD) /* we'd been told to redo it in non-rcu mode */ @@ -1733,7 +1733,7 @@ static inline int may_lookup(struct user_namespace *mnt_userns, return inode_permission(mnt_userns, nd->inode, MAY_EXEC); } -static int reserve_stack(struct nameidata *nd, struct path *link, unsigned seq) +static int reserve_stack(struct nameidata *nd, struct path *link) { if (unlikely(nd->total_link_count++ >= MAXSYMLINKS)) return -ELOOP; @@ -1748,7 +1748,7 @@ static int reserve_stack(struct nameidata *nd, struct path *link, unsigned seq) if (nd->flags & LOOKUP_RCU) { // we need to grab link before we do unlazy. And we can't skip // unlazy even if we fail to grab the link - cleanup needs it - bool grabbed_link = legitimize_path(nd, link, seq); + bool grabbed_link = legitimize_path(nd, link, nd->next_seq); if (!try_to_unlazy(nd) || !grabbed_link) return -ECHILD; @@ -1762,11 +1762,11 @@ static int reserve_stack(struct nameidata *nd, struct path *link, unsigned seq) enum {WALK_TRAILING = 1, WALK_MORE = 2, WALK_NOFOLLOW = 4}; static const char *pick_link(struct nameidata *nd, struct path *link, - struct inode *inode, unsigned seq, int flags) + struct inode *inode, int flags) { struct saved *last; const char *res; - int error = reserve_stack(nd, link, seq); + int error = reserve_stack(nd, link); if (unlikely(error)) { if (!(nd->flags & LOOKUP_RCU)) @@ -1776,7 +1776,7 @@ static const char *pick_link(struct nameidata *nd, struct path *link, last = nd->stack + nd->depth++; last->link = *link; clear_delayed_call(&last->done); - last->seq = seq; + last->seq = nd->next_seq; if (flags & WALK_TRAILING) { error = may_follow_link(nd, inode); @@ -1838,12 +1838,14 @@ static const char *pick_link(struct nameidata *nd, struct path *link, * to do this check without having to look at inode->i_op, * so we keep a cache of "no, this doesn't need follow_link" * for the common case. + * + * NOTE: dentry must be what nd->next_seq had been sampled from. */ static const char *step_into(struct nameidata *nd, int flags, - struct dentry *dentry, struct inode *inode, unsigned seq) + struct dentry *dentry, struct inode *inode) { struct path path; - int err = handle_mounts(nd, dentry, &path, &inode, &seq); + int err = handle_mounts(nd, dentry, &path, &inode); if (err < 0) return ERR_PTR(err); @@ -1858,23 +1860,22 @@ static const char *step_into(struct nameidata *nd, int flags, } nd->path = path; nd->inode = inode; - nd->seq = seq; + nd->seq = nd->next_seq; return NULL; } if (nd->flags & LOOKUP_RCU) { /* make sure that d_is_symlink above matches inode */ - if (read_seqcount_retry(&path.dentry->d_seq, seq)) + if (read_seqcount_retry(&path.dentry->d_seq, nd->next_seq)) return ERR_PTR(-ECHILD); } else { if (path.mnt == nd->path.mnt) mntget(path.mnt); } - return pick_link(nd, &path, inode, seq, flags); + return pick_link(nd, &path, inode, flags); } static struct dentry *follow_dotdot_rcu(struct nameidata *nd, - struct inode **inodep, - unsigned *seqp) + struct inode **inodep) { struct dentry *parent, *old; @@ -1891,6 +1892,7 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, nd->path = path; nd->inode = path.dentry->d_inode; nd->seq = seq; + // makes sure that non-RCU pathwalk could reach this state if (unlikely(read_seqretry(&mount_lock, nd->m_seq))) return ERR_PTR(-ECHILD); /* we know that mountpoint was pinned */ @@ -1898,7 +1900,8 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, old = nd->path.dentry; parent = old->d_parent; *inodep = parent->d_inode; - *seqp = read_seqcount_begin(&parent->d_seq); + nd->next_seq = read_seqcount_begin(&parent->d_seq); + // makes sure that non-RCU pathwalk could reach this state if (unlikely(read_seqcount_retry(&old->d_seq, nd->seq))) return ERR_PTR(-ECHILD); if (unlikely(!path_connected(nd->path.mnt, parent))) @@ -1909,14 +1912,13 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, return ERR_PTR(-ECHILD); if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-ECHILD); - *seqp = nd->seq; + nd->next_seq = nd->seq; *inodep = nd->path.dentry->d_inode; return nd->path.dentry; } static struct dentry *follow_dotdot(struct nameidata *nd, - struct inode **inodep, - unsigned *seqp) + struct inode **inodep) { struct dentry *parent; @@ -1940,14 +1942,12 @@ static struct dentry *follow_dotdot(struct nameidata *nd, dput(parent); return ERR_PTR(-ENOENT); } - *seqp = 0; *inodep = parent->d_inode; return parent; in_root: if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-EXDEV); - *seqp = 0; *inodep = nd->path.dentry->d_inode; return dget(nd->path.dentry); } @@ -1958,7 +1958,6 @@ static const char *handle_dots(struct nameidata *nd, int type) const char *error = NULL; struct dentry *parent; struct inode *inode; - unsigned seq; if (!nd->root.mnt) { error = ERR_PTR(set_root(nd)); @@ -1966,12 +1965,12 @@ static const char *handle_dots(struct nameidata *nd, int type) return error; } if (nd->flags & LOOKUP_RCU) - parent = follow_dotdot_rcu(nd, &inode, &seq); + parent = follow_dotdot_rcu(nd, &inode); else - parent = follow_dotdot(nd, &inode, &seq); + parent = follow_dotdot(nd, &inode); if (IS_ERR(parent)) return ERR_CAST(parent); - error = step_into(nd, WALK_NOFOLLOW, parent, inode, seq); + error = step_into(nd, WALK_NOFOLLOW, parent, inode); if (unlikely(error)) return error; @@ -1996,7 +1995,6 @@ static const char *walk_component(struct nameidata *nd, int flags) { struct dentry *dentry; struct inode *inode; - unsigned seq; /* * "." and ".." are special - ".." especially so because it has * to be able to know about the current root directory and @@ -2007,7 +2005,7 @@ static const char *walk_component(struct nameidata *nd, int flags) put_link(nd); return handle_dots(nd, nd->last_type); } - dentry = lookup_fast(nd, &inode, &seq); + dentry = lookup_fast(nd, &inode); if (IS_ERR(dentry)) return ERR_CAST(dentry); if (unlikely(!dentry)) { @@ -2017,7 +2015,7 @@ static const char *walk_component(struct nameidata *nd, int flags) } if (!(flags & WALK_MORE) && nd->depth) put_link(nd); - return step_into(nd, flags, dentry, inode, seq); + return step_into(nd, flags, dentry, inode); } /* @@ -2372,6 +2370,8 @@ static const char *path_init(struct nameidata *nd, unsigned flags) flags &= ~LOOKUP_RCU; if (flags & LOOKUP_RCU) rcu_read_lock(); + else + nd->seq = nd->next_seq = 0; nd->flags = flags; nd->state |= ND_JUMPED; @@ -2473,8 +2473,9 @@ static int handle_lookup_down(struct nameidata *nd) { if (!(nd->flags & LOOKUP_RCU)) dget(nd->path.dentry); + nd->next_seq = nd->seq; return PTR_ERR(step_into(nd, WALK_NOFOLLOW, - nd->path.dentry, nd->inode, nd->seq)); + nd->path.dentry, nd->inode)); } /* Returns 0 and nd will be valid on success; Retuns error, otherwise. */ @@ -3393,7 +3394,6 @@ static const char *open_last_lookups(struct nameidata *nd, struct dentry *dir = nd->path.dentry; int open_flag = op->open_flag; bool got_write = false; - unsigned seq; struct inode *inode; struct dentry *dentry; const char *res; @@ -3410,7 +3410,7 @@ static const char *open_last_lookups(struct nameidata *nd, if (nd->last.name[nd->last.len]) nd->flags |= LOOKUP_FOLLOW | LOOKUP_DIRECTORY; /* we _can_ be in RCU mode here */ - dentry = lookup_fast(nd, &inode, &seq); + dentry = lookup_fast(nd, &inode); if (IS_ERR(dentry)) return ERR_CAST(dentry); if (likely(dentry)) @@ -3464,7 +3464,7 @@ static const char *open_last_lookups(struct nameidata *nd, finish_lookup: if (nd->depth) put_link(nd); - res = step_into(nd, WALK_TRAILING, dentry, inode, seq); + res = step_into(nd, WALK_TRAILING, dentry, inode); if (unlikely(res)) nd->flags &= ~(LOOKUP_OPEN|LOOKUP_CREATE|LOOKUP_EXCL); return res; From patchwork Mon Jul 4 23:15:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905872 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C7D1C43334 for ; Mon, 4 Jul 2022 23:15:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C057C6B0071; Mon, 4 Jul 2022 19:15:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BB4026B0073; Mon, 4 Jul 2022 19:15:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7D656B0075; Mon, 4 Jul 2022 19:15:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9C0886B0071 for ; Mon, 4 Jul 2022 19:15:55 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 7BBB18014D for ; Mon, 4 Jul 2022 23:15:55 +0000 (UTC) X-FDA: 79650977070.07.A83C1DE Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf18.hostedemail.com (Postfix) with ESMTP id 26E531C004F for ; Mon, 4 Jul 2022 23:15:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=phosGs0ReDXhvjah48sBo0KCwl+d1jcc+HCEqeu58tc=; b=OwdNXG7l7nBZtBBbjR/cEwIL4N J37eUfI1FdldQ2JpF8o2dJ2JHAuu3g5AX9ZcC1ukryfycNUEebrOsH9bjzn5Ms14CE5CNkbnEFGqs MYAVKAe6mUxmINf7gP+SuhmQ5NOKLLqGiIJPJwezFX+CUPngnBDRuMi5xYoptjm99AXdxw+ozX7dd RUdGasO87hSmy74nZJTIKnsrM8O5sp4QAVlELB8ReYdLKpNELFQf2zgYT8cv6WZMmQox9iK66Kq9n TgHphg+f7OxwqwIe3eA3NnlxLV5SbKhPv91SC5i4ZlWxYcAa8/WRiJWOWIk4fkjPr0mRjx2KC0bk9 25vU+zrg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VHc-008AYW-Qq; Mon, 04 Jul 2022 23:15:12 +0000 Date: Tue, 5 Jul 2022 00:15:12 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 4/7] step_into(): lose inode argument Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976555; a=rsa-sha256; cv=none; b=m4Qkbgug/W70I1k4BOzak/nxmSfqthjqzdGs5WKuI1vGMGnewUmexRCbDXzwh+9/nSZbIy /4ajtpNqMCeyeBzaeJFvbUh8W4ag5ePEJ+mDWMQC0DGJfH2T/30S322D9HLeK0OYYKUpiY mAYQla9vN9Pc/ONqT1QNsnmIVmwxOSs= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=OwdNXG7l; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf18.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976555; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=phosGs0ReDXhvjah48sBo0KCwl+d1jcc+HCEqeu58tc=; b=z4S2NO2N5QUMglNIvdkrqShCakyqcJ+3UkIogkJGTzO31DcEIuJqp+KxpsyNCFExogW5tJ hU+Pb0uI+Ki8H035ab/FeSGAxE0Pd5Emw3URoijLUQAteHWf/6OL8ebD5aM9PnVqIKf7F/ Kjc0RhUkUTeqWCyZ39Nz7woGXRGOaE8= Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=OwdNXG7l; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf18.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk X-Rspam-User: X-Stat-Signature: owmyfmu7ohrdtkcciat8pu4qffr8op7n X-Rspamd-Queue-Id: 26E531C004F X-Rspamd-Server: rspam04 X-HE-Tag: 1656976555-603249 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: make handle_mounts() always fetch it. This is just the first step - the callers of step_into() will stop trying to calculate the sucker, etc. The passed value should be equal to dentry->d_inode in all cases; in RCU mode - fetched after we'd sampled ->d_seq. Might as well fetch it here. We do need to validate ->d_seq, which duplicates the check currently done in lookup_fast(); that duplication will go away shortly. After that change handle_mounts() always ignores the initial value of *inode and always sets it on success. Signed-off-by: Al Viro --- fs/namei.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index c7c9e88add85..dddbebf92b48 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1532,6 +1532,11 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, path->dentry = dentry; if (nd->flags & LOOKUP_RCU) { unsigned int seq = nd->next_seq; + *inode = dentry->d_inode; + if (read_seqcount_retry(&dentry->d_seq, seq)) + return -ECHILD; + if (unlikely(!*inode)) + return -ENOENT; if (likely(__follow_mount_rcu(nd, path, inode))) return 0; // *path and nd->next_seq might've been clobbered @@ -1842,9 +1847,10 @@ static const char *pick_link(struct nameidata *nd, struct path *link, * NOTE: dentry must be what nd->next_seq had been sampled from. */ static const char *step_into(struct nameidata *nd, int flags, - struct dentry *dentry, struct inode *inode) + struct dentry *dentry) { struct path path; + struct inode *inode; int err = handle_mounts(nd, dentry, &path, &inode); if (err < 0) @@ -1970,7 +1976,7 @@ static const char *handle_dots(struct nameidata *nd, int type) parent = follow_dotdot(nd, &inode); if (IS_ERR(parent)) return ERR_CAST(parent); - error = step_into(nd, WALK_NOFOLLOW, parent, inode); + error = step_into(nd, WALK_NOFOLLOW, parent); if (unlikely(error)) return error; @@ -2015,7 +2021,7 @@ static const char *walk_component(struct nameidata *nd, int flags) } if (!(flags & WALK_MORE) && nd->depth) put_link(nd); - return step_into(nd, flags, dentry, inode); + return step_into(nd, flags, dentry); } /* @@ -2474,8 +2480,7 @@ static int handle_lookup_down(struct nameidata *nd) if (!(nd->flags & LOOKUP_RCU)) dget(nd->path.dentry); nd->next_seq = nd->seq; - return PTR_ERR(step_into(nd, WALK_NOFOLLOW, - nd->path.dentry, nd->inode)); + return PTR_ERR(step_into(nd, WALK_NOFOLLOW, nd->path.dentry)); } /* Returns 0 and nd will be valid on success; Retuns error, otherwise. */ @@ -3464,7 +3469,7 @@ static const char *open_last_lookups(struct nameidata *nd, finish_lookup: if (nd->depth) put_link(nd); - res = step_into(nd, WALK_TRAILING, dentry, inode); + res = step_into(nd, WALK_TRAILING, dentry); if (unlikely(res)) nd->flags &= ~(LOOKUP_OPEN|LOOKUP_CREATE|LOOKUP_EXCL); return res; From patchwork Mon Jul 4 23:15:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905873 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81B21C43334 for ; Mon, 4 Jul 2022 23:16:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 219456B0073; Mon, 4 Jul 2022 19:16:16 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1CA9E6B0074; Mon, 4 Jul 2022 19:16:16 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B8816B0075; Mon, 4 Jul 2022 19:16:16 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id F33C16B0073 for ; Mon, 4 Jul 2022 19:16:15 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C4BF120B1E for ; Mon, 4 Jul 2022 23:16:15 +0000 (UTC) X-FDA: 79650977910.25.9704AD7 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf21.hostedemail.com (Postfix) with ESMTP id 7DC671C0020 for ; Mon, 4 Jul 2022 23:16:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=pcS1G1OlD4As8AY5pL4+nnYUmzJ1Ij+cKL5OnlOGDmU=; b=cKBVkMzKga1zgdYpuIgEZr2KWm zjdiJ7f7CS9ldtxwhWWtw4sihmhQKxN5emZRaBS9jz9oA3R3eb4Fkn0a0W33nox7HdYLnDlUNr/6G Psnqt1jJ8P5QR61stJ3JMV8J8BK6v09kWn53QEem/iwCdtQiK8SOAE0JzlGgKcZhUucgh27Oiu7lz aEeRATnReUfEXNbJCvaw+b5VGNUQFSxoZFQlWyltWhQes+GKvAzBug15ufyERHzOHgL+7gmZiv59Z 2NWA0siXYeeg03JY0lQiaEsGbaLecHbXYmxqaMIr48HJBBZzYvvCg2gGkGNKVjtlMc1NlrTtY5p/c WxOle0Wg==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VI2-008AZ2-Rt; Mon, 04 Jul 2022 23:15:38 +0000 Date: Tue, 5 Jul 2022 00:15:38 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 5/7] follow_dotdot{,_rcu}(): don't bother with inode Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976575; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pcS1G1OlD4As8AY5pL4+nnYUmzJ1Ij+cKL5OnlOGDmU=; b=5FgE46YYyKiQoqrPYytoq6shHGwDItBwbIdeOboS0e4Ey1bjrmaFqmTqgBCaf9k1MocVBp cKsONm6Vh46xBJXocqgnDO6AULJocMcOKmWhnTPt5YOIUnu8vOAHRXgJKieZ3pVunAxHE9 //95wznmo+CUOx33hLPCVPIfsoatvN4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=cKBVkMzK; spf=none (imf21.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976575; a=rsa-sha256; cv=none; b=bWyNMK4bTiuHe8PL3IME0Xw1c5uSFJrApkiCZ7eE5QaEUBmTw06cDhmzMk24t/UMt1L2vK Up2QvYJmjadFnrE7ErPqICcUBegWI3EBMdVvHhlnWHgJVeuu1gNveGmdJZKox/wD2f+ngV 4StLglkIKfaY0b5tukupVqSheJBjSB8= X-Stat-Signature: ownsqer6ehruatoitt8qdkqjpma8na1j X-Rspamd-Queue-Id: 7DC671C0020 Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=cKBVkMzK; spf=none (imf21.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk X-Rspam-User: X-Rspamd-Server: rspam02 X-HE-Tag: 1656976575-358051 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: step_into() will fetch it, TYVM. Signed-off-by: Al Viro --- fs/namei.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index dddbebf92b48..fe95fe39634c 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1880,8 +1880,7 @@ static const char *step_into(struct nameidata *nd, int flags, return pick_link(nd, &path, inode, flags); } -static struct dentry *follow_dotdot_rcu(struct nameidata *nd, - struct inode **inodep) +static struct dentry *follow_dotdot_rcu(struct nameidata *nd) { struct dentry *parent, *old; @@ -1905,7 +1904,6 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, } old = nd->path.dentry; parent = old->d_parent; - *inodep = parent->d_inode; nd->next_seq = read_seqcount_begin(&parent->d_seq); // makes sure that non-RCU pathwalk could reach this state if (unlikely(read_seqcount_retry(&old->d_seq, nd->seq))) @@ -1919,12 +1917,10 @@ static struct dentry *follow_dotdot_rcu(struct nameidata *nd, if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-ECHILD); nd->next_seq = nd->seq; - *inodep = nd->path.dentry->d_inode; return nd->path.dentry; } -static struct dentry *follow_dotdot(struct nameidata *nd, - struct inode **inodep) +static struct dentry *follow_dotdot(struct nameidata *nd) { struct dentry *parent; @@ -1948,13 +1944,11 @@ static struct dentry *follow_dotdot(struct nameidata *nd, dput(parent); return ERR_PTR(-ENOENT); } - *inodep = parent->d_inode; return parent; in_root: if (unlikely(nd->flags & LOOKUP_BENEATH)) return ERR_PTR(-EXDEV); - *inodep = nd->path.dentry->d_inode; return dget(nd->path.dentry); } @@ -1963,7 +1957,6 @@ static const char *handle_dots(struct nameidata *nd, int type) if (type == LAST_DOTDOT) { const char *error = NULL; struct dentry *parent; - struct inode *inode; if (!nd->root.mnt) { error = ERR_PTR(set_root(nd)); @@ -1971,9 +1964,9 @@ static const char *handle_dots(struct nameidata *nd, int type) return error; } if (nd->flags & LOOKUP_RCU) - parent = follow_dotdot_rcu(nd, &inode); + parent = follow_dotdot_rcu(nd); else - parent = follow_dotdot(nd, &inode); + parent = follow_dotdot(nd); if (IS_ERR(parent)) return ERR_CAST(parent); error = step_into(nd, WALK_NOFOLLOW, parent); From patchwork Mon Jul 4 23:16:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B42B8C43334 for ; Mon, 4 Jul 2022 23:17:09 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B5A16B0071; Mon, 4 Jul 2022 19:17:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 266726B0073; Mon, 4 Jul 2022 19:17:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1317C6B0074; Mon, 4 Jul 2022 19:17:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 051706B0071 for ; Mon, 4 Jul 2022 19:17:09 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id CE3E420B65 for ; Mon, 4 Jul 2022 23:17:08 +0000 (UTC) X-FDA: 79650980136.09.8BC4693 Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf20.hostedemail.com (Postfix) with ESMTP id 7A7081C002B for ; Mon, 4 Jul 2022 23:17:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=3Krii5I77Jw+hei/sBgWu8KHpDCvwQjbHpsrcFH98j0=; b=uElzEPYgfvz4HqmD+jumc9o+co 4FBZ1pk6PQNTVkvgkjC3MG/06CMseIjjitUto8/QeQHUXagTnvCAb8W+wHbp1y+0PE25vMAraPymx pBAJ4BMdkPHC4yk0lGril6Ldu8tC5JGT0Ml+TECjtL6OilAzpuMUAyPtb9cJUdoUTEvBCQtpjFquv 0ntqwDI+dNPzFIbuATQVwTTHzP/8gnaQgebhuvBNkx+pXafGftvZA1vS2d0sUaRd03mRrbJ/zdcOl A9gsBQXfx0pOAYR+qkLXUUl3bc7YsHFODJwB0eH4w3dZ3PfPoU5deV+z1cNtvWmo/zpxgRyi7OsVt 1tUxiT6A==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VIs-008AbM-Q3; Mon, 04 Jul 2022 23:16:31 +0000 Date: Tue, 5 Jul 2022 00:16:30 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 6/7] lookup_fast(): don't bother with inode Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976628; a=rsa-sha256; cv=none; b=8mz8lJU9xX8vcE3biwhSTcr71WG4SbKBqdLgaBCVQucAbVLBpUCXTHflO4sq7+XC1K2QN+ pVg9cx66Uw5hF1EZgAqxsSlLdOlblxTSZYd/LzQyW0mwppxfPcBo0ZuZcOg/XtFQMEop5e uR+3t+7ffEirSML3KXxBrxx7Y9zO31g= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=uElzEPYg; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf20.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976628; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3Krii5I77Jw+hei/sBgWu8KHpDCvwQjbHpsrcFH98j0=; b=Dc8I4xiz6iibkl+mEw/cXIsUPwuW4kuj4T5ZRcGoNMAM33QifkarVuYR78sE8cv0e8iyW2 AeGW32iVkOw7CO7aZynKX8gJiA9B7aDF0SHX/R48+MkgU/NNZbptUISJbzNqLgKPnbgJ2w VgmuhNJE88iwAk65s3kfT0noeYEbeLU= Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=uElzEPYg; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf20.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 7A7081C002B X-Rspam-User: X-Stat-Signature: p6kfc5mnixwpmxbtjqqjmq3uywchah8m X-HE-Tag: 1656976628-625557 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Note that validation of ->d_seq after ->d_inode fetch is gone, along with fetching of ->d_inode itself. Signed-off-by: Al Viro --- fs/namei.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index fe95fe39634c..cdb61d09df79 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1617,8 +1617,7 @@ static struct dentry *__lookup_hash(const struct qstr *name, return dentry; } -static struct dentry *lookup_fast(struct nameidata *nd, - struct inode **inode) +static struct dentry *lookup_fast(struct nameidata *nd) { struct dentry *dentry, *parent = nd->path.dentry; int status = 1; @@ -1636,22 +1635,11 @@ static struct dentry *lookup_fast(struct nameidata *nd, return NULL; } - /* - * This sequence count validates that the inode matches - * the dentry name information from lookup. - */ - *inode = d_backing_inode(dentry); - if (unlikely(read_seqcount_retry(&dentry->d_seq, nd->next_seq))) - return ERR_PTR(-ECHILD); - - /* + /* * This sequence count validates that the parent had no * changes while we did the lookup of the dentry above. - * - * The memory barrier in read_seqcount_begin of child is - * enough, we can use __read_seqcount_retry here. */ - if (unlikely(__read_seqcount_retry(&parent->d_seq, nd->seq))) + if (unlikely(read_seqcount_retry(&parent->d_seq, nd->seq))) return ERR_PTR(-ECHILD); status = d_revalidate(dentry, nd->flags); @@ -1993,7 +1981,6 @@ static const char *handle_dots(struct nameidata *nd, int type) static const char *walk_component(struct nameidata *nd, int flags) { struct dentry *dentry; - struct inode *inode; /* * "." and ".." are special - ".." especially so because it has * to be able to know about the current root directory and @@ -2004,7 +1991,7 @@ static const char *walk_component(struct nameidata *nd, int flags) put_link(nd); return handle_dots(nd, nd->last_type); } - dentry = lookup_fast(nd, &inode); + dentry = lookup_fast(nd); if (IS_ERR(dentry)) return ERR_CAST(dentry); if (unlikely(!dentry)) { @@ -3392,7 +3379,6 @@ static const char *open_last_lookups(struct nameidata *nd, struct dentry *dir = nd->path.dentry; int open_flag = op->open_flag; bool got_write = false; - struct inode *inode; struct dentry *dentry; const char *res; @@ -3408,7 +3394,7 @@ static const char *open_last_lookups(struct nameidata *nd, if (nd->last.name[nd->last.len]) nd->flags |= LOOKUP_FOLLOW | LOOKUP_DIRECTORY; /* we _can_ be in RCU mode here */ - dentry = lookup_fast(nd, &inode); + dentry = lookup_fast(nd); if (IS_ERR(dentry)) return ERR_CAST(dentry); if (likely(dentry)) From patchwork Mon Jul 4 23:17:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Al Viro X-Patchwork-Id: 12905875 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF0C1C43334 for ; Mon, 4 Jul 2022 23:17:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FBD96B0074; Mon, 4 Jul 2022 19:17:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6ABCD6B0075; Mon, 4 Jul 2022 19:17:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 59B366B0078; Mon, 4 Jul 2022 19:17:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 4C0046B0074 for ; Mon, 4 Jul 2022 19:17:40 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 1932F34FD6 for ; Mon, 4 Jul 2022 23:17:40 +0000 (UTC) X-FDA: 79650981480.13.0941E0E Received: from zeniv.linux.org.uk (zeniv.linux.org.uk [62.89.141.173]) by imf03.hostedemail.com (Postfix) with ESMTP id B720620003 for ; Mon, 4 Jul 2022 23:17:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=linux.org.uk; s=zeniv-20220401; h=Sender:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=NqVApc9I2DDH/ifQWSRH221HRDI5UWBJClZrnVtHcvg=; b=XMq6en3D1K42NPJrNPt3lNiGHU 6m4VZEYHJsKw2KJ8FEER0qK8oNGCVOWcuDgjwh8WQDSnCv3uQR6L+1i3enPxKWoWnbVOH3z6CEie3 nAp9cJJDd2p+yBlssUPpEpXlwKJbc2p27ShaRo5hao7QkcNFe3prUfJ6W5u0gQWZQnbKrS5pR1Tdf egeY54C3ZyVr7W9Zdixd+0O1ckAB19QGVGrBNLoZWZcqmBVvuhjabPDBCUO+bxM+6STEGYDHx9FyS toKCReTROXqjfeOSHKWpf/dSfOftFLkoYOpfUHq4w3tP7uXrpyQGmCSK5nAd/yJzARh/75JRzMNP4 bDBFQeCA==; Received: from viro by zeniv.linux.org.uk with local (Exim 4.95 #2 (Red Hat Linux)) id 1o8VJP-008AcR-HB; Mon, 04 Jul 2022 23:17:03 +0000 Date: Tue, 5 Jul 2022 00:17:03 +0100 From: Al Viro To: Linus Torvalds Cc: Alexander Potapenko , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Greg Kroah-Hartman , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Marco Elver , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux-MM , linux-arch , Linux Kernel Mailing List , Evgenii Stepanov , Nathan Chancellor , Nick Desaulniers , Segher Boessenkool , Vitaly Buka , linux-toolchains Subject: [PATCH 7/7] step_into(): move fetching ->d_inode past handle_mounts() Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=XMq6en3D; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf03.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1656976659; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NqVApc9I2DDH/ifQWSRH221HRDI5UWBJClZrnVtHcvg=; b=E4xo7TLcmOZ13U0bs98MFffi8IexFN8V7/b70O8L3g2yem+IqosehqoP7wAySOaTE575bn bm6cntSPeRuQ/cbbwMcK6wppAoQjZ075Y101oKgcG0ZVj2yr6ZQ0QabR5m+rIeSVPpJnJs ov82jsjwONwzQvAlCJqbCKmt3JqrL/o= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1656976659; a=rsa-sha256; cv=none; b=PE7mgfShWL1MsMhvxdAOSPQkNsiJTk31+udVPq73O4acYiVtxA+As335X19W4la9ACjRNy 7+OVSi3nS0+cfOkXMbne8T4Lwke34SNbS+ILT5QK1voAtSp/4qANZn0d1yh7rogX9gGk3h HnU28N3zqhqUvO7M4DTSBmNOY7VZSw8= X-Stat-Signature: 65acpwez3n1r59qg1doyfk55coy5cqas X-Rspamd-Queue-Id: B720620003 Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=linux.org.uk header.s=zeniv-20220401 header.b=XMq6en3D; dmarc=pass (policy=none) header.from=zeniv.linux.org.uk; spf=none (imf03.hostedemail.com: domain of viro@ftp.linux.org.uk has no SPF policy when checking 62.89.141.173) smtp.mailfrom=viro@ftp.linux.org.uk X-Rspamd-Server: rspam03 X-Rspam-User: X-HE-Tag: 1656976659-326747 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: ... and lose messing with it in __follow_mount_rcu() Signed-off-by: Al Viro --- fs/namei.c | 31 +++++++++++-------------------- 1 file changed, 11 insertions(+), 20 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index cdb61d09df79..f2c99e75b578 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1470,8 +1470,7 @@ EXPORT_SYMBOL(follow_down); * Try to skip to top of mountpoint pile in rcuwalk mode. Fail if * we meet a managed dentry that would need blocking. */ -static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, - struct inode **inode) +static bool __follow_mount_rcu(struct nameidata *nd, struct path *path) { struct dentry *dentry = path->dentry; unsigned int flags = dentry->d_flags; @@ -1501,13 +1500,6 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, dentry = path->dentry = mounted->mnt.mnt_root; nd->state |= ND_JUMPED; nd->next_seq = read_seqcount_begin(&dentry->d_seq); - *inode = dentry->d_inode; - /* - * We don't need to re-check ->d_seq after this - * ->d_inode read - there will be an RCU delay - * between mount hash removal and ->mnt_root - * becoming unpinned. - */ flags = dentry->d_flags; // makes sure that non-RCU pathwalk could reach // this state. @@ -1523,7 +1515,7 @@ static bool __follow_mount_rcu(struct nameidata *nd, struct path *path, } static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, - struct path *path, struct inode **inode) + struct path *path) { bool jumped; int ret; @@ -1532,12 +1524,7 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, path->dentry = dentry; if (nd->flags & LOOKUP_RCU) { unsigned int seq = nd->next_seq; - *inode = dentry->d_inode; - if (read_seqcount_retry(&dentry->d_seq, seq)) - return -ECHILD; - if (unlikely(!*inode)) - return -ENOENT; - if (likely(__follow_mount_rcu(nd, path, inode))) + if (likely(__follow_mount_rcu(nd, path))) return 0; // *path and nd->next_seq might've been clobbered path->mnt = nd->path.mnt; @@ -1557,8 +1544,6 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, dput(path->dentry); if (path->mnt != nd->path.mnt) mntput(path->mnt); - } else { - *inode = d_backing_inode(path->dentry); } return ret; } @@ -1839,15 +1824,21 @@ static const char *step_into(struct nameidata *nd, int flags, { struct path path; struct inode *inode; - int err = handle_mounts(nd, dentry, &path, &inode); + int err = handle_mounts(nd, dentry, &path); if (err < 0) return ERR_PTR(err); + inode = path.dentry->d_inode; if (likely(!d_is_symlink(path.dentry)) || ((flags & WALK_TRAILING) && !(nd->flags & LOOKUP_FOLLOW)) || (flags & WALK_NOFOLLOW)) { /* not a symlink or should not follow */ - if (!(nd->flags & LOOKUP_RCU)) { + if (nd->flags & LOOKUP_RCU) { + if (read_seqcount_retry(&path.dentry->d_seq, nd->next_seq)) + return ERR_PTR(-ECHILD); + if (unlikely(!inode)) + return ERR_PTR(-ENOENT); + } else { dput(nd->path.dentry); if (nd->path.mnt != path.mnt) mntput(nd->path.mnt);