[RFC,v4,46/69] move handle_dots(), follow_dotdot() and follow_dotdot_rcu() past step_into()

Message ID	20200313235357.2646756-46-viro@ZenIV.linux.org.uk (mailing list archive)
State	New, archived
Headers	show Return-Path: <SRS0=ymZa=46=vger.kernel.org=linux-fsdevel-owner@kernel.org> From: Al Viro <viro@ZenIV.linux.org.uk> To: linux-fsdevel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, Linus Torvalds <torvalds@linux-foundation.org> Subject: [RFC][PATCH v4 46/69] move handle_dots(), follow_dotdot() and follow_dotdot_rcu() past step_into() Date: Fri, 13 Mar 2020 23:53:34 +0000 Message-Id: <20200313235357.2646756-46-viro@ZenIV.linux.org.uk> In-Reply-To: <20200313235357.2646756-1-viro@ZenIV.linux.org.uk> References: <20200313235303.GP23230@ZenIV.linux.org.uk> <20200313235357.2646756-1-viro@ZenIV.linux.org.uk> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk
Series	[RFC,v4,01/69] do_add_mount(): lift lock_mount/unlock_mount into callers \| expand [RFC,v4,01/69] do_add_mount(): lift lock_mount/unlock_mount into callers [RFC,v4,02/69] fix automount/automount race properly [RFC,v4,03/69] follow_automount(): get rid of dead^Wstillborn code [RFC,v4,04/69] follow_automount() doesn't need the entire nameidata [RFC,v4,05/69] make build_open_flags() treat O_CREAT \| O_EXCL as implying O_NOFOLLOW [RFC,v4,06/69] handle_mounts(): start building a sane wrapper for follow_managed() [RFC,v4,07/69] atomic_open(): saner calling conventions (return dentry on success) [RFC,v4,08/69] lookup_open(): saner calling conventions (return dentry on success) [RFC,v4,09/69] do_last(): collapse the call of path_to_nameidata() [RFC,v4,10/69] handle_mounts(): pass dentry in, turn path into a pure out argument [RFC,v4,11/69] lookup_fast(): consolidate the RCU success case [RFC,v4,12/69] teach handle_mounts() to handle RCU mode [RFC,v4,13/69] lookup_fast(): take mount traversal into callers [RFC,v4,14/69] step_into() callers: dismiss the symlink earlier [RFC,v4,15/69] new step_into() flag: WALK_NOFOLLOW [RFC,v4,16/69] fold handle_mounts() into step_into() [RFC,v4,17/69] LOOKUP_MOUNTPOINT: fold path_mountpointat() into path_lookupat() [RFC,v4,18/69] expand the only remaining call of path_lookup_conditional() [RFC,v4,19/69] merging pick_link() with get_link(), part 1 [RFC,v4,20/69] merging pick_link() with get_link(), part 2 [RFC,v4,21/69] merging pick_link() with get_link(), part 3 [RFC,v4,22/69] merging pick_link() with get_link(), part 4 [RFC,v4,23/69] merging pick_link() with get_link(), part 5 [RFC,v4,24/69] merging pick_link() with get_link(), part 6 [RFC,v4,25/69] finally fold get_link() into pick_link() [RFC,v4,26/69] sanitize handling of nd->last_type, kill LAST_BIND [RFC,v4,27/69] namei: invert the meaning of WALK_FOLLOW [RFC,v4,28/69] pick_link(): check for WALK_TRAILING, not LOOKUP_PARENT [RFC,v4,29/69] link_path_walk(): simplify stack handling [RFC,v4,30/69] namei: have link_path_walk() maintain LOOKUP_PARENT [RFC,v4,31/69] massage __follow_mount_rcu() a bit [RFC,v4,32/69] new helper: traverse_mounts() [RFC,v4,33/69] atomic_open(): return the right dentry in FMODE_OPENED case [RFC,v4,34/69] atomic_open(): lift the call of may_open() into do_last() [RFC,v4,35/69] do_last(): merge the may_open() calls [RFC,v4,36/69] do_last(): don't bother with keeping got_write in FMODE_OPENED case [RFC,v4,37/69] do_last(): rejoing the common path earlier in FMODE_{OPENED,CREATED} case [RFC,v4,38/69] do_last(): simplify the liveness analysis past finish_open_created [RFC,v4,39/69] do_last(): rejoin the common path even earlier in FMODE_{OPENED,CREATED} case [RFC,v4,40/69] split the lookup-related parts of do_last() into a separate helper [RFC,v4,41/69] path_connected(): pass mount and dentry separately [RFC,v4,42/69] path_parent_directory(): leave changing path->dentry to callers [RFC,v4,43/69] expand path_parent_directory() in its callers [RFC,v4,44/69] follow_dotdot{,_rcu}(): lift switching nd->path to parent out of loop [RFC,v4,45/69] follow_dotdot{,_rcu}(): lift LOOKUP_BENEATH checks out of loop [RFC,v4,46/69] move handle_dots(), follow_dotdot() and follow_dotdot_rcu() past step_into() [RFC,v4,47/69] handle_dots(), follow_dotdot{,_rcu}(): preparation to switch to step_into() [RFC,v4,48/69] follow_dotdot{,_rcu}(): switch to use of step_into() [RFC,v4,49/69] lift all calls of step_into() out of follow_dotdot/follow_dotdot_rcu [RFC,v4,50/69] follow_dotdot{,_rcu}(): massage loops [RFC,v4,51/69] follow_dotdot_rcu(): be lazy about changing nd->path [RFC,v4,52/69] follow_dotdot(): be lazy about changing nd->path [RFC,v4,53/69] helper for mount rootwards traversal [RFC,v4,54/69] non-RCU analogue of the previous commit [RFC,v4,55/69] fs/namei.c: kill follow_mount() [RFC,v4,56/69] pick_link(): more straightforward handling of allocation failures [RFC,v4,57/69] pick_link(): pass it struct path already with normal refcounting rules [RFC,v4,58/69] fold path_to_nameidata() into its only remaining caller [RFC,v4,59/69] pick_link(): take reserving space on stack into a new helper [RFC,v4,60/69] reserve_stack(): switch to __nd_alloc_stack() [RFC,v4,61/69] __nd_alloc_stack(): make it return bool [RFC,v4,62/69] link_path_walk(): sample parent's i_uid and i_mode for the last component [RFC,v4,63/69] take post-lookup part of do_last() out of loop [RFC,v4,64/69] open_last_lookups(): consolidate fsnotify_create() calls [RFC,v4,65/69] open_last_lookups(): don't abuse complete_walk() when all we want is unlazy [RFC,v4,66/69] open_last_lookups(): lift O_EXCL\|O_CREAT handling into do_open() [RFC,v4,67/69] open_last_lookups(): move complete_walk() into do_open() [RFC,v4,68/69] atomic_open(): no need to pass struct open_flags anymore [RFC,v4,69/69] lookup_open(): don't bother with fallbacks to lookup+create

diff --git a/fs/namei.c b/fs/namei.c index 577dc541a4d4..3ee3b8719505 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -1363,70 +1363,6 @@ static inline int handle_mounts(struct nameidata *nd, struct dentry *dentry, return ret; } -static int follow_dotdot_rcu(struct nameidata *nd) -{ - struct dentry *parent = NULL; - struct inode *inode = nd->inode; - unsigned seq; - - while (1) { - if (path_equal(&nd->path, &nd->root)) - break; - if (nd->path.dentry != nd->path.mnt->mnt_root) { - struct dentry *old = nd->path.dentry; - - parent = old->d_parent; - inode = parent->d_inode; - seq = read_seqcount_begin(&parent->d_seq); - if (unlikely(read_seqcount_retry(&old->d_seq, nd->seq))) - return -ECHILD; - if (unlikely(!path_connected(nd->path.mnt, parent))) - return -ECHILD; - break; - } else { - struct mount *mnt = real_mount(nd->path.mnt); - struct mount *mparent = mnt->mnt_parent; - struct dentry *mountpoint = mnt->mnt_mountpoint; - struct inode *inode2 = mountpoint->d_inode; - unsigned seq = read_seqcount_begin(&mountpoint->d_seq); - if (unlikely(read_seqretry(&mount_lock, nd->m_seq))) - return -ECHILD; - if (&mparent->mnt == nd->path.mnt) - break; - if (unlikely(nd->flags & LOOKUP_NO_XDEV)) - return -ECHILD; - /* we know that mountpoint was pinned */ - nd->path.dentry = mountpoint; - nd->path.mnt = &mparent->mnt; - inode = inode2; - nd->seq = seq; - } - } - if (unlikely(!parent)) { - if (unlikely(nd->flags & LOOKUP_BENEATH)) - return -ECHILD; - } else { - nd->path.dentry = parent; - nd->seq = seq; - } - while (unlikely(d_mountpoint(nd->path.dentry))) { - struct mount *mounted; - mounted = __lookup_mnt(nd->path.mnt, nd->path.dentry); - if (unlikely(read_seqretry(&mount_lock, nd->m_seq))) - return -ECHILD; - if (!mounted) - break; - if (unlikely(nd->flags & LOOKUP_NO_XDEV)) - return -ECHILD; - nd->path.mnt = &mounted->mnt; - nd->path.dentry = mounted->mnt.mnt_root; - inode = nd->path.dentry->d_inode; - nd->seq = read_seqcount_begin(&nd->path.dentry->d_seq); - } - nd->inode = inode; - return 0; -} - /* * Skip to top of mountpoint pile in refwalk mode for follow_dotdot() */ @@ -1443,38 +1379,6 @@ static void follow_mount(struct path *path) } } -static int follow_dotdot(struct nameidata *nd) -{ - struct dentry *parent = NULL; - while (1) { - if (path_equal(&nd->path, &nd->root)) - break; - if (nd->path.dentry != nd->path.mnt->mnt_root) { - /* rare case of legitimate dget_parent()... */ - parent = dget_parent(nd->path.dentry); - if (unlikely(!path_connected(nd->path.mnt, parent))) { - dput(parent); - return -ENOENT; - } - break; - } - if (!follow_up(&nd->path)) - break; - if (unlikely(nd->flags & LOOKUP_NO_XDEV)) - return -EXDEV; - } - if (unlikely(!parent)) { - if (unlikely(nd->flags & LOOKUP_BENEATH)) - return -EXDEV; - } else { - dput(nd->path.dentry); - nd->path.dentry = parent; - } - follow_mount(&nd->path); - nd->inode = nd->path.dentry->d_inode; - return 0; -} - /* * This looks up the name in dcache and possibly revalidates the found dentry. * NULL is returned if the dentry does not exist in the cache. @@ -1654,40 +1558,6 @@ static inline int may_lookup(struct nameidata *nd) return inode_permission(nd->inode, MAY_EXEC); } -static inline int handle_dots(struct nameidata *nd, int type) -{ - if (type == LAST_DOTDOT) { - int error = 0; - - if (!nd->root.mnt) { - error = set_root(nd); - if (error) - return error; - } - if (nd->flags & LOOKUP_RCU) - error = follow_dotdot_rcu(nd); - else - error = follow_dotdot(nd); - if (error) - return error; - - if (unlikely(nd->flags & LOOKUP_IS_SCOPED)) { - /* - * If there was a racing rename or mount along our - * path, then we can't be sure that ".." hasn't jumped - * above nd->root (and so userspace should retry or use - * some fallback). - */ - smp_rmb(); - if (unlikely(__read_seqcount_retry(&mount_lock.seqcount, nd->m_seq))) - return -EAGAIN; - if (unlikely(__read_seqcount_retry(&rename_lock.seqcount, nd->r_seq))) - return -EAGAIN; - } - } - return 0; -} - enum {WALK_TRAILING = 1, WALK_MORE = 2, WALK_NOFOLLOW = 4}; static const char *pick_link(struct nameidata *nd, struct path *link, @@ -1817,6 +1687,136 @@ static const char *step_into(struct nameidata *nd, int flags, return pick_link(nd, &path, inode, seq, flags); } +static int follow_dotdot_rcu(struct nameidata *nd) +{ + struct dentry *parent = NULL; + struct inode *inode = nd->inode; + unsigned seq; + + while (1) { + if (path_equal(&nd->path, &nd->root)) + break; + if (nd->path.dentry != nd->path.mnt->mnt_root) { + struct dentry *old = nd->path.dentry; + + parent = old->d_parent; + inode = parent->d_inode; + seq = read_seqcount_begin(&parent->d_seq); + if (unlikely(read_seqcount_retry(&old->d_seq, nd->seq))) + return -ECHILD; + if (unlikely(!path_connected(nd->path.mnt, parent))) + return -ECHILD; + break; + } else { + struct mount *mnt = real_mount(nd->path.mnt); + struct mount *mparent = mnt->mnt_parent; + struct dentry *mountpoint = mnt->mnt_mountpoint; + struct inode *inode2 = mountpoint->d_inode; + unsigned seq = read_seqcount_begin(&mountpoint->d_seq); + if (unlikely(read_seqretry(&mount_lock, nd->m_seq))) + return -ECHILD; + if (&mparent->mnt == nd->path.mnt) + break; + if (unlikely(nd->flags & LOOKUP_NO_XDEV)) + return -ECHILD; + /* we know that mountpoint was pinned */ + nd->path.dentry = mountpoint; + nd->path.mnt = &mparent->mnt; + inode = inode2; + nd->seq = seq; + } + } + if (unlikely(!parent)) { + if (unlikely(nd->flags & LOOKUP_BENEATH)) + return -ECHILD; + } else { + nd->path.dentry = parent; + nd->seq = seq; + } + while (unlikely(d_mountpoint(nd->path.dentry))) { + struct mount *mounted; + mounted = __lookup_mnt(nd->path.mnt, nd->path.dentry); + if (unlikely(read_seqretry(&mount_lock, nd->m_seq))) + return -ECHILD; + if (!mounted) + break; + if (unlikely(nd->flags & LOOKUP_NO_XDEV)) + return -ECHILD; + nd->path.mnt = &mounted->mnt; + nd->path.dentry = mounted->mnt.mnt_root; + inode = nd->path.dentry->d_inode; + nd->seq = read_seqcount_begin(&nd->path.dentry->d_seq); + } + nd->inode = inode; + return 0; +} + +static int follow_dotdot(struct nameidata *nd) +{ + struct dentry *parent = NULL; + while (1) { + if (path_equal(&nd->path, &nd->root)) + break; + if (nd->path.dentry != nd->path.mnt->mnt_root) { + /* rare case of legitimate dget_parent()... */ + parent = dget_parent(nd->path.dentry); + if (unlikely(!path_connected(nd->path.mnt, parent))) { + dput(parent); + return -ENOENT; + } + break; + } + if (!follow_up(&nd->path)) + break; + if (unlikely(nd->flags & LOOKUP_NO_XDEV)) + return -EXDEV; + } + if (unlikely(!parent)) { + if (unlikely(nd->flags & LOOKUP_BENEATH)) + return -EXDEV; + } else { + dput(nd->path.dentry); + nd->path.dentry = parent; + } + follow_mount(&nd->path); + nd->inode = nd->path.dentry->d_inode; + return 0; +} + +static inline int handle_dots(struct nameidata *nd, int type) +{ + if (type == LAST_DOTDOT) { + int error = 0; + + if (!nd->root.mnt) { + error = set_root(nd); + if (error) + return error; + } + if (nd->flags & LOOKUP_RCU) + error = follow_dotdot_rcu(nd); + else + error = follow_dotdot(nd); + if (error) + return error; + + if (unlikely(nd->flags & LOOKUP_IS_SCOPED)) { + /* + * If there was a racing rename or mount along our + * path, then we can't be sure that ".." hasn't jumped + * above nd->root (and so userspace should retry or use + * some fallback). + */ + smp_rmb(); + if (unlikely(__read_seqcount_retry(&mount_lock.seqcount, nd->m_seq))) + return -EAGAIN; + if (unlikely(__read_seqcount_retry(&rename_lock.seqcount, nd->r_seq))) + return -EAGAIN; + } + } + return 0; +} + static const char *walk_component(struct nameidata *nd, int flags) { struct dentry *dentry;

[RFC,v4,46/69] move handle_dots(), follow_dotdot() and follow_dotdot_rcu() past step_into()

Commit Message

Patch