From patchwork Wed Nov 20 11:20:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 13881064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8682ED6392E for ; Wed, 20 Nov 2024 11:20:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19E856B0085; Wed, 20 Nov 2024 06:20:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1277E6B0088; Wed, 20 Nov 2024 06:20:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E6BE96B0089; Wed, 20 Nov 2024 06:20:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id C029E6B0085 for ; Wed, 20 Nov 2024 06:20:52 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 738BF80AFF for ; Wed, 20 Nov 2024 11:20:52 +0000 (UTC) X-FDA: 82806227532.17.E0FE82A Received: from mail-ed1-f54.google.com (mail-ed1-f54.google.com [209.85.208.54]) by imf29.hostedemail.com (Postfix) with ESMTP id F3E7B120013 for ; Wed, 20 Nov 2024 11:19:40 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jQR0DE7Y; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732101465; a=rsa-sha256; cv=none; b=Hqxh8a5o9Br/AaVnmn7VVZEq2p4EMQkgW11z7ElRm/xVZv8RFA41+T0GUjdxeNTo0Smjge uvWbk6dFy9KBIIU9EvouZ3y/luq+GKci14W4Y4hKi5/0pI8KQmZ0iTAVAQXiPg3e6T4gvX /9MXBz0OD5GESgApbcaqwfhr8KMNIKI= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=jQR0DE7Y; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf29.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.208.54 as permitted sender) smtp.mailfrom=mjguzik@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732101465; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vUaKEruejfDCGRJdpj1eBL3gJs8S2bH9vNecg7yvIhA=; b=FR+2nnQ6R+s4bhPEX1jYE7UqKkUl3agslqgr9K1n9LlpXsEG/EEJqUYOljTzpHGlOMTfOs JTGAuGCs0pH801tZOxdMSKPfbG3j3SmP66mNwMeUyqKWXm9VUmSvDR1EVa7ePVmnSQYWZ7 BaShMxLAIQGA8F9o11Cgn5/xDytm72c= Received: by mail-ed1-f54.google.com with SMTP id 4fb4d7f45d1cf-5cedf5fe237so5233212a12.3 for ; Wed, 20 Nov 2024 03:20:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732101649; x=1732706449; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vUaKEruejfDCGRJdpj1eBL3gJs8S2bH9vNecg7yvIhA=; b=jQR0DE7YLaL8Jo4SAUoJ0JLhaH0PHa6VvXiM2YL052fCHZlD4Y79FwTJKVZv1sBs8o Nfl2q2X6QA6CxIy0AI547yh5VFK6FNhnF3VMGtHD3OJAIrEP17U2gU8J1MyCN6CCAz6Y VHfA5qtr6E3zVq4eZHLtqaD5hQxfN+0Ue2Q74dymUNwXZQq7ZnpXWX0sbM/UR8T5/QDi 10XsgGg0hEJoTeO7BRlne79flXRTStaVktxdZQLsEfDNk/6duo1TLOs1aOJo1w/jPd8d jz9Tb3VZw1VloDNo+Yj77yoiiXthv4pQqfGhh1lorbSJXMhWFAy9vpSnCPTyjMuSFd/f l+Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732101649; x=1732706449; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vUaKEruejfDCGRJdpj1eBL3gJs8S2bH9vNecg7yvIhA=; b=bRmw7WUBM4ulrQ5XLLJUSWOWdeguMin7ZhNWfVVGKZTAvE1HnwZXuubtG/qnIwvMMA 8L8eigzzcqJ53YuUYfysDjXNYcszWAh4EM9KaUy6pL78wTNM6tGEHC00PZ8SOSYjBLb/ yrtg2mQsl+DkgIsXGvO7uVE5q4BxkcxWVHCvdrZG6SfQLd4neKfy7paHAEtfHVoxQgGD q9oblqvRhSW2/nV/HoSEAJJlBrqxih55PhgA99/4d3n5m7evkKYz2aSflMKTqiWYdPmY qQJF/pxaOROCQ/Ey4wKxErbVFDbcSSTXrZBI3cvLHgtCK861CXG/Qm/Znj7hSeRwFgd4 Ihjw== X-Forwarded-Encrypted: i=1; AJvYcCVHJ42m8gD+HfTBjnwSWMqbyHqp+KlQgkWOBP+/2oRiECZ6EX0nq84+dha1E718DlTgWJ3oVKvzMA==@kvack.org X-Gm-Message-State: AOJu0Yz2MqN6sVatc49UdhJ+i9mSzB7woRfIw1/X1dtmU2WyjljrkKD6 sufX7SccTQWZ1rQHgWe9CH2B9K9iQQnI56V/gpzYumY1LMtBBXZM X-Google-Smtp-Source: AGHT+IF/eB9UNR6VnoZKt3nFhYHrh56urFiBKWBZ6vnCYzDyugb1WbUzdkpcnja3XeP3jZyhS9PA3Q== X-Received: by 2002:a17:907:c1f:b0:a9d:e1d6:42a1 with SMTP id a640c23a62f3a-aa4dd57c0ecmr210229766b.30.1732101648787; Wed, 20 Nov 2024 03:20:48 -0800 (PST) Received: from f.. (cst-prg-93-87.cust.vodafone.cz. [46.135.93.87]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-aa20df5690csm758559566b.75.2024.11.20.03.20.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Nov 2024 03:20:48 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, hughd@google.com, linux-ext4@vger.kernel.org, tytso@mit.edu, linux-mm@kvack.org, Mateusz Guzik Subject: [PATCH v3 1/3] vfs: support caching symlink lengths in inodes Date: Wed, 20 Nov 2024 12:20:34 +0100 Message-ID: <20241120112037.822078-2-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241120112037.822078-1-mjguzik@gmail.com> References: <20241120112037.822078-1-mjguzik@gmail.com> MIME-Version: 1.0 X-Stat-Signature: db7mu9nex4t9jm8ypu5m54era41kyyrw X-Rspamd-Queue-Id: F3E7B120013 X-Rspamd-Server: rspam08 X-Rspam-User: X-HE-Tag: 1732101580-804806 X-HE-Meta: U2FsdGVkX18niNVHBR/UEWdzuU3d7k4oJ3AdJMHlEGHRlIi/+jl0OvIRH45na4QA6cxukUfAwyfb+3Iyecyg/598vkfQHUcP8snyDoN+Kyy86l/dS96qnIwZQWpZkheREEr9pdpmqnyv+Or4zNkyN8acsxC7aYopdy1ipo/a0/O+8QS9V6js7KgvvU+pamiTqke0Tcf06EcLasAa3RQ+t3icrtULQqvPXR3nHMCrzKludIsppj/3YY5cqV7iEc7WWcrDjbuLDRz915wJgbF72vQwQ4a2rQefFS6Rqbi0ajCfbMrO12CUQilMx4FgTFs6YplPaBigAuYx86dNZj5poUop0wXSdecfD0kVKgT0wkeMny0NWUp0x6ovmaCK5eySO+G6ja14xF9UWU1vfnuXVSkniF80Wyr0a3eXUOE8MNJNl8El6Rf3xW6FLtdB7WAgvK9d7OibduLDrM8BFJ/hu1m++ey6zt/SJuHTadwmGxVDjmrhjfFYONnqGOTz36VAyqIDlpuQ+DAcflsdmhvZgMuXbCHzLk/jih38fYiFYhKIXt2+PRZJICniQSvnjVvuo2r7ivVj4GVTME+y3/WrQGJ3AQz0LMEuCHr2hVFdhjtoZ1GCw2L6FB4OharccDiSp7445uBECilb5u6/osA0Io0bqzu7ziVlWNQqXoTTzxBMm4riFXC1sa2AQWcoXOWMgQZUgn31ZrNQ2hPtymPN27HupuX3Wq4iLvYX8L3SUK61q+qOYwe6PMa1hK1cIp4NkQnP/4A5WVHNMRuuOluMUy1uxwmZ1vTBES6FwS0jMpRcWBh0I3IuxWnPbcMfde4rNlsb2DVjYhMdAxEmT53GZod0avM7S6gxtpbGvMTqRvOxQH4UVrD+SVZFTPAtMVSwrUIAbX27mv0n8QfNWzkXtN6IBPFkpEJqcvlNJA09Ftb2Z3lou020D5oI1yOPbDAkFPTwFfcFnCjlSx3QtCH uijCrJB5 VChFUPiSwIWBo3b5oC8wC4FMEBkfPvxWkqC8p+QZiYQVhkm1ndvoVbPyhsNcB641qMooGaQba7b+fM4pp8CZT4vzypc0qUSi/YyQAfeH5zKihjebggsjjYSGVRAiCsYbGIlo0HFPdfUbGqv1AKK/gC6eQTr+4rhqJMTCTuHdfEWe30qZnwwNQiMQLRV0wWP1udsooWif2rRi3j2uoUWBm1ROk9rPHIX0oSVuB32W9A17cE4oAa/3w5xWKW2gdRQnU8cTBIsBcGNNdUvvRAzYQnn8Ki7wa23Ja9w6AY5eIHEuJWaIf4sZTlcy3sEbXD37y3zjZqR9IcG203s4oes2dKFEo2OVPWbsrl92S/tY8jFownoQRwmBD1BrgVA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When utilized it dodges strlen() in vfs_readlink(), giving about 1.5% speed up when issuing readlink on /initrd.img on ext4. Filesystems opt in by calling inode_set_cached_link() when creating an inode. The size is stored in a new union utilizing the same space as i_devices, thus avoiding growing the struct or taking up any more space. Churn-wise the current readlink_copy() helper is patched to accept the size instead of calculating it. Signed-off-by: Mateusz Guzik --- fs/namei.c | 34 +++++++++++++++++++--------------- fs/proc/namespaces.c | 2 +- include/linux/fs.h | 15 +++++++++++++-- security/apparmor/apparmorfs.c | 2 +- 4 files changed, 34 insertions(+), 19 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index 9d30c7aa9aa6..e56c29a22d26 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -5272,19 +5272,16 @@ SYSCALL_DEFINE2(rename, const char __user *, oldname, const char __user *, newna getname(newname), 0); } -int readlink_copy(char __user *buffer, int buflen, const char *link) +int readlink_copy(char __user *buffer, int buflen, const char *link, int linklen) { - int len = PTR_ERR(link); - if (IS_ERR(link)) - goto out; + int copylen; - len = strlen(link); - if (len > (unsigned) buflen) - len = buflen; - if (copy_to_user(buffer, link, len)) - len = -EFAULT; -out: - return len; + copylen = linklen; + if (unlikely(copylen > (unsigned) buflen)) + copylen = buflen; + if (copy_to_user(buffer, link, copylen)) + copylen = -EFAULT; + return copylen; } /** @@ -5304,6 +5301,9 @@ int vfs_readlink(struct dentry *dentry, char __user *buffer, int buflen) const char *link; int res; + if (inode->i_opflags & IOP_CACHED_LINK) + return readlink_copy(buffer, buflen, inode->i_link, inode->i_linklen); + if (unlikely(!(inode->i_opflags & IOP_DEFAULT_READLINK))) { if (unlikely(inode->i_op->readlink)) return inode->i_op->readlink(dentry, buffer, buflen); @@ -5322,7 +5322,7 @@ int vfs_readlink(struct dentry *dentry, char __user *buffer, int buflen) if (IS_ERR(link)) return PTR_ERR(link); } - res = readlink_copy(buffer, buflen, link); + res = readlink_copy(buffer, buflen, link, strlen(link)); do_delayed_call(&done); return res; } @@ -5391,10 +5391,14 @@ EXPORT_SYMBOL(page_put_link); int page_readlink(struct dentry *dentry, char __user *buffer, int buflen) { + const char *link; + int res; + DEFINE_DELAYED_CALL(done); - int res = readlink_copy(buffer, buflen, - page_get_link(dentry, d_inode(dentry), - &done)); + link = page_get_link(dentry, d_inode(dentry), &done); + res = PTR_ERR(link); + if (!IS_ERR(link)) + res = readlink_copy(buffer, buflen, link, strlen(link)); do_delayed_call(&done); return res; } diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c index 8e159fc78c0a..c610224faf10 100644 --- a/fs/proc/namespaces.c +++ b/fs/proc/namespaces.c @@ -83,7 +83,7 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl if (ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) { res = ns_get_name(name, sizeof(name), task, ns_ops); if (res >= 0) - res = readlink_copy(buffer, buflen, name); + res = readlink_copy(buffer, buflen, name, strlen(name)); } put_task_struct(task); return res; diff --git a/include/linux/fs.h b/include/linux/fs.h index 7e29433c5ecc..2cc98de5af43 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -626,6 +626,7 @@ is_uncached_acl(struct posix_acl *acl) #define IOP_XATTR 0x0008 #define IOP_DEFAULT_READLINK 0x0010 #define IOP_MGTIME 0x0020 +#define IOP_CACHED_LINK 0x0040 /* * Keep mostly read-only and often accessed (especially for @@ -723,7 +724,10 @@ struct inode { }; struct file_lock_context *i_flctx; struct address_space i_data; - struct list_head i_devices; + union { + struct list_head i_devices; + int i_linklen; + }; union { struct pipe_inode_info *i_pipe; struct cdev *i_cdev; @@ -749,6 +753,13 @@ struct inode { void *i_private; /* fs or device private pointer */ } __randomize_layout; +static inline void inode_set_cached_link(struct inode *inode, char *link, int linklen) +{ + inode->i_link = link; + inode->i_linklen = linklen; + inode->i_opflags |= IOP_CACHED_LINK; +} + /* * Get bit address from inode->i_state to use with wait_var_event() * infrastructre. @@ -3351,7 +3362,7 @@ extern const struct file_operations generic_ro_fops; #define special_file(m) (S_ISCHR(m)||S_ISBLK(m)||S_ISFIFO(m)||S_ISSOCK(m)) -extern int readlink_copy(char __user *, int, const char *); +extern int readlink_copy(char __user *, int, const char *, int); extern int page_readlink(struct dentry *, char __user *, int); extern const char *page_get_link(struct dentry *, struct inode *, struct delayed_call *); diff --git a/security/apparmor/apparmorfs.c b/security/apparmor/apparmorfs.c index 01b923d97a44..60959cfba672 100644 --- a/security/apparmor/apparmorfs.c +++ b/security/apparmor/apparmorfs.c @@ -2611,7 +2611,7 @@ static int policy_readlink(struct dentry *dentry, char __user *buffer, res = snprintf(name, sizeof(name), "%s:[%lu]", AAFS_NAME, d_inode(dentry)->i_ino); if (res > 0 && res < sizeof(name)) - res = readlink_copy(buffer, buflen, name); + res = readlink_copy(buffer, buflen, name, strlen(name)); else res = -ENOENT;