From patchwork Tue Nov 19 09:45:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mateusz Guzik X-Patchwork-Id: 13879532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2ED80D62060 for ; Tue, 19 Nov 2024 09:46:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B53756B0085; Tue, 19 Nov 2024 04:46:18 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B02F26B0089; Tue, 19 Nov 2024 04:46:18 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A3256B008A; Tue, 19 Nov 2024 04:46:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 749706B0085 for ; Tue, 19 Nov 2024 04:46:18 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 36DBF80348 for ; Tue, 19 Nov 2024 09:46:18 +0000 (UTC) X-FDA: 82802363532.28.32FB8DC Received: from mail-ej1-f45.google.com (mail-ej1-f45.google.com [209.85.218.45]) by imf14.hostedemail.com (Postfix) with ESMTP id 9ECB6100008 for ; Tue, 19 Nov 2024 09:45:20 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=PEhiomJt; spf=pass (imf14.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.218.45 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732009332; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2k+LOXLkuwwtIxGlSwTxzA89tsnqiig0usDZlI6j5Rc=; b=sQR6WBPc8Lee7Eokt7bx7x8dHak1EtYHu3+GipDb+tonYp3RAfQiU5CHjNEBgjtUne5jiW izX52k2cwSbQshgADEyP5RQCD/ai3Y9DeVxK5FHMMuMwyrBfG+qGnW+V+chd9h271XzbHz I4HOHuAwprks9c4TkXaUXxXMlhleB9s= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=PEhiomJt; spf=pass (imf14.hostedemail.com: domain of mjguzik@gmail.com designates 209.85.218.45 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1732009332; a=rsa-sha256; cv=none; b=lDGENbmM0zm/LLdLYLX5EZAosbjwOPlPZBZ1r4qsJ7Wkb4xvvL2a/GRsOChxtPixeI5OBy SaZxnuB8oby0bAuqU9DlAGNhIym1Rtv4bMLDLZYkO4XNRC6WX7L9Tfd1//0oSJNodqBMjl OVh2GNsCBap+8U1Ymr+DsaIg1InK5WA= Received: by mail-ej1-f45.google.com with SMTP id a640c23a62f3a-a99fa009adcso218275966b.0 for ; Tue, 19 Nov 2024 01:46:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732009575; x=1732614375; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2k+LOXLkuwwtIxGlSwTxzA89tsnqiig0usDZlI6j5Rc=; b=PEhiomJtPZy/wl1XyzLI2OaLHKmGNvL8/q9Wyyd1kNbumSmXyoqhlIkrrpvLGVccDs N+wXGje0gtrSWdkxAx+bWql0K9mB15tOI9A8iG6PEsP+FLDjUkodSfJIYJK1IVUfyQZm YRhKQ7/S8J6e4HnLbDJyutvfRXJBWQUOmjFaWcd6UkFNRicOBLijKBJqk/qdnjVZezHE heD52BdgIeGylC3Lg9nw5C+jziB7XKcG8hQVXKIgEcfN5dE39zHt16MjnmNztkRodf1W 1UGRVuZbmvZO3oIOK0T4D1/T10hHxFWcOXp58FCPIKbRBRcjKIQNKC34oW9JL72M+Unt PLLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732009575; x=1732614375; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2k+LOXLkuwwtIxGlSwTxzA89tsnqiig0usDZlI6j5Rc=; b=MeLC6rh2xwfswFyWdA/cTEv3bDy1yvQnXO1QB9onQD4rSz4cnmC0tK1qY7uKPYTGY7 tyPB61lc+rSMpTyflYgJ7/GSrcP0LfadVn+ICAB+XzZ8Q2sUyf7q3kPfnM4rkihTU/Dc eRfHybhp95UoFzoGZ75RCybc0w83HIvvHMTjdR7GqaQOkycLrogLZgJQ1wJlozMct9GO teTtMkQAWv5EN5rLPHKlNS40qeMOCan0E4CUDGGTCsN0swwKC8ROAnqeEXWaVc3zQkSm yzX3joNhZe3zjrVd+Wfg92hNPNYqrJRRL6dICmInvrjysUQSj5wA+ptB+ToV2tzXEo3v rKMg== X-Forwarded-Encrypted: i=1; AJvYcCVOkEIyFHkC6Gz4mZeMwL6kvKnM++ESW3f/nYuJvlLYyPWRVlAU9k8bA7cMqo+qnwVz04yv3jjjCQ==@kvack.org X-Gm-Message-State: AOJu0Yw5p9YfSq6Hav+tCiD5CO3krlBcHWegFSK8g8IhiH7YO7jE91NT OPw9+X7oDMqKLtQka//pxT1S5kBZGkMUsboH3tWbLm3UZxtG1n63 X-Google-Smtp-Source: AGHT+IHq3gRQkbq4vilVBjoFrqlc2rP/Dxb0QaL818D1CYDKfch0hN3L/I87OgXz6zPDisBWU8VEfw== X-Received: by 2002:a05:6402:524d:b0:5cf:c1a3:b109 with SMTP id 4fb4d7f45d1cf-5cfc1a3b3d8mr8659092a12.2.1732009574442; Tue, 19 Nov 2024 01:46:14 -0800 (PST) Received: from f.. (cst-prg-93-87.cust.vodafone.cz. [46.135.93.87]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5cfcb3edce9sm1821154a12.35.2024.11.19.01.46.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Nov 2024 01:46:13 -0800 (PST) From: Mateusz Guzik To: brauner@kernel.org Cc: viro@zeniv.linux.org.uk, jack@suse.cz, linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, hughd@google.com, linux-ext4@vger.kernel.org, tytso@mit.edu, linux-mm@kvack.org, Mateusz Guzik Subject: [PATCH v2 1/3] vfs: support caching symlink lengths in inodes Date: Tue, 19 Nov 2024 10:45:53 +0100 Message-ID: <20241119094555.660666-2-mjguzik@gmail.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20241119094555.660666-1-mjguzik@gmail.com> References: <20241119094555.660666-1-mjguzik@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9ECB6100008 X-Stat-Signature: 3tgqcjnd7upo8wpndwknic3ars96za4j X-Rspam-User: X-HE-Tag: 1732009520-503457 X-HE-Meta: U2FsdGVkX19qgkMZLP9f1qQRRhVPC3yTwjhEu6PML5MQlrgptQ18i5EtxXBskQEC2Eo/a5dKQ4SneohthHiXB6nTVXrfd9u2/V7yJDGwTjc3ui9Z2cYtFf0boqqxqiNBIr19iAwUjVXFJc+yLviCwllwoWrlLl462H+ZVzxMkbUHxsxiPjYTm3DJpzEj3w7SOTqJVPDNT8kSb9jOU48cSMiGB5ROvdDVoYozPMWchXx2788HlVkcAhZwcF6QeEa7leI/UXCIFfrM+97n1+QKq3FDrGhHWiL2olJiHIJ/sGyyxfrypmpAfnhptsYIF1y8Cl/1RVWbP8/IHBuCHgaGaE5ZNIUqMOrI9AaakdPIMrkpeHh6qZDJbsFyIoGKSQo6xLvXFvRkFU7hteCV7oaCIpqWgjuR41VicxWsQH4Dn2ETT3WRbr2ua7UobYoYtsWyra+avRVTKhrb6hyxHh1VHR6HnP9PEWun/i83J5Fr+QYOjtcOPirDnGkusIx9ltS+zzzChm+CQK6dFgozFA97HrmitwuXPKRe4W4xQOOmqif99fsp6DE43q5kWpq2axsiThxHWRQB+F8h42A1flPSzY2yQ+a6Xr1hS+ToAM0ED0bF23iMP3SOxGAEVNWft3iL1k//kbXbW5EuqDMiil/d3wp158LujcIgy1izazOxi90tJ6jeu31aZTXTGWGgX9/ndRSIsY7oF1DGrxH+pUo6TEF5OixjIjQb+R/9EuZkzAZ62TZcz7Jyp277wHpUN0aiQijevNVB+faXKDy/XIjrw1au7bOvYRE4QA1IfhaS9GLJWAgIAzi7HhME2rb2rPy+wR7X5u0l08XNedeJSCv3lPCkrqpr5eCCNUWvTwqqGUwA1lT+D1Ep3p7ZnacC9WMh+ToDxWV9z9yU3A4GsXPipY6T2MtFX2CmonQOdRGRikRs8UgMl0AYtXfbzGM3BTSfRGF2255jPk2zDYhd/5K d9YdBnAT ylHEId7S897UZoWxmFFNtiuiCOpKmhKg4DuxwzKlBwqTpnThbxM0Qp60uCcDCsJoCKIB2vUxKW6ISclMu1dQoAyeS7Q9+xFphrnpmUoIYPg/HmKVqgTJxTUC70hQ+puUWdFEt27sT/eIouVURxVCrCV1tABOjcqrAvq/4IggGabBuzCHwYYTZCchOdgu1iznZz2RbyK+/OnQVMfifLSbMwaX2Xyb3bxheQmBQHBKS5XLa37qdYV7TDCS8+UEOGQoVBKNjUbprgS7KHOuyYaOEt9WKludGqOkjypUFB7Fsk0a9ufGg1/p7y6iWiiuG9jrNL/32yybPr2sO+NUu46haoYLenMl/4tdzeNtv+m8mNdt9e2xCQUOj8/mBrg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When utilized it dodges strlen() in vfs_readlink(), giving about 1.5% speed up when issuing readlink on /initrd.img on ext4. Filesystems opt in by calling inode_set_cached_link() when creating an inode. The size is stored in what used to be a 4-byte hole. If necessary the field can be made smaller and converted into a union with something not used with symlinks. Churn-wise the current readlink_copy() helper is patched to accept the size instead of calculating it. Signed-off-by: Mateusz Guzik --- fs/namei.c | 34 +++++++++++++++++++--------------- fs/proc/namespaces.c | 2 +- include/linux/fs.h | 12 ++++++++++-- security/apparmor/apparmorfs.c | 2 +- 4 files changed, 31 insertions(+), 19 deletions(-) diff --git a/fs/namei.c b/fs/namei.c index 9d30c7aa9aa6..e56c29a22d26 100644 --- a/fs/namei.c +++ b/fs/namei.c @@ -5272,19 +5272,16 @@ SYSCALL_DEFINE2(rename, const char __user *, oldname, const char __user *, newna getname(newname), 0); } -int readlink_copy(char __user *buffer, int buflen, const char *link) +int readlink_copy(char __user *buffer, int buflen, const char *link, int linklen) { - int len = PTR_ERR(link); - if (IS_ERR(link)) - goto out; + int copylen; - len = strlen(link); - if (len > (unsigned) buflen) - len = buflen; - if (copy_to_user(buffer, link, len)) - len = -EFAULT; -out: - return len; + copylen = linklen; + if (unlikely(copylen > (unsigned) buflen)) + copylen = buflen; + if (copy_to_user(buffer, link, copylen)) + copylen = -EFAULT; + return copylen; } /** @@ -5304,6 +5301,9 @@ int vfs_readlink(struct dentry *dentry, char __user *buffer, int buflen) const char *link; int res; + if (inode->i_opflags & IOP_CACHED_LINK) + return readlink_copy(buffer, buflen, inode->i_link, inode->i_linklen); + if (unlikely(!(inode->i_opflags & IOP_DEFAULT_READLINK))) { if (unlikely(inode->i_op->readlink)) return inode->i_op->readlink(dentry, buffer, buflen); @@ -5322,7 +5322,7 @@ int vfs_readlink(struct dentry *dentry, char __user *buffer, int buflen) if (IS_ERR(link)) return PTR_ERR(link); } - res = readlink_copy(buffer, buflen, link); + res = readlink_copy(buffer, buflen, link, strlen(link)); do_delayed_call(&done); return res; } @@ -5391,10 +5391,14 @@ EXPORT_SYMBOL(page_put_link); int page_readlink(struct dentry *dentry, char __user *buffer, int buflen) { + const char *link; + int res; + DEFINE_DELAYED_CALL(done); - int res = readlink_copy(buffer, buflen, - page_get_link(dentry, d_inode(dentry), - &done)); + link = page_get_link(dentry, d_inode(dentry), &done); + res = PTR_ERR(link); + if (!IS_ERR(link)) + res = readlink_copy(buffer, buflen, link, strlen(link)); do_delayed_call(&done); return res; } diff --git a/fs/proc/namespaces.c b/fs/proc/namespaces.c index 8e159fc78c0a..c610224faf10 100644 --- a/fs/proc/namespaces.c +++ b/fs/proc/namespaces.c @@ -83,7 +83,7 @@ static int proc_ns_readlink(struct dentry *dentry, char __user *buffer, int bufl if (ptrace_may_access(task, PTRACE_MODE_READ_FSCREDS)) { res = ns_get_name(name, sizeof(name), task, ns_ops); if (res >= 0) - res = readlink_copy(buffer, buflen, name); + res = readlink_copy(buffer, buflen, name, strlen(name)); } put_task_struct(task); return res; diff --git a/include/linux/fs.h b/include/linux/fs.h index 972147da71f9..30e332fb399d 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -626,6 +626,7 @@ is_uncached_acl(struct posix_acl *acl) #define IOP_XATTR 0x0008 #define IOP_DEFAULT_READLINK 0x0010 #define IOP_MGTIME 0x0020 +#define IOP_CACHED_LINK 0x0040 /* * Keep mostly read-only and often accessed (especially for @@ -686,7 +687,7 @@ struct inode { /* Misc */ u32 i_state; - /* 32-bit hole */ + int i_linklen; /* for symlinks */ struct rw_semaphore i_rwsem; unsigned long dirtied_when; /* jiffies of first dirtying */ @@ -749,6 +750,13 @@ struct inode { void *i_private; /* fs or device private pointer */ } __randomize_layout; +static inline void inode_set_cached_link(struct inode *inode, char *link, int linklen) +{ + inode->i_link = link; + inode->i_linklen = linklen; + inode->i_opflags |= IOP_CACHED_LINK; +} + /* * Get bit address from inode->i_state to use with wait_var_event() * infrastructre. @@ -3351,7 +3359,7 @@ extern const struct file_operations generic_ro_fops; #define special_file(m) (S_ISCHR(m)||S_ISBLK(m)||S_ISFIFO(m)||S_ISSOCK(m)) -extern int readlink_copy(char __user *, int, const char *); +extern int readlink_copy(char __user *, int, const char *, int); extern int page_readlink(struct dentry *, char __user *, int); extern const char *page_get_link(struct dentry *, struct inode *, struct delayed_call *); diff --git a/security/apparmor/apparmorfs.c b/security/apparmor/apparmorfs.c index 01b923d97a44..60959cfba672 100644 --- a/security/apparmor/apparmorfs.c +++ b/security/apparmor/apparmorfs.c @@ -2611,7 +2611,7 @@ static int policy_readlink(struct dentry *dentry, char __user *buffer, res = snprintf(name, sizeof(name), "%s:[%lu]", AAFS_NAME, d_inode(dentry)->i_ino); if (res > 0 && res < sizeof(name)) - res = readlink_copy(buffer, buflen, name); + res = readlink_copy(buffer, buflen, name, strlen(name)); else res = -ENOENT;