From patchwork Sat Mar 11 10:15:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Biggers X-Patchwork-Id: 9618533 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0E446604CC for ; Sat, 11 Mar 2017 10:15:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id ECBD2287B9 for ; Sat, 11 Mar 2017 10:15:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E0196287C0; Sat, 11 Mar 2017 10:15:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CAB6D287B9 for ; Sat, 11 Mar 2017 10:15:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932907AbdCKKPe (ORCPT ); Sat, 11 Mar 2017 05:15:34 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:35659 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932826AbdCKKPb (ORCPT ); Sat, 11 Mar 2017 05:15:31 -0500 Received: by mail-pf0-f195.google.com with SMTP id 67so13468998pfg.2; Sat, 11 Mar 2017 02:15:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id; bh=C6e7QO580m48D5W28UfkQitxAIUU1FmVJXGcVQ+16Cw=; b=sXsE2wrr9cJDNEI1YqruoEspav3ktOza3vwCW/NciQtNk48I7yQQxRrOPiP7vDxVUw Dv6DeuLScWZS0y4Dzqp2KEgIUmkylmMiYxPvGsiVOMJ/yVubYEhfef3K5egGcXK4jZC5 fB0yYmAiUYymg3XZaajYoSguVGi3DPDbYUFTOqaJjsirQtcpQ7wqUh02BywYtc8WJBpm aibm8IDVrgKOiEPLHz0UYKAoXTZap6vKLYf9lY501CU/spsL343LqNpMAZGpaZWUK3P6 kwCtUT9gTXesRpExvom+jpONyI2AJUYKWLNb1yNYXtulkOfOqQJ2pHW8EGNSbGyhCgj1 4Eeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=C6e7QO580m48D5W28UfkQitxAIUU1FmVJXGcVQ+16Cw=; b=Q8kU9KyFGBPscXgbHDHmy4yFdQ4HKkCNlUwUdcyxJOrc+xwB7qBz2CAGAXJ5turIl0 xLrl+koA0MNj+lmgmiAmtahsJjL5tuHOFnsOFiDJtvM/ifhgVxYUrxw2gTF8sl3lm2kb dnuI4WHdURgCOLhJERLVYAmgMLymzO9X4YmS89kjHXDiR+ZA5+W8pd2FtoTQXIDFTCXL qUCXAlDocXuuAktXP5EZuh0xcopggIgt9Rndyh96IhWKK1W3RvTiL27G8gRq25vgCRAD RQ9v8fbZNtBe/OS+klwmHJ63Ws/J1QlBtdaRcQLMbkfiGLu7VqOB7bt/BmxzWm7crHck T5Zg== X-Gm-Message-State: AMke39meEJF5HjWO0ooOypWcZ4oJ3I4Xa5ZvNyL47fI4vb1MIidULtKZlnBEZDN0epmj/g== X-Received: by 10.99.126.11 with SMTP id z11mr25698468pgc.51.1489227329461; Sat, 11 Mar 2017 02:15:29 -0800 (PST) Received: from zzz.hsd1.wa.comcast.net (c-73-239-167-150.hsd1.wa.comcast.net. [73.239.167.150]) by smtp.gmail.com with ESMTPSA id s21sm23462018pgg.65.2017.03.11.02.15.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 11 Mar 2017 02:15:28 -0800 (PST) From: Eric Biggers To: linux-fsdevel@vger.kernel.org Cc: Al Viro , David Howells , linux-kernel@vger.kernel.org, Eric Biggers Subject: [PATCH] statx: optimize copy of struct statx to userspace Date: Sat, 11 Mar 2017 02:15:10 -0800 Message-Id: <20170311101510.6504-1-ebiggers3@gmail.com> X-Mailer: git-send-email 2.12.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Eric Biggers I found that statx() was significantly slower than stat(). As a microbenchmark, I compared 10,000,000 invocations of fstat() on a tmpfs file to the same with statx() passed a NULL path: $ time ./stat_benchmark real 0m1.464s user 0m0.275s sys 0m1.187s $ time ./statx_benchmark real 0m5.530s user 0m0.281s sys 0m5.247s statx is expected to be a little slower than stat because struct statx is larger than struct stat, but not by *that* much. It turns out that most of the overhead was in copying struct statx to userspace, apparently mostly in all the stac/clac instructions that got generated for each __put_user() call. (This was on x86_64, but some other architectures, e.g. arm64, have something similar now too.) stat() instead initializes its struct on the stack and copies it to userspace with a single call to copy_to_user(). This turns out to be much faster, and changing statx to do this makes it almost as fast as stat: $ time ./statx_benchmark real 0m1.573s user 0m0.229s sys 0m1.344s Signed-off-by: Eric Biggers Acked-by: David Howells --- fs/stat.c | 70 ++++++++++++++++++++++++++++++--------------------------------- 1 file changed, 33 insertions(+), 37 deletions(-) diff --git a/fs/stat.c b/fs/stat.c index b792dd201c31..2cb3d0229232 100644 --- a/fs/stat.c +++ b/fs/stat.c @@ -510,46 +510,41 @@ SYSCALL_DEFINE4(fstatat64, int, dfd, const char __user *, filename, } #endif /* __ARCH_WANT_STAT64 || __ARCH_WANT_COMPAT_STAT64 */ -static inline int __put_timestamp(struct timespec *kts, - struct statx_timestamp __user *uts) +static inline void init_statx_timestamp(struct statx_timestamp *uts, + const struct timespec *kts) { - return (__put_user(kts->tv_sec, &uts->tv_sec ) || - __put_user(kts->tv_nsec, &uts->tv_nsec ) || - __put_user(0, &uts->__reserved )); + uts->tv_sec = kts->tv_sec; + uts->tv_nsec = kts->tv_nsec; + uts->__reserved = 0; } -/* - * Set the statx results. - */ -static long statx_set_result(struct kstat *stat, struct statx __user *buffer) +static int cp_statx(const struct kstat *stat, struct statx __user *buffer) { - uid_t uid = from_kuid_munged(current_user_ns(), stat->uid); - gid_t gid = from_kgid_munged(current_user_ns(), stat->gid); - - if (__put_user(stat->result_mask, &buffer->stx_mask ) || - __put_user(stat->mode, &buffer->stx_mode ) || - __clear_user(&buffer->__spare0, sizeof(buffer->__spare0)) || - __put_user(stat->nlink, &buffer->stx_nlink ) || - __put_user(uid, &buffer->stx_uid ) || - __put_user(gid, &buffer->stx_gid ) || - __put_user(stat->attributes, &buffer->stx_attributes ) || - __put_user(stat->blksize, &buffer->stx_blksize ) || - __put_user(MAJOR(stat->rdev), &buffer->stx_rdev_major ) || - __put_user(MINOR(stat->rdev), &buffer->stx_rdev_minor ) || - __put_user(MAJOR(stat->dev), &buffer->stx_dev_major ) || - __put_user(MINOR(stat->dev), &buffer->stx_dev_minor ) || - __put_timestamp(&stat->atime, &buffer->stx_atime ) || - __put_timestamp(&stat->btime, &buffer->stx_btime ) || - __put_timestamp(&stat->ctime, &buffer->stx_ctime ) || - __put_timestamp(&stat->mtime, &buffer->stx_mtime ) || - __put_user(stat->ino, &buffer->stx_ino ) || - __put_user(stat->size, &buffer->stx_size ) || - __put_user(stat->blocks, &buffer->stx_blocks ) || - __clear_user(&buffer->__spare1, sizeof(buffer->__spare1)) || - __clear_user(&buffer->__spare2, sizeof(buffer->__spare2))) - return -EFAULT; - - return 0; + struct statx tmp; + + tmp.stx_mask = stat->result_mask; + tmp.stx_blksize = stat->blksize; + tmp.stx_attributes = stat->attributes; + tmp.stx_nlink = stat->nlink; + tmp.stx_uid = from_kuid_munged(current_user_ns(), stat->uid); + tmp.stx_gid = from_kgid_munged(current_user_ns(), stat->gid); + tmp.stx_mode = stat->mode; + memset(tmp.__spare0, 0, sizeof(tmp.__spare0)); + tmp.stx_ino = stat->ino; + tmp.stx_size = stat->size; + tmp.stx_blocks = stat->blocks; + memset(tmp.__spare1, 0, sizeof(tmp.__spare1)); + init_statx_timestamp(&tmp.stx_atime, &stat->atime); + init_statx_timestamp(&tmp.stx_btime, &stat->btime); + init_statx_timestamp(&tmp.stx_ctime, &stat->ctime); + init_statx_timestamp(&tmp.stx_mtime, &stat->mtime); + tmp.stx_rdev_major = MAJOR(stat->rdev); + tmp.stx_rdev_minor = MINOR(stat->rdev); + tmp.stx_dev_major = MAJOR(stat->dev); + tmp.stx_dev_minor = MINOR(stat->dev); + memset(tmp.__spare2, 0, sizeof(tmp.__spare2)); + + return copy_to_user(buffer, &tmp, sizeof(tmp)) ? -EFAULT : 0; } /** @@ -582,7 +577,8 @@ SYSCALL_DEFINE5(statx, error = vfs_statx_fd(dfd, &stat, mask, flags); if (error) return error; - return statx_set_result(&stat, buffer); + + return cp_statx(&stat, buffer); } /* Caller is here responsible for sufficient locking (ie. inode->i_lock) */