From patchwork Mon Sep 17 20:10:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Brandenburg X-Patchwork-Id: 10603305 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C48D617D5 for ; Mon, 17 Sep 2018 20:11:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B46692A6F7 for ; Mon, 17 Sep 2018 20:11:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B28C92A6A3; Mon, 17 Sep 2018 20:11:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 529862A6F7 for ; Mon, 17 Sep 2018 20:11:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728448AbeIRBkF (ORCPT ); Mon, 17 Sep 2018 21:40:05 -0400 Received: from mail-qk1-f195.google.com ([209.85.222.195]:34338 "EHLO mail-qk1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728407AbeIRBkE (ORCPT ); Mon, 17 Sep 2018 21:40:04 -0400 Received: by mail-qk1-f195.google.com with SMTP id d15-v6so9763058qkc.1 for ; Mon, 17 Sep 2018 13:11:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=omnibond-com.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cB+35A/7ZQ46TKL7ut14HnfM4ymxRMTCzupJyIEHGgo=; b=Y3LKUTupet3ME6w/ALRJfxyt6cVSqhku5s32SnA0PwEyLwiz25xqjSLY9ZPrNSzMf5 MyRByYjQLW928ETalvNdoiCvj/DcmiwLhT3ruAQu5faSEN1nbnX8T3KlTObIK613oRQm F3wBYIizXTc5G+fgmKJ2fzs9oWWIZzTxclU7H4x7CyseZ/05YOK/ZrE5jhrX0LdrecbK 4HqmLHeF5hawHD/rMdPL6nY45eMWI59sWusQNm7H4ySVLEblS56YkojCrZf9YeclnHtK WAAqI8E6BpJg3l4UlxS+uXQldgBBmAYDHwvtuz7vi85LK6Zxp7Mo3gQ20xBwf1Blk7Dz A4vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cB+35A/7ZQ46TKL7ut14HnfM4ymxRMTCzupJyIEHGgo=; b=YCSjTokqtgmjmT1txyXHnXt/5t/CambZqMCpkdGqosBTn5rYXJiC6NxTJ4Lj4SfciI v2gcM9plYr1B902paKHSqpnJhDTH0Ixaps3Z/MOFbhpSnD0LHdg8mhGCoFoY+PN9CtAT VjwJxeOxeY4jYL85e7fD+rC4vhCpeEY6QvJADlm117/CXpqB2vrp7HN7I5lDNF7/VCFV /2SFAdgnuQ1qEg9XY7t2aa75iHhmzh0MENqZwKq5AJF1Ov3iQIUF8dIe8O4UgEUXTzqa 9VmtNum8I5XVZtsP//Ik2QZcpLuLimenxpC/WBLR+DLFheSzfIM1qdy4a9xHJU6H71f3 wLkA== X-Gm-Message-State: APzg51DgVN5mFcJF26+qOO43SN4Wu6vbKEC5NcRuSv46bx+7FR2btdLG j9YQ4Gnsh1l9y8AQyLpEdICzqA== X-Google-Smtp-Source: ANB0VdY3fYxkerd9V5a8jDH5E0GnlMXgVp3LFrcWcucbQZMLhENnIg1VJUxjDZTPkqf3PW0hquWDyw== X-Received: by 2002:a37:84c4:: with SMTP id g187-v6mr18609728qkd.49.1537215072313; Mon, 17 Sep 2018 13:11:12 -0700 (PDT) Received: from ip-172-31-22-34.ec2.internal (ec2-18-215-252-133.compute-1.amazonaws.com. [18.215.252.133]) by smtp.gmail.com with ESMTPSA id n8-v6sm11053480qtk.38.2018.09.17.13.11.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 17 Sep 2018 13:11:11 -0700 (PDT) From: Martin Brandenburg To: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, devel@lists.orangefs.org Cc: Martin Brandenburg Subject: [PATCH 01/17] orangefs: implement xattr cache Date: Mon, 17 Sep 2018 20:10:38 +0000 Message-Id: <20180917201054.3530-2-martin@omnibond.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20180917201054.3530-1-martin@omnibond.com> References: <20180917201054.3530-1-martin@omnibond.com> MIME-Version: 1.0 Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This uses the same timeout as the getattr cache. This substantially increases performance when writing files with smaller buffer sizes. When writing, the size is (often) changed, which causes a call to notify_change which calls security_inode_need_killpriv which needs a getxattr. Caching it reduces traffic to the server. Signed-off-by: Martin Brandenburg --- fs/orangefs/inode.c | 1 + fs/orangefs/orangefs-kernel.h | 10 ++++ fs/orangefs/super.c | 9 +++ fs/orangefs/xattr.c | 104 ++++++++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+) diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c index 31932879b716..a7a8d3647ffe 100644 --- a/fs/orangefs/inode.c +++ b/fs/orangefs/inode.c @@ -367,6 +367,7 @@ static int orangefs_set_inode(struct inode *inode, void *data) struct orangefs_object_kref *ref = (struct orangefs_object_kref *) data; ORANGEFS_I(inode)->refn.fs_id = ref->fs_id; ORANGEFS_I(inode)->refn.khandle = ref->khandle; + hash_init(ORANGEFS_I(inode)->xattr_cache); return 0; } diff --git a/fs/orangefs/orangefs-kernel.h b/fs/orangefs/orangefs-kernel.h index 17b24ad6b264..0c76b8899fd1 100644 --- a/fs/orangefs/orangefs-kernel.h +++ b/fs/orangefs/orangefs-kernel.h @@ -193,6 +193,8 @@ struct orangefs_inode_s { unsigned long getattr_time; u32 getattr_mask; + + DECLARE_HASHTABLE(xattr_cache, 4); }; /* per superblock private orangefs info */ @@ -217,6 +219,14 @@ struct orangefs_stats { unsigned long writes; }; +struct orangefs_cached_xattr { + struct hlist_node node; + char key[ORANGEFS_MAX_XATTR_NAMELEN]; + char val[ORANGEFS_MAX_XATTR_VALUELEN]; + ssize_t length; + unsigned long timeout; +}; + extern struct orangefs_stats orangefs_stats; /* diff --git a/fs/orangefs/super.c b/fs/orangefs/super.c index dfaee90d30bd..4c36481208f5 100644 --- a/fs/orangefs/super.c +++ b/fs/orangefs/super.c @@ -128,6 +128,15 @@ static void orangefs_i_callback(struct rcu_head *head) { struct inode *inode = container_of(head, struct inode, i_rcu); struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode); + struct orangefs_cached_xattr *cx; + struct hlist_node *tmp; + int i; + + hash_for_each_safe(orangefs_inode->xattr_cache, i, tmp, cx, node) { + hlist_del(&cx->node); + kfree(cx); + } + kmem_cache_free(orangefs_inode_cache, orangefs_inode); } diff --git a/fs/orangefs/xattr.c b/fs/orangefs/xattr.c index 03bcb871544d..998c3563bcdd 100644 --- a/fs/orangefs/xattr.c +++ b/fs/orangefs/xattr.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0 /* * (C) 2001 Clemson University and The University of Chicago + * Copyright 2018 Omnibond Systems, L.L.C. * * See COPYING in top-level directory. */ @@ -50,6 +51,35 @@ static inline int convert_to_internal_xattr_flags(int setxattr_flags) return internal_flag; } +static unsigned int xattr_key(const char *key) +{ + unsigned int i = 0; + while (key) + i += *key++; + return i % 16; +} + +static struct orangefs_cached_xattr *find_cached_xattr(struct inode *inode, + const char *key) +{ + struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode); + struct orangefs_cached_xattr *cx; + struct hlist_head *h; + struct hlist_node *tmp; + h = &orangefs_inode->xattr_cache[xattr_key(key)]; + if (hlist_empty(h)) + return NULL; + hlist_for_each_entry_safe(cx, tmp, h, node) { +/* if (!time_before(jiffies, cx->timeout)) { + hlist_del(&cx->node); + kfree(cx); + continue; + }*/ + if (!strcmp(cx->key, key)) + return cx; + } + return NULL; +} /* * Tries to get a specified key's attributes of a given @@ -65,6 +95,7 @@ ssize_t orangefs_inode_getxattr(struct inode *inode, const char *name, { struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode); struct orangefs_kernel_op_s *new_op = NULL; + struct orangefs_cached_xattr *cx; ssize_t ret = -ENOMEM; ssize_t length = 0; int fsuid; @@ -93,6 +124,27 @@ ssize_t orangefs_inode_getxattr(struct inode *inode, const char *name, down_read(&orangefs_inode->xattr_sem); + cx = find_cached_xattr(inode, name); + if (cx && time_before(jiffies, cx->timeout)) { + if (cx->length == -1) { + ret = -ENODATA; + goto out_unlock; + } else { + if (size == 0) { + ret = cx->length; + goto out_unlock; + } + if (cx->length > size) { + ret = -ERANGE; + goto out_unlock; + } + memcpy(buffer, cx->val, cx->length); + memset(buffer + cx->length, 0, size - cx->length); + ret = cx->length; + goto out_unlock; + } + } + new_op = op_alloc(ORANGEFS_VFS_OP_GETXATTR); if (!new_op) goto out_unlock; @@ -117,6 +169,15 @@ ssize_t orangefs_inode_getxattr(struct inode *inode, const char *name, " does not exist!\n", get_khandle_from_ino(inode), (char *)new_op->upcall.req.getxattr.key); + cx = kmalloc(sizeof *cx, GFP_KERNEL); + if (cx) { + strcpy(cx->key, name); + cx->length = -1; + cx->timeout = jiffies + + orangefs_getattr_timeout_msecs*HZ/1000; + hash_add(orangefs_inode->xattr_cache, &cx->node, + xattr_key(cx->key)); + } } goto out_release_op; } @@ -156,6 +217,23 @@ ssize_t orangefs_inode_getxattr(struct inode *inode, const char *name, ret = length; + if (cx) { + strcpy(cx->key, name); + memcpy(cx->val, buffer, length); + cx->length = length; + cx->timeout = jiffies + HZ; + } else { + cx = kmalloc(sizeof *cx, GFP_KERNEL); + if (cx) { + strcpy(cx->key, name); + memcpy(cx->val, buffer, length); + cx->length = length; + cx->timeout = jiffies + HZ; + hash_add(orangefs_inode->xattr_cache, &cx->node, + xattr_key(cx->key)); + } + } + out_release_op: op_release(new_op); out_unlock: @@ -168,6 +246,9 @@ static int orangefs_inode_removexattr(struct inode *inode, const char *name, { struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode); struct orangefs_kernel_op_s *new_op = NULL; + struct orangefs_cached_xattr *cx; + struct hlist_head *h; + struct hlist_node *tmp; int ret = -ENOMEM; if (strlen(name) >= ORANGEFS_MAX_XATTR_NAMELEN) @@ -209,6 +290,16 @@ static int orangefs_inode_removexattr(struct inode *inode, const char *name, "orangefs_inode_removexattr: returning %d\n", ret); op_release(new_op); + + h = &orangefs_inode->xattr_cache[xattr_key(name)]; + hlist_for_each_entry_safe(cx, tmp, h, node) { + if (!strcmp(cx->key, name)) { + hlist_del(&cx->node); + kfree(cx); + break; + } + } + out_unlock: up_write(&orangefs_inode->xattr_sem); return ret; @@ -226,6 +317,9 @@ int orangefs_inode_setxattr(struct inode *inode, const char *name, struct orangefs_inode_s *orangefs_inode = ORANGEFS_I(inode); struct orangefs_kernel_op_s *new_op; int internal_flag = 0; + struct orangefs_cached_xattr *cx; + struct hlist_head *h; + struct hlist_node *tmp; int ret = -ENOMEM; gossip_debug(GOSSIP_XATTR_DEBUG, @@ -287,6 +381,16 @@ int orangefs_inode_setxattr(struct inode *inode, const char *name, /* when request is serviced properly, free req op struct */ op_release(new_op); + + h = &orangefs_inode->xattr_cache[xattr_key(name)]; + hlist_for_each_entry_safe(cx, tmp, h, node) { + if (!strcmp(cx->key, name)) { + hlist_del(&cx->node); + kfree(cx); + break; + } + } + out_unlock: up_write(&orangefs_inode->xattr_sem); return ret;