From patchwork Mon Oct 1 14:41:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Trond Myklebust X-Patchwork-Id: 10622277 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7C2411515 for ; Mon, 1 Oct 2018 14:42:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6033128DDB for ; Mon, 1 Oct 2018 14:42:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 52EF72921C; Mon, 1 Oct 2018 14:42:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B46072944D for ; Mon, 1 Oct 2018 14:42:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729182AbeJAVUl (ORCPT ); Mon, 1 Oct 2018 17:20:41 -0400 Received: from mail-it1-f193.google.com ([209.85.166.193]:39210 "EHLO mail-it1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729479AbeJAVUk (ORCPT ); Mon, 1 Oct 2018 17:20:40 -0400 Received: by mail-it1-f193.google.com with SMTP id w200-v6so11735680itc.4 for ; Mon, 01 Oct 2018 07:42:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=clOqvGBHrepHq9+TvDgQDJT+2NmZ556LWuy6ooxnDMg=; b=bsWnXT9Xn/ZffDVCOH4DfWS9Skj0jhlsq6CuS8blZTFwgAAweREGIGUpyjiryxcZUI Q9fvqwVHkGVcQbwddAQJvei8fZL6WzG7FBj5acCEBqJ3og3aJ50aQl+xX+ykYuMPihBd zz2ns8927B3+Rcd5q6ESpTVhrcIJwGWMlThT3hbBPN/LAngR0GVryZHj25Da9Wh/MlHV 1/4VgongItTiKRzEB1xZ6U2Gv2wSo/kOzksqpT+QIovi+P23xOHXo+45ced/IUdewRhA EK1bSW+jTmOVQbcUTn6+7Z4aEH9aUFI+JUaL3gZRGgksXJCJh8uixPqbZMoEsjdJQiHx cV/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=clOqvGBHrepHq9+TvDgQDJT+2NmZ556LWuy6ooxnDMg=; b=OZ4mNuPdPOc+/P9soC9vQGAJzErJGUvuKEFnFkPxMTWviGx9ETpTcHSFmKfqaFC3wd rmTdOZ4uOs1I6uvgCcQbFBF21xVOlD4xMdFNgFQ9M91CmjtwwsAh4Cf8Jh3PxfnDZBGd sw+O7o0SP1nIvk0MVGf3Gcax+aCS0BgJu0NGhL53JN55hxOmNTF4yo0xx+hDn9RVVCRH Lpns9+W5eQgdTsBWvgw3J27VsxW/CWZg6WXxzmBNNrliJOmCxNKy0R/P8wbFpJ9KAC0u HWTFcn9A72Cy4R7x/Nj4lcC0Z+k+TuBsGbryD6kkkLiLKjG9n+JbXrXKjTaRCP1CJjsz 4y+Q== X-Gm-Message-State: ABuFfohGeS95N+nfJPqdFj//hpVq9Z+omtFyOha8cgq/um4NE5s2TXGC wDcc73lIyzXdFUeEXhGDZg== X-Google-Smtp-Source: ACcGV62/XPtinOdTtO5qlslL7SqLx6qXckwZL2V/E3u1l75tR07ZIbl7hO8y7vwC/4mDuHNd/PJKlg== X-Received: by 2002:a02:1b4f:: with SMTP id l76-v6mr8422953jad.57.1538404951018; Mon, 01 Oct 2018 07:42:31 -0700 (PDT) Received: from leira.trondhjem.org.localdomain (c-68-40-195-73.hsd1.mi.comcast.net. [68.40.195.73]) by smtp.gmail.com with ESMTPSA id j75-v6sm2055597itj.33.2018.10.01.07.42.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 01 Oct 2018 07:42:30 -0700 (PDT) From: Trond Myklebust X-Google-Original-From: Trond Myklebust To: "J . Bruce Fields" Cc: linux-nfs@vger.kernel.org Subject: [PATCH 13/15] knfsd: Simplify NFS duplicate replay cache Date: Mon, 1 Oct 2018 10:41:55 -0400 Message-Id: <20181001144157.3515-14-trond.myklebust@hammerspace.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20181001144157.3515-13-trond.myklebust@hammerspace.com> References: <20181001144157.3515-1-trond.myklebust@hammerspace.com> <20181001144157.3515-2-trond.myklebust@hammerspace.com> <20181001144157.3515-3-trond.myklebust@hammerspace.com> <20181001144157.3515-4-trond.myklebust@hammerspace.com> <20181001144157.3515-5-trond.myklebust@hammerspace.com> <20181001144157.3515-6-trond.myklebust@hammerspace.com> <20181001144157.3515-7-trond.myklebust@hammerspace.com> <20181001144157.3515-8-trond.myklebust@hammerspace.com> <20181001144157.3515-9-trond.myklebust@hammerspace.com> <20181001144157.3515-10-trond.myklebust@hammerspace.com> <20181001144157.3515-11-trond.myklebust@hammerspace.com> <20181001144157.3515-12-trond.myklebust@hammerspace.com> <20181001144157.3515-13-trond.myklebust@hammerspace.com> MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Simplify the duplicate replay cache by initialising the preallocated cache entry, so that we can use it as a key for the cache lookup. Note that the 99.999% case we want to optimise for is still the one where the lookup fails, and we have to add this entry to the cache, so preinitialising should not cause a performance penalty. Signed-off-by: Trond Myklebust --- fs/nfsd/cache.h | 6 +-- fs/nfsd/nfscache.c | 94 ++++++++++++++++++++++------------------------ 2 files changed, 47 insertions(+), 53 deletions(-) diff --git a/fs/nfsd/cache.h b/fs/nfsd/cache.h index b7559c6f2b97..bd77d5f6fe53 100644 --- a/fs/nfsd/cache.h +++ b/fs/nfsd/cache.h @@ -24,13 +24,13 @@ struct svc_cacherep { unsigned char c_state, /* unused, inprog, done */ c_type, /* status, buffer */ c_secure : 1; /* req came from port < 1024 */ - struct sockaddr_in6 c_addr; __be32 c_xid; - u32 c_prot; + __wsum c_csum; u32 c_proc; + u32 c_prot; u32 c_vers; unsigned int c_len; - __wsum c_csum; + struct sockaddr_in6 c_addr; unsigned long c_timestamp; union { struct kvec u_vec; diff --git a/fs/nfsd/nfscache.c b/fs/nfsd/nfscache.c index cef4686f87ef..527ce4c65765 100644 --- a/fs/nfsd/nfscache.c +++ b/fs/nfsd/nfscache.c @@ -121,7 +121,7 @@ nfsd_cache_hash(__be32 xid) } static struct svc_cacherep * -nfsd_reply_cache_alloc(void) +nfsd_reply_cache_alloc(struct svc_rqst *rqstp, __wsum csum) { struct svc_cacherep *rp; @@ -130,6 +130,16 @@ nfsd_reply_cache_alloc(void) rp->c_state = RC_UNUSED; rp->c_type = RC_NOCACHE; INIT_LIST_HEAD(&rp->c_lru); + + rp->c_xid = rqstp->rq_xid; + rp->c_proc = rqstp->rq_proc; + memset(&rp->c_addr, 0, sizeof(rp->c_addr)); + rpc_copy_addr((struct sockaddr *)&rp->c_addr, svc_addr(rqstp)); + rpc_set_port((struct sockaddr *)&rp->c_addr, rpc_get_port(svc_addr(rqstp))); + rp->c_prot = rqstp->rq_prot; + rp->c_vers = rqstp->rq_vers; + rp->c_len = rqstp->rq_arg.len; + rp->c_csum = csum; } return rp; } @@ -141,9 +151,11 @@ nfsd_reply_cache_free_locked(struct svc_cacherep *rp) drc_mem_usage -= rp->c_replvec.iov_len; kfree(rp->c_replvec.iov_base); } - list_del(&rp->c_lru); - atomic_dec(&num_drc_entries); - drc_mem_usage -= sizeof(*rp); + if (rp->c_state != RC_UNUSED) { + list_del(&rp->c_lru); + atomic_dec(&num_drc_entries); + drc_mem_usage -= sizeof(*rp); + } kmem_cache_free(drc_slab, rp); } @@ -319,24 +331,23 @@ nfsd_cache_csum(struct svc_rqst *rqstp) } static bool -nfsd_cache_match(struct svc_rqst *rqstp, __wsum csum, struct svc_cacherep *rp) +nfsd_cache_match(const struct svc_cacherep *key, const struct svc_cacherep *rp) { /* Check RPC XID first */ - if (rqstp->rq_xid != rp->c_xid) + if (key->c_xid != rp->c_xid) return false; /* compare checksum of NFS data */ - if (csum != rp->c_csum) { + if (key->c_csum != rp->c_csum) { ++payload_misses; return false; } /* Other discriminators */ - if (rqstp->rq_proc != rp->c_proc || - rqstp->rq_prot != rp->c_prot || - rqstp->rq_vers != rp->c_vers || - rqstp->rq_arg.len != rp->c_len || - !rpc_cmp_addr(svc_addr(rqstp), (struct sockaddr *)&rp->c_addr) || - rpc_get_port(svc_addr(rqstp)) != rpc_get_port((struct sockaddr *)&rp->c_addr)) + if (key->c_proc != rp->c_proc || + key->c_prot != rp->c_prot || + key->c_vers != rp->c_vers || + key->c_len != rp->c_len || + memcmp(&key->c_addr, &rp->c_addr, sizeof(key->c_addr)) != 0) return false; return true; @@ -345,19 +356,18 @@ nfsd_cache_match(struct svc_rqst *rqstp, __wsum csum, struct svc_cacherep *rp) /* * Search the request hash for an entry that matches the given rqstp. * Must be called with cache_lock held. Returns the found entry or - * NULL on failure. + * inserts an empty key on failure. */ static struct svc_cacherep * -nfsd_cache_search(struct nfsd_drc_bucket *b, struct svc_rqst *rqstp, - __wsum csum) +nfsd_cache_insert(struct nfsd_drc_bucket *b, struct svc_cacherep *key) { - struct svc_cacherep *rp, *ret = NULL; + struct svc_cacherep *rp, *ret = key; struct list_head *rh = &b->lru_head; unsigned int entries = 0; list_for_each_entry(rp, rh, c_lru) { ++entries; - if (nfsd_cache_match(rqstp, csum, rp)) { + if (nfsd_cache_match(key, rp)) { ret = rp; break; } @@ -374,6 +384,7 @@ nfsd_cache_search(struct nfsd_drc_bucket *b, struct svc_rqst *rqstp, atomic_read(&num_drc_entries)); } + lru_put_end(b, ret); return ret; } @@ -389,9 +400,6 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) { struct svc_cacherep *rp, *found; __be32 xid = rqstp->rq_xid; - u32 proto = rqstp->rq_prot, - vers = rqstp->rq_vers, - proc = rqstp->rq_proc; __wsum csum; u32 hash = nfsd_cache_hash(xid); struct nfsd_drc_bucket *b = &drc_hashtbl[hash]; @@ -410,52 +418,38 @@ nfsd_cache_lookup(struct svc_rqst *rqstp) * Since the common case is a cache miss followed by an insert, * preallocate an entry. */ - rp = nfsd_reply_cache_alloc(); - spin_lock(&b->cache_lock); - if (likely(rp)) { - atomic_inc(&num_drc_entries); - drc_mem_usage += sizeof(*rp); + rp = nfsd_reply_cache_alloc(rqstp, csum); + if (!rp) { + dprintk("nfsd: unable to allocate DRC entry!\n"); + return rtn; } - /* go ahead and prune the cache */ - prune_bucket(b); - - found = nfsd_cache_search(b, rqstp, csum); - if (found) { - if (likely(rp)) - nfsd_reply_cache_free_locked(rp); + spin_lock(&b->cache_lock); + found = nfsd_cache_insert(b, rp); + if (found != rp) { + nfsd_reply_cache_free_locked(rp); rp = found; goto found_entry; } - if (!rp) { - dprintk("nfsd: unable to allocate DRC entry!\n"); - goto out; - } - nfsdstats.rcmisses++; rqstp->rq_cacherep = rp; rp->c_state = RC_INPROG; - rp->c_xid = xid; - rp->c_proc = proc; - rpc_copy_addr((struct sockaddr *)&rp->c_addr, svc_addr(rqstp)); - rpc_set_port((struct sockaddr *)&rp->c_addr, rpc_get_port(svc_addr(rqstp))); - rp->c_prot = proto; - rp->c_vers = vers; - rp->c_len = rqstp->rq_arg.len; - rp->c_csum = csum; - lru_put_end(b, rp); + atomic_inc(&num_drc_entries); + drc_mem_usage += sizeof(*rp); + + /* go ahead and prune the cache */ + prune_bucket(b); out: spin_unlock(&b->cache_lock); return rtn; found_entry: - nfsdstats.rchits++; /* We found a matching entry which is either in progress or done. */ - lru_put_end(b, rp); - + nfsdstats.rchits++; rtn = RC_DROPIT; + /* Request being processed */ if (rp->c_state == RC_INPROG) goto out;