From patchwork Sun Nov 9 01:15:09 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 5259501 Return-Path: X-Original-To: patchwork-linux-nfs@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1EF509F387 for ; Sun, 9 Nov 2014 01:15:17 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3C27F2010C for ; Sun, 9 Nov 2014 01:15:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4D54420148 for ; Sun, 9 Nov 2014 01:15:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751247AbaKIBPM (ORCPT ); Sat, 8 Nov 2014 20:15:12 -0500 Received: from mail-ie0-f177.google.com ([209.85.223.177]:48804 "EHLO mail-ie0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750962AbaKIBPL (ORCPT ); Sat, 8 Nov 2014 20:15:11 -0500 Received: by mail-ie0-f177.google.com with SMTP id tp5so7081382ieb.8 for ; Sat, 08 Nov 2014 17:15:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=vW43QPkbCJMR2uLIURiQTByMbHzLm79T81eGFR7feAE=; b=YfHez8C59PR+J1vHsDHlX8boMHCDMPb1aEURMgHTh8qbFiTHGkv0Vh5GlNVzZOoHRR qfoA32xtjPVPwW5eX/5/5eK2p1Flk1OdEC/0CNfs+VGvd3YOAXbL6ii45QvFLfQwJRM1 A1jjkull6fVJJEsLLVYx8+/Mo8BH//1xtl3PqhFOgwoK+04LUO2OnYos2044vGFHvDty 3ynbqbySEyQo+xN6pB/EveAusoR9A4NnDHcm1/8oFBy3pUNHN/KoILhPCdFVsdFTQzdW fhHDiF7Y0Ffj72ikl73m6ZBooLxTMUTLUGtlmdRGHDfVhBQXaRC7X8o8gkztvPgKfvqM ecyQ== X-Received: by 10.42.80.143 with SMTP id v15mr27853511ick.54.1415495711060; Sat, 08 Nov 2014 17:15:11 -0800 (PST) Received: from manet.1015granger.net ([2604:8800:100:81fc:82ee:73ff:fe43:d64f]) by mx.google.com with ESMTPSA id th10sm2719858igb.12.2014.11.08.17.15.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 08 Nov 2014 17:15:10 -0800 (PST) Subject: [PATCH v2 08/10] SUNRPC: serialize iostats updates From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Sat, 08 Nov 2014 20:15:09 -0500 Message-ID: <20141109011509.8806.40278.stgit@manet.1015granger.net> In-Reply-To: <20141109010328.8806.5861.stgit@manet.1015granger.net> References: <20141109010328.8806.5861.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-3-g7d0f MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org X-Spam-Status: No, score=-5.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY, URIBL_RHS_DOB autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Occasionally mountstats reports a negative retransmission rate. Ensure that two RPCs completing concurrently don't confuse the sums in the transport's op_metrics array. Since pNFS filelayout can invoke rpc_count_iostats() on another transport from xprt_release(), we can't rely on simply holding the transport_lock in xprt_release(). There's nothing for it but hard serialization. One spin lock per RPC operation should make this as painless as it can be. Signed-off-by: Chuck Lever --- include/linux/sunrpc/metrics.h | 3 +++ net/sunrpc/stats.c | 21 ++++++++++++++++----- 2 files changed, 19 insertions(+), 5 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/include/linux/sunrpc/metrics.h b/include/linux/sunrpc/metrics.h index 1565bbe..eecb5a7 100644 --- a/include/linux/sunrpc/metrics.h +++ b/include/linux/sunrpc/metrics.h @@ -27,10 +27,13 @@ #include #include +#include #define RPC_IOSTATS_VERS "1.0" struct rpc_iostats { + spinlock_t om_lock; + /* * These counters give an idea about how many request * transmissions are required, on average, to complete that diff --git a/net/sunrpc/stats.c b/net/sunrpc/stats.c index 5453049..9711a15 100644 --- a/net/sunrpc/stats.c +++ b/net/sunrpc/stats.c @@ -116,7 +116,15 @@ EXPORT_SYMBOL_GPL(svc_seq_show); */ struct rpc_iostats *rpc_alloc_iostats(struct rpc_clnt *clnt) { - return kcalloc(clnt->cl_maxproc, sizeof(struct rpc_iostats), GFP_KERNEL); + struct rpc_iostats *stats; + int i; + + stats = kcalloc(clnt->cl_maxproc, sizeof(*stats), GFP_KERNEL); + if (stats) { + for (i = 0; i < clnt->cl_maxproc; i++) + spin_lock_init(&stats[i].om_lock); + } + return stats; } EXPORT_SYMBOL_GPL(rpc_alloc_iostats); @@ -135,20 +143,21 @@ EXPORT_SYMBOL_GPL(rpc_free_iostats); * rpc_count_iostats - tally up per-task stats * @task: completed rpc_task * @stats: array of stat structures - * - * Relies on the caller for serialization. */ void rpc_count_iostats(const struct rpc_task *task, struct rpc_iostats *stats) { struct rpc_rqst *req = task->tk_rqstp; struct rpc_iostats *op_metrics; - ktime_t delta; + ktime_t delta, now; if (!stats || !req) return; + now = ktime_get(); op_metrics = &stats[task->tk_msg.rpc_proc->p_statidx]; + spin_lock(&op_metrics->om_lock); + op_metrics->om_ops++; op_metrics->om_ntrans += req->rq_ntrans; op_metrics->om_timeouts += task->tk_timeouts; @@ -161,8 +170,10 @@ void rpc_count_iostats(const struct rpc_task *task, struct rpc_iostats *stats) op_metrics->om_rtt = ktime_add(op_metrics->om_rtt, req->rq_rtt); - delta = ktime_sub(ktime_get(), task->tk_start); + delta = ktime_sub(now, task->tk_start); op_metrics->om_execute = ktime_add(op_metrics->om_execute, delta); + + spin_unlock(&op_metrics->om_lock); } EXPORT_SYMBOL_GPL(rpc_count_iostats);