From patchwork Fri May 1 17:40:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523063 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C583514B4 for ; Fri, 1 May 2020 17:40:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AE50E2173E for ; Fri, 1 May 2020 17:40:35 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YDcgWDcG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729865AbgEARkf (ORCPT ); Fri, 1 May 2020 13:40:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729218AbgEARke (ORCPT ); Fri, 1 May 2020 13:40:34 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A620C061A0C; Fri, 1 May 2020 10:40:34 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id k12so8515433qtm.4; Fri, 01 May 2020 10:40:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=UuL4nrLdfSBwWKtLxiILrceo4WQfjllGgKQTN+8K0+A=; b=YDcgWDcG5jJC5QZRuDIXB0paYUm6sc9z3gOixXklrzrGQ+tU7S9plN9fJYx8Hje9vm mQGRi3jcaWatrP1bWQcZtr68zDiFkQ4++dBzBYCnLyNZ2wdl225eD1BhLSW+rgfLLWoD d9i9UGQo0n+0rOj4jdr3hwPY0s3VQeUBbR5dwT+o04g8Ou3wF09cIHk46sW0tu3bNk/W AAgCrN/g4RfPPuwJOWaapgyL6pdxc8z51zDY5Lk3SJWhdEBRX2WmbDFgqZVNABmWsIwl F2KG26Xv+oMcrfOieQ7pcLjaCVtrLUCqTacpDz4PRmjrw4CIV3s+HBJ6OACQBiHqGIpz FjbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=UuL4nrLdfSBwWKtLxiILrceo4WQfjllGgKQTN+8K0+A=; b=FiLQr5J0c8gcJBsUuyFaD8CWvKiFfW4HlR7z5TLH38NrDdRdRr1SJ91rna/U9Pcyqj mrpQm+OyN7W1JXfICQNF9aZTCcDh6C/CJwUgp0JZzRaQEnQZMixu+8ZmhdskvsxqXc/i 9tRPb903t2zvI3G0Z9z4Ar4YkDjBM8pdVHON/+OIrULpVjNKTpO3Fq+aiDRYMX8jlZxV g0v6x4ab3oPiMWAVlff7xdy+KsNdKXwRyUw1RgpKLy5dKhJiDJB0ruWjWMao7V+LtzBK pkzbvcFn6umnfZMOPHwDcBguk8+PI6xdfAOG4HcI6TjoAsUKg7UdQA996SDQsXQ0e8yH UKvg== X-Gm-Message-State: AGi0PuZOjPwswPW4FlzC/gOzWF5T6gCY31kxJ8CJvVF8eBpeS5X6ZzUG jWdeOuiBe81fiGvvXULNJBbCG2p8 X-Google-Smtp-Source: APiQypKC1JouBDQ8ZhPjze5b5xCMJJUSf0ktw/PlCASlMqYls2fdPviTwx4MjnX8E98+CwoYSm1jSw== X-Received: by 2002:ac8:3254:: with SMTP id y20mr4984778qta.67.1588354833437; Fri, 01 May 2020 10:40:33 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id q62sm2036503qke.22.2020.05.01.10.40.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:33 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HeVS3026765; Fri, 1 May 2020 17:40:31 GMT Subject: [PATCH v1 1/7] svcrdma: Clean up the tracing for rw_ctx_init errors From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:31 -0400 Message-ID: <20200501174031.3899.64778.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org - De-duplicate code - Rename the tracepoint with "_err" to allow enabling via glob - Report the sg_cnt for the failing rw_ctx - Replace one use of xpt_remotebuf - Fix a dumb signage issue Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 12 +++++--- net/sunrpc/xprtrdma/svc_rdma_rw.c | 56 +++++++++++++++++++++++-------------- 2 files changed, 43 insertions(+), 25 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index c25e11564598..72dc9f6146fb 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1591,28 +1591,32 @@ DECLARE_EVENT_CLASS(svcrdma_dma_map_class, DEFINE_SVC_DMA_EVENT(dma_map_page); DEFINE_SVC_DMA_EVENT(dma_unmap_page); -TRACE_EVENT(svcrdma_dma_map_rwctx, +TRACE_EVENT(svcrdma_dma_map_rw_err, TP_PROTO( const struct svcxprt_rdma *rdma, + unsigned int nents, int status ), - TP_ARGS(rdma, status), + TP_ARGS(rdma, nents, status), TP_STRUCT__entry( __field(int, status) + __field(unsigned int, nents) __string(device, rdma->sc_cm_id->device->name) __string(addr, rdma->sc_xprt.xpt_remotebuf) ), TP_fast_assign( __entry->status = status; + __entry->nents = nents; __assign_str(device, rdma->sc_cm_id->device->name); __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); ), - TP_printk("addr=%s device=%s status=%d", - __get_str(addr), __get_str(device), __entry->status + TP_printk("addr=%s device=%s nents=%u status=%d", + __get_str(addr), __get_str(device), __entry->nents, + __entry->status ) ); diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 23c2d3ce0dc9..db70709e165a 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -39,7 +39,7 @@ static void svc_rdma_wc_read_done(struct ib_cq *cq, struct ib_wc *wc); struct svc_rdma_rw_ctxt { struct list_head rw_list; struct rdma_rw_ctx rw_ctx; - int rw_nents; + unsigned int rw_nents; struct sg_table rw_sg_table; struct scatterlist rw_first_sgl[]; }; @@ -107,6 +107,34 @@ void svc_rdma_destroy_rw_ctxts(struct svcxprt_rdma *rdma) } } +/** + * svc_rdma_rw_ctx_init - Prepare a R/W context for I/O + * @rdma: controlling transport instance + * @ctxt: R/W context to prepare + * @offset: RDMA offset + * @handle: RDMA tag/handle + * @direction: I/O direction + * + * Returns on success, the number of WQEs that will be needed + * on the workqueue, or a negative errno. + */ +static int svc_rdma_rw_ctx_init(struct svcxprt_rdma *rdma, + struct svc_rdma_rw_ctxt *ctxt, + u64 offset, u32 handle, + enum dma_data_direction direction) +{ + int ret; + + ret = rdma_rw_ctx_init(&ctxt->rw_ctx, rdma->sc_qp, rdma->sc_port_num, + ctxt->rw_sg_table.sgl, ctxt->rw_nents, + 0, offset, handle, direction); + if (unlikely(ret < 0)) { + svc_rdma_put_rw_ctxt(rdma, ctxt); + trace_svcrdma_dma_map_rw_err(rdma, ctxt->rw_nents, ret); + } + return ret; +} + /* A chunk context tracks all I/O for moving one Read or Write * chunk. This is a a set of rdma_rw's that handle data movement * for all segments of one chunk. @@ -431,12 +459,10 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, goto out_noctx; constructor(info, write_len, ctxt); - ret = rdma_rw_ctx_init(&ctxt->rw_ctx, rdma->sc_qp, - rdma->sc_port_num, ctxt->rw_sg_table.sgl, - ctxt->rw_nents, 0, seg_offset, - seg_handle, DMA_TO_DEVICE); + ret = svc_rdma_rw_ctx_init(rdma, ctxt, seg_offset, seg_handle, + DMA_TO_DEVICE); if (ret < 0) - goto out_initerr; + return -EIO; trace_svcrdma_send_wseg(seg_handle, write_len, seg_offset); @@ -462,11 +488,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, out_noctx: dprintk("svcrdma: no R/W ctxs available\n"); return -ENOMEM; - -out_initerr: - svc_rdma_put_rw_ctxt(rdma, ctxt); - trace_svcrdma_dma_map_rwctx(rdma, ret); - return -EIO; } /* Send one of an xdr_buf's kvecs by itself. To send a Reply @@ -646,12 +667,10 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, goto out_overrun; } - ret = rdma_rw_ctx_init(&ctxt->rw_ctx, cc->cc_rdma->sc_qp, - cc->cc_rdma->sc_port_num, - ctxt->rw_sg_table.sgl, ctxt->rw_nents, - 0, offset, rkey, DMA_FROM_DEVICE); + ret = svc_rdma_rw_ctx_init(cc->cc_rdma, ctxt, offset, rkey, + DMA_FROM_DEVICE); if (ret < 0) - goto out_initerr; + return -EIO; list_add(&ctxt->rw_list, &cc->cc_rwctxts); cc->cc_sqecount += ret; @@ -664,11 +683,6 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, out_overrun: dprintk("svcrdma: request overruns rq_pages\n"); return -EINVAL; - -out_initerr: - trace_svcrdma_dma_map_rwctx(cc->cc_rdma, ret); - svc_rdma_put_rw_ctxt(cc->cc_rdma, ctxt); - return -EIO; } /* Walk the segments in the Read chunk starting at @p and construct From patchwork Fri May 1 17:40:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523065 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2858581 for ; Fri, 1 May 2020 17:40:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0B4F624953 for ; Fri, 1 May 2020 17:40:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mXYMmzeE" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729994AbgEARkk (ORCPT ); Fri, 1 May 2020 13:40:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33768 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARkj (ORCPT ); Fri, 1 May 2020 13:40:39 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C73E4C061A0C; Fri, 1 May 2020 10:40:39 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id y19so5092881qvv.4; Fri, 01 May 2020 10:40:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=9C4PcF/usiCJ5I83M8hMFfyUfv897E3z7ZSz4cgOCbs=; b=mXYMmzeERNOAPHg/vR6dpSm1IVHzwIZCBQF1VUmpjfaSwnjaLVQpGhE6xS4B2b4Iyw LgTm0LFjgbqcVE4hpLATC6ns/zevxt1K23NYXZ+MeqW+iHfvq1aHlNucKoMvMvUJ5cET dAxEyckVYUfL+tczoL/KCovGrYxt2a8S4rU47EsjpTxOUugwUjc9IKyQq7eY5gXr9MZ/ oIp04NB1Czrvc5yWHVoRryOyBpZWddRof2ZOT8egsDwFqaCa64kRziLTvqabBxxit47i SCp/aDKSZwXyT6WxC+chA8DJK/5+fXsSHOMeAHpWZ2pCo5cG5quVlepCtgocY+nZS25K uwAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=9C4PcF/usiCJ5I83M8hMFfyUfv897E3z7ZSz4cgOCbs=; b=NuA3kV5bb0LUzODAG4ZXHDjc34s35FU0RHkzxzQQNvptxfn6C+BPy+UxaSoe6CkW2E umvgoxchsUJLV3bG6jldvtcrcTQ7vyxrC00nBEZNoA2ukJPGGHBFs2JJzBhc6TEbHvbT ajavtGWBK3ZkCugDzmCGunjEd8en3VmQIpdS8Ovne5lVa0/2zbXs/73bwbS1MV01ocOd gp6atm0rLNwZ2YVZ2Z1ExmiHHM63825opYxTrhuQ9UN5JyWHDGaEW6fhlOX9mrqHavb1 v7Zm1AYb3UUNprhziNzPrdCwSnJ+hPyvAxAcY+B4OfTnyiPtdNGFHQDiq0jHxLHRrjkP Yf0g== X-Gm-Message-State: AGi0PuYtjdlxjrUWnqOIUVRkdumE/Ei83XiJtUXqPpdV2njgyS9qwgYu 6MgFcja00UdPxEt49Z5xFBfv+oJN X-Google-Smtp-Source: APiQypJihBRjJ2oXYOxdu2OatOrP+A+qHl8Tbzd+UF4bC7PBkOm3WvyKfYqOMvdpdXtN1ZtW8BQmBw== X-Received: by 2002:a0c:b659:: with SMTP id q25mr5152946qvf.48.1588354838880; Fri, 01 May 2020 10:40:38 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id p2sm3162256qkm.65.2020.05.01.10.40.38 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:38 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HebBQ026768; Fri, 1 May 2020 17:40:37 GMT Subject: [PATCH v1 2/7] svcrdma: Clean up handling of get_rw_ctx errors From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:37 -0400 Message-ID: <20200501174037.3899.88600.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Replace two dprintk call sites with a tracepoint. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 25 +++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 27 +++++++++++---------------- 2 files changed, 36 insertions(+), 16 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 72dc9f6146fb..a2fc5f3fb7d9 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1620,6 +1620,31 @@ TRACE_EVENT(svcrdma_dma_map_rw_err, ) ); +TRACE_EVENT(svcrdma_no_rwctx_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + unsigned int num_sges + ), + + TP_ARGS(rdma, num_sges), + + TP_STRUCT__entry( + __field(unsigned int, num_sges) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->num_sges = num_sges; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s num_sges=%d", + __get_str(addr), __get_str(device), __entry->num_sges + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index db70709e165a..c2d49f607cfe 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -67,19 +67,22 @@ svc_rdma_get_rw_ctxt(struct svcxprt_rdma *rdma, unsigned int sges) ctxt = kmalloc(struct_size(ctxt, rw_first_sgl, SG_CHUNK_SIZE), GFP_KERNEL); if (!ctxt) - goto out; + goto out_noctx; INIT_LIST_HEAD(&ctxt->rw_list); } ctxt->rw_sg_table.sgl = ctxt->rw_first_sgl; if (sg_alloc_table_chained(&ctxt->rw_sg_table, sges, ctxt->rw_sg_table.sgl, - SG_CHUNK_SIZE)) { - kfree(ctxt); - ctxt = NULL; - } -out: + SG_CHUNK_SIZE)) + goto out_free; return ctxt; + +out_free: + kfree(ctxt); +out_noctx: + trace_svcrdma_no_rwctx_err(rdma, sges); + return NULL; } static void svc_rdma_put_rw_ctxt(struct svcxprt_rdma *rdma, @@ -456,7 +459,7 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, ctxt = svc_rdma_get_rw_ctxt(rdma, (write_len >> PAGE_SHIFT) + 2); if (!ctxt) - goto out_noctx; + return -ENOMEM; constructor(info, write_len, ctxt); ret = svc_rdma_rw_ctx_init(rdma, ctxt, seg_offset, seg_handle, @@ -484,10 +487,6 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, dprintk("svcrdma: inadequate space in Write chunk (%u)\n", info->wi_nsegs); return -E2BIG; - -out_noctx: - dprintk("svcrdma: no R/W ctxs available\n"); - return -ENOMEM; } /* Send one of an xdr_buf's kvecs by itself. To send a Reply @@ -637,7 +636,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, sge_no = PAGE_ALIGN(info->ri_pageoff + len) >> PAGE_SHIFT; ctxt = svc_rdma_get_rw_ctxt(cc->cc_rdma, sge_no); if (!ctxt) - goto out_noctx; + return -ENOMEM; ctxt->rw_nents = sge_no; sg = ctxt->rw_sg_table.sgl; @@ -676,10 +675,6 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, cc->cc_sqecount += ret; return 0; -out_noctx: - dprintk("svcrdma: no R/W ctxs available\n"); - return -ENOMEM; - out_overrun: dprintk("svcrdma: request overruns rq_pages\n"); return -EINVAL; From patchwork Fri May 1 17:40:42 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523069 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1BEEA913 for ; Fri, 1 May 2020 17:40:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 040E024953 for ; Fri, 1 May 2020 17:40:46 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Nf8KUTFi" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729782AbgEARkp (ORCPT ); Fri, 1 May 2020 13:40:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARkp (ORCPT ); Fri, 1 May 2020 13:40:45 -0400 Received: from mail-qk1-x742.google.com (mail-qk1-x742.google.com [IPv6:2607:f8b0:4864:20::742]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E47C4C061A0C; Fri, 1 May 2020 10:40:44 -0700 (PDT) Received: by mail-qk1-x742.google.com with SMTP id b188so9922605qkd.9; Fri, 01 May 2020 10:40:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=bQ1GsoV6mL/QdiU1sTMcePB/ArTRzgDahu4Tj8VCGOE=; b=Nf8KUTFi3sMZDpmkQ0JpH+ureZnyfoeOr2VmDuOuy8FHyI1WiEZMPuwDIE490T+7Q2 5QAC+3sthE6+WHkXbjgVz8TJcVxzbUBpFRniY1LmIZVg8A4+/oNWUokXSs2VkogVy3y4 rbjwdaAtC2zxEXrfLRXDbzegVLoDUT0Re3luHmpJzHyMxepRAKiXNU/J+ozQx8ZszQnZ lVmltOlYUuSNDSAC7RXJI35a1pM3PLkBwqRpJCjY1zgGpfXIIs/c4dzZXVURa2v8l+0q yMKWUQcI5qF95u8rJQhkoh+GQDNJHeEaeAOMuhLUudkj/DKCb1bzNOfgSCw+52QZRu29 fl2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=bQ1GsoV6mL/QdiU1sTMcePB/ArTRzgDahu4Tj8VCGOE=; b=JeekNOYB0wWGU70fGVs32QZbbkIWp6Fw+I08PTTBG8ueJN9TxIZQHtvnLJsAkKQiK0 FQlsf6bJCz6NleV365McTJmQk+mtVTxdBfiVaOGFoWI5Joo2JvmK9LpO2NErjGorKjS/ R8IWasQcgNy8frP9QceDcgc2SIiq9mrDOh6TwyPtDu8dtdt2yAwYmY7ctxc6MJ1XGSQY DVClLHzbYMCGHx6QqhbXciYkIvYnMDCMTvJugU++6Zyrg2wzFscBejcymXkbSZ5LXySb BFghfmu2mq02aYUXbuP+l1eNiYsml5KKrRjJYa5MRAkOTm9QmGoaR8kewHIrlpnlYCNA p3zA== X-Gm-Message-State: AGi0PuY7Z0tJ+uN76VcxipkZp7YZb2Cgm2VjdUlmug+EXMQtbpj/8JIE wMQmuAZfW2rrLXM56PsQ4FcjPums X-Google-Smtp-Source: APiQypKHncCoHbHCRfh4b+u+AbhQUu4PAyuOdkpEAq+c995Ai5xD6IhiTP0u3y3vIcrcZaMyljESJg== X-Received: by 2002:a05:620a:13e1:: with SMTP id h1mr4750796qkl.10.1588354844058; Fri, 01 May 2020 10:40:44 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id o43sm3066347qtc.23.2020.05.01.10.40.43 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:43 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HegEv026771; Fri, 1 May 2020 17:40:42 GMT Subject: [PATCH v1 3/7] svcrdma: Trace page overruns when constructing RDMA Reads From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:42 -0400 Message-ID: <20200501174042.3899.51707.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Replace a dprintk call site with a tracepoint. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 28 ++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 2 +- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index a2fc5f3fb7d9..15dc1e852a0c 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1645,6 +1645,34 @@ TRACE_EVENT(svcrdma_no_rwctx_err, ) ); +TRACE_EVENT(svcrdma_page_overrun_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + const struct svc_rqst *rqst, + unsigned int pageno + ), + + TP_ARGS(rdma, rqst, pageno), + + TP_STRUCT__entry( + __field(unsigned int, pageno) + __field(u32, xid) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->pageno = pageno; + __entry->xid = __be32_to_cpu(rqst->rq_xid); + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s xid=0x%08x pageno=%u", __get_str(addr), + __get_str(device), __entry->xid, __entry->pageno + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index c2d49f607cfe..17098a11d2ad 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -676,7 +676,7 @@ static int svc_rdma_build_read_segment(struct svc_rdma_read_info *info, return 0; out_overrun: - dprintk("svcrdma: request overruns rq_pages\n"); + trace_svcrdma_page_overrun_err(cc->cc_rdma, rqstp, info->ri_pageno); return -EINVAL; } From patchwork Fri May 1 17:40:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523073 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B4ED181 for ; Fri, 1 May 2020 17:40:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9D68C24953 for ; Fri, 1 May 2020 17:40:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="akGLCr5J" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730267AbgEARkv (ORCPT ); Fri, 1 May 2020 13:40:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33798 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARku (ORCPT ); Fri, 1 May 2020 13:40:50 -0400 Received: from mail-qv1-xf42.google.com (mail-qv1-xf42.google.com [IPv6:2607:f8b0:4864:20::f42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56916C061A0C; Fri, 1 May 2020 10:40:50 -0700 (PDT) Received: by mail-qv1-xf42.google.com with SMTP id q2so5086616qvd.1; Fri, 01 May 2020 10:40:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=3Dub4iiczZWPgOci9+rLFCeI0KKy+abSYTnIWDORjBI=; b=akGLCr5Jer31o1RtQ/54wWkGoWTROmSluVJdJCeeLtwFDK7HeHAyeEGwtqCwY6+WHv ZW1OrKwiy+YXDAQK5jbl3adJkkQeuIG+1twkSFHAHe6D1hgp5Gf6iRo+31RGhVFGK6SZ DAPdwL6lWdo7amsga+2tfEk2vdluOGQHBZXKlWJ7/gyUfl+NefZuIiNqh//P9WgiTizl ZKo+jiUmv69IpKMoAKUAKV3qXKVQmu4SR7YYI1+T0AHOCQiry6kJqgFYkgEATxKIhgjW LvrA7cIHk6nUNFJmD+DI0LRq4om27iOfUNyTTXFX1lRE0/eaY+wR1JMa66rMetc7Src3 XTxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=3Dub4iiczZWPgOci9+rLFCeI0KKy+abSYTnIWDORjBI=; b=IcQbei2Lypqs74Qd0DBOsjvYG1kzAdUr9o4+2rL/41GvuSV2K/2H90Kw53z9KR54dQ 9KJDwYJzCX/V4QsCRvboiW+uK5WdZwvHRB3pSWxNrIdMl0fxiI9cdHm4GgU3VaS/K4tn jyUYhMpv4mi/hfxWqgoZukg0ELgkMGiYwnAgdMW0dtwcbCay4+ClK6O5ZBKOcIKpdRYb XrBejerweKgvsUcVFAiuSRP2bM5UgXw3VOtUHrIGqujS5zr906JXHs8TUT3R61h0BKtO UutpBzoquAjy29aQazEN11T+XV2UMm7WiaU6PY1O4k9YcvrwgvgigouiBZ4/AzaBdWog emXw== X-Gm-Message-State: AGi0PuakWFTxFk34X9P9FfL0kL5xhoMEdlfMQFtk/DTDHMESvgkO2jT5 SMl29jv8ApeorJoP5uqdmWbznusR X-Google-Smtp-Source: APiQypJxiME2jfh8PT6j62BqeuOrTGrnGdV/kO6b6rBYf3ls22OXrTM97rM60mOUB4Yny3OqQdlDhA== X-Received: by 2002:ad4:54c3:: with SMTP id j3mr5131857qvx.241.1588354849456; Fri, 01 May 2020 10:40:49 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id h13sm3134843qti.32.2020.05.01.10.40.48 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:48 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HelKU026774; Fri, 1 May 2020 17:40:47 GMT Subject: [PATCH v1 4/7] svcrdma: trace undersized Write chunks From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:47 -0400 Message-ID: <20200501174047.3899.79731.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: replace a dprintk call site This is the last remaining dprintk call site in svc_rdma_rw.c, so remove the dprintk infrastructure as well. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 32 ++++++++++++++++++++++++++++++++ net/sunrpc/xprtrdma/svc_rdma_rw.c | 7 ++----- 2 files changed, 34 insertions(+), 5 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 15dc1e852a0c..3390dd12a8dc 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1673,6 +1673,38 @@ TRACE_EVENT(svcrdma_page_overrun_err, ) ); +TRACE_EVENT(svcrdma_small_wrch_err, + TP_PROTO( + const struct svcxprt_rdma *rdma, + unsigned int remaining, + unsigned int seg_no, + unsigned int num_segs + ), + + TP_ARGS(rdma, remaining, seg_no, num_segs), + + TP_STRUCT__entry( + __field(unsigned int, remaining) + __field(unsigned int, seg_no) + __field(unsigned int, num_segs) + __string(device, rdma->sc_cm_id->device->name) + __string(addr, rdma->sc_xprt.xpt_remotebuf) + ), + + TP_fast_assign( + __entry->remaining = remaining; + __entry->seg_no = seg_no; + __entry->num_segs = num_segs; + __assign_str(device, rdma->sc_cm_id->device->name); + __assign_str(addr, rdma->sc_xprt.xpt_remotebuf); + ), + + TP_printk("addr=%s device=%s remaining=%u seg_no=%u num_segs=%u", + __get_str(addr), __get_str(device), __entry->remaining, + __entry->seg_no, __entry->num_segs + ) +); + TRACE_EVENT(svcrdma_send_pullup, TP_PROTO( unsigned int len diff --git a/net/sunrpc/xprtrdma/svc_rdma_rw.c b/net/sunrpc/xprtrdma/svc_rdma_rw.c index 17098a11d2ad..5eb35309ecef 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_rw.c +++ b/net/sunrpc/xprtrdma/svc_rdma_rw.c @@ -9,13 +9,10 @@ #include #include -#include #include "xprt_rdma.h" #include -#define RPCDBG_FACILITY RPCDBG_SVCXPRT - static void svc_rdma_write_done(struct ib_cq *cq, struct ib_wc *wc); static void svc_rdma_wc_read_done(struct ib_cq *cq, struct ib_wc *wc); @@ -484,8 +481,8 @@ svc_rdma_build_writes(struct svc_rdma_write_info *info, return 0; out_overflow: - dprintk("svcrdma: inadequate space in Write chunk (%u)\n", - info->wi_nsegs); + trace_svcrdma_small_wrch_err(rdma, remaining, info->wi_seg_no, + info->wi_nsegs); return -E2BIG; } From patchwork Fri May 1 17:40:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523077 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 09D9E81 for ; Fri, 1 May 2020 17:40:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E095924953 for ; Fri, 1 May 2020 17:40:56 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="bHPIDmZJ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730318AbgEARk4 (ORCPT ); Fri, 1 May 2020 13:40:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33816 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARkz (ORCPT ); Fri, 1 May 2020 13:40:55 -0400 Received: from mail-qt1-x844.google.com (mail-qt1-x844.google.com [IPv6:2607:f8b0:4864:20::844]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AD7B2C061A0C; Fri, 1 May 2020 10:40:55 -0700 (PDT) Received: by mail-qt1-x844.google.com with SMTP id v26so8515005qto.0; Fri, 01 May 2020 10:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=lCCExvyODMO0v1Z3g8fcgjDrSeXwtuhz60/Y3Fx4pvQ=; b=bHPIDmZJTJ23hGc8AflcZeNTv7Px8ZiYvWekxFSSHpqe0aV1xDqkcxJ28jWddJm8gk /qmAJ5muk0lEDeo9ynqVJmAZSEQ05YpXMHezNNVzEQV+uK2+vQE333NXq3WctMnvMObg +oVxTDzmaFRb0qlWNoq1YaR6cLOMpDpF/MFq1DvXepd8Tq3RIAdWeb0uyNtAflQGTp3E g8lcpGFhzJ44aEYNkGtCFZOeKwa8YkMr7JBV7vtW0uDB4+wuKzYjHAEwFepX6Y2B+sNn f+D/VdU0py5E8lb9OYjKjg1xGpgVBCDvWJnN3pHBaWInT8qHKLSXYF8zM0jxXT3HefwS e6VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=lCCExvyODMO0v1Z3g8fcgjDrSeXwtuhz60/Y3Fx4pvQ=; b=b2X7dkWrh7MuDRQ2oTzxisb0M6H6tZ550IgReTN9KAGONJXR08M5Sb7QExBm41xyEe oW8/Wwng7iDulbqrYkzI314kPAxdLJb+UvHC0r+dHYq+K6YXZjT1jQIIv/WTt/7jSiV+ YnKm7Ox9oyM7G2QwGvmT3YcimU9Om3mii0HbQwlIKRNe+Vz4cauOSfa5ZJaxVhyWGgM5 c1pjfeuk2rQ6QuaNY/2g1uk/JFa5rjiOADx7GLGMeqIu/5LN1GxM1nJcLFcnma+8BGgD DRdX/jyFebizufNxslPhMBH4nzdUJOADI4oPKBEfoCrJ9Bqw5Re4GxrDXGEtuLiG7JKD lUVg== X-Gm-Message-State: AGi0Puaq2DgaoM+hWooHqqhvO/+5/6I74LGOd6/5Nfa2QbSUg7US1jKB IkWFznB1QPpzhVlqGB6JmOjhH9Xi X-Google-Smtp-Source: APiQypJR+pqO+Vw7UC8fAgSo3MM2c4Nj4ptVlMbQYYuEYCSr42yZkV8MITLUQbjaiKSrmeD9lPCCpA== X-Received: by 2002:ac8:1ad1:: with SMTP id h17mr5061098qtk.9.1588354854729; Fri, 01 May 2020 10:40:54 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id m26sm3195717qta.53.2020.05.01.10.40.54 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:54 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041Herci026777; Fri, 1 May 2020 17:40:53 GMT Subject: [PATCH v1 5/7] svcrdma: Fix backchannel return code From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:53 -0400 Message-ID: <20200501174053.3899.37683.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Way back when I was writing the RPC/RDMA server-side backchannel code, I misread the TCP backchannel reply handler logic. When svc_tcp_recvfrom() successfully receives a backchannel reply, it does not return -EAGAIN. It sets XPT_DATA and returns zero. Update svc_rdma_recvfrom() to return zero. Here, XPT_DATA doesn't need to be set again: it is set whenever a new message is received, behind a spin lock in a single threaded context. Also, if handling the cb reply is not successful, the message is simply dropped. There's no special message framing to deal with as there is in the TCP case. Now that the handle_bc_reply() return value is ignored, I've removed the dprintk call sites in the error exit of handle_bc_reply() in favor of trace points in other areas that already report the error cases. Signed-off-by: Chuck Lever --- include/linux/sunrpc/svc_rdma.h | 5 +--- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 38 +++++++--------------------- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 11 ++++---- 3 files changed, 17 insertions(+), 37 deletions(-) diff --git a/include/linux/sunrpc/svc_rdma.h b/include/linux/sunrpc/svc_rdma.h index cbcfbd0521e3..8518c3f37e56 100644 --- a/include/linux/sunrpc/svc_rdma.h +++ b/include/linux/sunrpc/svc_rdma.h @@ -160,9 +160,8 @@ struct svc_rdma_send_ctxt { }; /* svc_rdma_backchannel.c */ -extern int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, - __be32 *rdma_resp, - struct xdr_buf *rcvbuf); +extern void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *rctxt); /* svc_rdma_recvfrom.c */ extern void svc_rdma_recv_ctxts_destroy(struct svcxprt_rdma *rdma); diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index d9aab4504f2c..b174f3b109a5 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -15,26 +15,25 @@ #undef SVCRDMA_BACKCHANNEL_DEBUG /** - * svc_rdma_handle_bc_reply - Process incoming backchannel reply - * @xprt: controlling backchannel transport - * @rdma_resp: pointer to incoming transport header - * @rcvbuf: XDR buffer into which to decode the reply + * svc_rdma_handle_bc_reply - Process incoming backchannel Reply + * @rqstp: resources for handling the Reply + * @rctxt: Received message * - * Returns: - * %0 if @rcvbuf is filled in, xprt_complete_rqst called, - * %-EAGAIN if server should call ->recvfrom again. */ -int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, - struct xdr_buf *rcvbuf) +void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, + struct svc_rdma_recv_ctxt *rctxt) { + struct svc_xprt *sxprt = rqstp->rq_xprt; + struct rpc_xprt *xprt = sxprt->xpt_bc_xprt; struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt); + struct xdr_buf *rcvbuf = &rqstp->rq_arg; struct kvec *dst, *src = &rcvbuf->head[0]; + __be32 *rdma_resp = rctxt->rc_recv_buf; struct rpc_rqst *req; u32 credits; size_t len; __be32 xid; __be32 *p; - int ret; p = (__be32 *)src->iov_base; len = src->iov_len; @@ -49,14 +48,10 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, __func__, (int)len, p); #endif - ret = -EAGAIN; - if (src->iov_len < 24) - goto out_shortreply; - spin_lock(&xprt->queue_lock); req = xprt_lookup_rqst(xprt, xid); if (!req) - goto out_notfound; + goto out_unlock; dst = &req->rq_private_buf.head[0]; memcpy(&req->rq_private_buf, &req->rq_rcv_buf, sizeof(struct xdr_buf)); @@ -77,25 +72,12 @@ int svc_rdma_handle_bc_reply(struct rpc_xprt *xprt, __be32 *rdma_resp, spin_unlock(&xprt->transport_lock); spin_lock(&xprt->queue_lock); - ret = 0; xprt_complete_rqst(req->rq_task, rcvbuf->len); xprt_unpin_rqst(req); rcvbuf->len = 0; out_unlock: spin_unlock(&xprt->queue_lock); -out: - return ret; - -out_shortreply: - dprintk("svcrdma: short bc reply: xprt=%p, len=%zu\n", - xprt, src->iov_len); - goto out; - -out_notfound: - dprintk("svcrdma: unrecognized bc reply: xprt=%p, xid=%08x\n", - xprt, be32_to_cpu(xid)); - goto out_unlock; } /* Send a backwards direction RPC call. diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index efa5fcb5793f..eee7c6478b30 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -878,12 +878,9 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) goto out_drop; rqstp->rq_xprt_hlen = ret; - if (svc_rdma_is_backchannel_reply(xprt, p)) { - ret = svc_rdma_handle_bc_reply(xprt->xpt_bc_xprt, p, - &rqstp->rq_arg); - svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); - return ret; - } + if (svc_rdma_is_backchannel_reply(xprt, p)) + goto out_backchannel; + svc_rdma_get_inv_rkey(rdma_xprt, ctxt); p += rpcrdma_fixed_maxsz; @@ -913,6 +910,8 @@ int svc_rdma_recvfrom(struct svc_rqst *rqstp) svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return ret; +out_backchannel: + svc_rdma_handle_bc_reply(rqstp, ctxt); out_drop: svc_rdma_recv_ctxt_put(rdma_xprt, ctxt); return 0; From patchwork Fri May 1 17:40:58 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523081 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2F57D81 for ; Fri, 1 May 2020 17:41:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1790224953 for ; Fri, 1 May 2020 17:41:02 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="U49FcE/u" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730347AbgEARlB (ORCPT ); Fri, 1 May 2020 13:41:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33834 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARlB (ORCPT ); Fri, 1 May 2020 13:41:01 -0400 Received: from mail-qv1-xf44.google.com (mail-qv1-xf44.google.com [IPv6:2607:f8b0:4864:20::f44]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E3056C061A0C; Fri, 1 May 2020 10:41:00 -0700 (PDT) Received: by mail-qv1-xf44.google.com with SMTP id fb4so5082984qvb.7; Fri, 01 May 2020 10:41:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=RdAX8caYJ0iXy3SCEZHENL/3iYUYiev43YHvBxvtV8I=; b=U49FcE/uMEaOm5bS2c3tdknrgPqtk9hz1umngR8R1649u/gMVLNPyJVOvA2Zbge8Hd ndFzmlhdfkFijBiqsLVDb3kTx8J9Fqh1bBLKX0IjmGaEfv4+x683Er9Ug5qJzhVNO+V6 cBlKcYua97vZOZRNgIyEBMEzx5uClzLVRNBBEk/WZ4azo/+TJAogj8SdT29YwqTmooHN AWoG9hJtcQXPFrnkPC3Jum1WlbnmbiyAjEQ6dTxG/w02CrKILT1jcZ9fe7vMn2odCEW8 3+WZnr014MmMfrukmDAxaOtwjjKn4ZaFnKsbdTM3jc/qdBSs9wt9LpkuNdmd4ny3XsV9 /feg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=RdAX8caYJ0iXy3SCEZHENL/3iYUYiev43YHvBxvtV8I=; b=ngMdJcrS4Yuu38BWEjF6Wabtv2toukCLAXaBsK0V80Tiu1cjGOzkOQVIeG1FttXled hCTWugMDbPie+tBOL/CLKcVUl59++KmB4sLVNcQZIRtLSvHUrS8qMYwm+9DnUcoY8qkt DSQpjqe6xCUZkgK+KIBOyZMY+SCX2TO5kvzsAkV7e0+obdKVyVmVb4mPwvKaF2yKTELi lvc3NNZp8K1KWtfIwj7q/c12mMu+uzOA/lXmVxZ6x/qc+1XTMvyYOIw0rrYYJsdPmRbt JVBGJYoSVW7ewvMdc5SM5EBU2MKvJsHiQs0Z9V09jJ7aP5RC3NpchdD2j7QzUpumSt6m S5kA== X-Gm-Message-State: AGi0PuY9o7iwMdd/m3bcgMWPp3iH+xQ1Q5yfFA0sKkro4TPAerCq9I02 kkBto+8SDomvZpdvLBPEOKvJ1wTZ X-Google-Smtp-Source: APiQypL/SiUpoKaEr/Q9PHYNBA6A0wZoHp/PGxGp5xLTLINpGYoFF+KvqyDEXmEC9Ybow+ak2vQmig== X-Received: by 2002:a0c:f84b:: with SMTP id g11mr4770792qvo.155.1588354859981; Fri, 01 May 2020 10:40:59 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id t14sm3076446qtj.71.2020.05.01.10.40.59 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:40:59 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041HewV3026780; Fri, 1 May 2020 17:40:58 GMT Subject: [PATCH v1 6/7] svcrdma: Remove backchannel dprintk call sites From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:40:58 -0400 Message-ID: <20200501174058.3899.54831.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 48 +++------------------------- 1 file changed, 5 insertions(+), 43 deletions(-) diff --git a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c index b174f3b109a5..1ee73f7cf931 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_backchannel.c +++ b/net/sunrpc/xprtrdma/svc_rdma_backchannel.c @@ -10,10 +10,6 @@ #include "xprt_rdma.h" #include -#define RPCDBG_FACILITY RPCDBG_SVCXPRT - -#undef SVCRDMA_BACKCHANNEL_DEBUG - /** * svc_rdma_handle_bc_reply - Process incoming backchannel Reply * @rqstp: resources for handling the Reply @@ -31,33 +27,17 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, __be32 *rdma_resp = rctxt->rc_recv_buf; struct rpc_rqst *req; u32 credits; - size_t len; - __be32 xid; - __be32 *p; - - p = (__be32 *)src->iov_base; - len = src->iov_len; - xid = *rdma_resp; - -#ifdef SVCRDMA_BACKCHANNEL_DEBUG - pr_info("%s: xid=%08x, length=%zu\n", - __func__, be32_to_cpu(xid), len); - pr_info("%s: RPC/RDMA: %*ph\n", - __func__, (int)RPCRDMA_HDRLEN_MIN, rdma_resp); - pr_info("%s: RPC: %*ph\n", - __func__, (int)len, p); -#endif spin_lock(&xprt->queue_lock); - req = xprt_lookup_rqst(xprt, xid); + req = xprt_lookup_rqst(xprt, *rdma_resp); if (!req) goto out_unlock; dst = &req->rq_private_buf.head[0]; memcpy(&req->rq_private_buf, &req->rq_rcv_buf, sizeof(struct xdr_buf)); - if (dst->iov_len < len) + if (dst->iov_len < src->iov_len) goto out_unlock; - memcpy(dst->iov_base, p, len); + memcpy(dst->iov_base, src->iov_base, src->iov_len); xprt_pin_rqst(req); spin_unlock(&xprt->queue_lock); @@ -66,7 +46,6 @@ void svc_rdma_handle_bc_reply(struct svc_rqst *rqstp, credits = 1; /* don't deadlock */ else if (credits > r_xprt->rx_buf.rb_bc_max_requests) credits = r_xprt->rx_buf.rb_bc_max_requests; - spin_lock(&xprt->transport_lock); xprt->cwnd = credits << RPC_CWNDSHIFT; spin_unlock(&xprt->transport_lock); @@ -174,10 +153,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) *p++ = xdr_zero; *p = xdr_zero; -#ifdef SVCRDMA_BACKCHANNEL_DEBUG - pr_info("%s: %*ph\n", __func__, 64, rqst->rq_buffer); -#endif - rqst->rq_xtime = ktime_get(); rc = svc_rdma_bc_sendto(rdma, rqst, ctxt); if (rc) @@ -188,7 +163,6 @@ rpcrdma_bc_send_request(struct svcxprt_rdma *rdma, struct rpc_rqst *rqst) svc_rdma_send_ctxt_put(rdma, ctxt); drop_connection: - dprintk("svcrdma: failed to send bc call\n"); return -ENOTCONN; } @@ -207,9 +181,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) container_of(sxprt, struct svcxprt_rdma, sc_xprt); int ret; - dprintk("svcrdma: sending bc call with xid: %08x\n", - be32_to_cpu(rqst->rq_xid)); - if (test_bit(XPT_DEAD, &sxprt->xpt_flags)) return -ENOTCONN; @@ -222,8 +193,6 @@ static int xprt_rdma_bc_send_request(struct rpc_rqst *rqst) static void xprt_rdma_bc_close(struct rpc_xprt *xprt) { - dprintk("svcrdma: %s: xprt %p\n", __func__, xprt); - xprt_disconnect_done(xprt); xprt->cwnd = RPC_CWNDSHIFT; } @@ -231,8 +200,6 @@ xprt_rdma_bc_close(struct rpc_xprt *xprt) static void xprt_rdma_bc_put(struct rpc_xprt *xprt) { - dprintk("svcrdma: %s: xprt %p\n", __func__, xprt); - xprt_rdma_free_addresses(xprt); xprt_free(xprt); } @@ -267,19 +234,14 @@ xprt_setup_rdma_bc(struct xprt_create *args) struct rpc_xprt *xprt; struct rpcrdma_xprt *new_xprt; - if (args->addrlen > sizeof(xprt->addr)) { - dprintk("RPC: %s: address too large\n", __func__); + if (args->addrlen > sizeof(xprt->addr)) return ERR_PTR(-EBADF); - } xprt = xprt_alloc(args->net, sizeof(*new_xprt), RPCRDMA_MAX_BC_REQUESTS, RPCRDMA_MAX_BC_REQUESTS); - if (!xprt) { - dprintk("RPC: %s: couldn't allocate rpc_xprt\n", - __func__); + if (!xprt) return ERR_PTR(-ENOMEM); - } xprt->timeout = &xprt_rdma_bc_timeout; xprt_set_bound(xprt); From patchwork Fri May 1 17:41:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 11523085 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 57F3E913 for ; Fri, 1 May 2020 17:41:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3E43A2173E for ; Fri, 1 May 2020 17:41:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="U7guWbLg" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729899AbgEARlG (ORCPT ); Fri, 1 May 2020 13:41:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729572AbgEARlG (ORCPT ); Fri, 1 May 2020 13:41:06 -0400 Received: from mail-qt1-x842.google.com (mail-qt1-x842.google.com [IPv6:2607:f8b0:4864:20::842]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A4ADC061A0C; Fri, 1 May 2020 10:41:06 -0700 (PDT) Received: by mail-qt1-x842.google.com with SMTP id k12so8516789qtm.4; Fri, 01 May 2020 10:41:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=wLc9kaJEBy8MqiGvCdXWsNnTsIFMmf1FjET+I7edc2s=; b=U7guWbLg4XKUumPeRDF/L+sqnG016ACBmSCO/vUfrRsx6rvyktJkkjcaq0QMn0AlMb RGeZkmeDHTXPtwj8Xh61HauxyPuO4dyhARVcZLabV0PtiLvpRzP9SGYpYKe2Fifjxzw9 ImYjXqaIF4DqmpVswbdyCMhVw8nJnuDHvLb1AtJRGVbVJVpdrHH6430d9mr2NSIHwm+M iQffj+uyXevZKX5WptG04Zy6A+HrXKlEfa2OaHgk+y9y/dheSKwhPpjKAxrMQhiC1xHF ZLPZkrZ61HDfcCBA67ASK4rI2T7T1czESRBFN9FWGgW5f9Mfj1ZMexfagyhOWrki2Z+l ICKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:from:to:date:message-id :in-reply-to:references:user-agent:mime-version :content-transfer-encoding; bh=wLc9kaJEBy8MqiGvCdXWsNnTsIFMmf1FjET+I7edc2s=; b=StsYAUHbYZIlbyAi9+Ms4hF/0lRmNgztX+Gf/AyPnCIIlB2qsrPobda/Rk5XibUn5P 7GETQcj95tpQptW1CJcob0nYKL0fhoKGLt0wJd/p3Snullm9Ed+gG3ISN1KRiOkOXVw9 bGgcEBtBjnBgXWqjA2Jq9sRlfH4+qTg7ryx2Oh8qW2JZYUivCRC+PK1MXQRJC3ooIHMY zf3jNV96xNcKl746dSc/KueyjfStvl789mc7TW8nlx1wsDRNMUko1RFHXcdcAibD3kqk 0vwBlVquQCIp5PrmWq0i+Slu0Jx1wrLcrjkauZCbQTcx48Joca2HQkjBJ2stHZOzrsWB zUYQ== X-Gm-Message-State: AGi0PubCH+6vnfJuBmvrw9M6MgjyLWhPN0z8/VZgjLnVN9EGdf1UkGBI IMjEf6s129krXaI28w67rqXCQ7kE X-Google-Smtp-Source: APiQypIgmsRVmypDzFdEuz4JjVI2UqThj9m0b2jdaue2mjj4PJmTny+XLenPkAY2uC7Qic/LQt6qOA== X-Received: by 2002:ac8:37ac:: with SMTP id d41mr4998999qtc.288.1588354865433; Fri, 01 May 2020 10:41:05 -0700 (PDT) Received: from gateway.1015granger.net (c-68-61-232-219.hsd1.mi.comcast.net. [68.61.232.219]) by smtp.gmail.com with ESMTPSA id d123sm1014337qkb.28.2020.05.01.10.41.04 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 01 May 2020 10:41:04 -0700 (PDT) Received: from klimt.1015granger.net (klimt.1015granger.net [192.168.1.55]) by gateway.1015granger.net (8.14.7/8.14.7) with ESMTP id 041Hf33A026783; Fri, 1 May 2020 17:41:03 GMT Subject: [PATCH v1 7/7] svcrdma: Rename tracepoints that record header decoding errors From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Fri, 01 May 2020 13:41:03 -0400 Message-ID: <20200501174103.3899.45497.stgit@klimt.1015granger.net> In-Reply-To: <20200501173903.3899.31567.stgit@klimt.1015granger.net> References: <20200501173903.3899.31567.stgit@klimt.1015granger.net> User-Agent: StGit/0.22-20-geafe MIME-Version: 1.0 Sender: linux-nfs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org Clean up: Use a consistent naming convention so that these trace points can be enabled quickly via a glob. Signed-off-by: Chuck Lever --- include/trace/events/rpcrdma.h | 5 +++-- net/sunrpc/xprtrdma/svc_rdma_recvfrom.c | 10 +++++----- 2 files changed, 8 insertions(+), 7 deletions(-) diff --git a/include/trace/events/rpcrdma.h b/include/trace/events/rpcrdma.h index 3390dd12a8dc..6b8ab7a51744 100644 --- a/include/trace/events/rpcrdma.h +++ b/include/trace/events/rpcrdma.h @@ -1363,7 +1363,7 @@ TRACE_EVENT(svcrdma_decode_rqst, show_rpcrdma_proc(__entry->proc), __entry->hdrlen) ); -TRACE_EVENT(svcrdma_decode_short, +TRACE_EVENT(svcrdma_decode_short_err, TP_PROTO( unsigned int hdrlen ), @@ -1407,7 +1407,8 @@ DECLARE_EVENT_CLASS(svcrdma_badreq_event, ); #define DEFINE_BADREQ_EVENT(name) \ - DEFINE_EVENT(svcrdma_badreq_event, svcrdma_decode_##name,\ + DEFINE_EVENT(svcrdma_badreq_event, \ + svcrdma_decode_##name##_err, \ TP_PROTO( \ __be32 *p \ ), \ diff --git a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c index eee7c6478b30..e426fedb9524 100644 --- a/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c +++ b/net/sunrpc/xprtrdma/svc_rdma_recvfrom.c @@ -665,23 +665,23 @@ static int svc_rdma_xdr_decode_req(struct xdr_buf *rq_arg, return hdr_len; out_short: - trace_svcrdma_decode_short(rq_arg->len); + trace_svcrdma_decode_short_err(rq_arg->len); return -EINVAL; out_version: - trace_svcrdma_decode_badvers(rdma_argp); + trace_svcrdma_decode_badvers_err(rdma_argp); return -EPROTONOSUPPORT; out_drop: - trace_svcrdma_decode_drop(rdma_argp); + trace_svcrdma_decode_drop_err(rdma_argp); return 0; out_proc: - trace_svcrdma_decode_badproc(rdma_argp); + trace_svcrdma_decode_badproc_err(rdma_argp); return -EINVAL; out_inval: - trace_svcrdma_decode_parse(rdma_argp); + trace_svcrdma_decode_parse_err(rdma_argp); return -EINVAL; }