From patchwork Sat Nov 7 01:07:06 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: jubin.john@intel.com X-Patchwork-Id: 7574341 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id F3BBD9F4F5 for ; Sat, 7 Nov 2015 01:07:27 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id E660F206C0 for ; Sat, 7 Nov 2015 01:07:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 66F5C206D6 for ; Sat, 7 Nov 2015 01:07:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751079AbbKGBHV (ORCPT ); Fri, 6 Nov 2015 20:07:21 -0500 Received: from mga01.intel.com ([192.55.52.88]:49584 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752594AbbKGBHS (ORCPT ); Fri, 6 Nov 2015 20:07:18 -0500 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 06 Nov 2015 17:07:18 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,254,1444719600"; d="scan'208";a="829726169" Received: from phlsvsds.ph.intel.com ([10.228.195.38]) by fmsmga001.fm.intel.com with ESMTP; 06 Nov 2015 17:07:17 -0800 Received: from phlsvsds.ph.intel.com (localhost.localdomain [127.0.0.1]) by phlsvsds.ph.intel.com (8.13.8/8.13.8) with ESMTP id tA717GR2030621; Fri, 6 Nov 2015 20:07:16 -0500 Received: (from jjohn@localhost) by phlsvsds.ph.intel.com (8.13.8/8.13.8/Submit) id tA717FTE030618; Fri, 6 Nov 2015 20:07:15 -0500 X-Authentication-Warning: phlsvsds.ph.intel.com: jjohn set sender to jubin.john@intel.com using -f From: Jubin John To: gregkh@linuxfoundation.org, devel@driverdev.osuosl.org Cc: linux-rdma@vger.kernel.org, dledford@redhat.com, mike.marciniszyn@intel.com, ira.weiny@intel.com Subject: [PATCH RESEND 11/11] staging/rdma/hfi1: Workaround to prevent corruption during packet delivery Date: Fri, 6 Nov 2015 20:07:06 -0500 Message-Id: <1446858426-30509-12-git-send-email-jubin.john@intel.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1446858426-30509-1-git-send-email-jubin.john@intel.com> References: <1446858426-30509-1-git-send-email-jubin.john@intel.com> Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, T_RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Mark F. Brown Disabling one receive context when RX_DMA is receiving a packet can cause incorrect packet delivery for a subsequent packet on another receive context. This is resolved by doing the following: 1. Programming dummy tail address for every receive context before enabling it 2. While deallocating receive context resetting tail address to dummy address 3. Leaving the dummy address in when disabling tail update 4. When disabling receive context leaving tail update enabled Reviewed-by: Dennis Dalessandro Reviewed-by: Mike Marciniszyn Reviewed-by: Mitko Haralanov Signed-off-by: Mark F. Brown Signed-off-by: Jubin John --- drivers/staging/rdma/hfi1/chip.c | 18 +++++++++++++++--- drivers/staging/rdma/hfi1/hfi.h | 4 ++++ drivers/staging/rdma/hfi1/init.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 47 insertions(+), 3 deletions(-) diff --git a/drivers/staging/rdma/hfi1/chip.c b/drivers/staging/rdma/hfi1/chip.c index 7658c76..e74c32e 100644 --- a/drivers/staging/rdma/hfi1/chip.c +++ b/drivers/staging/rdma/hfi1/chip.c @@ -7785,6 +7785,17 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt) } if (op & HFI1_RCVCTRL_CTXT_DIS) { write_csr(dd, RCV_VL15, 0); + /* + * When receive context is being disabled turn on tail + * update with a dummy tail address and then disable + * receive context. + */ + if (dd->rcvhdrtail_dummy_physaddr) { + write_kctxt_csr(dd, ctxt, RCV_HDR_TAIL_ADDR, + dd->rcvhdrtail_dummy_physaddr); + rcvctrl |= RCV_CTXT_CTRL_TAIL_UPD_SMASK; + } + rcvctrl &= ~RCV_CTXT_CTRL_ENABLE_SMASK; } if (op & HFI1_RCVCTRL_INTRAVAIL_ENB) @@ -7854,10 +7865,11 @@ void hfi1_rcvctrl(struct hfi1_devdata *dd, unsigned int op, int ctxt) if (op & (HFI1_RCVCTRL_TAILUPD_DIS | HFI1_RCVCTRL_CTXT_DIS)) /* * If the context has been disabled and the Tail Update has - * been cleared, clear the RCV_HDR_TAIL_ADDR CSR so - * it doesn't contain an address that is invalid. + * been cleared, set the RCV_HDR_TAIL_ADDR CSR to dummy address + * so it doesn't contain an address that is invalid. */ - write_kctxt_csr(dd, ctxt, RCV_HDR_TAIL_ADDR, 0); + write_kctxt_csr(dd, ctxt, RCV_HDR_TAIL_ADDR, + dd->rcvhdrtail_dummy_physaddr); } u32 hfi1_read_cntrs(struct hfi1_devdata *dd, loff_t pos, char **namep, diff --git a/drivers/staging/rdma/hfi1/hfi.h b/drivers/staging/rdma/hfi1/hfi.h index 190f7a2..02f2deb 100644 --- a/drivers/staging/rdma/hfi1/hfi.h +++ b/drivers/staging/rdma/hfi1/hfi.h @@ -1084,6 +1084,10 @@ struct hfi1_devdata { /* Save the enabled LCB error bits */ u64 lcb_err_en; u8 dc_shutdown; + + /* receive context tail dummy address */ + volatile __le64 *rcvhdrtail_dummy_kvaddr; + dma_addr_t rcvhdrtail_dummy_physaddr; }; /* 8051 firmware version helper */ diff --git a/drivers/staging/rdma/hfi1/init.c b/drivers/staging/rdma/hfi1/init.c index 4a79744..36a3065 100644 --- a/drivers/staging/rdma/hfi1/init.c +++ b/drivers/staging/rdma/hfi1/init.c @@ -693,6 +693,18 @@ int hfi1_init(struct hfi1_devdata *dd, int reinit) if (ret) goto done; + /* allocate dummy tail memory for all receive contexts */ + dd->rcvhdrtail_dummy_kvaddr = dma_zalloc_coherent( + &dd->pcidev->dev, sizeof(u64), + &dd->rcvhdrtail_dummy_physaddr, + GFP_KERNEL); + + if (!dd->rcvhdrtail_dummy_kvaddr) { + dd_dev_err(dd, "cannot allocate dummy tail memory\n"); + ret = -ENOMEM; + goto done; + } + /* dd->rcd can be NULL if early initialization failed */ for (i = 0; dd->rcd && i < dd->first_user_ctxt; ++i) { /* @@ -1268,6 +1280,14 @@ static void cleanup_device_data(struct hfi1_devdata *dd) tmp = dd->rcd; dd->rcd = NULL; spin_unlock_irqrestore(&dd->uctxt_lock, flags); + + if (dd->rcvhdrtail_dummy_kvaddr) { + dma_free_coherent(&dd->pcidev->dev, sizeof(u64), + (void *)dd->rcvhdrtail_dummy_kvaddr, + dd->rcvhdrtail_dummy_physaddr); + dd->rcvhdrtail_dummy_kvaddr = NULL; + } + for (ctxt = 0; tmp && ctxt < dd->num_rcv_contexts; ctxt++) { struct hfi1_ctxtdata *rcd = tmp[ctxt]; @@ -1523,6 +1543,14 @@ int hfi1_create_rcvhdrq(struct hfi1_devdata *dd, struct hfi1_ctxtdata *rcd) reg = (dd->rcvhdrsize & RCV_HDR_SIZE_HDR_SIZE_MASK) << RCV_HDR_SIZE_HDR_SIZE_SHIFT; write_kctxt_csr(dd, rcd->ctxt, RCV_HDR_SIZE, reg); + + /* + * Program dummy tail address for every receive context + * before enabling any receive context + */ + write_kctxt_csr(dd, rcd->ctxt, RCV_HDR_TAIL_ADDR, + dd->rcvhdrtail_dummy_physaddr); + return 0; bail_free: