From patchwork Thu Mar 16 03:24:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jitendra Bhivare X-Patchwork-Id: 9627053 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id B135160244 for ; Thu, 16 Mar 2017 03:26:15 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A634528604 for ; Thu, 16 Mar 2017 03:26:15 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B0F02863B; Thu, 16 Mar 2017 03:26:15 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.5 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5BFBD28604 for ; Thu, 16 Mar 2017 03:26:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751411AbdCPD0O (ORCPT ); Wed, 15 Mar 2017 23:26:14 -0400 Received: from mail-wr0-f174.google.com ([209.85.128.174]:34472 "EHLO mail-wr0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751229AbdCPD0B (ORCPT ); Wed, 15 Mar 2017 23:26:01 -0400 Received: by mail-wr0-f174.google.com with SMTP id l37so23187215wrc.1 for ; Wed, 15 Mar 2017 20:25:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TceXMLhaF0o1RT0OGFXcG+juHUdGLYG8vmON+qZU6U0=; b=AiGHGb31o5srKbvy9jrMe3S/LcuQL45N4f3gIYLG1pGAwQuVYKDMC2zKgUc5O22jzl ABq41r20SQe1DxjEqwWVqa2mmdw3XDkSnFT6UPkPLwxhogl5r16Jj32WbW0NZCBWOpW4 oNRry7d7lE0XeT2ZkkoPTJ8HcuTKrEnOf/0fE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TceXMLhaF0o1RT0OGFXcG+juHUdGLYG8vmON+qZU6U0=; b=ALQXRJ66eusN5oI6j2NLZMOoFpA3TTR5cipSbuXPiLDSO9ypcRYBzCE9I22bn1P2SK VQv86kwK7gF1mRHRNWaAFblzpWYpsIMLwNcBKVkveOmhIwXbs/7y6K65JqU5t0+IwCAz Bns169TgEzi3oIG2Xfq2PY5pdNcJMfHFPYewRMXmkrP1/EfxcDHOon0sGv3itfxfgBKQ 8XgpF80LpzI0/3Qujh6LGIzeVJ2riAzTLtbWWWM6wfqHBhV/G7FU+oXakx3fKhGXqQKj k5+s4iG5xMgC6iJJBtmv/B+La6dGTCSTKqwmgawfKeZ3ST2rXS/d3PmFbN/O7IoSCv5U 0+Sw== X-Gm-Message-State: AFeK/H1Fiyinui3+Vw1tfRW31Clf9X6GCABRTTeGKEl2nyK9ht+COfIY1FfWa+ezOpaz3UbN X-Received: by 10.223.136.33 with SMTP id d30mr6200284wrd.117.1489634732029; Wed, 15 Mar 2017 20:25:32 -0700 (PDT) Received: from android.dhcp.avagotech.net ([192.19.239.250]) by smtp.gmail.com with ESMTPSA id l21sm4454655wrl.59.2017.03.15.20.25.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 15 Mar 2017 20:25:31 -0700 (PDT) From: Jitendra Bhivare To: cleech@redhat.com, lduncan@suse.com Cc: linux-scsi@vger.kernel.org, Jitendra Bhivare Subject: [PATCH v2 06/10] be2iscsi: Use num_cons field in Rx CQE Date: Thu, 16 Mar 2017 08:54:41 +0530 Message-Id: <1489634685-4975-7-git-send-email-jitendra.bhivare@broadcom.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1489634685-4975-1-git-send-email-jitendra.bhivare@broadcom.com> References: <1489634685-4975-1-git-send-email-jitendra.bhivare@broadcom.com> Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP FW runs out of buffer if buffers are not posted back soon. ASYNC Rx CQE indicates that FW has consumed 8 RQEs. Use it to post back buffers instead of waiting for buffers to be processed and freed by driver. Signed-off-by: Jitendra Bhivare --- drivers/scsi/be2iscsi/be_main.c | 124 ++++++++++++++++------------------------ drivers/scsi/be2iscsi/be_main.h | 1 + 2 files changed, 51 insertions(+), 74 deletions(-) diff --git a/drivers/scsi/be2iscsi/be_main.c b/drivers/scsi/be2iscsi/be_main.c index b76fd26..1cd9c2d 100644 --- a/drivers/scsi/be2iscsi/be_main.c +++ b/drivers/scsi/be2iscsi/be_main.c @@ -1467,7 +1467,8 @@ beiscsi_hdl_put_handle(struct hd_async_context *pasync_ctx, static struct hd_async_handle * beiscsi_hdl_get_handle(struct beiscsi_conn *beiscsi_conn, struct hd_async_context *pasync_ctx, - struct i_t_dpdu_cqe *pdpdu_cqe) + struct i_t_dpdu_cqe *pdpdu_cqe, + u8 *header) { struct beiscsi_hba *phba = beiscsi_conn->phba; struct hd_async_handle *pasync_handle; @@ -1515,6 +1516,7 @@ beiscsi_hdl_get_handle(struct beiscsi_conn *beiscsi_conn, switch (code) { case UNSOL_HDR_NOTIFY: pasync_handle = pasync_ctx->async_entry[ci].header; + *header = 1; break; case UNSOL_DATA_DIGEST_ERROR_NOTIFY: error = 1; @@ -1547,6 +1549,7 @@ beiscsi_hdl_get_handle(struct beiscsi_conn *beiscsi_conn, /* FW has stale address - attempt continuing by dropping */ } + list_del_init(&pasync_handle->link); /** * Each CID is associated with unique CRI. * ASYNC_CRI_FROM_CID mapping and CRI_FROM_CID are totaly different. @@ -1554,11 +1557,6 @@ beiscsi_hdl_get_handle(struct beiscsi_conn *beiscsi_conn, pasync_handle->cri = BE_GET_ASYNC_CRI_FROM_CID(cid); pasync_handle->is_final = final; pasync_handle->buffer_len = dpl; - /* empty the slot */ - if (pasync_handle->is_header) - pasync_ctx->async_entry[ci].header = NULL; - else - pasync_ctx->async_entry[ci].data = NULL; /** * DEF PDU header and data buffers with errors should be simply @@ -1708,85 +1706,53 @@ beiscsi_hdl_gather_pdu(struct beiscsi_conn *beiscsi_conn, static void beiscsi_hdq_post_handles(struct beiscsi_hba *phba, - u8 header, u8 ulp_num) + u8 header, u8 ulp_num, u16 nbuf) { - struct hd_async_handle *pasync_handle, *tmp, **slot; + struct hd_async_handle *pasync_handle; struct hd_async_context *pasync_ctx; struct hwi_controller *phwi_ctrlr; - struct list_head *hfree_list; struct phys_addr *pasync_sge; u32 ring_id, doorbell = 0; u32 doorbell_offset; - u16 prod = 0, cons; - u16 index; + u16 prod, pi; phwi_ctrlr = phba->phwi_ctrlr; pasync_ctx = HWI_GET_ASYNC_PDU_CTX(phwi_ctrlr, ulp_num); if (header) { - cons = pasync_ctx->async_header.free_entries; - hfree_list = &pasync_ctx->async_header.free_list; + pasync_sge = pasync_ctx->async_header.ring_base; + pi = pasync_ctx->async_header.pi; ring_id = phwi_ctrlr->default_pdu_hdr[ulp_num].id; doorbell_offset = phwi_ctrlr->default_pdu_hdr[ulp_num]. doorbell_offset; } else { - cons = pasync_ctx->async_data.free_entries; - hfree_list = &pasync_ctx->async_data.free_list; + pasync_sge = pasync_ctx->async_data.ring_base; + pi = pasync_ctx->async_data.pi; ring_id = phwi_ctrlr->default_pdu_data[ulp_num].id; doorbell_offset = phwi_ctrlr->default_pdu_data[ulp_num]. doorbell_offset; } - /* number of entries posted must be in multiples of 8 */ - if (cons % 8) - return; - list_for_each_entry_safe(pasync_handle, tmp, hfree_list, link) { - list_del_init(&pasync_handle->link); - pasync_handle->is_final = 0; - pasync_handle->buffer_len = 0; - - /* handles can be consumed out of order, use index in handle */ - index = pasync_handle->index; - WARN_ON(pasync_handle->is_header != header); + for (prod = 0; prod < nbuf; prod++) { if (header) - slot = &pasync_ctx->async_entry[index].header; + pasync_handle = pasync_ctx->async_entry[pi].header; else - slot = &pasync_ctx->async_entry[index].data; - /** - * The slot just tracks handle's hold and release, so - * overwriting at the same index won't do any harm but - * needs to be caught. - */ - if (*slot != NULL) { - beiscsi_log(phba, KERN_ERR, BEISCSI_LOG_ISCSI, - "BM_%d : async PDU %s slot at %u not empty\n", - header ? "header" : "data", index); + pasync_handle = pasync_ctx->async_entry[pi].data; + WARN_ON(pasync_handle->is_header != header); + WARN_ON(pasync_handle->index != pi); + /* setup the ring only once */ + if (nbuf == pasync_ctx->num_entries) { + /* note hi is lo */ + pasync_sge[pi].hi = pasync_handle->pa.u.a32.address_lo; + pasync_sge[pi].lo = pasync_handle->pa.u.a32.address_hi; } - /** - * We use same freed index as in completion to post so this - * operation is not required for refills. Its required only - * for ring creation. - */ - if (header) - pasync_sge = pasync_ctx->async_header.ring_base; - else - pasync_sge = pasync_ctx->async_data.ring_base; - pasync_sge += index; - /* if its a refill then address is same; hi is lo */ - WARN_ON(pasync_sge->hi && - pasync_sge->hi != pasync_handle->pa.u.a32.address_lo); - WARN_ON(pasync_sge->lo && - pasync_sge->lo != pasync_handle->pa.u.a32.address_hi); - pasync_sge->hi = pasync_handle->pa.u.a32.address_lo; - pasync_sge->lo = pasync_handle->pa.u.a32.address_hi; - - *slot = pasync_handle; - if (++prod == cons) - break; + if (++pi == pasync_ctx->num_entries) + pi = 0; } + if (header) - pasync_ctx->async_header.free_entries -= prod; + pasync_ctx->async_header.pi = pi; else - pasync_ctx->async_data.free_entries -= prod; + pasync_ctx->async_data.pi = pi; doorbell |= ring_id & DB_DEF_PDU_RING_ID_MASK; doorbell |= 1 << DB_DEF_PDU_REARM_SHIFT; @@ -1803,20 +1769,26 @@ beiscsi_hdq_process_compl(struct beiscsi_conn *beiscsi_conn, struct hd_async_handle *pasync_handle = NULL; struct hd_async_context *pasync_ctx; struct hwi_controller *phwi_ctrlr; + u8 ulp_num, consumed, header = 0; u16 cid_cri; - u8 ulp_num; phwi_ctrlr = phba->phwi_ctrlr; cid_cri = BE_GET_CRI_FROM_CID(beiscsi_conn->beiscsi_conn_cid); ulp_num = BEISCSI_GET_ULP_FROM_CRI(phwi_ctrlr, cid_cri); pasync_ctx = HWI_GET_ASYNC_PDU_CTX(phwi_ctrlr, ulp_num); pasync_handle = beiscsi_hdl_get_handle(beiscsi_conn, pasync_ctx, - pdpdu_cqe); - if (!pasync_handle) - return; - - beiscsi_hdl_gather_pdu(beiscsi_conn, pasync_ctx, pasync_handle); - beiscsi_hdq_post_handles(phba, pasync_handle->is_header, ulp_num); + pdpdu_cqe, &header); + if (is_chip_be2_be3r(phba)) + consumed = AMAP_GET_BITS(struct amap_i_t_dpdu_cqe, + num_cons, pdpdu_cqe); + else + consumed = AMAP_GET_BITS(struct amap_i_t_dpdu_cqe_v2, + num_cons, pdpdu_cqe); + if (pasync_handle) + beiscsi_hdl_gather_pdu(beiscsi_conn, pasync_ctx, pasync_handle); + /* num_cons indicates number of 8 RQEs consumed */ + if (consumed) + beiscsi_hdq_post_handles(phba, header, ulp_num, 8 * consumed); } void beiscsi_process_mcc_cq(struct beiscsi_hba *phba) @@ -2775,6 +2747,7 @@ static int hwi_init_async_pdu_ctx(struct beiscsi_hba *phba) "BM_%d : No Virtual address for ULP : %d\n", ulp_num); + pasync_ctx->async_header.pi = 0; pasync_ctx->async_header.buffer_size = p->defpdu_hdr_sz; pasync_ctx->async_header.va_base = mem_descr->mem_array[0].virtual_address; @@ -2883,6 +2856,7 @@ static int hwi_init_async_pdu_ctx(struct beiscsi_hba *phba) ulp_num); idx = 0; + pasync_ctx->async_data.pi = 0; pasync_ctx->async_data.buffer_size = p->defpdu_data_sz; pasync_ctx->async_data.va_base = mem_descr->mem_array[idx].virtual_address; @@ -2913,11 +2887,12 @@ static int hwi_init_async_pdu_ctx(struct beiscsi_hba *phba) list_add_tail(&pasync_header_h->link, &pasync_ctx->async_header. free_list); + pasync_ctx->async_entry[index].header = + pasync_header_h; pasync_header_h++; pasync_ctx->async_header.free_entries++; INIT_LIST_HEAD(&pasync_ctx->async_entry[index]. wq.list); - pasync_ctx->async_entry[index].header = NULL; pasync_data_h->cri = -1; pasync_data_h->is_header = 0; @@ -2954,9 +2929,10 @@ static int hwi_init_async_pdu_ctx(struct beiscsi_hba *phba) list_add_tail(&pasync_data_h->link, &pasync_ctx->async_data. free_list); + pasync_ctx->async_entry[index].data = + pasync_data_h; pasync_data_h++; pasync_ctx->async_data.free_entries++; - pasync_ctx->async_entry[index].data = NULL; } } } @@ -3734,6 +3710,7 @@ static int hwi_init_port(struct beiscsi_hba *phba) unsigned int def_pdu_ring_sz; struct be_ctrl_info *ctrl = &phba->ctrl; int status, ulp_num; + u16 nbufs; phwi_ctrlr = phba->phwi_ctrlr; phwi_context = phwi_ctrlr->phwi_ctxt; @@ -3770,9 +3747,8 @@ static int hwi_init_port(struct beiscsi_hba *phba) for (ulp_num = 0; ulp_num < BEISCSI_ULP_COUNT; ulp_num++) { if (test_bit(ulp_num, &phba->fw_config.ulp_supported)) { - def_pdu_ring_sz = - BEISCSI_ASYNC_HDQ_SIZE(phba, ulp_num) * - sizeof(struct phys_addr); + nbufs = phwi_context->pasync_ctx[ulp_num]->num_entries; + def_pdu_ring_sz = nbufs * sizeof(struct phys_addr); status = beiscsi_create_def_hdr(phba, phwi_context, phwi_ctrlr, @@ -3800,9 +3776,9 @@ static int hwi_init_port(struct beiscsi_hba *phba) * let EP know about it. */ beiscsi_hdq_post_handles(phba, BEISCSI_DEFQ_HDR, - ulp_num); + ulp_num, nbufs); beiscsi_hdq_post_handles(phba, BEISCSI_DEFQ_DATA, - ulp_num); + ulp_num, nbufs); } } diff --git a/drivers/scsi/be2iscsi/be_main.h b/drivers/scsi/be2iscsi/be_main.h index 216f9b4..cebacac 100644 --- a/drivers/scsi/be2iscsi/be_main.h +++ b/drivers/scsi/be2iscsi/be_main.h @@ -634,6 +634,7 @@ struct hd_async_buf_context { * They are posted back to FW in groups of 8. */ struct list_head free_list; + u16 pi; }; /**