From patchwork Mon Oct 2 08:23:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 9980377 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 0F2156029B for ; Mon, 2 Oct 2017 08:24:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0190D28952 for ; Mon, 2 Oct 2017 08:24:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA8E528955; Mon, 2 Oct 2017 08:24:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C45D628952 for ; Mon, 2 Oct 2017 08:24:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750996AbdJBIYk (ORCPT ); Mon, 2 Oct 2017 04:24:40 -0400 Received: from mail-bl2nam02on0085.outbound.protection.outlook.com ([104.47.38.85]:59104 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1750943AbdJBIYi (ORCPT ); Mon, 2 Oct 2017 04:24:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=CkMNvmhAsGknWOx+xhRNKuCHOmm03+sXwBARJrISWgg=; b=KRHga3zDkPsZ4JU2g/8FEyQB2wAhg/bFLx47zgWJhRFWkUMOjEY8yVYaDyiEgGWne0dUIJJw0+mJ82wqi61O/zB3EeCE5XXLUk/G5PMjgka+F0ciii7LaR80Sh88zbbVZ8QreqkohlReNrCmNPQ2MkFd92nNcDtl+cRt528+OUg= Received: from SN4PR0701CA0016.namprd07.prod.outlook.com (10.161.192.154) by BY2PR07MB487.namprd07.prod.outlook.com (10.141.219.149) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Mon, 2 Oct 2017 08:24:32 +0000 Received: from BL2FFO11FD025.protection.gbl (2a01:111:f400:7c09::101) by SN4PR0701CA0016.outlook.office365.com (2603:10b6:803:28::26) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.56.9 via Frontend Transport; Mon, 2 Oct 2017 08:24:31 +0000 Authentication-Results: spf=none (sender IP is 50.232.66.26) smtp.mailfrom=cavium.com; vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=cavium.com; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BL2FFO11FD025.mail.protection.outlook.com (10.173.161.104) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.56.11 via Frontend Transport; Mon, 2 Oct 2017 08:24:30 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Mon, 2 Oct 2017 01:24:22 -0700 From: Michal Kalderon To: CC: , , , Michal Kalderon , "Ariel Elior" Subject: [PATCH net-next 01/12] qed: Add ll2 option to limit the number of bds per packet Date: Mon, 2 Oct 2017 11:23:47 +0300 Message-ID: <1506932638-26268-2-git-send-email-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1506932638-26268-1-git-send-email-Michal.Kalderon@cavium.com> References: <1506932638-26268-1-git-send-email-Michal.Kalderon@cavium.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(2980300002)(428002)(189002)(199003)(50226002)(189998001)(106466001)(69596002)(2906002)(36756003)(2351001)(105586002)(50986999)(76176999)(305945005)(4326008)(50466002)(77096006)(316002)(47776003)(5003940100001)(478600001)(72206003)(356003)(6666003)(16586007)(48376002)(54906003)(86362001)(5660300001)(104016004)(101416001)(2950100002)(6916009)(107886003)(8936002)(81166006)(81156014)(8676002); DIR:OUT; SFP:1101; SCL:1; SRVR:BY2PR07MB487; H:CAEXCH02.caveonetworks.com; FPR:; SPF:None; PTR:50-232-66-26-static.hfc.comcastbusiness.net; MX:1; A:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11FD025; 1:RAEc35oa266n6mAke72+NoHal9Z4kTuXrC+KLe74qmlKKHESk9XXM94NTa3OXc9Ie8a2a1npFonYX2QOYCH1bxQzD4TQRFz/lSplKjhTQyc4uh9VxYrMo18nT9qVZ1TK X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d14a0ac1-9511-4943-8edd-08d5096f0149 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:BY2PR07MB487; X-Microsoft-Exchange-Diagnostics: 1; BY2PR07MB487; 3:M5BCNP0Z+LfGazOX41i3yXgke72niArgJCXpOgqPxwGtJtnD0lnweSfNOwezA+siwNFRcVKf23zV8NwjrH7USC5iELsaF/9+a3a4wOiazaK/v7q77qfVqY0B8+4zwkEKUNhEff6b64iPmoUxP8hMT3pN7bXLE8ou6VORJjiPk5QQqGoMZqBCF1bgdz7BaFaGGq/28I1eJnCsQYhV0z5olh+9hiKNf843PasL38fr1n9Su+0AEx+igR4CZw6d5xftYnVlnWBphS0Ja/cU2T1UNPI4RlsPZgxyhK/cn7qenKWpqvMGu6WRnb1Jd4P15ym5FYwLWOpTNFytjqo3uXaKWg==; 25:t5HJBWZx6duJGp4/7KftyaKfbraaBbbfYmIhXCAHmT0lXUAPjHEWIyn5MHX3YcMa0fHsPzbZZUAnC3tZbEXmQwc/qqdENf58p0GB5+K0aMnNLJz8AGuDj06zOUINZsSfYtFSO7uic04HyVqqtP72USMMlpx9QrYqB4vV19lrmLSSYKUiCQ9Z4lH96XIsvu1GcPPyTzTr7iJPgCXn7x5ZwyehcTLBd5ZkIvVwIzzAs0ZnI3k3jZLJFA3KatrUi/CF0HM7crQ9DA7Q0k6f7APMttlKaZ7++Allihy90muAAswh9nkDLn0sLsIVhZTuYhK26O//7KioNh9pDdFSX6Pucg==; 31:Sro8O1aBxCEiUUixrMIOP16zEhxU8CuLid+VMSz6siIX0G8MMC4ZcbD+Xd0jPSY8IxxkFrbB3VRUbnRjzC7c5Dc8rXe6eNJz/kMi/4Bhk+tM8/lRCW6isUoz3NIKF2iY07uTf+nWzya9gLpPLWBeXW1P3iyPmzPHPG0M2QnaVGtlzZy1vDuhACDYFDhXrvzvIsH+Q85SWUrRMBLfhZfHEGBlBByUghZ0zPptNXC+hEI= X-MS-TrafficTypeDiagnostic: BY2PR07MB487: X-Microsoft-Exchange-Diagnostics: 1; BY2PR07MB487; 20:vnd6am5s/1mosg+y3XYKz6vQMymzVW9rr1XDAGrsiHt/hGmZSM4cSIV82hDHOz4NY+kc05yhDJlzkFB/cy1bgZVqaasGC0v+HMWITNpkjIdOrpbdGCrne02rKHO7Ye7xFlWa1wKLctADG7JN2Wub6AEekjYv/aarRg5hN7c1fwM1iRB92T/QC/+ExZdvoyK6Qj66Vef0/Kre0D9ggivt3Km+yX5ss2yqHNjz4aQR704YtrJDylaPf0Nyqz8XUIyVU9kMpbC3+NKq9QLvvOg1jvpvDdfQTsT+6GJRzMrX3DT+XG9lkxIScgty1RlmokI2UGnSfIJP77gg1weUcoSLYuZdgQOrfTEy4w6H5jEe6dpKXtmCeh1L7tTE6qWMESwknpnL0ksRZPbsNzU33bQYcHDKqmIFLuR9vW5Oiyq0fFPmH8oEBZo498aQB6pKnJAdEUdQA6GCu7F8KR+3kEUV1cIDtzMduxlpnAkV7M4yGYVWyGyEecVZUwGbzgpmne+r; 4:z38vbhALMFdeV8AZ4j5CcywmuWE7WU/3jMePzsRCs54qfvPVtVbIJVL84x88kahuhgJw/jnffWRw7XhJAApy+3q2aJBX8/xoO9blvDvG1c77Ra27jGDMxZQkzFCihR35Cm4LtAyhTYGmCx9ylsMy2WjqxsfdCkzWbYY71gClujCyI1X3irVYCj9AYtbBC4xPAhEfQXbFbxKJKT3NbbN8swKZzZPdP+gxTcZ//fWdQUJZo/sMBNQKyhAdmxkqD+hF X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(5005006)(8121501046)(3002001)(10201501046)(100000703101)(100105400095)(93006095)(93001095)(6041248)(20161123555025)(20161123558100)(20161123560025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:BY2PR07MB487; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2PR07MB487; X-Forefront-PRVS: 0448A97BF2 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BY2PR07MB487; 23:1CE1bBHwD4SWrc7hgT7nlt5VHi/tGbqrdN3oJx8dpP?= =?us-ascii?Q?DMzb7CIcK7H0yo2RERwgUifXFojgViPSpUe9b89utZDjiHoVxj4Fo6yM9kAk?= =?us-ascii?Q?Y9MEMbOx00oVL3x2Kr44FYCCH5HJPgPO0BwzFpWW03hRszif4xtPFvam/HK2?= =?us-ascii?Q?DmP+hE6q1DgN/zCedJUlysl3EDQ1kr1l+5XCSQHG+r2pzs3FJIBWiI0GGGBr?= =?us-ascii?Q?aGw6pp7r9PxIpWoDPAEtd90PlTBFIDwvSN2/T0fi6LiX8tH4Z+KeNG6jP0CJ?= =?us-ascii?Q?XvAL06dn1OuzaM58w/5dHIQvj3di3RlHNKF5nEsHYLWu7e50KfY4vgaZbD7h?= =?us-ascii?Q?+O3fqBs/RpvMrsb4Y8+T20jy59fF9NkEpQvm/UteaVy1eBoi10VvsdPOAv2f?= =?us-ascii?Q?yOLjzxM+BcOjDJKw3392otgaKBgOQwLobaqhCHdg3DU2xoIkpWUyCTXMeAWa?= =?us-ascii?Q?ehooCEThh68iHcIoWG1bbY6rx+MpI+nhGWnDdcFxoHTKt/+caICA4UzcP/od?= =?us-ascii?Q?UPjQ/7xAHbfjyQRs4TPgahfIteI7DOyWEmRRCljZ6mKFkJQPjF1tq2idNg2g?= =?us-ascii?Q?/hxui+e5/sWklEFrkP+UCHSnKpUsic9zvkMcjEOM4DuT71ZJ6Mxxj3wIhcat?= =?us-ascii?Q?e1yDZzaeJshM1YyXTPHZb04gbWuVJN6uDHGJ5TdPS38KZFU9gmHKBi3rKUMa?= =?us-ascii?Q?vWsJtwxxeLd1Lyw6/EkJDfrz2Fqb61RksjcCjUl8mYDpeE5AowFSD8BqPCsR?= =?us-ascii?Q?nBT9qf9tF4GAiD831DDUycaOqWcVjn3dA5/WZ0SR70cc0H2S3dniKubrPaHz?= =?us-ascii?Q?H3Gi9Tcm2qRqH7r1233swa0BpQwiy8+LzjTwoy4PeF9Q3kYRLuGjl6ygcHBC?= =?us-ascii?Q?6AE+L5CRO24xrNLzB7mFiYT0oCg5OIUESh2EdQ1TYV0SJ2ErPVL1vRdFYxMF?= =?us-ascii?Q?30WBGBM3/KFx/awikw1JRo9y07Zd5IVD9Jml5KugnBAaH57MnCNSEv1BEgmR?= =?us-ascii?Q?mSehsAIHkggf4WJIfo5jwm?= X-Microsoft-Exchange-Diagnostics: 1; BY2PR07MB487; 6:q4A/d2cz2rkIvrebP7U8ZCd0C9kImBBdVCq8dSOhHj4mx32csUb4N+HXi0z2PIQn5OcXennEO02qgrMjaQcBn2KOmjSCus5V/F3HK4FgN8JgUPIZazHo5udoFwT5CSsJkpoK/bYj99uyBpS0JJEhGhqxtb6MLxf8zAkIjUI12IwQtOeYeJ+SNFRMs9fSdotpwUk00/XG4zWK2TndqP+ZUsq/kpB7nulk3r25+GrtqgKxdS3cUL9/CCiGFstFqZhw5rC+6jVw5/cwT+SrXqO5Oh/QcwbpHS2Pv3U2ODyxiFHySbhIiVtiaXb8uNz19axmVFcHGpDqHoE7CNjzSAffrg==; 5:koyCD5Oup30+M+41bWyTuQUmeWkcIv2Y3VXsLeUZcX9986Ee3PHnRTWUAcZcld7fwvMce/690vGfhftgWF748+1hAhBFStLQEsxlkPooZyBa9HlPGz9ZMtNs3iT+d/A6H5HDl4054MpTF6uLagF9Bg==; 24:xv33crn06sc0W6y0ke+7k01Ib1Qx9e2WngarJJ50pAD5tTlr8EBgbRp5KyYeCLgiMOIC2l10nWBL2P65fniU8RS2ij/rLfY0sBHFoCkSqlU=; 7:iRpwtEw3gdsyup/Kh7dFN8Uqzmd9JpFkbOdbDQu1MG6rl0Df2MYD//kwwmbrG8AY7kzJzPoI3sutlxG75VpXtIRbvxLaJTqtXXz9CYWNziOm6fOoDiUJoFp/7Ha7pYWdjxfgLwCN3uMP5/Akgt3OcnCrZtHQYU/jI7zrJ4I+6elGy5CqwnNnQRkxXrA/7SR6B14XyHmbPUM92Ln/T4Vo9Y9oMdzIm8EV4dsK9l24zME= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Oct 2017 08:24:30.6978 (UTC) X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2PR07MB487 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP iWARP uses 3 ll2 connections, the maximum number of bds is known during connection setup. This patch modifies the static array in the ll2_tx_packet descriptor to be a flexible array and significantlly reduces memory size. In addition, some redundant fields in the ll2_tx_packet were removed, which also contributed to decreasing the descriptor size. Signed-off-by: Michal Kalderon Signed-off-by: Ariel Elior --- drivers/net/ethernet/qlogic/qed/qed_ll2.c | 25 +++++++++++++++++++------ drivers/net/ethernet/qlogic/qed/qed_ll2.h | 7 ++----- 2 files changed, 21 insertions(+), 11 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.c b/drivers/net/ethernet/qlogic/qed/qed_ll2.c index 250afa5..10e3a43 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.c +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.c @@ -1105,6 +1105,7 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, struct qed_ll2_info *p_ll2_info) { struct qed_ll2_tx_packet *p_descq; + u32 desc_size; u32 capacity; int rc = 0; @@ -1122,8 +1123,12 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, goto out; capacity = qed_chain_get_capacity(&p_ll2_info->tx_queue.txq_chain); - p_descq = kcalloc(capacity, sizeof(struct qed_ll2_tx_packet), - GFP_KERNEL); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_descq) + + (p_ll2_info->input.tx_max_bds_per_packet - 1) * + sizeof(p_descq->bds_set)); + + p_descq = kcalloc(capacity, desc_size, GFP_KERNEL); if (!p_descq) { rc = -ENOMEM; goto out; @@ -1359,11 +1364,13 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) { struct qed_hwfn *p_hwfn = cxt; struct qed_ll2_info *p_ll2_conn; + struct qed_ll2_tx_packet *p_pkt; struct qed_ll2_rx_queue *p_rx; struct qed_ll2_tx_queue *p_tx; struct qed_ptt *p_ptt; int rc = -EINVAL; u32 i, capacity; + u32 desc_size; u8 qid; p_ptt = qed_ptt_acquire(p_hwfn); @@ -1397,9 +1404,15 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) INIT_LIST_HEAD(&p_tx->sending_descq); spin_lock_init(&p_tx->lock); capacity = qed_chain_get_capacity(&p_tx->txq_chain); - for (i = 0; i < capacity; i++) - list_add_tail(&p_tx->descq_array[i].list_entry, - &p_tx->free_descq); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_pkt) + + (p_ll2_conn->input.tx_max_bds_per_packet - 1) * + sizeof(p_pkt->bds_set)); + + for (i = 0; i < capacity; i++) { + p_pkt = (void *)((u8 *)p_tx->descq_array + desc_size * i); + list_add_tail(&p_pkt->list_entry, &p_tx->free_descq); + } p_tx->cur_completing_bd_idx = 0; p_tx->bds_idx = 0; p_tx->b_completing_packet = false; @@ -1698,7 +1711,7 @@ int qed_ll2_prepare_tx_packet(void *cxt, p_tx = &p_ll2_conn->tx_queue; p_tx_chain = &p_tx->txq_chain; - if (pkt->num_of_bds > CORE_LL2_TX_MAX_BDS_PER_PACKET) + if (pkt->num_of_bds > p_ll2_conn->input.tx_max_bds_per_packet) return -EIO; spin_lock_irqsave(&p_tx->lock, flags); diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.h b/drivers/net/ethernet/qlogic/qed/qed_ll2.h index a822528..8019336 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.h +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.h @@ -63,17 +63,14 @@ struct qed_ll2_rx_packet { struct qed_ll2_tx_packet { struct list_head list_entry; u16 bd_used; - u16 vlan; - u16 l4_hdr_offset_w; - u8 bd_flags; bool notify_fw; void *cookie; - + /* Flexible Array of bds_set determined by max_bds_per_packet */ struct { struct core_tx_bd *txq_bd; dma_addr_t tx_frag; u16 frag_len; - } bds_set[ETH_TX_MAX_BDS_PER_NON_LSO_PACKET]; + } bds_set[1]; }; struct qed_ll2_rx_queue {