From patchwork Mon Oct 9 09:37:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Kalderon, Michal" X-Patchwork-Id: 9992439 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 72F3F60223 for ; Mon, 9 Oct 2017 09:38:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 648F4285E2 for ; Mon, 9 Oct 2017 09:38:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5978E28755; Mon, 9 Oct 2017 09:38:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 31DC6285E2 for ; Mon, 9 Oct 2017 09:38:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753983AbdJIJiV (ORCPT ); Mon, 9 Oct 2017 05:38:21 -0400 Received: from mail-bl2nam02on0043.outbound.protection.outlook.com ([104.47.38.43]:19261 "EHLO NAM02-BL2-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753728AbdJIJiT (ORCPT ); Mon, 9 Oct 2017 05:38:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=T8OZ1n9xL/fNtGaTA4zcfz6jXrpA/dxGPv5/bNjOPWM=; b=j78q5e0pdGO0kEbiiWRUNgpsVpXx6Ru26Ixs5GlbrdZ3Y9anqMf+uRMwWLy5zDOXtTCFhKcoAlqGmZMcJEoK/kyBXkiGuO9J9xJ1MMANu3INkNmlXXhW/TyuQSucFe0t8Mqbb+IcxGB/MnN3xU5BEHzXBbS5fJwJZy7UUMvq7Rs= Received: from CY1PR07CA0026.namprd07.prod.outlook.com (10.166.202.36) by SN1PR0701MB1950.namprd07.prod.outlook.com (10.163.131.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7; Mon, 9 Oct 2017 09:38:17 +0000 Received: from BL2FFO11OLC005.protection.gbl (2a01:111:f400:7c09::176) by CY1PR07CA0026.outlook.office365.com (2a01:111:e400:c60a::36) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.77.7 via Frontend Transport; Mon, 9 Oct 2017 09:38:17 +0000 Authentication-Results: spf=none (sender IP is 50.232.66.26) smtp.mailfrom=cavium.com; vger.kernel.org; dkim=none (message not signed) header.d=none; vger.kernel.org; dmarc=none action=none header.from=cavium.com; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) Received: from CAEXCH02.caveonetworks.com (50.232.66.26) by BL2FFO11OLC005.mail.protection.outlook.com (10.173.160.91) with Microsoft SMTP Server (version=TLS1_0, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384) id 15.20.77.10 via Frontend Transport; Mon, 9 Oct 2017 09:38:17 +0000 Received: from lb-tlvb-michal.il.qlogic.com.com (10.185.6.89) by CAEXCH02.caveonetworks.com (10.17.4.29) with Microsoft SMTP Server id 14.2.347.0; Mon, 9 Oct 2017 02:38:12 -0700 From: Michal Kalderon To: CC: , , , Michal Kalderon , "Ariel Elior" Subject: [PATCH v3 net-next 01/12] qed: Add ll2 option to limit the number of bds per packet Date: Mon, 9 Oct 2017 12:37:43 +0300 Message-ID: <1507541874-18344-2-git-send-email-Michal.Kalderon@cavium.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1507541874-18344-1-git-send-email-Michal.Kalderon@cavium.com> References: <1507541874-18344-1-git-send-email-Michal.Kalderon@cavium.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:50.232.66.26; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(6009001)(346002)(376002)(2980300002)(428002)(199003)(189002)(47776003)(81166006)(81156014)(86362001)(105586002)(54906003)(356003)(48376002)(2906002)(5003940100001)(6916009)(4326008)(8936002)(305945005)(189998001)(8676002)(69596002)(2950100002)(316002)(36756003)(107886003)(77096006)(106466001)(5660300001)(72206003)(101416001)(478600001)(16586007)(2351001)(104016004)(50466002)(50986999)(76176999)(50226002)(6666003); DIR:OUT; SFP:1101; SCL:1; SRVR:SN1PR0701MB1950; H:CAEXCH02.caveonetworks.com; FPR:; SPF:None; PTR:50-232-66-26-static.hfc.comcastbusiness.net; A:1; MX:1; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BL2FFO11OLC005; 1:iIzkbT18QbJ6ffCeZB1wc8/3afrz4Ltg6Tu3/vu4inaJUj88zSMLcLmI39qg42TZj2wduYeWltiXKAtidrbW4iwaJHNVBqavXlV2g+pvEmMuJT/dZxdms5uFiDGy5j8A X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: adf06407-24b4-4b2e-4636-08d50ef97895 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(2017030254152)(2017052603199)(201703131423075)(201703031133081)(201702281549075); SRVR:SN1PR0701MB1950; X-Microsoft-Exchange-Diagnostics: 1; SN1PR0701MB1950; 3:Okmdh+Zk57B4N8JgN+plj+EZ+NPsZ2Kh3p0p8K5caEB+j72tlW0pDWHJOttwWiDWf0mb2I9sxgbelC3nCOkjZ+NYYzKud73MS9eQ6+EGFH+FMdJg9R9BQ57LixVUEohxbu072clnFNuvX/mPMqgcEVykm0LAcdJe9YkT5OxgQpSnS+EFk341f4SVCIAh6yZuXU8/vNSVyWgQy1IojvIXlmQPs2d2Ffk0QQirxn0qr/HCU4FAsjZmk513A28dbLiE0FymlgaJp/hrRMVyX0h7WFFXfm5mZ2O3OZGCvp/Ge/fYIKNgmXtaXo2dk5j3yXNTZwrpV93tCA8sq3+J23wJnA==; 25:l7Z98cQVJBz4wjxO7MpI3cM+/SfyuhRqptDwUB1nQeIoXdXv8eY0HE+KIBRWTZsoSWy2hAO5F14gaqxYzLlqo8Zi7ENhlUMisXs+uXAY50GP+iwOF57E7fn/I4f0WJiHQaZ+KjT5SRbd9m0PFVI4IF4kzp3fGAHd4y7rjQdOtHMizFAKlnUoTux+TWS03BB+Bqmrd0pcnHb6ENSA8+OygmBTIQPZhLcbCo1iqpCyywASb49isAtkgRdaCQmkDiZ0iDtTOyFhtlu58W8w6KEi0+bOn4L7a+GxyEreQIZe6pAHzvujzOyA9vlkxoCe0ps9jgwhjX5kpqHvlXSlPgEeZQ==; 31:jBmvzXOftnSdnKaXedsW3fX/WxXBRzQm/qVLeJgme84vIOMRhcKeIgGRxAtvlubwGFBVceMADW3pZSl6ZuqOVb4jokVNU8dmyFLPnEC6qZGfO9f5viTdx91GjI39VAzkXFjPOEuTkStazyaYoo5L3ZYDoUtulQxZ9bwUPwDoruwLFs/+cg6iLCVpt00pGlTF5D6c/QHVzkIVAiQXN3NkmGbi8X9tjqktoqQmR2vsk+Y= X-MS-TrafficTypeDiagnostic: SN1PR0701MB1950: X-Microsoft-Exchange-Diagnostics: 1; SN1PR0701MB1950; 20:v5zYSmiflVUJakqVgN7k9z7USh+w4WQZrshCRIOBKfD7xsq9ttyx4Q4slsKRJnRvr6t4Wb0v+EBIBUJulCl/ByCHHnA6yNBLPffjkc7yEiXaJ0Rep1FPe/w6cBYuKCaGWVwz24BNSJitFZc0NMIap+LxUeDVAj7be2/GKJpv6J2/DYU0ETG1eO6O+9SxcL6tZVN1qI0y3l/cS8RMYE0pPiCjVBhucTTikyBFLQUYtsIf/xMcmsKUhTeYHq6ETwJduqo6hRCn+8maGXGK9R03C5Sp6rHPMB98FFiqK2Rxsgge0Kz9U3tBksJnxC1aEwtewXteQmkttMH9VCOkXVCMuXKr6eE4T5282DGLOrTUyRhnCvfoYcUzXX8oO8CXMdfPW58+j5soaRYAgS8mJ2kvbZCCY4p8pEKma2WGwTBMku5bCPwhFutkZjLtPIj7Cbwm6X180jdLGGnsE0ijXB+izGx0NGK4+z3YEMcNYkJhc18aUpOHytSCyb/EQHjUk6Lw; 4:ajAA+VmCw0GyNMGl7Cb4cNImgJNiSto4NDw6kuKM/kEiduu+3250w7Cu40xVXyTktFdptJ4PQ5HOib9Yxbu4F/XvNBI8iVZk8wOauA3p/cV988doyOWC2vmdMR9ovpukxTjzj9Ocb49YypenStfB4CEijMPloeSKoaRjh5Ru9ps5dqIjLpuswZHna8rJw6j15a0s+UwyavrJKf5fL7F4mKRHiyOpEZ5nho8NCYI+pZBK3l8aVwctKzcec7Ghctv1 X-Exchange-Antispam-Report-Test: UriScan:; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(93006095)(93001095)(3002001)(10201501046)(6041248)(20161123560025)(20161123558100)(20161123564025)(20161123562025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123555025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095); SRVR:SN1PR0701MB1950; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:SN1PR0701MB1950; X-Forefront-PRVS: 045584D28C X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; SN1PR0701MB1950; 23:dVr0/2UbuEKcf0NA9ARU9Ss4i2NdvHEi8UBx+HE?= =?us-ascii?Q?QCs/lMfRfKItad61ldU1FNYne6KHKcyJOcoE9vptbVHN1hxxSPzATujiHHfZ?= =?us-ascii?Q?mtFUTNbhI51lq//rZ/4fj3DvlJjCt5QG+Aee6Lry4Myvzt84ukaCFS31yXFf?= =?us-ascii?Q?mvKz7dvA6jQryqDdCHN9HoIMZJAbuqeYE/fm8uDEPTc2w3S7bTbMv0OKheOX?= =?us-ascii?Q?uLapWLTm3ixqgA3ZrdVYaCnA6Dx3GHVWEdiqKuJeRvpkcRRDpAHiCWtOBWv3?= =?us-ascii?Q?8PmQIn8h1US6nLZLsY+lvbGwhCmGxVYeJVIwQm+x8A25o23UMg5mQGrdngjs?= =?us-ascii?Q?EMVpWABJJ0ffXV9hxZT6TfvZauNI3Zi7XFsdhqZIspAvw2hxi2+46P4OHtrI?= =?us-ascii?Q?VaHSLXmiBA1LSpGI+S/u+5QFhTbTMeRkCtpv8i01vlHAM1yM9vfFlvmLUcfi?= =?us-ascii?Q?PIy+X0QlwtSKETiUiYgiX+Dpozep3SRVwX7LFvKYtCWTGhReP2bYVcqqrXsc?= =?us-ascii?Q?052anpAto9+anIN46TELXTw1Fo9wf5Db64K9icD5W258VwaUipXetGzXx7rS?= =?us-ascii?Q?YNPIR6IClaDCEQSi6cn4MyrNLmuMPONlWS7s/xddnAjHkqziush5cHJCQFvu?= =?us-ascii?Q?7VY1NafX3c26DAIqiKgI3MVYQKbcsOX8aWrpx7htO7Qo8k5qjTU/od/J6dHS?= =?us-ascii?Q?O4vkVdZDiedhjKjJ3cq6yd0QjLDILTYLpziDTkwr459By+r0WEB5FlkBhPWt?= =?us-ascii?Q?yKDcUACPHUdQK8Q0R6rNVTBTf+dY2RXyV0r5NRFAZVfrOH0tXVXhqrzg7dZJ?= =?us-ascii?Q?Sbt0eoSW1kCO12S+Tq5S+haqpXKTFr54k3vFNTq8SFWRE2os6DoN66044WGc?= =?us-ascii?Q?u2fStTJvo+YPNIAZITmuokyzk4kB9K7pppfBju2Xs2QDXecGJeF0Aen8gv/2?= =?us-ascii?Q?S2JmVMFuORervzqO8i92Jr+he+M/xbsfiUhTrRF0ShHhjfNlSmkyAJCzONwN?= =?us-ascii?Q?QxgS4mdIKhPKjik2kVG8AzRx7?= X-Microsoft-Exchange-Diagnostics: 1; SN1PR0701MB1950; 6:9FggZKWM0zrPkbd2CMWDFLGEqi2q/CeMKPu8W2rm+8xvUFuZ8FMRXqOeHDLBNXhG6jZ9u5k7c30TAUhABko1m9DpuwlcNEY6HeF8VxhxKKPbnwIysxht54vkV3lyio6s4JyFlWHx9uCrPTsV6fE+ojifQ/R9V1e5iFdXQaJfV71CF443qNqWDNwU/pcttVnE/dmXMQqCuo5COlog+jZ0DvFFyrcJ/kdI6yJJ06w8mR/WfWfsUZKBNTIqQJaIcT8KapSM54mmoegOWCk3YEzWZWxe/mXEPvfMCRFam3MMwcywcbOJymz/mNt9iyx7qQA+m/IhAXDSuacXYgOJ7OwcVA==; 5:0kMWaZeytMk1hU6EOXstiT+fNjo34HTnChIPZcjrDxJtB6oPHYlGUYK2vAivcMtteTApp8U4Aum+57YmaBZoi9judZ6bA0gasRIWFdSdCRWpGgD0q3RbduWewJPVjJvw+ky6HnsBpWnq5VMJM6n3cA==; 24:2RC0I2ROg/LSfDT5jOogIcJbEkX1UUB66M40FzZswJQkPQT3/9Lqmi82qdJF+Z7/Xn9pPobNPn9s+zLiCZAwpjzujfJH0NQ/VjJM5q4WinM=; 7:H0IAJyDDDlv8hRB4qRZfBiu3/pxiPsNksREY/qvyxbCMldBOr7R7RFdQYOufokbxFq1Zoc0ZwD6LIlreGNJbGWRixE/R+sm9LRwgTobYNe/lHmUVjXszfhRw1M0CrDdZ1M0wyvnQ7fSTQ4YTSDAiLvR9JaJPnXd5V5mHoXRmY0056fCON1cARN7hTlF/v8FFLzbMqyGlosQaxQZNUrRLT5wphBu0Cj3M/c1aFR3LSWU= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: cavium.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Oct 2017 09:38:17.1899 (UTC) X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=711e4ccf-2e9b-4bcf-a551-4094005b6194; Ip=[50.232.66.26]; Helo=[CAEXCH02.caveonetworks.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR0701MB1950 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP iWARP uses 3 ll2 connections, the maximum number of bds is known during connection setup. This patch modifies the static array in the ll2_tx_packet descriptor to be a flexible array and significantlly reduces memory size. In addition, some redundant fields in the ll2_tx_packet were removed, which also contributed to decreasing the descriptor size. Signed-off-by: Michal Kalderon Signed-off-by: Ariel Elior --- drivers/net/ethernet/qlogic/qed/qed_ll2.c | 29 +++++++++++++++++++++-------- drivers/net/ethernet/qlogic/qed/qed_ll2.h | 9 +++------ 2 files changed, 24 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.c b/drivers/net/ethernet/qlogic/qed/qed_ll2.c index 250afa5..75af40a 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.c +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.c @@ -1105,6 +1105,7 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, struct qed_ll2_info *p_ll2_info) { struct qed_ll2_tx_packet *p_descq; + u32 desc_size; u32 capacity; int rc = 0; @@ -1122,13 +1123,17 @@ static int qed_ll2_acquire_connection_tx(struct qed_hwfn *p_hwfn, goto out; capacity = qed_chain_get_capacity(&p_ll2_info->tx_queue.txq_chain); - p_descq = kcalloc(capacity, sizeof(struct qed_ll2_tx_packet), - GFP_KERNEL); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_descq) + + (p_ll2_info->input.tx_max_bds_per_packet - 1) * + sizeof(p_descq->bds_set)); + + p_descq = kcalloc(capacity, desc_size, GFP_KERNEL); if (!p_descq) { rc = -ENOMEM; goto out; } - p_ll2_info->tx_queue.descq_array = p_descq; + p_ll2_info->tx_queue.descq_mem = p_descq; DP_VERBOSE(p_hwfn, QED_MSG_LL2, "Allocated LL2 Txq [Type %08x] with 0x%08x buffers\n", @@ -1359,11 +1364,13 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) { struct qed_hwfn *p_hwfn = cxt; struct qed_ll2_info *p_ll2_conn; + struct qed_ll2_tx_packet *p_pkt; struct qed_ll2_rx_queue *p_rx; struct qed_ll2_tx_queue *p_tx; struct qed_ptt *p_ptt; int rc = -EINVAL; u32 i, capacity; + u32 desc_size; u8 qid; p_ptt = qed_ptt_acquire(p_hwfn); @@ -1397,9 +1404,15 @@ int qed_ll2_establish_connection(void *cxt, u8 connection_handle) INIT_LIST_HEAD(&p_tx->sending_descq); spin_lock_init(&p_tx->lock); capacity = qed_chain_get_capacity(&p_tx->txq_chain); - for (i = 0; i < capacity; i++) - list_add_tail(&p_tx->descq_array[i].list_entry, - &p_tx->free_descq); + /* First element is part of the packet, rest are flexibly added */ + desc_size = (sizeof(*p_pkt) + + (p_ll2_conn->input.tx_max_bds_per_packet - 1) * + sizeof(p_pkt->bds_set)); + + for (i = 0; i < capacity; i++) { + p_pkt = p_tx->descq_mem + desc_size * i; + list_add_tail(&p_pkt->list_entry, &p_tx->free_descq); + } p_tx->cur_completing_bd_idx = 0; p_tx->bds_idx = 0; p_tx->b_completing_packet = false; @@ -1698,7 +1711,7 @@ int qed_ll2_prepare_tx_packet(void *cxt, p_tx = &p_ll2_conn->tx_queue; p_tx_chain = &p_tx->txq_chain; - if (pkt->num_of_bds > CORE_LL2_TX_MAX_BDS_PER_PACKET) + if (pkt->num_of_bds > p_ll2_conn->input.tx_max_bds_per_packet) return -EIO; spin_lock_irqsave(&p_tx->lock, flags); @@ -1858,7 +1871,7 @@ void qed_ll2_release_connection(void *cxt, u8 connection_handle) qed_int_unregister_cb(p_hwfn, p_ll2_conn->tx_queue.tx_sb_index); } - kfree(p_ll2_conn->tx_queue.descq_array); + kfree(p_ll2_conn->tx_queue.descq_mem); qed_chain_free(p_hwfn->cdev, &p_ll2_conn->tx_queue.txq_chain); kfree(p_ll2_conn->rx_queue.descq_array); diff --git a/drivers/net/ethernet/qlogic/qed/qed_ll2.h b/drivers/net/ethernet/qlogic/qed/qed_ll2.h index a822528..9bdd08f 100644 --- a/drivers/net/ethernet/qlogic/qed/qed_ll2.h +++ b/drivers/net/ethernet/qlogic/qed/qed_ll2.h @@ -63,17 +63,14 @@ struct qed_ll2_rx_packet { struct qed_ll2_tx_packet { struct list_head list_entry; u16 bd_used; - u16 vlan; - u16 l4_hdr_offset_w; - u8 bd_flags; bool notify_fw; void *cookie; - + /* Flexible Array of bds_set determined by max_bds_per_packet */ struct { struct core_tx_bd *txq_bd; dma_addr_t tx_frag; u16 frag_len; - } bds_set[ETH_TX_MAX_BDS_PER_NON_LSO_PACKET]; + } bds_set[1]; }; struct qed_ll2_rx_queue { @@ -101,7 +98,7 @@ struct qed_ll2_tx_queue { struct list_head active_descq; struct list_head free_descq; struct list_head sending_descq; - struct qed_ll2_tx_packet *descq_array; + void *descq_mem; /* memory for variable sized qed_ll2_tx_packet*/ struct qed_ll2_tx_packet *cur_send_packet; struct qed_ll2_tx_packet cur_completing_packet; u16 cur_completing_bd_idx;