From patchwork Wed Sep 28 20:24:12 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Parav Pandit X-Patchwork-Id: 9354859 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id AE7456077B for ; Wed, 28 Sep 2016 20:24:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A11332871E for ; Wed, 28 Sep 2016 20:24:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9589F29735; Wed, 28 Sep 2016 20:24:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7F1F62871E for ; Wed, 28 Sep 2016 20:24:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753705AbcI1UYQ (ORCPT ); Wed, 28 Sep 2016 16:24:16 -0400 Received: from mail-pa0-f67.google.com ([209.85.220.67]:35026 "EHLO mail-pa0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752899AbcI1UYP (ORCPT ); Wed, 28 Sep 2016 16:24:15 -0400 Received: by mail-pa0-f67.google.com with SMTP id j3so2454408paj.2 for ; Wed, 28 Sep 2016 13:24:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:cc:subject:date:message-id; bh=owKD2ZaKZJVVkQOo1POG/twgD4wpR3Q6Y19KdYCzU+g=; b=d6DaRBNcL6jCwcEauyImduHr7FIU63vezBNVZJND/T+o5tQafJuxVVFhMz8aaFIVBb gjXu6gtTRPZM/0CWhYsMM+ttEXLIvPGbvMSIb93m1idG6IpHlhVoeKek/Xyjdoc0j40I iwD8ZO1iMHksSTelEuCVrvJWnSKq4XtPLowWd/qo0c0lAY6CDN2aGH69Q4a7rpKr8yCT pdDnd3q2ePFfL5GmmbpR87Psk29EAumNNVZ5Ph8MbKssTZQyM7ESmfLizWyzBGQ8qypL DQIl4ZOo6924ipbmdHBPrrO4NmoX9+oCIcgY0v88QBLpSPcVSArKRH+wmlo1Q4ZwTAV/ HtKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=owKD2ZaKZJVVkQOo1POG/twgD4wpR3Q6Y19KdYCzU+g=; b=gB3dNC2RQ2x3EM19+tSNICqHpVEih40Kticq1Njb3S/nyqv3UU0ljeo6fd54udW5a3 gsZzHBh9FI1qPH6q3j+LVNO50DS2mkjTYGC6D5pQTEf55MOLNQEbaxTuV3ioMvAbTqyo EkTKrHB3A/BsHUsTvrUe0kk7HFN5yPxl11LmnuHoceacOo6gwm/tlKwb2J7+8aZx0t6H 6h1VbFK84ShgY+01w4uHTUEQnMd+3HEzWdpgPGxgiBLIMRvkuLu2l1bG6yja2tD7I9yR DB/iR8kxgKrqj4Oq8yGju4DGbclbPBYGDxs2M0yUIgWbG41y6uNFZQkGnKq72GnSxrVt l3wg== X-Gm-Message-State: AE9vXwPPhFASIpsHgj9VekZvQm8dymuOc8BivdUeABgHROSjrQOCu0olQwIoFBK0cuuljQ== X-Received: by 10.66.248.229 with SMTP id yp5mr60457268pac.31.1475094255186; Wed, 28 Sep 2016 13:24:15 -0700 (PDT) Received: from ip-172-31-4-47.us-west-2.compute.internal (ec2-52-43-58-33.us-west-2.compute.amazonaws.com. [52.43.58.33]) by smtp.gmail.com with ESMTPSA id n19sm14535978pfi.50.2016.09.28.13.24.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 28 Sep 2016 13:24:14 -0700 (PDT) From: Parav Pandit To: monis@mellanox.com, linux-rdma@vger.kernel.org, dledford@redhat.com Cc: pandit.parav@gmail.com Subject: [PATCH] IB/rxe: Avoid scheduling tasklet for userspace QP Date: Wed, 28 Sep 2016 20:24:12 +0000 Message-Id: <1475094252-18304-1-git-send-email-pandit.parav@gmail.com> X-Mailer: git-send-email 2.7.4 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This patch avoids scheduing tasklet for WQE and protocol processing for user space QP. It performs the task in calling process context. To improve code readability kernel spacific post_send handling moved to post_send_kernel() function. Signed-off-by: Parav Pandit --- drivers/infiniband/sw/rxe/rxe_verbs.c | 38 +++++++++++++++++++++++------------ 1 file changed, 25 insertions(+), 13 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_verbs.c b/drivers/infiniband/sw/rxe/rxe_verbs.c index 4552be9..a5af691 100644 --- a/drivers/infiniband/sw/rxe/rxe_verbs.c +++ b/drivers/infiniband/sw/rxe/rxe_verbs.c @@ -801,26 +801,15 @@ err1: return err; } -static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, - struct ib_send_wr **bad_wr) +static int rxe_post_send_kernel(struct rxe_qp *qp, struct ib_send_wr *wr, + struct ib_send_wr **bad_wr) { int err = 0; - struct rxe_qp *qp = to_rqp(ibqp); unsigned int mask; unsigned int length = 0; int i; int must_sched; - if (unlikely(!qp->valid)) { - *bad_wr = wr; - return -EINVAL; - } - - if (unlikely(qp->req.state < QP_STATE_READY)) { - *bad_wr = wr; - return -EINVAL; - } - while (wr) { mask = wr_opcode_mask(wr->opcode, qp); if (unlikely(!mask)) { @@ -861,6 +850,29 @@ static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, return err; } +static int rxe_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr, + struct ib_send_wr **bad_wr) +{ + struct rxe_qp *qp = to_rqp(ibqp); + + if (unlikely(!qp->valid)) { + *bad_wr = wr; + return -EINVAL; + } + + if (unlikely(qp->req.state < QP_STATE_READY)) { + *bad_wr = wr; + return -EINVAL; + } + + if (qp->is_user) { + /* Utilize process context to do protocol processing */ + rxe_run_task(&qp->req.task, 0); + return 0; + } else + return rxe_post_send_kernel(qp, wr, bad_wr); +} + static int rxe_post_recv(struct ib_qp *ibqp, struct ib_recv_wr *wr, struct ib_recv_wr **bad_wr) {