From patchwork Fri Jul 1 06:10:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Zhijian Li (Fujitsu)" X-Patchwork-Id: 12902739 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6DA9C433EF for ; Fri, 1 Jul 2022 06:10:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232809AbiGAGK3 (ORCPT ); Fri, 1 Jul 2022 02:10:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232866AbiGAGK2 (ORCPT ); Fri, 1 Jul 2022 02:10:28 -0400 Received: from esa10.fujitsucc.c3s2.iphmx.com (esa10.fujitsucc.c3s2.iphmx.com [68.232.159.247]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35F4A1A83D for ; Thu, 30 Jun 2022 23:10:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fujitsu.com; i=@fujitsu.com; q=dns/txt; s=fj1; t=1656655827; x=1688191827; h=from:to:cc:subject:date:message-id: content-transfer-encoding:mime-version; bh=xLIpJZxZHpDAPGCWRK2e/Tzy5c7Q5EzbcG7vF1Yi2y0=; b=WpUBfWfjpyr4Z6qZJoLlLIPMQszSwRHr40g7vmImK7kUgIPmcIdVWSF0 WyW5BQkbtXsz5jUS54lOZYkl7jRz9e/YHLXwr6ukL8h2+MH1OO93fSbeP 2Y4HWPIqn+QWY18Fyf/v/YQcM6gKVDEiHJw39zGaPHQ0UhUGEdGE+2bTG kO0LRvXfNd1F42qYfbOkmP6Uf+4ZAQrnsX4IIEn03ZQepm9mdP1CjA/Xk aMf3fg3S4CKZ65WfMgDgfWyULV2BC91jwX32N/bm1AEYEF7/gPgwkGiVj PtqGRE80HRwVvBQxe46aR8knQnXRP9Y5VA1uL4+7yHLAaaqwpNX0NYEzJ w==; X-IronPort-AV: E=McAfee;i="6400,9594,10394"; a="59519661" X-IronPort-AV: E=Sophos;i="5.92,236,1650898800"; d="scan'208";a="59519661" Received: from mail-tycjpn01lp2177.outbound.protection.outlook.com (HELO JPN01-TYC-obe.outbound.protection.outlook.com) ([104.47.23.177]) by ob1.fujitsucc.c3s2.iphmx.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2022 15:10:23 +0900 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JkExqr0YTG7jEhrh9ezZYRjUSyvqzu7vwoB3oI6pMnKziMva+ZWNqb2PmOM5ceYOWJZ49ddDv4RzumFu/mNExsNvUA3mhk9MJ3F9LgUYmX+hlHgHPwFeEoUWXcyHySo7SeWWzvsJif+Lr0EKAoH74DP7uvx5XzJU+zMw8eOZ8aPaosMn77lNzffasmE45ZCFgNk0aUmqhLXETHPL0brrfeAI4p9UdNRunZlKukNpx5UuxNjpAH32vmCUsyNDkzRg80UffpwO4547skfOhPi9oPCx1obEVU/0D6czforxZEHnCGY1fFUJWFbu6NqxhEN+1I27XepZiMi+R/ddt3VJXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=xLIpJZxZHpDAPGCWRK2e/Tzy5c7Q5EzbcG7vF1Yi2y0=; b=YbFNBjcy8UfrowqW70xmIHOLTLpweKwqiG92/Ju9NXa8fxW+o5ve7MPIoXclJpzSBZ3WeM0kiWBgmNtOqA9RQ1WdrpVrdhdIyCjDu95eUJmMLtc6SK3XYdtNRRUlgTtfxG3EJx2pm25wHRx5YHZ9sUPixDPdFYs2s44qzkHto1IwkDFjBXDKsAlH8RXBo/d4n1Z1zrhrkJq4BiTxMajymdvqeIwhD00G9QUDDD1C6n1ElGqCXB1n/C3p78iqiENzc9WpY4G9QjtovjfXp2ANPm4gE3ACycc2vHQsyN0a5/X1VTKKKFv4OpYkMqKRNkFGVlKz05/Cbxih135DyLxOEA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=fujitsu.com; dmarc=pass action=none header.from=fujitsu.com; dkim=pass header.d=fujitsu.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fujitsu.onmicrosoft.com; s=selector2-fujitsu-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=xLIpJZxZHpDAPGCWRK2e/Tzy5c7Q5EzbcG7vF1Yi2y0=; b=o7abGmws+lWkiD2wMTo2mPJxxlk8VR/dZJzuUsutwws8MjapU3z4PYQapYQKyM9ukCbyjE4hNfxQcTlijMdpWEXekYQdBpFy4rafX0cDzITIUnnBkc0OLV51Ezj6ymKpV210MJj6RyKtAH9VqYVo2zQ1FrjRwtazXDvGlL76j60= Received: from TYCPR01MB9305.jpnprd01.prod.outlook.com (2603:1096:400:196::10) by OS3PR01MB6626.jpnprd01.prod.outlook.com (2603:1096:604:10c::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5395.14; Fri, 1 Jul 2022 06:10:20 +0000 Received: from TYCPR01MB9305.jpnprd01.prod.outlook.com ([fe80::8d7a:baa8:3b18:cd93]) by TYCPR01MB9305.jpnprd01.prod.outlook.com ([fe80::8d7a:baa8:3b18:cd93%6]) with mapi id 15.20.5395.015; Fri, 1 Jul 2022 06:10:20 +0000 From: "lizhijian@fujitsu.com" To: Yanjun Zhu , Jason Gunthorpe , Haakon Bugge , "linux-rdma@vger.kernel.org" , Bob Pearson CC: "lizhijian@fujitsu.com" Subject: [PATCH v4 0/3] RDMA/rxe: Fix no completion event issue Thread-Topic: [PATCH v4 0/3] RDMA/rxe: Fix no completion event issue Thread-Index: AQHYjRE9+VFNFEqc1ka0ZuIEnP5w+w== Date: Fri, 1 Jul 2022 06:10:19 +0000 Message-ID: <20220701061731.1582399-1-lizhijian@fujitsu.com> Accept-Language: zh-CN, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.31.1 authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=fujitsu.com; x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 5a40ca21-8e9b-4aa0-3f2b-08da5b2860d5 x-ms-traffictypediagnostic: OS3PR01MB6626:EE_ x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: GfKXV2arAYQ2hmLJeCiBhoSUsTt2JSpk6IIRbcAwK1IsX+V1Owq0zNbQYCoEs26Rn+rXeUh6+QzErJsyym+CunW9NUzxT4kRrl1nr/Ynfv1RA8PXQ9XEwIucZGyJ4ODcShRYD88dtjwy+xwPqysTWmgmAUA4wBiug0sLx4WZ+Vt5dWlZi2DqHmADO5jI/phz4y2oxnn84rvhjkBrs660DcCn1CM1eQCnJPSj+9Wv97x30qnuixV1VGttq8F2CqAeICzvaPEynwu1l+bFMyxvonsgGKgza3RLlMEpk4Lhj0IiYcS22NqeEUrN1t0OKahy5+GuKZAKP/wW7YStq7Ccdu3ssOTYElvWsMdOZ50ihUhZprS+Qxw5j+EjUaZRw2YUl20ASLmwgV26TzT73qfoDYef/vGBu0wAEwZR1y9GpjNZOaq+DqsvxjaOEoSHlpj/WuUJ++ikvRu7/457nLlRatUmZOOccE9yGh5u6vpUMwx1UiPBF/4gCggPKdYBvX0SbL21f+VPPS+iBCc+ugDtVSR0VJvdo8Ajs2j2HaD2u60l22yCAIj3v2lZOye/2X3MmG+XAc4XM2IMhyosqZP3NBDohjGzNH4wZMBqTy6T258wufHz2059NQB7O/ZKM9FmrwA+R3mDTyM2sSYUGU3IXG6Lw0L1AXSkPaHuQlXfIOQbamYTSpERO/1ghsKjejXtK+3tZsjwzFn9uUIT0Polrl4iDy0X6Bb3atS3Ivqpf62oEJsvgVWV81IhvGx0g7WBg9jgIDh/Vmqk2dRZUOEmtrJnVr4lM4SjmrafXEgzvsUc/erC8GTuDbWwcdWJJ4aD x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:TYCPR01MB9305.jpnprd01.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230016)(4636009)(346002)(366004)(396003)(376002)(136003)(39860400002)(5660300002)(36756003)(4326008)(64756008)(110136005)(8676002)(122000001)(8936002)(2906002)(38070700005)(66446008)(91956017)(186003)(41300700001)(83380400001)(85182001)(66476007)(316002)(76116006)(66946007)(6486002)(71200400001)(66556008)(82960400001)(38100700002)(478600001)(107886003)(86362001)(4744005)(1076003)(6512007)(26005)(2616005)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?eucgb2312_cn?b?SVJtbHVNMmdUR2pQdHFuQXB0?= =?eucgb2312_cn?b?bUo2Z0RETTVtbHMwbTBWTFFYRG5nK1J5OC9pdzZwZGRCS2FvdTdXbEJoTFNCLzB3?= =?eucgb2312_cn?b?eUI4RWh2WTN1MkdqbjJ1TWtMNDEzVnYwMEpZTDN4cDBLUXo3QWx5WmkvWmNYWndV?= =?eucgb2312_cn?b?dSsxL09Qa01LWUppOU9PbmpnRnJqWHJWUW5SVzRHZmFaeXlWclpIL0gyYUx3WFlU?= =?eucgb2312_cn?b?REZucmtrVnZPN2VmTzBFd3B6Z3J4Y0RFWkx3d3RUR1lvSXAvaW9Kd240dlhJeFM4?= =?eucgb2312_cn?b?QitoaWdOUTVma2g3cENWR3RSWWg3Rm0yVldlUWZNbHBqSVZpUFE5YWMrUm9ncTdY?= =?eucgb2312_cn?b?SzMyd0dkOTVrQ3MxUkdiZ2lPVmQ3aEFtbHFiRTUzV3M2WmhlTUVJOHEwbHczYWx5?= =?eucgb2312_cn?b?eFgxa1dpKy9HdmVELzhYeEJSVTZ0WnArWUVtNC9sQ044Z1ZIUXhESXVtT1VvQ0ps?= =?eucgb2312_cn?b?VDZiampvb1U4UDVHckNsNTFSYVU4R2JpRnMrblRnMDIwTUpkMHZpSTZWbVlvcU9H?= =?eucgb2312_cn?b?eFV2M3Fxd1J0M3lYNVA1dGNPVDFpd3plZUhVdElDNmQ4M3dCTlFndmdnR29CSVMz?= =?eucgb2312_cn?b?bXM3M21MZGhoS1lkZTJ4aEcybmNROUlsTFBqQ0xmTVo2dmgvQzRNcDBEQmk3clhD?= =?eucgb2312_cn?b?UEZXcVBYK3I1ZzBpSUMyRkhsbmUrUGYwR0lWcjRrMm1mbWlaQkYxbC9pdFZEckpj?= =?eucgb2312_cn?b?eTlkOEw3ZTVhazd2T1JWYjZrdW5BZkRMVUxoVk9oVWtjTFltNE04aU1kSTFiSi9q?= =?eucgb2312_cn?b?OW5zYlBGUTBvTHN0dFBEVHpBRW5mRnkzemIrVnhpT3Z6OHNUWVVOVEJPd0lUTWxz?= =?eucgb2312_cn?b?L0p2c25pMm1mQllQVkJGSzI0UXhwdDlUbzUvRWtBL3VzSmRxNGxCT25XLy9JL1R0?= =?eucgb2312_cn?b?ZnJQZndDZWlMQWdES1VDVEk0bzZCcjljWi9iTE1wcDBFWDFHd2xQV3VOYVViZXJD?= =?eucgb2312_cn?b?akw4ckhIYlZaNWp3TWJrbi9tSmRzU2xuYXF6Y2dnQUF5Yi9RYkFnempqQnVrRGtl?= =?eucgb2312_cn?b?YmtneWpOYncvSjZMcDAyVHFRQW5ZaUd4NytWWnJXYnVQWjc0dFkzRlVlR2xVOW9o?= =?eucgb2312_cn?b?SEZVajc2SXpDcEZPMXhkOXZJazBmU1pTRVo3WTYrUURiU3lFN2FkSGVKZEFjcE5x?= =?eucgb2312_cn?b?Z29PaFY4Yk1iNFA1Mnp5SHZRZGhIR01HeHUwc1g2bE1nOWsrYVNwSzRLT1FIWVk0?= =?eucgb2312_cn?b?MFZpR2syZTViM2NRL2hlYmV1d2xTSzBZS2QvUWd5R29jV0dGWTNtSitlS01odGNi?= =?eucgb2312_cn?b?azNBQ1EwUjN0aVZLbXNyNGwydFhCNU9NNURyYzhQVmNDQjk1WlRaWnQrZWUveXU0?= =?eucgb2312_cn?b?ejhjQnRSaUU0RmhBdlhyN2FaUWNFN2U5N1BHQ3dyNndCRnh6WXFiMHJCMldHNCtk?= =?eucgb2312_cn?b?cXF3ZUJuYkJFUjJocWVZOEUyK216MkJzdUQzNGI1K041TGNWdkk0WjFhaVI4QnhS?= =?eucgb2312_cn?b?OVR6M0lCZ1pGZUpFS3ZkaFAxZjJRS3ZjcGRpQml6RXJUT1M0QVVjdi9WcmlWWHRD?= =?eucgb2312_cn?b?WHNsa2pRQUVRUnJXeElybG1POWFncVZURmsrdm5URXhGZkZ3eVlTc2V4MDBZU2hN?= =?eucgb2312_cn?b?UUdKYUxkdG02TEw5QU9WMmJRb1oyenZDRVRaazFwT1F0Ri9JamtZVU1RMU0zV2hv?= =?eucgb2312_cn?b?L1UxVGpMS2dJMnpKZ0F6dWlOTnkwbEF3NFU3MXdtS3ROMFpYZkdhUXk2L0J6NmRG?= =?eucgb2312_cn?b?MTMyYW95YTlVeUtRZWNhcTc0VS82QkRPWXZTa3pBbFh5OFVhR0tjdy8vOWxVbm5h?= =?eucgb2312_cn?b?NXBQZyt3WmtIQnF3QWpQUVJmT09QQW5jQlpsMXJJeG1GTVdPTDRyd05lQmJYNmor?= =?eucgb2312_cn?b?MG8yMjdNRE1uSkpvOGFXZjc3azNLN2pIZjRaZGhyM2hXL1d6Vjk0eFJEaUNmMDNJ?= =?eucgb2312_cn?b?NDVqdTB3SXNLdDVuT0FoY21JVGVvZFpEbUxrSW1RTEZhQjFqUFp4dlRPN3BZOUdM?= =?eucgb2312_cn?b?cFlaWGt4ek5YSjM3djBWSTVqbFBIRHJBTU1HVFFUYWJSUWxUeDhNT2toSXRiWnpi?= =?eucgb2312_cn?b?Vm5PY3VxVVdWNllOcHcrdmpOY2VWQ05qcVc4SEhRU0xHVXdldjFqVDZkQkVVSWFE?= =?eucgb2312_cn?b?Um9DSnVvRzNZSmJzZGYxWFRjcWc9PQ==?= MIME-Version: 1.0 X-OriginatorOrg: fujitsu.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: TYCPR01MB9305.jpnprd01.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5a40ca21-8e9b-4aa0-3f2b-08da5b2860d5 X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Jul 2022 06:10:20.7129 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: 09m/+k+IL5BLFsUPYnHzCG6neHKlPbRipNYxbYQ50V5fv5gPFyTyjqerkhl2Vo2KU3pRnzWIOGXVqsayRa/WBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: OS3PR01MB6626 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org It's observed that no more completion occurs after a few incorrect posts. Actually, it will block the polling. we can easily reproduce it by the below pattern. a. post correct RDMA_WRITE b. poll completion event while true { c. post incorrect RDMA_WRITE(wrong rkey for example) d. poll completion event <<<< block after 2 incorrect RDMA_WRITE posts } V4 add new patch from Bob where it make requester stop executing qp operation as soon as possible. Both blktests and pyverbs tests are passed fine. Bob Pearson (1): RDMA/rxe: Split qp state for requester and completer Li Zhijian (2): RDMA/rxe: Update wqe_index for each wqe error completion RDMA/rxe: Generate error completion for error requester QP state drivers/infiniband/sw/rxe/rxe_comp.c | 6 +++--- drivers/infiniband/sw/rxe/rxe_qp.c | 5 +++++ drivers/infiniband/sw/rxe/rxe_req.c | 18 +++++++++++++++++- drivers/infiniband/sw/rxe/rxe_verbs.h | 1 + 4 files changed, 26 insertions(+), 4 deletions(-)