From patchwork Fri Jun 28 22:35:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 11023505 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0778F13B1 for ; Fri, 28 Jun 2019 22:36:11 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E07D28812 for ; Fri, 28 Jun 2019 22:36:11 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 128FD28875; Fri, 28 Jun 2019 22:36:11 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9749928812 for ; Fri, 28 Jun 2019 22:36:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726931AbfF1WgJ (ORCPT ); Fri, 28 Jun 2019 18:36:09 -0400 Received: from mail-eopbgr70049.outbound.protection.outlook.com ([40.107.7.49]:23750 "EHLO EUR04-HE1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726695AbfF1WgJ (ORCPT ); Fri, 28 Jun 2019 18:36:09 -0400 ARC-Seal: i=1; a=rsa-sha256; s=testarcselector01; d=microsoft.com; cv=none; b=UxSEbw0ofDOQI2XWEAmSSgBV1YOlkS4pYqVy+TU8COk2BgE8yTCw1H9zFuz4XYpDnSjQCerj0V3HybpK/d1rKw2t+VvY43CnRqkLfBfm40HLWTFbp6bckjN9W2IHtUfDKacjkuCS534ABT9979qTp3fRMrz4R5J+BJtbVFZAv8M= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=testarcselector01; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kJ+uCPUzHLJNEY4ko/vvh8JMZUWh02qkbmS68vqidB4=; b=RnQNf5/Cn/mtPLnBDb9zrT0dOJvDbdpHFrpnmP+hN7llJahITeY45LZHm9YhEBpJZazZtg08oDKrhR3fbb6HoXGL7947tw8H5MVXhI7IeEPAnXuPiKXXqXrVnRuMpwD6wXzRhP2Wy0a/C08U+nRlRezi+32oURnkqQmx12+8S50= ARC-Authentication-Results: i=1; test.office365.com 1;spf=none;dmarc=none;dkim=none;arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kJ+uCPUzHLJNEY4ko/vvh8JMZUWh02qkbmS68vqidB4=; b=rZfG0ga6RVWsMMm1XlBFH4MvYshRHsVxhQyQp5IzYyq/xK8AaBPg2jl+qlyPEh4RZTg832UJsJFskTXleuNrQUJijzoOmBfHK994PBNmxBEXOFJyF6yFBKMBRIILb/6lItuiEt4GYgvYHkrcyGQGpiI47+XlVt7BjnbenUuBThw= Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com (10.172.227.7) by DB6PR0501MB2357.eurprd05.prod.outlook.com (10.168.56.24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.17; Fri, 28 Jun 2019 22:35:56 +0000 Received: from DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278]) by DB6PR0501MB2759.eurprd05.prod.outlook.com ([fe80::a901:6951:59de:3278%2]) with mapi id 15.20.2008.014; Fri, 28 Jun 2019 22:35:56 +0000 From: Saeed Mahameed To: Saeed Mahameed , Leon Romanovsky CC: "netdev@vger.kernel.org" , "linux-rdma@vger.kernel.org" , Bodong Wang , Mark Bloch , Parav Pandit Subject: [PATCH mlx5-next 05/18] RDMA/mlx5: Cleanup rep when doing unload Thread-Topic: [PATCH mlx5-next 05/18] RDMA/mlx5: Cleanup rep when doing unload Thread-Index: AQHVLgHZomX2Wz/SpE6y8RkMDT1zag== Date: Fri, 28 Jun 2019 22:35:55 +0000 Message-ID: <20190628223516.9368-6-saeedm@mellanox.com> References: <20190628223516.9368-1-saeedm@mellanox.com> In-Reply-To: <20190628223516.9368-1-saeedm@mellanox.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-mailer: git-send-email 2.21.0 x-originating-ip: [209.116.155.178] x-clientproxiedby: BY5PR13CA0035.namprd13.prod.outlook.com (2603:10b6:a03:180::48) To DB6PR0501MB2759.eurprd05.prod.outlook.com (2603:10a6:4:84::7) authentication-results: spf=none (sender IP is ) smtp.mailfrom=saeedm@mellanox.com; x-ms-exchange-messagesentrepresentingtype: 1 x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 2253c2e2-8835-42b7-360c-08d6fc18fbf0 x-ms-office365-filtering-ht: Tenant x-microsoft-antispam: BCL:0;PCL:0;RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(4618075)(2017052603328)(7193020);SRVR:DB6PR0501MB2357; x-ms-traffictypediagnostic: DB6PR0501MB2357: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:4502; x-forefront-prvs: 00826B6158 x-forefront-antispam-report: SFV:NSPM;SFS:(10009020)(4636009)(39860400002)(396003)(346002)(366004)(136003)(376002)(189003)(199004)(107886003)(6436002)(1076003)(3846002)(6116002)(256004)(66446008)(50226002)(6636002)(66066001)(53936002)(36756003)(71200400001)(71190400001)(186003)(8676002)(52116002)(4326008)(5660300002)(14444005)(26005)(6506007)(305945005)(450100002)(11346002)(446003)(478600001)(76176011)(64756008)(99286004)(386003)(102836004)(6512007)(110136005)(2906002)(316002)(66476007)(68736007)(14454004)(81166006)(486006)(86362001)(81156014)(73956011)(66946007)(2616005)(6486002)(476003)(66556008)(8936002)(25786009)(54906003)(7736002);DIR:OUT;SFP:1101;SCL:1;SRVR:DB6PR0501MB2357;H:DB6PR0501MB2759.eurprd05.prod.outlook.com;FPR:;SPF:None;LANG:en;PTR:InfoNoRecords;MX:1;A:1; received-spf: None (protection.outlook.com: mellanox.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: OwYZRE0LwElWhHs+1PmHNeeW0bcpqCBWw0J/VV4Cj+pmSS7XBTw9NTVbc1OyMXW5frI07QTuGxx4QDyn7cVXJ9KBZo5SxFtyw2OXtnZDjPATbGvVYPJrSCInS8qhwWzf/P4kGlCoeFeumSvJQaBHJzeXrRo9MIMPTxzLjFrutsQPQC2dSr5Ym3VRFkEFGx7JSb633UESHYgBCvb3M4ky857XK+ZxLBoncL2RImC125EKIOw3nI4pwme6UZorRWw3lxjylOD94+8dR+pL8L/pYfKXwzayINFxyS4Mmv42TW6yeX3fOoXg8DxR3M24dIgl/B0RjmUluzpRGmjTdIl9LHBcoWtJV2ARYF5mwa1OuDUJ99fIAwmKyH9Ho0y+K0xt6RTImdEOkB9FLbZG/jXU8AdKmVqENH8yL/dVJPpKLAI= MIME-Version: 1.0 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2253c2e2-8835-42b7-360c-08d6fc18fbf0 X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Jun 2019 22:35:55.8674 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: saeedm@mellanox.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB6PR0501MB2357 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Bodong Wang When an IB rep is loaded, netdev for the same vport is saved for later reference. However, it's not cleaned up when doing unload. For ECPF, kernel crashes when driver is referring to the already removed netdev. Following steps lead to a shown call trace: 1. Create n VFs from host PF 2. Distroy the VFs 3. Run "rdma link" from ARM Call trace: mlx5_ib_get_netdev+0x9c/0xe8 [mlx5_ib] mlx5_query_port_roce+0x268/0x558 [mlx5_ib] mlx5_ib_rep_query_port+0x14/0x34 [mlx5_ib] ib_query_port+0x9c/0xfc [ib_core] fill_port_info+0x74/0x28c [ib_core] nldev_port_get_doit+0x1a8/0x1e8 [ib_core] rdma_nl_rcv_msg+0x16c/0x1c0 [ib_core] rdma_nl_rcv+0xe8/0x144 [ib_core] netlink_unicast+0x184/0x214 netlink_sendmsg+0x288/0x354 sock_sendmsg+0x18/0x2c __sys_sendto+0xbc/0x138 __arm64_sys_sendto+0x28/0x34 el0_svc_common+0xb0/0x100 el0_svc_handler+0x6c/0x84 el0_svc+0x8/0xc Cleanup the rep and netdev reference when unloading IB rep. Fixes: 26628e2d58c9 ("RDMA/mlx5: Move to single device multiport ports in switchdev mode") Signed-off-by: Bodong Wang Reviewed-by: Mark Bloch Reviewed-by: Parav Pandit Signed-off-by: Saeed Mahameed --- drivers/infiniband/hw/mlx5/ib_rep.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/drivers/infiniband/hw/mlx5/ib_rep.c b/drivers/infiniband/hw/mlx5/ib_rep.c index 1de16a93fc64..3065c5d0ee96 100644 --- a/drivers/infiniband/hw/mlx5/ib_rep.c +++ b/drivers/infiniband/hw/mlx5/ib_rep.c @@ -17,6 +17,7 @@ mlx5_ib_set_vport_rep(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) vport_index = rep->vport_index; ibdev->port[vport_index].rep = rep; + rep->rep_data[REP_IB].priv = ibdev; write_lock(&ibdev->port[vport_index].roce.netdev_lock); ibdev->port[vport_index].roce.netdev = mlx5_ib_get_rep_netdev(dev->priv.eswitch, rep->vport); @@ -68,15 +69,18 @@ mlx5_ib_vport_rep_load(struct mlx5_core_dev *dev, struct mlx5_eswitch_rep *rep) static void mlx5_ib_vport_rep_unload(struct mlx5_eswitch_rep *rep) { - struct mlx5_ib_dev *dev; - - if (!rep->rep_data[REP_IB].priv || - rep->vport != MLX5_VPORT_UPLINK) - return; + struct mlx5_ib_dev *dev = mlx5_ib_rep_to_dev(rep); + struct mlx5_ib_port *port; - dev = mlx5_ib_rep_to_dev(rep); - __mlx5_ib_remove(dev, dev->profile, MLX5_IB_STAGE_MAX); + port = &dev->port[rep->vport_index]; + write_lock(&port->roce.netdev_lock); + port->roce.netdev = NULL; + write_unlock(&port->roce.netdev_lock); rep->rep_data[REP_IB].priv = NULL; + port->rep = NULL; + + if (rep->vport == MLX5_VPORT_UPLINK) + __mlx5_ib_remove(dev, dev->profile, MLX5_IB_STAGE_MAX); } static void *mlx5_ib_vport_get_proto_dev(struct mlx5_eswitch_rep *rep)