From patchwork Mon Jul 8 12:00:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Dragos Tatulea X-Patchwork-Id: 13726516 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2057.outbound.protection.outlook.com [40.107.237.57]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E23C81ACB; Mon, 8 Jul 2024 12:02:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.237.57 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720440148; cv=fail; b=RiwZ0+9kVC4NfKXxLTF3oLH4q4TXocJI2vJigsmLsgfGtXxVzSHWdh4bMISHqC70/uBoDFEST0WxLKGeAkSUgr4+RIx5HEoCsLcFw8THNsfFlf5Jq+C8xFmnG4cgQ5/Nut24pGqj8xZ0EZR1zRchSDGFdlYZ9HC9SmRDIltrZho= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1720440148; c=relaxed/simple; bh=TwPw47efXO1B+Q3nzT6e//56UQUzyCnKn2YznnWBahM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-ID:References: In-Reply-To:To:CC; b=KcGUg7swFA9zG3Z+L8FUX38A9fgKqOPGSxjIqGWQnGLS2mAp6SC2fbTakz2r8u8sOc1OqVPwhU0Xs1F/9RhBmNzWBDg/STa2ebeW3wFcUAJgZup7qv4ZrHenY0qKxly5GHeUrxj25ZCLWjtIwSBEpM7e+Z//sRaUZxmbQUaR8dE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=U+aUWkrh; arc=fail smtp.client-ip=40.107.237.57 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="U+aUWkrh" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=V3PDOU8L32KUjmNbs3ObvUc7TLNntYYreUZsgQrlQqbLX2IP2Y3Y/dhB/tpaaYUGOdBU5V77iZTYfK5JM223Xkz3HjwHFOJlZzPrx5i6g2dneZjms6ZpxyprbTKRH7VuGouUT9MIM6Sgz+yKOPLOoaxqT+Nida9s7qndiuh17nNy/5J5V1Ujwv0vwowlDkBX1D5rsw2U7x2j7StGsVn4F8aWgnwwAl81QU8FTsos2s9hBViXSD5jyEJPFBAYGWGLB7F2boNStivIl7RUgTmyfWMWpy3b1RmdiwVT5aH6vWC0i8KNkz66h6e2ndetHLSAhix5nHPmVf7bUHJZ9XSbGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=aWB+O0G5uLWJ1mMDlgSXHc++Kxn74xpH2LYJgVRT2xo=; b=ljIlKQMhmx15uHrD3Q5fytmqY5aXLgUA7F9xjuaqhobB+p2qCL/QR5jYmt4M8S5F4+5svGvebS3ozcWe825xAHScjK/94/Vhv60KVJu938wWH2DNAboH04I3v1StwMphrmXlLBLcBTc943xPDyNLThEHIv4w53AI6NRLO4VoaTRAbM9F/eCBrbvJXjc417Ktnsv/FihhRL3g+Oaap3g26W9MJlzL8j1iFzrw8F9wZSnNzThIjwVTvSs+eE6GlarQGITtJZJSHea2hGWwhF/YiVI5JvkOZ2uhcu6EE9QInA+qCbqeElmSBveuHO/7jxmFrRpnGJuVSTBTXe1zqK0Esw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=oracle.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aWB+O0G5uLWJ1mMDlgSXHc++Kxn74xpH2LYJgVRT2xo=; b=U+aUWkrhf7WqzVjQ9HjOt3LcNAZBGygyKVue5sGEswwyajWBKoln0vxCjhJAeaub7uYaUw9seL3MDwpUisXXlSIoYf5aiupukUIAsfla3bPJ+znVT9MktsN2jmlwGObEjExscnflWW3TYYBZEMCSd/mMAAg0EDXSOAq4hvJf53Ba9SUBZSsDuEmLZqAqUovrvsniJ3oLmDwbC1DJM8JFwZ2X93awlqRcpoCFxKAO3dQdt9a1fh2SSBTQMYZ9kliaZEyQr58+2c0ANe6g56EPI7v+/exaR1Y1FEXbLuBKTEvKGekYoV6i6DRC3FnfHsKAxWVEB+PVyGg8psMre3djWw== Received: from BL1PR13CA0270.namprd13.prod.outlook.com (2603:10b6:208:2ba::35) by PH7PR12MB6586.namprd12.prod.outlook.com (2603:10b6:510:212::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7741.35; Mon, 8 Jul 2024 12:02:20 +0000 Received: from BL6PEPF0001AB73.namprd02.prod.outlook.com (2603:10b6:208:2ba:cafe::da) by BL1PR13CA0270.outlook.office365.com (2603:10b6:208:2ba::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7677.22 via Frontend Transport; Mon, 8 Jul 2024 12:02:20 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BL6PEPF0001AB73.mail.protection.outlook.com (10.167.242.166) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7762.17 via Frontend Transport; Mon, 8 Jul 2024 12:02:20 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 8 Jul 2024 05:01:59 -0700 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Mon, 8 Jul 2024 05:01:59 -0700 Received: from dev-l-177.mtl.labs.mlnx (10.127.8.11) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.4 via Frontend Transport; Mon, 8 Jul 2024 05:01:55 -0700 From: Dragos Tatulea Date: Mon, 8 Jul 2024 15:00:39 +0300 Subject: [PATCH vhost v3 15/24] vdpa/mlx5: Allow creation of blank VQs Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <20240708-stage-vdpa-vq-precreate-v3-15-afe3c766e393@nvidia.com> References: <20240708-stage-vdpa-vq-precreate-v3-0-afe3c766e393@nvidia.com> In-Reply-To: <20240708-stage-vdpa-vq-precreate-v3-0-afe3c766e393@nvidia.com> To: "Michael S. Tsirkin" , Jason Wang , Xuan Zhuo , =?utf-8?q?Eugenio_P=C3=A9rez?= , Saeed Mahameed , Leon Romanovsky , Tariq Toukan , Si-Wei Liu CC: , , , , Cosmin Ratiu , Dragos Tatulea X-Mailer: b4 0.13.0 X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL6PEPF0001AB73:EE_|PH7PR12MB6586:EE_ X-MS-Office365-Filtering-Correlation-Id: defc8862-54ac-4982-7bc4-08dc9f45d21a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|7416014|1800799024|376014|36860700013; X-Microsoft-Antispam-Message-Info: =?utf-8?q?OOG9oE67zZb5Z678hqoiquMy5YiY5N9?= =?utf-8?q?cp1i0twG+r1ZDerPrQhyIhURMPXHcXnD4P0LO0TVFUWs8V4hO2PPoUwZ3+MdIvuOO?= =?utf-8?q?ZduKLmBfMyRuX7qeLeoJ0iuDk0VtpwyXTutcZB7t7MNXyNd7gDNzVbe7kGh9vqHNR?= =?utf-8?q?TlTNWr0DUsuC5HvtK1MtMt5eQsi/cBq+fNOMXUrA3FVGMx7IVHKPxny6mfWqXcKlb?= =?utf-8?q?RQjp6JbrzydVX75ISOOIHQbyqYZydnuFzLX89HC0f0S6slEYlCnK5Q2ma8+ENHHhI?= =?utf-8?q?a+c3Wms4E0e3da+GNKoNUFGoyBswulnfkWIo+hi97WLmHaOeUg0OpAz77jYHyDP4C?= =?utf-8?q?4WUbnhZ41XRZgvyxECj2X2dUN7fNnxB/njrlV4bEg+F4b+iKwv4r9+dy4In6iHBYM?= =?utf-8?q?os49ZOIacpDQOy0oWfrw8+FgVIJMsXq80SqJFrZhwIaH9A8eeydu5XFM5zBBm3/rQ?= =?utf-8?q?ZTzc5xtVRBZGnQ81eS1Nwa54zmTLUzyjeYm38c/5C8MDKx8aUW5dEpxsllAmYnMkc?= =?utf-8?q?NytFBaxmRPfow+H8vAdEOSBmQCq/tsdiz/NGd/ta5fSWYtziO75i4RE61pzDdtFC+?= =?utf-8?q?m2JotYcxWFIt28i9d6VPZywPuvsUaO2o7/aLkuECUGq5Nh6VXXHyjBWIMFkpE/P2C?= =?utf-8?q?k5/Lq/+P7FSzdrqOg5qrLVg2JEhj8GAchfw1GzE1fZtjmNF/Fqr3YX89JwW3VBqxj?= =?utf-8?q?JMZFMVeL2luG3sqGfHKQVZmjXjVBGBsBS2RKm3JUgthRG1LdejDbT++XQx0ZBGFdL?= =?utf-8?q?PUpnEFjZxdXHSZmOOd/U0xCkCbS/TT4XYqbJMrpoBEyyUWro+UKbiqw+6WmAWuaEe?= =?utf-8?q?VSaU/2B1mekJICWuxHXmBVzHKRGKGpGYlrx8Xtaz29PqnXOZPeQS99b0T9LbAXtDY?= =?utf-8?q?aGxjNTB2mbBiWnslPtjA2O/CG37+avV0zGZUSpraKDiYy/fJg4mvFVUTadQmnm5Dx?= =?utf-8?q?/S9aQzmWboguM/1FPGS87g8HdAjB8YpYGzdDfUiwPQz805Y8EQ+sBggpHe/d+iiOc?= =?utf-8?q?XVxWALiT9jwHdpX4b2gBd9cqzPbESLSaZBRv+bG9rT+qb/+9K3Y//2wWGZ0LPMW2O?= =?utf-8?q?taQEZN8XT5fOztp0RdDfIDiOTQvfNSv71+KqzZY3YzEP25dSKcMVwH7Q0GK8/5vic?= =?utf-8?q?J7mnKglyzTxIY05APyS/7Z7TJMwfx0hO13IUbzl105zKUxbhLGorEZZUuECTmbIXq?= =?utf-8?q?/g/ReD1erEyveL/SShwODBNwa3C/sVr59v99xBXbl6MF6SCUkgqS0ItfyzYZpmvrT?= =?utf-8?q?qspHIMKtxGz1WbxLYAhUAv67tPIVUDxyRsErJTrF9F1Xgw0ZPTrqa/bvL6QNexMad?= =?utf-8?q?pZHsh+d0k3dVyft/TknszKSWdRhhjMTFNkr1MyioicwlHnM2ipQeyRyYkc2O8j4wb?= =?utf-8?q?vF7sUdJEuHw?= X-Forefront-Antispam-Report: CIP:216.228.117.161;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge2.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(7416014)(1800799024)(376014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Jul 2024 12:02:20.4247 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: defc8862-54ac-4982-7bc4-08dc9f45d21a X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.161];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL6PEPF0001AB73.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6586 Based on the filled flag, create VQs that are filled or blank. Blank VQs will be filled in later through VQ modify. Later patches will make use of this to pre-create blank VQs at vdpa device creation. Reviewed-by: Cosmin Ratiu Acked-by: Eugenio Pérez Signed-off-by: Dragos Tatulea --- drivers/vdpa/mlx5/net/mlx5_vnet.c | 85 +++++++++++++++++++++++++-------------- 1 file changed, 55 insertions(+), 30 deletions(-) diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c index a8ac542f30f7..0a62ce0b4af8 100644 --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c @@ -158,7 +158,7 @@ static bool is_index_valid(struct mlx5_vdpa_dev *mvdev, u16 idx) static void free_fixed_resources(struct mlx5_vdpa_net *ndev); static void mvqs_set_defaults(struct mlx5_vdpa_net *ndev); -static int setup_vq_resources(struct mlx5_vdpa_net *ndev); +static int setup_vq_resources(struct mlx5_vdpa_net *ndev, bool filled); static void teardown_vq_resources(struct mlx5_vdpa_net *ndev); static bool mlx5_vdpa_debug; @@ -874,13 +874,16 @@ static bool msix_mode_supported(struct mlx5_vdpa_dev *mvdev) pci_msix_can_alloc_dyn(mvdev->mdev->pdev); } -static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq) +static int create_virtqueue(struct mlx5_vdpa_net *ndev, + struct mlx5_vdpa_virtqueue *mvq, + bool filled) { int inlen = MLX5_ST_SZ_BYTES(create_virtio_net_q_in); u32 out[MLX5_ST_SZ_DW(create_virtio_net_q_out)] = {}; struct mlx5_vdpa_dev *mvdev = &ndev->mvdev; struct mlx5_vdpa_mr *vq_mr; struct mlx5_vdpa_mr *vq_desc_mr; + u64 features = filled ? mvdev->actual_features : mvdev->mlx_features; void *obj_context; u16 mlx_features; void *cmd_hdr; @@ -898,7 +901,7 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque goto err_alloc; } - mlx_features = get_features(ndev->mvdev.actual_features); + mlx_features = get_features(features); cmd_hdr = MLX5_ADDR_OF(create_virtio_net_q_in, in, general_obj_in_cmd_hdr); MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, opcode, MLX5_CMD_OP_CREATE_GENERAL_OBJECT); @@ -906,8 +909,6 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque MLX5_SET(general_obj_in_cmd_hdr, cmd_hdr, uid, ndev->mvdev.res.uid); obj_context = MLX5_ADDR_OF(create_virtio_net_q_in, in, obj_context); - MLX5_SET(virtio_net_q_object, obj_context, hw_available_index, mvq->avail_idx); - MLX5_SET(virtio_net_q_object, obj_context, hw_used_index, mvq->used_idx); MLX5_SET(virtio_net_q_object, obj_context, queue_feature_bit_mask_12_3, mlx_features >> 3); MLX5_SET(virtio_net_q_object, obj_context, queue_feature_bit_mask_2_0, @@ -929,17 +930,36 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque MLX5_SET(virtio_q, vq_ctx, queue_index, mvq->index); MLX5_SET(virtio_q, vq_ctx, queue_size, mvq->num_ent); MLX5_SET(virtio_q, vq_ctx, virtio_version_1_0, - !!(ndev->mvdev.actual_features & BIT_ULL(VIRTIO_F_VERSION_1))); - MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr); - MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr); - MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); - vq_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]]; - if (vq_mr) - MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, vq_mr->mkey); - - vq_desc_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_DESC_GROUP]]; - if (vq_desc_mr && MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, desc_group_mkey_supported)) - MLX5_SET(virtio_q, vq_ctx, desc_group_mkey, vq_desc_mr->mkey); + !!(features & BIT_ULL(VIRTIO_F_VERSION_1))); + + if (filled) { + MLX5_SET(virtio_net_q_object, obj_context, hw_available_index, mvq->avail_idx); + MLX5_SET(virtio_net_q_object, obj_context, hw_used_index, mvq->used_idx); + + MLX5_SET64(virtio_q, vq_ctx, desc_addr, mvq->desc_addr); + MLX5_SET64(virtio_q, vq_ctx, used_addr, mvq->device_addr); + MLX5_SET64(virtio_q, vq_ctx, available_addr, mvq->driver_addr); + + vq_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]]; + if (vq_mr) + MLX5_SET(virtio_q, vq_ctx, virtio_q_mkey, vq_mr->mkey); + + vq_desc_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_DESC_GROUP]]; + if (vq_desc_mr && + MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, desc_group_mkey_supported)) + MLX5_SET(virtio_q, vq_ctx, desc_group_mkey, vq_desc_mr->mkey); + } else { + /* If there is no mr update, make sure that the existing ones are set + * modify to ready. + */ + vq_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_GROUP]]; + if (vq_mr) + mvq->modified_fields |= MLX5_VIRTQ_MODIFY_MASK_VIRTIO_Q_MKEY; + + vq_desc_mr = mvdev->mr[mvdev->group2asid[MLX5_VDPA_DATAVQ_DESC_GROUP]]; + if (vq_desc_mr) + mvq->modified_fields |= MLX5_VIRTQ_MODIFY_MASK_DESC_GROUP_MKEY; + } MLX5_SET(virtio_q, vq_ctx, umem_1_id, mvq->umem1.id); MLX5_SET(virtio_q, vq_ctx, umem_1_size, mvq->umem1.size); @@ -959,12 +979,15 @@ static int create_virtqueue(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtque kfree(in); mvq->virtq_id = MLX5_GET(general_obj_out_cmd_hdr, out, obj_id); - mlx5_vdpa_get_mr(mvdev, vq_mr); - mvq->vq_mr = vq_mr; + if (filled) { + mlx5_vdpa_get_mr(mvdev, vq_mr); + mvq->vq_mr = vq_mr; - if (vq_desc_mr && MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, desc_group_mkey_supported)) { - mlx5_vdpa_get_mr(mvdev, vq_desc_mr); - mvq->desc_mr = vq_desc_mr; + if (vq_desc_mr && + MLX5_CAP_DEV_VDPA_EMULATION(mvdev->mdev, desc_group_mkey_supported)) { + mlx5_vdpa_get_mr(mvdev, vq_desc_mr); + mvq->desc_mr = vq_desc_mr; + } } return 0; @@ -1442,7 +1465,9 @@ static void dealloc_vector(struct mlx5_vdpa_net *ndev, } } -static int setup_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq) +static int setup_vq(struct mlx5_vdpa_net *ndev, + struct mlx5_vdpa_virtqueue *mvq, + bool filled) { u16 idx = mvq->index; int err; @@ -1471,7 +1496,7 @@ static int setup_vq(struct mlx5_vdpa_net *ndev, struct mlx5_vdpa_virtqueue *mvq) goto err_connect; alloc_vector(ndev, mvq); - err = create_virtqueue(ndev, mvq); + err = create_virtqueue(ndev, mvq, filled); if (err) goto err_vq; @@ -2062,7 +2087,7 @@ static int change_num_qps(struct mlx5_vdpa_dev *mvdev, int newqps) } else { ndev->cur_num_vqs = 2 * newqps; for (i = cur_qps * 2; i < 2 * newqps; i++) { - err = setup_vq(ndev, &ndev->vqs[i]); + err = setup_vq(ndev, &ndev->vqs[i], true); if (err) goto clean_added; } @@ -2558,14 +2583,14 @@ static int verify_driver_features(struct mlx5_vdpa_dev *mvdev, u64 features) return 0; } -static int setup_virtqueues(struct mlx5_vdpa_dev *mvdev) +static int setup_virtqueues(struct mlx5_vdpa_dev *mvdev, bool filled) { struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev); int err; int i; for (i = 0; i < mvdev->max_vqs; i++) { - err = setup_vq(ndev, &ndev->vqs[i]); + err = setup_vq(ndev, &ndev->vqs[i], filled); if (err) goto err_vq; } @@ -2877,7 +2902,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, if (teardown) { restore_channels_info(ndev); - err = setup_vq_resources(ndev); + err = setup_vq_resources(ndev, true); if (err) return err; } @@ -2888,7 +2913,7 @@ static int mlx5_vdpa_change_map(struct mlx5_vdpa_dev *mvdev, } /* reslock must be held for this function */ -static int setup_vq_resources(struct mlx5_vdpa_net *ndev) +static int setup_vq_resources(struct mlx5_vdpa_net *ndev, bool filled) { struct mlx5_vdpa_dev *mvdev = &ndev->mvdev; int err; @@ -2906,7 +2931,7 @@ static int setup_vq_resources(struct mlx5_vdpa_net *ndev) if (err) goto err_setup; - err = setup_virtqueues(mvdev); + err = setup_virtqueues(mvdev, filled); if (err) { mlx5_vdpa_warn(mvdev, "setup_virtqueues\n"); goto err_setup; @@ -3000,7 +3025,7 @@ static void mlx5_vdpa_set_status(struct vdpa_device *vdev, u8 status) goto err_setup; } register_link_notifier(ndev); - err = setup_vq_resources(ndev); + err = setup_vq_resources(ndev, true); if (err) { mlx5_vdpa_warn(mvdev, "failed to setup driver\n"); goto err_driver;