From patchwork Fri Mar 24 18:17:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 13187227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78D89C77B71 for ; Fri, 24 Mar 2023 18:18:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231872AbjCXSST (ORCPT ); Fri, 24 Mar 2023 14:18:19 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230350AbjCXSSO (ORCPT ); Fri, 24 Mar 2023 14:18:14 -0400 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B3AECC10; Fri, 24 Mar 2023 11:18:11 -0700 (PDT) Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 32OI0SD2022105; Fri, 24 Mar 2023 18:17:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : subject : date : message-id : mime-version : content-transfer-encoding; s=corp-2022-7-12; bh=1v7ZLeL97hWYChdBpR/rAN0z9r+BoNViKNQTonIgoCs=; b=dklQLpv5Kih9JVdYi1iO2ImypuDTYkW4F+sme8XZYHtjXMI0JGJwkdeW4lDW2IeRBmfH 3I2ntPpGeIzk11tOaMcV1bNoGPAbQ0hTjXtjXXclvUGOKdyOxoRjcG7LD65Rvpp5xwrd yaHgfOOpDxdRe3+ncYvEX4jdkqh4RvqDfJNWKMFaUwqxzOhfDkduEHS7+/ZtYlugQZsi E+xMsDiXR7w7ek9bDVLPOyEFuc1jfmuafwBkXAjxBCfYxpU7/3Ad01Gq+rExVNAJz5Lf cCgB+txoCNtMcPWNsFOVrACgPIJRV31gci7NttV3wqYP5XYkTkPoRXqrhHm3Nvujdvkn OA== Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.appoci.oracle.com [138.1.37.129]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3phgrg81bc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Mar 2023 18:17:46 +0000 Received: from pps.filterd (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 32OIAArs001370; Fri, 24 Mar 2023 18:17:45 GMT Received: from pps.reinject (localhost [127.0.0.1]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3pgxk4bx2j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 24 Mar 2023 18:17:45 +0000 Received: from phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 32OIHiEJ021017; Fri, 24 Mar 2023 18:17:45 GMT Received: from mnchrist-mac.us.oracle.com (dhcp-10-154-153-54.vpn.oracle.com [10.154.153.54]) by phxpaimrmta03.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTP id 3pgxk4bx19-1; Fri, 24 Mar 2023 18:17:44 +0000 From: Mike Christie To: bvanassche@acm.org, hch@lst.de, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, james.bottomley@hansenpartnership.com, linux-block@vger.kernel.org, dm-devel@redhat.com, snitzer@kernel.org, axboe@kernel.dk, linux-nvme@lists.infradead.org, chaitanyak@nvidia.com, kbusch@kernel.org, target-devel@vger.kernel.org Subject: [PATCH v5 00/18] Use block pr_ops in LIO Date: Fri, 24 Mar 2023 13:17:23 -0500 Message-Id: <20230324181741.13908-1-michael.christie@oracle.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22 definitions=2023-03-24_11,2023-03-24_01,2023-02-09_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 bulkscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2303200000 definitions=main-2303240143 X-Proofpoint-ORIG-GUID: 42_53J3ULHbeAIb4lt_dUZZeDskoyq4B X-Proofpoint-GUID: 42_53J3ULHbeAIb4lt_dUZZeDskoyq4B Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The patches in this thread allow us to use the block pr_ops with LIO's target_core_iblock module to support cluster applications in VMs. They were built over Linus's tree. They also apply over linux-next and Martin's tree and Jens's trees. Currently, to use windows clustering or linux clustering (pacemaker + cluster labs scsi fence agents) in VMs with LIO and vhost-scsi, you have to use tcmu or pscsi or use a cluster aware FS/framework for the LIO pr file. Setting up a cluster FS/framework is pain and waste when your real backend device is already a distributed device, and pscsi and tcmu are nice for specific use cases, but iblock gives you the best performance and allows you to use stacked devices like dm-multipath. So these patches allow iblock to work like pscsi/tcmu where they can pass a PR command to the backend module. And then iblock will use the pr_ops to pass the PR command to the real devices similar to what we do for unmap today. The patches are separated in the following groups: Patch 1 - 2: - Add block layer callouts for reading reservations and rename reservation error code. Patch 3 - 5: - SCSI support for new callouts. Patch 6: - DM support for new callouts. Patch 7 - 13: - NVMe support for new callouts. Patch 14 - 18: - LIO support for new callouts. This patchset has been tested with the libiscsi PGR ops and with window's failover cluster verification test. Note that for scsi backend devices we need this patchset: https://lore.kernel.org/linux-scsi/20230123221046.125483-1-michael.christie@oracle.com/T/#m4834a643ffb5bac2529d65d40906d3cfbdd9b1b7 to handle UAs. To reduce the size of this patchset that's being done separately to make reviewing easier. And to make merging easier this patchset and the one above do not have any conflicts so can be merged in different trees. v5: - Use []/struct_size with nvme reservation structs - Add Keith's copywrite to pr.c - Drop else in nvme_send_pr_command - Fix PR_EXCLUSIVE_ACCESS_ALL_REGS use in block_pr_type_from_nvme v4: - Pass read_keys number of keys instead of array len - Keep the switch use when converting between block and scsi/nvme PR types. Drop default case so compiler spits out warning if in the future a new value is added. - Add helper for handling nvme_send_ns_head_pr_command/nvme_send_ns_pr_command - Use void * instead of u8* for passing data buffer. - Rename status variable to rs. - Have caller init buffer/structs instead of nvme/scsi callouts. - Drop blk_status to err code. v3: - Fix patch subject formatting. - Fix coding style. - Rearrange patches so helpers are added with users to avoid compilation errors. - Move pr type conversion to array and add nvme_pr_type. - Add Extended Data Structure control flag enum and use in code for checks. - Move nvme pr code to new file. - Add more info to patch subjects about why we need to add blk_status to pr_ops. - Use generic SCSI passthrough error handling interface. - Fix checkpatch --strict errors. Note that I kept the existing coding style that it complained about because it looked like it was the preferred style for the code and I didn't want a mix and match. v2: - Drop BLK_STS_NEXUS rename changes. Will do separately. - Add NVMe support. - Fixed bug in target_core_iblock where a variable was not initialized mentioned by Christoph. - Fixed sd pr_ops UA handling issue found when running libiscsi PGR tests. - Added patches to allow pr_ops to pass up a BLK_STS so we could return a RESERVATION_CONFLICT status when a pr_ops callout fails.