From patchwork Tue Feb 9 12:38:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077939 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 516D8C433E6 for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1ACF164EB1 for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230049AbhBIMjr (ORCPT ); Tue, 9 Feb 2021 07:39:47 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:43160 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229601AbhBIMjn (ORCPT ); Tue, 9 Feb 2021 07:39:43 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CY9UQ054636; Tue, 9 Feb 2021 12:38:58 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=mwRx5RJwoINrrW6DHSPguPEI22CgZPzUP20hTWdabpQ=; b=PgriokHMJs7Z2Gn02e2ei6tyygT32HKG6DqR3v4dlvgC5otzBAdMgygxuZlbw/r5Fr1B Cza9AJHCRxZseObKNSDcpovMYLWbQnWzgfRVaGSszWYQ1X5KRmWDjlV4cOuhdPMn4R5h Y1AJ20DptsMiLA8ow98HQXoC3da4uWAXnONkAtKhFLQMQIwF64z2N/D+eq6B2NgxFyfN mlwas6BlS87MJgopkOqv4FdyYSercppRTaqwp40Exl84bqADMJBQN3Ad5iAgVaL6olMk NNetnIsQn2yfdgdYBYusGH6rTpFcI4GydL+xHJTjNoxHPeDbJp6qDKq94CyT1kIaIA1D Tg== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 36hgmafe4g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:58 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIE178533; Tue, 9 Feb 2021 12:38:57 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by aserp3030.oracle.com with ESMTP id 36j4pnn07j-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:57 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=EIhJm6TcQr/Up+iG7LVPOdHA7+sdBpZC2VNsFipbUSCjpdvFlrG25WFqCH+Afm3qKDQ1K05JFa2INPkfQpPDQKIAO2JcrEPvJeCLmu5LaHDRJscIKTdGe423+ptX5iZ1AOZuHZjAM0kDegasMM+L4Y32WgIB1L9sUO3OfhYO3SWo61E3IfOb29xsGvcmgyr9CwUhu6ASVOp+1ygmyb7xe2r8hn7enzJmJmr30oimEZSNUZzfhETg9axWY9Y1WoCY+sFqOAI7lwiCvK6djl/k5hVkL7RitARZJcV9VmWUWyMjUCVo3+BtCroJB4DJgZJ9meZm+XaNqgkJbnWoszW+Yw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mwRx5RJwoINrrW6DHSPguPEI22CgZPzUP20hTWdabpQ=; b=efLZUa32lTm6py/VkpIQMZx9mTrADamGqH8GIisTluZX2cehpHNAlSthp3xcn1USPXkcGro6V/gIGIxqHFQwBlauum+HzvxWjj028pKYIXWdZ4BgECnO26kux4b3ptkE7RyThMfw28bYCjhIsOgkoI41e+lcsBiEslMOOccJHFz1Fy+SSNWuPAQNWBh12iGNpBPeHGUnBbftvMJXnCUNGw3cKPrXByY6TMGPaDowjtZKf2k3tlWAIXzaFqvIK0EodpkBt7Gx/TXXEvLI70lXcXC3X3Nbr/JTMFnjTySLp8WbrtuifNwKIPHfh3wyDoDj4MHhgQRB31dxIjxT4KY4Vg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mwRx5RJwoINrrW6DHSPguPEI22CgZPzUP20hTWdabpQ=; b=hFEGuGPdHAg1VZHRYx36nfuQCSECLQP4NEwssW40HHK6hGusXmA7qLtG2gp2UvslrlKoxVDt2Bhuqhn+7WJRaVEUyWigZ8hd8ixNT40DXEPFxBzfeY3flCO8EUdFIM8DV0R/z7gw4GraUcWc1IN3HQXmsBMSx6w+l13mCCPwGoQ= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:38:55 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:38:55 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 01/13] target: move t_task_cdb initialization Date: Tue, 9 Feb 2021 06:38:33 -0600 Message-Id: <20210209123845.4856-2-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:38:54 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b85d68d4-4da8-4a37-b692-08d8ccf7a9f9 X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: cJAA6QqmfGm7ijREpTNGxwbyfY9ixfRTkVQJP+jASuqljPZftKuziLL74j87d+BXBQKQRi7KjpDINuNLn/UVagpqkYvN3we/rV3tlT+A1kn+fTbbWzbQ9aaKI3dHjUWcH+cnB+sYY7i1vBXSJhzq5jKrqwU1HD/TKcT9VvOa6IVA3u/koDUGar2ol3F1AXLIPdau4Y5VZNSJn59ZinySsjziNBGuTZXVQhTxk/ePkQ3N11Ha0gd9DCsbirBmOxBYC8Q42LwMadfsdTCcvU0US4vL/W+PmjkIRChsJkqEvr8ZYFhiE/yHuoPV4HZkScot2qkHVrRwt0fgRdN2d9spArogwc9U4xF19AAJy+q58N0414xaB6op5sKFyqxxqSmGtyT9z5C+6EVJis0uKqtsnDnBdPRRPUEAkiuDaL/qR+Fy+qAOld/UuJqft/qO37/521OREuznl8EX4IYXpHP0sI/oBY1lybPeAcV9zGlkmu87n0W5VzHoWL0Dk06jWYNfMPiN77dN2gjFkdqq/ddViLXYdAW42TV2/sfFtsizpPfeTiBqoYDhK1efyi7KwkOjrLF4NElFfN9CuCtBJ/wQYw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: MX9hvd+rQ7dNSlzjmxi+MCRkTRDfIu13JluDs3hNtBEnZY4A+0hsCzWclYRjfmS2NApaxTSoeiQlXc4xuKmwqpTs5rA6bAu3Ek2ReC1gGCKwQGQ4UYjlfptrOD6zYeu+si31UCpvnhMvAYg4qJm8OLccr+NvcSg1f8xXZU6Qeu2m6B32j2W2lXKcxSaufq7M2LZxXtH4ytNSCeN3wqgcToTfRRKSzGCb5ZUSdcOI8rFZJu8u9umuODIp/aDrOgMA90n69lX1qMMt36tnlKR5mot68tZk2ksiD5ajUuOomQNxHdncVdjrIW0cZO5ZC1LV8IffObNzr+m4WhHxMaPDnoii+usWET5kNIKjGZB5uvhCNfjVhv5MXYYXEm48NyI0SYopGxjelI2g95oHqdi4auFAC9v5H4Vt+VVW//0DDML1K9xL7WHVsKLjVKND7wVFySaKz5dUSrO1CI2yamY/nszoIiNGT6k7+7OEZBnXyGTccljhMRfbp3PXUkCwxQ34fqP37vl2Hz57Le+o70b7ql2rfKkOFxMPWIEjlkgKDwnCUFG/ui3asyJGNGEipGdumos7Cd9p7A2JEirLV22LfwSdWi2sEy3cN1MvV9ntz8vdXKp7fnZgTPWsYyLrq56jPGgv8nWciDhiAt/ywWEhY9Ih1ilzlKzzPHjzPG9q7wnF8Watu+81kUUn1EmMPwj/pnok6ERoFrz414k9tfvuusZ1i0m+iqpH+OPMtbVukygO56zwvN4kNhCfSgLrVdwqcQ3ZbNvHfqwcJfmgq9Sa6BIXcQECAGM69/SxrvZPgFLLEHHRfFeBQEQyhNXqDFnG7vtbqW92WvKV+Pk9CFqByQMbNSI8F8fa2tEIo8oPpTTWr2mV7253CI1pXhfw3aeh0v5QAGQ3XCiSwzb6okztB5OwIlbFn1ozydcZvLoWKN9EFRugrvgjKx6kt5U8NZCbrgFkO2uPKOrIROQd7lI99R4xzE71GsIKRI7yuMMQCLoEmjXo3PRxTl7JiAhxL/6LnsZwAvUWwfVV3g0G9lKyNCnIY8g9LH3NhDr/Z/WMVspAF4cLoxW4XrboffOjdl4mp/SCHnYxaAdBrSxgD+KYsWuH3uuIcg3JzQHxc655RExZlOuVLZ5nNxq+/ifiCKaln6oxWCQDoQgXDafQPUPpKmouV198kVuK/wc7ikCsTXoNI59CVJ8TMeQ1hZTRFfPo8wiSC3iJBMYwbZw2+o4CduBxDa/bu9HdRdbqer8WgUv8GAME2zz9iLFI+EPGX1pD1ZUKhZjMd4+uuFDFYfDjCqVfkbxtHDP+f9lgyhea7OQ/ybI1ltFvnsCJ7Vow+/kc X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b85d68d4-4da8-4a37-b692-08d8ccf7a9f9 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:38:55.5827 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ufdE7tOsuX8Nh0rM0YJ1YwnCr5shjDSJmbaMPJwmMDQKc19fjdQn1JvX/nSoJw2A+tnrDpXzqRvdO9VC/cBWyLKzIJBmlesgMiNXL6dzDus= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 suspectscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org The next patch splits target_submit_cmd_map_sgls so the initialization and submission part can be called at different times. If the init part fails we can reference the t_task_cdb early in some of the logging and tracing code. This moves it to transport_init_se_cmd so we don't hit NULL pointer crashes. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig --- drivers/target/target_core_transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index d47bfd8b0f87..5c4adde96d5e 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -1389,6 +1389,7 @@ void transport_init_se_cmd( INIT_WORK(&cmd->work, NULL); kref_init(&cmd->cmd_kref); + cmd->t_task_cdb = &cmd->__t_task_cdb[0]; cmd->se_tfo = tfo; cmd->se_sess = se_sess; cmd->data_length = data_length; @@ -1430,7 +1431,6 @@ target_cmd_init_cdb(struct se_cmd *cmd, unsigned char *cdb) { sense_reason_t ret; - cmd->t_task_cdb = &cmd->__t_task_cdb[0]; /* * Ensure that the received CDB is less than the max (252 + 8) bytes * for VARIABLE_LENGTH_CMD From patchwork Tue Feb 9 12:38:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077941 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83D69C433E9 for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C90A64E50 for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230209AbhBIMj5 (ORCPT ); Tue, 9 Feb 2021 07:39:57 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:39094 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230182AbhBIMjs (ORCPT ); Tue, 9 Feb 2021 07:39:48 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZFlq048534; Tue, 9 Feb 2021 12:38:59 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=fOS1NCKuge0pO5LPozjq1i+tEYEZOZqyqx9RH3H2gd8=; b=PXOPIBOeNavBZvW/ZrrfpaxFbCrXXIKAF5D/A+IdYMpd71w5qC8XXoA1eBXFHypNODDZ M2KPhCspgK//wmKvxV2TrYZgABdr/wfTvUWIeQdt622bzZqkXNtHvsgj2T95jgtm1FJA R9LGqCwVT8Qw/SV5spfI57cXqwjmcpNCRUW6bTPzRafcurtKbIMezbIAhgJN7Lf52gel PIJ76JUTcWa4gYGP+VuOJpdOyiIeU/BhZtU7G8modgArE2JIcIgIEFsZQLTXGNV+AkIL eN9lc2mtvY9RwIvnKk6fAsADrbsxAAjw/CS7k5z2pQtD1eks6jg6YY8Wrq3wyz+U13o1 xA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 36hkrmy8c6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:59 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIG178533; Tue, 9 Feb 2021 12:38:58 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by aserp3030.oracle.com with ESMTP id 36j4pnn07j-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:58 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YKIzj5wXtP2yB/jqtIBkaV1nE8EujUtjlX7ZKUIb9zag6mRCXx1m4bs57KhvxQxkUxzczT4HmwRsc84TcqZe6+2EgdeXzQ4p7ajgVuAkqT9jGX0O0AbJbC8EwLEmmhkdWTvujiUTT68Plf+UxZNqxLs+9zHnR3XbXqKpg0W0+KyFzcAVjUhWYuANzcwgsug6Y8Qb+LdoK31ESLZbnJGtQ4qQ3ZKZ5c+PZ/HuoqDvjaDGBxq7MVxScwVu02pvKJjpOpn7Dnl2gy2BQ4pu46nvDJYd77wsJdR6zW+IMqZQO9ZZ2wF+uYSCbp3rXYAqDDs4OrWr2gaKfSwuDkFSqIxnZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fOS1NCKuge0pO5LPozjq1i+tEYEZOZqyqx9RH3H2gd8=; b=J6KcrokKWdbuCGssylZC7Y+X6ClW285GtVMBMTdkqg8QnnPhc3hLCVEe/qyI6uExCzqIOWtMVXwyBrctfViIux+VcQ8LC1uW2nRZZQYrDpYywaMK5M8il++BWno3GE/wQ8n3MIlpfwkI/eMXmSlGIVDL/+wpfwekNx2dbQ6cMUAdtak8gByknjCJafejETJLG1HcHs5YcxGAF7PFNKGGE4pMBAdEbxvL+6kKO0yJDu4UfN7AosVhuPib59uK7vUmDsyavSzl9ZxkqdztGjWBPPdpm2dGkeAjijorjT5VEiuTp/+093ffWgnn+ppdRFSndBfPg6/N/4rqXi+cUWd/lg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fOS1NCKuge0pO5LPozjq1i+tEYEZOZqyqx9RH3H2gd8=; b=qqadSXhr+3WvSql50WJVJP7TZlTAGSPPOVofsOavX/1yPJP6ZGQ6zNXMIX7f9M7g9D8OFQcPL5WuJfrEkga1ItK9S29YKNxyzK7q3oK2aO8ugHKh+GBIaIw4T9UCGORROfSvuhQM9P0hmZ8cOr03im5udd7N/dXJ032X8ZSOm+U= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:38:56 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:38:56 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 02/13] target: split target_submit_cmd_map_sgls Date: Tue, 9 Feb 2021 06:38:34 -0600 Message-Id: <20210209123845.4856-3-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:38:55 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ecf90914-b6ee-4c2c-5368-08d8ccf7aab1 X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PGThId7bYh/Ftbno3bd1tYfs7Z/twD3vpxiFbvukSxuru1Mp3o/JPledBmc4dsUgC6CJwLO5exdZn52RQyYm3eoPT1W8gN47KqUdkvQzsVvd6z3NgnV/j14EUWeEasdLUNgwGYyDV2J7BPkibtxy5yeDZYNTHRdKPkFbvHDJqNIVXC4S66+vNAX+B4JZCJdbGlSxOJR/Atnks2Y18y3qMl/ecjmEpzUeZXaX5ZskSsJHVU53YW8oyZVSwyuDCH7vx9sEBKwyiYoi/+RuaiXYaBXMAuQ865KJSSC+AiPD5QpvXFNdzdp3Lo37D8UboDInh0xgEFE6RiyeytaIcEA22uYbbTp0+Twtg8vZHEOonVycWlQxCC1L5atVnIL8W0H4Wph5J/UITT4RU9+UDWhxSUeOFCFhYiYteV96i7wLuGc1oW4PWkiLp3iPhWas3tSRkBJMh9TuVAiMoozqdDHjrWPMImBVHITQ5TfQmVyXXHaFmjoa/xafzMPNSyxhKlcTs2W1L9GQazypiwkoP1rSdJhtW3AOFdFB6A1scNQxjr0hqFY48n/QpdQzZ0bLGdHTcj7v61QaSYqJ8KRbotSbkA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 2HKg306y7PuGrg7cRyZ/05F8gideUcqAWJpioj/1/kGi7PMtYRRKhIwSlvAjRwzP7nk1u6chURnZJkXjJfAfDt46Lw8RoxdUr14BLyMBoqOsNLWd0rIXvy0neg/x5LRa7z+w7dz899LcpCjo0TkVESUDEltGES7gwOsRqPVNmRI7+7COdNAwsftd4C7EyIzYSqfvOHSBRhO57ybfm8XV7l0rXa0u/Uw6dBuWX9YpVu82vLqHD5xixEl5/X5PhrIVXGEoobMYWV4GZmorlFy0KORw1nVIDLs7QlTl2oe0ehzGFDde4Pw9vVH7ij8EpMPF1mbpilOE3SERyk1Yiqgch8u3k5OgN20yUdZyWadH8FIm2qcPGi2NV0Gp3I155k8Lh2bT4T7n2CnApK3yYI3OCEw4xlM+iRbnAYc97Ep207RhZPRJAsUwV4yNCz4c8buu2AiNIKJFzpULdZd2bz+nXtyd0FighsDNzIh0uojOBBzqbx6lf/LMCOi/oPLEMZHG1eHMudA9MVD+PjSKk622I3nrsWgddgJ1y3X9i5BDMp0bdrY9E+Qv0hG2Har4WBU4lEiBKhoQ/5GU5mUzoVKRPdMPJdgDfcyj67zcl5RlXgt4BQP3wU88xTckhArynRThy434BcoSmORdal3cZKOOsimJZLPzgTaYSsD9rXovYH7BWo6OX2+7XOjlcJwx38IyRd+0mEBU5TIp7JTY3MUKcywi39BpCPAvkhjQLoYOpotDnY0iKaKMXzBPjxbH1mKd0RKp8Xmym4RTto6KZWEXhr16oAQsclwLkfrK6XpzC6DU+Jo2+ZYF1GYLuV+er1aIzkmBGBUD/j5sm6WpL41UUUrNKtTJmqLa+p+zv9GOGCjMktAgTvGkRSqt2XmiRUEIe7rujoVfCuBeSzTf3u2o56KIn+WJdczpjS1RDYwKa0ODOOl6ybs3qvWA8BVq9Ah2xhn75GUgJR4yOhAS6qVde2fAgdvh1RS9Up9s7LeiQDKDPmtoVmuceARPXNEOTwIOhgPZpArRaLhQPZCC0acw0A+6BoWE0o0mS5bK+D9lSKWtQx77JfFl9LptO6TlN5y1/XyUDrsmcZpRXG85bU1oia79R1sEcVKFYG3q5TttP1ktFqfy/jyWYfyNmlmgagTn3SJyumKiY1YYKDFYQkGp9VzAQpUOUfTaZ9Ac+VkDP5//5cXqAEt6SDFDAuDBQCuPKenUvxEa+FWp0cj5XXqvkiYj5+j9bdN1X7ACPP2W6lcUyMQkThLiOcIWSQda9hNIai8PukFSb1ym+huSw88A+mNAOsz/RGNYIgI/2NcEhQAzqw2DgVViLm5YZgM1jIxj X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ecf90914-b6ee-4c2c-5368-08d8ccf7aab1 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:38:56.7310 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: sxht9DPTNxS0XVr+1l5ZdxsaDtgVhnhcdvNlBVfZW7llEg/9Iz5rn5YzKojSxbtaENNpQcwDvS4ndNct0kE3Tb+HhSdvsE8hjN7tHJ0m1QE= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Separate target_submit_cmd_map_sgls into the part that does: - the initial cmd setup - will not sleep - and gives us access to the se_device and the part that: - can sleep - handles the actual submission This will be needed for loop in the next patches which needs to add the cmd to the lio workqueue and can't sleep in that initial submission path. Signed-off-by: Mike Christie Reported-by: kernel test robot --- drivers/target/target_core_transport.c | 149 ++++++++++++++++++------- 1 file changed, 109 insertions(+), 40 deletions(-) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 5c4adde96d5e..71b0a862608b 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -1571,12 +1571,9 @@ transport_generic_map_mem_to_cmd(struct se_cmd *cmd, struct scatterlist *sgl, } /** - * target_submit_cmd_map_sgls - lookup unpacked lun and submit uninitialized - * se_cmd + use pre-allocated SGL memory. - * + * target_submit_prep - prep cmd for submission to lio core * @se_cmd: command descriptor to submit * @se_sess: associated se_sess for endpoint - * @cdb: pointer to SCSI CDB * @sense: pointer to SCSI sense buffer * @unpacked_lun: unpacked LUN to reference for struct se_lun * @data_length: fabric expected data transfer length @@ -1592,26 +1589,29 @@ transport_generic_map_mem_to_cmd(struct se_cmd *cmd, struct scatterlist *sgl, * * Task tags are supported if the caller has set @se_cmd->tag. * - * Returns non zero to signal active I/O shutdown failure. All other - * setup exceptions will be returned as a SCSI CHECK_CONDITION response, - * but still return zero here. + * Returns: + * - less than zero to signal active I/O shutdown failure + * - zero on success. + * - one for all other setup exceptions. The cmd will be returned as a + * SCSI CHECK_CONDITION response in this case. * - * This may only be called from process context, and also currently - * assumes internal allocation of fabric payload buffer by target-core. + * This may only be called from interrupt context if the caller's + * queue_status and release_cmd callouts do not block. + * + * This assumes internal allocation of fabric payload buffer by target-core. */ -int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess, - unsigned char *cdb, unsigned char *sense, u64 unpacked_lun, - u32 data_length, int task_attr, int data_dir, int flags, - struct scatterlist *sgl, u32 sgl_count, - struct scatterlist *sgl_bidi, u32 sgl_bidi_count, - struct scatterlist *sgl_prot, u32 sgl_prot_count) +static int +target_submit_prep(struct se_cmd *se_cmd, struct se_session *se_sess, + unsigned char *sense, u64 unpacked_lun, + u32 data_length, int task_attr, int data_dir, int flags, + struct scatterlist *sgl, u32 sgl_count, + struct scatterlist *sgl_bidi, u32 sgl_bidi_count, + struct scatterlist *sgl_prot, u32 sgl_prot_count) { struct se_portal_group *se_tpg; sense_reason_t rc; int ret; - might_sleep(); - se_tpg = se_sess->se_tpg; BUG_ON(!se_tpg); BUG_ON(se_cmd->se_tfo || se_cmd->se_sess); @@ -1642,14 +1642,6 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess */ if (flags & TARGET_SCF_BIDI_OP) se_cmd->se_cmd_flags |= SCF_BIDI; - - rc = target_cmd_init_cdb(se_cmd, cdb); - if (rc) { - transport_send_check_condition_and_sense(se_cmd, rc, 0); - target_put_sess_cmd(se_cmd); - return 0; - } - /* * Locate se_lun pointer and attach it to struct se_cmd */ @@ -1657,13 +1649,7 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess if (rc) { transport_send_check_condition_and_sense(se_cmd, rc, 0); target_put_sess_cmd(se_cmd); - return 0; - } - - rc = target_cmd_parse_cdb(se_cmd); - if (rc != 0) { - transport_generic_request_failure(se_cmd, rc); - return 0; + return 1; } /* @@ -1684,6 +1670,43 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess if (sgl_count != 0) { BUG_ON(!sgl); + rc = transport_generic_map_mem_to_cmd(se_cmd, sgl, sgl_count, + sgl_bidi, sgl_bidi_count); + if (rc != 0) { + transport_generic_request_failure(se_cmd, rc); + return 1; + } + } + + return 0; +} + +/** + * target_submit - perform final initialization and submit cmd to LIO core + * @se_cmd: command descriptor to submit + * @cdb: pointer to SCSI CDB + * + * target_submit_prep must have been called on the cmd, and this must be + * called from process context. + */ +static void target_submit(struct se_cmd *se_cmd, unsigned char *cdb) +{ + struct scatterlist *sgl = se_cmd->t_data_sg; + unsigned char *buf = NULL; + int rc; + + might_sleep(); + + rc = target_cmd_init_cdb(se_cmd, cdb); + if (rc) + goto fail; + + rc = target_cmd_parse_cdb(se_cmd); + if (rc != 0) + goto fail; + + if (se_cmd->t_data_nents != 0) { + BUG_ON(!sgl); /* * A work-around for tcm_loop as some userspace code via * scsi-generic do not memset their associated read buffers, @@ -1694,8 +1717,6 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess */ if (!(se_cmd->se_cmd_flags & SCF_SCSI_DATA_CDB) && se_cmd->data_direction == DMA_FROM_DEVICE) { - unsigned char *buf = NULL; - if (sgl) buf = kmap(sg_page(sgl)) + sgl->offset; @@ -1705,12 +1726,6 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess } } - rc = transport_generic_map_mem_to_cmd(se_cmd, sgl, sgl_count, - sgl_bidi, sgl_bidi_count); - if (rc != 0) { - transport_generic_request_failure(se_cmd, rc); - return 0; - } } /* @@ -1720,6 +1735,60 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess core_alua_check_nonop_delay(se_cmd); transport_handle_cdb_direct(se_cmd); + return; + +fail: + transport_generic_request_failure(se_cmd, rc); +} + +/** + * target_submit_cmd_map_sgls - lookup unpacked lun and submit uninitialized + * se_cmd + use pre-allocated SGL memory. + * + * @se_cmd: command descriptor to submit + * @se_sess: associated se_sess for endpoint + * @cdb: pointer to SCSI CDB + * @sense: pointer to SCSI sense buffer + * @unpacked_lun: unpacked LUN to reference for struct se_lun + * @data_length: fabric expected data transfer length + * @task_attr: SAM task attribute + * @data_dir: DMA data direction + * @flags: flags for command submission from target_sc_flags_tables + * @sgl: struct scatterlist memory for unidirectional mapping + * @sgl_count: scatterlist count for unidirectional mapping + * @sgl_bidi: struct scatterlist memory for bidirectional READ mapping + * @sgl_bidi_count: scatterlist count for bidirectional READ mapping + * @sgl_prot: struct scatterlist memory protection information + * @sgl_prot_count: scatterlist count for protection information + * + * Task tags are supported if the caller has set @se_cmd->tag. + * + * Returns non zero to signal active I/O shutdown failure. All other + * setup exceptions will be returned as a SCSI CHECK_CONDITION response, + * but still return zero here. + * + * This may only be called from process context, and also currently + * assumes internal allocation of fabric payload buffer by target-core. + */ +int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess, + unsigned char *cdb, unsigned char *sense, u64 unpacked_lun, + u32 data_length, int task_attr, int data_dir, int flags, + struct scatterlist *sgl, u32 sgl_count, + struct scatterlist *sgl_bidi, u32 sgl_bidi_count, + struct scatterlist *sgl_prot, u32 sgl_prot_count) +{ + int ret; + + ret = target_submit_prep(se_cmd, se_sess, sense, unpacked_lun, + data_length, task_attr, data_dir, flags, + sgl, sgl_count, sgl_bidi, sgl_bidi_count, + sgl_prot, sgl_prot_count); + if (ret < 0) + return ret; + else if (ret > 0) + return 0; + + target_submit(se_cmd, cdb); return 0; } EXPORT_SYMBOL(target_submit_cmd_map_sgls); From patchwork Tue Feb 9 12:38:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077945 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73BE3C4332E for ; Tue, 9 Feb 2021 12:40:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3249A64E88 for ; Tue, 9 Feb 2021 12:40:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230232AbhBIMkB (ORCPT ); Tue, 9 Feb 2021 07:40:01 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:43206 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230183AbhBIMjs (ORCPT ); Tue, 9 Feb 2021 07:39:48 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CYA8k054681; Tue, 9 Feb 2021 12:39:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=/PA4gd1D6jVrBXFmI67kErEr/sUV9xPGhvQiBeIrQXk=; b=DUwaFX0gvqgSRe3VeXsh+8pKHu+T3MmB1gOsLNq5UG4MHEL4Go0qS9xXbbKOKQS0J6gW wlH2rjy0r3FIsB529NmDZ7S4/qO4WAS/tAlvqvsESNOnHnTixalY+x0/S5bQXgOarQ8P BJ4MXKTrRUI0WvCD3HvZnoaPVK01kaQnf9Yw546g++IJ4O/DpnSbC0KO8mmRghtxqsVu bLqSOM+vdbrgXCUdQMkz4bIVFNpj0Ae1aPGTjVo7hT9O6BDrnRF8+4G9cmdRkxFzuTBh bifLGkjdF/wJJ+/uH0tT1X3aoCyddVLZQmhdmBMARHoON2FcKOU+MBJmkdItm+CwhA68 6A== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 36hgmafe4p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:59 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIH178533; Tue, 9 Feb 2021 12:38:59 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by aserp3030.oracle.com with ESMTP id 36j4pnn07j-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:38:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VCq+LcRGgk8e5A80+oDvV7gUvLEK9+nP5SZbT5vuO0PPJx6Lk69/KZdcsVfdkp0IlqQWh2rOjdX/xQeQ6vGj8E2zFxNFFJBp+sniZLAH9bCkr+lmmLYaBX8VmsymcTVJayigGIqW2qLP90CknWYx62qu7SookT7sIz5L46oCDjUpFl+G23BTRy9U0Rwp5CN829O47zTv2/TssfjEykL3fwOxGv0ZPPKELX6K8zezY6ccHp88IEvuYmW29UA22P88DF4sabECNA7dZdpczKXjzX8puopDpws303vz7Mycq0EBV8AAxHlPTU20TPkzWormV8bYZ/349RGoWzdhLFTRow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/PA4gd1D6jVrBXFmI67kErEr/sUV9xPGhvQiBeIrQXk=; b=GajDKcug2p8V+ZLDgt8tW4Pz1uFVTDdZzxxWJicbZPiOI+PlDJ8HWdmrT0xn+G8p5MvIOyJuFd8FP6xo58CRw8BTVLMU+xnTI1BCNNOTnrwkACD0aPpE8dGvyEKL8q0HgSsRS13t66Q2uFzf4e3ocD9YbkvK3h9fxXOFTZABX13WQmBxpxWkBWSpP3tmZ3Hy5YPgPQyUKfM84tMi6J35Moh2nAkOL97QsMjiBpfNiGwPK1vhuV96s3xRo1LjmKJvuenE9tRcvC3Vn5/d/LkCI9IpwLfN9rXo5JJ6/mQs8tXzEMF5laVajN74AAXTm+5CSfYPaM6tfizZif8VO0TaWQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/PA4gd1D6jVrBXFmI67kErEr/sUV9xPGhvQiBeIrQXk=; b=puzm1Mw3dv1ghnbG9hCC51TucUizDDTnW5sAl8D8RW+uRZh3AtGPdJG91sZORzJ0fXfzPs8N2DeWhGMyERPHvydvq8XdM73KjEqNKQdEf4y33OquijPsHpShtvr4EwnQXgyXbdmvqB7MCkk5ICSjkF29QWv8JB6W8PB81LKu5XY= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:38:58 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:38:58 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 03/13] target: add workqueue based cmd submission Date: Tue, 9 Feb 2021 06:38:35 -0600 Message-Id: <20210209123845.4856-4-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:38:56 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 685d3e76-995a-446f-76a8-08d8ccf7ab6a X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BlRGrxny6ExQ+lf5UlDesm1FJVggZiEAl5Rh28I0IHhoQI6SskTNy1HQlry7f6m4mWAwx30FFIsK/ga1tVnLQ3NSLXA2rDzy0Fg01yfJE8JamdrocdzRIREgDqkU4yG2/ApVLXh6cjMGaHdkujaNNv0wdRZn8IJF2EwSE4m67ON0FSffKYBgMPRE7ZZ27DLLlHgS9HI2jogCDRFSy4shkVc4efCSSGTvjMdYaa03FH/q2U2IjjXVNKMtW9mX+hhDw4u215af5oLfP/Lfw0Q/6pxUSCh6FUYFvzK/bNaMrIoEcHcBLPo82l8QScaBwknMl/GA7B9X2UpSOa5f7yj8H9r7KobHTBZT262CIC8k9Ze94wtrT8qj3ME6Pb1sz1vQuYyODMfzCHKRpz9ZOHnOBTRIHcOIeqCvB+mrkfjXDB9tLSQ/qigKGBdWMJcO/6LBbA8rpqaExQAlq+KFMVnrAu01NHSmkj7eg7oDDRLkbREFZBrNPU4WY4OjcFOYMSj35HFFCKIZhUPXpHeR5mOvGUF/SmBFxbitKv48kojQOwu5kGykR69IhenTizy7kjov+ldOT4tPdErW6c1/4HKRpw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: trq/OHb3UlXaMAOP+E8spcc2cCTLeNekSdv/hSCPHnbaHphDH7p1T8NLTJWDfLDbGeOMRXTe3K29tbsk5Ywke08O/XIsG8t28Z/hPGIpziujNJS/5Zj4QYuadbmcDr3Tjybt7BbfAvSfWVwmSYNcpSq37F/5fwBXaqdzfSBWxOROMBPdBYuul2x6/fzTjAvXHSK5JT0vhLISgbqYzVLhukB0Qx5912wufjlgtSWFdTvzgLIY0rDkEZipMiYhIcCWYj23DBNbPwu+IezdBE8rhFiM/7EKvXyiwiVnFemZUa8xsYbLptcxf1S4OH77W7DqBI1+6LRTmSSo2yMGrLiH7z1Rj4RtmYs0Z2iQiLBaczqOEyoA+9P5U8KwrG/O7t3U2g/TWeklEGfRYIRYpAhh2minTEJrGhDiSCGkxx55jZY7mBdzYXV9v+dXvtzsDGlfjgIl4fWatUBVdqSAuqgx5R3aIl1Gy4p71Z5vhmX6Kr0VI6tG9Ge3sOHFxKd1glz7UrM/5+v4fVDvEJPcbJKA2+9BAH1AF8NJ2MsX7pc+7wBsqjF2IHDpLuYVDFrAxggG8Um/cCHqa6ypBG8rgqBt6j4E3aRWS7M4OFm47S77QLAgADLOJ0W7M0kut8xRJOKOAuc7oE7H6UdiCWtGm78Y+ox4jXIjPGvM1vCjfMJgmIz73BdmJcevNtKLW3S2AJbADuzk98Il+aZ2VFdeCF4dOVKgH1sAUuC8Hpm6an/UcUBWvvc2wJVD+n/+AsUz47EL0GKDdCsUHqy7bgXiiwdsX+9wCPF9fbCww5oHnMGwMiSrWx0P15YUoALAo5esO5Osi5Z/i1oJSS6a923geXVFDPC0WNoDQGfpapfi6km0K+ek4zv58CPxzAk4d9KXDkNdbnzc1UvXtXN0wCPG9ta68jwMqi5eNY3nWT8bEci5FvWCGVQj5fuLwlOc54fCnj0+bC6Eqs3sVKnI6o1kVJkfVyYtB8Z3IUWLDF6mRZBVk3R5zs6VP9rgFUhBrU3mUM6viHObf+jvLYcOr4vGq3ckYDx6qMK1V92lRcq7FQ0GxqV+lNzn/dHzCuumcACCAwPxXfOPOx39hXWzfI9bPox+DAZXLAvQqoRxouWoK/uarvOhzHdZyshUYlEYRASLEBY4rPcOBTn0/ysi9NcSd03/HRgEBFLkVkOADXPufYYQcTwad44QDYFi77ZgJ086MiIF59nzNDCxq/f7YpagoS1nOvoMJTO2jkmJ5sJ+77i8762vgsAQ6WV80V3/D0+spXS5zTx/qq+Mq63Rkn+jB+EFu9yu9K5xmAX6HsugDE75igy/MJELuXwi3UqLjcYr6cAj X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 685d3e76-995a-446f-76a8-08d8ccf7ab6a X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:38:57.9323 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: jNbNFNqI+BEEl2c+4zQTS6BG979/w77BH83Ff9cy9FcGKIEVP2iuqBDy/3mEX1cyDhizs9m3W+nA1zPTOjMSP+01DQVoMaP1ZyMMbLAQbm0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 suspectscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org loop and vhost-scsi do their target cmd submission from driver workqueues. This allows them to avoid an issue where the backend may block waiting for resources like tags/requests, mem/locks, etc and that ends up blocking their entire submission path and for the case of vhost-scsi both the submission and completion path. This patch adds a helper drivers can use to submit from a lio workqueue. This code will then be extended in the next patches to fix the plugging of backend devices. Note: I'm only converting vhost/loop initially, but the workqueue based submission will work for other drivers and have similar benefits where the main target loops will not end up blocking one some backend resource. I'll port others when I have more time to test as I think we might want to make it configurable for some drivers. Signed-off-by: Mike Christie --- drivers/target/target_core_device.c | 10 ++- drivers/target/target_core_internal.h | 1 + drivers/target/target_core_transport.c | 91 +++++++++++++++++++++++++- include/target/target_core_base.h | 8 ++- include/target/target_core_fabric.h | 7 ++ 5 files changed, 113 insertions(+), 4 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 7787c527aad3..74d3a4896588 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -735,8 +735,14 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name) dev->queue_cnt = nr_cpu_ids; for (i = 0; i < dev->queue_cnt; i++) { - INIT_LIST_HEAD(&dev->queues[i].state_list); - spin_lock_init(&dev->queues[i].lock); + struct se_device_queue *q; + + q = &dev->queues[i]; + INIT_LIST_HEAD(&q->state_list); + spin_lock_init(&q->lock); + + init_llist_head(&q->sq.cmd_list); + INIT_WORK(&q->sq.work, target_queued_submit_work); } dev->se_hba = hba; diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index e7b3c6e5d574..56f841fd7f04 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -153,6 +153,7 @@ void target_qf_do_work(struct work_struct *work); bool target_check_wce(struct se_device *dev); bool target_check_fua(struct se_device *dev); void __target_execute_cmd(struct se_cmd *, bool); +void target_queued_submit_work(struct work_struct *work); /* target_core_stat.c */ void target_stat_setup_dev_default_groups(struct se_device *); diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 71b0a862608b..46e20c42119e 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -41,6 +41,7 @@ #include static struct workqueue_struct *target_completion_wq; +static struct workqueue_struct *target_submission_wq; static struct kmem_cache *se_sess_cache; struct kmem_cache *se_ua_cache; struct kmem_cache *t10_pr_reg_cache; @@ -129,8 +130,15 @@ int init_se_kmem_caches(void) if (!target_completion_wq) goto out_free_lba_map_mem_cache; + target_submission_wq = alloc_workqueue("target_submission", + WQ_MEM_RECLAIM, 0); + if (!target_submission_wq) + goto out_free_completion_wq; + return 0; +out_free_completion_wq: + destroy_workqueue(target_completion_wq); out_free_lba_map_mem_cache: kmem_cache_destroy(t10_alua_lba_map_mem_cache); out_free_lba_map_cache: @@ -153,6 +161,7 @@ int init_se_kmem_caches(void) void release_se_kmem_caches(void) { + destroy_workqueue(target_submission_wq); destroy_workqueue(target_completion_wq); kmem_cache_destroy(se_sess_cache); kmem_cache_destroy(se_ua_cache); @@ -1380,7 +1389,6 @@ void transport_init_se_cmd( { INIT_LIST_HEAD(&cmd->se_delayed_node); INIT_LIST_HEAD(&cmd->se_qf_node); - INIT_LIST_HEAD(&cmd->se_cmd_list); INIT_LIST_HEAD(&cmd->state_list); init_completion(&cmd->t_transport_stop_comp); cmd->free_compl = NULL; @@ -1827,6 +1835,87 @@ int target_submit_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, } EXPORT_SYMBOL(target_submit_cmd); +void target_queued_submit_work(struct work_struct *work) +{ + struct se_cmd_queue *sq = container_of(work, struct se_cmd_queue, work); + struct se_cmd *se_cmd, *next_cmd; + struct llist_node *cmd_list; + unsigned char *cdb; + + cmd_list = llist_del_all(&sq->cmd_list); + if (!cmd_list) + /* Previous call took what we were queued to submit */ + return; + + cmd_list = llist_reverse_order(cmd_list); + llist_for_each_entry_safe(se_cmd, next_cmd, cmd_list, se_cmd_list) { + cdb = se_cmd->se_tfo->get_cdb(se_cmd); + target_submit(se_cmd, cdb); + } +} + +static void target_queue_cmd_work(struct se_cmd_queue *q, struct se_cmd *se_cmd, + int cpu) +{ + llist_add(&se_cmd->se_cmd_list, &q->cmd_list); + queue_work_on(cpu, target_submission_wq, &q->work); +} + +/** + * target_queue_cmd_submit - queue the cmd to run on the LIO workqueue + * @se_cmd: command descriptor to submit + * @se_sess: associated se_sess for endpoint + * @sense: pointer to SCSI sense buffer + * @unpacked_lun: unpacked LUN to reference for struct se_lun + * @data_length: fabric expected data transfer length + * @task_attr: SAM task attribute + * @data_dir: DMA data direction + * @flags: flags for command submission from target_sc_flags_tables + * @sgl: struct scatterlist memory for unidirectional mapping + * @sgl_count: scatterlist count for unidirectional mapping + * @sgl_bidi: struct scatterlist memory for bidirectional READ mapping + * @sgl_bidi_count: scatterlist count for bidirectional READ mapping + * @sgl_prot: struct scatterlist memory protection information + * @sgl_prot_count: scatterlist count for protection information + * + * Task tags are supported if the caller has set @se_cmd->tag. + * + * Returns non zero to signal active I/O shutdown failure. All other + * setup exceptions will be returned as a SCSI CHECK_CONDITION response, + * but still return zero here. + * + * This may only be called from interrupt context if the caller's + * queue_status and release_cmd callouts do not block. + * + * This assumes internal allocation of fabric payload buffer by target-core. + */ +int +target_queue_cmd_submit(struct se_cmd *se_cmd, struct se_session *se_sess, + unsigned char *sense, u64 unpacked_lun, u32 data_length, + int task_attr, int data_dir, int flags, + struct scatterlist *sgl, u32 sgl_count, + struct scatterlist *sgl_bidi, u32 sgl_bidi_count, + struct scatterlist *sgl_prot, u32 sgl_prot_count) +{ + struct se_device *se_dev; + int cpu, ret; + + ret = target_submit_prep(se_cmd, se_sess, sense, unpacked_lun, + data_length, task_attr, data_dir, flags, + sgl, sgl_count, sgl_bidi, sgl_bidi_count, + sgl_prot, sgl_prot_count); + if (ret < 0) + return ret; + else if (ret > 0) + return 0; + + cpu = se_cmd->cpuid; + se_dev = se_cmd->se_dev; + target_queue_cmd_work(&se_dev->queues[cpu].sq, se_cmd, cpu); + return 0; +} +EXPORT_SYMBOL_GPL(target_queue_cmd_submit); + static void target_complete_tmr_failure(struct work_struct *work) { struct se_cmd *se_cmd = container_of(work, struct se_cmd, work); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 63dd12124139..815de4c97230 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -487,7 +487,7 @@ struct se_cmd { /* Only used for internal passthrough and legacy TCM fabric modules */ struct se_session *se_sess; struct se_tmr_req *se_tmr_req; - struct list_head se_cmd_list; + struct llist_node se_cmd_list; struct completion *free_compl; struct completion *abrt_compl; const struct target_core_fabric_ops *se_tfo; @@ -764,9 +764,15 @@ struct se_dev_stat_grps { struct config_group scsi_lu_group; }; +struct se_cmd_queue { + struct llist_head cmd_list; + struct work_struct work; +}; + struct se_device_queue { struct list_head state_list; spinlock_t lock; + struct se_cmd_queue sq; }; struct se_device { diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h index d60a3eb7517a..d7b54f3e04da 100644 --- a/include/target/target_core_fabric.h +++ b/include/target/target_core_fabric.h @@ -80,6 +80,7 @@ struct target_core_fabric_ops { int (*queue_status)(struct se_cmd *); void (*queue_tm_rsp)(struct se_cmd *); void (*aborted_task)(struct se_cmd *); + unsigned char *(*get_cdb)(struct se_cmd *); /* * fabric module calls for target_core_fabric_configfs.c */ @@ -160,6 +161,12 @@ int target_submit_cmd_map_sgls(struct se_cmd *, struct se_session *, struct scatterlist *, u32); int target_submit_cmd(struct se_cmd *, struct se_session *, unsigned char *, unsigned char *, u64, u32, int, int, int); +int target_queue_cmd_submit(struct se_cmd *se_cmd, struct se_session *se_sess, + unsigned char *sense, u64 unpacked_lun, u32 data_length, + int task_attr, int data_dir, int flags, struct scatterlist *sgl, + u32 sgl_count, struct scatterlist *sgl_bidi, u32 sgl_bidi_count, + struct scatterlist *sgl_prot, u32 sgl_prot_count); + int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess, unsigned char *sense, u64 unpacked_lun, void *fabric_tmr_ptr, unsigned char tm_type, From patchwork Tue Feb 9 12:38:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077943 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E20D8C4332D for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A45AB64E50 for ; Tue, 9 Feb 2021 12:40:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230182AbhBIMj7 (ORCPT ); Tue, 9 Feb 2021 07:39:59 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:39130 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230184AbhBIMjs (ORCPT ); Tue, 9 Feb 2021 07:39:48 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CYEdh047856; Tue, 9 Feb 2021 12:39:02 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=u+xiQn5GyXdQkBxVmYnI8zNMC01FeeMnEGS9mCrFlsI=; b=IGWiRwAmag8htC30vXBslO5cGvW8qZ9O1S4uJbOQbkufyiYYA19h78qg0PAI9ALyVC7m zjdPKnqu9YQn6WXRGyIDoC6GLvQglAie6D6GQDlKU9NlCcPsV6fymqE48RkO7aU9thLI P8BbtjGg1umCgBhW8+26Tu+pCJ66pTGGDo2Ze60sIF+1McOYRi4U/hz1wmIQbl74cWNK cnQrLQNgg2mN2tYuDonQOcf4SrSfzkVqkCP/IiVRklbnluO9Krmjju3Gr+c0dsOBfK5p UW8AXWZz64oZ/YwLD7odoagluFjGMVIDFiYXyPOK5iHUneRjvp+ELimTdPkneKq7otYL KA== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by userp2120.oracle.com with ESMTP id 36hkrmy8c9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:02 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIK178533; Tue, 9 Feb 2021 12:39:01 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3030.oracle.com with ESMTP id 36j4pnn0cr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eQoQrmbIqfpvTUTNBjebcHeKkrxErMHK7ZAQ1frc/M1LcNamUz80zMiTe2kmUQiddWBm86PmSpIJdibbzN6avZfPmN84ORBhTe63mfxJUb5iLmLqKig5HpW45Jpzyt7mHi3m5wk6vDBXcnILX41q8SzWiAwtIgD0yLG+GTAwbNjpT3fK8rgXTwNvYsRlrhU/re2gGtxe1F3pAIDkiGpkFNaXk14L8ArzYB4Rcy9AgzOkaVIq4rp0q29tMmH1Yqpq1keO3Ghs3fCtwRMbrGRYXVTamaQn2mHCfBKhD9QSsAYrXfyktrJ/DIp3zubUL5B/BKOi+YaSiqgiYpj6fPk9/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u+xiQn5GyXdQkBxVmYnI8zNMC01FeeMnEGS9mCrFlsI=; b=WIYaW0FnORRHXtQZImkd7gN/Lfy8oY6UevGBqSXcnSnOLCeuaApT8HM2P4JR+UT7W9YqVzXNj0ioOZur5j4HNFBttHAd+dFFzBYb6GIMVlToKxf1weH4AQ/uMcdC2r+Ye9nmXuFnUOS6pgKCFNvxrEiZKm3zkyS7PR5RUuNGCJvh+e1BhVKIKNi46oAPKjQQG3qFbTpKHGQdha5NTf0Kf1iwYmZW3BWUmppM0sax4T8gkMtK7NMt1t9u+u5L01g5MKTGFQebiRdL73CXFWdFuV+1VM01GGI0Wr913XhYqBv/jesNclxUUkPYNYFP8SOXplnv9PnQKfQC/1rF4AoRKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=u+xiQn5GyXdQkBxVmYnI8zNMC01FeeMnEGS9mCrFlsI=; b=cjBi7/qjSHB471ZHvJiSEtvYtrhu3ocv5XfxkA5p+xySZWU4z4crLjiE8BLQ7t5tLmpYrBQ8LAPH51Y7U8ORr9tx42d/Ec87d2z8qm8VrBAoDWkYfNddUf5JNVZ01fq5bVbb7YMo7ivMFVXv12zK0GagmtJmmM0PFO/B19IwbUs= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:38:59 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:38:59 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 04/13] vhost scsi: use lio wq cmd submission helper Date: Tue, 9 Feb 2021 06:38:36 -0600 Message-Id: <20210209123845.4856-5-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:38:58 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4c746f92-72c5-4ad0-e89f-08d8ccf7ac1a X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5797; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: myEZXt4oc2Y0kFe7u2v8Dsn4foE+L5lhJTnHepPnk/9b1+u2Op4PfzsvYA13xYvrGi4tfn8STFJcQGg03PHZk/OTEwOenmzxgaKQzCbod4X7yphsTtULNaVmjTigyRrXMbU3HBjSsScJUqrC+qKP/BvI7kxq4JJyHbNLv/OSS/9oh/OOuEfELlZt2rJWG8Eos0EISHonnZMVudEOEm7HtLAFIFWPDooRRZm9Bf6RwKLTs1c+SKx3V0DRMZUUe2LeRGiF071c7/d+LKmT5RcPXasLjkZ3ao1TysGBv0zS7rDFKqBD9vzCAn7USbpDZOkYboYDILuYfDsb2+BGz+WLCInCmg95p9z7Un5dGC3h7V/kAZJ0DeX4IZa8XwBYyuWBbvmq39aNoQdcM44EmuTUoys4gAEK8hcs4ojNifmCrNWXGtUu7joCtxM8/W5vhV45K+pdJzZrLCXN8/bdKA8EkHXppBTxojRGKKvIjApkjYzz35pLhR4n9kny/xowCkvds/4j43b+Duv2Ta1G1mtirnqdFFfInZ7hiJGnbSQfNaZtI+mgfuu0VZvOQtynqnwOEyZoK/Pz42ma38G87CQ5hQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: luXGpZeQ9CHR7Nif3aD0tv+VuhNgsRCR0+MwltmY2TC4+uEelnxhEuQDqg6UlPelFHvNgPpR3U7PRVv4ijr5O0reJMznXF1K19ahfCFcB+ByOKKrFhB62vwK4fohEp084wxyaZ+cjsKv2bvXGy8Y+SzIzeeS9i+7DGjJyjYw3G5UN4z9SJNVBMmCfC3O81/ls4m+mxhdy5q/F5enlP6NkLdb06jYMPuG+yrubFkcAoo44fVy/Svjd/LvJjCkk31xdiPpgkl79ikl2oCpv7m1YavqEkN9OAx75y3gXqUfCQ3mzjDY5K0hbyKDmm0zOsex5vB69d6W5zXMvvz1+Gw0EIwNyTMkb08P1oAEZxp0NV0/4GvqpaItMYVYzxjbk7xx5iFxmXBUjAr9cC3UNARlBTQWzKk2IUEZCWVqbdl/+9dNTc/0TyO36Zp2Jm1UD5AKfCZ5unXo2K8F1wQo8b1FZrn3HpmHuBDATOdo/nZcTJ0zV3UsawtlcRAHRAcYfQ1P5hn+5/dlYgFTZwrf5aZSLLdHgw4SUnRbE70mdtOCfqQz3F2Yh3L/qLJ2NalZtyy62yBR66w/8kbx/ntEC+zzjIgBlgfbJ9RXQYPnvybskxWb2aE6i0pwUxRh6uMy7KtE3Vfci0RO1tGyNGEET1+g3I96g9Eo32pl31Jya23Td4hHfqVPuhmeYZpEBcZt0sJEN+31b/qaHku2Ep8E4XHuret5eTHXL3qlnm6dEKI/DdHjhUf1+oVJA7Sohx0x5YF2GdnnJfOKYLraCmM+vesp1HNqBToSOodE7WOZyDwK0cQuas0HXyX1lcjhWpGqsLkSjVsdbg0Ud0DRB853utT9tnSciURvp1fx5xjW91UFoXqN+PFQLeGrGDk3+W54pvAMe6VEF0pNgWT2fkxV76/nLcjVkKHHOpwOvQtJ13UOih5FJIGw09k1bBv+1h2VlLccdFq6PqCpjFXvKNA4z05JbNzlq09BamJ9nfQ7W1CY5G2xWaNa+jGyg9B7lvTzPpj5Nex859T359Y7i2jJIQNS7TT2I+VzqNscKNSTR57XojFAUtLlyepMi2Usi3GqIfQj0gx2n66oYJkPijA58i9la2onZJ/l5q5yiPKVti5AXx03WtHPsR0w8pANaJLvRZctaceCKChs+Asr+Gvic9ZWQy/aiNCv4kEsbh2/VX1twnONGjw1A63Nq4OJmhbRxvKR7RiScjIM9ed6aAdy/fU28bTk74a4KK1Rl+Mgu23RzDbfqN6YMe+HF3bfhDxbh+smlz0Dqaa+XxTq+CiMtQxYxRT1Z+BF+pL6G2qNGC+Q6mPWm4fP6UBiu470ArNnRmqp X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4c746f92-72c5-4ad0-e89f-08d8ccf7ac1a X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:38:59.1896 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: vgYbfa6PEx+HgdbLGEnHpcpMnbhT0s4D8J8HAAfrzWOZM2doEPg+MJuw3oXH37EPnXk9xjBkwi1TI+5W74kudhlw/+sasDUVrxWdkW/Mz5E= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Convert vhost-scsi to use the lio wq cmd submission helper. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Michael S. Tsirkin --- drivers/vhost/scsi.c | 49 +++++++++++++++----------------------------- 1 file changed, 17 insertions(+), 32 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 4ce9f00ae10e..99909c6f3960 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -85,7 +85,7 @@ struct vhost_scsi_cmd { /* The number of scatterlists associated with this cmd */ u32 tvc_sgl_count; u32 tvc_prot_sgl_count; - /* Saved unpacked SCSI LUN for vhost_scsi_submission_work() */ + /* Saved unpacked SCSI LUN for vhost_scsi_target_queue_cmd() */ u32 tvc_lun; /* Pointer to the SGL formatted memory from virtio-scsi */ struct scatterlist *tvc_sgl; @@ -101,8 +101,6 @@ struct vhost_scsi_cmd { struct vhost_scsi_nexus *tvc_nexus; /* The TCM I/O descriptor that is accessed via container_of() */ struct se_cmd tvc_se_cmd; - /* work item used for cmwq dispatch to vhost_scsi_submission_work() */ - struct work_struct work; /* Copy of the incoming SCSI command descriptor block (CDB) */ unsigned char tvc_cdb[VHOST_SCSI_MAX_CDB_SIZE]; /* Sense buffer that will be mapped into outgoing status */ @@ -240,8 +238,6 @@ struct vhost_scsi_ctx { struct iov_iter out_iter; }; -static struct workqueue_struct *vhost_scsi_workqueue; - /* Global spinlock to protect vhost_scsi TPG list for vhost IOCTL access */ static DEFINE_MUTEX(vhost_scsi_mutex); static LIST_HEAD(vhost_scsi_list); @@ -782,12 +778,10 @@ static int vhost_scsi_to_tcm_attr(int attr) return TCM_SIMPLE_TAG; } -static void vhost_scsi_submission_work(struct work_struct *work) +static void vhost_scsi_target_queue_cmd(struct vhost_scsi_cmd *cmd) { - struct vhost_scsi_cmd *cmd = - container_of(work, struct vhost_scsi_cmd, work); - struct vhost_scsi_nexus *tv_nexus; struct se_cmd *se_cmd = &cmd->tvc_se_cmd; + struct vhost_scsi_nexus *tv_nexus; struct scatterlist *sg_ptr, *sg_prot_ptr = NULL; int rc; @@ -805,9 +799,9 @@ static void vhost_scsi_submission_work(struct work_struct *work) tv_nexus = cmd->tvc_nexus; se_cmd->tag = 0; - rc = target_submit_cmd_map_sgls(se_cmd, tv_nexus->tvn_se_sess, - cmd->tvc_cdb, &cmd->tvc_sense_buf[0], - cmd->tvc_lun, cmd->tvc_exp_data_len, + rc = target_queue_cmd_submit(se_cmd, tv_nexus->tvn_se_sess, + &cmd->tvc_sense_buf[0], cmd->tvc_lun, + cmd->tvc_exp_data_len, vhost_scsi_to_tcm_attr(cmd->tvc_task_attr), cmd->tvc_data_direction, TARGET_SCF_ACK_KREF, sg_ptr, cmd->tvc_sgl_count, NULL, 0, sg_prot_ptr, @@ -819,6 +813,14 @@ static void vhost_scsi_submission_work(struct work_struct *work) } } +static unsigned char *vhost_scsi_get_cdb(struct se_cmd *se_cmd) +{ + struct vhost_scsi_cmd *cmd = container_of(se_cmd, struct vhost_scsi_cmd, + tvc_se_cmd); + + return cmd->tvc_cdb; +} + static void vhost_scsi_send_bad_target(struct vhost_scsi *vs, struct vhost_virtqueue *vq, @@ -1132,14 +1134,7 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) * vhost_scsi_queue_data_in() and vhost_scsi_queue_status() */ cmd->tvc_vq_desc = vc.head; - /* - * Dispatch cmd descriptor for cmwq execution in process - * context provided by vhost_scsi_workqueue. This also ensures - * cmd is executed on the same kworker CPU as this vhost - * thread to gain positive L2 cache locality effects. - */ - INIT_WORK(&cmd->work, vhost_scsi_submission_work); - queue_work(vhost_scsi_workqueue, &cmd->work); + vhost_scsi_target_queue_cmd(cmd); ret = 0; err: /* @@ -2466,6 +2461,7 @@ static const struct target_core_fabric_ops vhost_scsi_ops = { .queue_status = vhost_scsi_queue_status, .queue_tm_rsp = vhost_scsi_queue_tm_rsp, .aborted_task = vhost_scsi_aborted_task, + .get_cdb = vhost_scsi_get_cdb, /* * Setup callers for generic logic in target_core_fabric_configfs.c */ @@ -2489,17 +2485,9 @@ static int __init vhost_scsi_init(void) " on "UTS_RELEASE"\n", VHOST_SCSI_VERSION, utsname()->sysname, utsname()->machine); - /* - * Use our own dedicated workqueue for submitting I/O into - * target core to avoid contention within system_wq. - */ - vhost_scsi_workqueue = alloc_workqueue("vhost_scsi", 0, 0); - if (!vhost_scsi_workqueue) - goto out; - ret = vhost_scsi_register(); if (ret < 0) - goto out_destroy_workqueue; + goto out; ret = target_register_template(&vhost_scsi_ops); if (ret < 0) @@ -2509,8 +2497,6 @@ static int __init vhost_scsi_init(void) out_vhost_scsi_deregister: vhost_scsi_deregister(); -out_destroy_workqueue: - destroy_workqueue(vhost_scsi_workqueue); out: return ret; }; @@ -2519,7 +2505,6 @@ static void vhost_scsi_exit(void) { target_unregister_template(&vhost_scsi_ops); vhost_scsi_deregister(); - destroy_workqueue(vhost_scsi_workqueue); }; MODULE_DESCRIPTION("VHOST_SCSI series fabric driver"); From patchwork Tue Feb 9 12:38:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B8AFC43332 for ; Tue, 9 Feb 2021 12:40:04 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 01C7D64E50 for ; Tue, 9 Feb 2021 12:40:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230188AbhBIMkD (ORCPT ); Tue, 9 Feb 2021 07:40:03 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:43228 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229988AbhBIMjs (ORCPT ); Tue, 9 Feb 2021 07:39:48 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CYAaC054680; Tue, 9 Feb 2021 12:39:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=AakTnBcU2oH3AQ71ZMPcSB4AA5mBPEJzCX8nCoSYz1U=; b=T6JvKBuJn42fjKd9fb9+986NB1Fmjor/ovsyQ2sH1J+aWlBZ7zNrLCqHkOswLcyKXdyQ IGEHDiALWbc90xzTRFCYPfyCTDambmD9UkoI8XmpvXg3pY4/7ZDN1W4QRlLotyb9Om+Y TwZSaygv3neRc37mNEs9d/Zt1f8T8tX57fr+3r9fZqfVDoELhci06eMKm8Vf5JSEtEa/ zYFvyZbFY0Vav/KD11f9E7EfEVjClWP6moD/KikyLIhbh3HU2gmUdZv+DLuNzVKKl72O 7uaAl9CM3QnyWDCB5QdvkzFeOK+5Or+rsb/VToeoLxmBGHA/GAA/JkCbM6almipnjLyh ag== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2130.oracle.com with ESMTP id 36hgmafe4s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:03 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIM178533; Tue, 9 Feb 2021 12:39:02 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3030.oracle.com with ESMTP id 36j4pnn0cr-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=k8aF2mYTUypktRB1zJrRcOciPzN6dWfvr2EypReDp2SedrW/aPYtXDEfLC0g8KysoZWgqBo+BOXVNXrGn8+Hlo4FU4UhR1WDI72Ue73O1gLsMp0+C4YEEPYc9OfeO1NLm9kNLXxlZmSr6XdmHDFHBDDsOnmGYufns9IDiFG//Pzk4JfuEu4OO7FzPE5NPG8CD3uEtb8Dy0l3jLHHNMa3RXoRx5fus8lBgHhyxQ02YH7qzWv4s/FzXZmIKmulqaKwVYyD6ase3zEMF7x75cf7gKt+Qago6JOexmHoMnkpzpLNyACR4QqcT3u8KmHg5Wpt05kQS/2OOQFyHsfBl9RQDQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AakTnBcU2oH3AQ71ZMPcSB4AA5mBPEJzCX8nCoSYz1U=; b=beX95R03TGCizceeTKdgI3u33u8lLgUU0icSyTRgVEADy8xHVPgL5Gp2G8jYApiBJ2fBpaZCfWwnxtykwyUOt+wKrABsHRcx2THkFgtcnPpAwBZ3ZTL0Y+VZbsqJAPoGReANVsZOw6WYQ+nyjOc9aUXvCMGzmZ7x+t5f+xsEvoxEO0Gd8+5ews3Fxw32+WCNXiECiwul22jnXlOCoDc3FdzptuiTkSzFcXj9V3hud2+jbBnJNXszy8Rbtgyz0TyL3xrY+pdQ14tGVJ8B4qguFxNmzij0s9C3dPNw/rXoEo4Ve4CaxB6YvPXkoNgOA8kgR9/vVRd5gSR5WBOCEr3mQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AakTnBcU2oH3AQ71ZMPcSB4AA5mBPEJzCX8nCoSYz1U=; b=MMr51YEvQLQGykTwz5BbGNWwLDyQmlyBWrvMMPTuxFojh8zK7MVj0LTqAr0JcfHyUA0C1uFX6ofjiIII7BpkkVn4TO0uqt/AI6dshlH8hjROr1jalR4ZiIT3yCEF4zGCv6OUtAKVzOVe4Py0AeVIVQbJKUkRYtvE1Awb9NvLLF4= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:00 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:00 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 05/13] tcm loop: use blk cmd allocator for se_cmds Date: Tue, 9 Feb 2021 06:38:37 -0600 Message-Id: <20210209123845.4856-6-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:38:59 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 207522d1-9b49-468e-7806-08d8ccf7acd8 X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1002; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N8LwlNjGT6BoSm1J9wOAnHXBf8elO/pw9sW9MAYIqNa1PY8bJjzbtr1/9oJntRLx3iK8Eaq+iC/4TQJfoOABuh4ofKRZ+rDfmQaF8SoOUL4XyI93VCc433QtaKIVk5WnYgrMyYsdaabyssZDG9/9jALrCtLCVTk+nRDu+tme3P7mbpC6SjM76QKmooxnknvBpqrxeGErVa0DwbcuqpgTGO7rPvSjOLczrCfpM6Up0eYNVrEOyXSUH4DVbm5XXbF3aMPyO5BPH/f2d0zYjZhN62nA5pKxBXZhGe0ebOm3TIa+mG2dRMaM3sRVb9zu2EBvkUsxpjiBYbjBXSuiS3dTIYOoWU5YMciKqASzLp1EE+LVHsOq/judPI2pEpR4jDa7g2Pj8a8hma0u2AUKwu8/8s/ce3ALuYjKbE4s55Ss5rvLfHp02qIWrkVs5N3UsJuT/VfFYaC/Ci3kuSikKOhNrPnKwR25JwMdq0ZAA1Tyx+xo7wSeLGIynfqS8PxcWRH9n9L5kHPVTTYmZmh7LliHFLUGDiUHV/X5Xy2z7eUBZ2Lz9lTtdGfLL4EGNDB8TetLk9JmRee/liwK8zvENx6tBw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: mreRhkh6qsX55VEMAnCGqiibKDuDaD68ET9DMDJ6gfD/mbG/3pjCr1bi0umq6otmFXGhUhZsAYd/5U0rFpgJV8RD7pN0rsegwXDp5h9EfB6ufzOXWQvdrhV0RZNDYQ5pjwmKb33fCmdRYu7C1St41Y+aw2OPeO6BVqfKxOdoxDX7WU3IVdlq0q1ZEOlKG80cI4szoSzKq9lY5GUV/C6h8uTFT9XkRg56+FOtQ61TgETL1tBE2Q0d/JHdUlHMYn1yETqmPb5shA2yn/YVhCVOhxnGT1p+1O0b+xy7KkSI0YA9sLbyQyI2FVNYgzJglvs5NI8v3JtoA5En5BZUWrLsswISDgKOjuzf3lgXtAngfoI4aQ7FYw1IWjkGvHjqWopEfBfh5m6AgIjHaB4sVbwq3TWut85G4yh5UGR9oqKLLSeAV0P3JB/UFy7yUAwSoesKzCd0ngnOlPfxxrUofGmApSnZfOeg6t17CyCcWA7Mcn6+kqZDTYEXI5+85knIBkFV0roCLoGJFtvspTACgtjPibVLmACGsLt4VWg9V8JHSg8d+kOzvXFC1gyFTrD4LY1AI42A7AYPbucn1iQZy6aapFp/Nl3olz2/qgI4OED4YyBpkm5ebVfAmzV3M3T4JkMGaKNK5uehaT3ghTub9Ranjch/LCMEgTiDvv/6url74H34haFv8lmIos43kNoSoh4SKsz76wdGGrESg3rMIRCahriUgs+DZBrf/QnX0nNlg8FOiow6I5+9LEtA1gVYMFYlxgTCFgJWMKNk7xsM682VklDF5ZztWy0HRntYk40h/k/J0t5RoIsfHbQJDS91sSjN+9LYht5VNDWFvc+ircA7YYVL+XGvN6QFYKUpFBf3X9hOAkhymh7gUjY43EOyB31nInpu5w3RdvUcv+KzBXPd8JwtZhstCUKorrseHncsBHevYjsCv3vNA+tP0qIcqkRBEACLdQ5IHErDivZXh5BzblhPQ1vvdUVgPl3J3GweZqvXWgIUOknjLE0J/V/n1YQgJoGldd20szXA3Od2TVhuIPRQvUEyrSYvwvYkSoyrOxkHkLSuaXgkeAwPi1cNJ0bR8kTVdwrrpFpIsTvG0cUxqtBBGU7ZRk9ejF3P3zl1g68McumOb6jRY87gK0priQk8q3QDhPXGWzfaAY4c8S4rZFmRO3MAGsN3O65AsvbxPzRT1TopLnmFoxnXvGSLA58WJwj61YyC96sEvGRB361OFearoas4P6y/gwoMUHVEErpr27olOQmmlgVej5D5weOdOpMS8uFc+fclbUL5kweZUKnUQpwovZSS6r6YdDkcJXpBrVlZcyZYp/xT4ihrUWae X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 207522d1-9b49-468e-7806-08d8ccf7acd8 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:00.3830 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pkvd0jU5EcTcYujEbD5AVlIqoNkYzjsSMbIwtUhwWXRn3uJ3YfG9Jd00fSjAD/lYDmRFk8cFeew+VVQIu1Lws3BJjFEPnLHfhhr7xllcoN0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 suspectscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This just has tcm loop use the block layer cmd allocator for se_cmds instead of using the tcm_loop_cmd_cache. In future patches when we can use the host tags for internal requests like TMFs we can completely kill the tcm_loop_cmd_cache. Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig --- drivers/target/loopback/tcm_loop.c | 19 ++++++++----------- 1 file changed, 8 insertions(+), 11 deletions(-) diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c index badba437e5f9..274826a2b0bd 100644 --- a/drivers/target/loopback/tcm_loop.c +++ b/drivers/target/loopback/tcm_loop.c @@ -67,8 +67,12 @@ static void tcm_loop_release_cmd(struct se_cmd *se_cmd) { struct tcm_loop_cmd *tl_cmd = container_of(se_cmd, struct tcm_loop_cmd, tl_se_cmd); + struct scsi_cmnd *sc = tl_cmd->sc; - kmem_cache_free(tcm_loop_cmd_cache, tl_cmd); + if (se_cmd->se_cmd_flags & SCF_SCSI_TMR_CDB) + kmem_cache_free(tcm_loop_cmd_cache, tl_cmd); + else + sc->scsi_done(sc); } static int tcm_loop_show_info(struct seq_file *m, struct Scsi_Host *host) @@ -165,7 +169,6 @@ static void tcm_loop_submission_work(struct work_struct *work) return; out_done: - kmem_cache_free(tcm_loop_cmd_cache, tl_cmd); sc->scsi_done(sc); } @@ -175,20 +178,14 @@ static void tcm_loop_submission_work(struct work_struct *work) */ static int tcm_loop_queuecommand(struct Scsi_Host *sh, struct scsi_cmnd *sc) { - struct tcm_loop_cmd *tl_cmd; + struct tcm_loop_cmd *tl_cmd = scsi_cmd_priv(sc); pr_debug("%s() %d:%d:%d:%llu got CDB: 0x%02x scsi_buf_len: %u\n", __func__, sc->device->host->host_no, sc->device->id, sc->device->channel, sc->device->lun, sc->cmnd[0], scsi_bufflen(sc)); - tl_cmd = kmem_cache_zalloc(tcm_loop_cmd_cache, GFP_ATOMIC); - if (!tl_cmd) { - set_host_byte(sc, DID_ERROR); - sc->scsi_done(sc); - return 0; - } - + memset(tl_cmd, 0, sizeof(*tl_cmd)); tl_cmd->sc = sc; tl_cmd->sc_cmd_tag = sc->request->tag; INIT_WORK(&tl_cmd->work, tcm_loop_submission_work); @@ -320,6 +317,7 @@ static struct scsi_host_template tcm_loop_driver_template = { .dma_boundary = PAGE_SIZE - 1, .module = THIS_MODULE, .track_queue_depth = 1, + .cmd_size = sizeof(struct tcm_loop_cmd), }; static int tcm_loop_driver_probe(struct device *dev) @@ -580,7 +578,6 @@ static int tcm_loop_queue_data_or_status(const char *func, if ((se_cmd->se_cmd_flags & SCF_OVERFLOW_BIT) || (se_cmd->se_cmd_flags & SCF_UNDERFLOW_BIT)) scsi_set_resid(sc, se_cmd->residual_count); - sc->scsi_done(sc); return 0; } From patchwork Tue Feb 9 12:38:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B3DAC433DB for ; Tue, 9 Feb 2021 12:40:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 09C2764E50 for ; Tue, 9 Feb 2021 12:40:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230246AbhBIMkP (ORCPT ); Tue, 9 Feb 2021 07:40:15 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:42494 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230187AbhBIMjz (ORCPT ); Tue, 9 Feb 2021 07:39:55 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZS4I008786; Tue, 9 Feb 2021 12:39:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=cGlYDZEw8p3YrWD264K2nbK0C8Xegrsr3k+uwkZIpcA=; b=f+GqMoELucM1ftVYrL1ll4i1cXxS1TKQWxEKDBePCOG0bIckAl+73VvipbIlyq6XM7ba 0cEuyrG5mTCBNE3PTyx4txmIkvm/lExi+e0kAq4zMMe6kICOiehhe6L7a+XL8tT6kzUf Or1xMFQgxO65yAgNngYlek6GliBpxYv26ehV1WhG6eut2FuOLBYYya9ZEdf/T6WaciWA 7+jWH6fbUzEGQiJna+bd/9TaAfImMaB+uDtI69sbfYVl3ecoshUoI1SwvPsTKhQGGQaA LtDxpYmWgf5hm5rhxdvM4jCC/STSar7mxWVLdGmlALnCK7fVHfpTfxDLCjX71V51x5aQ Mw== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 36hk2kfajf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:03 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIO178533; Tue, 9 Feb 2021 12:39:03 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3030.oracle.com with ESMTP id 36j4pnn0cr-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PV7N/ra6vekW8MmtVhvdxUzv/0gqxHhOJBR2G5L++cfJNwPzmFEmbLvINFiKcofv8pi1q7D5RNJz5L7ZekGh3e5uQPESleYqEgqd7+/kKLPMLuWUuBkOjYZxH99AgLPVrx8J0ALdXV05PID7PhZm6SENvTe6Em6P0cD8wuKPkX6p5iptLyF+Pu0sajf1EKYc3RiN9COOpeaMiOfwiJmKZl4VVqXQQZD/1bHIiSRDsA2GIiihDzM2vokgygPEFBeG/ltkha51PFAGUjqH6CXmj7hmVv2XFmpV9dmt0r+dquuPt8tJAJJqxiViuAlmHIz8WvZJ717gTsMOKbDc84LWWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cGlYDZEw8p3YrWD264K2nbK0C8Xegrsr3k+uwkZIpcA=; b=IflDQc6MHdDYT+ajy5CvceHJL5kDtAig+ijKcoeyIhJvBIm25+r7O7ta9NWnW1stlvjXjGDEDmS5mo/6Qd3DbqabXUx6w0tPvaq9lK0PA3J242hYVa6TAiKP4dPsJCyt5QEjTpLIHzYDhH+3JQkg/7IAZbNCgS0K29h+7d+xK1xiZS+N7/y26DDQmPulPQcL0AA7ghKfFFs2TC0lQuIdNIvF6KvwUtUpgFnw4JN5cEwFK/qILpcYZ1/+W5rOyX4evC+BnjVJu+8F0J34IfpyEuiEYdAOPjvRaPe4gw9X+bd+i65mKsK5ssUwTGRykFrp808Zm6U3O3uOVddWRWDEKg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cGlYDZEw8p3YrWD264K2nbK0C8Xegrsr3k+uwkZIpcA=; b=Ix1eFSZoLrtm+qCJX4g3lhtTInqnKRuS6Pk8hfYZKUGapXti62FfqPgZHZfqwlAjk3Lv3ZiJDaniJbBsorc/wUDbLcaurVjkPQRqDZjxxvvjKaIwzRchx+KU9p+JFycr+quvyy71fyL/QADQ2U2ywPdn/Blrdud9jPOsCjJLqc0= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:01 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:01 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 06/13] tcm loop: use lio wq cmd submission helper Date: Tue, 9 Feb 2021 06:38:38 -0600 Message-Id: <20210209123845.4856-7-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:00 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8708b622-0164-4ded-b582-08d8ccf7ad8e X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1002; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0z2DkV6LxD8K+1ej4hwzOiAiuT1NAAM/6acFv/iKJjhOqtxS05dYHWkPvSau3jvOEC2KrsqpHBgbbe1jDRTdR6UQg9xPSpzqr1lazbECiepRKiMq5bQ1a2sjmPGQyPpUmMBmOQBLemupljX2Us0jmUMI8vcj4S8x46y59N85GL40mDRII9284smKlzxvSAWxwQQalgf65EZQQufymngXxiISw0b/I3lEGptI3xcek2DYkAUIxJMk/17wI3RQPgi2zCe0lU9S+43rWhnqqfPnkB4jGbwAtjgQvoXKAWPiMvaMXsIi457Rt3sS8Q+4dIeeF9j9UE8ZEvvqFGVOzMxmaoGGmbqNVI2mpbBDczcYkSYsEopeWp6QzJlpXAhN9O1+H4VHwIFgqaN64Rp87Qi0qs16eyzJQfTybef1wt8ySP034ZRzr5OVda4NojaaGfCh1o2lImoZiNtZrbPtvXucTkF8Sn8Ln3YFXEJZdrmZ7ltkwhlCjJcEGTo8B0FvBsWpFGp6crbbwr8NnsmOv3a8ge4zVwJS7YxQA7ViZLtxhhDh5wkTC7IcsoKAnioS2Y3LsMvikgM3V47nN0R3K1Z/OA== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: P+FjAD95DCQxxaNTwP8fKGMLR1cj0sQTdsu4lyBu4nY9hJKq6BMUc8DfM1IUp1M4jHDps3XdMH6XMrhatWxpKJBizblRQTcg6IfwI2tX0saQHbR8HQaLsuBiOEOFobTtzBAOyJKMdJoG4sEhEOPN9jGZ+c0qPAcYi+sc5jBd9EVgZglhWilC4RgWpTkB0sy50GxC6OK00dMSVob4nCkm+2g/fzUDNTRr+ES9YXKXYNdtzdMqQQNCngNunVfMhuHhKjEFRKjgEoCF3oqrbGxXe8TNcnrrEpryaGs3qlOQi0XDcyAp7vXHhYOpg6YdWssfRN8iDMur2YVoLsL9jI1f+rYoLzBWHuUxoN7qq4V85GI3xJKNt0xkhp9leKAkrTy7JNc4qauZKFwi2wt0cFxdkSrHNdMZRxkqTXCw9iE8e40I08DWzSuA932H4CQ2nUjA1abymdOec4wAIIHzZVRL+SS+ZXkqp+s8SfWEtXhtpvp9Xulgf/TBXj1G5hjORD2KhVKAf2Ic4Z1QkLK72EmFsqQ//m5yjRgW2aDts7fCMtdtCVWALDdZ5Thxm53ETRPeBY4y1puzIZkQK/YHrinNRHUG7mzBIeN8izLzYZ0p9tXd2l0KoNYkcgM9eaL6+vxOpB8R/THNQ99WBgyNCtJJ0BbtMM4+4roonpmP7L6bEWvU+pnf5nUywb1gauag2sJWN9ybO3VVrSD6ADmDDXyjd1RTmGBm83v0Zv4suTQMuTY2bCw4na46T2WXb8LMY7ZXPGk0DYRjTQIdE2Mxeug5ZkO3TAjfmmT35RSBber165E/bv/H8h5707q/+r17hQ4VcIW5ozngraFSymRhdxBcFQqUgbFLA0CueSaexpYAiuD8Hv1Wv3B2wIqm5h+K41LvKHPdAc1FLK+e7KhadRKHwGqAflEC1eq4326h0tz9cVsPaFw5ooFqi7gubW2DqsTGGxBimBYuGYVwIepkbvJeJF2E1eOYgqH4qXXbV0u4nin4Rib+rbLyfPwsh4pwuaQc/g8DyGtLmr3kRWUZMDfncMv6z/pxBwx08pZFcxpwY/rtlvKYzUcGz7Cxqy6cy83LOP8i9dO+nagPFcW1glQ73IGCM50VMdBQaD8jzZaPc4DxCIj7J3aIwmo6PB7BUnw0ot8cJ17jfzHxt4lgju5GC8ioaB4asTEKTfoya3ef8VRNTzqcWw8GWYAwTHMpBxk6Gpmv52DwdJyBUtq5zHiDK9pBbWzqZTNMGk37vIiopfSHbsaF4teGrSHcwTRQ7z6/s/2L7PfZHsWWfP/JuhoblhQJsadUjnTHnC586+T2ITSXq1pe1zFuCCJ8cTHpipr2 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8708b622-0164-4ded-b582-08d8ccf7ad8e X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:01.5393 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FGOA7iM4c/rq+oKNfwj0HRD/HZlB71xzlI+VQHGs3+xyeicG3T+/GGJNyavHm7DY+ia3XDqgQyCEb1oDWRYB32BguHaFAvkHsyLnhH3/uS4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Convert loop to use the lio wq cmd submission helper. Signed-off-by: Mike Christie Reported-by: kernel test robot --- drivers/target/loopback/tcm_loop.c | 34 +++++++++++++++++------------- drivers/target/loopback/tcm_loop.h | 1 - 2 files changed, 19 insertions(+), 16 deletions(-) diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c index 274826a2b0bd..3642f381067e 100644 --- a/drivers/target/loopback/tcm_loop.c +++ b/drivers/target/loopback/tcm_loop.c @@ -39,7 +39,6 @@ #define to_tcm_loop_hba(hba) container_of(hba, struct tcm_loop_hba, dev) -static struct workqueue_struct *tcm_loop_workqueue; static struct kmem_cache *tcm_loop_cmd_cache; static int tcm_loop_hba_no_cnt; @@ -106,10 +105,16 @@ static struct device_driver tcm_loop_driverfs = { */ static struct device *tcm_loop_primary; -static void tcm_loop_submission_work(struct work_struct *work) +static unsigned char *tcm_loop_get_cdb(struct se_cmd *se_cmd) +{ + struct tcm_loop_cmd *tl_cmd = container_of(se_cmd, struct tcm_loop_cmd, + tl_se_cmd); + + return tl_cmd->sc->cmnd; +} + +static void tcm_loop_target_queue_cmd(struct tcm_loop_cmd *tl_cmd) { - struct tcm_loop_cmd *tl_cmd = - container_of(work, struct tcm_loop_cmd, work); struct se_cmd *se_cmd = &tl_cmd->tl_se_cmd; struct scsi_cmnd *sc = tl_cmd->sc; struct tcm_loop_nexus *tl_nexus; @@ -155,7 +160,7 @@ static void tcm_loop_submission_work(struct work_struct *work) } se_cmd->tag = tl_cmd->sc_cmd_tag; - rc = target_submit_cmd_map_sgls(se_cmd, tl_nexus->se_sess, sc->cmnd, + rc = target_queue_cmd_submit(se_cmd, tl_nexus->se_sess, &tl_cmd->tl_sense_buf[0], tl_cmd->sc->device->lun, transfer_length, TCM_SIMPLE_TAG, sc->sc_data_direction, 0, @@ -179,6 +184,11 @@ static void tcm_loop_submission_work(struct work_struct *work) static int tcm_loop_queuecommand(struct Scsi_Host *sh, struct scsi_cmnd *sc) { struct tcm_loop_cmd *tl_cmd = scsi_cmd_priv(sc); + struct tcm_loop_hba *tl_hba; + struct tcm_loop_tpg *tl_tpg; + + tl_hba = *(struct tcm_loop_hba **)shost_priv(sc->device->host); + tl_tpg = &tl_hba->tl_hba_tpgs[sc->device->id]; pr_debug("%s() %d:%d:%d:%llu got CDB: 0x%02x scsi_buf_len: %u\n", __func__, sc->device->host->host_no, sc->device->id, @@ -188,8 +198,8 @@ static int tcm_loop_queuecommand(struct Scsi_Host *sh, struct scsi_cmnd *sc) memset(tl_cmd, 0, sizeof(*tl_cmd)); tl_cmd->sc = sc; tl_cmd->sc_cmd_tag = sc->request->tag; - INIT_WORK(&tl_cmd->work, tcm_loop_submission_work); - queue_work(tcm_loop_workqueue, &tl_cmd->work); + + tcm_loop_target_queue_cmd(tl_cmd); return 0; } @@ -1146,6 +1156,7 @@ static const struct target_core_fabric_ops loop_ops = { .queue_status = tcm_loop_queue_status, .queue_tm_rsp = tcm_loop_queue_tm_rsp, .aborted_task = tcm_loop_aborted_task, + .get_cdb = tcm_loop_get_cdb, .fabric_make_wwn = tcm_loop_make_scsi_hba, .fabric_drop_wwn = tcm_loop_drop_scsi_hba, .fabric_make_tpg = tcm_loop_make_naa_tpg, @@ -1161,17 +1172,13 @@ static int __init tcm_loop_fabric_init(void) { int ret = -ENOMEM; - tcm_loop_workqueue = alloc_workqueue("tcm_loop", 0, 0); - if (!tcm_loop_workqueue) - goto out; - tcm_loop_cmd_cache = kmem_cache_create("tcm_loop_cmd_cache", sizeof(struct tcm_loop_cmd), __alignof__(struct tcm_loop_cmd), 0, NULL); if (!tcm_loop_cmd_cache) { pr_debug("kmem_cache_create() for tcm_loop_cmd_cache failed\n"); - goto out_destroy_workqueue; + goto out; } ret = tcm_loop_alloc_core_bus(); @@ -1188,8 +1195,6 @@ static int __init tcm_loop_fabric_init(void) tcm_loop_release_core_bus(); out_destroy_cache: kmem_cache_destroy(tcm_loop_cmd_cache); -out_destroy_workqueue: - destroy_workqueue(tcm_loop_workqueue); out: return ret; } @@ -1199,7 +1204,6 @@ static void __exit tcm_loop_fabric_exit(void) target_unregister_template(&loop_ops); tcm_loop_release_core_bus(); kmem_cache_destroy(tcm_loop_cmd_cache); - destroy_workqueue(tcm_loop_workqueue); } MODULE_DESCRIPTION("TCM loopback virtual Linux/SCSI fabric module"); diff --git a/drivers/target/loopback/tcm_loop.h b/drivers/target/loopback/tcm_loop.h index d3110909a213..437663b3905c 100644 --- a/drivers/target/loopback/tcm_loop.h +++ b/drivers/target/loopback/tcm_loop.h @@ -16,7 +16,6 @@ struct tcm_loop_cmd { struct scsi_cmnd *sc; /* The TCM I/O descriptor that is accessed via container_of() */ struct se_cmd tl_se_cmd; - struct work_struct work; struct completion tmr_done; /* Sense buffer that will be mapped into outgoing status */ unsigned char tl_sense_buf[TRANSPORT_SENSE_BUFFER]; From patchwork Tue Feb 9 12:38:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CC7CC433E6 for ; Tue, 9 Feb 2021 12:40:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EDC3F64E88 for ; Tue, 9 Feb 2021 12:40:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229988AbhBIMkF (ORCPT ); Tue, 9 Feb 2021 07:40:05 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:42512 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230192AbhBIMjw (ORCPT ); Tue, 9 Feb 2021 07:39:52 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZS5m008736; Tue, 9 Feb 2021 12:39:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=ZrhSxDpGiuoIC3wgDeHHH1SB/vn9aAiRH+pEFj/pm/M=; b=qPjmaWUS4XSq4bgQZ+uejXCyZb3qp3Axslx1vXmVctM8ReKoSCYirrvpDuOTN5R5C1kR 4RzsKT4o58H4HxS3tpbHhKpja+FnNedhiG4IpMqNbUlq+OauoOU9okn6RIVBHRwHtrWE r7YjuA28LxLgoAtuV63dBKl2WmczN3691sKZ30HhRihGHxsXCv+9UkdwCm6Q1yvks09m CbSdJ3LSau3rF9Y1CCzYzaLVkiJTsnMFfrHAW51V36DieTLT2xmTWulkwo3hUTzSKkuX VXf4qv2otsrEeod/xOzZnyxWuBvYSNypkLD3P3zATK5fWtM6lNkLMmQ2Tcq2m7oaS6lV +A== Received: from aserp3030.oracle.com (aserp3030.oracle.com [141.146.126.71]) by aserp2120.oracle.com with ESMTP id 36hk2kfajh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:04 +0000 Received: from pps.filterd (aserp3030.oracle.com [127.0.0.1]) by aserp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaDIP178533; Tue, 9 Feb 2021 12:39:04 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2108.outbound.protection.outlook.com [104.47.58.108]) by aserp3030.oracle.com with ESMTP id 36j4pnn0cr-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:04 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=g0a9U37Xbz0sqI5fBhiRVpxbE+sdfzA5fhxtcWrzHssCZESxtgemOndzVlyLNFtG9FTbDfCQiY4TSmP+wDOWbEgnEERdyCSeMTbADPM4WjKj+PJUneNX+8Ln89JgDChhvE/gzHBfmUTy6j/HpiVk4NPovftONoxyydQzG25o1Vpwla5FcqT/tkHm4eyy7a+WNAUmJaapq4TSkDyMMLlRQUjAmjfiFdP6RD4wnEKxX6i6IUcRhQ/VoDuphflXH9WPm4FdkhXR0Xh8Am9FMtvqGmgADdPkyqiXNZizFcm5nIwRVNZD8MioRZiGiwTiKgZGELxKMP6z+wGI+IsO4/dvsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZrhSxDpGiuoIC3wgDeHHH1SB/vn9aAiRH+pEFj/pm/M=; b=biCjObRhPmOxByk3M43T8ZeEg/BmpA1DWYUBGaXRlFzNH1pZT1RM4KZUgEieWgrz98UGx6JPPs0iC6WQ73QOxCk2IsKRFCPxdgFzL0t5UD24VGsU08ZN+yIqfN9PgyjV75MKfetDCSQOqsHlnDdpFm5WbDTXfB941N/0Da3srDK+obZEPkumH2O2F4DPQ3S42rpQoAwHKfW4f0rNTQ2Z5z7XK+wLFu+utzk7/610ABG91po6KmND6j4RLcGFbUhh4DsUeILhCI45B5yxAmNuEuxZ4P5BOzZDvXECfbc8y9z7w/jMTuepaz/752e87Z7B2xWuLIK5zTuVSXqjBnysCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZrhSxDpGiuoIC3wgDeHHH1SB/vn9aAiRH+pEFj/pm/M=; b=rHQWUOhH1r70tGWjr5RiqprUMZgJtMUs30C7C4Xx4y6MlukVCjmPoWBUMczSnJxsRiPbFA0FBiLyilLIeKFpesYraLjF1H22KRYQjUlc+HQEYOzilBU0QXjFGx08M6AJ2WNkkZECVxXzEWWdhTKfk2fulIpHY4Eoj+sk6hFk/xU= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:02 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:02 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie , Chaitanya Kulkarni Subject: [PATCH 07/13] target: cleanup cmd flag bits Date: Tue, 9 Feb 2021 06:38:39 -0600 Message-Id: <20210209123845.4856-8-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:01 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f790b8bb-2390-409b-9777-08d8ccf7ae4a X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:9508; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SWpIDagmsVtyI2CnigSVvHK7m3mYStX+c0SncX+Zgdie6thlAoREY93udVedMYIlbtmdk/nBe6OVz1zs9B464amdFByljkZgMPsDnwIqxRgMiYwIDN2j1Pt/lZ4lEWzA4GzlXpqUk2r20R/vXxNW1+KwC/amzz8j/fD+IkdsAUQW5akLDlM8nW4NtOnBl/wtXYxZT9Yj+YCWspCvgBlSaYCI0qQ7JxxBj3iXFoVnd4bAeGA07ewFp1PuY+J4yX5eBd52ThSkpWhtIjPfusVqCfJy8wzBJvvEBaU0ky9JS5xE+337A0uB96iWTd+EXgKtdCc30GPnPjZmHJGOdpIiTlnUGT0sjf6HUaNKX7ZNopCa/vC3xh5L7i0i5tNnOTXagD8oL05J5CphcX0SZJBT1FHybXRC5i/o9NGsSqGaGho4zQq+3R+IMovTMuHYZuUi8s7E5NZvZ2sPLOc+4N6XawiW5VmgY8juI4w/5kU0Ej6/3IoIQhfsKrMUWOyxkVG0MXPrvTLEh1hO2OgaWNuuYEkGruOE4H2jGIXmjhxUvC+Q4+3pDZdlfZld0cjb5rccPNbvzIV4M0qcBYrJjCMa2g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(54906003)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: plCyI+7naSniA/tvxzH36cBDq0yW0GN8weOQKi+G3SBqd+YnDhkXtAdlbRnVHOrb4IkB+7HBrSvUf8+HUYF+nmsITrzJLKnjrDdKoZNgiVPBIm6gGnk22ePaCcMlUBmYGZB6jz8jtC7gVVlDoHxypzcRjfk10FCV6/CpgiwucK+5viuu9SvLjaCkowz5Lc7GgPVib/St7KspDNMks3QgSeZ06z9yXBlIbTAAbE0xYvwUN8z73PJ8WXeejn9iJ/peKkB7TOytPAsf4ebbBJIDPCR91qmf7Rix0+zCcACaQ9+J/PZd+Zo/NI5JhqomO5cqKCRx19sqy4JGPcVhAxNZXG0hXNVcQXEZUk1WmcPzd8GLNYe60jSp0torxWrU+0o1cQHZvrjrrPNN69ydat+aYFCZhkY0p+M09ZCAMboARe8ZoLmHzQvqakIa8jpwWq5xwRr3a7M/Frxx0r091OsPIMinlQg4dfsTXwV++laPeUCQhIVmhZngh6jo1vfXdH/a9DPVIBrHm9E9VrqkVUBSgMzXXKoEVwfBR2LYM5bZoFy0aKmaiCkx7WrDTKOeg8PbxvPpz0MD3qVm7g0p8uuluce/1gpztn025JZgABXJu+8wYurCAA4r/65XKxC5Yk+ylkCD38bolHvmOw1dW7RMr0Bq4RLyr7STJ2lIh/e47zDHiGHRSciajnCaVzC2aOoqlP/lLfCgsCZSBqXR1Lf0WYYC0+AZ6EXaFXJmv+und2O+ITrffUzkM4sBhUQcz1eV7zCzDqnOjq6C3kYgC2enwKSBaBxxXWNPA0RomqQuyZVfEMRXIgGXOWL+1NK67lb6PjJbIQ1Ndaq7ybp9Xh39Ms6SWRmFsdNKeSAzhCrfMt/ywD9KSOR8zB7L8QgAXru0SQ/HmE2HfeyXgpYwb+vChWeAUULKkf3h/a5oJwGr8QJaf5m2kajB1exFZNv0njrWfE42rJzVrwbz7I0VqTw5FvroRkGfoNdTaAgS2d/LCGVaQTdzoUzGtErvI3yvEoNHelUirUDM5C0DTe6ry7SliHNJLj8jCRNE+Cg4wy0R4uE5ImetaPYgsGnQjyZc5EacrF0v2sUerMuyGH2HC2zhOaHlYCeMvzESr2wyvWfR0/0LBGQ6VZACZjIOgeFjGYuRTUOa+nogdXVswK+5KNdAaS6dtyVx5cPbOoHUAlTBcgH88Vnn4vo1DxFW4Jr5lX8+neP2ti+xiOEyXlJUYcbdUQJgGjfFmvwnrT41Pu1TspehngnbLVyN/ieKhA+Q9ddBtOtVvDjQ6Ki9xr7DiCcWuh0iRxOhPOYNisqMAOne71iORGjLn6LHJB1wL/4hy3U6 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f790b8bb-2390-409b-9777-08d8ccf7ae4a X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:02.7736 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8c7w2qzZWB4EI0SIp2Y+SDru5m8gQLNlgqeZuZGU4Ta7lcYOFupGn+72wwcijp64UUSTzT+uGyFYsf1pOuBmjKloqpaHrqFlTKys6ZC8Zl4= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 bulkscore=0 phishscore=0 spamscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We have a couple holes in the cmd flags definitions. This cleans up the definitions to fix that and make it easier to read. Signed-off-by: Mike Christie Reviewed-by: Chaitanya Kulkarni Reviewed-by: Christoph Hellwig --- include/target/target_core_base.h | 38 +++++++++++++++---------------- 1 file changed, 19 insertions(+), 19 deletions(-) diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 815de4c97230..5e6703ca102d 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -127,25 +127,25 @@ enum transport_state_table { /* Used for struct se_cmd->se_cmd_flags */ enum se_cmd_flags_table { - SCF_SUPPORTED_SAM_OPCODE = 0x00000001, - SCF_TRANSPORT_TASK_SENSE = 0x00000002, - SCF_EMULATED_TASK_SENSE = 0x00000004, - SCF_SCSI_DATA_CDB = 0x00000008, - SCF_SCSI_TMR_CDB = 0x00000010, - SCF_FUA = 0x00000080, - SCF_SE_LUN_CMD = 0x00000100, - SCF_BIDI = 0x00000400, - SCF_SENT_CHECK_CONDITION = 0x00000800, - SCF_OVERFLOW_BIT = 0x00001000, - SCF_UNDERFLOW_BIT = 0x00002000, - SCF_ALUA_NON_OPTIMIZED = 0x00008000, - SCF_PASSTHROUGH_SG_TO_MEM_NOALLOC = 0x00020000, - SCF_COMPARE_AND_WRITE = 0x00080000, - SCF_PASSTHROUGH_PROT_SG_TO_MEM_NOALLOC = 0x00200000, - SCF_ACK_KREF = 0x00400000, - SCF_USE_CPUID = 0x00800000, - SCF_TASK_ATTR_SET = 0x01000000, - SCF_TREAT_READ_AS_NORMAL = 0x02000000, + SCF_SUPPORTED_SAM_OPCODE = (1 << 0), + SCF_TRANSPORT_TASK_SENSE = (1 << 1), + SCF_EMULATED_TASK_SENSE = (1 << 2), + SCF_SCSI_DATA_CDB = (1 << 3), + SCF_SCSI_TMR_CDB = (1 << 4), + SCF_FUA = (1 << 5), + SCF_SE_LUN_CMD = (1 << 6), + SCF_BIDI = (1 << 7), + SCF_SENT_CHECK_CONDITION = (1 << 8), + SCF_OVERFLOW_BIT = (1 << 9), + SCF_UNDERFLOW_BIT = (1 << 10), + SCF_ALUA_NON_OPTIMIZED = (1 << 11), + SCF_PASSTHROUGH_SG_TO_MEM_NOALLOC = (1 << 12), + SCF_COMPARE_AND_WRITE = (1 << 13), + SCF_PASSTHROUGH_PROT_SG_TO_MEM_NOALLOC = (1 << 14), + SCF_ACK_KREF = (1 << 15), + SCF_USE_CPUID = (1 << 16), + SCF_TASK_ATTR_SET = (1 << 17), + SCF_TREAT_READ_AS_NORMAL = (1 << 18), }; /* From patchwork Tue Feb 9 12:38:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077953 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CD35C433DB for ; Tue, 9 Feb 2021 12:40:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 013CE64E50 for ; Tue, 9 Feb 2021 12:40:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230242AbhBIMkK (ORCPT ); Tue, 9 Feb 2021 07:40:10 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:42542 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230196AbhBIMjz (ORCPT ); Tue, 9 Feb 2021 07:39:55 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CaULe009428; Tue, 9 Feb 2021 12:39:06 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=Ca/rM6cNPFhPKHaYbHHm8zkO5GUgL/b3cEo0tBUZF2w=; b=lEX2vem6u6sj8bbTz/QQ8GAhScC/x4wlW36p6czYC+97NsSM+W9rHHvwqwCbwNkDuqzx 9Sbw2LSu/Yz4V9zv3beTnrxtMxQvogAUItF2nqo4ymXNbWp226VzxKrCR1NngGdQoeoL Lxcy0C56JGj8+dvsHIwy7XQqf4tEGkgwKoaYTW8h4WmSCcuwS4XFawIsOvK8KUquglkd eSQ1sdVmd9x5qAe9J/xDIZSt8M/w8zeyxhEtLHKTnDvbwiMSEodbyi7rRLQKe1ehOoNE DxhmoozpgDKO/Qt2qQe5Ycz8a5GM8JmDktApsIFnCFPonkKSJq1ixqXj05qd+E4wS+GM RA== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by aserp2120.oracle.com with ESMTP id 36hk2kfajn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:06 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZeCf029740; Tue, 9 Feb 2021 12:39:05 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2107.outbound.protection.outlook.com [104.47.58.107]) by aserp3020.oracle.com with ESMTP id 36j5115bft-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:05 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GMCxnaPpGNYQToV45zbgsJ5T12GENcA16QhJ9y3A4sSsQuyTChwjA8OcmsPWHQcgsKAcUoFi6NVzhrQzVfXkT2LAohHM8f17UpbKVk52e/2Rb1/heRg1izyfsh07ibZarMvEG4zCUiWdMUlCdJaoIzM7g+ASM9J0/Qmj3YeCfq9HswSmhpkpENimCEQ0sCnQL/gvnF7E6LB9gsQ1U5QOc7DGZbHgIkyJv4BGv+3ZieBJWc/fL/9PFVDBYB2fF015wrWdb048zmTHfJftYK4wWVQRKyZE/PTwCjmjXslSv3GYltY0nR9yZtDcOYdtXKx8KD2iOocf0Wn5Psl25XP9bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ca/rM6cNPFhPKHaYbHHm8zkO5GUgL/b3cEo0tBUZF2w=; b=JKefdRsCIAHpCc6xucpY1DpXBpEJLqXOSWPsGp3Ly4WuZq12dXVQA9MlP3F1SHmgM0HyK4cp0JWLtgpIT6FTByrPwEGeLHzgImSlmq5hUwNHd1fmIwZrLCRedMjdy0cA2LLxRaGAktDGZbgIWv0Y1bmnHVG3JlJRI4IixcIsY2otRByeYXt+SWucjY5l65Fec/R/AGp+ks0dLzs221Nbpn7EpfaJopFRpFNpFR2O23tcpd7IIN+dHnuRB3a89DWpWvmy2jUfUcL3TlBw7SqDyH2q2ln3m9oFTroAvkPgd+nAhgtxcGQ4NK7/0uGIccxqZi3eqZ2B17i399Q2DZhrmw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Ca/rM6cNPFhPKHaYbHHm8zkO5GUgL/b3cEo0tBUZF2w=; b=Z4hPe/h9nQmoSr3hgLou9T0BLYPUbUK/mToZz0yvIevL4tGD1Jm5NeBcuwv8Vn6s0FcyZT1he8w3091kxAbaGUDvVgA99xtmQ3D9dN5iuVcFIRIYLnEkzIDJWiQijpU2cM50g6Mg5a9RYyWElibx/CQzoqEDYKbjcx318D7Oi7E= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:04 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:03 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 08/13] target: fix backend plugging Date: Tue, 9 Feb 2021 06:38:40 -0600 Message-Id: <20210209123845.4856-9-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:02 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 30b11baf-076f-49e3-577a-08d8ccf7aefb X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8JMuQcoXWuniIsOT1S7iuia9omd8vI6aUuNKcm2otOhm8oVjG7du4N1gN01mXnzddxWQZfnqy68eLMz1ZjzdfkvfBzO+TT4yw8ft6+pBFTB9XPz7OrOxHjAQDqX3nRUxJr2y1Di2JQRJdYZqe8eLkRdyBLEtnMX2200a2SF3plDR+q8Y/yf/AoXU/EH4igG3Nd9+pZXe4BIwEeVJtLyjufmx5FQW1F7evAKnJCcpmKVYKIf8L+9p+nQLAMyh2GvU4pEUEbuPRS+N5FQ2Itdy0xV4t8MLzvW9wUaPbwfCK+gX2GKI/ZbLbFXeTXb3JBkYNCULDLcj2R9fZk56Ob91cYYMdUe0GUxnbyuQ+gzIirnpakEKt6Q7IYX5Yh+nQP0j/OJ1p9yvV+kcq99ZaCJRTdYondD4SgjtR9APRmhgZnLQmi9Nl5DrkghmjOwwCZw/T/njAomzgsuE76/5XkfjLVfelknnvcdZnolgiDER2UPXUueMHGhuI0nuWHNtisVxLBFhZUge+RbvfylL89gu0TnCXg6WFozs/JWpJ6cr1h+MIHrLkPKjDqndDA7I/ojP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: /aeWlDcZ8vH89v8kTjrMo0YgO+M8phCg3DbYZ5I70o565MLVR9BMrRxfazJb37ppxaceDxYc/B6ChiUm3DnSCrW+//Mb3IbL+w4ip7JaPtX76+HY6C/xx4IB1E1CqG9nINJr0lKjceghn/SU+XHzT6Cgxs5j+XSjteA3zxaOZ7G+TN5lMnRm4rYsQLDXL2KAsBoJO5Jp8ioujbJb2b4mdhMR0KtoJSpFF7JSvkGnrti5/5sX5YsbjahM9QhSsxiHZCbYrDwXAHONwkpztNYkTgchf4SW7tQq2foQtXIChGsRo0o7qHM5bsEH6tJ9jVtTu61hahRAJFUPnwha+9c+BGe38Lk6ak70k7JXHPhlJ5V//SjeKOECMWRa+fMvUUTs7YaM9rrUqDJw3f7/zTF8LYgiOxl3C9n8kkImOHliLgVMPnB6eA0KZBEkTxspvVxkAFLV7Z/o70iGCqp8UohyFKc4cby8PuSG0F+pWOLewXRodyDywj3STglxYr6GcWsDSyjmgW4rkQMSbaUtmvKyligyFvyzPZhVqwNCagXs/0pYDGRQwPzE0LiLOPvn6n4E9Lv/YHwTOljDkcY1hS1Gsp90jp1LcMmAuQceAfvdDAOR3A6tYN6stQfu31HzbjVCJWRbFWgcRjadpVtF+adVvmBbfjVqUbkjCa3arcjW55mJMsT7Jtstwk10O8Srzy1UvSmW+2swnt8ZI3HX4JL7we1ltkPEohjjiiS0h/kAQbG1capuRKdpLK+IYNuYVrGnMk8o++ESwHvSfmMrIxtr5pfzBX668AKqOkvDrvAAI0k+xX/R8BVk3V2W6U3aCFxiv0oyCmtxnXYa2SzmMxOoWHmeG1OdZhL2HyDU/9QRUtBP5TwAHSf/naCb/OjW1sWKU+NBpYpzX/TBe4PMt6QDt3mh8Pyor/dMcNzpEbOFGGC5jJFCrkPe7kr6NgIw5nQNqNNUC8ODYZvoSXTq/v87OrSHpFm8eYgsa466lkleK4weMgrj05WjAAQw8irOXeM0hsnzsU0VwNdzKX1oX7jlJlihQoIS/xez/NXO9Avbld5qY+eycT2nucz4BOL7II1DxkP2UOELZLZaRdDUmxC3avSvd3oGiCVPldg0kmn1heTCLKbklAaz7dkljQ2zQRBrYc7Mf9Sd4A3EhFtPnU+aVzkpIc5rSsZlZXoc+RciCvmjLi0n+Mcgt/lbjdVp0rbyOmWS/+rLpdEOW3CHitlCxWcRaPih96KWy7R/QZA2N6XQOKBImqGFyrCvFVGOqupqCyAj/7CRDgLldf7JB78UmW9o36bfJrQyomD1AIay9CGZ5P2A7gB/lSIjtDKV+Grf X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30b11baf-076f-49e3-577a-08d8ccf7aefb X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:03.9090 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XLjyBk2jusrm0s1cThETpzA8foz+19VTqDGRxGXGKzrGjzcf9AH+BI5W+cqlLCJB8rJWHOE7HlS/Tk42aKKtIKUlD0ehnfp0p/mTG0ZSBrg= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 bulkscore=0 adultscore=0 mlxlogscore=999 phishscore=0 spamscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org target_core_iblock is plugging and unplugging on every command and this is causing perf issues for drivers that prefer batched cmds. With the last patches we can now take multiple cmds from a fabric driver queue and then pass them down the backend drivers in a batch. This patch adds this support by adding 2 callouts to the backend for plugging and unplugging the device. The next 2 patches add support for iblock and tcmu device plugging. Signed-off-by: Mike Christie --- drivers/target/target_core_transport.c | 40 ++++++++++++++++++++++++++ include/target/target_core_backend.h | 2 ++ include/target/target_core_base.h | 4 +++ 3 files changed, 46 insertions(+) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 46e20c42119e..a90f20222894 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -1835,10 +1835,42 @@ int target_submit_cmd(struct se_cmd *se_cmd, struct se_session *se_sess, } EXPORT_SYMBOL(target_submit_cmd); + +static struct se_dev_plug *target_plug_device(struct se_device *se_dev) +{ + struct se_dev_plug *se_plug; + + if (!se_dev->transport->plug_device) + return NULL; + + se_plug = se_dev->transport->plug_device(se_dev); + if (!se_plug) + return NULL; + + se_plug->se_dev = se_dev; + /* + * We have a ref to the lun at this point, but the cmds could + * complete before we unplug, so grab a ref to the se_device so we + * can call back into the backend. + */ + config_group_get(&se_dev->dev_group); + return se_plug; +} + +static void target_unplug_device(struct se_dev_plug *se_plug) +{ + struct se_device *se_dev = se_plug->se_dev; + + se_dev->transport->unplug_device(se_plug); + config_group_put(&se_dev->dev_group); +} + void target_queued_submit_work(struct work_struct *work) { struct se_cmd_queue *sq = container_of(work, struct se_cmd_queue, work); struct se_cmd *se_cmd, *next_cmd; + struct se_dev_plug *se_plug = NULL; + struct se_device *se_dev = NULL; struct llist_node *cmd_list; unsigned char *cdb; @@ -1849,9 +1881,17 @@ void target_queued_submit_work(struct work_struct *work) cmd_list = llist_reverse_order(cmd_list); llist_for_each_entry_safe(se_cmd, next_cmd, cmd_list, se_cmd_list) { + if (!se_dev) { + se_dev = se_cmd->se_dev; + se_plug = target_plug_device(se_dev); + } + cdb = se_cmd->se_tfo->get_cdb(se_cmd); target_submit(se_cmd, cdb); } + + if (se_plug) + target_unplug_device(se_plug); } static void target_queue_cmd_work(struct se_cmd_queue *q, struct se_cmd *se_cmd, diff --git a/include/target/target_core_backend.h b/include/target/target_core_backend.h index 6336780d83a7..aa5f83e55501 100644 --- a/include/target/target_core_backend.h +++ b/include/target/target_core_backend.h @@ -34,6 +34,8 @@ struct target_backend_ops { int (*configure_device)(struct se_device *); void (*destroy_device)(struct se_device *); void (*free_device)(struct se_device *device); + struct se_dev_plug *(*plug_device)(struct se_device *se_dev); + void (*unplug_device)(struct se_dev_plug *se_plug); ssize_t (*set_configfs_dev_params)(struct se_device *, const char *, ssize_t); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 5e6703ca102d..b8e0a3250bd0 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -769,6 +769,10 @@ struct se_cmd_queue { struct work_struct work; }; +struct se_dev_plug { + struct se_device *se_dev; +}; + struct se_device_queue { struct list_head state_list; spinlock_t lock; From patchwork Tue Feb 9 12:38:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 017EFC433E0 for ; Tue, 9 Feb 2021 12:40:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B892F64E50 for ; Tue, 9 Feb 2021 12:40:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230239AbhBIMkH (ORCPT ); Tue, 9 Feb 2021 07:40:07 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:43548 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230197AbhBIMjy (ORCPT ); Tue, 9 Feb 2021 07:39:54 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CYkxq038321; Tue, 9 Feb 2021 12:39:07 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=KclUObLCUqKE/XMsQZmNavMXW0k/hkril4g11ked4Ls=; b=eHOnPLs0nHFN/+NGFDVB/oIcrKAHMfc42o3FmKUPRzHp2veKIa3DwOLAcUvPPIpJGjR4 +HfhP4IQKrhHuaf6X3CqWuB8pCwsTHy691OeS2fIQVu7wAcPoHLPuGkfo9D4l8ylavLd WK6cfxOGL7ACJ7J+Q7ChzMlWtjAi9C4k92T6N829PrFZenk3XnWRJgDbfYHO490lNYu2 R4A4QgdOtvTnRdyjmhiV6kp08basfjZa2B/JTm6rQviIsez1YvUpw5IwpnlrhhLqbrRY tUixTdT2agDkk03kyjsbXUoiY+HzIw06dqPMS0UAPh7+rbSzywyVdWl3zbIHXNJ1Af4q +A== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2130.oracle.com with ESMTP id 36hjhqqagn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:07 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZeCg029740; Tue, 9 Feb 2021 12:39:06 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2107.outbound.protection.outlook.com [104.47.58.107]) by aserp3020.oracle.com with ESMTP id 36j5115bft-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J0dkMDl+C1Rfu+M2tVd7ssbgWZ/5diJjDlGwUya8vV7kVSu9QtPT+7aJQ2rirADYjnWhOtamIpEuf+mamvZukiFg+kZQ5n9NsC91ygWbCyi2z+y1iK6gbtn/huBmwPapO0DmxQfSYns6yYOMJ2Z0xnVaKfmh44V4+oMojqAsmemdcaNJKQHWFfxLOs/i6a0CLuaEnYJK90HtTegj5W9wHIE/Ziv2zeGTcgHvPcJfhw/CbAf8rXxokROqGLJhyrMTORPbMEdb7OsJBRdN54QuV3aO7XqInYOrmCi+YKqCrnCmEAfzAYfndzr/eFbCNi0LBFCIy2by/Sv2rEbekeDCnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KclUObLCUqKE/XMsQZmNavMXW0k/hkril4g11ked4Ls=; b=oX0ZPfAz8N8N2aupyofM8PG2yqrU1yjrgdgfQ/wx36N1hx9Hz3Sk4S9jXsc0l8Tp6jJdPbhmuDtJxJvSLJbumbaEcx5zYcU9Zn50s/MPpPs+AKk5ECR0bBwBs0pYpuEK28NVMTGqzKDKM+mcWjcA1Y1Llz2FDNka7g+RvPlEoisDu4eSLe/UdKH+k9w0CxDozX5Uf6FGnv5PORNrVbocItrvGPw0YcUMs4zVX4/Np0Vb/Vneok/tqSAk7F9j57gl0A27wpIkOzSEX6vTOr8/et+hTdVDV5Q7WAORgHqfuqAZ21vhs3rnlg+q8YYJ5wxpZKRoIU/F5aUPWgfsbnW9Hg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KclUObLCUqKE/XMsQZmNavMXW0k/hkril4g11ked4Ls=; b=v8qQMPOOXwoLodO5RYhwxUVRGB9WDNE+j1FWi2BFB44eQfRiHohEd7EqxWaJ9/XfDzIV/XGjpDoo2Nb06H915rjkkUvUD3Z5SZrfKDHHh88CHSes/Fw0pWFFwxecioSmUfLqAIiHrkUO2o4EN0+1nsVL4Esql+zMhEzmFvENyHA= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:05 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:05 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 09/13] target iblock: add backend plug/unplug callouts Date: Tue, 9 Feb 2021 06:38:41 -0600 Message-Id: <20210209123845.4856-10-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:04 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 2b6e7608-6456-42f7-3e84-08d8ccf7afac X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3276; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /msOQ3hP3wgOqfArpBkPpLNeQRjnav4TpVRDoRvrvFo+8bXJ8rww7LzL3CnW0fFGIkNB/8pwy1lNFJT1LK0wGpAmvVFLUbGB7VXTZEH18fp9ZA+tvFpoMfIZ1wWlKoDyCXNFcGHsGZRq1RihZayvsjXgas31URXHqm+/4jyAlmxM47Kb431Lq4ibk3LSlU7uIYZFZIVqQBa373BL05v6YT6gY8l34KQH9BevI/h3hudEhz5Pu8IQIApKNS8t4gwo6FeyqyXt0rb/Br34bhHyHHouJSwwS0gb3I2qk2PY0uAdw+URwsSOq6UpBKSE6CWgvjEcvRu20vANmh0fFO6UyfxV2suFCA//lAKQN9X6uxd7liUYx3h9xk2oQxX4W2UDEzsVLtq8mwpwXDo97zveZBpQNoPRJOy5hirExbNXDfLia/SOWF7y/ajIvRIku78FMZjUeYQ+gBgmyV2AUwD7eVzIl4qF94Ar5yCgk859XoGNbcGMPygkzq3XbuNx/fsM0Rfg/xWzaPoPMprMXVDy+9REyTRH2Mp50wVbEsmcW64lHQ1rN6kjUrGA3HhKS+jPwuElH6CtbWb/OuP7gVTv4Q== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4aZZDE3kHZlzvOOpuzjRgk58d15d9tB90xvH+nXZxghKycVuf0Alo7GLygKWXd/OYSTZ/jtW2FwDEUzzfa0AsEcScHARUJPNFN53xLx+GciV76f++keZwWa7sGaRaoRZLeUgLo8aluLHvpvAgcAiMf4YIkz3IGMux7pIW9ngXr9JRFRSaiWqM2QrkwCs1ZrX1SlX7XbRIEd20eurECK1QqvVJ533ao2yClYvZmTKQN82xnqLVJFyswrpSF2l0sR2b4ZWXrPg740uYMHEvUl5n9sMqj9m/B5poK9LH910VqZ+BMklgzz1WzBnUePvkRe6hZ+gMWtb/MckZ4emrJ2uW2ik8vVwoWu7Qv0jhbi7MF43LLDr6QIXw2hEjZhFJRJX3Wjuxy6aXCNdq2YUuo55HQRNgLUlmro1OZkJSN/QWAEWEx7ErFmS+0xx4jJzjcLquVIDW0z9ESN/GqVr3I3ANjQ75R4fwEGUt18aBWJuj/8oIKCBQA6dfftPQ3E2Zx6P/bSyyVwtNoscri/kDtpU7hBZEUDSZD/vJ8ebS0W9dps8sU41YDCC9lJOOkEpVy2hpwGs2p6AxSt7eO3YiSnRQ3gbkeSZg2iKvn0QxkvAqwpR8qyNDBcqkZZLCS5i+afjwZJpl+dUHMME79l3n6gv8Mz+JddY20vljxgjzAVTG6fd7GA886MpkWglFrPrLFIPMXFQRxzQwm4lxYL/1Y/38o+ncgkiclokkplt5PJTptdBUbxhy91xXjtkTD2e6GetDiHBdpagvMlIwrf/xMH52//D+w2iF1ZYR60sm5fNQDV5bilUc676OIYu0sdfiIlzQE0msCEFXvhm+ki3RG05CiKe1w2xuMymOQxCBiaT0fvQYn19v9UMY35PmYWEtpzJQDN+4peUEBnKe7VXoYkSt+HCIFxh5F63qATSyqqo+xb+JOb3HZDZd1+Fjo/q7T7MHrniEi5Oq4N4v8fzW27bL5qk3hcVJyJ+I5z3T9R1VELvJ/HPeENyoy1ol4EAS1nlHWxNRmwLagf5IAgi8mUZPUSlzjZECeIawxv8p66GqHyMiaFxl60NZsmUk6B7PkjO14vQiWOdppWNZBmjBGZncge6TVhzhPKxpusFIkQ7VnSEwJAPkt5ysdvev99mtj/XjVtJ6IqH2UlpGUUDgwfWjRpqJQtKGp/ie3eklxnNaWGFKZ8w8R88M0lBu2Mz7DU6tHCWZU813Fw7yX4Ox2H2ORi9/+9zXsKZr3+Y4WDmlxuLTinMp5AkaLiY0lePxlnB2hYy9p7j84/jtjYllMCGVe88ooBR38whUfTnqZVwp0QohGBJeuMHFTB1CSXyDtaI X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2b6e7608-6456-42f7-3e84-08d8ccf7afac X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:05.0573 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: BIKT34xiAnPMvQojJQw639yb7IvN6nqwuWkai3x7YMzl4c3L13/WSsg5wlRxSh6vwIiZhW+CUXRyslJ2d+ts5KquCX4RATbzUhYHXgItKto= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 bulkscore=0 adultscore=0 mlxlogscore=999 phishscore=0 spamscore=0 suspectscore=0 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 impostorscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 mlxscore=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This patch adds plug/unplug callouts for iblock. For initiator drivers like iscsi which wants to pass multiple cmds to its xmit thread instead of one cmd at a time, this increases IOPs by around 10% with vhost-scsi (combined with the last patches we can see a total 40-50% increase). For driver combos like tcm_loop and faster drivers like the iser initiator, we can still see IOPs increase by 20-30% when tcm_loop's nr_hw_queues setting is also increased. Signed-off-by: Mike Christie --- drivers/target/target_core_iblock.c | 39 ++++++++++++++++++++++++++++- drivers/target/target_core_iblock.h | 10 ++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/drivers/target/target_core_iblock.c b/drivers/target/target_core_iblock.c index 8ed93fd205c7..58e0d47d5163 100644 --- a/drivers/target/target_core_iblock.c +++ b/drivers/target/target_core_iblock.c @@ -61,9 +61,18 @@ static struct se_device *iblock_alloc_device(struct se_hba *hba, const char *nam return NULL; } + ib_dev->ibd_plug = kcalloc(nr_cpu_ids, sizeof(*ib_dev->ibd_plug), + GFP_KERNEL); + if (!ib_dev->ibd_plug) + goto free_dev; + pr_debug( "IBLOCK: Allocated ib_dev for %s\n", name); return &ib_dev->dev; + +free_dev: + kfree(ib_dev); + return NULL; } static int iblock_configure_device(struct se_device *dev) @@ -171,6 +180,7 @@ static void iblock_dev_call_rcu(struct rcu_head *p) struct se_device *dev = container_of(p, struct se_device, rcu_head); struct iblock_dev *ib_dev = IBLOCK_DEV(dev); + kfree(ib_dev->ibd_plug); kfree(ib_dev); } @@ -188,6 +198,28 @@ static void iblock_destroy_device(struct se_device *dev) bioset_exit(&ib_dev->ibd_bio_set); } +static struct se_dev_plug *iblock_plug_device(struct se_device *se_dev) +{ + struct iblock_dev *ib_dev = IBLOCK_DEV(se_dev); + struct iblock_dev_plug *ib_dev_plug; + + ib_dev_plug = &ib_dev->ibd_plug[smp_processor_id()]; + if (test_and_set_bit(IBD_PLUGF_PLUGGED, &ib_dev_plug->flags)) + return NULL; + + blk_start_plug(&ib_dev_plug->blk_plug); + return &ib_dev_plug->se_plug; +} + +static void iblock_unplug_device(struct se_dev_plug *se_plug) +{ + struct iblock_dev_plug *ib_dev_plug = container_of(se_plug, + struct iblock_dev_plug, se_plug); + + blk_finish_plug(&ib_dev_plug->blk_plug); + clear_bit(IBD_PLUGF_PLUGGED, &ib_dev_plug->flags); +} + static unsigned long long iblock_emulate_read_cap_with_block_size( struct se_device *dev, struct block_device *bd, @@ -337,7 +369,10 @@ static void iblock_submit_bios(struct bio_list *list) { struct blk_plug plug; struct bio *bio; - + /* + * The block layer handles nested plugs, so just plug/unplug to handle + * fabric drivers that didn't support batching and multi bio cmds. + */ blk_start_plug(&plug); while ((bio = bio_list_pop(list))) submit_bio(bio); @@ -870,6 +905,8 @@ static const struct target_backend_ops iblock_ops = { .configure_device = iblock_configure_device, .destroy_device = iblock_destroy_device, .free_device = iblock_free_device, + .plug_device = iblock_plug_device, + .unplug_device = iblock_unplug_device, .parse_cdb = iblock_parse_cdb, .set_configfs_dev_params = iblock_set_configfs_dev_params, .show_configfs_dev_params = iblock_show_configfs_dev_params, diff --git a/drivers/target/target_core_iblock.h b/drivers/target/target_core_iblock.h index cefc641145b3..8c55375d2f75 100644 --- a/drivers/target/target_core_iblock.h +++ b/drivers/target/target_core_iblock.h @@ -4,6 +4,7 @@ #include #include +#include #include #define IBLOCK_VERSION "4.0" @@ -17,6 +18,14 @@ struct iblock_req { #define IBDF_HAS_UDEV_PATH 0x01 +#define IBD_PLUGF_PLUGGED 0x01 + +struct iblock_dev_plug { + struct se_dev_plug se_plug; + struct blk_plug blk_plug; + unsigned long flags; +}; + struct iblock_dev { struct se_device dev; unsigned char ibd_udev_path[SE_UDEV_PATH_LEN]; @@ -24,6 +33,7 @@ struct iblock_dev { struct bio_set ibd_bio_set; struct block_device *ibd_bd; bool ibd_readonly; + struct iblock_dev_plug *ibd_plug; } ____cacheline_aligned; #endif /* TARGET_CORE_IBLOCK_H */ From patchwork Tue Feb 9 12:38:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077955 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88B0BC433DB for ; Tue, 9 Feb 2021 12:40:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5973B64EB1 for ; Tue, 9 Feb 2021 12:40:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230244AbhBIMkM (ORCPT ); Tue, 9 Feb 2021 07:40:12 -0500 Received: from aserp2130.oracle.com ([141.146.126.79]:43260 "EHLO aserp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230198AbhBIMjz (ORCPT ); Tue, 9 Feb 2021 07:39:55 -0500 Received: from pps.filterd (aserp2130.oracle.com [127.0.0.1]) by aserp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CYADG054675; Tue, 9 Feb 2021 12:39:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=tidXB8n4fN0LxkN855F3fC3XEAjGcZwFTrYtF55Q/e8=; b=K2mexNLD3LpN6fypIYMFVcq++KY2XAEhy9brjWTes8hstOB85y7HZFMM5Pr/sqDgDsc8 BRHw6s1ey9RWd63JgWs4WbS72AmC2VqX15T3jmYjXMtwt3SB4VP5T/5Dcr2W5sf2Csh5 jE7KMjNMFnYVM+SEx3Ds+jWJJjEoLxTXwrun7seFfHq6VfS/KI4tQ7jYJinMVYgo7JCS 8o8kHQUTM3qcjqGHZ4vxxp33Cbijbkj/hBqWKXk+k34qxLNR2mYUe+VCKf73ne/Vl0F2 JtavGsjsA6GiA1W2lpIo2KvKdA/BUGCFrcun1yx/IjTMeyDFA+t49mTjdo2TszTnEutg 2Q== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2130.oracle.com with ESMTP id 36hgmafe4y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:09 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZJIc035198; Tue, 9 Feb 2021 12:39:08 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by userp3030.oracle.com with ESMTP id 36j51w15v2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:08 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=LBDtvP9tGGI7VZUPg70e/uYyJhBR0POUPPLVYn0WNG0cLJaRc8fweuOiFeXjPEFJfpmuUQqTCu4QafpF1DhtaF4zgClWY/wRRjYzR+IoUzEL8kapdb1lY2YVb36l/Fx7KKEXnnK1/ejba+a22AzCs7AxgLbStGTnyndcJ8eC1Lb6P3j30B7Cq5DJagAbQt1hrTPo4sJDy/stlVPVvzodjGyiN02YlHjpTIl7GxbUD7eli7C6qE1GneSiQLOh/jm7Vmljm3+a4use1pq6gouEtJ7ZJhB5P5Q9oHXTFxG2ebJS85nlcPsV6062GFCtk5mBBTfXogoAklnZIaATsvmMPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tidXB8n4fN0LxkN855F3fC3XEAjGcZwFTrYtF55Q/e8=; b=dpFdzlvByb+cNYuiSHCLa7fsQ6lOV0uFrH3ZPB3XvH1Z2mNcpIwivrfHlJfDwI7NCfgXMPZSJwayAQJCsPEC0zj6ADmyG1VD8X9mJSWAstpSEEF9iboIeIGUXTSS2dwvlUR7hf1UFJiBASJVTfTaZW91293ZhMqS50eRfBViM6UNbpnOi9Bt2oVkTfGfKBc7QErkc+TcskRGZ3Bxc/OrsbsgBmYDjyXlXB9tx4bL5PED+LkB7x+tveU8Ou5U6BLMGdfL9oKScrJIdiMflEg3Jm2DL1rvNpTQSJmO67Ypkr4KCvnT9ooxhJvqtJRPhMNxiZeJW+Oc3ywnCm0T0hbuZQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tidXB8n4fN0LxkN855F3fC3XEAjGcZwFTrYtF55Q/e8=; b=GxJrQhXHt3jWW6BG9J/lscgywhTcylBVhrzznnbTlPRLErXDS5n4Lwl9i0Df4CGNjZ81uj48wPYZGblv6DjOTbdTDTyStrvzUBIEPMkrV9jZFJWJtaNEm24KMuw/A1atfZCMzHqlyKk4iQmLZXmwknD7zCYLuBxb+ztYCtmA0yE= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:06 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:06 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 10/13] target_core_user: add backend plug/unplug callouts Date: Tue, 9 Feb 2021 06:38:42 -0600 Message-Id: <20210209123845.4856-11-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:05 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: fcce3427-d1f5-47a8-9ab5-08d8ccf7b05d X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uWYeThUDL4emqhKytf2p6Am3DzAvR/7111WoMKqcoRNCyGzZ/eytExz3knDmM/WRY24Trh+FVEYcW9WcssUWml5zN8gK0TyIKRHv/Zm1Z2EzjAmjPr/Ebq3LhF11BssQUwk/XrZSwig7qtzX02NvUroAzUhs5LcYBHe1ztmCOgfG3GA9KRkYdsB8DS3E/WA47rGLtoHXh0QaqHVguegGVAdDD3YFKK90w6d9bvL1dhQSS1AcVSMdGyTo/vz6uJ4JXw2W/WZkNFj8mBPQQwywBMY8whSIi+gHQtw0elDW0o0bbwV/nXUKjEaCGtTM1DXY4K1dkBpbzcBzjR1wY4BheLAtFYOJezVxEcZyj1YzODo8fs6sdysBiMwcIKEwoBQCir4L/GSkjyqznudBSus1765Z63VUsmmpXq7kCboUlMXcZA6PohB0FjfLgVsSoqguFC5IWydpXMN+f3ip79uGFMEmPxPEVPYmyyo3y0U2m4eGETcqXNCJdlKJwZqSEh9HpWiR7HotsT8zawgiMeN8H/DuwGDzhP3pWdLN7NoxGzKhxPru8EcJIVhVlprjxWr6jVJmleJccnLbTTGqiiYNxw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: sSLHnddR6K/stmV58IGe1ogoV0mnAt0aCF1eWSb2bZ0fGSyl7xm+l1xd/iQTLQvU4wov6ZgiBt/dbxbnz432LcZf8XPZubq0XTbk+byN/xBAo40xrfgW3Nkl5KIoXXyx1N8goLYma/KfOkU6rv91AfNYFuRfJW3rfVEQyU4HOP02JBjjP88iZZ9LZSVzAOEg7RRPPqyST3Ud+KjK4+XZoVuBoehWFbK5g8xQmI8g8M3YVhCak8r90eZGTO+0BEq4/0nhPsa8cbyt8rEU6Ndk4qlN+AUOTnM1SvYu4BrPQP7NgM1IUqcc0ivOrDuE8jTOrRt3lsSK0bpJpnEr5Fom0lyv7SEbpbAkA8qf6G0qg35GFSMQCAWfwnUkOir7jHJYneSi9wjewCDBNDanefRxVwsebXwsW83dxnbQfVcbZZptyloWSa0PceT3Uc7wnzHT/7xpIJyrpSSfHhIOTazap2Nf6Duap3nAXq6Zg0j8ewt9tqm1uNhzOouzkET/QQWvvNhHBtRNdbRaR9Zw/KtsyU8FwrepNUrTbHEvv7D9gWcxogEjg9kdn3+fs9CQoWCaScs5c/Z3IA31PGSdzBWewBrHsSBL4OFfhPjE57SVIH7Y1HEoGrdRC8iUSZNogSL+t3Tb2w9PmrnNMonbThx464HKUbk5tTyRejfR/MdZlyRJ4AKFmpWnuTXdNuQ6QyQKA3Waeu7+9AajUef/LWQMjS4p4A6PByb0aUOpX+EH/ur6FAlolubp2WdhxxM7O5Lo76A3nnLf7Hup/MPvTwF2ULqNwz5En5zL3ycqY667XfHzkPvjRGbUl63HYBAgHdWa3o2bb2OuSBPm1YPnZ1zWK3/tqYPJpWnHNF81WCTUPijLc7+o9YtFKRdqRsn6jBx0yeDQvtc9tvhISy7cHt7IjQJ3Z0sW4rvyje4Q16/UR6ytDSxV31DPQc28klYcGact1pw8Tb8RTsyFF574sv8Hx5bb7rE1sIFK3CxTjJomwLIcqsUfhC1B01U/79LW6e0JrFG/eekUtCNBHgCab0smTMtMYX6LJt3vPFEAxBTwd2mM1k5J2h2/tebYM0Atqy4vilVZFgOxh46WNxelZDtcxCW/dgZjrAxDUHbBZHSbRxbYAa7ruDK8HcfMoOxKOcAfDbaP4msSnzkaSbgdduWMBf3qMmfUM6Gou4aOb8c9gz+WJj6nDjTmPQP7GEML+Zd1DF1t7d+w2tqXB9IU1j5QwzSguDeofU6lGmhkFZXWMLvL0GJSqBNp06vrfh9nSwkfcpBEjuTzeujkHFzm2hDnjNI4ExnKJKc4EuH6RVmrAvuKeA7Fr1pcFkmaDRblWcXs X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: fcce3427-d1f5-47a8-9ab5-08d8ccf7b05d X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:06.2607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ys5npo55v8yuGTVfaYYG1+o3gZpctixo6lM0M7hslIo6zPnXiVt+hS3NmPAjh2ItXF4sIxGR65AdW1ZX9kov+n3+EaeGElVfrp7VfEqDNes= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 priorityscore=1501 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 adultscore=0 impostorscore=0 suspectscore=0 mlxscore=0 clxscore=1015 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This patch adds plug/unplug callouts for tcmu, so we can avoid the number of times we switch to userspace. Using this driver with tcm loop is a common config, and dependng on the nr_hw_queues (nr_hw_queues=1 performs much better) and fio jobs (lower num jobs around 4) this patch can increase IOPs by only around 5-10% because we hit other issues like the big per tcmu device mutex. Signed-off-by: Mike Christie --- drivers/target/target_core_user.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) diff --git a/drivers/target/target_core_user.c b/drivers/target/target_core_user.c index a5991df23581..a030ca6f0f4c 100644 --- a/drivers/target/target_core_user.c +++ b/drivers/target/target_core_user.c @@ -111,6 +111,7 @@ struct tcmu_dev { struct kref kref; struct se_device se_dev; + struct se_dev_plug se_plug; char *name; struct se_hba *hba; @@ -119,6 +120,7 @@ struct tcmu_dev { #define TCMU_DEV_BIT_BROKEN 1 #define TCMU_DEV_BIT_BLOCKED 2 #define TCMU_DEV_BIT_TMR_NOTIFY 3 +#define TCM_DEV_BIT_PLUGGED 4 unsigned long flags; struct uio_info uio_info; @@ -959,6 +961,25 @@ static uint32_t ring_insert_padding(struct tcmu_dev *udev, size_t cmd_size) return cmd_head; } +static void tcmu_unplug_device(struct se_dev_plug *se_plug) +{ + struct se_device *se_dev = se_plug->se_dev; + struct tcmu_dev *udev = TCMU_DEV(se_dev); + + uio_event_notify(&udev->uio_info); + clear_bit(TCM_DEV_BIT_PLUGGED, &udev->flags); +} + +static struct se_dev_plug *tcmu_plug_device(struct se_device *se_dev) +{ + struct tcmu_dev *udev = TCMU_DEV(se_dev); + + if (!test_and_set_bit(TCM_DEV_BIT_PLUGGED, &udev->flags)) + return &udev->se_plug; + + return NULL; +} + /** * queue_cmd_ring - queue cmd to ring or internally * @tcmu_cmd: cmd to queue @@ -1086,8 +1107,8 @@ static int queue_cmd_ring(struct tcmu_cmd *tcmu_cmd, sense_reason_t *scsi_err) list_add_tail(&tcmu_cmd->queue_entry, &udev->inflight_queue); - /* TODO: only if FLUSH and FUA? */ - uio_event_notify(&udev->uio_info); + if (!test_bit(TCM_DEV_BIT_PLUGGED, &udev->flags)) + uio_event_notify(&udev->uio_info); return 0; @@ -2840,6 +2861,8 @@ static struct target_backend_ops tcmu_ops = { .configure_device = tcmu_configure_device, .destroy_device = tcmu_destroy_device, .free_device = tcmu_free_device, + .unplug_device = tcmu_unplug_device, + .plug_device = tcmu_plug_device, .parse_cdb = tcmu_parse_cdb, .tmr_notify = tcmu_tmr_notify, .set_configfs_dev_params = tcmu_set_configfs_dev_params, From patchwork Tue Feb 9 12:38:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077959 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1819C433E0 for ; Tue, 9 Feb 2021 12:40:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 683DF64EB1 for ; Tue, 9 Feb 2021 12:40:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230249AbhBIMkR (ORCPT ); Tue, 9 Feb 2021 07:40:17 -0500 Received: from userp2130.oracle.com ([156.151.31.86]:43564 "EHLO userp2130.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230199AbhBIMjz (ORCPT ); Tue, 9 Feb 2021 07:39:55 -0500 Received: from pps.filterd (userp2130.oracle.com [127.0.0.1]) by userp2130.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZKSr038496; Tue, 9 Feb 2021 12:39:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=kdgAUNG5jwO4SxRDy5VaZFEvV0i4DmeYmZbMwRrGGAA=; b=P90y/BPdW7dbMxUm3Dbu9kO/RCMucgKNLoYkpAfDBbSiTVlCpV28xoI7Bp0rsC/JtL1b +aNk8qSJpFGiCxYjwhx6UduZZtPG4z6P4Fv/6PdU/39G5WODI6mNJlEsobzYqVifv2TM ZBakrZXzezXF/g0ZC168uk3HPxp1F8n2lN/VIJb9OgxxixmQsmppEaNlUvxED8g3L0Cj RsouJBZhdWFTUdsAZGpJQLWdOmrI90QtySN/3uU5FcpXWNbY+0/sVHi3n2KfM4raYwKD ox9nPgDiE5TkQAR0puiWIaAFDqNeJhEV9wwn8gHC1coZbFuTy9iIOu5nBgwfzQ8LGuuD Pg== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2130.oracle.com with ESMTP id 36hjhqqagq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:09 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZJId035198; Tue, 9 Feb 2021 12:39:09 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by userp3030.oracle.com with ESMTP id 36j51w15v2-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:09 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=RYFyGqIzI6aUX2XR0UfiW4gS6PNAcOJW0A2LwrNNYZ9lv9qH8CWDdeJhIbOnxACqaI6tkihyTbESXBmxc/U1KLzB7VMW04RLjffDstHe87sCBmq8Mnaz2syFFTfAaS8I/eZLSeZetkjX7YB6urTM/CJ3E0eOTrfdOKONfZ+VRWzrVRz4pPnSfCpgKrQlxvhbNmENp2rWgRZGEt/sCj0oUopTp5orCTJPDj07mj1YopStu1jpMHOTSBAJFeoM8RTAASzjemxawUgskLqGezqftIAGc5/7DHGbm3gjifKTtZVNxKbLbb66qlyHCrPwNvzU4CDRHauY8g3BQ5ZeRIG/MA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kdgAUNG5jwO4SxRDy5VaZFEvV0i4DmeYmZbMwRrGGAA=; b=hEj4QPiiJq4csQDPIm0gmmUgRz1Fazm+AwryAE4ycY1JmeYz8E52TAkoRyMXH40PmW5ucYwLwEo3p5ZCUx5FyzRSxhM4umZ4LljtIkiYUHwvUmZJMAfA9LYbWcfZ0XWO/9O+DmOENkNUVMlNN8e2/6d56RzbVDLoc0Pk6hkGBM+pXkeIRZz/jExOA+Gid6qm4+oXzSzDhS4gRtPmwd9gYBJfEdL/oj3SdAl+QBZy624EQOg3S3oosrELtzp+7oeMiV3Bpe+ONSwOrE2dPsUSDrIghTsJiBojHwYYKMeQWyFBlQPH6GgDGA41g3hReRQJmzHClHqxpxk23kkvODvjww== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kdgAUNG5jwO4SxRDy5VaZFEvV0i4DmeYmZbMwRrGGAA=; b=g1nHAgWvC2t7B2kQxHOzGv9PiniOeRGSEOZJ22Ley/2StRAq27BOqsQsOCBgeSLY3FH2fhW3szKClUqRxBLIG8lZwcON4/kxn4DcMwuISfUehZqmB9niHx+uFTPS6c165lOrHjFEIJdQ6pxKjnOdG0xqKbZS311yCgNuKgPLb3A= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:07 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:07 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 11/13] target: replace work per cmd in completion path Date: Tue, 9 Feb 2021 06:38:43 -0600 Message-Id: <20210209123845.4856-12-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:06 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 962cd03c-5f3e-4969-fac1-08d8ccf7b115 X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3968; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZuqsPJkTS4FtD6VOQ40N3QTeAdA/3lAwi7p3s2QgX1TwaUkV33aj/gI158DD2RIVjSqGzM1+6LypZC7yC1k93MXKP6gM90FzXGmXS7cGbuiJ+6Nx7IPjOyv0h80GdWiJzxfhijvUU4yY34kLOK+PeCb7s8VGHkH3yFjGTHsU2uDczKtbXPF8wzuhUfyBtVLtyupLp2YCPjSKrFUiM3b2AlRalQJ8inKaz3B1kTTIHZ9X7QiGsxIhyrjMp9CfFTQ3v0zvvH9/TnNpAmQP4XY/xatKQYXOx2d2Q+TQKG8rAStd7XJWblHriYOjAPSitJ+BugnGVj24MzCFQKf1qzD7a5O4+USDcR2JOAN6dCSHdsyHZKeTidB/cLb7vB1go9V1upETSoNuVuDKcKj0lD5jf1cgPmSYaDhPTQ75rtS/Lbped/ah2d7gEmcLHIn8t/aiY3ettCqCDrpiPoXH5eVCbrTVOPSauzOgCo3mkw4IxCzS7iJaS0B82PY2VXVf7GLXrCpA3lLD+a8QWk1ZrTNmTGjbktxSPL36uuW+uZ6yJ6rlZ80tBLkCz9+kawcXdZT51dcJyKqq0Vy9icR4ZWtR9A== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: li1rVKHsHu6Og4kt+X8d4c3HjbtRaSGmcdCzC5ij4qGMAhszLFyLrlmmXzM/Brbeo+bMyeYF+SaqrjqGcK2uTSpqPimP1CFGigrhxBCVCDZg56MKdSUQgNkZCtOw7lKIgQkRJREiXnPaRKzr3BNnKzAX47lJK/8gk33XEvwriMXzfpR/JTTgi8rmHvTuHd40v07y1gIGhyolpqAXoY+HnK4dN/IYDGlkcFliD3JE6/60qs/pCVOoptr0RR+t5Fj5mOWWDvkeIWRE6kcI87Ccl2HirX83ZIF7M7ffM7tjDHWrf3dgKCif4CDKyksSK/fcCUPHijNMAI5yhROV5GwEXpy6eazY21gzIooGbCkWGt9TjraV9hZA99mb5KZUNNFIlk1KMWgq/p8PW9QfTcYd5xkjlgYciHGFfAumLaXY1Pk7hUtc1Kausfk0hK1YIrJIJ/BWA7zyIGIdwRHlIM0LF6V8u7MvIzNYzimm1tkynLj3FecE2q40P6Wa52mW+dVEoPlzbX4853rohOsJ/5/GbrVnC1uVsN8gvuif1o+jwt8EEyU2iZAVZGWJADkSpT0h07fBQ1SDw3KYMcJzyTsQcBj2WcP3SYpLrdTRPPh8k/OSN6lGWVXAq7C9002heUlBt5Y4nm+DbA5GVSnJAUMYLgAREYy2HSmypmy6FGJlu67nQB+go+RsFurw0+Pbae5la7h8zoEKSoycrh9+qeq8lNdehnZrAYuTfRc6Z6D0VH8bJ244LrzPrkAT0VjJz7cdLXN1ODDCp7eI0g5PxkRPSKJtIUQbXhR3WmwPtKPDC8dukaw0vvyUzE+3Gq4dqJSdZC+1ozc50ObkeZ1ZK/iZzTiwTNxu8WfyXygowrjXS3ypqnWzHMp1SNg/wJzLYFpD7AJ9Y4rzdoVx+YFq2ruzSoZ+lBFqLIaErZ5bSj3U7gFuL0e1drNnwtsd785mcyUvgj9gXrSHrljY0dzedZ9IgseOgiNdffV3bjWh/g81VXYiBKy/18TFYyiq4mFT6RgARGXETkGA1iDGnmd1ocvKswINPUNNm6IMvZzWqDROtwpQWvuIt0MRTbeRAQYZO6iKwx6I45tW0B9mEk5YqECQjhXR7UksAYTyifBbMLck+MMUjDuUBLyQPsUKAz75kFj73CqcUQjglDdNoZROEdtx8aDWEVmGnwD9uEW9XT5a1l5d5wZwu8qxCma0NckhqZNJnulKgQCXFEbPlk7ENl7MmWeeRm1nIGq02rPONc7A59a5LkF2Q/hJTzerhYtvgbJWXb9KKrBr8SZicp7fQSgnVFwp6YnYtmsCMpLl7ODXy63ut4wO4GPOEqkmqo0muIC9 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 962cd03c-5f3e-4969-fac1-08d8ccf7b115 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:07.4250 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: z05Ga95RKoAjVWSwJOaV3JrX7/Ooun6/raMwbJ6AQ4a/YTqZwOGpAY8y0VdCwuYypy5kERs8T1yLJmwWLzlKVICudsmAgvur8zqYZbrpLj0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 impostorscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 mlxscore=0 phishscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 spamscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org Doing a work per cmd can lead to lots of threads being created. This patch just replaces the completion work per cmd with a per cpu list. Combined with the first patches this allows tcm loop on top of initiators like iser to go from around 700K IOPs to 1000K and reduces the number of threads that get created when the system is under heavy load and hitting the initiator drivers tagging limits. Signed-off-by: Mike Christie --- drivers/target/target_core_device.c | 3 + drivers/target/target_core_internal.h | 1 + drivers/target/target_core_transport.c | 98 +++++++++++++++----------- include/target/target_core_base.h | 1 + 4 files changed, 60 insertions(+), 43 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 74d3a4896588..eaa2323843c0 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -743,6 +743,9 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name) init_llist_head(&q->sq.cmd_list); INIT_WORK(&q->sq.work, target_queued_submit_work); + + init_llist_head(&q->cq.cmd_list); + INIT_WORK(&q->cq.work, target_queued_compl_work); } dev->se_hba = hba; diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index 56f841fd7f04..e54d05ae8dfd 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -154,6 +154,7 @@ bool target_check_wce(struct se_device *dev); bool target_check_fua(struct se_device *dev); void __target_execute_cmd(struct se_cmd *, bool); void target_queued_submit_work(struct work_struct *work); +void target_queued_compl_work(struct work_struct *work); /* target_core_stat.c */ void target_stat_setup_dev_default_groups(struct se_device *); diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index a90f20222894..b1f920c1b816 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -55,7 +55,6 @@ static void transport_complete_task_attr(struct se_cmd *cmd); static void translate_sense_reason(struct se_cmd *cmd, sense_reason_t reason); static void transport_handle_queue_full(struct se_cmd *cmd, struct se_device *dev, int err, bool write_pending); -static void target_complete_ok_work(struct work_struct *work); int init_se_kmem_caches(void) { @@ -732,14 +731,6 @@ static void transport_lun_remove_cmd(struct se_cmd *cmd) percpu_ref_put(&lun->lun_ref); } -static void target_complete_failure_work(struct work_struct *work) -{ - struct se_cmd *cmd = container_of(work, struct se_cmd, work); - - transport_generic_request_failure(cmd, - TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE); -} - /* * Used when asking transport to copy Sense Data from the underlying * Linux/SCSI struct scsi_cmnd @@ -827,11 +818,20 @@ static void target_handle_abort(struct se_cmd *cmd) transport_cmd_check_stop_to_fabric(cmd); } -static void target_abort_work(struct work_struct *work) +static void target_queue_cmd_work(struct se_cmd_queue *q, struct se_cmd *se_cmd, + int cpu, struct workqueue_struct *wq) { - struct se_cmd *cmd = container_of(work, struct se_cmd, work); + llist_add(&se_cmd->se_cmd_list, &q->cmd_list); + queue_work_on(cpu, wq, &q->work); +} - target_handle_abort(cmd); +static void target_queue_cmd_compl(struct se_cmd *se_cmd) +{ + struct se_device *se_dev = se_cmd->se_dev; + int cpu = se_cmd->cpuid; + + target_queue_cmd_work(&se_dev->queues[cpu].cq, se_cmd, cpu, + target_completion_wq); } static bool target_cmd_interrupted(struct se_cmd *cmd) @@ -841,8 +841,8 @@ static bool target_cmd_interrupted(struct se_cmd *cmd) if (cmd->transport_state & CMD_T_ABORTED) { if (cmd->transport_complete_callback) cmd->transport_complete_callback(cmd, false, &post_ret); - INIT_WORK(&cmd->work, target_abort_work); - queue_work(target_completion_wq, &cmd->work); + + target_queue_cmd_compl(cmd); return true; } else if (cmd->transport_state & CMD_T_STOP) { if (cmd->transport_complete_callback) @@ -857,7 +857,6 @@ static bool target_cmd_interrupted(struct se_cmd *cmd) /* May be called from interrupt context so must not sleep. */ void target_complete_cmd(struct se_cmd *cmd, u8 scsi_status) { - int success; unsigned long flags; if (target_cmd_interrupted(cmd)) @@ -866,25 +865,11 @@ void target_complete_cmd(struct se_cmd *cmd, u8 scsi_status) cmd->scsi_status = scsi_status; spin_lock_irqsave(&cmd->t_state_lock, flags); - switch (cmd->scsi_status) { - case SAM_STAT_CHECK_CONDITION: - if (cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE) - success = 1; - else - success = 0; - break; - default: - success = 1; - break; - } - cmd->t_state = TRANSPORT_COMPLETE; cmd->transport_state |= (CMD_T_COMPLETE | CMD_T_ACTIVE); spin_unlock_irqrestore(&cmd->t_state_lock, flags); - INIT_WORK(&cmd->work, success ? target_complete_ok_work : - target_complete_failure_work); - queue_work_on(cmd->cpuid, target_completion_wq, &cmd->work); + target_queue_cmd_compl(cmd); } EXPORT_SYMBOL(target_complete_cmd); @@ -1894,13 +1879,6 @@ void target_queued_submit_work(struct work_struct *work) target_unplug_device(se_plug); } -static void target_queue_cmd_work(struct se_cmd_queue *q, struct se_cmd *se_cmd, - int cpu) -{ - llist_add(&se_cmd->se_cmd_list, &q->cmd_list); - queue_work_on(cpu, target_submission_wq, &q->work); -} - /** * target_queue_cmd_submit - queue the cmd to run on the LIO workqueue * @se_cmd: command descriptor to submit @@ -1951,7 +1929,8 @@ target_queue_cmd_submit(struct se_cmd *se_cmd, struct se_session *se_sess, cpu = se_cmd->cpuid; se_dev = se_cmd->se_dev; - target_queue_cmd_work(&se_dev->queues[cpu].sq, se_cmd, cpu); + target_queue_cmd_work(&se_dev->queues[cpu].sq, se_cmd, cpu, + target_submission_wq); return 0; } EXPORT_SYMBOL_GPL(target_queue_cmd_submit); @@ -2054,8 +2033,7 @@ void transport_generic_request_failure(struct se_cmd *cmd, cmd->transport_complete_callback(cmd, false, &post_ret); if (cmd->transport_state & CMD_T_ABORTED) { - INIT_WORK(&cmd->work, target_abort_work); - queue_work(target_completion_wq, &cmd->work); + target_queue_cmd_compl(cmd); return; } @@ -2480,10 +2458,32 @@ static bool target_read_prot_action(struct se_cmd *cmd) return false; } -static void target_complete_ok_work(struct work_struct *work) +static void target_complete_cmd_work(struct se_cmd *cmd) { - struct se_cmd *cmd = container_of(work, struct se_cmd, work); - int ret; + int ret, success; + + if (cmd->transport_state & CMD_T_ABORTED) { + target_handle_abort(cmd); + return; + } + + switch (cmd->scsi_status) { + case SAM_STAT_CHECK_CONDITION: + if (cmd->se_cmd_flags & SCF_TRANSPORT_TASK_SENSE) + success = 1; + else + success = 0; + break; + default: + success = 1; + break; + } + + if (!success) { + transport_generic_request_failure(cmd, + TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE); + return; + } /* * Check if we need to move delayed/dormant tasks from cmds on the @@ -2625,6 +2625,18 @@ static void target_complete_ok_work(struct work_struct *work) transport_handle_queue_full(cmd, cmd->se_dev, ret, false); } +void target_queued_compl_work(struct work_struct *work) +{ + struct se_cmd_queue *cq = container_of(work, struct se_cmd_queue, + work); + struct se_cmd *se_cmd, *next_cmd; + struct llist_node *cmd_list; + + cmd_list = llist_del_all(&cq->cmd_list); + llist_for_each_entry_safe(se_cmd, next_cmd, cmd_list, se_cmd_list) + target_complete_cmd_work(se_cmd); +} + void target_free_sgl(struct scatterlist *sgl, int nents) { sgl_free_n_order(sgl, nents, 0); diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index b8e0a3250bd0..f2ba7de59da7 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -777,6 +777,7 @@ struct se_device_queue { struct list_head state_list; spinlock_t lock; struct se_cmd_queue sq; + struct se_cmd_queue cq; }; struct se_device { From patchwork Tue Feb 9 12:38:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077963 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 873A9C433E0 for ; Tue, 9 Feb 2021 12:40:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5630264EB1 for ; Tue, 9 Feb 2021 12:40:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230077AbhBIMk1 (ORCPT ); Tue, 9 Feb 2021 07:40:27 -0500 Received: from aserp2120.oracle.com ([141.146.126.78]:42596 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230207AbhBIMj4 (ORCPT ); Tue, 9 Feb 2021 07:39:56 -0500 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZSep008730; Tue, 9 Feb 2021 12:39:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=OSCFIcw+tBeadXvOC1n4T7x3USrV/C467H2FsDnmOF0=; b=JlJGFFDkLgBo8NycMgJIMYd2qUXPfng6CxosqMp28wxnHQwRt2URyuL9n5s7xkErg4X0 9DOoifLbPFXTQLsg6LHn8W45Lpgem2Ut1iif48QP/V0E6NVlwreT5RNDZep2RkfO+ZxH oCdfeGWPcTxvqhlyvjyywtDQzQimO/wxEp3FkYxjT5JlcuPMMH2a5E9SOH19mxQufxxs CUXyNT+nQ/5n1KMchLvdTNUek2UaktXYI8QCAo5yCsHadzm7Vp1Ahvpop+dZ9IDAHt/f avG4bIFvqaB2H5L2Xz209yytb0xjJ7hSBaU2oLEZLtUClEv2LEtNlRsTJQmXb3RuldDp xQ== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by aserp2120.oracle.com with ESMTP id 36hk2kfajt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:10 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZJIf035198; Tue, 9 Feb 2021 12:39:10 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2100.outbound.protection.outlook.com [104.47.58.100]) by userp3030.oracle.com with ESMTP id 36j51w15v2-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:10 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=F8SH0VwrQtcQN7kgDBEPbfdPlC8BBxsozlLoTvs5hstBndaXeKz5MrWDrvqt8MdJeiN3fw6PDC40RjOn9tK3grKrfSTbuyzfG2y6tODmPDxZrqdlhO0RP/EwTMhHKgGiy37xM1nERGdYoJobALIfGB2c7wFM8QHxQsk4pwSQbAQQaa4vM3Nx06kXa0JrKC0RrATZafae1wU/PmDF66bYbygZp7d4123BkYfZfCunzHThPOxaVvI6AEIoyUCvp8/CnokRyBJxwlC+mVAmNS9pcLlk2hjfTMuh92567qozPTvxMrU3AlbR+ILJidsRsboTy4pklZuVfaM6uUBCx7OkZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OSCFIcw+tBeadXvOC1n4T7x3USrV/C467H2FsDnmOF0=; b=V7LzHMfaamHcxiEA1DWib9/aEsxAJX3/vZWD+8mK2cLtIWrJpbHCRNMbBlFF0XwFJ0PSTEWjo1UpadwG0m1qqjHMjiYIkrNKME+wtsIzF4B17WD+/Uon23fDuvyToKzM4Sw+RtU+Mr4TuMmtQkjXz/uwbKx2efVja9+D9Wlm77ApV+1EeszUzWdVlYBB0fO0cwUGZxtXdfodfZMOD1+ATfyli769MelLbkgcqnu7qwKTMeGBUH5byXL37665chRxJFhs/MetckDpoOiEoq0poH+xXqEZZdD0Mk3DvnG7R2T/DPiGThsDltQX923WKRy8X886KcWVcX6JOwNh+gVRCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=OSCFIcw+tBeadXvOC1n4T7x3USrV/C467H2FsDnmOF0=; b=rMVeXr7aEJ3pk9Pvjh5VB8IcgDMn7WX1ypAsTFqpyBZtes8ecQwaTo21yoELit2OOuC/RIslkzBRkklmuRe6kYtnfvmGhm6g303IlLQPAbE+fc8Av+ZWfkVJ/q9q9F0dRLMvFv5GpnffUYHdfS8tqwIeg1iKw4n++dpsIpQQ3xs= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:08 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:08 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 12/13] target, vhost-scsi: don't switch cpus on completion Date: Tue, 9 Feb 2021 06:38:44 -0600 Message-Id: <20210209123845.4856-13-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:07 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9e6ea9c1-be35-45a8-71c3-08d8ccf7b1bb X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: GNdoq/FM+ILuwB3XErAeLREVUev7zV05pBQ39q+/m3gaQqsTe6ZC9ebwWaI5EyvyZzNqyTgBgOhrcvLNb9WcfD4qDZMZKhU7Hjz/o1s0RlTCrJf2L3QnwyFMl7iqBWlRyKRgnZjqzFq9ZXBJ/B78OE/HPfi54TOdjd813lHAPasWJdEOCyGCBDb+6RbjMpz2KF58AMW1+LMcwEbXUrBfzVWvBTtgiTrGqGsRit8+LecBAG66PRgg19ZAB11GwpD2MLHhmomioxab+eEd2tp7bX8XyP+fqLFaZ1RMERQ6GCBU/Xoay8AP6OaoaJ1pD9D2vMsVdunAitR4oYvJeUcXUpAwAvf74gq0Tpvfez9ygAN4dG1CiED8WijulcPDSSguXtS26EShiModkrMSVAXj9AMVV2HXjoomKLghliksnlsBbkk/jiWa5jtEbThpzx2Y2h2POO8D8SEmMNMV01K7h6CYWrkVVsT2XYtQdQGcJ9n9EOb3R1mpYyL6jiBcW90uwaF0G/y8BafWHJabxrpXXyilr6tWN3hgWPmN70OAtqiZsW9b+qePec5pXCBpKmyT9Y2vK7UUjV5xTCQub5YsCg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ArKyj6Z/+LdgVEYOjHYUfsOy5itSmVGRbBERn3fnR4QXIH2eMt2bEFGnszYIUqMA69DMpfqmh3FD5rNDAIoOQuHBuHZvYBT//jOMjljUdgNRz+4KlC7bw4/fNNvoi8L/TrjqtGg6s0eV6plz5bZOTAMUchOOcxGpPaxxxW5CHkbzgqMKw+r6r50hCESOAcNMpGv0Zf3pJp7LW97JCeXzYVHkaccYdEguDOa0ZnimjVEAA9McajMaTcBtavSzIPyfB+MIsEZAtdNVX/OMhxBnha/8XL5iw0UOA7cnj/+aDIn9Y6teOYjFDPfyQF8I68KDMmYaswBnMLZ5x7+n3oPkqW0l2xqy5TH63DyaxTAWfSxPF2nIeYHBNB1egEj9s4QgBs7algVrAX/CVtTSwpxqd5DCqJmQxn697MwXf+vNgxn/lO/Y7o6gwlzB7N8TZzxIpuTqg8os8RFuFjg/3wN2fUBnO5TC0svuxk2S3DDTYQix0GsUVGnyMDp1h6LI57G/14e1ZVES+nVg+OzUgJG7m0RvRMc0W+GURbdcfQ2jBLAQOkLrCaMhhWYQzURoIvPsEO17hPUGdP5TjADwjkP72rFnMCi6o3Fjskm9V8N557q/4XmoLNxkiIzHbJqRqAe5W4LTVKOaKQw7VHEYa3cEYVYKykngQrh+uGerEoaJdnmOZ2/pQDdBTfIUE6AUTkILrARf3fgo95Yq0M+yEQQQh+eLWD9WvlYNeqBK5YQh1EDWUSG5N6uxVj4kok63y4pzW84ch3HlQaPnA4gQRHBI/sO03e2+JXpK109zH7XzHe7zp33IeIcL1F88bOwQLp5QFAM/wLPX7yrI7ww+3CdgWuBR3/rr0OkwzWXdsaOV1iLYmLBT1UwLRZihG4Ss8Alm7ono+SieugM8TgtrcregLKX+NHPfUnrxVN7+knBIabgYiGALOvwR/Ef5J379vh8/+B4Rz3fe1JUYsGyf9AplLMEhnFxxh7A/4t/5hhLw9H5Cvum3Ak2aKwtoIU21v3uMZ00+b8u4fVXbhFXri6pykVSm9MONNLFTRSEboyBfby0HvY9R2+8DD+A4ABTkEVjfdKNvjHlPZXODZGFKw4kg2rb5SNRNIDNqFHe4yj9RD2W0P29gDTFoq5IPS10qthTp8yL2c15BNwhl0u4llBwz+hXy1fFeoXY4dV+GTlvVIR7aHZOArMceqari0Zw5VSlIaCYesEvVtrLoNWPo9c3f+S7Mg8ug0THjcYavpcnNg6NyQmkl63aU4gT94sb3pAsiuZeM33BYVXgua4lx8FNaOzZgJmcXl+fDQfELLUeaLvuUHuuS17XvrAmn8LGr+uku X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e6ea9c1-be35-45a8-71c3-08d8ccf7b1bb X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:08.5214 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Vc8TMmdPilGRu0JkrHpF7K0vlBxB047HiXpEtUVZr73zRQ4WGc9ZDvmb3RwLpy+Nf0vgLVdSQdRHpzb2CQJXdvDnVqiMiBqoLYm9/7gF3UI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 priorityscore=1501 clxscore=1015 impostorscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 adultscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org LIO wants to complete a cmd on the CPU it was submitted on, because most drivers have per cpu or hw queue handlers. But, for vhost-scsi which has the single thread for submissions and completions this is not always the best thing to do since the thread could be running on a different CPU now, and it conflicts with what the user has setup in the lower levels with settings like the block layer rq_affinity or for network block devices what the user has setup on their nic. This patch has vhost-scsi tell LIO to complete the cmd on the CPU the layer below LIO has completed the cmd on. We then stop fighting the block, net and whatever layer/setting is below us. With this patch and the previous ones I see an increase in IOPs by about 50% (234K -> 350K) for random 4K workloads like: fio --filename=/dev/sda --direct=1 --rw=randrw --bs=4k --ioengine=libaio --iodepth=128 --numjobs=8 --time_based --group_reporting --runtime=60 Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Michael S. Tsirkin --- drivers/target/target_core_transport.c | 10 +++++++++- drivers/vhost/scsi.c | 3 ++- include/target/target_core_base.h | 2 ++ 3 files changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index b1f920c1b816..e5e555dcd73a 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -828,7 +828,12 @@ static void target_queue_cmd_work(struct se_cmd_queue *q, struct se_cmd *se_cmd, static void target_queue_cmd_compl(struct se_cmd *se_cmd) { struct se_device *se_dev = se_cmd->se_dev; - int cpu = se_cmd->cpuid; + int cpu; + + if (se_cmd->se_cmd_flags & SCF_IGNORE_CPUID_COMPL) + cpu = smp_processor_id(); + else + cpu = se_cmd->cpuid; target_queue_cmd_work(&se_dev->queues[cpu].cq, se_cmd, cpu, target_completion_wq); @@ -1609,6 +1614,9 @@ target_submit_prep(struct se_cmd *se_cmd, struct se_session *se_sess, BUG_ON(!se_tpg); BUG_ON(se_cmd->se_tfo || se_cmd->se_sess); + if (flags & TARGET_SCF_IGNORE_CPUID_COMPL) + se_cmd->se_cmd_flags |= SCF_IGNORE_CPUID_COMPL; + if (flags & TARGET_SCF_USE_CPUID) se_cmd->se_cmd_flags |= SCF_USE_CPUID; /* diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 99909c6f3960..b9addd5fdd28 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -803,7 +803,8 @@ static void vhost_scsi_target_queue_cmd(struct vhost_scsi_cmd *cmd) &cmd->tvc_sense_buf[0], cmd->tvc_lun, cmd->tvc_exp_data_len, vhost_scsi_to_tcm_attr(cmd->tvc_task_attr), - cmd->tvc_data_direction, TARGET_SCF_ACK_KREF, + cmd->tvc_data_direction, + TARGET_SCF_ACK_KREF | TARGET_SCF_IGNORE_CPUID_COMPL, sg_ptr, cmd->tvc_sgl_count, NULL, 0, sg_prot_ptr, cmd->tvc_prot_sgl_count); if (rc < 0) { diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index f2ba7de59da7..bb4ac7e6f560 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -146,6 +146,7 @@ enum se_cmd_flags_table { SCF_USE_CPUID = (1 << 16), SCF_TASK_ATTR_SET = (1 << 17), SCF_TREAT_READ_AS_NORMAL = (1 << 18), + SCF_IGNORE_CPUID_COMPL = (1 << 19) }; /* @@ -195,6 +196,7 @@ enum target_sc_flags_table { TARGET_SCF_ACK_KREF = 0x02, TARGET_SCF_UNKNOWN_SIZE = 0x04, TARGET_SCF_USE_CPUID = 0x08, + TARGET_SCF_IGNORE_CPUID_COMPL = 0x10, }; /* fabric independent task management function values */ From patchwork Tue Feb 9 12:38:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12077961 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7972DC433E6 for ; Tue, 9 Feb 2021 12:40:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 37F1964E50 for ; Tue, 9 Feb 2021 12:40:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230199AbhBIMkW (ORCPT ); Tue, 9 Feb 2021 07:40:22 -0500 Received: from userp2120.oracle.com ([156.151.31.85]:39208 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230208AbhBIMj4 (ORCPT ); Tue, 9 Feb 2021 07:39:56 -0500 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CY7U1047664; Tue, 9 Feb 2021 12:39:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2020-01-29; bh=T5zSD8qo191UlsCAuxlAcPE/y3JaSiRJjIgMFsvXhIU=; b=SGSU7K5PYdPDmZhdl+YWDcY2pPxLMm7CYYp9clGCIseNurocuX95t031/kB0EVhq9guo I6VXUC85I2m8H29ThigdyofvjJIqEz9b0BPj6L8pXIMhq7zXCElS4RSWoqPnPmyPoIRf UsrvRyVxEwxuuOKFei4V6uSm80Os8EMHBd/R29RMcZ9CreMmPUyCixjtsYzTcvBXsS5I +a2JmGTGyyhq6Hzp7O6SPex4i0x4v7f/A6NhwOGm6bPErfVtqKRuEZ8So+G2dD/zxhft jckQdDp7wU75ggvFs/2BWF/Wpv36x9UhAiVsln5+H1z4Ayl9Y2v1z4o1ClcCYlnsOFKK NA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by userp2120.oracle.com with ESMTP id 36hkrmy8cp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:12 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 119CZEqh034098; Tue, 9 Feb 2021 12:39:11 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2107.outbound.protection.outlook.com [104.47.58.107]) by userp3030.oracle.com with ESMTP id 36j51w15yf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 09 Feb 2021 12:39:11 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Gf2CCaMDcJxDBMnfbudXfn018jrqkqcUUaxUp3hpIOAfJOsCRXLvc7T9AUV5sQIXKtwo9YKwdcyUXYp/KNA2TdbJ2ntjKndmErRmn3N9u/EoM00O/wW2XJAg3HAN8vH8KfHgpYbrlpRgbze/vabCKKStJISPC6fuGV0klZFU1nFmHfSPFUQlvUwy1jE9zJ9Blw+6MpfBgVqY6pSUFdQTCxu/vqyIBeSpyArT1zo71naqzy2MhKjc45gUxbsCFtY14tuqGT3YhPV3Udv1PsdWuigfvpOWIfD7WdrQ9OyZGy6xfg2biSbfLvArsbh763cgds/cStUtarJCQzou+QEqoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T5zSD8qo191UlsCAuxlAcPE/y3JaSiRJjIgMFsvXhIU=; b=QC15mOvue3X1qoNT6W+fuQZNrIBNC8UMfhQ7zjQWAU7gxVdMChR0rpUzY0cGBMqspr95vwD/IXMuj/lXjxBi26a43O/EI0Bba0f78sdLUtDNiYb9hAtnejWz0Dq8uDe9KETi+MV7sldRAeImdnO9VlpJ/riBG6gZHB/yEsFO9SUhpi1NZYLdNxTPLr/BiYJDqANReWF6n2AGKeUyobvSwpy2+T0nd0BCJ+TKIs9wtarOFYLVozGg0I5fsrwyMtAKFw2ZlTKiynlN0EE3D2t2QEwXIUUKDWiyEQmTtiGYhKL7VoCEYR2VvfyUC0cNEhQ/dMrw11dAypROUEzjvV/AAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=T5zSD8qo191UlsCAuxlAcPE/y3JaSiRJjIgMFsvXhIU=; b=EF3EhD58EeOyUyS0P72esegUTdDC9/xlrce9WeX4PDf4m0bFeF22S9Z02muBNl2ds8R1LHG7QRVxsvwMuP3nmMM2taynK2l6FlCejZeN4L7xWOThyqbnTBL18tV+BnCQc030Uj3QN8hkO7axNFDW9rUQW8VPUPs5JyUTfudQ9l4= Authentication-Results: wdc.com; dkim=none (message not signed) header.d=none;wdc.com; dmarc=none action=none header.from=oracle.com; Received: from BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) by BYAPR10MB2600.namprd10.prod.outlook.com (2603:10b6:a02:aa::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3825.25; Tue, 9 Feb 2021 12:39:09 +0000 Received: from BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20]) by BYAPR10MB3573.namprd10.prod.outlook.com ([fe80::1d86:b9d7:c9ef:ba20%7]) with mapi id 15.20.3825.030; Tue, 9 Feb 2021 12:39:09 +0000 From: Mike Christie To: Chaitanya.Kulkarni@wdc.com, loberman@redhat.com, martin.petersen@oracle.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, mst@redhat.com, stefanha@redhat.com, virtualization@lists.linux-foundation.org Cc: Mike Christie Subject: [PATCH 13/13] target: flush submission work during TMR processing Date: Tue, 9 Feb 2021 06:38:45 -0600 Message-Id: <20210209123845.4856-14-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210209123845.4856-1-michael.christie@oracle.com> References: <20210209123845.4856-1-michael.christie@oracle.com> X-Originating-IP: [73.88.28.6] X-ClientProxiedBy: CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) To BYAPR10MB3573.namprd10.prod.outlook.com (2603:10b6:a03:11e::32) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from localhost.localdomain (73.88.28.6) by CH2PR03CA0007.namprd03.prod.outlook.com (2603:10b6:610:59::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3846.25 via Frontend Transport; Tue, 9 Feb 2021 12:39:08 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f922750b-e646-4d8f-e316-08d8ccf7b274 X-MS-TrafficTypeDiagnostic: BYAPR10MB2600: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: HOmcC8u9b/00dXrA2hMcTceAEeVfUw02xtOIE320htcVaHreyEfMYR6dmZ5Oo5f1LZiauNMEkpB1hgNEN1OjkkXT/q1iNLjzjIxtvXXgzbF4jchQezaq0H0fDw704hsTCu8hGqNXUNDWk9MlzqujZ6IR/RLJ853B9Y+2R0l05spCaDzLFatBm+zFdrp/by6pxCkyVjmjwtW8QgHF2/gfFiUapdlfXW2mVrQ9fzwUYXmIlYyTOpdkOIORxw/lPDAKwEV7eI4SsqkT3gSaja5kSQg07jYxK6JMaGB7IOzW2L9yAhd/KmXJ49T61c2JSgJIcoHIncEv4ZZSwVhT3Px4/243vdUAlruy6PdqeVDrZEfh1i8kjqRen/0CTyaQkXaIS/dVctVS/E/BMI2qhdTsHlZZTgVDZF1b+wQIOYFxiowQmyfaKgvBj1aEcCKv3XCqefkGzNHLNsyye+oVngv1U1cx3Ee3UvSXdYvtJza6QhTO39a/efb5mfYDA/fVTkQJcA2UAOFG0ZQVINWFWxVcTm1n8XT1s3UMRWtiJ7CfbRpoESyvg4nXw2simWniPXDN9MxXYX8/xqwvJVZLPtriog== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3573.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(39860400002)(346002)(376002)(366004)(136003)(396003)(2906002)(6486002)(16526019)(8676002)(86362001)(8936002)(5660300002)(4326008)(6666004)(107886003)(1076003)(478600001)(6512007)(6506007)(52116002)(26005)(69590400011)(186003)(66946007)(66476007)(36756003)(66556008)(316002)(83380400001)(956004)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData: i5Dop7nDRLfhzq6WUV+0LzHtn92LNBMC4h+H01mQ3r0wgUn1ahRyPLNlN4+26usSbLohJQLpO1FyrAs6RCgucFim5opLC4LhDHvGRWwZz3rEODkSItZZEis4olPTWWTM9wlQjUBvT5qdAvXm7sMu00lPOKnNgaVG82jg/IwZxPZ4Zm7Vd2Fdq/eZHvz+Eucy5eSnc4mANJQxQMlw8qRHQtQ7/7f/KgrMC8X6oWTmtYjX76duLpLHe/97koeIfIv0Ss1vZ3CCx3gW+/1Xw59fJqvPPpE7Gqi2ifu+C1DTPRz3H/VzRVfnqrwPUmANAVkktHTJMvCjjVXZeODWONmJmLg3AihNvD0v8a5S++tluC3zdSYc/5JANZt0UzNfPmLxqeRntCHKaB82g0of4CewzcF8QGldY7nNRoZxtehCkQEaiE+f0a4I6qdgLlXrnfzPp4ZXhcYIEo94jAw7UN3Ndu/wH4a6CrrOvIbuZqwRDQ28W1aAs0CMpAazU2Kdt42EV8Np1Ioz0oyROZta5c0QLcCNfHaUOlXPywYkaBMD8SO4XD9w9+++EI11kTbAMVVl7ZmCaGxERsChNjSq2r5P1j+A87xHo7nAftBsjD49Yyutad8IQvFnW0Hm2FmVnuRRKCgaC90X9FKNg7zBh0zwsrCtpKiU3qQz0fn0HADZwkJewFkx3XOhGxe7f/SSGivyVclub/gpkO+VEHSzacD1pcc3V95v3q5jN5A1/HCsFimbrAhq3dFD9CaczbBOpRUVSfxLpnxGRt5MnDGxrMoMmZ7mX7KE4DAg4pyk4hJtL2cUWORsiBEZIlOkjGhleCx/tKE2kc1uxFaQy88f3KSGHRd3f51vIEj1km2w1C4m2RYLv0q8VqVhp00UgvtQQbmrzsHk9a4DiKCyMsCY3n2O9o4Dq8Jvet072wjQUXlw1gkOufwHjCNLdCLuzsuCxy89nh8HTOW9qsV7X/rxYL/Iv/NGmlZjsrdJycsrouZ/PdVGSQoV7/BCpXDBcdd6T+g+Y3y6P7998z7oCdR0jxjtjrDwm7ZYVUWiwTCdz4Y7nqobDaqTfZvLc7bNcBsdUDkyZy00/Z1AONf/6lRkAtpme9sDgWzL4ErwHbZxiP1BbClAWBQ5GoF6ZyNCaN4fqTSdD2t2FmDbcEhsRCkuOuci5qkhvOuU/qKESNX4gWlsH4XQRzr70YulcLMaqg1RUFk5UMzaWuXIf/vhY9FDnf9jXLODwgLZUO17OiZq7DRJZoT+dwZrTdM1tDnAIEP9IgZ4Vrw/ZCVqCKfgEU7hNhpFPmQ4nHRLe6Sh026zjoH80qWt+ZTgKtx/UZ+0HewOv8Uz X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f922750b-e646-4d8f-e316-08d8ccf7b274 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3573.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Feb 2021 12:39:09.7637 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: k1FmfDFEXfzdypVT5vsG1c1ctjSeRMAFRdje4/7hBHcY7NMcFoIOKS5TyMn8Qwr5qd+8eMuJGSIkXWMLHC9o5EBcnJbN6MOHh5J3a3cfyZI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2600 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 phishscore=0 mlxscore=0 malwarescore=0 mlxlogscore=999 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9889 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 priorityscore=1501 bulkscore=0 spamscore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2102090066 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org If a cmd is on the submission workqueue then the TMR code will miss it, and end up returning task not found or success for lun resets. The fabric driver might then tell the initiator that the running cmds have been handled when they are about to run. This adds a cancel when we are processing TMRs. Signed-off-by: Mike Christie --- drivers/target/target_core_tmr.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/target/target_core_tmr.c b/drivers/target/target_core_tmr.c index 7347285471fa..9b7f159f9341 100644 --- a/drivers/target/target_core_tmr.c +++ b/drivers/target/target_core_tmr.c @@ -124,6 +124,8 @@ void core_tmr_abort_task( int i; for (i = 0; i < dev->queue_cnt; i++) { + cancel_work_sync(&dev->queues[i].sq.work); + spin_lock_irqsave(&dev->queues[i].lock, flags); list_for_each_entry_safe(se_cmd, next, &dev->queues[i].state_list, state_list) { @@ -302,6 +304,8 @@ static void core_tmr_drain_state_list( * in the Control Mode Page. */ for (i = 0; i < dev->queue_cnt; i++) { + cancel_work_sync(&dev->queues[i].sq.work); + spin_lock_irqsave(&dev->queues[i].lock, flags); list_for_each_entry_safe(cmd, next, &dev->queues[i].state_list, state_list) {