From patchwork Thu Sep 30 02:04:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12527117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2DB15C433F5 for ; Thu, 30 Sep 2021 02:04:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 16B506141B for ; Thu, 30 Sep 2021 02:04:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344407AbhI3CGV (ORCPT ); Wed, 29 Sep 2021 22:06:21 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:46338 "EHLO mx0b-00069f02.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232383AbhI3CGS (ORCPT ); Wed, 29 Sep 2021 22:06:18 -0400 Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U0LepK026125; Thu, 30 Sep 2021 02:04:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=SoplooHbn6j4ogQH190X7aqxDfz8vcfRkXJpQKW7JAg=; b=pozzxHDM0VWFXVT0MHXAehcbhmTVWiNlMEGH2U9TK+KRLnRgpK9511guvBPRPCJYVC0o tzpsjhMHCiHKF3L43UM+mg40owKSK0OavcnQEAmHTWdEVMHj1B8P81SR1+JM45JVc+2H ay0TN15FC4lnI2YrctLHEtdKeprOUY4D9UiI2VcYX93ZqUVM1X8YXhfSs+qOJflBKf2X JGfZNfepDt0hGT3gtmhqT27PvSa+jWr/NJP4HxTISjaw80bV/7ygU33ACbjopSQKP2F/ 414sgRV6sl5z/Ii3xKk+77+BGu57fklRyB4HtsYxVYvtHDldR6cPWeK9pm+mcjEdtlaS mA== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3bcf6d099w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:34 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18U20UQf108324; Thu, 30 Sep 2021 02:04:33 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by userp3030.oracle.com with ESMTP id 3bc3bkv6f2-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:32 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Wd9lpV+xU1CG7fL2KCrUFzgCOiogGhp1NUC72c4BAkQJMi8gXOw7WX3PBp4tqKyrcJyXCtwRapYks3SLDzW4SY9q/AzpzouQxdFsJGo3/obxFD568XXzlhopQ+uUu8Mi4tNEXc2okruUvVX/XXXoa6yMB48Fs84b+TGtbTtWKcFQTDmFs1LLV6318s5mCwvSv7agcN65Y1G345i2+ocU4eWW6v4Wa6/a2CmvAw1glHuFRCXUyxi67yTnbqk8ht8QvNizV6lZc7AnM38nkW/PzBlFEltmK115QP9MKD4CnTYbCPehwffO3zDqCEyRZcLhw89TljHN20WLtQ7YSAA8fQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=SoplooHbn6j4ogQH190X7aqxDfz8vcfRkXJpQKW7JAg=; b=K5cIPBVat8vbbIXJzwzzMiqfWP2cV8PkVm/g9z1B+hxMuF2Juwdq6A0hDbt1w90bESNGNP4kZehFYfSfkKyzu1WI/jAeFGRM233WXP12FrXm6CBGR4Mqhz4+CfjHGiZS6bSk9/wMRHtDM3y/93SjBM43jRE1COSh/hNJ8hcZV1LsEF0l+alPHnshUsQnBm2xGwBP5dBtVCdEuBR+Vsasp+Yd50orzBx33A2SG6Nqxf1sFyN1IOqIGAk1Jxp9FJD+j0CSGD0bUuAG2H+uEh881D43NXzdtqz7V5JLDCpp9N30QH9KDQlDwByQ70a7J66ioL5avfCcJO8O4cJrOkGsAQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SoplooHbn6j4ogQH190X7aqxDfz8vcfRkXJpQKW7JAg=; b=pW7h+PN9c20RpAUuK59v+LUa9IQX8S62VoRD+LdMYwSMGRCY2xBST4JVqfoyueYg9Y48FSpSbFPK1QqIfhzAs4UgElnPwUn9GQY8sECw72x+nfPDk3Mw2Cd8y1Ue9BMpUhM1N1q6GdNh4Prb3jk+CDJ8IudpRwqg+qC7j9urmX4= Authentication-Results: oracle.com; dkim=none (message not signed) header.d=none;oracle.com; dmarc=none action=none header.from=oracle.com; Received: from DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) by DS7PR10MB4909.namprd10.prod.outlook.com (2603:10b6:5:3b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep 2021 02:04:31 +0000 Received: from DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531]) by DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531%5]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021 02:04:31 +0000 From: Mike Christie To: martin.petersen@oracle.com, james.bottomley@hansenpartnership.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: Mike Christie Subject: [PATCH 1/5] target: fix ordered CMD_T_SENT handling Date: Wed, 29 Sep 2021 21:04:18 -0500 Message-Id: <20210930020422.92578-2-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930020422.92578-1-michael.christie@oracle.com> References: <20210930020422.92578-1-michael.christie@oracle.com> X-ClientProxiedBy: DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) To DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) MIME-Version: 1.0 Received: from localhost.localdomain (73.88.28.6) by DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 02:04:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c98598e7-af74-42d8-6d80-08d983b6a443 X-MS-TrafficTypeDiagnostic: DS7PR10MB4909: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3513; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ipKZcdpLUQTkBamRvwqhCgeg66MaG/4WAt7ACs7kn45Qx9+Cj/cU0V2RMNMDlnnbTLCnWmycxOuh9FBwXzOL2lDN1vywK0+lo8pyiHofOO0defQCeKJzn2EoubExoGK1kvjQok0kpS1SUgAH+UZjJBusJgyhLYwmtn9ne3TURxWj3e+64xlEoBbjdWkZwkLK4rmMO3vJMHOczdiVkJUXRW+hwHv2jwr1ZM9QPoK3MmPeqznJTw6mHGRYzING/Cmjw9vi3mBB3X9fYtf1hPtWqT2oITW1D8DfEzUma0dKwTtjmJ3lNPBcpwGlEhqrm3eoKVztKCQDqiBMUL23QF44hf6DHxlcKH7J/PJIHfSkII6x9D9giFKp/fb8cbWGRS54OwigSZoacjvTTw3MO/Pp6CSYyrIp6w3D4WNBXcD5FCyduTDjZ3wttbnZIqYD6cPCniAYzhv2jBLZS1hXIa/ztlKmeaOQJqDW6f+mbvmmUebXM6acEhXlq2PSuY36c0kbeT55qXR+N7b98vw0ZtWIkI3hsG161NSTzXdlgEG4oWeKahoSl2YDWy2zjS9ltQxF73nQLwigGiqCqCRATRiLhCTpiRiRcNthOlKFrt9QeT1uaIa7pXQiPDO6dVpFFh0dUP0ukPi1Mb4anEbtGbtflD/wZ+zWWc266lDCMHc1Fjh774B3MmM8TBYwPmOziREFT7p3pr2iTdPJyNsGNkxm7w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1466.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(38100700002)(38350700002)(4326008)(6512007)(6666004)(6486002)(8936002)(83380400001)(316002)(6506007)(66476007)(1076003)(107886003)(508600001)(2906002)(26005)(66946007)(52116002)(5660300002)(8676002)(86362001)(186003)(66556008)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: WH9/sDj+EE54lb+ft9DO3SeHkiS1LGqd3Xy0GiyPGgHVRAfEPi2f8FwzM1OglxmgQkeeTI3ClewZTzC5DF6DgLdFSkUJb6G6/RSci/YhSPHW5rC7vmMqCDdhYpaJ/EUyaw78smEXImx/qwTERr5jLR1F+6nbBIl2E8qviINorch0UKbK43H1oiGVDnRtKXyyTxXNs2aLZW+JUdll5lxgjHOlUFFf7NLelhei4VFVEUAWzhE4Yy1wgsplDRv6E1CUkRwlpFgliP9Bz86DM7I7kHq7NCqfi691d9PezkKjK7UPnZQAmVyQDx3umOme6+2cMndpYRtNSmT01VwP1jLAfvMV34CUXh11GLSXtSVsN07mgsLVmI5BFnxRi2tCUDrXlXamBOIq59+AWSxLiYM258uVBqv3t61cOXZ+MPtHGPPUDUeylRlyUGsPJa7WHObszLSpRhoHA2dM0PPEaBjQbRgj7yAWP0eznUly10VN+FlSRPphEjkikvZ1H3Oq3ayvAiRYn8oLohl0SCz9Z47ofseMrHYm/9N+/tXNZY4mVsHn8g0LSIUrtxDE02gVImOOpy9VTVEJ2EoNY0l3uVsW1u5HNmtagHjz6daG5Hl6xPTJxFa0YG4iBXGf4yvz1/oE+sIuB5QmyrHNnc0UkNoFXJjSWSJ7vlIQEn6eE4MKt8U2kaGNvl9aPUDdbmYKlcQ32bugKeZVy33yZuzDlz9fJV55ClDhvmvUrEchWQIAY2CPC55xybIPy6l4fKF+xldtm2ipDiKRzosyqgQqevWI+jI3PynTUAutDNPoGXbUZlrx0iwIrPQrRLs78JZr9ccNCejbMFE21ZB0dAeQH5KbTKju5hr2CseINByrE28y6lS8QsDWNvkXLZdqCeRcboRrZM/Kt+ByYU33ocDbfg61AXcgFsxroa1Bff/CXgdgUZinKbHM1H4rMEPRb9ETD9n1HKUT4d4XnIx+jEDOUxICPrWvj86WOe1BQfo3pnzlR9NOiCkmDWNkKB06OGdiaNsXLSuYzHvhaisukp4C6//LecQAPtawWmkOPtsjITxChKEVzcP/I4EvIK+3m9qOaLt9Kr+8UBNGtCtQtIGImkVDQBotHKZYsjfpVhONLYDcSxVDpXJ8sz9l1TCtxGiyycrkTXHKY+bpR0E6d/sHMg59Z2gpWTcw+ChHlfhszvqGEDTEYRqZMYniwqwBDrI47IUXBiYBN+Q9uhAwQKAbPWwHspMQoPWBfY+9EgRYgpKKN4EMJDRDx/R+FetSotrX7zVRbQcP4MPfWQEAfRJakNP3ToKyloB7SHjRy2mc+7d+5iqHsx7FnEWxfLYmgsaK0p4h X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: c98598e7-af74-42d8-6d80-08d983b6a443 X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 02:04:31.4533 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Kv5V8Rf3KjHSxnELuMJyK9fZ2DmYadntde6V9Y4kvk3WLevSbqoWydzEeP3Ds8dCYJGYryiySOY8nAQD8ps04LS4H6ig5nVaPfR+kPsjUEw= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB4909 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0 mlxscore=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109300009 X-Proofpoint-GUID: qa3192nd9AQLbPIAcd8jaEvMNygSgiHB X-Proofpoint-ORIG-GUID: qa3192nd9AQLbPIAcd8jaEvMNygSgiHB Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We can race where target_handle_task_attr has put the cmd on the delayed_cmd_list. Then target_restart_delayed_cmds has removed it and set CMD_T_SENT, but then target_execute_cmd now clears that bit. This patch moves the clearing to before we've put the cmd on the list. Signed-off-by: Mike Christie Reviewed-by: Lee Duncan --- drivers/target/target_core_transport.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index 14c6f2bb1b01..e02173a4b7bc 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -2200,6 +2200,10 @@ static bool target_handle_task_attr(struct se_cmd *cmd) if (atomic_read(&dev->dev_ordered_sync) == 0) return false; + spin_lock_irq(&cmd->t_state_lock); + cmd->transport_state &= ~CMD_T_SENT; + spin_unlock_irq(&cmd->t_state_lock); + spin_lock(&dev->delayed_cmd_lock); list_add_tail(&cmd->se_delayed_node, &dev->delayed_cmd_list); spin_unlock(&dev->delayed_cmd_lock); @@ -2228,12 +2232,8 @@ void target_execute_cmd(struct se_cmd *cmd) if (target_write_prot_action(cmd)) return; - if (target_handle_task_attr(cmd)) { - spin_lock_irq(&cmd->t_state_lock); - cmd->transport_state &= ~CMD_T_SENT; - spin_unlock_irq(&cmd->t_state_lock); + if (target_handle_task_attr(cmd)) return; - } __target_execute_cmd(cmd, true); } From patchwork Thu Sep 30 02:04:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12527113 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA1A2C433FE for ; Thu, 30 Sep 2021 02:04:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8EC6861875 for ; Thu, 30 Sep 2021 02:04:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346842AbhI3CGS (ORCPT ); Wed, 29 Sep 2021 22:06:18 -0400 Received: from mx0a-00069f02.pphosted.com ([205.220.165.32]:45524 "EHLO mx0a-00069f02.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233113AbhI3CGR (ORCPT ); Wed, 29 Sep 2021 22:06:17 -0400 Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U0SAJA032191; Thu, 30 Sep 2021 02:04:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=WHARaM2rSyrT+ZENzReQKaSW2MwmQLkBeq+SyIM+kQY=; b=EBbidjhOC5dZmApfCLcrGyIBuK7MTqLTK+gbzecbgfblcDdid7Mn0CIiRlnM2Dp4aaTp 68VJR0AApx8ojCrX6EBf8tpiuk/p75U/PTwxnowYRxZvaLSImLyuG34ipCRenotIQcJQ fX7a53It/85uocT04yZD+uakXGcPtwOt9Bp3MG8v8RIQiu8E5VJDgaJQRPI/+20PaCci VhFOqyeJS8QXKsEuIjja2jCWYOqr82TA8jS3HZO+3VjJG5BRhp38dbJGgQBJrwaSJnIA tJsppuCOhaRrY26Xem4z1aWZZ6wCV7LWgFGIOkYzAQk29YEPF/UeGui/tMRq8hrbPNjY gw== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3bchfkynbn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:34 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18U20UQg108324; Thu, 30 Sep 2021 02:04:33 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by userp3030.oracle.com with ESMTP id 3bc3bkv6f2-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=H+JD4GqdOETPprHOhdgmPFyY5zYlEwgRk1INXM/w08Sxyt/JyN1gwjoHlAZku+uRig2N4MvLgkznTficCxsEg6aXjMh/Rt7/r4YiHDLubNbKDN6t3M9qVEeAblhVtGlQlCkfzeXg0yyiNWFUDfOqCuRdkSnmnVXpqQZ2d4CH+X1lctyjD9Muk6hjIG4cpcPe6t3x/qaRFtr58ZL5fltNCPY24boDqKQOhCDghRdP+ZvnhS5oN5c77xerGZiFatzOOu7izGQqRZOmBrchnst/8vv0v6OGm6lYS6o2kQZOYJTUL2xsWxpRUFActL9R/ZEb0jPfZHDzytHpuqyLU59QIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=WHARaM2rSyrT+ZENzReQKaSW2MwmQLkBeq+SyIM+kQY=; b=bOib/pFV/MnncRDNzfFoOLcxcKTniMmecMYMvZQownoyrTcqOLLT8NeK1VDTyY+p4GkdoSI8N2xDkPihzaHD8tuxNr9FsxTXZfRGC78J629kjMx7bRNXqmZCCd81S6HcFNLoOPbJnVJhSP29cE88EXdL7Z0BDmnOV+NZs82Qt+1kM6jXAKIR5sCXdCyigCPxcFmc9yOhcz1SJzBx+gq7F4foxXWxQPBDDW7ws2vBygbLOW60B4G0O2QDKtuUa0pH6ndRK+Y7Xzw28ovwX2SWwNTnw+XBqnK/oVdjl5ynsrzXu7pTrfmwHOBrhf3LK23DD27r96MXNdVPCaOPcBwwuA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WHARaM2rSyrT+ZENzReQKaSW2MwmQLkBeq+SyIM+kQY=; b=ikvudvxPczD2tsEJkOuuZyn3nQzFuGzIveI7lkObD5kO0PF5SIw37mJePNTdP3J5m0YwJQx2UeXw0wXKv2r/YcFBu+aywEoqnwaaunTA7mKvxHJO2bztyi6HaJS7vDAopvRAZy7KFXP24JwdWlR7HkS72vJXoKKHzfdTfLIoIKM= Authentication-Results: oracle.com; dkim=none (message not signed) header.d=none;oracle.com; dmarc=none action=none header.from=oracle.com; Received: from DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) by DS7PR10MB4909.namprd10.prod.outlook.com (2603:10b6:5:3b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep 2021 02:04:32 +0000 Received: from DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531]) by DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531%5]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021 02:04:32 +0000 From: Mike Christie To: martin.petersen@oracle.com, james.bottomley@hansenpartnership.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: Mike Christie Subject: [PATCH 2/5] target: fix ordered tag handling Date: Wed, 29 Sep 2021 21:04:19 -0500 Message-Id: <20210930020422.92578-3-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930020422.92578-1-michael.christie@oracle.com> References: <20210930020422.92578-1-michael.christie@oracle.com> X-ClientProxiedBy: DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) To DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) MIME-Version: 1.0 Received: from localhost.localdomain (73.88.28.6) by DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 02:04:31 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b9dde67e-6764-42c0-0f8c-08d983b6a491 X-MS-TrafficTypeDiagnostic: DS7PR10MB4909: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1tQIuuta/prdzUW31AfUxYF0UsuRBL6H9mloato/JCJMMB1PxNfrLVK/zdNPOrGl7jNp4eblmUosyIKb39G8k4RE5oHPhKHSBiWjQVWERMxOpd4oNEiGqmvsxTb2n2NuHa4b8DFgmIvY2tcyt2ME+s1P3bJA77uKMhOIy3OKkfbjjPU9i84e+sX9FHcLkGuTCe2P8uP4mv+0Zhum9r3YgIzRu74Lra7H9NNoFQou8q0IduZdguWZtKhJIUbia1+CT409qnO1tYAt7Cr2p9A6S2YtUG04qgKFc23Yd25sS+xE/1uHcgCD/iSCxlLZuQ9C8YQzxH7/5Dn3x9mnVF0ioonY3yG5gDhCpxI97yEcKqP3pBw/ykpVCmKGfEA/qX2NT536ciGaYMZD6sF5inIB0kqI0mAEyCZF0Gb07ac4xHkWqVHpqnejitz+5w0Z4WTo8APsDbxk3H096fbDe3ArZ4ObyZYtfiF6qo0AOELxbhD38/Wdz+KQtbP5VQqA8FZX1GnSKFTwkM8NBBXnrJABTTITw4qdWo1OrmcPkE+y/uMIpdTWTY8pE7EE7V6gcfA5TOisN3FlqWdmmSxvbBDVbcpkWRkzYczINrBXDiMFnFW/NaEe7I5WW07ZlI+ehnmzgj+yrzwkD75MTqswpE7fACYY+zK5eYFIDnhzC08uPsmf/tclBZQs/oiD/AjYJ+Aj7noobZQeiTPh2+H4HOicdw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1466.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(38100700002)(38350700002)(4326008)(6512007)(6666004)(6486002)(8936002)(83380400001)(316002)(6506007)(66476007)(1076003)(107886003)(508600001)(2906002)(26005)(66946007)(52116002)(5660300002)(8676002)(86362001)(186003)(66556008)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 64gs5EUj+VAbjQrX2Y1AHBl6pAFuDMR28/SXgmA4Fe+Ewc6x+n7Bc4F+FnMjiLgUoGSba4VCy8uib7AvAHNJq35s/Wz8U1BNQzO9UC+r7HMf3S8FsS8Mn1dQy9bULUtajrAgdYoHZ74Tud613H7e2MpvAN5Z4iyJvZS7uezszXpsf14eePwcxeiFMbH87QJ3d3LA5syaiPVZ1ckTVni4UL/FX9mnRL1uhA+lTaZ3C8xNN08nfklu1zNhg/30ferX7xumzk9L6e4bzLte2lC5L71JBzXwAYA6628VcoaopNF7+PjJQ20s6IaZ9m/rIbtVvLaZu2MgyeI8Sh3FRfTq2EpsLyAuVJRUmR1hahTq47wIDVovsf547GZ5cFXud8jLze6MBIGl0MLBSkZCf9rxK+5HN/bynd+UK9eogV2tT/DmcVD4cUwXFeuBQlQInDsYRe3QabI8fF5geQDASas0MttL4QzYBWho6cvJ0wdS6tAUaDRzXSiR3QtVg7dZ5jbmyNjwK425jKfW76jUTyGjK9MnN/r4aN4DURRDgl4JFKjjNcMC46ZO8AZYxnvSh04pgQllAALK/kRHWufIJBnnuzfFScFzkQbuh4SmEGsMHJVIDWzQCQCo1XgZgAcU8d7fYFoVShehDfVxM5ZqUt7X10AwFm7jnenL62EZ7vaYO66q1RgMhevt30Kt5vGsC+VHHJReors/I1nSBY6b6j7v5lTqOzhwnA1zWFeLllSc1yjprN3RteBhDdJppR1ZQdtM/XplLRjdpsRc+VLch3i52s0MOLKMPBdW/QRI2wn+NjB5q6DxL4zl3DbLWYwBO1ldWb2M4T1EiQnF2W/+DexIyBYGToEM3wmHjikZzVwVbKhyx1ukKhN5krgEDj/zvk0gjHI1Abl2XsjNV19YcDNKfJYB+dhAyjhywKPI7fl4EIEBzfRpit+svNoOwu87m6peAEx0oG8l0PEY+rI1pqbEInWU1ea6lFg5aY6pmXW+3DK3XzF0mB7RvXRZcJMy+J/8H/lpycF3Ib4ZGt2Qu9qkHblF3NgXPSdCtbZBE+zmd/Dx64f6rzUmVijOL3MonqMZCLnJ01X5BJ9Vms5oAe3wQm9vFJkKwmXlY9IpCbpFT8rtSsek2RUxPSEPd8wOr6qcE8AB4W5PGl9rghoWbe+pES6v9H3eoL//OzLc2XnS3oOXY3aj51JTLawBNOS4tThm8OVI4zJxVpe2Zbr7HSuH+ObrdFo14PZ+SXQL0+v5iizFjNi3AUWymHxI7o8jNAPVvX2AFsWgkhz8PGW/QkZGbgtPd9Gj8BNdAMtFp5hJNVCH+7eDyqXD4xilf/Ja6ciW X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b9dde67e-6764-42c0-0f8c-08d983b6a491 X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 02:04:31.9511 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9HO5j+8KcslX53eIHmVgljcBf9es22LONpfT6XSjnaEiJ0rKziRHTj928LbLAvL0CYZlB9g7MioMBxbE9hgrXD6GcI4YzgWhoCooaqhCoIY= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB4909 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0 mlxscore=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109300009 X-Proofpoint-ORIG-GUID: aQbrtIMKfllHJljSPM8b-a7udI_d0xfr X-Proofpoint-GUID: aQbrtIMKfllHJljSPM8b-a7udI_d0xfr Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org This patch fixes the following bugs: 1. If there are multiple ordered cmds queued and multiple simple cmds completing, target_restart_delayed_cmds could be called on different CPUs and each instance could start a ordered cmd. They could then run in different orders than they were queued. 2. target_restart_delayed_cmds and target_handle_task_attr can race where: 1. target_handle_task_attr has passed the simple_cmds == 0 check. 2. transport_complete_task_attr then decrements simple_cmds to 0. 3. transport_complete_task_attr runs target_restart_delayed_cmds and it does not see any cmds on the delayed_cmd_list. 4. target_handle_task_attr adds the cmd to the delayed_cmd_list. The cmd will then end up timing out. 3. If we are sent > 1 ordered cmds and simple_cmds == 0, we can execute them out of order, because target_handle_task_attr will hit that simple_cmds check first and return false for all ordered cmds sent. 4. We run target_restart_delayed_cmds after every cmd completion, so if there is more than 1 simple cmd running, we start executing ordered cmds after that first cmd instead of waiting for all of them to complete. 5. Ordered cmds are not supposed to start until HEAD OF QUEUE and all older cmds have completed, and not just simple. 6. It's not a bug but it doesn't make sense to take the delayed_cmd_lock for every cmd completion when ordered cmds are almost never used. Just replacing that lock with an atomic increases IOPs by up to 10% when completions are spread over multiple CPUs and there are multiple sessions/ mqs/thread accessing the same device. This patch moves the queued delayed handling to a per device work to serialze the cmd executions for each device and adds a new counter to track HEAD_OF_QUEUE and SIMPLE cmds. We can then check the new counter to determine when to run the work on the completion path. Signed-off-by: Mike Christie --- drivers/target/target_core_device.c | 2 + drivers/target/target_core_internal.h | 1 + drivers/target/target_core_transport.c | 76 ++++++++++++++++++-------- include/target/target_core_base.h | 6 +- 4 files changed, 61 insertions(+), 24 deletions(-) diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c index 8cb1fa0c0585..44bb380e7390 100644 --- a/drivers/target/target_core_device.c +++ b/drivers/target/target_core_device.c @@ -772,6 +772,8 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name) INIT_LIST_HEAD(&dev->t10_alua.lba_map_list); spin_lock_init(&dev->t10_alua.lba_map_lock); + INIT_WORK(&dev->delayed_cmd_work, target_do_delayed_work); + dev->t10_wwn.t10_dev = dev; /* * Use OpenFabrics IEEE Company ID: 00 14 05 diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h index a343bcfa2180..a889a6237d9c 100644 --- a/drivers/target/target_core_internal.h +++ b/drivers/target/target_core_internal.h @@ -151,6 +151,7 @@ int transport_dump_vpd_ident(struct t10_vpd *, unsigned char *, int); void transport_clear_lun_ref(struct se_lun *); sense_reason_t target_cmd_size_check(struct se_cmd *cmd, unsigned int size); void target_qf_do_work(struct work_struct *work); +void target_do_delayed_work(struct work_struct *work); bool target_check_wce(struct se_device *dev); bool target_check_fua(struct se_device *dev); void __target_execute_cmd(struct se_cmd *, bool); diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c index e02173a4b7bc..913f31561531 100644 --- a/drivers/target/target_core_transport.c +++ b/drivers/target/target_core_transport.c @@ -2173,32 +2173,35 @@ static bool target_handle_task_attr(struct se_cmd *cmd) */ switch (cmd->sam_task_attr) { case TCM_HEAD_TAG: + atomic_inc_mb(&dev->non_ordered); pr_debug("Added HEAD_OF_QUEUE for CDB: 0x%02x\n", cmd->t_task_cdb[0]); return false; case TCM_ORDERED_TAG: - atomic_inc_mb(&dev->dev_ordered_sync); + atomic_inc_mb(&dev->delayed_cmd_count); pr_debug("Added ORDERED for CDB: 0x%02x to ordered list\n", cmd->t_task_cdb[0]); - - /* - * Execute an ORDERED command if no other older commands - * exist that need to be completed first. - */ - if (!atomic_read(&dev->simple_cmds)) - return false; break; default: /* * For SIMPLE and UNTAGGED Task Attribute commands */ - atomic_inc_mb(&dev->simple_cmds); + atomic_inc_mb(&dev->non_ordered); + + if (atomic_read(&dev->delayed_cmd_count) == 0) + return false; break; } - if (atomic_read(&dev->dev_ordered_sync) == 0) - return false; + if (cmd->sam_task_attr != TCM_ORDERED_TAG) { + atomic_inc_mb(&dev->delayed_cmd_count); + /* + * We will account for this when we dequeue from the delayed + * list. + */ + atomic_dec_mb(&dev->non_ordered); + } spin_lock_irq(&cmd->t_state_lock); cmd->transport_state &= ~CMD_T_SENT; @@ -2210,6 +2213,12 @@ static bool target_handle_task_attr(struct se_cmd *cmd) pr_debug("Added CDB: 0x%02x Task Attr: 0x%02x to delayed CMD listn", cmd->t_task_cdb[0], cmd->sam_task_attr); + /* + * We may have no non ordered cmds when this function started or we + * could have raced with the last simple/head cmd completing, so kick + * the delayed handler here. + */ + schedule_work(&dev->delayed_cmd_work); return true; } @@ -2243,29 +2252,48 @@ EXPORT_SYMBOL(target_execute_cmd); * Process all commands up to the last received ORDERED task attribute which * requires another blocking boundary */ -static void target_restart_delayed_cmds(struct se_device *dev) +void target_do_delayed_work(struct work_struct *work) { - for (;;) { + struct se_device *dev = container_of(work, struct se_device, + delayed_cmd_work); + + spin_lock(&dev->delayed_cmd_lock); + while (!dev->ordered_sync_in_progress) { struct se_cmd *cmd; - spin_lock(&dev->delayed_cmd_lock); - if (list_empty(&dev->delayed_cmd_list)) { - spin_unlock(&dev->delayed_cmd_lock); + if (list_empty(&dev->delayed_cmd_list)) break; - } cmd = list_entry(dev->delayed_cmd_list.next, struct se_cmd, se_delayed_node); + + if (cmd->sam_task_attr == TCM_ORDERED_TAG) { + /* + * Check if we started with: + * [ordered] [simple] [ordered] + * and we are now at the last ordered so we have to wait + * for the simple cmd. + */ + if (atomic_read(&dev->non_ordered) > 0) + break; + + dev->ordered_sync_in_progress = true; + } + list_del(&cmd->se_delayed_node); + atomic_dec_mb(&dev->delayed_cmd_count); spin_unlock(&dev->delayed_cmd_lock); + if (cmd->sam_task_attr != TCM_ORDERED_TAG) + atomic_inc_mb(&dev->non_ordered); + cmd->transport_state |= CMD_T_SENT; __target_execute_cmd(cmd, true); - if (cmd->sam_task_attr == TCM_ORDERED_TAG) - break; + spin_lock(&dev->delayed_cmd_lock); } + spin_unlock(&dev->delayed_cmd_lock); } /* @@ -2283,14 +2311,17 @@ static void transport_complete_task_attr(struct se_cmd *cmd) goto restart; if (cmd->sam_task_attr == TCM_SIMPLE_TAG) { - atomic_dec_mb(&dev->simple_cmds); + atomic_dec_mb(&dev->non_ordered); dev->dev_cur_ordered_id++; } else if (cmd->sam_task_attr == TCM_HEAD_TAG) { + atomic_dec_mb(&dev->non_ordered); dev->dev_cur_ordered_id++; pr_debug("Incremented dev_cur_ordered_id: %u for HEAD_OF_QUEUE\n", dev->dev_cur_ordered_id); } else if (cmd->sam_task_attr == TCM_ORDERED_TAG) { - atomic_dec_mb(&dev->dev_ordered_sync); + spin_lock(&dev->delayed_cmd_lock); + dev->ordered_sync_in_progress = false; + spin_unlock(&dev->delayed_cmd_lock); dev->dev_cur_ordered_id++; pr_debug("Incremented dev_cur_ordered_id: %u for ORDERED\n", @@ -2299,7 +2330,8 @@ static void transport_complete_task_attr(struct se_cmd *cmd) cmd->se_cmd_flags &= ~SCF_TASK_ATTR_SET; restart: - target_restart_delayed_cmds(dev); + if (atomic_read(&dev->delayed_cmd_count) > 0) + schedule_work(&dev->delayed_cmd_work); } static void transport_complete_qf(struct se_cmd *cmd) diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index fb11c7693b25..2121a323fd6c 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -812,8 +812,9 @@ struct se_device { atomic_long_t read_bytes; atomic_long_t write_bytes; /* Active commands on this virtual SE device */ - atomic_t simple_cmds; - atomic_t dev_ordered_sync; + atomic_t non_ordered; + bool ordered_sync_in_progress; + atomic_t delayed_cmd_count; atomic_t dev_qf_count; u32 export_count; spinlock_t delayed_cmd_lock; @@ -834,6 +835,7 @@ struct se_device { struct list_head dev_sep_list; struct list_head dev_tmr_list; struct work_struct qf_work_queue; + struct work_struct delayed_cmd_work; struct list_head delayed_cmd_list; struct list_head qf_cmd_list; /* Pointer to associated SE HBA */ From patchwork Thu Sep 30 02:04:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12527115 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C3E8C4332F for ; Thu, 30 Sep 2021 02:04:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5F10D61875 for ; Thu, 30 Sep 2021 02:04:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233113AbhI3CGS (ORCPT ); Wed, 29 Sep 2021 22:06:18 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:46286 "EHLO mx0b-00069f02.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1344407AbhI3CGS (ORCPT ); Wed, 29 Sep 2021 22:06:18 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U0ETBJ017460; Thu, 30 Sep 2021 02:04:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=yDS1OxcVp8ly10V8vePxRfPpns6A/RLsJBT3Y0aBIiU=; b=HXY4GOhDzol7RUFfQMQocpxrlJFZHznt6fyicwzZXWk7Z/BV2hDcy3cogkBY7X/9mPUz NmvSqyrIHRmzaXQpaQrsAPvXR/QsQhHTE486S7f9606SYI41z1jol6rgX+RRf/dJrgpd R7ikqhwobWaABGZVRJyOlAyjDDTbOimIMxpucJtvjFAMekqMbE+0Kne1krL4swsoC+rP q73dSW5hCw/qxuYgPFOwfBJrOgZgsriJqtFoFnbLRkzmgivtfqJ8rOrOhNdGfTDlB7ZV WysbZ/J2/u2fTBqYMa2veV8SrkU0eFSBpMpcd9U9ydC+P0ADMas5Qj1UKWx2sqYwdcuW Bg== Received: from userp3030.oracle.com (userp3030.oracle.com [156.151.31.80]) by mx0b-00069f02.pphosted.com with ESMTP id 3bcg3hswrf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:34 +0000 Received: from pps.filterd (userp3030.oracle.com [127.0.0.1]) by userp3030.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18U20UQh108324; Thu, 30 Sep 2021 02:04:33 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2044.outbound.protection.outlook.com [104.47.56.44]) by userp3030.oracle.com with ESMTP id 3bc3bkv6f2-4 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:33 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YFfDpo4tTgqVYnwFLNFxAku5YAGfY7N5oCKcRg8Xs7EsqDjRu5W/lrzOKePYHgYvt5HNnXdPpL/TudAj2KjxMw/ekHAvj7cgWgF4nD9DwdLJ3XAVV2yj7+DTs52tSkPFnC7b2kDrMU+DfiLtlF0zHfNkrcf3WtGG4rEQKdTSKOtUj2L32B4mAIl5xiqdE77uMaFxIvcCPH/h8meFZSGFgALbbWzEuGi1grWWKQmAJ2LKBwzJlkhVeavrMXDBbJwTlQ96DhNve0hHm3y4pi83aHigabLzSMeLl0Uo/4CdCXt8r2xBD72LNU+7pym+BWkR20vVmL5PmVcundwfSEHoMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=yDS1OxcVp8ly10V8vePxRfPpns6A/RLsJBT3Y0aBIiU=; b=FN4c4bRfImDsBQGhFfpCQi1evgoU/l1f/RAaBmMQZW0LDuWVeRCRIasjpjgdmG66EQDs4Le6WovqUOvts/nKM/BPF/mrvpRnR7xanPXBUGkUUarPEMasF3MCJiTpXQWq/7GtfiAUPHPLcmziTwl2JKYVGKsz777D+nyTLNK0LmCHuNy7dI6QS17iWmcvcgecQ6ap0QJF94rXfyTH2lu8EZCFX1dGTGuU6hclrysVW9GhQGsNQXUd/T92NV+f5HydUZBr6jmnOXmHjfUWcy9JW4YGHr4PkRhV9/2/18L7K4apdR0WgQYLc9ApsK+L1esPyoEjdhUSnvPd+AsCsLXWtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yDS1OxcVp8ly10V8vePxRfPpns6A/RLsJBT3Y0aBIiU=; b=OjCmsYS4gXoTN/RwdNg5ZJ7BRsSFugGh9mP5gsBKBPolMn2kyOfYSWYnnLUa0UlXQWSlmFFhjtsr7Q9cD3xg3dlxLFdK94NQTHo1szAGvPLfvbNUo4YMAsrYxMoNKghJlBMOl1142GmtarXI3HD6x4zFyIGcTwGZz1tMNUzd+is= Authentication-Results: oracle.com; dkim=none (message not signed) header.d=none;oracle.com; dmarc=none action=none header.from=oracle.com; Received: from DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) by DS7PR10MB4909.namprd10.prod.outlook.com (2603:10b6:5:3b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep 2021 02:04:32 +0000 Received: from DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531]) by DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531%5]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021 02:04:32 +0000 From: Mike Christie To: martin.petersen@oracle.com, james.bottomley@hansenpartnership.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: Mike Christie Subject: [PATCH 3/5] target: fix alua_tg_pt_gps_count tracking Date: Wed, 29 Sep 2021 21:04:20 -0500 Message-Id: <20210930020422.92578-4-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930020422.92578-1-michael.christie@oracle.com> References: <20210930020422.92578-1-michael.christie@oracle.com> X-ClientProxiedBy: DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) To DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) MIME-Version: 1.0 Received: from localhost.localdomain (73.88.28.6) by DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 02:04:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 21ba12b3-e7f5-4657-cbb4-08d983b6a4db X-MS-TrafficTypeDiagnostic: DS7PR10MB4909: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8882; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MZbRlycx86dHXgDioiSMqonCtrdhWAqWKF3dRwyxIUCKUrkM90evKU+6agD7Fu/WT0onfe1eQxWPhQotu2+Yge2K399joLVNJSearL5CoxRmn+Tn6R65pkUGCbx1TzQRBsWcOszjqyBr1/slnnQmjTTScSFpBb+rsJG7Ay5DQMvzexE4v9f5HV1Yu6Q10PYlubzfdiPM+Ob3lu5M1jXRaNMbDf94sinjfPxUaGIokXJdyhIQSsk3d+8i/FdAcRUoN/zsmzYNbkFknYKYjVRjSyDuFOYiYDCkN4dWvspphybOru/HQSnVsPIySs0+nTtvnEkX3woj4JD2DucyHJFYx9KXdG5s4qnsaqJVFzn1FcKC5gTRZlua+tb6SLtIHYKBXq5TF5X7djo5j/QOb9eMOCq6ShJ3WrOmomZmINXsgKu0MSq2cLbCLBYLpkY2OrlCjL7wsOs+Mn0FB6SNpZVVYAx0bKVbGV19xDkhOo/92MinXWeEnM/DeAu/mdHyGkM83lB4qVb/4c/djUSP0piScWltM0V1092huPDSuhh/0+3v9WUxgwVXcEeGs8Mb33pchHSNNBWR/0AevzRIjJVlT3pSG9A//NfJ72SmvlDBqNLEsuFNsANXXE6K2k3GlYEWv/ZcL/NyiIkWuak5E1EHMCMGqxVNxm/p6hcemd2a0c0sXsFqWjWIJx4Mb0amS7sAAsFDNoCyHCwQ81qBkLCbNg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1466.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(38100700002)(38350700002)(4326008)(6512007)(6666004)(6486002)(8936002)(83380400001)(316002)(6506007)(66476007)(1076003)(107886003)(508600001)(2906002)(26005)(66946007)(52116002)(5660300002)(8676002)(86362001)(186003)(66556008)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ifHm/ySwstZR/hD67asokfQrWehHEOB1LBb4qsCxGJrxi27SaE5bp3j1JSAK53IwwpYxSSuoBhY9A5QYLqWT4vmd3MxSVs7EIeauJzvO3ltWJvlA2sOFjXp01+8Co9Zv53ic7sct9WOr2GImmZ8Vx3kDcmQLGxuvEG8mo+OkI3uH87+YHdnTXTdOA4xgO2UKkDvJq6F2oNYOH12ffLVAEajfbcTIRosXwMSr51V8TMJCvYh+vzEUr2ScOdlRtVdQlStpf81cDYiZk8hf9YBApojXdAkS2QJHms8wjff9rhQ80kaR1YIJcoiKB0uVguFyBzxp2f3RzrkxA9MRV7P+UdJjJ1FHuCYOvRsaNVBYlr8l/Wgl6TkzPcGr0YUU5ILmsIMKCMpirJI6MQZ6067+738Yh4KvWm+dbAW0lL2CmJd0jypFW0RdnVeQLFOKyKbncjBXhbBTEyrOqHlRV+IvAURUQBS2w1dmL9kJs+3zsdRhbYdWRDrwFYotDkpw7WNGbV5u4WHU6DAC7o0UTXc4kgECpem6wjpqAVPujvufU83hWIqh2eRiwdxTCh4vukVsqsFNARGuRMuhuSohmecmEBnVORRakml1veXYbNz3U6lKlrxNLv+Ep+upApmDrxTWGx0lwztgs4YRSAMEDDGtnJ4ycIRw9M/LE756Q8GdUWfn8b2tYfvkzuL8CNXfmRtDxqlBMeiTOABIX8rpEQD9+L1pr7Bs9tUi4qJE/7uLIVe4ZMU0K2ClfbTsCxplh3nWXpfya8KKj9j5JML+JlZJ7D/B1SRnG/XnwSDUcGC9FSNvd1z8FpFQ/9yQa8fwlLg1vtcoP9hTUfpppUD7YIcPi8SbLssslhQrrQcNhc4EJnJPOd+95rQ8YoxtuIN+CEl0Yk7ZxZC2yXXTimLZVYzAqri0TJY6HnBvbVmkGZLMpbQVP2V0Cqrkf5XpUW7Tw3E2H/SQ9SRv/d203XCwpokjAgAmi11bcmXl9vFOWWdqrwFbXLKZC+MpCyfoy6lx5BU15GIBOO6ABRvjeMrCSCN52ixkhKxYpNzYnLbfklfzHQNmLaFyIwglMEe+dnYepSuBQpyRCQeoxJ7k/cSypwZkCSK4LOFYeZeSzgakeFm58jXdWdsL3h8aEt1au+QjB+nQoAYK10V2J3ZQpNdQdpdIaB8STiGXNHsUZyRDoBI/NxCUW3CgtSZzWIym38T9+5QVp0vkUZA0uxHU/uueeN70PAzNFCTvWx4xX6POfD/pcOZ49+crLuFxmULRdGSja6ElCWFjFmMTxwoI0h9y6HlTP4u3cD+Q5+r5Sd+pYR79hDKAb7chaNS7+8Q/Os1UZjMZ X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 21ba12b3-e7f5-4657-cbb4-08d983b6a4db X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 02:04:32.5689 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0OI8TL1BR/BAxq9GsNVlPkhcYgVgIH31IrD87TarNH1oQ97lgEmbpDhGzBGMLqN3KankPtXbiq4k2UL6GqaQLZoKNDNyuqANNpBwGfpJhUM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB4909 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0 mlxscore=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109300009 X-Proofpoint-GUID: W7XJDD1_ymQdPJNd-zSz_VAVevfgl00x X-Proofpoint-ORIG-GUID: W7XJDD1_ymQdPJNd-zSz_VAVevfgl00x Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We can't free the tg_pt_gp in core_alua_set_tg_pt_gp_id because it's still accessed via configfs. It's release must go through the normal configfs/refcount process. I think the max alua_tg_pt_gps_count check should have been done in core_alua_allocate_tg_pt_gp, but with the current code userspace could have created 0x0000ffff + 1 groups, but only set the id for 0x0000ffff. Then it could have deleted a group with an id set, and then set the id for that extra group and it would work ok. It's unlikely, but just in case this patch continues to allow that type of behavior, and just fixes the kfree while in use bug. Signed-off-by: Mike Christie --- drivers/target/target_core_alua.c | 1 - 1 file changed, 1 deletion(-) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index cb1de1ecaaa6..bd0f2ce011dd 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -1674,7 +1674,6 @@ int core_alua_set_tg_pt_gp_id( pr_err("Maximum ALUA alua_tg_pt_gps_count:" " 0x0000ffff reached\n"); spin_unlock(&dev->t10_alua.tg_pt_gps_lock); - kmem_cache_free(t10_alua_tg_pt_gp_cache, tg_pt_gp); return -ENOSPC; } again: From patchwork Thu Sep 30 02:04:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12527119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E38CC433FE for ; Thu, 30 Sep 2021 02:04:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 769226141B for ; Thu, 30 Sep 2021 02:04:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1347438AbhI3CGX (ORCPT ); Wed, 29 Sep 2021 22:06:23 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:47956 "EHLO mx0b-00069f02.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1346934AbhI3CGT (ORCPT ); Wed, 29 Sep 2021 22:06:19 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U0C6DK017464; Thu, 30 Sep 2021 02:04:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=lByVmpQiT7Xcjm6HY4ceH48atnToinN4nMJT3IH9wvo=; b=ARTY2L6QztrnFdkUGO6zrLPFyNrebC57IQSO/LIYnat2T/fDpF6MI2NcuiKRV/v4+VdM a+M5DFRL7wSjmxEASvqxM6FzLKfcj5havPqHCjB3fxO+wFWfhPIwQfqRLrB/QMxaslQz EWxylU2e1iR7TSeSxWk24DNA07DYh7hMMFXsR6nic+R2Pj5JgSFjhO5Ddf+s+e8NFrpU WAhDxD3fhxKaqUI2g9squd8obmylSzQGBZEQBXuZUWiPUMkjJoKAUfLVlzQBMhF2A2m7 F1q7SDbeI9RvKIVW1bY6fm2xS2VnFjDoIqoJAdcocsgRy/pfhY3M4iXVqab7s5hMQhVA Kg== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3bcg3hswrm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:36 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18U20Fu0160315; Thu, 30 Sep 2021 02:04:35 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2047.outbound.protection.outlook.com [104.47.56.47]) by userp3020.oracle.com with ESMTP id 3bc3cf5ava-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ZGKJJ/zO/BUc8QR/5fPouzUOk8Zuu+qp7nLp0oKoo1FHdVUOHnQ5yeBIuW2RhaxsJ2o37BGh1zKIjyUoFLOgDWN4cUda/2C9CMZ4xCOGgkfSYUoVds5gUbka9tlNGsMdzW/nQK7Hay2d7P1FLEx0ssFBezkuE4H+77jKsnNEEuJ/CZIpSfCjCzpJ8LMxn1EJL/QRjKE6lzPdMp8c4MAw5jteya/lPLMCLIpmdnm9AUu8D9Ovym8W2G6Yn7pRhlrUOfY2SLyqN3uwxwAH2rcW18/H1PpVja06LWnY9SYmRiWd/dgijdyKAvz5oy2DrshUvB90aT5SQYdW8swgiBLrUQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=lByVmpQiT7Xcjm6HY4ceH48atnToinN4nMJT3IH9wvo=; b=On+63doDbUqqcoWgFsMHQLNsRnQBCU2A0D/Mv458oFwJGmyQgDMmPRkL7LV7iN+1j7jxt3SMe5KDtQpq6h8Qi34O9y9m9dEPjfgrffBY9IUvmk1K2YMQGG+nDxA2W1FG+o7VzMwaUf8kOxWYQqv/UQMavwNlMZ1E8duKJpbwi46SRQkdkWNiLWaswqwsFmc/XGWGxNaj474m56F6RLwb7fGG9DliusLnrxRIgFX4nz1pXesZ8upl+4ZvxHG+9gnCbXvKl8DQ4sUW0c96qi0CZ1SH045nC89Q2nm3aCpiBke0mjNm49186/1BRdwlYBZIeN5jnJylKueZUB+qdSmxxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lByVmpQiT7Xcjm6HY4ceH48atnToinN4nMJT3IH9wvo=; b=QO5TF1yDvPpJKqDyfDvt8nQ33J1eor7fXLkvxmAvnM+VIj3/Gp60HCNqUDn+d2RgJ7tq8somIBq3qdjEI66LOSoYnVszmIW4CJAhDA2dWe03l2UQQ991Eo+p9yhyLTiPUFHvbj13hY+tXybx/TfkQRdVoRn/Eis3eK/+fzYATF4= Authentication-Results: oracle.com; dkim=none (message not signed) header.d=none;oracle.com; dmarc=none action=none header.from=oracle.com; Received: from DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) by DS7PR10MB4909.namprd10.prod.outlook.com (2603:10b6:5:3b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep 2021 02:04:33 +0000 Received: from DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531]) by DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531%5]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021 02:04:33 +0000 From: Mike Christie To: martin.petersen@oracle.com, james.bottomley@hansenpartnership.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: Mike Christie Subject: [PATCH 4/5] target: replace lun_tg_pt_gp_lock with rcu in IO path Date: Wed, 29 Sep 2021 21:04:21 -0500 Message-Id: <20210930020422.92578-5-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930020422.92578-1-michael.christie@oracle.com> References: <20210930020422.92578-1-michael.christie@oracle.com> X-ClientProxiedBy: DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) To DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) MIME-Version: 1.0 Received: from localhost.localdomain (73.88.28.6) by DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 02:04:33 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6729f218-bdd7-4d39-4d34-08d983b6a564 X-MS-TrafficTypeDiagnostic: DS7PR10MB4909: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2MGlEp0lT7CM9FeckobZq13wF4c6co3JdJ3iokh16OMhbTq9hmr/S2bC6imuWKqLcmHwwmlqGD6174SFaF6qgc3KTzQaA4mU2QSjO++LKAa+I3T7mLzPr2Kn0jeOG8jkAqu8xMQ4nB33xI6+UnniucFN8yH5W6jgMrFqiiguogghSM8j9GSDDy1KmIpe6akXkofuF2dQU4fvD+8hIT3q3yF793gIDzlQ/X2+4jP0SmTiG4rTmka3uRJZ/rpjK6bScqQZMazY/4A4yFt6BLRkc50qq+g5YMkd/OrZFSlxhWQz8WbrOLJXfk9l2ORJnZnAUby7ay16GjijkoQHCTnp5XfFMEHvq065fTsZPBCBaIgJaTsaupgkrRMqD16kIxZFn1a7n4Tddpxn871SK6+T80oVBzOIpBSEZo68mtSihiXs+olOIMqPykDw+nE4TOTxSOYX5IbFc9PU4Dc/Qhco3EuqwNbyKJOGnhlR78f9tEdV3Sy/8Nf6JjEulKGySxgqKqNoqCTxWWETC3i/SXxC64LciPw+NWbweqEARe671qzbQKjx05XfPZNvBb+rxqG+xXxN4XK1OAwL6mUp6v1MspLB3VUHvu2463xttzJ5sX/loZ9hE1feAvlA/LcA+WCGQJAj/RHqx/gtp2fuyKPQQ4XSyO7Xh/nvsehDoDSi4++Vb9wvERBGCZIhSifSDIduQwyGJbX4EwBE0PELds+Lsg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1466.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(38100700002)(38350700002)(4326008)(6512007)(6666004)(6486002)(8936002)(83380400001)(316002)(6506007)(66476007)(1076003)(107886003)(508600001)(2906002)(26005)(66946007)(52116002)(5660300002)(8676002)(86362001)(186003)(66556008)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ISBBD+s8zZfpb4YeltjFFrwjjvMaNzzNsUBCXRv/BXeSdui1u14oUOIhqFm6dSk7o3qEN/E/9fKehFlf7LUzKF7ihIXNEXu5ZpnliVBoPWz6F/jtvepvDIv1WvEgQDOqfg4nzdAvjiomSyA6Xg+5wSI56O3hi3Qibf0CjAJE15Y91iI+uRKwa6fERsnKES8PiByYnrdMIeDTcz1d7NylHjZpmBFRislD9gZWt3YPdRDelw8DzkjrbPSCV3mS+iouxNUzYWOwRdu2l/h97VBR1aaHPX4qVjN3G9nDoZRVU2h1f7VjpxTUFXll6H4LRBQhmWot4wB0IflRhXyQiX6dOimhtfaTVjUpEKzZGwoW/t7MNb0pZD9/JL3ELDVxSxDrwt6SCvpf52jflGSvyMpC5d+qhHEeX7WrXwJgE/07F6Pd+EMx+tAtAnzSm6DtAohTWmlPtA0rUKHbiMp/98tsLIDHDMk3VkhrE4I9AM5RJyDtMXQBGPmK2ueWGIRZFKbILSu0pvb4QQ7eXeEoz3QTwxcC4QsSXCGQCEtALOngvbxjR8mHzyerPJ/zxnBAd3TtMAX086tUcGTRbRvRkEbxPhHXRUShSpFTWDc+W3XIF3pIc6lN024AMf8IGrMKQ+uw5Ji0tB9MdV78vVqLB5ZPP8c6sSU50QYWvx5KZ9kmdaOa4TTCidirwJLJTEZ3wAqPYmR/SIlVbAd+3YQkXMLep4O1nsqikekoRNE33dMz2pzbm0EIQpHcqCpjR4VnFYlEqcneOmmMn8/sZL5XgfaHmCdaAaGO2cVnc6jnk07DZB/oDDUpRyZ2n2G3jFnNfn5z42mR55HALOC8mEELhm8FouRv6HsLe+WglUpHoh96mvErNp95NANwNjGeXPuzz5sNKxw7R/5v5X68SYy7bOy7809cu89nT1NmUoH9py1K6u7B/021XJjaWqDEZrRDCZtxLktH3HQVtSlSHppJh4+tdDgqZRfaKOzZrOlLkZqg5qtwwcKg57o3ENdRy0vAxM9/p6eZZyxqkpu0mkPisXCGbzVQ6vezoJlllrSLvyS5pfAn1DOtNIUuyxVTdxIem1oZ3195tXM7Dqnbw+h8ow+QXIMcaOLeGMIiAwEvPzSDfQSJsYUZjzrFmFqFSUw1mearCg6jHWDrX/ylCG+TgbSccy+0GWW/YPa5CV9Q9x5C1ZR6C8xNQqQ5cxOJ8LBeTzySttOVHVs96IoU9vdKqbrrrkN6fPTVcNIUMbR7yTXgNJLiXs/tdxlrsTstk25Uen/JTAicVCFH4FDE/tqW9xxINO6pWvDJobmEAI95ifrB8CSQnh4ip2QXzsmQbCaX9QNR X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6729f218-bdd7-4d39-4d34-08d983b6a564 X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 02:04:33.4395 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JTSPxp/omoNPgpJaIsjM6CY0BA+q7q0RYv2xUyEgyxD1P86nUyzGFt2ZQd91pe3bGNH0Lk4t41fD/gnnTS+ZIDSeCRjPwRYKaiyOQ4WdBVQ= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB4909 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109300009 X-Proofpoint-GUID: zavX18Q5GES3gMcxgYjdQ5bKexU_Qhw8 X-Proofpoint-ORIG-GUID: zavX18Q5GES3gMcxgYjdQ5bKexU_Qhw8 Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org We are only holding the lun_tg_pt_gp_lock in target_alua_state_check to make sure tg_pt_gp is not freed from under us while we copy the state, delay, id values. We can instead use RCU here to access the tg_pt_gp. With this patch IOPs can increase up to 10% for jobs like: fio --filename=/dev/sdX --direct=1 --rw=randrw --bs=4k \ --ioengine=libaio --iodepth=64 --numjobs=N when there aire mulitple sessions (you are running that fio command to each /dev/sdX or using multipath and there are over 8 paths), or more than 8 queues for the loop or vhost with multiple threads case and numjobs > 8. Signed-off-by: Mike Christie --- drivers/target/target_core_alua.c | 61 +++++++++++++++++-------------- include/target/target_core_base.h | 2 +- 2 files changed, 35 insertions(+), 28 deletions(-) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index bd0f2ce011dd..74944b914b4e 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -247,11 +247,11 @@ target_emulate_report_target_port_groups(struct se_cmd *cmd) * this CDB was received upon to determine this value individually * for ALUA target port group. */ - spin_lock(&cmd->se_lun->lun_tg_pt_gp_lock); - tg_pt_gp = cmd->se_lun->lun_tg_pt_gp; + rcu_read_lock(); + tg_pt_gp = rcu_dereference(cmd->se_lun->lun_tg_pt_gp); if (tg_pt_gp) buf[5] = tg_pt_gp->tg_pt_gp_implicit_trans_secs; - spin_unlock(&cmd->se_lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); } transport_kunmap_data_sg(cmd); @@ -292,24 +292,24 @@ target_emulate_set_target_port_groups(struct se_cmd *cmd) * Determine if explicit ALUA via SET_TARGET_PORT_GROUPS is allowed * for the local tg_pt_gp. */ - spin_lock(&l_lun->lun_tg_pt_gp_lock); - l_tg_pt_gp = l_lun->lun_tg_pt_gp; + rcu_read_lock(); + l_tg_pt_gp = rcu_dereference(l_lun->lun_tg_pt_gp); if (!l_tg_pt_gp) { - spin_unlock(&l_lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); pr_err("Unable to access l_lun->tg_pt_gp\n"); rc = TCM_UNSUPPORTED_SCSI_OPCODE; goto out; } if (!(l_tg_pt_gp->tg_pt_gp_alua_access_type & TPGS_EXPLICIT_ALUA)) { - spin_unlock(&l_lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); pr_debug("Unable to process SET_TARGET_PORT_GROUPS" " while TPGS_EXPLICIT_ALUA is disabled\n"); rc = TCM_UNSUPPORTED_SCSI_OPCODE; goto out; } valid_states = l_tg_pt_gp->tg_pt_gp_alua_supported_states; - spin_unlock(&l_lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); ptr = &buf[4]; /* Skip over RESERVED area in header */ @@ -662,17 +662,17 @@ target_alua_state_check(struct se_cmd *cmd) " target port\n"); return TCM_ALUA_OFFLINE; } - - if (!lun->lun_tg_pt_gp) + rcu_read_lock(); + tg_pt_gp = rcu_dereference(lun->lun_tg_pt_gp); + if (!tg_pt_gp) { + rcu_read_unlock(); return 0; + } - spin_lock(&lun->lun_tg_pt_gp_lock); - tg_pt_gp = lun->lun_tg_pt_gp; out_alua_state = tg_pt_gp->tg_pt_gp_alua_access_state; nonop_delay_msecs = tg_pt_gp->tg_pt_gp_nonop_delay_msecs; tg_pt_gp_id = tg_pt_gp->tg_pt_gp_id; - - spin_unlock(&lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); /* * Process ALUA_ACCESS_STATE_ACTIVE_OPTIMIZED in a separate conditional * statement so the compiler knows explicitly to check this case first. @@ -1219,10 +1219,10 @@ static int core_alua_set_tg_pt_secondary_state( struct t10_alua_tg_pt_gp *tg_pt_gp; int trans_delay_msecs; - spin_lock(&lun->lun_tg_pt_gp_lock); - tg_pt_gp = lun->lun_tg_pt_gp; + rcu_read_lock(); + tg_pt_gp = rcu_dereference(lun->lun_tg_pt_gp); if (!tg_pt_gp) { - spin_unlock(&lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); pr_err("Unable to complete secondary state" " transition\n"); return -EINVAL; @@ -1246,7 +1246,7 @@ static int core_alua_set_tg_pt_secondary_state( "implicit", config_item_name(&tg_pt_gp->tg_pt_gp_group.cg_item), tg_pt_gp->tg_pt_gp_id, (offline) ? "OFFLINE" : "ONLINE"); - spin_unlock(&lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); /* * Do the optional transition delay after we set the secondary * ALUA access state. @@ -1754,13 +1754,14 @@ void core_alua_free_tg_pt_gp( __target_attach_tg_pt_gp(lun, dev->t10_alua.default_tg_pt_gp); } else - lun->lun_tg_pt_gp = NULL; + rcu_assign_pointer(lun->lun_tg_pt_gp, NULL); spin_unlock(&lun->lun_tg_pt_gp_lock); spin_lock(&tg_pt_gp->tg_pt_gp_lock); } spin_unlock(&tg_pt_gp->tg_pt_gp_lock); + synchronize_rcu(); kmem_cache_free(t10_alua_tg_pt_gp_cache, tg_pt_gp); } @@ -1805,7 +1806,7 @@ static void __target_attach_tg_pt_gp(struct se_lun *lun, assert_spin_locked(&lun->lun_tg_pt_gp_lock); spin_lock(&tg_pt_gp->tg_pt_gp_lock); - lun->lun_tg_pt_gp = tg_pt_gp; + rcu_assign_pointer(lun->lun_tg_pt_gp, tg_pt_gp); list_add_tail(&lun->lun_tg_pt_gp_link, &tg_pt_gp->tg_pt_gp_lun_list); tg_pt_gp->tg_pt_gp_members++; spin_lock(&lun->lun_deve_lock); @@ -1822,6 +1823,7 @@ void target_attach_tg_pt_gp(struct se_lun *lun, spin_lock(&lun->lun_tg_pt_gp_lock); __target_attach_tg_pt_gp(lun, tg_pt_gp); spin_unlock(&lun->lun_tg_pt_gp_lock); + synchronize_rcu(); } static void __target_detach_tg_pt_gp(struct se_lun *lun, @@ -1834,7 +1836,7 @@ static void __target_detach_tg_pt_gp(struct se_lun *lun, tg_pt_gp->tg_pt_gp_members--; spin_unlock(&tg_pt_gp->tg_pt_gp_lock); - lun->lun_tg_pt_gp = NULL; + rcu_assign_pointer(lun->lun_tg_pt_gp, NULL); } void target_detach_tg_pt_gp(struct se_lun *lun) @@ -1842,10 +1844,12 @@ void target_detach_tg_pt_gp(struct se_lun *lun) struct t10_alua_tg_pt_gp *tg_pt_gp; spin_lock(&lun->lun_tg_pt_gp_lock); - tg_pt_gp = lun->lun_tg_pt_gp; + tg_pt_gp = rcu_dereference_check(lun->lun_tg_pt_gp, + lockdep_is_held(&lun->lun_tg_pt_gp_lock)); if (tg_pt_gp) __target_detach_tg_pt_gp(lun, tg_pt_gp); spin_unlock(&lun->lun_tg_pt_gp_lock); + synchronize_rcu(); } ssize_t core_alua_show_tg_pt_gp_info(struct se_lun *lun, char *page) @@ -1854,8 +1858,8 @@ ssize_t core_alua_show_tg_pt_gp_info(struct se_lun *lun, char *page) struct t10_alua_tg_pt_gp *tg_pt_gp; ssize_t len = 0; - spin_lock(&lun->lun_tg_pt_gp_lock); - tg_pt_gp = lun->lun_tg_pt_gp; + rcu_read_lock(); + tg_pt_gp = rcu_dereference(lun->lun_tg_pt_gp); if (tg_pt_gp) { tg_pt_ci = &tg_pt_gp->tg_pt_gp_group.cg_item; len += sprintf(page, "TG Port Alias: %s\nTG Port Group ID:" @@ -1871,7 +1875,7 @@ ssize_t core_alua_show_tg_pt_gp_info(struct se_lun *lun, char *page) "Offline" : "None", core_alua_dump_status(lun->lun_tg_pt_secondary_stat)); } - spin_unlock(&lun->lun_tg_pt_gp_lock); + rcu_read_unlock(); return len; } @@ -1918,7 +1922,8 @@ ssize_t core_alua_store_tg_pt_gp_info( } spin_lock(&lun->lun_tg_pt_gp_lock); - tg_pt_gp = lun->lun_tg_pt_gp; + tg_pt_gp = rcu_dereference_check(lun->lun_tg_pt_gp, + lockdep_is_held(&lun->lun_tg_pt_gp_lock)); if (tg_pt_gp) { /* * Clearing an existing tg_pt_gp association, and replacing @@ -1941,7 +1946,7 @@ ssize_t core_alua_store_tg_pt_gp_info( dev->t10_alua.default_tg_pt_gp); spin_unlock(&lun->lun_tg_pt_gp_lock); - return count; + goto sync_rcu; } __target_detach_tg_pt_gp(lun, tg_pt_gp); move = 1; @@ -1958,6 +1963,8 @@ ssize_t core_alua_store_tg_pt_gp_info( tg_pt_gp_new->tg_pt_gp_id); core_alua_put_tg_pt_gp_from_name(tg_pt_gp_new); +sync_rcu: + synchronize_rcu(); return count; } diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h index 2121a323fd6c..d7d31a508dec 100644 --- a/include/target/target_core_base.h +++ b/include/target/target_core_base.h @@ -749,7 +749,7 @@ struct se_lun { /* ALUA target port group linkage */ struct list_head lun_tg_pt_gp_link; - struct t10_alua_tg_pt_gp *lun_tg_pt_gp; + struct t10_alua_tg_pt_gp __rcu *lun_tg_pt_gp; spinlock_t lun_tg_pt_gp_lock; struct se_portal_group *lun_tpg; From patchwork Thu Sep 30 02:04:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Christie X-Patchwork-Id: 12527121 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B368C433F5 for ; Thu, 30 Sep 2021 02:04:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 395CE6141B for ; Thu, 30 Sep 2021 02:04:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346934AbhI3CGX (ORCPT ); Wed, 29 Sep 2021 22:06:23 -0400 Received: from mx0b-00069f02.pphosted.com ([205.220.177.32]:48572 "EHLO mx0b-00069f02.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1347294AbhI3CGU (ORCPT ); Wed, 29 Sep 2021 22:06:20 -0400 Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.16.1.2/8.16.1.2) with SMTP id 18U0C6DL017464; Thu, 30 Sep 2021 02:04:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2021-07-09; bh=UtDC9ouH3Q4WGuZ12fLcrluLt3CZ8j2TAPeAdEuTzVU=; b=RMmxnVwNGa9hkBWlxmqdzjmDKLuoy7CC3OR8Vk8dJB3IUh+UgVnB+ysH884eycwDbqm2 GhssPExcCNrMwbruhDV8GTdVsvpc2YVcdssTbOGH3aerZpWls5RsFa8GikPtdmVZUYqi iWzPYZfsJuGEBkRMve/oLb7Qtd1SamJR62ZC2WxYD28UaDCQQDuDtuHOQ59Yl1zHcxRA x9HFGGssfeWc5meJGtsGa5VCtAV4n4C4T1KtZL4y0o8zCEWiwIhmSOxsdECsdbMm6tQ8 qo9WGMSL7TLDMwh37lxSwzOLTul1yIhn9oHW4nwyY7l5dvDMmiRjCAlVTImjDTiPGNgt jw== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by mx0b-00069f02.pphosted.com with ESMTP id 3bcg3hswrn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:36 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 18U20Fu2160315; Thu, 30 Sep 2021 02:04:35 GMT Received: from nam02-dm3-obe.outbound.protection.outlook.com (mail-dm3nam07lp2047.outbound.protection.outlook.com [104.47.56.47]) by userp3020.oracle.com with ESMTP id 3bc3cf5ava-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 30 Sep 2021 02:04:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nibDc6n+e2mv3zxS3r3T5PzBQFFj3T1FCc+MTXoCYgSOvZRxluSF2zYdDZVQMHWoOjFoXF2GH2tAmeNbD7sID8GsaHbOQWgTwvpDJCs6mnPnZQPJRSijykeZRuFYoPnHrYV4xp6w/+/w5plgU440HGP3GU12L/yw/phx93RwXnUPyC74i+dL9B+LDYsVcvV2LX3Uvw4my6c8SDGcznElCzFvWSo9yUBLSqCeuwJp9g3MUbxkBwxK1JX1nzmhEEpZ8lvCqVVRU4wJxtm9Gz0H17U63bYOoXM7K4C6/5Y7B+poPcRN45qK9kntMiV1GSspbBSJc9a8QGTcM8c5PNDLGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=UtDC9ouH3Q4WGuZ12fLcrluLt3CZ8j2TAPeAdEuTzVU=; b=YbMAhuHtGV53bvR0qwjgHiRKZqGc2UjTlG/JpUH1pfxBpRACi6syXseMRsg6gx2n3HB5Ko3VudYVZQ8xbrTLOZvxkqK/4mkDtVvk7RaOHctYBO14UiLUvn2S3RoOURS6q3weUHVLYEy78+qBU3EaA22TI7ZFjdL2d+b0fMgW9P2/g+41rPpb0KFGzQRtrf6F+CT1asdphgcb4RAT6N31q6Lws3lfxZETiobl5TLNj0/Jhr8XaRkguA8m3Zb0GY/Bp5ylAcReYxPkTR6s9z+AmmHzMvsIEgK0pRHXvctSLgtWVZvK9UhS5JvkvFbQrMxY16tYS0Ywg4T6pRsYzbpbGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UtDC9ouH3Q4WGuZ12fLcrluLt3CZ8j2TAPeAdEuTzVU=; b=dNIQVBD+NZrIpPFQsUyK+1BKXXXOa2OaTRoN0y6/DNePLAckCdzWLC793P3Y4Mx9gwFcEBgy4TDzIqBzZGNi1VKbDT86W6ZFFWfMRe3UxyYWKyvvIJDHHfEcz2r3b7VP+cYHTDVR8dKE4Gr5yV+3Fdgc9PTHNl2kmeVBUiAM/Hs= Authentication-Results: oracle.com; dkim=none (message not signed) header.d=none;oracle.com; dmarc=none action=none header.from=oracle.com; Received: from DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) by DS7PR10MB4909.namprd10.prod.outlook.com (2603:10b6:5:3b0::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4544.14; Thu, 30 Sep 2021 02:04:34 +0000 Received: from DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531]) by DM5PR10MB1466.namprd10.prod.outlook.com ([fe80::195:7e6b:efcc:f531%5]) with mapi id 15.20.4544.023; Thu, 30 Sep 2021 02:04:34 +0000 From: Mike Christie To: martin.petersen@oracle.com, james.bottomley@hansenpartnership.com, linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: Mike Christie Subject: [PATCH 5/5] target: perform alua group changes in one step Date: Wed, 29 Sep 2021 21:04:22 -0500 Message-Id: <20210930020422.92578-6-michael.christie@oracle.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210930020422.92578-1-michael.christie@oracle.com> References: <20210930020422.92578-1-michael.christie@oracle.com> X-ClientProxiedBy: DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) To DM5PR10MB1466.namprd10.prod.outlook.com (2603:10b6:3:b::7) MIME-Version: 1.0 Received: from localhost.localdomain (73.88.28.6) by DM5PR07CA0159.namprd07.prod.outlook.com (2603:10b6:3:ee::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4566.15 via Frontend Transport; Thu, 30 Sep 2021 02:04:33 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 8a7499f1-76ff-457f-809f-08d983b6a5ec X-MS-TrafficTypeDiagnostic: DS7PR10MB4909: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:530; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ZuKJEv2vQDClzyiDWzEmks1USOxZi7WBps1XdsSLKGy15lp0qnNDK0CUvhPdoLBK64InV1WsOPFdRBAdcU5XRFRYOoFqNnYyVnLzhi17F6B57WW7+WqOH3K35jLpA3lQ5lAM177cZhNFpIFFABjxYIYlyfSsJhjRmo8BvTWRmIAp12D3EUPJQh5gP/KKdjDaNm4RnZLxuKW6p4SObbmCKQ/9/CDrWAT7Cvoywtdeb13PcR+4lz3ohBa28l060GYGAp8um6llTVlws3mKUsIZYcHWbIXcKy6BofeoP+Cu2NIwqSAOHGUWev9bO7HG3Jbcxj3TH0xWvngszm4WzOVHuiyL7M3tLTSCa6Hh8eme0IoeTbNPmAKKNpR2QkhHLfBTDx8BsjcoNEe+qrcVANw+yUn+ciDtWfqkCDXvjSbJSbcYAiCCdGDtRno+EoGu5BGqVOZ9HjBxOpTV61SkasTwKiTQzxD1VIQAKy5P5WKxioBClJaEiZM+W/WUvIMpXBgQvAhRjTuHq7tIdX0f+kYpCcua741/Jm60clpUBnp6C5JvjxSP2heCLyfSFgs1a/wydWth7zq5bdPigaT1b+/2N3X+CxmFSSuPR7d//QPl2TKk6ixKK5nbFxoWR3D3ujfmMfLaF1RpWGarNVQxY6Ff4niaz1GeWhYc5fETT1qyhRro2JV1MjeED5HgGJMIx+jQGKdM7gEZS6hp5g6AbUCM0w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR10MB1466.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(366004)(2616005)(956004)(38100700002)(38350700002)(4326008)(6512007)(6666004)(6486002)(8936002)(83380400001)(316002)(6506007)(66476007)(1076003)(107886003)(508600001)(2906002)(26005)(66946007)(52116002)(5660300002)(8676002)(86362001)(186003)(66556008)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: nBVI2v6DZeKO0LqIC5fmnC3h1H//CNpC6z6H3CIfVjEt8gWfmfi+vVWE8h1OOL04QR5rwME2Mx36XHIaYQlDQBmKj7GTJ2RKFaYPRHM6O2m3bW93FrOJO9LzZom6GSMR7scXN9JO+WFirXY/C6C6Bj7mVn5pHWHOaCL4Ks+125ZjIp7iyTYM0M5JGcFagwYS9vbP4RNPiOUESBCpHF26wziWuPARMHbupQEw+/FLcEdWyCkvDmmMWdmQ4M288nGYAtmYAWnKKf03jhV/KgvW0x8o7Te3allB7racBUYzfDLv3I5gJr/1vNAOPl+EzlAZw8smB6mcO1+L5bNcuN6rPJWxFnlKKuzEqnTVDV11h6IDtXMc61m54sQ7DaC2Pu/jxvquZEiImCVflX8RJPxyD4nnAggHT20k4b97LdkeHg6/a98HZnaWK8nH7nuEU5vryED27RlzNheuI8n7PpVKwrbJT+TJPfliFhQQRK3WUchWGTeNzdSPk2F1OmBAdNTjlSlwKeAEdjk4XCkk1hXcnjNkSTSTrQF4uz0XDfVN+ROMyZrOYbSoYPpUWYIdmXKSMPaSBT186qMAvNklgsGnPmxnMowUDZDB5D+kJCxpOVrl0kOIcBoybSthRnSFVR7m679fA1aA0fE5xB3WlgIKf8s03lX9xXKuORLFvR+UKjNvTaOEuB9LrvEvRbcUnCTFmFJU/I5fMmlphkX1A4xWbIJVWZfnQeJJC1cyqOzO8G94Hg78OAvNLghOsUGVwmSqcMTbqlTR2VUw2tg4JBdxbbru9xY67sjPxP635kWnZMrI0O/L1mmmZzhW0jhmNy0vuhYOka1Na26CLjJnNc0td4OPXijW3MkW5H3lzQ1xZcCESAPOMcln2DwZbujImBniGSgZcdgUOGOIY8CMA4kazF1/iStUB2XLxNudNtQZ6cqclrTATdnaACDFyGImln/WwVAHFa/4zsAMKwTAqy1uZRLq+q1s6QPb2M/VHCmvQKwTlkUvUigFatajSSzEEKMD5RWJjDSkJFHSE3uB/+JsM/RQWK+kUJTSw4na/4EYTOCXhC2H9D7rycOJ99RhClkwA5ejPnMy2nsll+nvkzQmaV2K6dZ/jKmKz+VoYTITbyTHtoJOPEwfu644BwLu/bCBCrNITAY9WeTqREvcLhBDs+sFKSMaM4kQbEeW+DqhniKAKHj+vHSGCPQLVKmGjwqv/XHrZ6bYcyp1nrMV2skI2OHNoIBoQTInfhGyNo7Zq9bLyiFD1SbTZxGBQTghUo3y+5pkYTPYBjp19grFYBqQzMh22JiTal+MDJyzt5UUScK18RpimLaTZHTBZ58+esWN X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8a7499f1-76ff-457f-809f-08d983b6a5ec X-MS-Exchange-CrossTenant-AuthSource: DM5PR10MB1466.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 02:04:34.2031 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: nR7Hd/t7T0McqOTUdj67cEgvu2xy/s3eXofVrrv7PJBnvorfCNK7/3aL3HKe51NWGtAOOp9M2LQkhGBsLU8CsNIib4W1QWLQwuFPA1ea+3c= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB4909 X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10122 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001 definitions=main-2109300009 X-Proofpoint-GUID: tz-8ZkBl9oqPlvDqWW4LvHW6d9S12Ojg X-Proofpoint-ORIG-GUID: tz-8ZkBl9oqPlvDqWW4LvHW6d9S12Ojg Precedence: bulk List-ID: X-Mailing-List: target-devel@vger.kernel.org When userspace changes the lun's alua group, it will set the lun's group to NULL then to the new group. Before the new group is set, target_alua_state_check will return 0 and allow the IO to execute. This has us skip the NULL stage, and just swap in the new group. Signed-off-by: Mike Christie --- drivers/target/target_core_alua.c | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c index 74944b914b4e..b56ef8af66e7 100644 --- a/drivers/target/target_core_alua.c +++ b/drivers/target/target_core_alua.c @@ -1835,8 +1835,6 @@ static void __target_detach_tg_pt_gp(struct se_lun *lun, list_del_init(&lun->lun_tg_pt_gp_link); tg_pt_gp->tg_pt_gp_members--; spin_unlock(&tg_pt_gp->tg_pt_gp_lock); - - rcu_assign_pointer(lun->lun_tg_pt_gp, NULL); } void target_detach_tg_pt_gp(struct se_lun *lun) @@ -1846,12 +1844,25 @@ void target_detach_tg_pt_gp(struct se_lun *lun) spin_lock(&lun->lun_tg_pt_gp_lock); tg_pt_gp = rcu_dereference_check(lun->lun_tg_pt_gp, lockdep_is_held(&lun->lun_tg_pt_gp_lock)); - if (tg_pt_gp) + if (tg_pt_gp) { __target_detach_tg_pt_gp(lun, tg_pt_gp); + rcu_assign_pointer(lun->lun_tg_pt_gp, NULL); + } spin_unlock(&lun->lun_tg_pt_gp_lock); synchronize_rcu(); } +static void target_swap_tg_pt_gp(struct se_lun *lun, + struct t10_alua_tg_pt_gp *old_tg_pt_gp, + struct t10_alua_tg_pt_gp *new_tg_pt_gp) +{ + assert_spin_locked(&lun->lun_tg_pt_gp_lock); + + if (old_tg_pt_gp) + __target_detach_tg_pt_gp(lun, old_tg_pt_gp); + __target_attach_tg_pt_gp(lun, new_tg_pt_gp); +} + ssize_t core_alua_show_tg_pt_gp_info(struct se_lun *lun, char *page) { struct config_item *tg_pt_ci; @@ -1941,18 +1952,16 @@ ssize_t core_alua_store_tg_pt_gp_info( &tg_pt_gp->tg_pt_gp_group.cg_item), tg_pt_gp->tg_pt_gp_id); - __target_detach_tg_pt_gp(lun, tg_pt_gp); - __target_attach_tg_pt_gp(lun, + target_swap_tg_pt_gp(lun, tg_pt_gp, dev->t10_alua.default_tg_pt_gp); spin_unlock(&lun->lun_tg_pt_gp_lock); goto sync_rcu; } - __target_detach_tg_pt_gp(lun, tg_pt_gp); move = 1; } - __target_attach_tg_pt_gp(lun, tg_pt_gp_new); + target_swap_tg_pt_gp(lun, tg_pt_gp, tg_pt_gp_new); spin_unlock(&lun->lun_tg_pt_gp_lock); pr_debug("Target_Core_ConfigFS: %s %s/tpgt_%hu/%s to ALUA" " Target Port Group: alua/%s, ID: %hu\n", (move) ?