From patchwork Mon Feb 6 15:54:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Josef Bacik X-Patchwork-Id: 9558267 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 39F4B60413 for ; Mon, 6 Feb 2017 15:55:12 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 299B125EA6 for ; Mon, 6 Feb 2017 15:55:12 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1C9D327D5E; Mon, 6 Feb 2017 15:55:12 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 812D025EA6 for ; Mon, 6 Feb 2017 15:55:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752240AbdBFPzK (ORCPT ); Mon, 6 Feb 2017 10:55:10 -0500 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:48541 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751829AbdBFPzJ (ORCPT ); Mon, 6 Feb 2017 10:55:09 -0500 Received: from pps.filterd (m0001303.ppops.net [127.0.0.1]) by m0001303.ppops.net (8.16.0.20/8.16.0.20) with SMTP id v16FppvU015442; Mon, 6 Feb 2017 07:54:59 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : subject : date : message-id : mime-version : content-type; s=facebook; bh=XlT46YhddhExyXdBZWewUTU6zkF/xxPMZ5SqxfoDEp8=; b=Kge6j90NAXzfLJ1npnodnzVzYfaUP6VH4qD21FuD0sQmfhYYB+J7HS6fYVfyiPMmL4rZ WOol6CEAbD+386XxSqy3V+XNKQCSaIYmXdB8HOXTbGORKlNnQv+dEcSCzG85vyuWpMsn Eoncukzk2KqXWZffsI1c5NCFeGaj2xD2/3w= Received: from mail.thefacebook.com ([199.201.64.23]) by m0001303.ppops.net with ESMTP id 28dbggmeu1-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Mon, 06 Feb 2017 07:54:59 -0800 Received: from NAM01-BY2-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.13) with Microsoft SMTP Server (TLS) id 14.3.294.0; Mon, 6 Feb 2017 07:54:57 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=XlT46YhddhExyXdBZWewUTU6zkF/xxPMZ5SqxfoDEp8=; b=a9YZFQ+ksVCKLE1pHmxslpFB5XlXJM4vQ/Lx7zM2BIU9QdfjtqxsgNVHA73ZKGKd6GfeKeIv2djkVhBqtU1XqMb6ZS1WzKS7N9lzIpWWEurUohd+C1SmIf+cFcmZqyn8gWOeeJbNYd+W6SZ/ZlivdkQKYWhhJge992lfugfx5Jc= Received: from localhost (2606:a000:4381:1201:225:22ff:feb3:e51a) by DM5PR15MB1915.namprd15.prod.outlook.com (10.174.247.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.888.16; Mon, 6 Feb 2017 15:54:56 +0000 From: Josef Bacik To: , , , Subject: [PATCH] nbd: freeze the queue before making changes Date: Mon, 6 Feb 2017 10:54:49 -0500 Message-ID: <1486396489-2911-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-Originating-IP: [2606:a000:4381:1201:225:22ff:feb3:e51a] X-ClientProxiedBy: DM5PR18CA0024.namprd18.prod.outlook.com (10.175.218.162) To DM5PR15MB1915.namprd15.prod.outlook.com (10.174.247.143) X-MS-Office365-Filtering-Correlation-Id: 095ebd5a-07ae-4ab9-ed6c-08d44ea87f3c X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:DM5PR15MB1915; X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1915; 3:oaFDF+QWYZ6pPJLx5ImzGV7gCghpPnoX9OS2bsi0ANLlsu77nTZa3Zaq4d2JY7uAIDRsNOOEF6eW3J36dnIUcBXMdIQcnZ8v29OER4zKDUS1WxpS9zvLBK+8vhahZy4ABs80U4EDHE4NzhWpCoVHM33j067NkC7BJxbcBPNgAungdiL28OpWEXhC6OvH1FpferKfBTYIscGVaJ/BjS+UbhJHxeuR51D/6QWPy7tm7AESLEEIHV3mnE3ts5R9i1EIwJpa9BcahHWBx4rMJT2OOg==; 25:303wn7sMzPqz77xhtiEHMASkzogiQcE8En29v9YZYIwe/2exr23wKKzwiJHk6cHcnv6lSzsBbmu6xsNKoyexhtJ4cbLNbAZADgEyLKQT5m3LlC+8N2kdXjNHcpJLyfXABpGCeZDo0YyY59n//8ucxmnHjzy8wNZfgWSj0qpfxruVtD7CY+HdZMhbSOJ2KES3qjoj9DoiIP4jNoAec7nRjPH9jLHzZWmEZM2G8e3scffKLI1iicFxB8RbqtuR7TiJQLBoGEP1AuT7oneZ6fhXXac/vDleBmgb4UbkyrQ8kPbDHzWgWqfD2StlLm0Php5M5snBl6LeBnDEKXzPC4E6fJo/3kpOxufb8uZZcLp7WRSOkag8PD+ckYTofKXby+HtL9rnwvK0N5f8rDnNY6wlw8i1czzca/0V9Tqqcef2oquTXwGr3oZBdsvXF8H2X2Jt X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1915; 31:NV61hXqyWuteEbQ+aFhjiztYk2rmuAzBxuaL9803DNZGDlK+7AAlY/AfxXlyS9w52FCd5YjvcgAtoalwwjHwhhWyjmoPWDKqBZA4wsoRSPijpyZ06b616o/8ir214MZf275aHqiSQX0UCJo2g2R7f7Fcuao9vnlqkVNnxfETDEuUYpH7mhIaBBigE/dT1EDUHmYLTibcQ4bsQMeN60IgdE+VAKHfpZPU4+zNxigzBpd24IFqxqNhHrg/GiGSofYFXKJ6e13HpjSpCP4mUqjZCQ==; 20:YxJmAkCBPeOEjVmU9+Yg+KuP7ru86pFIIku8tKc7ofirVkwnHEmDs9fJamzQKjn1N5d/gCiWbNbqFP7tH7cb1erq/c5RrN36f0jDKlhFDyXmCKXU0BmSr+hZU6aeN0pQL2sP8cDR0bxPvA/aWcQK6EjHocQGUZjfiQl8s/2u9Q3g2T638lGFk/U4QawrteaITyekaZyswxQkfICTI8KtOpHbaVys9dk9Hd9vp6VrrbrRSs+rIK0cnL0ZoYjAUtZrnpsd6hbY4GDvqIGJbR6/MGBoH5OA4MYFKiTayHFctDs1R1Rr7ZsDIP95G/R9c6Nziz0v9Oi1byNBuJ2ThCWJxKvwlFLcvPjsap3Y5LOTfQ0smkpbicNGqSsxuD62RsQKU0WQDhdxIB4nofDDYa3gBQVFJIyQzWMLK2dTSKHRNU8JsX+EwXbsRY5fGp5N2nzKdUK0ie8chVbSQspGZdGngefpTCkqL/KtuWVeF0W+Z2m/0xco98W1eBDZT/ePPLPX X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(20170203043)(8121501046)(10201501046)(3002001)(6041248)(20161123560025)(20161123555025)(20161123562025)(20161123564025)(20161123558025)(6072148); SRVR:DM5PR15MB1915; BCL:0; PCL:0; RULEID:; SRVR:DM5PR15MB1915; X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1915; 4:z04ZnCfBvmZBY7nh/mvoxKZFZdHGmDsVGDZQ4yvCf3nZXN8bd1JgAxevFrsVC2/9td0zTAG0djh/4OWOcrKMYl1YrV0faoIAoVmaoxbifpL7Tovd10YLaE6V5Dk/AuiNbvIk9sFk54pKSYPsB9arfBBcUidEjrtCNKdZlKageyIY6OZeEDPgUgWueyJvC/7BWJVhQIsqOYDCuWUdepzK2KWBDq3Be2kAZXaaOR4TxGUWVS+mhxPRZ+IcBa2KQBfXDE3qAzAU0VjsOA+qtcXjfgTQ+ZBnYKLsqlNKhtaNPRfMJXq1YlHgjo/ptR5N5R0qY+2dskO70Hma7LqfDj12nFMmSwMmmS+VM0tHyNh6EuQW8sERtqmxwzFqktyFN9ZKejZvh/TUix9Pq6wdB8ai2seUN9LH1SYjEv/FWwkVo8URSfBtuPnEBdIFkLQ6/+2zbEzjHnXQdLXLUeRca9465zcL5TKZA4b5UM+8AKUwwKBsdtJe5Lj8yFqOQq0HpuL2YBu39eYJ4adT0hCs/6tFltRK2/HSGRtlY55lvYROthbWEKEIme6LdDD8bRcDb8tVGw00KqyYNOeZjUMrfbPX2KjSSCt4kPZCLdSrPb3CY99eLbXVlcGqkXlf6Lk4ma2ZwbPdpZIGa8bQ+IlyslEb9JvNxj4NYXjHFiNMO7WH/RYel+7FczibjbpRXJ+uJMUE X-Forefront-PRVS: 0210479ED8 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6069001)(6009001)(7916002)(39450400003)(189002)(199003)(45904002)(101416001)(8676002)(81156014)(33646002)(2201001)(86362001)(6116002)(50986999)(81166006)(105586002)(106356001)(5003940100001)(50466002)(53936002)(76506005)(6496005)(68736007)(48376002)(42186005)(92566002)(47776003)(36756003)(50226002)(97736004)(5001770100001)(305945005)(6666003)(189998001)(6486002)(107886002)(38730400001)(5660300001)(2906002)(25786008)(7736002); DIR:OUT; SFP:1102; SCL:1; SRVR:DM5PR15MB1915; H:localhost; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR15MB1915; 23:B2VLI2w8PzPjy+q+3tSiCaED9jCjUUJsGHrFQ7g8d?= =?us-ascii?Q?KNDmBvO2NTRwNlEf/66lRX6Mvcq2Q7x/dlB+r1EsGyEQFML73BQ7K6ydJHON?= =?us-ascii?Q?k7XxACImsu7yLCqF3TeUPkOyDwPLBS4psPfFLv3HjND9o0BPNY9WJoxKZwVw?= =?us-ascii?Q?T2wjoTuz2MbU5H94X1PBSbBmOEqlrRUByQOGNLUUX5g0o6JLvo4wuvFpeytD?= =?us-ascii?Q?mlRJruHYng1iEPHcUDb2kQIxR6XgBuBoWae40FIGmzOK2cZlhmELmyjED6ai?= =?us-ascii?Q?dLm5zZM10/d0aVT29JAsbSEE9ERYa3ePhSOdvwrfBVbnFZ/SfmQ2BoS8Uha1?= =?us-ascii?Q?vZumYsAHx9hN0aPUTddWxkm5hMOui30LCja/0FwpRxUrg6LaUBSxit/W8vQg?= =?us-ascii?Q?us/PTpLYw8CrtD9Xq/ICEf4ZMdcN98ceAA/fOWXUDY4DjMCgmE5W5STDzeEn?= =?us-ascii?Q?aCBImugoi8Ol4NZ87gJKFv4L9Pa3IH74/SCj9E6zZnT4PaCKDrbX7tk9lzpe?= =?us-ascii?Q?BD3prN7Dgybc/qgQZeyb8pXnOBgBnicLm6423MG6VXO9ASPvgcrD0APjp3vp?= =?us-ascii?Q?NCTtMpLPDER+EHHQQln6C1A6gJcogEe3UTL9fhZHVgvhZUBy18+6adG8qq/i?= =?us-ascii?Q?Cb1q+FAZ414c6Qkj5s684nWJfWHqI61UTQCunTHfQZNxAIdUFvOKz+Mm8ysn?= =?us-ascii?Q?JKeB3jbGwGu0KwOblZrUsjgBWLQYv8E4Bp8DHmcbE8zpwiwiF3RY4hNckw3y?= =?us-ascii?Q?SjXM+hDo7yM234FSjVqT8hphs3zXUzP/waRWDObv7H/7ZFJvSqhSYTtWmLaf?= =?us-ascii?Q?D4s/8smT/ZLmGmeSQDjsDWz8PBBSY1eJmm7XeW2abOiZxMVa/GgvdVQyW6dm?= =?us-ascii?Q?6YxWt4bemz+Cmnzsj/hAl5dlAfcs3H/8wSWLICbhNsQywVHXAkpDqMGs/5bD?= =?us-ascii?Q?XCzekLbe6VZG6ZQBtca2+74ypSAgXFHOXLhtVs5u8BH1S4HMLT4pEPXZ7tsU?= =?us-ascii?Q?VxU6koTOJNYt4Y3oeEzIiI/B6HGvQgkffvLTBiVim0eqSWuUlpi9S4dpOZPN?= =?us-ascii?Q?a5OmVkIB/mw8ID+as2i2ZhSo7o+?= X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1915; 6:BMs9wf2HokIv+iJE/Lz8ZizpYTTRKxB3/UOanwkJQnQ7I+kXrBbcCJTYZOE7zi+WYZWyayyV5GN/9skc7S2qI1Nm4ZEecWRc0lJLMLJK9QywHeOE+G/8Vpl26jpUjk66PrGfVEF78Aww2NYJdMD/JUjKr+HN0JcU13xsvNEC0JgfM3YhPlbAEn6x6ykLyewZuSrOFD+ijZEvj3m5IrsTOt7NIbkec03rNinvG3RkBZzDWJGhrnuQ8GMSSCWk9zl4ixa3rke3frl2UKwzwsc0o4EZgwsZxugAOriC2dwwXAUmnNrd5fUDYIUJO9lUv32tZpdvc1GJ+bfkEqJw6TxEu+qkPZztYZIBSAGITly2tYWx9BcvhShYQfo6m4vIxSTWzVg1Nv7TTtRW8TkYQ14zlg==; 5:GlpK0noGwDS0y4masx1DFqIyCmLQ3M9TdRtR1GieI12r6YY1cezUxk6Oi5OZ7OA4P4SvxP8/LAtmBPtj6tCE64/U+Ydd74Aw4gRDzMY2JZOW+knXMkS5qqb4MU63PUouHTPKq7BpvMooMnwn5kE4NO6/1Y2oKYiKtYatPLb7t+A=; 24:62G5xiWQu7Oc0XsGsaw3smXcivU6em9urb7iemfQNMwutzWYso1HkJprCOwcUQzvdMdLBspv36NyrLfR8Fs73eU0jKjZgxeebX148BPGy0k= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR15MB1915; 7:0JJSOQNvyPr8njQsejoMY9CnOvLmjec+qjyf+RP6J+OLVVxOTC+xKUDp2AzdrggEUd49WpeLnJYL0ogbKyuwc5skvtJuD5YUohHThMF8Xxj9CfcxKVDGQv+++JeR8ZPqGc1XzcUan0JITtrf8UGe9UCBD2rkFoBdl5Cjb90Bf4CG16iM8rbTz3yu1oS510yqHGdCQxVFMtKK2GvIS2n5q0p3PfSZL9NwojIUy/twvZrAEfO/RK8Ygw63k7992WWpKpKSmO1BbDWEf5BQsWD2zvawpHwvmmLmj1yk6AfiCobLDlF2AEtTWf71CNbpA5ItUEqsuzpKmHlpKiKJ3OdLWcEduW4eNgUd4yTS1gBo/L5dgXaTyXzfoPVdtJSi3IteJ0haTiQ4Mq3rKHJUXCHxvvYiiiM/0VgMc1mkanEfweQIRC+pDxn7vGuO/saSEvNtbQJwER4qbOacwqKfl7A2vyraR8UW71rHrzLyjbgtOQxSHboVc32UYJC0rDcNlr3OVOoaL6cR8Prqpt8WJtRZ7g==; 20:mTfmS7rguoL811+CUnJ3d6bZsiLwXTdz3jnh5jz5ODs3pzKE5gsuQuLeHau5jq87nG9if2TTo0ev3iOYUN3nc+12eMEt5ixH8UKzpxfZFp/+/osm/0I2gU5VuJMfiJF1fPeOiicam77vpUrBGl6yT9rUKUUW4rbjJ+rRLjrMA3I= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2017 15:54:56.0242 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1915 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-02-06_05:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The way we make changes to the NBD device is inherently racey, as we could be in the middle of a request and suddenly change the number of connections. In practice this isn't a big deal, but with timeouts we have to take the config_lock in order to protect ourselves since it is important those values don't change. Fix this by freezing the queue before we do any of our device operations. Signed-off-by: Josef Bacik --- drivers/block/nbd.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 164a548..f8b3ecd 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -199,7 +199,6 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req, if (nbd->num_connections > 1) { dev_err_ratelimited(nbd_to_dev(nbd), "Connection timed out, retrying\n"); - mutex_lock(&nbd->config_lock); /* * Hooray we have more connections, requeue this IO, the submit * path will put it on a real connection. @@ -213,11 +212,9 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req, kernel_sock_shutdown(nsock->sock, SHUT_RDWR); mutex_unlock(&nsock->tx_lock); } - mutex_unlock(&nbd->config_lock); blk_mq_requeue_request(req, true); return BLK_EH_RESET_TIMER; } - mutex_unlock(&nbd->config_lock); } else { dev_err_ratelimited(nbd_to_dev(nbd), "Connection timed out\n"); @@ -225,9 +222,7 @@ static enum blk_eh_timer_return nbd_xmit_timeout(struct request *req, set_bit(NBD_TIMEDOUT, &nbd->runtime_flags); req->errors++; - mutex_lock(&nbd->config_lock); sock_shutdown(nbd); - mutex_unlock(&nbd->config_lock); return BLK_EH_HANDLED; } @@ -756,7 +751,9 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd, return -EINVAL; mutex_unlock(&nbd->config_lock); + blk_mq_unfreeze_queue(nbd->disk->queue); fsync_bdev(bdev); + blk_mq_freeze_queue(nbd->disk->queue); mutex_lock(&nbd->config_lock); /* Check again after getting mutex back. */ @@ -870,10 +867,14 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd, args[i].index = i; queue_work(recv_workqueue, &args[i].work); } + blk_mq_unfreeze_queue(nbd->disk->queue); + wait_event_interruptible(nbd->recv_wq, atomic_read(&nbd->recv_threads) == 0); for (i = 0; i < num_connections; i++) flush_work(&args[i].work); + + blk_mq_freeze_queue(nbd->disk->queue); nbd_dev_dbg_close(nbd); nbd_size_clear(nbd, bdev); device_remove_file(disk_to_dev(nbd->disk), &pid_attr); @@ -924,9 +925,11 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t mode, BUG_ON(nbd->magic != NBD_MAGIC); + blk_mq_freeze_queue(nbd->disk->queue); mutex_lock(&nbd->config_lock); error = __nbd_ioctl(bdev, nbd, cmd, arg); mutex_unlock(&nbd->config_lock); + blk_mq_unfreeze_queue(nbd->disk->queue); return error; }