From patchwork Fri Oct 21 07:24:55 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tim Dawson X-Patchwork-Id: 9388159 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id D6F3E607F0 for ; Fri, 21 Oct 2016 07:25:10 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C805F29E6F for ; Fri, 21 Oct 2016 07:25:10 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BBB6229E71; Fri, 21 Oct 2016 07:25:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2622229E6F for ; Fri, 21 Oct 2016 07:25:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754266AbcJUHZJ (ORCPT ); Fri, 21 Oct 2016 03:25:09 -0400 Received: from mail-pf0-f169.google.com ([209.85.192.169]:36446 "EHLO mail-pf0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751002AbcJUHZI (ORCPT ); Fri, 21 Oct 2016 03:25:08 -0400 Received: by mail-pf0-f169.google.com with SMTP id e6so52150285pfk.3 for ; Fri, 21 Oct 2016 00:25:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nyriad-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:mime-version:content-disposition :user-agent; bh=WqaWMx8crJI8KnwAu03FGRheuBXL9zSu7IHBlRcYiH4=; b=SXdeiAsIvlO1aTi3NSRGsZHWdWsUMt5mXuOkMb6cRJvm9IhOpf8YuKHdl+pB2pggI5 A7+Gf1ZVeXaTWwecekFb5flWImvmyD9Oqxlxfl6vUxGQOlNSaYNJ4Cj3pQzcZVgwXlbd dgs9NfyXNBHFMCRfxOqsO1A/lXwDOAb3npE9i8ioANubxLJ88iRJho0GNqT67tmWxnf3 S4JFZa7pYbBqEDr4hDWZFK3x4v35dUggFQuhhFAvuCRqmaWDZ4yDtXop60c5P1twxfAE EPF9uk4+WSwCLoHzOPHGIUi+Rw4U7l4n7fXLfwDUmP1Y1hxy7l4Nml72DbgST7cga9l9 qWJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:mime-version :content-disposition:user-agent; bh=WqaWMx8crJI8KnwAu03FGRheuBXL9zSu7IHBlRcYiH4=; b=FWRGGECaEhI4hqHVszhUA/bBPbI45iC01qTD+vZ//fNvchGvaQQqMx9m9PmSe0oNAt dCeZolcb8I8rNfv1PbrRebJb+npQyABKDTfVTZodXTOO/Zi+5G9/Oj2brsBZfa6q+zGW 1XkGdiQF2X5L8wHKgtacOiWiuPa/Sc2RnhvfDQWZZlOVJIAUiw+cBTdVpWshSKyw3ehO 8l7QhokJeWQC8h8+Uky71UgJEIM/n+3iXGNsbautWxW8/LHeg8I1cQf1QyZRo2VDX6QT lOKLSo3h6GdUcHeUGxga3EMthT+eOgn85iR0oVhuH1pXqxOSC6Cgy5HgwsYDyaR9eRPL 2jxw== X-Gm-Message-State: AA6/9Rle0dL5cDZpawLu9ltvp5lynq4/9wDQmrBJs0XpGLhC4MHPMdCi9B891IT++RMmj6z6 X-Received: by 10.98.60.157 with SMTP id b29mr9084942pfk.129.1477034708113; Fri, 21 Oct 2016 00:25:08 -0700 (PDT) Received: from tim-dev (114-134-6-99.static.lightwire.co.nz. [114.134.6.99]) by smtp.gmail.com with ESMTPSA id a23sm2328283pfc.59.2016.10.21.00.25.05 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 21 Oct 2016 00:25:07 -0700 (PDT) Date: Fri, 21 Oct 2016 20:24:55 +1300 From: Tim Dawson To: axboe@kernel.dk Cc: linux-block@vger.kernel.org Subject: [PATCH] nbd: allow socket reconnect without block IO errors Message-ID: <20161021072455.GA7101@tim-dev> MIME-Version: 1.0 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP this patch introduces a new ioctl that prevents requests being aborted on a socket error and instead the request is retried on the next connection or is cleaned up on timeout, disconnect or clear_sock. Signed-off-by: Tim Dawson --- drivers/block/nbd.c | 80 +++++++++++++++++++++++++++++++++++++++--------- include/uapi/linux/nbd.h | 1 + 2 files changed, 66 insertions(+), 15 deletions(-) diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index ba405b5..727f5b5 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -43,6 +43,7 @@ #define NBD_TIMEDOUT 0 #define NBD_DISCONNECT_REQUESTED 1 +#define NBD_RESEND_DO_IT 2 struct nbd_device { u32 flags; @@ -433,8 +434,6 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct block_device *bdev) return ret; } - nbd_size_update(nbd, bdev); - while (1) { cmd = nbd_read_stat(nbd); if (IS_ERR(cmd)) { @@ -442,10 +441,19 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct block_device *bdev) break; } + struct request *req = blk_mq_rq_from_pdu(cmd); + + if (test_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags) && req->errors) { + /* reset errors - we will reconnect and fix these */ + req->errors = 0; + ret = -EIO; + dev_warn(disk_to_dev(nbd->disk), "Handling failed recv %p\n", req); + break; + } + nbd_end_request(cmd); - } - nbd_size_clear(nbd, bdev); + } device_remove_file(disk_to_dev(nbd->disk), &pid_attr); return ret; @@ -503,7 +511,10 @@ static void nbd_handle_cmd(struct nbd_cmd *cmd) goto error_out; } - if (nbd_send_cmd(nbd, cmd) != 0) { + /* if not NBD_RESEND_DO_IT and there is an error, + * we dont want to nbd_end_request if we can retry later + */ + if (nbd_send_cmd(nbd, cmd) != 0 && !test_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags)) { dev_err(disk_to_dev(nbd->disk), "Request send failed\n"); req->errors++; nbd_end_request(cmd); @@ -515,8 +526,32 @@ static void nbd_handle_cmd(struct nbd_cmd *cmd) return; error_out: - req->errors++; - nbd_end_request(cmd); + if (test_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags)) { + nbd->task_send = NULL; + mutex_unlock(&nbd->tx_lock); + return; + } + + req->errors++; + nbd_end_request(cmd); +} + + +static void nbd_resend_req(struct request *req, void *data, bool reserved) +{ + struct nbd_device *nbd = data; + struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); + + dev_err(nbd_to_dev(nbd), "resend request %p\n", cmd); + nbd_handle_cmd(cmd); +} + +static void nbd_resend_pending(struct nbd_device *nbd) +{ + BUG_ON(nbd->magic != NBD_MAGIC); + + blk_mq_tagset_busy_iter(&nbd->tag_set, nbd_resend_req, nbd); + dev_dbg(disk_to_dev(nbd->disk), "queue cleared\n"); } static int nbd_queue_rq(struct blk_mq_hw_ctx *hctx, @@ -661,6 +696,10 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd, nbd->flags = arg; return 0; + case NBD_SET_RESEND: + set_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags); + return 0; + case NBD_DO_IT: { int error; @@ -675,26 +714,37 @@ static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd, nbd_parse_flags(nbd, bdev); + /* Resend remaining commands from last connection */ + if (test_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags)) + nbd_resend_pending(nbd); + + nbd_size_update(nbd, bdev); + nbd_dev_dbg_init(nbd); - error = nbd_thread_recv(nbd, bdev); + error = nbd_thread_recv(nbd, bdev); //blocking nbd_dev_dbg_close(nbd); mutex_lock(&nbd->tx_lock); nbd->task_recv = NULL; sock_shutdown(nbd); - nbd_clear_que(nbd); - kill_bdev(bdev); - nbd_bdev_reset(bdev); + if (!test_bit(NBD_RESEND_DO_IT, &nbd->runtime_flags) || + test_bit(NBD_DISCONNECT_REQUESTED, &nbd->runtime_flags)) { - /* user requested, ignore socket errors */ - if (test_bit(NBD_DISCONNECT_REQUESTED, &nbd->runtime_flags)) + dev_info(disk_to_dev(nbd->disk), "NBD_DO_IT cleanup\n"); + nbd_size_clear(nbd, bdev); + nbd_clear_que(nbd); + kill_bdev(bdev); + nbd_bdev_reset(bdev); + nbd_reset(nbd); + + /* user requested, ignore socket errors */ error = 0; + } + if (test_bit(NBD_TIMEDOUT, &nbd->runtime_flags)) error = -ETIMEDOUT; - nbd_reset(nbd); - return error; } diff --git a/include/uapi/linux/nbd.h b/include/uapi/linux/nbd.h index e08e413..9031c87 100644 --- a/include/uapi/linux/nbd.h +++ b/include/uapi/linux/nbd.h @@ -28,6 +28,7 @@ #define NBD_DISCONNECT _IO( 0xab, 8 ) #define NBD_SET_TIMEOUT _IO( 0xab, 9 ) #define NBD_SET_FLAGS _IO( 0xab, 10) +#define NBD_SET_RESEND _IO( 0xab, 11) enum { NBD_CMD_READ = 0,