From patchwork Fri May 11 09:12:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yan, Zheng" X-Patchwork-Id: 10393703 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 8B49460236 for ; Fri, 11 May 2018 09:12:17 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AC43628E3B for ; Fri, 11 May 2018 09:12:17 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A0C7028E3E; Fri, 11 May 2018 09:12:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00, MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 33A8328E3B for ; Fri, 11 May 2018 09:12:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752813AbeEKJMO (ORCPT ); Fri, 11 May 2018 05:12:14 -0400 Received: from mx3-rdu2.redhat.com ([66.187.233.73]:58690 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752604AbeEKJMN (ORCPT ); Fri, 11 May 2018 05:12:13 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 1080B818A6AA; Fri, 11 May 2018 09:12:13 +0000 (UTC) Received: from localhost.localdomain (ovpn-12-29.pek2.redhat.com [10.72.12.29]) by smtp.corp.redhat.com (Postfix) with ESMTP id 205032166BAD; Fri, 11 May 2018 09:12:08 +0000 (UTC) From: "Yan, Zheng" To: ceph-devel@vger.kernel.org Cc: idryomov@gmail.com, jlayton@redhat.com, "Yan, Zheng" Subject: [PATCH] ceph: abort osd requests on force umount Date: Fri, 11 May 2018 17:12:02 +0800 Message-Id: <20180511091202.65250-1-zyan@redhat.com> X-Scanned-By: MIMEDefang 2.78 on 10.11.54.6 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 11 May 2018 09:12:13 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.11.55.8]); Fri, 11 May 2018 09:12:13 +0000 (UTC) for IP:'10.11.54.6' DOMAIN:'int-mx06.intmail.prod.int.rdu2.redhat.com' HELO:'smtp.corp.redhat.com' FROM:'zyan@redhat.com' RCPT:'' Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP this avoid force umount getting stuck at ceph_osdc_sync() Signed-off-by: "Yan, Zheng" --- fs/ceph/super.c | 1 + include/linux/ceph/osd_client.h | 5 ++++- net/ceph/osd_client.c | 43 ++++++++++++++++++++++++++++++++++++----- 3 files changed, 43 insertions(+), 6 deletions(-) diff --git a/fs/ceph/super.c b/fs/ceph/super.c index 3c1155803444..40664e13cc0f 100644 --- a/fs/ceph/super.c +++ b/fs/ceph/super.c @@ -793,6 +793,7 @@ static void ceph_umount_begin(struct super_block *sb) if (!fsc) return; fsc->mount_state = CEPH_MOUNT_SHUTDOWN; + ceph_osdc_abort_requests(&fsc->client->osdc, -EIO); ceph_mdsc_force_umount(fsc->mdsc); return; } diff --git a/include/linux/ceph/osd_client.h b/include/linux/ceph/osd_client.h index b73dd7ebe585..f61736963236 100644 --- a/include/linux/ceph/osd_client.h +++ b/include/linux/ceph/osd_client.h @@ -347,6 +347,7 @@ struct ceph_osd_client { struct rb_root linger_map_checks; atomic_t num_requests; atomic_t num_homeless; + int abort_code; struct delayed_work timeout_work; struct delayed_work osds_timeout_work; #ifdef CONFIG_DEBUG_FS @@ -377,7 +378,9 @@ extern void ceph_osdc_handle_reply(struct ceph_osd_client *osdc, struct ceph_msg *msg); extern void ceph_osdc_handle_map(struct ceph_osd_client *osdc, struct ceph_msg *msg); -void ceph_osdc_update_epoch_barrier(struct ceph_osd_client *osdc, u32 eb); +extern void ceph_osdc_update_epoch_barrier(struct ceph_osd_client *osdc, u32); +extern void ceph_osdc_abort_requests(struct ceph_osd_client *osdc, + int err_code); extern void osd_req_op_init(struct ceph_osd_request *osd_req, unsigned int which, u16 opcode, u32 flags); diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index 08b5fc1f90cc..959af0e165ba 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -2165,9 +2165,9 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) struct ceph_osd_client *osdc = req->r_osdc; struct ceph_osd *osd; enum calc_target_result ct_res; + bool abort_code = 0; bool need_send = false; bool promoted = false; - bool need_abort = false; WARN_ON(req->r_tid); dout("%s req %p wrlocked %d\n", __func__, req, wrlocked); @@ -2183,7 +2183,9 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) goto promote; } - if (osdc->osdmap->epoch < osdc->epoch_barrier) { + if (osdc->abort_code) { + abort_code = osdc->abort_code; + } else if (osdc->osdmap->epoch < osdc->epoch_barrier) { dout("req %p epoch %u barrier %u\n", req, osdc->osdmap->epoch, osdc->epoch_barrier); req->r_t.paused = true; @@ -2208,7 +2210,7 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) req->r_t.paused = true; maybe_request_map(osdc); if (req->r_abort_on_full) - need_abort = true; + abort_code = -ENOSPC; } else if (!osd_homeless(osd)) { need_send = true; } else { @@ -2225,8 +2227,8 @@ static void __submit_request(struct ceph_osd_request *req, bool wrlocked) link_request(osd, req); if (need_send) send_request(req); - else if (need_abort) - complete_request(req, -ENOSPC); + else if (abort_code) + complete_request(req, abort_code); mutex_unlock(&osd->lock); if (ct_res == CALC_TARGET_POOL_DNE) @@ -2366,6 +2368,37 @@ void ceph_osdc_update_epoch_barrier(struct ceph_osd_client *osdc, u32 eb) } EXPORT_SYMBOL(ceph_osdc_update_epoch_barrier); +void ceph_osdc_abort_requests(struct ceph_osd_client *osdc, int err_code) +{ + struct rb_node *n, *m; + struct ceph_osd *osd; + struct ceph_osd_request *req; + + dout("%s err_code %d\n", __func__, err_code); + + down_write(&osdc->lock); + osdc->abort_code = err_code; + + for (n = rb_first(&osdc->osds); n; n = rb_next(n)) { + osd = rb_entry(n, struct ceph_osd, o_node); + m = rb_first(&osd->o_requests); + while (m) { + req = rb_entry(m, struct ceph_osd_request, r_node); + m = rb_next(m); + abort_request(req, err_code); + } + } + + m = rb_first(&osdc->homeless_osd.o_requests); + while (m) { + req = rb_entry(m, struct ceph_osd_request, r_node); + m = rb_next(m); + abort_request(req, err_code); + } + up_write(&osdc->lock); +} +EXPORT_SYMBOL(ceph_osdc_abort_requests); + /* * Drop all pending requests that are stalled waiting on a full condition to * clear, and complete them with ENOSPC as the return code. Set the