From patchwork Tue Nov 4 14:34:25 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Spray X-Patchwork-Id: 5228131 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 1347B9F3EE for ; Tue, 4 Nov 2014 14:35:30 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 036A4200F0 for ; Tue, 4 Nov 2014 14:35:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0A42B20173 for ; Tue, 4 Nov 2014 14:35:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753401AbaKDOfT (ORCPT ); Tue, 4 Nov 2014 09:35:19 -0500 Received: from mail-wg0-f48.google.com ([74.125.82.48]:61203 "EHLO mail-wg0-f48.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754066AbaKDOfO (ORCPT ); Tue, 4 Nov 2014 09:35:14 -0500 Received: by mail-wg0-f48.google.com with SMTP id m15so8259823wgh.35 for ; Tue, 04 Nov 2014 06:35:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references; bh=tspO4NfimkrHjaMszgcwy6Wt/a6baar8BXpRRM/J7mA=; b=VjhH+fRPD8toLCrD76aeQ/YJI/gJCPXysTVKy/D7+VtkcbYiZXTs1wCFuPVtAwSmYD 1i82Ff9/Xs81cQNDK7LBMqJmfx52F2WtZ07wcO8goWQIoQmvKvZzQFBtgCjr7AHKaGcz 5zTKENqF9R89zZDcRz/8uu3pjbATMZp8P51bEEl3ScQ+TaaDUOngWL8b+q13BGGGfIOb i3vokcf/Q7bcbiHsOCG6O0FxX3t5KhlO31+7yV8qNLfFcpAH2VKyNYjq05tj5beuWBvq Z9hq5lZz9YZL1B62e5ZnO7BL7fXX+Wo6X1cJbny6qrWkgt6BN1dc74FfO5eo9jmjKbyi paJw== X-Gm-Message-State: ALoCoQmlImCbwG+nl9qL8L4oLPD3yIuPrcktYCDxGlXgamvVBEPmDOU0GRpskVLWOtsFGZDPmEsY X-Received: by 10.180.187.130 with SMTP id fs2mr24224954wic.24.1415111712836; Tue, 04 Nov 2014 06:35:12 -0800 (PST) Received: from anthracite.localdomain (82-71-55-202.dsl.in-addr.zen.co.uk. [82.71.55.202]) by mx.google.com with ESMTPSA id w5sm1229004wif.18.2014.11.04.06.35.11 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 04 Nov 2014 06:35:12 -0800 (PST) From: John Spray To: ceph-devel@vger.kernel.org Cc: John Spray Subject: [PATCH 3/7] ceph: add ceph_osdc_cancel_writes Date: Tue, 4 Nov 2014 14:34:25 +0000 Message-Id: <1415111669-26246-4-git-send-email-john.spray@redhat.com> X-Mailer: git-send-email 1.9.3 In-Reply-To: <1415111669-26246-1-git-send-email-john.spray@redhat.com> References: <1415111669-26246-1-git-send-email-john.spray@redhat.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP To allow us to abort writes in ENOSPC conditions, instead of having them block indefinitely. Signed-off-by: John Spray --- include/linux/ceph/osd_client.h | 8 +++++ net/ceph/osd_client.c | 67 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/include/linux/ceph/osd_client.h b/include/linux/ceph/osd_client.h index 7cb5cea..f82000c 100644 --- a/include/linux/ceph/osd_client.h +++ b/include/linux/ceph/osd_client.h @@ -21,6 +21,7 @@ struct ceph_authorizer; /* * completion callback for async writepages */ +typedef void (*ceph_osdc_full_callback_t)(struct ceph_osd_client *, void *); typedef void (*ceph_osdc_callback_t)(struct ceph_osd_request *, struct ceph_msg *); typedef void (*ceph_osdc_unsafe_callback_t)(struct ceph_osd_request *, bool); @@ -226,6 +227,9 @@ struct ceph_osd_client { u64 event_count; struct workqueue_struct *notify_wq; + + ceph_osdc_full_callback_t map_cb; + void *map_p; }; extern int ceph_osdc_setup(void); @@ -331,6 +335,7 @@ extern void ceph_osdc_put_request(struct ceph_osd_request *req); extern int ceph_osdc_start_request(struct ceph_osd_client *osdc, struct ceph_osd_request *req, bool nofail); +extern u32 ceph_osdc_cancel_writes(struct ceph_osd_client *osdc, int r); extern void ceph_osdc_cancel_request(struct ceph_osd_request *req); extern int ceph_osdc_wait_request(struct ceph_osd_client *osdc, struct ceph_osd_request *req); @@ -361,5 +366,8 @@ extern int ceph_osdc_create_event(struct ceph_osd_client *osdc, void *data, struct ceph_osd_event **pevent); extern void ceph_osdc_cancel_event(struct ceph_osd_event *event); extern void ceph_osdc_put_event(struct ceph_osd_event *event); + +extern void ceph_osdc_register_map_cb(struct ceph_osd_client *osdc, + ceph_osdc_full_callback_t cb, void *data); #endif diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index 75ab07c..eb7e735 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -836,6 +836,59 @@ __lookup_request_ge(struct ceph_osd_client *osdc, return NULL; } +/* + * Drop all pending write/modify requests and complete + * them with the `r` as return code. + * + * Returns the highest OSD map epoch of a request that was + * cancelled, or 0 if none were cancelled. + */ +u32 ceph_osdc_cancel_writes( + struct ceph_osd_client *osdc, + int r) +{ + struct ceph_osd_request *req; + struct rb_node *n = osdc->requests.rb_node; + u32 latest_epoch = 0; + + dout("enter cancel_writes r=%d", r); + + mutex_lock(&osdc->request_mutex); + + while (n) { + req = rb_entry(n, struct ceph_osd_request, r_node); + n = rb_next(n); + + ceph_osdc_get_request(req); + if (req->r_flags && CEPH_OSD_FLAG_WRITE) { + req->r_result = r; + complete_all(&req->r_completion); + complete_all(&req->r_safe_completion); + + if (req->r_callback) { + // Requires callbacks used for write ops are + // amenable to being called with NULL msg + // (e.g. writepages_finish) + req->r_callback(req, NULL); + } + + __unregister_request(osdc, req); + + if (*req->r_request_osdmap_epoch > latest_epoch) { + latest_epoch = *req->r_request_osdmap_epoch; + } + } + ceph_osdc_put_request(req); + } + + mutex_unlock(&osdc->request_mutex); + + dout("complete cancel_writes latest_epoch=%ul", latest_epoch); + + return latest_epoch; +} +EXPORT_SYMBOL(ceph_osdc_cancel_writes); + static void __kick_linger_request(struct ceph_osd_request *req) { struct ceph_osd_client *osdc = req->r_osdc; @@ -2102,6 +2155,10 @@ done: downgrade_write(&osdc->map_sem); ceph_monc_got_osdmap(&osdc->client->monc, osdc->osdmap->epoch); + if (osdc->map_cb) { + osdc->map_cb(osdc, osdc->map_p); + } + /* * subscribe to subsequent osdmap updates if full to ensure * we find out when we are no longer full and stop returning @@ -2125,6 +2182,14 @@ bad: up_write(&osdc->map_sem); } +void ceph_osdc_register_map_cb(struct ceph_osd_client *osdc, + ceph_osdc_full_callback_t cb, void *data) +{ + osdc->map_cb = cb; + osdc->map_p = data; +} +EXPORT_SYMBOL(ceph_osdc_register_map_cb); + /* * watch/notify callback event infrastructure * @@ -2553,6 +2618,8 @@ int ceph_osdc_init(struct ceph_osd_client *osdc, struct ceph_client *client) spin_lock_init(&osdc->event_lock); osdc->event_tree = RB_ROOT; osdc->event_count = 0; + osdc->map_cb = NULL; + osdc->map_p = NULL; schedule_delayed_work(&osdc->osds_timeout_work, round_jiffies_relative(osdc->client->options->osd_idle_ttl * HZ));