From patchwork Sat Mar 9 16:03:36 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2241841 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id 061F2DF2F2 for ; Sat, 9 Mar 2013 16:03:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932727Ab3CIQDj (ORCPT ); Sat, 9 Mar 2013 11:03:39 -0500 Received: from mail-ie0-f177.google.com ([209.85.223.177]:36688 "EHLO mail-ie0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932401Ab3CIQDi (ORCPT ); Sat, 9 Mar 2013 11:03:38 -0500 Received: by mail-ie0-f177.google.com with SMTP id 16so3119052iea.8 for ; Sat, 09 Mar 2013 08:03:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=WAgjrVl4eynA51eHXvgiuaWEWKA6IFl+g0DKuC/6POc=; b=TjxP2mNGm4f0QTiTiPeHqre+RkmxJx5cmykB4djo7TUsW9nS2f6A8JPZF8zLHvNRZn FzR+1DEUaEbCWbMTd62aLqkbakElWtCYtYBXJJVe+lMEQp9DR6DXmog7xxlNybRSa6P2 TNDwNS/yTF/mJj8RuHrwQGqKCM8YHFahe8Xt4HQwlzmLP2/m2SiLyYvL4afY8+0wMVcE ijL255iH0MQMRzmIkjcTlWdmpksFgTokhb51WEcxLJQP46LKavFHlbLM6/TO93Ht0kZu szuEcjv2aX+g0hTuXUVcPEMVufjU8+eSbsSvaQXWHPunPJ04I1xP5RDmmvkw5/zKaAvt wsAQ== X-Received: by 10.50.57.232 with SMTP id l8mr2737185igq.49.1362845018247; Sat, 09 Mar 2013 08:03:38 -0800 (PST) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPS id dy5sm4706024igc.1.2013.03.09.08.03.36 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Sat, 09 Mar 2013 08:03:37 -0800 (PST) Message-ID: <513B5D58.8060809@inktank.com> Date: Sat, 09 Mar 2013 10:03:36 -0600 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 6/7] libceph: set response data fields earlier References: <513B5CAF.4010702@inktank.com> In-Reply-To: <513B5CAF.4010702@inktank.com> X-Gm-Message-State: ALoCoQmkIkJK5G5x+TW2ky8N8caz0bRKpxBWiWWx4vBlJplfnTHifbar9nGsdlQBCqgbRDi59xIi Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org When an incoming message is destined for the osd client, the messenger calls the osd client's alloc_msg method. That function looks up which request has the tid matching the incoming message, and returns the request message that was preallocated to receive the response. The response message is therefore known before the request is even started. Between the start of the request and the receipt of the response, the request and its data fields will not change, so there's no reason we need to hold off setting them. In fact it's preferable to set them just once because it's more obvious that they're unchanging. So set up the fields describing where incoming data is to land in a response message at the beginning of ceph_osdc_start_request(). Define a helper function that sets these fields, and use it to set the fields for both outgoing data in the request message and incoming data in the response. This resolves: http://tracker.ceph.com/issues/4284 Signed-off-by: Alex Elder --- net/ceph/osd_client.c | 43 ++++++++++++++++++++----------------------- 1 file changed, 20 insertions(+), 23 deletions(-) *skip = 0; diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index 4402e91..37d8961 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -1744,32 +1744,36 @@ bad: return; } -/* - * Register request, send initial attempt. - */ -int ceph_osdc_start_request(struct ceph_osd_client *osdc, - struct ceph_osd_request *req, - bool nofail) +static void ceph_osdc_msg_data_set(struct ceph_msg *msg, + struct ceph_osd_data *osd_data) { - int rc = 0; - struct ceph_osd_data *osd_data; - - /* Set up outgoing data */ - - osd_data = &req->r_data_out; if (osd_data->type == CEPH_OSD_DATA_TYPE_PAGES) { BUG_ON(osd_data->length > (u64) SIZE_MAX); if (osd_data->length) - ceph_msg_data_set_pages(req->r_request, - osd_data->pages, osd_data->length, - osd_data->alignment); + ceph_msg_data_set_pages(msg, osd_data->pages, + osd_data->length, osd_data->alignment); #ifdef CONFIG_BLOCK } else if (osd_data->type == CEPH_OSD_DATA_TYPE_BIO) { - ceph_msg_data_set_bio(req->r_request, osd_data->bio); + ceph_msg_data_set_bio(msg, osd_data->bio); #endif } else { BUG_ON(osd_data->type != CEPH_OSD_DATA_TYPE_NONE); } +} + +/* + * Register request, send initial attempt. + */ +int ceph_osdc_start_request(struct ceph_osd_client *osdc, + struct ceph_osd_request *req, + bool nofail) +{ + int rc = 0; + + /* Set up response incoming data and request outgoing data fields */ + + ceph_osdc_msg_data_set(req->r_reply, &req->r_data_in); + ceph_osdc_msg_data_set(req->r_request, &req->r_data_out); if (req->r_trail.length) ceph_msg_data_set_trail(req->r_request, &req->r_trail); @@ -2130,13 +2134,6 @@ static struct ceph_msg *get_reply(struct ceph_connection *con, m = NULL; goto out; } - BUG_ON(osd_data->length > (u64) SIZE_MAX); - ceph_msg_data_set_pages(m, osd_data->pages, - osd_data->length, osd_data->alignment); -#ifdef CONFIG_BLOCK - } else if (osd_data->type == CEPH_OSD_DATA_TYPE_BIO) { - ceph_msg_data_set_bio(m, osd_data->bio); -#endif } }