From patchwork Tue Mar 5 13:53:33 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2219281 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork2.kernel.org (Postfix) with ESMTP id B8687DF24C for ; Tue, 5 Mar 2013 13:53:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755163Ab3CENxh (ORCPT ); Tue, 5 Mar 2013 08:53:37 -0500 Received: from mail-ia0-f176.google.com ([209.85.210.176]:53963 "EHLO mail-ia0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755069Ab3CENxg (ORCPT ); Tue, 5 Mar 2013 08:53:36 -0500 Received: by mail-ia0-f176.google.com with SMTP id i18so6059077iac.35 for ; Tue, 05 Mar 2013 05:53:36 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=in2wXmZ4jtKaQDeqQ9CaFE8fExEYAkDFNJjmtGVpSus=; b=oq2dplxYPoP7gWDyyCM9r4ApejQehO3N/nvSEUkUgSL266pLJs5N3/PKb2BAf8NalZ C5ahWpMFxYLp8uWd7zqRrbIWODJZpbeBK1/Z0PGE1KVedUQQE8uMvyZ6A33WWalp+R/w ovg80Hv5CoXdKNOnzQoNgcTVak1+SybzO0i3YrOMeLUfP0RQTyVAd2kD4b6EuuQWmOUG k8bap6/nzOiR4mNWIUGqdR+pDCrQI0T7Fy9/Bhs2EB+CKyOT8zhDlj3JCFxm9kBgcTS/ 4Mlx8FaE1/BC774J0HR+WJ/dyEd7MBXGmGcGpZB30g2zYfdi9dnApvUBbdBDIjbMt7+6 vz8A== X-Received: by 10.50.53.176 with SMTP id c16mr6014873igp.36.1362491615986; Tue, 05 Mar 2013 05:53:35 -0800 (PST) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPS id l2sm15683400igb.1.2013.03.05.05.53.33 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 05 Mar 2013 05:53:34 -0800 (PST) Message-ID: <5135F8DD.3030602@inktank.com> Date: Tue, 05 Mar 2013 07:53:33 -0600 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 4/5] libceph: set response data fields earlier References: <5135F859.1090606@inktank.com> In-Reply-To: <5135F859.1090606@inktank.com> X-Gm-Message-State: ALoCoQneCt100N0W3W9xXrjiOoPK4zuthtMxUFRj6P89fm1ac+xAAy7yEHQQQEOd21cNhcFDvu+H Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org When an incoming message is destined for the osd client, the messenger calls the osd client's alloc_msg method. That function looks up which request has the tid matching the incoming message, and returns the request message that was preallocated to receive the response. The response message is therefore known before the request is even started. Between the start of the request and the receipt of the response, the request and its data fields will not change, so there's no reason we need to hold off setting them. In fact it's preferable to set them just once because it's more obvious that they're unchanging. So set up the fields describing where incoming data is to land in a response message at the beginning of ceph_osdc_start_request(). Define a helper function that sets these fields, and use it to set the fields for both outgoing data in the request message and incoming data in the response. This resolves: http://tracker.ceph.com/issues/4284 Signed-off-by: Alex Elder --- net/ceph/osd_client.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) @@ -2126,13 +2131,6 @@ static struct ceph_msg *get_reply(struct ceph_connection *con, m = NULL; goto out; } - - ceph_msg_data_set_pages(m, osd_data->pages, - osd_data->num_pages, osd_data->alignment); -#ifdef CONFIG_BLOCK - } else if (osd_data->type == CEPH_OSD_DATA_TYPE_BIO) { - ceph_msg_data_set_bio(m, osd_data->bio); -#endif } } *skip = 0; diff --git a/net/ceph/osd_client.c b/net/ceph/osd_client.c index c19188a..1bb2be9 100644 --- a/net/ceph/osd_client.c +++ b/net/ceph/osd_client.c @@ -1739,6 +1739,21 @@ bad: return; } +static void ceph_osdc_msg_data_set(struct ceph_msg *msg, + struct ceph_osd_data *osd_data) +{ + if (osd_data->type == CEPH_OSD_DATA_TYPE_PAGES) { + ceph_msg_data_set_pages(msg, osd_data->pages, + osd_data->num_pages, osd_data->alignment); +#ifdef CONFIG_BLOCK + } else if (osd_data->type == CEPH_OSD_DATA_TYPE_BIO) { + ceph_msg_data_set_bio(msg, osd_data->bio); +#endif + } else { + BUG_ON(osd_data->type != CEPH_OSD_DATA_TYPE_NONE); + } +} + /* * Register request, send initial attempt. */ @@ -1747,22 +1762,12 @@ int ceph_osdc_start_request(struct ceph_osd_client *osdc, bool nofail) { int rc = 0; - struct ceph_osd_data *osd_data; - /* Set up outgoing data */ + /* Set up request outgoing data and response incoming data fields */ - osd_data = &req->r_data_out; - if (osd_data->type == CEPH_OSD_DATA_TYPE_PAGES) { - ceph_msg_data_set_pages(req->r_request, osd_data->pages, - osd_data->num_pages, osd_data->alignment); -#ifdef CONFIG_BLOCK - } else if (osd_data->type == CEPH_OSD_DATA_TYPE_BIO) { - ceph_msg_data_set_bio(req->r_request, osd_data->bio); -#endif - } else { - BUG_ON(osd_data->type != CEPH_OSD_DATA_TYPE_NONE); - } + ceph_osdc_msg_data_set(req->r_request, &req->r_data_out); ceph_msg_data_set_trail(req->r_request, &req->r_trail); + ceph_osdc_msg_data_set(req->r_reply, &req->r_data_in); register_request(osdc, req);