From patchwork Fri Jan 23 11:11:25 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Chaitanya Huilgol X-Patchwork-Id: 5692941 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 3B2B69F302 for ; Fri, 23 Jan 2015 11:11:34 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 5EE5B202B4 for ; Fri, 23 Jan 2015 11:11:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 132B8202E5 for ; Fri, 23 Jan 2015 11:11:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755149AbbAWLLa (ORCPT ); Fri, 23 Jan 2015 06:11:30 -0500 Received: from mail-pa0-f50.google.com ([209.85.220.50]:56177 "EHLO mail-pa0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755140AbbAWLL3 (ORCPT ); Fri, 23 Jan 2015 06:11:29 -0500 Received: by mail-pa0-f50.google.com with SMTP id rd3so7194633pab.9 for ; Fri, 23 Jan 2015 03:11:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:date:to:subject:message-id:user-agent:mime-version :content-type; bh=G00hQ+VRahuv/hWNnn555wE1NCWAboVEK41bNaoOt/k=; b=bBgpgWSyy4KQk8YhII871nj7O9znmHDAq7eso8t93H35T5naF9qDG0lCtbRZqBHNLi TrbJrRfgkNc8Ia99Spda1etuTpPuf8qcCu+uHhIXPfBzVVQvB0qD18lAr+SERT9yQaoi hG4HE9y/o+6EvgNAHaypxEALowA7lXIh335vJg//DG94I31J71poP4eoUtMHnuKb4sfh HiHyAdPaatpCXBtKdQ1zQJhw3gnM4ElULYre8/sMY4lhatcbumt+MYdpv39xSleau3gk ddAc+0f0r2a2rtgzMuUqYM8yFKagcXNzSIkHtsOWx/IbdXxqOzvPU6ClQ5Woi9Aftm8K yrlg== X-Received: by 10.70.42.208 with SMTP id q16mr10616764pdl.56.1422011488901; Fri, 23 Jan 2015 03:11:28 -0800 (PST) Received: from omr-cteravm-gm.sdcorp.global.sandisk.com ([182.71.241.147]) by mx.google.com with ESMTPSA id w4sm1597352pdm.43.2015.01.23.03.11.27 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Jan 2015 03:11:28 -0800 (PST) From: Chaitanya Huilgol X-Google-Original-From: Chaitanya Huilgol Date: Fri, 23 Jan 2015 16:41:25 +0530 (IST) To: ceph-devel@vger.kernel.org Subject: [PATCH 1/2] libceph: tcp_nodelay support Message-ID: User-Agent: Alpine 2.10 (DEB 1266 2009-07-14) MIME-Version: 1.0 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_HI, T_DKIM_INVALID, T_RP_MATCHES_RCVD, T_TVD_MIME_EPI,UNPARSEABLE_RELAY autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Chaitanya Huilgol TCP_NODELAY socket option set on connection sockets, disables Nagle’s algorithm and improves latency characteristics. tcp_nodelay(default)/no_tcp_nodelay option flags provided to enable/disable setting the socket option. Signed-off-by: Chaitanya Huilgol Reviewed-by: Ilya Dryomov --- include/linux/ceph/libceph.h | 1 + include/linux/ceph/messenger.h | 4 +++- net/ceph/ceph_common.c | 14 +++++++++++++- net/ceph/messenger.c | 21 ++++++++++++++++++++- 4 files changed, 37 insertions(+), 3 deletions(-) diff --git a/include/linux/ceph/libceph.h b/include/linux/ceph/libceph.h index 8b11a79..4919eaa 100644 --- a/include/linux/ceph/libceph.h +++ b/include/linux/ceph/libceph.h @@ -30,6 +30,7 @@ #define CEPH_OPT_MYIP (1<<2) /* specified my ip */ #define CEPH_OPT_NOCRC (1<<3) /* no data crc on writes */ #define CEPH_OPT_NOMSGAUTH (1<<4) /* not require cephx message signature */ +#define CEPH_OPT_NO_TCP_NODELAY (1<<5) /* no TCP_NODELAY on TCP sockets */ #define CEPH_OPT_DEFAULT (0) diff --git a/include/linux/ceph/messenger.h b/include/linux/ceph/messenger.h index d9d396c..e50cade 100644 --- a/include/linux/ceph/messenger.h +++ b/include/linux/ceph/messenger.h @@ -57,6 +57,7 @@ struct ceph_messenger { atomic_t stopping; bool nocrc; + bool no_tcp_nodelay; /* * the global_seq counts connections i (attempt to) initiate @@ -264,7 +265,8 @@ extern void ceph_messenger_init(struct ceph_messenger *msgr, struct ceph_entity_addr *myaddr, u64 supported_features, u64 required_features, - bool nocrc); + bool nocrc, + bool no_tcp_nodelay); extern void ceph_con_init(struct ceph_connection *con, void *private, const struct ceph_connection_operations *ops, diff --git a/net/ceph/ceph_common.c b/net/ceph/ceph_common.c index 5d5ab67..55d5e84 100644 --- a/net/ceph/ceph_common.c +++ b/net/ceph/ceph_common.c @@ -239,6 +239,8 @@ enum { Opt_nocrc, Opt_cephx_require_signatures, Opt_nocephx_require_signatures, + Opt_tcp_nodelay, + Opt_no_tcp_nodelay, }; static match_table_t opt_tokens = { @@ -259,6 +261,8 @@ static match_table_t opt_tokens = { {Opt_nocrc, "nocrc"}, {Opt_cephx_require_signatures, "cephx_require_signatures"}, {Opt_nocephx_require_signatures, "nocephx_require_signatures"}, + {Opt_tcp_nodelay, "tcp_nodelay"}, + {Opt_no_tcp_nodelay, "no_tcp_nodelay"}, {-1, NULL} }; @@ -464,6 +468,13 @@ ceph_parse_options(char *options, const char *dev_name, opt->flags |= CEPH_OPT_NOMSGAUTH; break; + case Opt_tcp_nodelay: + opt->flags &= ~CEPH_OPT_NO_TCP_NODELAY; + break; + case Opt_no_tcp_nodelay: + opt->flags |= CEPH_OPT_NO_TCP_NODELAY; + break; + default: BUG_ON(token); } @@ -521,7 +532,8 @@ struct ceph_client *ceph_create_client(struct ceph_options *opt, void *private, ceph_messenger_init(&client->msgr, myaddr, client->supported_features, client->required_features, - ceph_test_opt(client, NOCRC)); + ceph_test_opt(client, NOCRC), + ceph_test_opt(client, NO_TCP_NODELAY)); /* subsystems */ err = ceph_monc_init(&client->monc, client); diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 33a2f20..6464a17 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -469,6 +469,20 @@ static void set_sock_callbacks(struct socket *sock, /* * socket helpers */ +static void ceph_tcp_set_sock_options(struct ceph_connection *con) +{ + int rc; + + if (!con->msgr->no_tcp_nodelay) { + /* Not requested to disable TCP_NODELAY, set it by default */ + int optval = 1; + rc = kernel_setsockopt(con->sock, IPPROTO_TCP, TCP_NODELAY, + (char *)&optval, sizeof(optval)); + if (rc != 0) { + dout("Error: CEPH_CON_OPT: TCP_NODELAY: %d\n", rc); + } + } +} /* * initiate connection to a remote socket. @@ -513,6 +527,9 @@ static int ceph_tcp_connect(struct ceph_connection *con) sk_set_memalloc(sock->sk); con->sock = sock; + /* process socket options if any */ + ceph_tcp_set_sock_options(con); + return 0; } @@ -2922,7 +2939,8 @@ void ceph_messenger_init(struct ceph_messenger *msgr, struct ceph_entity_addr *myaddr, u64 supported_features, u64 required_features, - bool nocrc) + bool nocrc, + bool no_tcp_nodelay) { msgr->supported_features = supported_features; msgr->required_features = required_features; @@ -2937,6 +2955,7 @@ void ceph_messenger_init(struct ceph_messenger *msgr, get_random_bytes(&msgr->inst.addr.nonce, sizeof(msgr->inst.addr.nonce)); encode_my_addr(msgr); msgr->nocrc = nocrc; + msgr->no_tcp_nodelay = no_tcp_nodelay; atomic_set(&msgr->stopping, 0);