From patchwork Fri Nov 23 12:40:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10695749 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8F8DC175A for ; Fri, 23 Nov 2018 12:40:43 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 80A2F2C013 for ; Fri, 23 Nov 2018 12:40:43 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7515C2C080; Fri, 23 Nov 2018 12:40:43 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 13D502C057 for ; Fri, 23 Nov 2018 12:40:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2409863AbeKWXYo (ORCPT ); Fri, 23 Nov 2018 18:24:44 -0500 Received: from mail-wr1-f68.google.com ([209.85.221.68]:45788 "EHLO mail-wr1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2409857AbeKWXYn (ORCPT ); Fri, 23 Nov 2018 18:24:43 -0500 Received: by mail-wr1-f68.google.com with SMTP id v6so12198226wrr.12 for ; Fri, 23 Nov 2018 04:40:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=RasxTOW8z8b2iOFHaISwLWQ0ieYMTTVtj45gAbBESWs=; b=ZX9KoSIFemTpbLYbPd12eqmqEn4fG12gEsO3Q5aiA3+JDD9XwBR0EaRA6t1lwUc64s WnU8OjDVGmF131uqO5P52aDpP9p4YoMoxu87geh/op4XdkkrbflIhZdrBVPYpdHp0nDZ iO2ml+Qd2k6vd9nOj2knTKwOfYKv92vOpmKwU0ng8FZ/6pssVLTwgz6/ryZ9DiW2/6QE SlwMKrgyxBG3PC5isnuPARWd2R1V7g7mQsU+vG9ku5ub0VtiiSnQJyme60No7z0f66Gf E/7kStEFOAraXgI18232UTcPovddfp4E0DQxnomlwZ32Vkgaw+5qKCUs/znDqFHgc3Mg puIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=RasxTOW8z8b2iOFHaISwLWQ0ieYMTTVtj45gAbBESWs=; b=cLg7Haxw/gFfjYThBoF9gMwx4YJPZxl1mDAUQO4hVS4CIP92F8K4tfC52Y1JhNRHNY MPqk29lUZwt9l118w1bahYirLEPYPi7mEIvHCvy6lZDjTiXfAFg10+xVKad3xus064Mh KSXNmuMXDQIjBhEHT02FpC4AequlykWh2nAizlyv1wKCPJs+TPRZ8CTsFphNvB2ykxyi av2aziDGEkAN3KsI9yZmejx+Yo8W8TXUKVo3Tl8TZ44iK7CuG7It7AMjKkZDbTXUPxQz OfU8d5cdiJ0RoUvvKddaQTtTEcul45ZZ7qYqbs5L+greqfi5+gx114rGHwWfMCdygpgN MsgA== X-Gm-Message-State: AA+aEWbCc6Howe4wE6JrhUL8Y7uPPd+y/30XD10D+CD+rzvcTmPmcEMQ jOa4jx2dlToZZIT0h5vXsPftQ53v X-Google-Smtp-Source: AFSGD/UCWL1FTlFZNtA7Fz1sdJnWQdIBhCPSTaFGmWg0IMz7J0XFKOLnQ31dQsIaj+qBCIAZJWdjFQ== X-Received: by 2002:a5d:6302:: with SMTP id i2mr11746890wru.14.1542976839202; Fri, 23 Nov 2018 04:40:39 -0800 (PST) Received: from orange.local (ip-94-112-136-201.net.upcbroadband.cz. [94.112.136.201]) by smtp.gmail.com with ESMTPSA id l3sm31077857wru.36.2018.11.23.04.40.38 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Nov 2018 04:40:38 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 1/4] libceph: drop last_piece logic from write_partial_message_data() Date: Fri, 23 Nov 2018 13:40:17 +0100 Message-Id: <20181123124020.4637-2-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20181123124020.4637-1-idryomov@gmail.com> References: <20181123124020.4637-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP last_piece is for the last piece in the current data item, not in the entire data payload of the message. This is harmful for messages with multiple data items. On top of that, we don't need to signal the end of a data payload either because it is always followed by a footer. We used to signal "more" unconditionally, until commit fe38a2b67bc6 ("libceph: start defining message data cursor"). Part of a large series, it introduced cursor->last_piece and also mistakenly inverted the hint by passing last_piece for "more". This was corrected with commit c2cfa1940097 ("libceph: Fix ceph_tcp_sendpage()'s more boolean usage"). As it is, last_piece is not helping at all: because Nagle algorithm is disabled, for a simple message with two 512-byte data items we end up emitting three packets: front + first data item, second data item and footer. Go back to the original pre-fe38a2b67bc6 behavior -- a single packet in most cases. Signed-off-by: Ilya Dryomov --- net/ceph/messenger.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 2f126eff275d..cca96d32ac64 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -1592,7 +1592,6 @@ static int write_partial_message_data(struct ceph_connection *con) struct page *page; size_t page_offset; size_t length; - bool last_piece; int ret; if (!cursor->resid) { @@ -1600,10 +1599,9 @@ static int write_partial_message_data(struct ceph_connection *con) continue; } - page = ceph_msg_data_next(cursor, &page_offset, &length, - &last_piece); - ret = ceph_tcp_sendpage(con->sock, page, page_offset, - length, !last_piece); + page = ceph_msg_data_next(cursor, &page_offset, &length, NULL); + ret = ceph_tcp_sendpage(con->sock, page, page_offset, length, + true); if (ret <= 0) { if (do_datacrc) msg->footer.data_crc = cpu_to_le32(crc); From patchwork Fri Nov 23 12:40:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10695751 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4AA5D15A7 for ; Fri, 23 Nov 2018 12:40:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3B10B2C013 for ; Fri, 23 Nov 2018 12:40:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2F7D42C018; Fri, 23 Nov 2018 12:40:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B458F2C080 for ; Fri, 23 Nov 2018 12:40:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2409866AbeKWXYp (ORCPT ); Fri, 23 Nov 2018 18:24:45 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:35159 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405500AbeKWXYp (ORCPT ); Fri, 23 Nov 2018 18:24:45 -0500 Received: by mail-wm1-f68.google.com with SMTP id c126so11836756wmh.0 for ; Fri, 23 Nov 2018 04:40:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=JyWduGT42lpp+GmDS7+UVatnQsk6KZz0kFiRaNmIeBk=; b=n1JQR8GJtphd3Vuxm61BLj0tc444oOg1CsrNXjIEM7DSO5J3lrTFky/2+Kp2MLteEk v1L9Q4FJNA4kvw/ubKfugyZVzVkI3z9hOeVbH4VvkowUHYWBS8HXE7S9eNwa7dToaWBu JpI2M8kQNTDm9gdtC7U2qPTcux77P37uHs+Nw91Qy5qR1Oicaf1TyY37MOV+zsoOofqx wL2OZIZsckl/VnB0HdXcTxsMmOgru6xexhsWlLgzVThMtw469xtDdrHD4xK9C4m8Tzqb yiZ1S4VqhieTvkJGgNb5Ie1zzu2BOwunDab+uzjsqn1lOFPB1G7yh4JPn7h4hQHJNwFe DrjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=JyWduGT42lpp+GmDS7+UVatnQsk6KZz0kFiRaNmIeBk=; b=YAQEQ8berQFE9+DKm4WW+TtTMd4BpSYKDOAyPLFnpbHRzJW7uk3gDwdiDbfa0ZsVJ/ wpOuk9KYfCFL7bPW9xIvR2uahaa6Ufzjron+6k/LBPoAtyrLV49sNUyGXbZgJ3xDePyB xo39zXdA22XMDwrws8+yNpCnu4Dgbj9CvzNb9D3wGZO5RKW7P58qVzsShfMV/BD/S/zG qHkTritpN3GflqDY55/a6zhn5gxcRxB9hFOs6ZN5iH4zbFBE6m0N7Gek0gk8B+UeRkhW esuic+sv1sC1bjLkjsqF9Kf/fRuizpAN6LLGEldntNKWRj+1mChd0aXM2bemkcRvx5EX Lw6A== X-Gm-Message-State: AGRZ1gJOZVIxrA1rRPA3dxD6RJ/0ESVS5JAdKZGfI/XOSw42qY0W7SWP NdanbXsCJ7cdAbCwG1MvdRsEezPD X-Google-Smtp-Source: AJdET5etQKHfxNKuBX0ZMQAbwo+QIId0rxr7VedGWHicKMfvA+QBzTJKvCokIsoXGR8xFlUY6D6bZA== X-Received: by 2002:a1c:2d90:: with SMTP id t138-v6mr3347671wmt.0.1542976840061; Fri, 23 Nov 2018 04:40:40 -0800 (PST) Received: from orange.local (ip-94-112-136-201.net.upcbroadband.cz. [94.112.136.201]) by smtp.gmail.com with ESMTPSA id l3sm31077857wru.36.2018.11.23.04.40.39 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Nov 2018 04:40:39 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 2/4] libceph: use sock_no_sendpage() as a fallback in ceph_tcp_sendpage() Date: Fri, 23 Nov 2018 13:40:18 +0100 Message-Id: <20181123124020.4637-3-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20181123124020.4637-1-idryomov@gmail.com> References: <20181123124020.4637-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP sock_no_sendpage() makes the code cleaner. Also, don't set MSG_EOR. sendpage doesn't act on MSG_EOR on its own, it just honors the setting from the preceding sendmsg call by looking at ->eor in tcp_skb_can_collapse_to(). Signed-off-by: Ilya Dryomov --- net/ceph/messenger.c | 32 ++++++-------------------------- 1 file changed, 6 insertions(+), 26 deletions(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index cca96d32ac64..21a743a3bd29 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -560,24 +560,12 @@ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov, return r; } -static int __ceph_tcp_sendpage(struct socket *sock, struct page *page, - int offset, size_t size, bool more) -{ - int flags = MSG_DONTWAIT | MSG_NOSIGNAL | (more ? MSG_MORE : MSG_EOR); - int ret; - - ret = kernel_sendpage(sock, page, offset, size, flags); - if (ret == -EAGAIN) - ret = 0; - - return ret; -} - static int ceph_tcp_sendpage(struct socket *sock, struct page *page, int offset, size_t size, bool more) { - struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL }; - struct bio_vec bvec; + ssize_t (*sendpage)(struct socket *sock, struct page *page, + int offset, size_t size, int flags); + int flags = MSG_DONTWAIT | MSG_NOSIGNAL | (more ? MSG_MORE : 0); int ret; /* @@ -589,19 +577,11 @@ static int ceph_tcp_sendpage(struct socket *sock, struct page *page, * triggers one of hardened usercopy checks. */ if (page_count(page) >= 1 && !PageSlab(page)) - return __ceph_tcp_sendpage(sock, page, offset, size, more); - - bvec.bv_page = page; - bvec.bv_offset = offset; - bvec.bv_len = size; - - if (more) - msg.msg_flags |= MSG_MORE; + sendpage = sock->ops->sendpage; else - msg.msg_flags |= MSG_EOR; /* superfluous, but what the hell */ + sendpage = sock_no_sendpage; - iov_iter_bvec(&msg.msg_iter, WRITE, &bvec, 1, size); - ret = sock_sendmsg(sock, &msg); + ret = sendpage(sock, page, offset, size, flags); if (ret == -EAGAIN) ret = 0; From patchwork Fri Nov 23 12:40:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10695753 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6EC31175A for ; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5D04C2C013 for ; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 51A272C018; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E65AB2C050 for ; Fri, 23 Nov 2018 12:40:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2409869AbeKWXYq (ORCPT ); Fri, 23 Nov 2018 18:24:46 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:40103 "EHLO mail-wm1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2409857AbeKWXYq (ORCPT ); Fri, 23 Nov 2018 18:24:46 -0500 Received: by mail-wm1-f65.google.com with SMTP id q26so11825228wmf.5 for ; Fri, 23 Nov 2018 04:40:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=from:to:subject:date:message-id:in-reply-to:references; bh=EWw8HerqCtPx4Hysb0uHq2kzbCLssiK+EkAIKzeXRHA=; b=jh/RI7BvPjjVYEh/0fTMUcJxKL3ftyP5MHAxOAHigOMCIgLH3EDBGgpQR4/uzfyBiV xwxePZG1MgFI8uepv7Z1K8ZxxNCbA9fugrMsur94f+06xeUOc1KV9fCk2X1OxV6ihktc B7avI0e+znwdHmH5fj+SwECfVRUXKjBHTMFe3WGt5mW8x4kjoPSfHPlst3tMAq50G9/f XK7a6HiEOEoa80/+TOSioGGmpfbXCZJ0ia7bLmIIX3P9tvT9lOUQG57YPhEwJQ/ckNCx Cw7WzjJoqw3qdWZ5fBFVjCtC3TfER2BCOeA/CUKmnlyd4l8BeSrva9QGFncfFjZ8RSpf EctQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=EWw8HerqCtPx4Hysb0uHq2kzbCLssiK+EkAIKzeXRHA=; b=RqGdplOWHbzY5P9uV6mLPg+TZ8PmQzjjrLblD8qC0I+fL5jiWFxErfo52/ARQcoP24 z5T7DV40Ob/1F1oY6g5eGtAgSvZXWOZdYGJKwgMbfFNY4Tl3PEwdeMpT8NEtcwqtYoK/ 0yXGmtWUXmISSXe0dtXY/DCv3BYjxO7BUTNpr2mk11pupuAOthKZpCtoub6suy02nlhy 2wk709xuJDlwXAUTfGv5Vso+mzwrMKLarLfSX+8QAarcyMm0hqKhzzF5oLvCUQgyLi82 Itx7zzaAD2ERm0jMJZSNzmi0g6Jz87wzRa6oxhSXAQTVchZFmMi7692NmzTBieLYlTRZ vLIQ== X-Gm-Message-State: AGRZ1gLRovsJC8wpycWloc70aozXf2sqsy3OapFuoyV/Kk16boi23PZc 7O1KZLCcmDwS00duCaHAJau+ecCF X-Google-Smtp-Source: AJdET5fQczT75/f7Mn0/qbj6jYIOQ/+3BHuT172yDQds+gwoVwl8cnq2nc0JWBxVHvrEX4y3osp3kA== X-Received: by 2002:a1c:2d90:: with SMTP id t138-v6mr3347685wmt.0.1542976841137; Fri, 23 Nov 2018 04:40:41 -0800 (PST) Received: from orange.local (ip-94-112-136-201.net.upcbroadband.cz. [94.112.136.201]) by smtp.gmail.com with ESMTPSA id l3sm31077857wru.36.2018.11.23.04.40.40 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Nov 2018 04:40:40 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 3/4] libceph: use MSG_SENDPAGE_NOTLAST with ceph_tcp_sendpage() Date: Fri, 23 Nov 2018 13:40:19 +0100 Message-Id: <20181123124020.4637-4-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20181123124020.4637-1-idryomov@gmail.com> References: <20181123124020.4637-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Prevent do_tcp_sendpages() from calling tcp_push() (at least) once per page. Instead, arrange for tcp_push() to be called (at least) once per data payload. This results in more MSS-sized packets and fewer packets overall (5-10% reduction in my tests with typical OSD request sizes). See commits 2f5338442425 ("tcp: allow splice() to build full TSO packets"), 35f9c09fe9c7 ("tcp: tcp_sendpages() should call tcp_push() once") and ae62ca7b0321 ("tcp: fix MSG_SENDPAGE_NOTLAST logic") for details. Here is an example of a packet size histogram for 128K OSD requests (MSS = 1448, top 5): Before: SIZE COUNT 1448 777700 952 127915 1200 39238 1219 9806 21 5675 After: SIZE COUNT 1448 897280 21 6201 1019 2797 643 2739 376 2479 We could do slightly better by explicitly corking the socket but it's not clear it's worth it. Signed-off-by: Ilya Dryomov --- net/ceph/messenger.c | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 21a743a3bd29..649faa626b35 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -560,12 +560,15 @@ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov, return r; } +/* + * @more: either or both of MSG_MORE and MSG_SENDPAGE_NOTLAST + */ static int ceph_tcp_sendpage(struct socket *sock, struct page *page, - int offset, size_t size, bool more) + int offset, size_t size, int more) { ssize_t (*sendpage)(struct socket *sock, struct page *page, int offset, size_t size, int flags); - int flags = MSG_DONTWAIT | MSG_NOSIGNAL | (more ? MSG_MORE : 0); + int flags = MSG_DONTWAIT | MSG_NOSIGNAL | more; int ret; /* @@ -1552,6 +1555,7 @@ static int write_partial_message_data(struct ceph_connection *con) struct ceph_msg *msg = con->out_msg; struct ceph_msg_data_cursor *cursor = &msg->cursor; bool do_datacrc = !ceph_test_opt(from_msgr(con->msgr), NOCRC); + int more = MSG_MORE | MSG_SENDPAGE_NOTLAST; u32 crc; dout("%s %p msg %p\n", __func__, con, msg); @@ -1580,8 +1584,10 @@ static int write_partial_message_data(struct ceph_connection *con) } page = ceph_msg_data_next(cursor, &page_offset, &length, NULL); + if (length == cursor->total_resid) + more = MSG_MORE; ret = ceph_tcp_sendpage(con->sock, page, page_offset, length, - true); + more); if (ret <= 0) { if (do_datacrc) msg->footer.data_crc = cpu_to_le32(crc); @@ -1611,13 +1617,16 @@ static int write_partial_message_data(struct ceph_connection *con) */ static int write_partial_skip(struct ceph_connection *con) { + int more = MSG_MORE | MSG_SENDPAGE_NOTLAST; int ret; dout("%s %p %d left\n", __func__, con, con->out_skip); while (con->out_skip > 0) { size_t size = min(con->out_skip, (int) PAGE_SIZE); - ret = ceph_tcp_sendpage(con->sock, zero_page, 0, size, true); + if (size == con->out_skip) + more = MSG_MORE; + ret = ceph_tcp_sendpage(con->sock, zero_page, 0, size, more); if (ret <= 0) goto out; con->out_skip -= ret; From patchwork Fri Nov 23 12:40:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilya Dryomov X-Patchwork-Id: 10695755 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id DD0B11926 for ; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id C9E3C2C013 for ; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id BED252C080; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 79D412C013 for ; Fri, 23 Nov 2018 12:40:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2409871AbeKWXYq (ORCPT ); Fri, 23 Nov 2018 18:24:46 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:52660 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2405500AbeKWXYq (ORCPT ); Fri, 23 Nov 2018 18:24:46 -0500 Received: by mail-wm1-f68.google.com with SMTP id r11-v6so11751700wmb.2 for ; Fri, 23 Nov 2018 04:40:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references; bh=q1vJXZTbD7LmwITQsF9hJF4lBCsj0g7NI+Gyf7j2EKU=; b=EV9P7jA0ZZrBec7zMpRnTfAeCRiJHu8Ftl6vvOGyK9mBfHaIngWxaNIExVeAC5XcKb hO74oLUozRzEVkjWSm6vL0vRUPOkSRHLx6d87AyctWm5Fb/JGzQoT/70vMgeHuQRkDv7 g3cl6PSMzuUMUpP0ucp14A4iYCKUscnUlCyDSQ+swLQCqpCSgpegAwBp24Imv4Al3ZhK oE9AtjY6vL/FrKM7jYxXlih7d9ZTrzNZ1Zbiqhw3lRBldoGvfMpAwhyyLgH+SdYiC3mv ofoXDvOEDerAukLyLPlCoiO48YL2FRaDrHwiazWxQ/dbEu+gyO3UaG+Zkz73QcM7Taey rWWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=q1vJXZTbD7LmwITQsF9hJF4lBCsj0g7NI+Gyf7j2EKU=; b=PhYtcRvQjFEgbEBd+Acq+drV7VQelYQ+b4lVsjBKYw4qUMouNoQdQoMs7sXEVkXppg uN/zK7qdP/7ybrJ2RV8NKRgKObp1H5kdksjF0fvpRntb7SfyaXmwo9xwIhJFGCUly6dE IrCxrhuTPbe4fkFeFY80CGlkTZzv/waefqm+xgOKko+z+TBJQYm6qGXHfkmhJKf8Ee2k 4ZuET8Dabi0ej995ieTtm0wcljYpvp6sGBNnjQYJlFOgjp5PBlZlmjRUQ53whdCjjuOL /9VkFJk9mLsuScM+dW8nUtLjXjNMMIZNxMTJCUj0Qj5zlcQtNAhGIynMT35Q5OhvnU/r D4VQ== X-Gm-Message-State: AGRZ1gKCItvKBKb3PyS8AmMy+8JdYWc72PctVUZfpY7cyjM7FL6sx50y lc1ITKS+QXCX/3kU6eoQ2mdAO1S3 X-Google-Smtp-Source: AJdET5dzYbzLmj34qBbo8Fjd9mGLx/k4HEUg9qw9rNaUFnV2vwqe3l8naxcbsw/dpdw+1ZAWJhOmEA== X-Received: by 2002:a7b:c1d6:: with SMTP id a22mr13079663wmj.48.1542976842272; Fri, 23 Nov 2018 04:40:42 -0800 (PST) Received: from orange.local (ip-94-112-136-201.net.upcbroadband.cz. [94.112.136.201]) by smtp.gmail.com with ESMTPSA id l3sm31077857wru.36.2018.11.23.04.40.41 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Nov 2018 04:40:41 -0800 (PST) From: Ilya Dryomov To: ceph-devel@vger.kernel.org Subject: [PATCH 4/4] libceph: switch more to bool in ceph_tcp_sendmsg() Date: Fri, 23 Nov 2018 13:40:20 +0100 Message-Id: <20181123124020.4637-5-idryomov@gmail.com> X-Mailer: git-send-email 2.14.4 In-Reply-To: <20181123124020.4637-1-idryomov@gmail.com> References: <20181123124020.4637-1-idryomov@gmail.com> Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Unlike in ceph_tcp_sendpage(), it's a bool. Signed-off-by: Ilya Dryomov --- net/ceph/messenger.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 649faa626b35..d5718284db57 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -544,7 +544,7 @@ static int ceph_tcp_recvpage(struct socket *sock, struct page *page, * shortly. */ static int ceph_tcp_sendmsg(struct socket *sock, struct kvec *iov, - size_t kvlen, size_t len, int more) + size_t kvlen, size_t len, bool more) { struct msghdr msg = { .msg_flags = MSG_DONTWAIT | MSG_NOSIGNAL }; int r;