Message ID | 1484860129-14727-1-git-send-email-jbacik@fb.com (mailing list archive) |
---|---|
State | New, archived |
Headers | show
Return-Path: <linux-block-owner@kernel.org> Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1E3A46020B for <patchwork-linux-block@patchwork.kernel.org>; Thu, 19 Jan 2017 21:09:02 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0509928451 for <patchwork-linux-block@patchwork.kernel.org>; Thu, 19 Jan 2017 21:09:02 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id EA84C28639; Thu, 19 Jan 2017 21:09:01 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F1B7428451 for <patchwork-linux-block@patchwork.kernel.org>; Thu, 19 Jan 2017 21:08:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751144AbdASVI6 (ORCPT <rfc822;patchwork-linux-block@patchwork.kernel.org>); Thu, 19 Jan 2017 16:08:58 -0500 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:54446 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751316AbdASVI4 (ORCPT <rfc822;linux-block@vger.kernel.org>); Thu, 19 Jan 2017 16:08:56 -0500 Received: from pps.filterd (m0044012.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v0JL4ip2025963 for <linux-block@vger.kernel.org>; Thu, 19 Jan 2017 13:08:56 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : subject : date : message-id : mime-version : content-type; s=facebook; bh=VrBxPEyPEWZlQAQwNmSqNCLRu2T6wDUd95Fp01swXKU=; b=p2fPd5QkobpdfBEN2BdqjG3AJLE93j0swuCoHynIIC262Jj93M0dj1nRGJAKgo7ubdto +Qqq+hcdEBTLeFZvG5xIvxaaiwvLmb3ajmPM1L6nPvHTpLfzWzq7zffIiGxqWm/pflsj 3EgXKf60+YPPLGMZukntveH1u0AEh7zM3ds= Received: from mail.thefacebook.com ([199.201.64.23]) by mx0a-00082601.pphosted.com with ESMTP id 28306h96w8-2 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for <linux-block@vger.kernel.org>; Thu, 19 Jan 2017 13:08:56 -0800 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (192.168.54.28) by o365-in.thefacebook.com (192.168.16.12) with Microsoft SMTP Server (TLS) id 14.3.294.0; Thu, 19 Jan 2017 13:08:55 -0800 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=VrBxPEyPEWZlQAQwNmSqNCLRu2T6wDUd95Fp01swXKU=; b=hgyQWXYG2qJF5eL+d8GrfIGIfXov+7dA61t53+jNkNqi4xwrwiv2mTt3o42cuoiwEx7v4MqbmHQs9Rigk1J9W9JZf9T2/uC52YmFd2zKN8dEmV8bHcl0hXxbuZoAoYi9B8zHMS8yWW7gnGay/5QRDSw2qL4LkIA534rUaVY7xVU= Received: from localhost (2620:10d:c091:180::2530) by CY4PR15MB1320.namprd15.prod.outlook.com (10.172.182.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.845.12; Thu, 19 Jan 2017 21:08:53 +0000 From: Josef Bacik <jbacik@fb.com> To: <linux-block@vger.kernel.org>, <axboe@fb.com> Subject: [PATCH] nbd: only set MSG_MORE when we have more to send Date: Thu, 19 Jan 2017 16:08:49 -0500 Message-ID: <1484860129-14727-1-git-send-email-jbacik@fb.com> X-Mailer: git-send-email 2.5.5 MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [2620:10d:c091:180::2530] X-ClientProxiedBy: DM5PR09CA0005.namprd09.prod.outlook.com (10.172.32.15) To CY4PR15MB1320.namprd15.prod.outlook.com (10.172.182.8) X-MS-Office365-Filtering-Correlation-Id: e7732a47-ed24-4c5e-b9e3-08d440af5fb5 X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001); SRVR:CY4PR15MB1320; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 3:bx/FKtgQCRPT/GaTic6hYD68xE73n8eY9FS5+EBgAQE5E7UFeL8rclNTh3PsIvdND5JuxcblvvaTtxXCoV4gcIE6UHd365RmVga3u3h6oGfWbsfrde2dSXLtYxK3dLdtH/q/o41G2jPXSEAU656udYL7p5bsNJjJ3alZLV35zQnDxImLCgh6fbJyisCNBUdBFUGnwcd+iCgRZn4WqioDzg5j6/yauNXmg0LZ576DQJb9rWbKpg5feIUXxj71istCAKr45wsGNsQPR6QjeRNrTA== X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 25:hdWUIo/vQ1rvzf/hemc9Y2vdmDCcfO5w02xSQP7y6eEjt4wSihXPLGSV2SSdX/d3A8lbHvQ6lu/Cn8yDJUMnCEd77ArNcYDGDvuuqCyWRbva8zjt5dHFbEa52zzKx0cfv13rRFMVZxDceNEJjm0gojxYiI5uSvcrSZWjLx8X2YUnERwjIrk7n4g/HxPhWIgJGkGFsODzo2ZQYBZScLzZujjtKw5W8DN2J4nNYkA4xGeZffxCKcHMTTAfZoh4r61qBE2eej8OlHmQGaBhsvu+6dpO+1aa1sLscJpkgRcTWMhn27NXEWeOX0iVL/CeswbrdNZ9aPlNRFpqCY/49qLdx7tqAaL/fyRR9BkhA/GfC2Y3EKQj/kLZBF9f9aKnGTmulmXRPlP7elYpKShUnesvXcLuHxxoDYKdztwrCjOKvVxEAKZZQY+tc1D6qcb+fgZp6/VHDCd9pQ5nCX99rOxS9hIwbhRkzsehPCyclhUEgnEuuUjbsqeRx8d4OAtvqhkOUlc75ioR5WsMzRjBBim71ydPV+1CVl+Pq86+bOKFurTzMvKc+9SzkWLrjJE+LvShp3abj6ayTGEtLR/1Nw9tNGK/zxj/WuzpftLZuYOU9axypOXlpSsfTIzBvNd2uJFgyEXA/322h7Dojoi44tKmzQQ3a8mIZ2n99S8guOLfQMpuyWHX9/TWEf4zc+/xKK4gZay6fZvfWIBNkyGFJUO1f2ntHQ8/aDQrJaG8M8QC63atHZOVrpnwOv9kEhr+xqz9 X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 31:cWnrLB51mwjwL/5zpz8Wo9brB+j/sQytixrWDOi5gXMpGC3ZUFVAc+pAy/pNYvlDBqFcP8iqu7eucGxk7+tzsRus++nxpH9BJPY1ZtKDmeFfSBx0ZjcAc2rV3p8l1gYrkiK7CaVsFDywnxO6U8Z5qjWMmXZEqos/mBRuD8x9cdfjIMvLqKTecxuPkUPcgvQLzgkROI3WUrP9s8U8R3VJZh5pzo1wUiM6PvvvvyRLUCn0knG0PegCL9LqULehpNVA; 20:zzPk6CfPNu/nxAdW6+mcKSixigLSBM4mrU4J/AXSSKOzagElG1YTJZ+UC8Gxy5QHuvGjK9qzK+Hfi9GtU+WQQsFeBkt4vVJ4JHOv3LzshdZN2N61T/IZuWthrnyfhBBIpYOsvnD04wqtNwb2AMLQi8Z+eCv55r0xZLuhvEpjaBfHGRkXdkbtTTFnvBuIJESsoOnsSiMEnhpKJFSpOT+mE18y8ZD0DsXrmaniEJXlsa3RODLVCVDBG73ZTLpDuNxm8IggCxK7MVRhU7pWvQFg0DNwtUIWUDars+gqoAvpiDZ50KmjdI9iSZ8FUY0Fji/252dN4gAp/t53ZFHTsl2YmrQbdTI1yzSrLysMXkzGKZK2X40uf2Xiwal6tHwgEWLMLtEQHcucz+qJUNHggMrUof6Ha5SXEMUeh6vmiHr2NLOdrreMfbO5I7MEghzrFtdOJ6DUSpAM6fbAlPZD8oljHbLdEq565SGT69lfEd7AtU6H3RMO00d2xuK98VDpBsu+ X-Microsoft-Antispam-PRVS: <CY4PR15MB1320EB168F41BC851439B322B97E0@CY4PR15MB1320.namprd15.prod.outlook.com> X-Exchange-Antispam-Report-Test: UriScan:(67672495146484); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040375)(601004)(2401047)(5005006)(8121501046)(10201501046)(3002001)(6041248)(20161123555025)(20161123564025)(20161123562025)(20161123560025)(6072148); SRVR:CY4PR15MB1320; BCL:0; PCL:0; RULEID:; SRVR:CY4PR15MB1320; X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 4:q2ywF6jgItlvktcV+/bIZzRFFH9ILYNrBWMKOUPoKVWsxvF7iAva0mk/o8laeguOjXRFdlDoRfKh/Eedbnft7dPTSRhyqGxRyz6ujhEvL26bqHSjViib/fzsfiN4Pdy/g+2sRUbL7RJlBtBTevMZFIk9SqEomgDQrqH6SODiOmVGf/N1mFF9UGgTeBTrOf6N0UM5RXm0MHvm5QWlxV6sMvdwd/OJGaAqKnVHyS6bhKyyAiCjwhTLHWhpURAKVReXIYaBTXQDC7GbGapuDlCDGwigCVTm//1NRH6xODVsf3OGi91vxk0jb4wW04EmVJKzI8OqzompLfgJH+WBTQ2HP1C+lt1DWJMfLIENR45eRthwINtl/f8MAZ0/KRzzy3hV5BOtRm9HKhwQlBR2yWlA+I1ABKP5797qGNh4/cYK74vQUQ2KYSl1IFYvSy8CcBlzz4dekp7ftlydwwLu/bTTW693dwfgJWZTG9dNFrrdhl0exrouhEsixCM27xcxSX/+WyBKtlYJxVF+sVX+3prvxXAgaDwtwLMi5r9yvflVWhklSke+zFQz+C9EAHbE6Wk2Sw1jHEy9FcAv36261MPY1DKefWxhO2lqBfkqh4jXIJIPyB7Atvd3d0zNsua07e0Y X-Forefront-PRVS: 0192E812EC X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(4630300001)(6069001)(6009001)(7916002)(39830400002)(39450400003)(39410400002)(199003)(189002)(189998001)(50226002)(38730400001)(5003940100001)(53936002)(2906002)(68736007)(5001770100001)(97736004)(101416001)(36756003)(105586002)(50466002)(107886002)(106356001)(47776003)(33646002)(8676002)(50986999)(81156014)(6116002)(76506005)(5660300001)(86362001)(81166006)(48376002)(6496003)(92566002)(450100001)(25786008)(6636002)(6486002)(305945005)(6666003)(7736002)(42186005); DIR:OUT; SFP:1102; SCL:1; SRVR:CY4PR15MB1320; H:localhost; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en; Received-SPF: None (protection.outlook.com: fb.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; CY4PR15MB1320; 23:AWB+SvUEtP4wpP+5mvLohAGJaQZzyobZQl0HkApwr?= =?us-ascii?Q?gO9tmhAKtiwoiBEwnBDpvxPF/XBehOB6aG5+5SyMzsS/1Zil9sNdzvGMkbfM?= =?us-ascii?Q?98+hr6ZX9iUUStngR1acuTVff21L2hBUr90TnUZE8MkdnDlmn46PbB1LcQkN?= =?us-ascii?Q?5EbasZGQAa9jmpO0I/mEWeGvhioa/tmg4FBlyWzAaaFXXTPwI2O7CEvuQLGk?= =?us-ascii?Q?sEpfhqScFIsx9EwQlM0Fi94No4jD6eOa97Uca+57TLhg7aH43+m6nZOkXY2l?= =?us-ascii?Q?7FL0rq0NUACWnQKbLW9fDY5sZ7zLtgm1p5w23T5DkgXQOqevLrPiDiewebA0?= =?us-ascii?Q?v0PBbmJ53Ck42C6YR1Co4VU9YR5h/uy5cjU8u46vZMoJynWfjTHj0UDLTNiA?= =?us-ascii?Q?1vwHMOs6lkGrFj3INAAA3270HRaidOlh3+UjxkcTWeeLwXV4oJ5p80EbdqsX?= =?us-ascii?Q?0pB1XrF3fHxrln5x4TNH6C12RiKS5HgpX4DP4BEX8GUoUzmh4LXn9tTLRUgP?= =?us-ascii?Q?NUNTOzXQMS4zmnMmtpX72XLRYWYExT4ZB7WjWrjP6FEAzqRwi+nQEaUbaO6k?= =?us-ascii?Q?Hjm+sH7tl8PRN6crN5r7GOybL0ryvzHe4QYLMiakFeYIVeVW42z0BjDlVmIC?= =?us-ascii?Q?qMS83WfBn0jyoMd+0IXTHqMuXLdr/qi/vRIjOtfIJeuGtQgVjC2EGYbxNpqk?= =?us-ascii?Q?lKxAMziafJ59CN3PyMpkmyIlAGCUSpYG1QAHy0kCpykBgqPt+0i7ten9fEQu?= =?us-ascii?Q?cs4OEF+LRSEB+IpSi4ouGMbhoxSCP2nHIAF5/lKO40pIIB0cLOZr+tScUPtC?= =?us-ascii?Q?2js/x8gGxhTkcwPnGep2hJLv98OBZbNAU36SKr4Wvas2O2GOadwTwadf+UTn?= =?us-ascii?Q?ZE19d8cY6HEeZhgcJHtkYJilrLxXArbfAE+PwYvXFueYYm8pnonskHLoNjs8?= =?us-ascii?Q?UfCQFEaDd0f1TAFJaILPNMLTDHNmy3H7xQy6UZllxf9F+6zpdh7r2COi5jOT?= =?us-ascii?Q?TmktIF1EW8NOLYRvEv9WCcmNGruqcVMJF1EVP0kemAkrqdBjr6kc9wcB9sWM?= =?us-ascii?Q?4uVYrLCieJXHk1tHT8cM12wqYztH7w816hejQFE6GvAO0dSze0SXjSw1CUgu?= =?us-ascii?Q?F1ZK9IolWA=3D?= X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 6:euF4GJzVQxKIpKcUmZ2ZdfE2fu7+rmvqgFrBrKtPBmUnjxYIuJ5mg0fM3ftmqebqIl8Hnhq9EwQ+f9/F0Qq2g8R/xcPjpHuF6zA72GIH/uDyT1Tt5NOlCT0U/+Rc9CbFRGXQbogNQgiNKEgmERacRPgzJC+l2OBf+UeN2mIEARLq7IHPfJCrBxoNtiehYUA7g6F0i28JJmXn2g0Mu9B1H1IueWAb3JcerP8S6PnQOOBQARSWkO5h6JKec34u2vZEMwjmL0dSPW2F7Nssn0gP+3qCM+I1nSHeBAZXL3icBm+tFNG1E8nrQ3W1igaPEvnDFXrBXl7kA98GOXimA+l/bVxg0cB2Ksx+Djhoy7ky2VqEzvwhvxd2yZDsQb8xrwLXmC5VNfN383EoUdN0YmvIPWiXf6IsUtiDR7vH9AXof4c=; 5:2dUtSkpuvL9gEsQMMBwqgYiHuz+ELl2qd6y+QUAWX7qupeuwyORsTlPh4eglAEfo0JvrG1o9NmtdPMgC91NCez94Ay//ohWiIqxejZvIkW4OwLahStvFK8rikixKfcWo+LNVPnd+vRSLjEeM6wJ++XAKkLUmA2vdLMGKFW+YYyQ=; 24:j8rf953NjK022moQBGan1On8uraZlr7Cd3d/NMrW+ptUKstqMA5fu/rnzik+DHEpZB6zqfbV6qsOhlAA/4WGnDX8uUArIAnaX7kJkRHSuMk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; CY4PR15MB1320; 7:eMONrPcZXYuv+ZlKSFRxEFFw8DWvYbTne9E0FwXGPDuY5Y9P7BVhfQqXGGQ3MlD3rwsmjmYwI9XbQp5H7yVZ3gmCYLRoOrKXtc3XFVTjbZiBBvkCMz9PmVAZ3JGd24tsWcEnVdOTk9g8U5b4qw97ZfGbMiOcW1YNoVP3fxR0aOGVyvDuc8z4FetH7qRnfxuUoTBRXmVYnbx0D/sIMD6dbHcqpzkMecxjJ4BysUxwCTfQ8vgDVgF8G7afYIrKXiCk/bESuGo7/BqzmDtXrDBWeklB/fLkixzGFKkuCqQX/uNiDzNx3+rkPaHIigatZfZ/6aYwwJAb18t2W8heo0gkrK6eZflwILwjHVkh8G4MRTdop4I81Jl4kW8bHI+a/zQ2B3gYft4LglvopKLuoHP8kOi4WoihmpNqhO7kPKnNJEC2j3l9zU6uH/tYUO8IOpLS4VnG0nVYxaZa5nHjpPxXRQ==; 20:ABwfwFENPxqG0Wv43wEpfsWxrcbhqKLhyZpoiDGaPmTeYKHQis1eWf+xcFyvC1vXfNrXo+l/f9YKn+Z+CRmkkUt9ysRM30WDgxZaZCzBl2PtF1B6GS5xgakD4gob/gUUae1N9klG8JJCLWRGvMvwhXTyHohejHpAaxUUV0cQPN8= X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2017 21:08:53.2910 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR15MB1320 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-01-19_08:, , signatures=0 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: <linux-block.vger.kernel.org> X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP |
On 01/19/2017 01:08 PM, Josef Bacik wrote: > A user noticed that write performance was horrible over loopback and we > traced it to an inversion of when we need to set MSG_MORE. It should be > set when we have more bvec's to send, not when we are on the last bvec. > This patch made the test go from 20 iops to 78k iops. Added for 4.10, thanks.
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 454f770..510ae02 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -275,7 +275,7 @@ static inline int sock_send_bvec(struct nbd_device *nbd, int index, static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index) { struct request *req = blk_mq_rq_from_pdu(cmd); - int result, flags; + int result; struct nbd_request request; unsigned long size = blk_rq_bytes(req); struct bio *bio; @@ -314,7 +314,6 @@ static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index) if (type != NBD_CMD_WRITE) return 0; - flags = 0; bio = req->bio; while (bio) { struct bio *next = bio->bi_next; @@ -323,9 +322,8 @@ static int nbd_send_cmd(struct nbd_device *nbd, struct nbd_cmd *cmd, int index) bio_for_each_segment(bvec, bio, iter) { bool is_last = !next && bio_iter_last(bvec, iter); + int flags = is_last ? 0 : MSG_MORE; - if (is_last) - flags = MSG_MORE; dev_dbg(nbd_to_dev(nbd), "request %p: sending %d bytes data\n", cmd, bvec.bv_len); result = sock_send_bvec(nbd, index, &bvec, flags);
A user noticed that write performance was horrible over loopback and we traced it to an inversion of when we need to set MSG_MORE. It should be set when we have more bvec's to send, not when we are on the last bvec. This patch made the test go from 20 iops to 78k iops. Signed-off-by: Josef Bacik <jbacik@fb.com> --- drivers/block/nbd.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-)