From patchwork Wed Jun 7 18:19:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271087 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0D63C7EE25 for ; Wed, 7 Jun 2023 18:19:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01ABF900002; Wed, 7 Jun 2023 14:19:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0CD66B0074; Wed, 7 Jun 2023 14:19:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6075900002; Wed, 7 Jun 2023 14:19:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id C7A426B0072 for ; Wed, 7 Jun 2023 14:19:37 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 6DE321C79A4 for ; Wed, 7 Jun 2023 18:19:37 +0000 (UTC) X-FDA: 80876764794.25.361CBDE Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 9FC8120029 for ; Wed, 7 Jun 2023 18:19:34 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X+75NLCH; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686161974; a=rsa-sha256; cv=none; b=U1gN3xnR6pkcargki5+TnDV1OMU3EA8sRQbBWwRH/8d5I3H/o5jx/ZpxTbUNPCK2wBfNB3 ZPrdS76oy/iXX/G8xQqkiwt+3mrC9YMruYVfCWs/XSqYjfJiLPjMYAHkE36Yg+YFUoxiuF yToTXkOjAxZy0imdVl5PbA2YxfnohTA= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=X+75NLCH; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686161974; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=XYVIUsFQJ9Y7j3fAxVQKQu7hRxA9LzjD7JNvAH18L7gQRt0+FrWkpf2qp+58i6UfxX+Auf l981PRYFRU97oTTz/CNqQZA/brTkSUbMJq1XTJfwpDVo2kxFl3W8aAdV5tEjFTyClZtf4B qdEl4AMEeOFvxj4p9FxPewFVRvJMDz4= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686161973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=704Gul1es5vKB8f3fNlqLP0xfed1dEhsuWb+SLm3klg=; b=X+75NLCHdNw9IUdjV0vY4keSL11jwbhdJAxDrikGBHUkfYJGVjJRj8ZaUW3Uskf5RMZg9E 2IOkwpXxe6PsJdgmdyvDAw7fCJ/JZ2sMrkCmVdLqwq0AypDbRBZJ2ozmCsvKLL8jFxS0/z 0ABj1JALZcyy4sshI3xFqbR5r0xRwHA= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-mJrSACM1MtWkA-dGsaZ40A-1; Wed, 07 Jun 2023 14:19:30 -0400 X-MC-Unique: mJrSACM1MtWkA-dGsaZ40A-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B083C3C0F220; Wed, 7 Jun 2023 18:19:28 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4158140D1B66; Wed, 7 Jun 2023 18:19:26 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 01/14] net: Block MSG_SENDPAGE_* from being passed to sendmsg() by userspace Date: Wed, 7 Jun 2023 19:19:07 +0100 Message-ID: <20230607181920.2294972-2-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 9FC8120029 X-Stat-Signature: y1hsxuhwbzh13iqyondc5m8zfxdjf7rg X-Rspam-User: X-HE-Tag: 1686161974-955396 X-HE-Meta: U2FsdGVkX1+HQK7BoCOL5JXJDZvl4R9rwdZuszGmuqkXHtDterJ2P3owrgXjgfBWvVtMk45PEfkUKTPmy26NgsUE/kzzAb9q4muoSk3p09tvU9481M2uX9lqroxH3t/mldyD2eEnO/7DhomaPKDZ3w6ntcF6lGriFRkDOd0hUWAB/gdYQLOf0DzhkrVAd2Q4cyk0y8yrEzmaYxslGZh6Aony4o3Vv3BwGcS+QBYn1QzTjKaunL/oBUVhlbrhy+8Z7OFv8otE9KkEvTem7YuSftI6LdF5xnFm0EShtPbj53q9JRj+GsXqwOctjj4jKn7X20PU/ATp6KKM3fgxzALflaNpo2rxut6FpgNIQw5fNQSeWG4EjrTaAYj13Fp8tGO82D+6ZrTZTKxQoruXqRBTwfpTXIsMWvwrqqg1qTkg9QSeVR2J8+TkUW2l5d6OwFMg/4lSdLRRR2+BSx0Ri9IcFMNFQTDnYTdZZKMKWlqPwqy/+3hFyXcFv3hQf125q8fwPQEBZRE+3/uK1Vr/GE8pREzHM3gH/y1dbdqWIvhu4D0HxieQ+Dx3lud7/1tnnMro1RVBlp46yKWFbU/IoKTWfVttHtVwtyfLotWDeX6F4k/p2n0XQ/cmY/IeMqygB6jajphw2/SVCmSnZz8rq+qeCBE2IBI3L55Fh9jcn6Y9r5CPdRiMicX+3K07H3Wk3XtXdlCB8UZFiQhwcXFVJj9iM4rwGUDO2/tOzXlVXUHSkQ950ZMs2qR8SyRo4bI+6+BbgK+nQkEwp1C+d6rKUnoZ7bvozB9WFwT79JrbvyVHXuBFltj5pBruggSdcMI62+Y4iEV87RtAbBqOAUvaHBasoZcf7lAnTkCO+eks3/ZNyrXNB05bu9w0lGZ336tBxZgwsO52CSfolzlqqsznlRYNA0M/t1f+0cWUTNhSVS3iLqv61kGBg8dU+GeMDQCDQcAiazfKHODyn0nZ/3f/Uyy 1U3KzTCH nFs4AsCJWfl0qroBtoD+l/rbTVSmPVsIw8QcSirreuVmGiCguPYlEwRcjqRH959Wkj1h8tFHV1v95p8TU/jTHRSxj6nlqn2aijHzotrWscpHPg3nyiJ8ohfxuwEdrXoJ845o/v+C3XsLe82N8UQNtsGwrvE81z0ye18pt16Bt5VtFole+RWD8LePKNdyW6Rg7ZGBAaqtlgDtniaiIygGhUVsoZpTYsgsPKh+wlK2uUJ8XmL6ZA96v0y9Zhs2J554/zkF18Ol5QjKcbu8VFD6cwdLNg5kTM+HppjDjqvf49XVDMkVavDxSspKjzvyAENYJ2GDEAkuHUWcXlMixwh7Jv0HPjHdZ2qGr4wvzd4oG2W4mDKs/ol5IGMuT39+h9Bvk5cZPGVCXov+5bDvE/sf/mo+VJY6DnsLg4oshrZEsd6SYgOrCn4aIoXRTGw9zc4DwX34nJQt0b6NxSrOL89rTKtpNho07xFgU5SA7/JU0eK3o0Mp+IyYiqcoQfHrfmUn1IFetAcZdUzRh1Mj+uSiGcW/uAp+UXkqCV72NuevmcVJORGHLeBVuOJ/gkA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: It is necessary to allow MSG_SENDPAGE_* to be passed into ->sendmsg() to allow sendmsg(MSG_SPLICE_PAGES) to replace ->sendpage(). Unblocking them in the network protocol, however, allows these flags to be passed in by userspace too[1]. Fix this by marking MSG_SENDPAGE_NOPOLICY, MSG_SENDPAGE_NOTLAST and MSG_SENDPAGE_DECRYPTED as internal flags, which causes sendmsg() to object if they are passed to sendmsg() by userspace. Network protocol ->sendmsg() implementations can then allow them through. Note that it should be possible to remove MSG_SENDPAGE_NOTLAST once sendpage is removed as a whole slew of pages will be passed in in one go by splice through sendmsg, with MSG_MORE being set if it has more data waiting in the pipe. Signed-off-by: David Howells cc: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org Link: https://lore.kernel.org/r/20230526181338.03a99016@kernel.org/ [1] --- include/linux/socket.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/include/linux/socket.h b/include/linux/socket.h index bd1cc3238851..3fd3436bc09f 100644 --- a/include/linux/socket.h +++ b/include/linux/socket.h @@ -339,7 +339,9 @@ struct ucred { #endif /* Flags to be cleared on entry by sendmsg and sendmmsg syscalls */ -#define MSG_INTERNAL_SENDMSG_FLAGS (MSG_SPLICE_PAGES) +#define MSG_INTERNAL_SENDMSG_FLAGS \ + (MSG_SPLICE_PAGES | MSG_SENDPAGE_NOPOLICY | MSG_SENDPAGE_NOTLAST | \ + MSG_SENDPAGE_DECRYPTED) /* Setsockoptions(2) level. Thanks to BSD these must match IPPROTO_xxx */ #define SOL_IP 0 From patchwork Wed Jun 7 18:19:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271088 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 249F9C7EE23 for ; Wed, 7 Jun 2023 18:19:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B3E5F900003; Wed, 7 Jun 2023 14:19:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AEE546B0075; Wed, 7 Jun 2023 14:19:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9DD87900003; Wed, 7 Jun 2023 14:19:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8DAB56B0074 for ; Wed, 7 Jun 2023 14:19:40 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5655D1604E3 for ; Wed, 7 Jun 2023 18:19:40 +0000 (UTC) X-FDA: 80876764920.28.7D975AF Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf24.hostedemail.com (Postfix) with ESMTP id 80EB8180021 for ; Wed, 7 Jun 2023 18:19:38 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IIQSJ74c; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686161978; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oJUkpOQ7e9hBKCJRUR3yZoWIoUWj9MtOId6LGPzA6h0=; b=j4nCFaB26+d3BAqYMGCEUtE0haNDWw/v6nG/gtBUih4Dg3BpCyZUbs2BOITN6bFqu+C4co oHrxAU4uqcK7L1dtYwnqqx7faU6j30kqXP4p6TEWBU1RUUwyowtfmSAodtrwHHoqXNQxN/ V/6uKB/Ao1Pe68l6ZVo646k7oOhTMbU= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686161978; a=rsa-sha256; cv=none; b=PWREkA5AIU6rYl07+K21yGKyzwcxIMlXSY2I3YP6X0/LlPh2ME22xAJOAXMmTXG+52NuKV 4svdlVzgyJ9ZlyQW7uvxsBDd+7bI9H6zzDcZYQ7G2Wki07WoPGTcA1Ce7UddN6q/FvSOL6 ssbAJOR2hHKM34IEwYxb8V0brb3qu4U= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IIQSJ74c; spf=pass (imf24.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686161977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oJUkpOQ7e9hBKCJRUR3yZoWIoUWj9MtOId6LGPzA6h0=; b=IIQSJ74cThUV6q8f4+cjn1KMy3qlaDW8OtssUV7XbarLHW9PsdyP1r2F6xY6pUkN1uecDy IM+etLPidkb42derRtlGR3C5KarkOU6uT7oCKa+BPqJ735dUEXL+wDsGaR7lHCkbP8hrHd jW0VwxmnhuymIfXxDtWYh0rFQaq2YAw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-412-TTybqZy5N2KGRf1jLAwqPg-1; Wed, 07 Jun 2023 14:19:35 -0400 X-MC-Unique: TTybqZy5N2KGRf1jLAwqPg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0B408101A58B; Wed, 7 Jun 2023 18:19:33 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 94E792026D49; Wed, 7 Jun 2023 18:19:30 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 02/14] tls: Allow MSG_SPLICE_PAGES but treat it as normal sendmsg Date: Wed, 7 Jun 2023 19:19:08 +0100 Message-ID: <20230607181920.2294972-3-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Stat-Signature: dupntx5jaxpza5d4spsfyn1kxb9yt3hf X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 80EB8180021 X-Rspam-User: X-HE-Tag: 1686161978-272714 X-HE-Meta: U2FsdGVkX1/R/TEWcVSYC7dQjzTLYTmK42t7hIm2UIax2Dr07suJOHoH5WUX62YNU9SvTjXNPq9avn1cptXKuuF5iDtECoM4C4xYbMt2aNOMdYetFDMwC6wi27VXDVy7Y9QMs4Fz7NiofHlvJ2Brp+PUyVLR2JuOh+sYinjk8uu/4cJfUudMpWFLxLEADunLsAu+WeiKlOYgndgz8hBViWrFLyMfdS7q/zhPeMHXmDcJbUbx565VBvCaaofs92LBa4BrZWef6bJQ0TcYe1d/IheLRMzEdvbexyIa3pl9SYZUNMKqORiCKrAbSVpDraiJL0oHxx87i4tjd+C+JE7/el5OBxbEuJPd+cqgjbo21FUNkIIbk5B+KibUClDWuZbqjDVqfKm7g1J/TPC5CHFLqdmJWpsupxNLtSnvtMdBHiKE+bhOc6kNR7Q9ntD5HbkTOhGnblHd0fHUHCVh7MQUJtmARcR+ofOll2ZrWwJuLWOydaog9/d+0HcRVejKxPjoivBiqt7dg7xLVoy9h24MAkYI9gnid5Qivx6+aHjGwazKUchyBoB6SG3DL6dLsCb2XU385TN+4415/yZJ9jsomOll8SvAEsiQzR3650AcTYkY6dBXDKmz7UjnhmSDZIlROhI/cEXEmcp1SCkk/gK1OCYTXHn87lTyTncohe8y265IIyRHyOqo+0XFBey80clfAQd3YXKtv0JnM1lIhkKQzktzDKBdDG8DB1n2WPhdCnHtryDSRTL+NCchcajof2rCnc5Skho8gsXTGsIzu4Pe9IDTqFkE6ijl4voZdoKYgdLztDWJRDwe3ipcrj/3oO60/GskZZydHV4fvbmbjYCGssqKp0Vk2ppKrHZbvCfRFmmOvCzayum6kBcC4omwls/t+AwkhopB5lDdrEdJb7cCzPsEtfpBa/OlRxffv68e6Rbxe+HzBE49FBN1/I7GfWAqWVcU+ssmmX8OUtLoXa2 WIjzjqBM i48vnE0SPT01OXWsrUKXO2IPj7tvgT+FZUZ4w9K3KyB7L9Obhj8GbvXlUNp34xa4+qvB2u6FKfrdQGn4fVAEmOXT0mUX62vxvDPfzhNrgHmhOq37OR4dRbmmhD5wVJSajK05gVeO8m8hz+NRTjiiJ+PQVA/su/9fldtVLdPO4gWJZKO9YYVmx8PLbhBUqHCMGBZjzL5uPtArhbj9BjdzB/jMTcaCGgbDsK8Utug9RT97xQCpRT1BfJkVJ2KQRFsqdTFVCgz+/O1lwYGxjAUsO8Y6TUHxsT5qW/bkhZ538YlOSD6E80kTgocmucLUcCk3AbUhhpc+5HlCBfKmQrk5EeeTeuErMgihfvJIecNHkTI8dtZcu5A3gwLSxLyVzJAWftBLpzL5y+AhHv76NQrZ1aJmec43JFWsCVkOt3mshTvDVt4CvhiDW1GGi1GI6KVL5mLzwC1zJH5VJGA1hC0j3zeucycnoEvgprwJg7EJZ4EPXpORx5L0V0JX3pe9QxhCCRYd0GzP9tCssPZOyy9QU7IMFFP8S2JvCkmHhKH/WyUA8FAKayDzeahtFHA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow MSG_SPLICE_PAGES to be specified to sendmsg() but treat it as normal sendmsg for now. This means the data will just be copied until MSG_SPLICE_PAGES is handled. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 3 ++- net/tls/tls_sw.c | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index a959572a816f..9ef766e41c7a 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -447,7 +447,8 @@ static int tls_push_data(struct sock *sk, long timeo; if (flags & - ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST)) + ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | + MSG_SPLICE_PAGES)) return -EOPNOTSUPP; if (unlikely(sk->sk_err)) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 1a53c8f481e9..38acc27a0dd0 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -955,7 +955,7 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int pending; if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | - MSG_CMSG_COMPAT)) + MSG_CMSG_COMPAT | MSG_SPLICE_PAGES)) return -EOPNOTSUPP; ret = mutex_lock_interruptible(&tls_ctx->tx_lock); From patchwork Wed Jun 7 18:19:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271089 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC234C7EE23 for ; Wed, 7 Jun 2023 18:19:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4AE096B0074; Wed, 7 Jun 2023 14:19:46 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 46164900005; Wed, 7 Jun 2023 14:19:46 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32664900004; Wed, 7 Jun 2023 14:19:46 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 25AE96B0074 for ; Wed, 7 Jun 2023 14:19:46 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id D1A25404C1 for ; Wed, 7 Jun 2023 18:19:45 +0000 (UTC) X-FDA: 80876765130.05.5EAC40B Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 0558F80003 for ; Wed, 7 Jun 2023 18:19:43 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JXNucl9Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686161984; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JYxfz2KU2rDSnmTtDbyA2cmeEZ/0Wy2M8yN6PdMuUwk=; b=VjN1i6izDC1MONQD+G5FOKmdlu+ZMQzVgvhZEB9/lRsa3HQ3IBF+NkBir4aJbUFXLMicSv 6C7EBV+ipH06Shsgu29bs+R+7J8dtZDy5DMh+EDwNhohT8MGhxrIDmd1rD7O1kJHiAjn/U 7qpSw/0FEefT3vHcbLa2lVZkV0gk9P0= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=JXNucl9Z; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686161984; a=rsa-sha256; cv=none; b=Kj6lZMANrBdMxEXmV2KtK7oxrqwCBOL+7sYu69F1B/MlNj/7mlbyEWTCJasRFvngh42Bfj LQ7I/i617NMVoXzdGS2lwaJ5P42OI5FozKWQUDst27epo/c26ktCkuOZrz3rg0KiMc1oOW odQZeFbSKvyo7bjuLI+GYKHXjns6Esc= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686161983; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JYxfz2KU2rDSnmTtDbyA2cmeEZ/0Wy2M8yN6PdMuUwk=; b=JXNucl9ZD8u+kZEJojjgbqSmmzCFpd1dZgw0wbpuse+3uEWzHzbeaFAN0SzdrZkDN1NKKP A0naBZ6QBo1HONRkQ2bxyzCaCj0qBX4h+ZsXirVSY2DG7AW9M46MrrC5mtVjsZdByCAsOx a+3XTDXjhIdbUoBbIiXLKd1NVsQ9ltk= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-507-hsnqYwbtNHan5Bnn4xTm2w-1; Wed, 07 Jun 2023 14:19:37 -0400 X-MC-Unique: hsnqYwbtNHan5Bnn4xTm2w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.rdu2.redhat.com [10.11.54.5]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 104C13802139; Wed, 7 Jun 2023 18:19:37 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 15D3C9E8D; Wed, 7 Jun 2023 18:19:33 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 03/14] splice, net: Use sendmsg(MSG_SPLICE_PAGES) rather than ->sendpage() Date: Wed, 7 Jun 2023 19:19:09 +0100 Message-ID: <20230607181920.2294972-4-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.5 X-Rspamd-Queue-Id: 0558F80003 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: cx8c7xmtsawi9cwf66jqgdfmkhpmmm3n X-HE-Tag: 1686161983-687786 X-HE-Meta: U2FsdGVkX1/kKGQ8ZRN8pb4xyh022zKAO9VHFj6IoHsJzDyU19aw4knNTEG2C6z079/E9RSOfHU+l1JuIdSg3nVYvPceMO9Iz2d+4saLtv49CRsGIaKoHlIJWSevWlgMX/HY9dPHYGagt1HkSnvhL/FMOeMZGgaRCluZwxyTy/Nvj4RmJMxnJUU5sQORFBDO0hgyyaaVzHjBXElVtnsZaFgeWAvERmVWqDXcWIIE0NN9tuf36Q1rnl05N8Zl2OTXdsWb35SLi8Y+KEmE9do3/5VitSIzIyG1tlQCvpreumfQfzOROeBHuiHa6VXcoE0yXEmMyRBy45Y8j0u3EAzVJYOyzVZ4/Tc5kz4vgS9ck9MpUrdzXCiQoA89Ap7Tx6L0nexfBxPeWPfyU/QbkQuoTWV+EUuWDIYoPDqwS2m8cPjWX7rRhXh1UxAQj21M2XM/ysVxiYfjnZIroIYNAi2VG6n0o5dcTAZ/pBF2j0e80M+ust/LYMlcsc+1i5GtxZ/pRJPc+W+WiNjsVPh7+WeWXimm530+TQruzjEMtKKNtfFrGm8CGVXHmaWDWmtNidgpHtczYpEMZCJR3yciyd6EqcJ5zJIjfdu2dMV0L/iMN14ni57xR89y77JJEdrs+5M6rUuw6zhnkQQmOQXEfKQdrfdlakNpIOm6vsdht0IV5MYtJfzcWST/Yd3V6XPAWs4eK5+pyYrK4/GwZIJFNqA9WroA/CvhlCnux+kSKuPFVpNyDrKDIUNiySs3vUTX+hbTNkxolbI1FQ3sPSMBYb12G17P2W1YEDwVD6UHa5V6dbA1xqMeZx1cljja3Epq3UCswPG5PUgYjgMXhYjJs4SYgj6PTqdkOuqr8C0eBYPhJill3IvI1l/9C6vfXNlagGHju73XuCGOaS/rOSjnvJB0ejpsEA8hzuafaJK/+FPMm48WQYWC/BkNKkc6EkmJ1CrUWozmBnyTZVRo0wfJ9Do GC5ID2fx RM6yNMa1IDOSa8jYTnFX48mFjZN4/b+6nierYjoTeMp/MHCOee9sIxdbNCak/h8oLqI3YsF9dsn0gx4wK7jmI1khlKkO/I3mZQHHmHPyTh9vD7jrKW9sg7FktImro2ljOmzzsDV1IfNSw0doqnXpW+l1INE5TvBoA20tzBlBM2qtXrNKjvNKp+SOjVsUhbkiprvZd7xUdjKWL2b454hzVz4LmK/NbpgJff+luu16KPJCqGBN+WUP8kHqr63oaYNFfJqieuooDq5vd25QMdry0fT7OwvBNAn19UorPpKlgEXATYBgdMJ+Ngq6J90URgl3jX44vPbNbKOk08knqN0pw3VwdFVa2OCoo79Xdah8/SwcEjMZTDGYviD2uts6JbSjuXNe9cy7LrsG57BnWTBWaB5tn0K/IBhGeLnGE3GxoXUDGnRGAoCkMvYQP7zD5H1JZxtHTPpi5AKEXUh6hIa+AbDZczupggONKhyLqw/cQT6KBr9+dNH8/xCe2wA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Replace generic_splice_sendpage() + splice_from_pipe + pipe_to_sendpage() with a net-specific handler, splice_to_socket(), that calls sendmsg() with MSG_SPLICE_PAGES set instead of calling ->sendpage(). MSG_MORE is used to indicate if the sendmsg() is expected to be followed with more data. This allows multiple pipe-buffer pages to be passed in a single call in a BVEC iterator, allowing the processing to be pushed down to a loop in the protocol driver. This helps pave the way for passing multipage folios down too. Protocols that haven't been converted to handle MSG_SPLICE_PAGES yet should just ignore it and do a normal sendmsg() for now - although that may be a bit slower as it may copy everything. Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: "David S. Miller" cc: Eric Dumazet cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #5) - Preclear ret just in case len is 0. fs/splice.c | 158 +++++++++++++++++++++++++++++++++-------- include/linux/fs.h | 2 - include/linux/splice.h | 2 + net/socket.c | 26 +------ 4 files changed, 131 insertions(+), 57 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 3e06611d19ae..e337630aed64 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -33,6 +33,7 @@ #include #include #include +#include #include #include @@ -448,30 +449,6 @@ const struct pipe_buf_operations nosteal_pipe_buf_ops = { }; EXPORT_SYMBOL(nosteal_pipe_buf_ops); -/* - * Send 'sd->len' bytes to socket from 'sd->file' at position 'sd->pos' - * using sendpage(). Return the number of bytes sent. - */ -static int pipe_to_sendpage(struct pipe_inode_info *pipe, - struct pipe_buffer *buf, struct splice_desc *sd) -{ - struct file *file = sd->u.file; - loff_t pos = sd->pos; - int more; - - if (!likely(file->f_op->sendpage)) - return -EINVAL; - - more = (sd->flags & SPLICE_F_MORE) ? MSG_MORE : 0; - - if (sd->len < sd->total_len && - pipe_occupancy(pipe->head, pipe->tail) > 1) - more |= MSG_SENDPAGE_NOTLAST; - - return file->f_op->sendpage(file, buf->page, buf->offset, - sd->len, &pos, more); -} - static void wakeup_pipe_writers(struct pipe_inode_info *pipe) { smp_mb(); @@ -652,7 +629,7 @@ static void splice_from_pipe_end(struct pipe_inode_info *pipe, struct splice_des * Description: * This function does little more than loop over the pipe and call * @actor to do the actual moving of a single struct pipe_buffer to - * the desired destination. See pipe_to_file, pipe_to_sendpage, or + * the desired destination. See pipe_to_file, pipe_to_sendmsg, or * pipe_to_user. * */ @@ -833,8 +810,9 @@ iter_file_splice_write(struct pipe_inode_info *pipe, struct file *out, EXPORT_SYMBOL(iter_file_splice_write); +#ifdef CONFIG_NET /** - * generic_splice_sendpage - splice data from a pipe to a socket + * splice_to_socket - splice data from a pipe to a socket * @pipe: pipe to splice from * @out: socket to write to * @ppos: position in @out @@ -846,13 +824,131 @@ EXPORT_SYMBOL(iter_file_splice_write); * is involved. * */ -ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe, struct file *out, - loff_t *ppos, size_t len, unsigned int flags) +ssize_t splice_to_socket(struct pipe_inode_info *pipe, struct file *out, + loff_t *ppos, size_t len, unsigned int flags) { - return splice_from_pipe(pipe, out, ppos, len, flags, pipe_to_sendpage); -} + struct socket *sock = sock_from_file(out); + struct bio_vec bvec[16]; + struct msghdr msg = {}; + ssize_t ret = 0; + size_t spliced = 0; + bool need_wakeup = false; + + pipe_lock(pipe); + + while (len > 0) { + unsigned int head, tail, mask, bc = 0; + size_t remain = len; + + /* + * Check for signal early to make process killable when there + * are always buffers available + */ + ret = -ERESTARTSYS; + if (signal_pending(current)) + break; -EXPORT_SYMBOL(generic_splice_sendpage); + while (pipe_empty(pipe->head, pipe->tail)) { + ret = 0; + if (!pipe->writers) + goto out; + + if (spliced) + goto out; + + ret = -EAGAIN; + if (flags & SPLICE_F_NONBLOCK) + goto out; + + ret = -ERESTARTSYS; + if (signal_pending(current)) + goto out; + + if (need_wakeup) { + wakeup_pipe_writers(pipe); + need_wakeup = false; + } + + pipe_wait_readable(pipe); + } + + head = pipe->head; + tail = pipe->tail; + mask = pipe->ring_size - 1; + + while (!pipe_empty(head, tail)) { + struct pipe_buffer *buf = &pipe->bufs[tail & mask]; + size_t seg; + + if (!buf->len) { + tail++; + continue; + } + + seg = min_t(size_t, remain, buf->len); + seg = min_t(size_t, seg, PAGE_SIZE); + + ret = pipe_buf_confirm(pipe, buf); + if (unlikely(ret)) { + if (ret == -ENODATA) + ret = 0; + break; + } + + bvec_set_page(&bvec[bc++], buf->page, seg, buf->offset); + remain -= seg; + if (seg >= buf->len) + tail++; + if (bc >= ARRAY_SIZE(bvec)) + break; + } + + if (!bc) + break; + + msg.msg_flags = MSG_SPLICE_PAGES; + if (flags & SPLICE_F_MORE) + msg.msg_flags |= MSG_MORE; + if (remain && pipe_occupancy(pipe->head, tail) > 0) + msg.msg_flags |= MSG_MORE; + + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, bvec, bc, + len - remain); + ret = sock_sendmsg(sock, &msg); + if (ret <= 0) + break; + + spliced += ret; + len -= ret; + tail = pipe->tail; + while (ret > 0) { + struct pipe_buffer *buf = &pipe->bufs[tail & mask]; + size_t seg = min_t(size_t, ret, buf->len); + + buf->offset += seg; + buf->len -= seg; + ret -= seg; + + if (!buf->len) { + pipe_buf_release(pipe, buf); + tail++; + } + } + + if (tail != pipe->tail) { + pipe->tail = tail; + if (pipe->files) + need_wakeup = true; + } + } + +out: + pipe_unlock(pipe); + if (need_wakeup) + wakeup_pipe_writers(pipe); + return spliced ?: ret; +} +#endif static int warn_unsupported(struct file *file, const char *op) { diff --git a/include/linux/fs.h b/include/linux/fs.h index 133f0640fb24..df92f4b3d122 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -2759,8 +2759,6 @@ extern ssize_t generic_file_splice_read(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int); extern ssize_t iter_file_splice_write(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int); -extern ssize_t generic_splice_sendpage(struct pipe_inode_info *pipe, - struct file *out, loff_t *, size_t len, unsigned int flags); extern long do_splice_direct(struct file *in, loff_t *ppos, struct file *out, loff_t *opos, size_t len, unsigned int flags); diff --git a/include/linux/splice.h b/include/linux/splice.h index a55179fd60fc..991ae318b6eb 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -84,6 +84,8 @@ extern long do_splice(struct file *in, loff_t *off_in, extern long do_tee(struct file *in, struct file *out, size_t len, unsigned int flags); +extern ssize_t splice_to_socket(struct pipe_inode_info *pipe, struct file *out, + loff_t *ppos, size_t len, unsigned int flags); /* * for dynamic pipe sizing diff --git a/net/socket.c b/net/socket.c index 3df96e9ba4e2..c4d9104418c8 100644 --- a/net/socket.c +++ b/net/socket.c @@ -57,6 +57,7 @@ #include #include #include +#include #include #include #include @@ -126,8 +127,6 @@ static long compat_sock_ioctl(struct file *file, unsigned int cmd, unsigned long arg); #endif static int sock_fasync(int fd, struct file *filp, int on); -static ssize_t sock_sendpage(struct file *file, struct page *page, - int offset, size_t size, loff_t *ppos, int more); static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); @@ -162,8 +161,7 @@ static const struct file_operations socket_file_ops = { .mmap = sock_mmap, .release = sock_close, .fasync = sock_fasync, - .sendpage = sock_sendpage, - .splice_write = generic_splice_sendpage, + .splice_write = splice_to_socket, .splice_read = sock_splice_read, .show_fdinfo = sock_show_fdinfo, }; @@ -1066,26 +1064,6 @@ int kernel_recvmsg(struct socket *sock, struct msghdr *msg, } EXPORT_SYMBOL(kernel_recvmsg); -static ssize_t sock_sendpage(struct file *file, struct page *page, - int offset, size_t size, loff_t *ppos, int more) -{ - struct socket *sock; - int flags; - int ret; - - sock = file->private_data; - - flags = (file->f_flags & O_NONBLOCK) ? MSG_DONTWAIT : 0; - /* more is a combination of MSG_MORE and MSG_SENDPAGE_NOTLAST */ - flags |= more; - - ret = kernel_sendpage(sock, page, offset, size, flags); - - if (trace_sock_send_length_enabled()) - call_trace_sock_send_length(sock->sk, ret, 0); - return ret; -} - static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags) From patchwork Wed Jun 7 18:19:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271090 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E055C7EE2E for ; Wed, 7 Jun 2023 18:19:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BB3B36B0075; Wed, 7 Jun 2023 14:19:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B63A0900005; Wed, 7 Jun 2023 14:19:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A2C32900004; Wed, 7 Jun 2023 14:19:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 91AE36B0075 for ; Wed, 7 Jun 2023 14:19:47 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 60B92AEFCD for ; Wed, 7 Jun 2023 18:19:47 +0000 (UTC) X-FDA: 80876765214.23.09DE128 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf28.hostedemail.com (Postfix) with ESMTP id 9053CC0011 for ; Wed, 7 Jun 2023 18:19:45 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=izEH8nlj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686161985; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=OL+TnlQNDjtbdgR92AyZlN3k/q2TbFv+1S0kh3gXGzs=; b=yS1fpqGhVhcs2PMPBhSynkONpZ0XXBzQV96VimwthrFVDO+a2ucbYxeh/y2gQPQQgJXjJR U+z2H+ffcYeCG8Ul5a+DwR31kcF1J4I2T1y2jD0LiHQ7+8TmNFcwcD1BiF5xkqhQhdMY7q pE0rFKTvNbfpOLupROHt+7R0alezLU8= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=izEH8nlj; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf28.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686161985; a=rsa-sha256; cv=none; b=0WckHQA/EbkivhLwlSqOES95BI6ssvSxOBUodkfCr5gIvz7s/uEblT7K96Nzuuseqa6HWr 2bxPnLAk5wZo2U4KTBns8FjuvWpZhBZAXlOk+3bfJ7pFTzD2lFFV37xqXNusZc5QYTupDA j4+x37yCaotW0faZGey4dunNgXG76Eg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686161984; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OL+TnlQNDjtbdgR92AyZlN3k/q2TbFv+1S0kh3gXGzs=; b=izEH8nlj8wOMl1gqQZlQtLJeGyEg8CgqGO6RG59FTRsbUTINJdkKc1DNpO6NwyKT4uFBOY aEnnsmYPOmuDLB9OSvH4KUBcLPZreNK9HOWVluLuVfrFy5d6nHJtTohb0KkZRjouVjrMLt uqipZLzfygiENb2gLMVIhAVXP71hriQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-480-BNnBFgYAMHmHweMCL8Mjlg-1; Wed, 07 Jun 2023 14:19:41 -0400 X-MC-Unique: BNnBFgYAMHmHweMCL8Mjlg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 525B5811E7F; Wed, 7 Jun 2023 18:19:40 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id C4BF82166B25; Wed, 7 Jun 2023 18:19:37 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Al Viro , Jan Kara , Jeff Layton , David Hildenbrand , Christian Brauner , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH net-next v6 04/14] splice, net: Add a splice_eof op to file-ops and socket-ops Date: Wed, 7 Jun 2023 19:19:10 +0100 Message-ID: <20230607181920.2294972-5-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 9053CC0011 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: tngp5y5de3a8e1eztkzpwazdpe6ppmdm X-HE-Tag: 1686161985-496479 X-HE-Meta: U2FsdGVkX1/4uVMvQWIkmbaMy96HmWIaOEQBNKmcwCyBDKuvI2MGuBcThWIOEZv8a6wFlO1bftOIJQELpfXGi37K77tgcUr867he5YASpK44A6y6ZFiWfzjw4BUzGld2cJNs5Bl2r94d7eBYq2+yxgDvg/46/KE+FEb6mjE1aMx8Ly9vyExLKt84zeUTl4b9gtYKILw273033KAWKwX9abqdiZWGt7h41q2CYEI7jk9pkdgGOp+3vDC2Yfa/RtyR9oexlMSGA6esOiQyjJK1SKOOI9gWC6icVgqRTf/NPUA6wYET4tyY3sQNlH5xG/oZtJYqNgR13GcGYS/giuX2HYv8FMLcD8ORYG6Kx9PR8zydxRA+p8f4KR1OZaAQXVpFhYo+Yo8wmLuMiwzwFzUfvm4u/Hb5xjWcsSmQzFCAG+AgMOX7R39eR71xI8qlcYMeu1uZIESlfbviQQbWBVGosUe4ja/32h5h3PuMQJenK1UmyDu+Daf4mcuk8yYqOlYXHT50l+9l/ENSaz5sN+NW3l8VfmdewmtdDPs0ZnINgYbfSkefb+tMeQF+5m/otAKkGRIZ+UNDb7WGXQQgCw3FYSokJ/zIdH2DF5swj6AK1PUACSz1qElAn4pZUy0a9B6TfFzohy2wLUfY9WU3+aiwFkGwnoLwh1SAdFXo3RjeekyZi3kewnE83+HAaR6UDHnBgyVpxSj7Gop/VRQ9Ilom1CDjZmS4yS9gmFl6e1h7fdqeOQIZidPsFeAUsLwsGrxL3TejEVCQ8O0GiMmTj1VcwVFKjfv5UWv5x9duGJCGy+zctQVGyIACAImUzmD6JLrR3Gst5i9r6SXJKBdx0I8vFegf/Ay0TG2+8xS6LuIUhKzNlhdmyORwEqA6SfofmmqBGFv942cc9IsYvDB/wLzZiT7ryDln0bUOQVHVzH1FcWKxCTndAPVOZpHAgMbSG8crabCbmyC5ZQ093eTZ77n xSPXzLlu faj3ujy4sEP4dEHip+ThLUqQ6QXEGqULu8IzCmu2yJNkbpDeREC7BuS3aubPLhiShHMIZS+JQ3dWMCGpOg5w13cNU3tYGKofSLRfZadsnzpoLrQ2EuckSPUH8e2MNusv/2mmC32pgUBYngSS1lHXx2QzKeb2Z/HBrBzDsEUMHJJ1Z4eyRiNGOpB/RZrPSMpMDsy0hgKdRTRD0p1M7je8SZt1fwBtb1tzIwGxiCZAthq0X+x5264W38bi6C85M4E/O/hd/7hJSHx7c0Y7O8GzztUCSnL9+uQ7PqK1M8c0OKUho5LOqZqxpkPpALBVq3QMQku39zoqKnwKR+u/0b9ht/0qLTzM28NLAbEK9SsHDuxPwG6WbDQZXC3WDJc429IrpyN7gvDTs0dmek/VswJB+032RocpGi734IgNaPs468nA8Zs98aHPAyqVlNJAfIg1KyOhryPQwyy0yxaxfjt/ZB3TJZkuawTMgBUPSLnddmv8rqb1GcN5K2npp9RpGH08VJe+qUcMR5I8/6NwsiMentkaD07xq+gllewxCgCcSPVHx4B8P8Z2xNCuGrKxlBciWCocdZtR+AylpnzUwZ8HUF0Mb97Dl2+Wy6jD6bIxe19bffrkILcxadRvYYPPKGfo75Xf8vsDJ3HW6Ih8RXElbAmIsKF1FWUlklK0jpbfJ0xxpz/+ZV3A1BDkYdTy6+KHr3Ak5wCMaBzVIXliTGyVa0vDJ8xKNDhq7LPlV0m+CEaNKrzY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add an optional method, ->splice_eof(), to allow splice to indicate the premature termination of a splice to struct file_operations and struct proto_ops. This is called if sendfile() or splice() encounters all of the following conditions inside splice_direct_to_actor(): (1) the user did not set SPLICE_F_MORE (splice only), and (2) an EOF condition occurred (->splice_read() returned 0), and (3) we haven't read enough to fulfill the request (ie. len > 0 still), and (4) we have already spliced at least one byte. A further patch will modify the behaviour of SPLICE_F_MORE to always be passed to the actor if either the user set it or we haven't yet read sufficient data to fulfill the request. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: Matthew Wilcox cc: Jan Kara cc: Jeff Layton cc: David Hildenbrand cc: Christian Brauner cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org --- fs/splice.c | 31 ++++++++++++++++++++++++++++++- include/linux/fs.h | 1 + include/linux/net.h | 1 + include/linux/splice.h | 1 + include/net/sock.h | 1 + net/socket.c | 10 ++++++++++ 6 files changed, 44 insertions(+), 1 deletion(-) diff --git a/fs/splice.c b/fs/splice.c index e337630aed64..67dbd85db207 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -969,6 +969,17 @@ static long do_splice_from(struct pipe_inode_info *pipe, struct file *out, return out->f_op->splice_write(pipe, out, ppos, len, flags); } +/* + * Indicate to the caller that there was a premature EOF when reading from the + * source and the caller didn't indicate they would be sending more data after + * this. + */ +static void do_splice_eof(struct splice_desc *sd) +{ + if (sd->splice_eof) + sd->splice_eof(sd); +} + /* * Attempt to initiate a splice from a file to a pipe. */ @@ -1068,7 +1079,7 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, ret = do_splice_to(in, &pos, pipe, len, flags); if (unlikely(ret <= 0)) - goto out_release; + goto read_failure; read_len = ret; sd->total_len = read_len; @@ -1108,6 +1119,15 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, file_accessed(in); return bytes; +read_failure: + /* + * If the user did *not* set SPLICE_F_MORE *and* we didn't hit that + * "use all of len" case that cleared SPLICE_F_MORE, *and* we did a + * "->splice_in()" that returned EOF (ie zero) *and* we have sent at + * least 1 byte *then* we will also do the ->splice_eof() call. + */ + if (ret == 0 && !more && len > 0 && bytes) + do_splice_eof(sd); out_release: /* * If we did an incomplete transfer we must release @@ -1136,6 +1156,14 @@ static int direct_splice_actor(struct pipe_inode_info *pipe, sd->flags); } +static void direct_file_splice_eof(struct splice_desc *sd) +{ + struct file *file = sd->u.file; + + if (file->f_op->splice_eof) + file->f_op->splice_eof(file); +} + /** * do_splice_direct - splices data directly between two files * @in: file to splice from @@ -1161,6 +1189,7 @@ long do_splice_direct(struct file *in, loff_t *ppos, struct file *out, .flags = flags, .pos = *ppos, .u.file = out, + .splice_eof = direct_file_splice_eof, .opos = opos, }; long ret; diff --git a/include/linux/fs.h b/include/linux/fs.h index df92f4b3d122..de2cb1132f07 100644 --- a/include/linux/fs.h +++ b/include/linux/fs.h @@ -1796,6 +1796,7 @@ struct file_operations { int (*flock) (struct file *, int, struct file_lock *); ssize_t (*splice_write)(struct pipe_inode_info *, struct file *, loff_t *, size_t, unsigned int); ssize_t (*splice_read)(struct file *, loff_t *, struct pipe_inode_info *, size_t, unsigned int); + void (*splice_eof)(struct file *file); int (*setlease)(struct file *, long, struct file_lock **, void **); long (*fallocate)(struct file *file, int mode, loff_t offset, loff_t len); diff --git a/include/linux/net.h b/include/linux/net.h index b73ad8e3c212..8defc8f1d82e 100644 --- a/include/linux/net.h +++ b/include/linux/net.h @@ -210,6 +210,7 @@ struct proto_ops { int offset, size_t size, int flags); ssize_t (*splice_read)(struct socket *sock, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); + void (*splice_eof)(struct socket *sock); int (*set_peek_off)(struct sock *sk, int val); int (*peek_len)(struct socket *sock); diff --git a/include/linux/splice.h b/include/linux/splice.h index 991ae318b6eb..4fab18a6e371 100644 --- a/include/linux/splice.h +++ b/include/linux/splice.h @@ -38,6 +38,7 @@ struct splice_desc { struct file *file; /* file to read/write */ void *data; /* cookie */ } u; + void (*splice_eof)(struct splice_desc *sd); /* Unexpected EOF handler */ loff_t pos; /* file position */ loff_t *opos; /* sendfile: output position */ size_t num_spliced; /* number of bytes already spliced */ diff --git a/include/net/sock.h b/include/net/sock.h index b418425d7230..ae2d74a0bc4c 100644 --- a/include/net/sock.h +++ b/include/net/sock.h @@ -1271,6 +1271,7 @@ struct proto { size_t len, int flags, int *addr_len); int (*sendpage)(struct sock *sk, struct page *page, int offset, size_t size, int flags); + void (*splice_eof)(struct socket *sock); int (*bind)(struct sock *sk, struct sockaddr *addr, int addr_len); int (*bind_add)(struct sock *sk, diff --git a/net/socket.c b/net/socket.c index c4d9104418c8..b778fc03c6e0 100644 --- a/net/socket.c +++ b/net/socket.c @@ -130,6 +130,7 @@ static int sock_fasync(int fd, struct file *filp, int on); static ssize_t sock_splice_read(struct file *file, loff_t *ppos, struct pipe_inode_info *pipe, size_t len, unsigned int flags); +static void sock_splice_eof(struct file *file); #ifdef CONFIG_PROC_FS static void sock_show_fdinfo(struct seq_file *m, struct file *f) @@ -163,6 +164,7 @@ static const struct file_operations socket_file_ops = { .fasync = sock_fasync, .splice_write = splice_to_socket, .splice_read = sock_splice_read, + .splice_eof = sock_splice_eof, .show_fdinfo = sock_show_fdinfo, }; @@ -1076,6 +1078,14 @@ static ssize_t sock_splice_read(struct file *file, loff_t *ppos, return sock->ops->splice_read(sock, ppos, pipe, len, flags); } +static void sock_splice_eof(struct file *file) +{ + struct socket *sock = file->private_data; + + if (sock->ops->splice_eof) + sock->ops->splice_eof(sock); +} + static ssize_t sock_read_iter(struct kiocb *iocb, struct iov_iter *to) { struct file *file = iocb->ki_filp; From patchwork Wed Jun 7 18:19:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271091 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94C15C7EE23 for ; Wed, 7 Jun 2023 18:20:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1C813900005; Wed, 7 Jun 2023 14:20:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1516D900004; Wed, 7 Jun 2023 14:20:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F0D5E900005; Wed, 7 Jun 2023 14:20:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DB41C900004 for ; Wed, 7 Jun 2023 14:20:06 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8C7AD80353 for ; Wed, 7 Jun 2023 18:20:06 +0000 (UTC) X-FDA: 80876766012.29.E11E143 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf19.hostedemail.com (Postfix) with ESMTP id 9F4C11A0002 for ; Wed, 7 Jun 2023 18:20:04 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=GVv0lUv+; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162004; a=rsa-sha256; cv=none; b=NlWq0Asr01m+vKvIEOWj5QDNq7yx/T81R8xi1LT3rtirh3LS72J87nMbpMNH/eV1QBJNAu ia2R0ET1kZEt2Mvv9JMY2LmjxbtBbnjHjUCVjuq9GtgrZjocBt1IpaIN9IonInrnxLJQ5E YDDW+pMrNG2Y9LVQQbqyHXnKij4CO3o= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=GVv0lUv+; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf19.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162004; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=XqvSAbXVPa8XwKzNr0PvZXw2FsrUtjvMm73oMSig34E=; b=w5hi/gzv5tXf1pBUfLwrM/lTzhL9/ifel/9Dx8LEOQMLjv+0mF2+44koCX1HKh3Qfb6CSV HLu++dawdxGiVfJaTqG/P0Y7JedyoXE8FO3RCDP4q1C+ZlgGb43hePf4H5O7tnOvD+5ANr Bx6ddxQ76q69MLD8kA1al7CXaecvDPo= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162003; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XqvSAbXVPa8XwKzNr0PvZXw2FsrUtjvMm73oMSig34E=; b=GVv0lUv+mi+vUq/sBMaukckmfGHxoogweveadMhrWZ3LpExnFMvF37ghL/qj1l+ooI04fJ y2m+YGcEsFUJS0Q9B8pOnM2uo54Q1RTfQP49NePpN2ZcW09lAZSGcARH18xr/43HScy16E qT2yw5L6ST/5cVeot4UTAAprUdwZfLs= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-617-EoBIkxF_MvKXdSL0ouQGYQ-1; Wed, 07 Jun 2023 14:20:00 -0400 X-MC-Unique: EoBIkxF_MvKXdSL0ouQGYQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A04B23806704; Wed, 7 Jun 2023 18:19:59 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 249F82166B25; Wed, 7 Jun 2023 18:19:41 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 05/14] tls/sw: Use splice_eof() to flush Date: Wed, 7 Jun 2023 19:19:11 +0100 Message-ID: <20230607181920.2294972-6-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 9F4C11A0002 X-Stat-Signature: jjpc4ay9rwetybkdisp1xz19yisukejk X-HE-Tag: 1686162004-739047 X-HE-Meta: U2FsdGVkX1+RH218h5WSew5DOUh2IC9QfK/POCYry/L2Bz6hpcImJ2C49tIHLqeT70hx6JSpjq21YeLY54qkXSeVLr81/icWbbDLTo3uJmjb04RhqTmNuj8ho2Y6V6ToaeFHYH1A5wZ6y2P1m4lEn9cdOJ8eo4LaMMfriVTJJz/2EgcEPbreZ+KgSvkzT0ZLCGxK7m9DtVIoDDsH+W4SgbZoGj56Pl31srNvv68J50nGb3qiq1f6uaLRfvZuy3d8IEt6qEAuPHpmp0hED+Iczo/Dv5uZY3v7g7gtgFL2dq9vKclDEFQnjdETVvDMkTAcbtOs7nUaBoIIQMykdF3QnvPHenhbcyqWRyCmfz8sT5EKwpngzMU7DiI0eTmWN34HL8a/XSaxKyOBRgUzO8E37Y1qC4bTJ79654IxKk/hGmbkdwdONLGBCDTgWOqdCiB5urfWrclCWFi2Ybo6Z1cNsiFPQgWMasB2ei1uGERVm6gg8QXMFHUSilkT48YbInv0sTdco9B7//uTJ8DCiUzXt2n3uSoKcwh3aLI6TAKxG2RUDxPK/pYL1DWMIUFIB46ETUk4kkodn+m8L9bjKNYniaUSrgluewwBywRayOOXbiNTVNvInyn8bUypBLKOLgk7WCSOGqbdDzufhwpe30DaH4/h2V2fre+SDsrdTJICQm+me4LQNzdrADxu+w4ii/5y87dcteY2fPwUSaxQsm5WnKqt++YkSdoYoE3jk5QRumDtyGd1CB1MapdSwHJGLrOljn2lU1HD+3K1wyFsNIbbIeCQ1p8HlYIXfxYhQ5QSYDyaCV4z/NfpcNqsx+ZreC/B17ewEgiCRV0TStTlClDqy31VQfKSm/IE4PFpZIO3AbNIsoC+JrXTjRmHN5d1hyjXvQ5SA4ilaZhZ+6/bJ51Jaxsbg301KP9LZC1bptRKkZnCvuqnG4wPHdfWujH9etGyUB/FJ66Xhb0fC/NXKI1 B+XAtcPx aLfsB+7f9EQrf0RdqQCUCtnIu1FL0RmfLc1ki/DTdcZAM+RZu2/cFOmblJWvz3oD8epx9ZllfVmbduoNRV2jAKr3r3LjHF7zDf9ks4mUeCo+mTAKOwqLqquBC5qGcP6E9eSrt06AbSaLY21bxVSCMWEBKYOc2nEhyvcNuQRshmw9dFyhpJdwzVD8GjcEgaeIDtWdlN8mqo/IjBo0oesML+N3R7dUZXv6iNc9UjzSVRgPFJgx3IkWduKTNgkSyGVH1w7zdnBF2ZW4X275Crxf8CUX7sleQiizRiB2Q8BG7tUCUw9ARlNbr6APfEQ20ZteyK+MQ508RFQUL2w0Hjg+d/H7UpXKXPPANDRONSPN1qwcHlYAbiSv+pEP0Cr+dKVVhB82jwawoDJgi3O5+f8DUn2f1ARe/GEH0xU62h1teEyxlzPkQSCs6Ind+JAK0rMnjLMvR6SEqr2XADMWqXZujzqJH0R83RamzArdBmj4Y7s0ntqu7CHouqPqoHdHYAGmVJen33bzVO5U6ro8f0eiC3HepKpxrm623a7geBUu+fovJDT63P+PvyrdLTlQUMiN18qvl3gvhBU1LK7INK2XDzqXVD3Xpv0W5lZzi5vCOj3nKYHpMXt2fhB+IAQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called TLS with a sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls.h | 1 + net/tls/tls_main.c | 2 ++ net/tls/tls_sw.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 77 insertions(+) diff --git a/net/tls/tls.h b/net/tls/tls.h index 0672acab2773..4922668fefaa 100644 --- a/net/tls/tls.h +++ b/net/tls/tls.h @@ -97,6 +97,7 @@ void tls_update_rx_zc_capable(struct tls_context *tls_ctx); void tls_sw_strparser_arm(struct sock *sk, struct tls_context *ctx); void tls_sw_strparser_done(struct tls_context *tls_ctx); int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); +void tls_sw_splice_eof(struct socket *sock); int tls_sw_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tls_sw_sendpage(struct sock *sk, struct page *page, diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index e02a0d882ed3..82ec5c654f32 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -957,6 +957,7 @@ static void build_proto_ops(struct proto_ops ops[TLS_NUM_CONFIG][TLS_NUM_CONFIG] ops[TLS_BASE][TLS_BASE] = *base; ops[TLS_SW ][TLS_BASE] = ops[TLS_BASE][TLS_BASE]; + ops[TLS_SW ][TLS_BASE].splice_eof = tls_sw_splice_eof; ops[TLS_SW ][TLS_BASE].sendpage_locked = tls_sw_sendpage_locked; ops[TLS_BASE][TLS_SW ] = ops[TLS_BASE][TLS_BASE]; @@ -1027,6 +1028,7 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG], prot[TLS_SW][TLS_BASE] = prot[TLS_BASE][TLS_BASE]; prot[TLS_SW][TLS_BASE].sendmsg = tls_sw_sendmsg; + prot[TLS_SW][TLS_BASE].splice_eof = tls_sw_splice_eof; prot[TLS_SW][TLS_BASE].sendpage = tls_sw_sendpage; prot[TLS_BASE][TLS_SW] = prot[TLS_BASE][TLS_BASE]; diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 38acc27a0dd0..a2fb0256ff1c 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -1157,6 +1157,80 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) return copied > 0 ? copied : ret; } +/* + * Handle unexpected EOF during splice without SPLICE_F_MORE set. + */ +void tls_sw_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tls_context *tls_ctx = tls_get_ctx(sk); + struct tls_sw_context_tx *ctx = tls_sw_ctx_tx(tls_ctx); + struct tls_rec *rec; + struct sk_msg *msg_pl; + ssize_t copied = 0; + bool retrying = false; + int ret = 0; + int pending; + + if (!ctx->open_rec) + return; + + mutex_lock(&tls_ctx->tx_lock); + lock_sock(sk); + +retry: + rec = ctx->open_rec; + if (!rec) + goto unlock; + + msg_pl = &rec->msg_plaintext; + + /* Check the BPF advisor and perform transmission. */ + ret = bpf_exec_tx_verdict(msg_pl, sk, false, TLS_RECORD_TYPE_DATA, + &copied, 0); + switch (ret) { + case 0: + case -EAGAIN: + if (retrying) + goto unlock; + retrying = true; + goto retry; + case -EINPROGRESS: + break; + default: + goto unlock; + } + + /* Wait for pending encryptions to get completed */ + spin_lock_bh(&ctx->encrypt_compl_lock); + ctx->async_notify = true; + + pending = atomic_read(&ctx->encrypt_pending); + spin_unlock_bh(&ctx->encrypt_compl_lock); + if (pending) + crypto_wait_req(-EINPROGRESS, &ctx->async_wait); + else + reinit_completion(&ctx->async_wait.completion); + + /* There can be no concurrent accesses, since we have no pending + * encrypt operations + */ + WRITE_ONCE(ctx->async_notify, false); + + if (ctx->async_wait.err) + goto unlock; + + /* Transmit if any encryptions have completed */ + if (test_and_clear_bit(BIT_TX_SCHEDULED, &ctx->tx_bitmask)) { + cancel_delayed_work(&ctx->tx_work.work); + tls_tx_records(sk, 0); + } + +unlock: + release_sock(sk); + mutex_unlock(&tls_ctx->tx_lock); +} + static int tls_sw_do_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { From patchwork Wed Jun 7 18:19:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95F0DC7EE2E for ; Wed, 7 Jun 2023 18:20:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A545390000B; Wed, 7 Jun 2023 14:20:30 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DFAB900004; Wed, 7 Jun 2023 14:20:30 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 794CE90000B; Wed, 7 Jun 2023 14:20:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 63422900004 for ; Wed, 7 Jun 2023 14:20:30 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 28535A034B for ; Wed, 7 Jun 2023 18:20:30 +0000 (UTC) X-FDA: 80876767020.28.8F8C960 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 4B93C40018 for ; Wed, 7 Jun 2023 18:20:28 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ioAUqIfG; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162028; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IkTAZr7FImbpw2xGK0s50kcwLgB1Jva9F8sfMs3my/4=; b=g3tsY6J9jCpkF8WBPj4HyOrlnGQIbUxOSHa4E+QFc5QVQ7fJy38Scbq9lvCcqbQQuuxPqB +gTdIgAJKpWs/+VNoAm1Znsj0eGVKtzVbEShR3KgKL2xSI1qT4NQBehEdymOlJIGnfNGyg YbRX/QtnkbLZ40bDDzlaGoYc5u+U66A= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162028; a=rsa-sha256; cv=none; b=eISXSijfLIscRPgVQobpOKDbof+wi+6Hnchgd9KH5n4J4nnNiLKqniqClb8j08wDPSFXB8 TxlA0G41yz/f4w2yzoEiwTUTJQNrj6UIrtSkjJH/yHgzFwMrcd4aWMXxpAVUgb1V4hcz9Y GSZOK6Z+5+3fNStzp43DsODOkrYkNnk= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=ioAUqIfG; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162027; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IkTAZr7FImbpw2xGK0s50kcwLgB1Jva9F8sfMs3my/4=; b=ioAUqIfG4h4zxyH8Cb+PFoSyJwFhiQ6Sdn1OuIZdQfT5A22FMzguHPfsuvhyXj4UffFxcA p5qsukxaYB94sk8mypg/gIPQ/5WZZdAYd1HkyJtl/AxLU9Z2BCVWehQgugw+TqivtIX1Xq OuMmmKxUwMEaQa7DXwWY45MuZ4D9FQ8= Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-389-fnwzd9GMNVGNYa4DGCVeZg-1; Wed, 07 Jun 2023 14:20:09 -0400 X-MC-Unique: fnwzd9GMNVGNYa4DGCVeZg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 89E833806704; Wed, 7 Jun 2023 18:20:03 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 692A51121314; Wed, 7 Jun 2023 18:20:00 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 06/14] tls/device: Use splice_eof() to flush Date: Wed, 7 Jun 2023 19:19:12 +0100 Message-ID: <20230607181920.2294972-7-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 X-Rspamd-Queue-Id: 4B93C40018 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: i99nzdkpm9mhnefgi8umtzp6tr969ju9 X-HE-Tag: 1686162028-462596 X-HE-Meta: U2FsdGVkX18Vjo2brBU42UX8OtiEPkdk6gDs2EhPJ5ZgiyL0sNyLwkp/qJVzMwW9rOZb+XSGqQQryzclVHnXJJOltPgtkqkXgjUN0ichfWGiIA6birYebfAOgaaaqKlG8/fVn5mSFcFN78+/Obdg6vDCxbK817fsms9xKsfYL7daeOjlno+ngbQFsSHU94MrxVWpA4HzCJ7No18ZaCUX65Y3XlZW2NVRhdBRZQdp3qabrUgYMxjNCNBMOKN8jlmVxBV4mOF6QXgPjV78A7BwBzb3vJMCTd1pooTrndj0SoZqmtYZ+kL6WkdZQiVBS7r1HJmApp1c3GWbFDNZp9ZHcptqIVeCwkw/3hqrdmiopZ8t4HFmUEMYwCK+L0TCCv5TnbjnDa55n11ub9WOz9XBhK4C5/mtcrrFfhQ7Y3nMbCFULmkPsHp/+EJ9Eq0YDUNoUlMVxHqifBzVS0Nf5r/yeb1YaFb1xHo5SMNIA52Qxqvxy86ayyPVkT8myLy8lPRy33vsUXoxV0EBOISTKYcnz33ipgdslJl5ulzyc0zq1jcMmrVVcT46sfPWib/7pXAeZVoilVSRFkKcgN/qoxrGmenLR2Hp0X2dsLJ1BPd6mT+aOmCuoZDEXwKyjyShCRXGmK4HePs6xpbYH4V0ood6gB0Hs50Aq2PiPMdTcWHfv1CarFWha4QmoRs9nghQQhLDUsY96B4iBtUzNvCmrr2M8jMvgiinvzAg//YOPqagFum1tXwqM49VinHvzpIXgrmY2gdDK0GjxyvLqfx+8qkKV5jSIHbQ+cKqKHGDhoOJzNgX2spWb5kiguf8gscwe7YcWN2Tzc8Kh+eYdeB+vWUdcShz3lMrq7FarrLW+cG5QnIhythRseLd6hifkdFBoTro17cTQsQHLSxC1DzsYqn06/wtj/kS39m9a6bOZdP882Tvk3kUXbRJCmNoJfGhCqRAVRSDLddb/iQJ7oP3FF1 7gmhe/yI C4Gs4yXP4+6ucsyjdw4VPE8b46fGE0a0Dohq8wUCV36TxqYCrP1DNhOsZHnR63r/eFqlBK4DIBzB+pttkSDZPLkB8sR1P3asp6J+7IRSagfd2rUiAZGBQMXWBmYGz6eLaD0SdnAbwGZ2ugR/bSvHaf8ioAuTyXGg1P7n2GvvC46Ob3cwFvmxZVwA0fMHfZ0LctCn8pU6IKR2FNzarL9UMEkxmqKunBBmHxsLPJYNOV72Mv/0SbU5c+0GTjwyc107bJRhGojFyKDwAyYMoxDgKTNovem18adL0NojInxaQA1kY1nPnDL6a4051neOBOfzDC12sK9eFYBDG8PX+qFisi/qC5BP4fJ3SoBjqNnLPisee5fC+WSbE5JWikGS2+RUhdnxuH9gVlK82pmkIP7BPQYKt33WymB/E/SdBOWZtVPqW4iBIe2M/xmBopPCNTpjidFTWn7P+7y1R3GGspcKzqzITQl2uHf7IaEYbwBKjirEQRgdKqQzp7cAD6GcaKyPo5H52Pkcsg4GsLybOT9AbdnHm1+b5LDnRodtC3VUJt4JVL0m9w0mzWiWhzI8cJGlioMYEI8niBgqUOWP26lVrLjSBX2mpsX2acUx4IQX/vZe3OE85qjmQM8qcJw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called TLS with a sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls.h | 1 + net/tls/tls_device.c | 23 +++++++++++++++++++++++ net/tls/tls_main.c | 2 ++ 3 files changed, 26 insertions(+) diff --git a/net/tls/tls.h b/net/tls/tls.h index 4922668fefaa..d002c3af1966 100644 --- a/net/tls/tls.h +++ b/net/tls/tls.h @@ -116,6 +116,7 @@ ssize_t tls_sw_splice_read(struct socket *sock, loff_t *ppos, size_t len, unsigned int flags); int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); +void tls_device_splice_eof(struct socket *sock); int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tls_tx_records(struct sock *sk, int flags); diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 9ef766e41c7a..439be833dcf9 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -590,6 +590,29 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) return rc; } +void tls_device_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tls_context *tls_ctx = tls_get_ctx(sk); + union tls_iter_offset iter; + struct iov_iter iov_iter = {}; + + if (!tls_is_partially_sent_record(tls_ctx)) + return; + + mutex_lock(&tls_ctx->tx_lock); + lock_sock(sk); + + if (tls_is_partially_sent_record(tls_ctx)) { + iov_iter_bvec(&iov_iter, ITER_SOURCE, NULL, 0, 0); + iter.msg_iter = &iov_iter; + tls_push_data(sk, iter, 0, 0, TLS_RECORD_TYPE_DATA, NULL); + } + + release_sock(sk); + mutex_unlock(&tls_ctx->tx_lock); +} + int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { diff --git a/net/tls/tls_main.c b/net/tls/tls_main.c index 82ec5c654f32..7b9c83dd7de2 100644 --- a/net/tls/tls_main.c +++ b/net/tls/tls_main.c @@ -1044,10 +1044,12 @@ static void build_protos(struct proto prot[TLS_NUM_CONFIG][TLS_NUM_CONFIG], #ifdef CONFIG_TLS_DEVICE prot[TLS_HW][TLS_BASE] = prot[TLS_BASE][TLS_BASE]; prot[TLS_HW][TLS_BASE].sendmsg = tls_device_sendmsg; + prot[TLS_HW][TLS_BASE].splice_eof = tls_device_splice_eof; prot[TLS_HW][TLS_BASE].sendpage = tls_device_sendpage; prot[TLS_HW][TLS_SW] = prot[TLS_BASE][TLS_SW]; prot[TLS_HW][TLS_SW].sendmsg = tls_device_sendmsg; + prot[TLS_HW][TLS_SW].splice_eof = tls_device_splice_eof; prot[TLS_HW][TLS_SW].sendpage = tls_device_sendpage; prot[TLS_BASE][TLS_HW] = prot[TLS_BASE][TLS_SW]; From patchwork Wed Jun 7 18:19:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271092 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD8D0C7EE23 for ; Wed, 7 Jun 2023 18:20:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6AA60900006; Wed, 7 Jun 2023 14:20:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6334D900004; Wed, 7 Jun 2023 14:20:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ACC6900006; Wed, 7 Jun 2023 14:20:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 3A7CA900004 for ; Wed, 7 Jun 2023 14:20:12 -0400 (EDT) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 0E1C8140487 for ; Wed, 7 Jun 2023 18:20:12 +0000 (UTC) X-FDA: 80876766264.05.29ED028 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 415F11C001B for ; Wed, 7 Jun 2023 18:20:10 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QLm43nxc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162010; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=bcZVQkV8pZRBmqqDLmAW/OVdZTCqUxl/BY0qE6seg5Q=; b=g9ZdQNiVgJlFi+wUrWy1iweQYHOQZWJVViVSFbul7yRXw3aJ8YzvgR5fAxnvQV2vKVtEcb d1ZbDugzNI4scE8hbvAqm1bxwKwR9TGryTprzxtAOpDjD2pebSBUuGOZMIWzkra3kG3vYx p1WwflngHtbuVu3rOlqyJjVkijeZ4PE= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=QLm43nxc; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162010; a=rsa-sha256; cv=none; b=ruympyvTCMvFXbQ970a6XytOpcV17BfBxGScySmsoHqGhIik9Yty5tSkVBLW+dSgaXOkAG iYSzHJHtXaPo9IyBwhVq3erFZ88buFGTREEXfSQUZYiW1XBWZJ29VHmP+wJZZchK/0c4Fl aYuh+G6Eah4ykgpy6xSE7MPOq4huXSM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162009; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bcZVQkV8pZRBmqqDLmAW/OVdZTCqUxl/BY0qE6seg5Q=; b=QLm43nxciLax4SfVKwyhbVMZQvjUWIHFn7cFw8V5dAqBLszOLwKZGPChQM2wOpoZgb+CIb BolfQ26eI8T2wvD7BsXw9ml1eMEXDXrgnmLtF5+X7ItcgqqHjlxNaQwq12Si1+/gSr7aGo 8Qo4xOtN/LjGq7Se+hcTbueLv19xtw8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-58-ImQvr5EoMlSQKIjUJYTR_w-1; Wed, 07 Jun 2023 14:20:08 -0400 X-MC-Unique: ImQvr5EoMlSQKIjUJYTR_w-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BB93B8030D1; Wed, 7 Jun 2023 18:20:06 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AD2E2026D49; Wed, 7 Jun 2023 18:20:04 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Kuniyuki Iwashima Subject: [PATCH net-next v6 07/14] ipv4, ipv6: Use splice_eof() to flush Date: Wed, 7 Jun 2023 19:19:13 +0100 Message-ID: <20230607181920.2294972-8-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Queue-Id: 415F11C001B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 5h868cstw71ozr1xi9ye6bgg4ff83dyc X-HE-Tag: 1686162010-117982 X-HE-Meta: U2FsdGVkX1+vd5vmfcm6Sf764nlarA1OxKNhgLsnqffkgiCwnxqFpIA0n8HWQVP3TS53Mso3UDcLww8PNlyneHE+Zg/ydi549WOyXFlH87Cazfq8HGyPvJtJIeDh9iC0uP9dgW3k0sr3znvBIwQCXC4nzLkAIfTZIdVmsguTU4l7fCYRGwJukOswRtD1SarJoMCQuRCRztTOL5DpqJJdejhEjJAGWVUwbPsO6Noz+e+JupK17+qgFZyVNpc7onXbUFYR6AM/MDvUxXTWE4yMTxJkpH/Fn4pxP7ig4zF5HWHnS1Hh28mgMs7eLYM97g3MB0tm8ELf7K3fOncMasDahlsefuJaWZGxxcjgPtYWE8+EvaX2RkZp9lHF5+EeBUMxCJHhFsEny73Dm7S0hv2OCjiivEKS7dnyqGnt8M/GrNqM0Wa2VNdzB/58Sc5z4Ce23twWov9tttdYwX9hja5U6FOTjR+9ElnAnf6OoWMhDBpbHIuJ4JyRSeIz/B4gepUtvzDlywOvUmF4hME4H+nXsej6ihJZivf2kvnUZR57vpk0Tnz9euSSgz57RJtfWfL6brOkdgQdE2qejzYaPU8GjDlnjpasbGwZ+1ScF6NH7duOIFePJgMzl/vU5RpCUreKhbSxRmOVh6qADJFFOFibFTQhFFRMmMnr49+MgnvOTXzyj/ESpzQ4Q7I9vzyWKdYPXVRNdwW6fZOwqy2tGmGcD0lk+rH/1ZWceC8R4iH3fDpUvzRq2PCmsoSRJK6JjC+TFrafaHyy9Cf6GbIYB4cl1yuXCMmgIXjd5KYCuxbC9JiiOe4KnPGagjKb3nhPbjutNzmjA8ovKZMLGnXRTJ9lzn9ql/174m9Ty5TuxuA8xsynzEd5vmwqGNAP6fKQQN2YF4+vciGWr79uoxIxMkYfybOHQo6nAw0JfYS1UtxpTmnifB751DxuhsycVBi3oAjHEdwRbNagLAzBKCizGbf /kI32q9u H3ZnwkVDrV23cVINeBh95j3RllB5sFfj/tu7Eq6TIkOjo1yanxWNPHvzBR3X9NfZDu7qgUfJdnpbe39hF7qInpIqrbYAKwtXC2seQhK5lf1KSoVx2PJtP1RK7/tvsMBwiN05WQGEaRB2kKgzbAL2IeFyQ/3b/r/lP4Dwg9sYTbk4DnbVMRW7oWdRZ6o4lghwDRPFvtjlemoq0CmuYmyttCjdDpCUHMPxLkZQhhybnuXixlJv8o0Y450PXVyMYqUwRQ3soN8lLE8dt/lZFyl6BDQIaYbL+x5LZBkFsPX4l8dTXXUPQ5tD5khOP9PAWxk/7GQo0h/i0xpFEu+Gs44qeaL+/S/9GkITkmUMAcQcT787TCj5zhSA4lZZKtPsyN/iNM0F9vWmtSOBKV/SSzqcWQMJOT+ldDzdBEEXtT5tvB5op4xsrcsWrwqVB3lzEQUJsW8CWW1G4xN7meWM6BHYoVETWn3H0y4NlM+5uyjiDsunaWaOfFMRZnLvBM9JtTgJLSb+9dltkAR2bT3tAwsotzCQPDAU+M0+tTthNNRVtjtKPP3SSg2Jas+S58vNbXsPV9S8s56BpzU/Ga4uThlvLhBRbHmZov7r+/SQG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to undo the effects of MSG_MORE after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. For UDP, a pending packet will not be emitted if the socket is closed before it is flushed; with this change, it be flushed by ->splice_eof(). For TCP, it's not clear that MSG_MORE is actually effective. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Kuniyuki Iwashima cc: Eric Dumazet cc: Willem de Bruijn cc: David Ahern cc: "David S. Miller" cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - In inet_splice_eof(), use prot after deref of sk->sk_prot. - In udpv6_splice_eof(), use udp_v6_push_pending_frames(). - In udpv6_splice_eof(), don't check for AF_INET. include/net/inet_common.h | 1 + include/net/tcp.h | 1 + include/net/udp.h | 1 + net/ipv4/af_inet.c | 18 ++++++++++++++++++ net/ipv4/tcp.c | 16 ++++++++++++++++ net/ipv4/tcp_ipv4.c | 1 + net/ipv4/udp.c | 16 ++++++++++++++++ net/ipv6/af_inet6.c | 1 + net/ipv6/tcp_ipv6.c | 1 + net/ipv6/udp.c | 15 +++++++++++++++ 10 files changed, 71 insertions(+) diff --git a/include/net/inet_common.h b/include/net/inet_common.h index 77f4b0ef5b92..a75333342c4e 100644 --- a/include/net/inet_common.h +++ b/include/net/inet_common.h @@ -35,6 +35,7 @@ void __inet_accept(struct socket *sock, struct socket *newsock, struct sock *newsk); int inet_send_prepare(struct sock *sk); int inet_sendmsg(struct socket *sock, struct msghdr *msg, size_t size); +void inet_splice_eof(struct socket *sock); ssize_t inet_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags); int inet_recvmsg(struct socket *sock, struct msghdr *msg, size_t size, diff --git a/include/net/tcp.h b/include/net/tcp.h index 68990a8f556a..49611af31bb7 100644 --- a/include/net/tcp.h +++ b/include/net/tcp.h @@ -327,6 +327,7 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); int tcp_sendmsg_locked(struct sock *sk, struct msghdr *msg, size_t size); int tcp_sendmsg_fastopen(struct sock *sk, struct msghdr *msg, int *copied, size_t size, struct ubuf_info *uarg); +void tcp_splice_eof(struct socket *sock); int tcp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int tcp_sendpage_locked(struct sock *sk, struct page *page, int offset, diff --git a/include/net/udp.h b/include/net/udp.h index 5cad44318d71..4ed0b47c5582 100644 --- a/include/net/udp.h +++ b/include/net/udp.h @@ -278,6 +278,7 @@ int udp_get_port(struct sock *sk, unsigned short snum, int udp_err(struct sk_buff *, u32); int udp_abort(struct sock *sk, int err); int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len); +void udp_splice_eof(struct socket *sock); int udp_push_pending_frames(struct sock *sk); void udp_flush_pending_frames(struct sock *sk); int udp_cmsg_send(struct sock *sk, struct msghdr *msg, u16 *gso_size); diff --git a/net/ipv4/af_inet.c b/net/ipv4/af_inet.c index b5735b3551cf..fd233c4195ac 100644 --- a/net/ipv4/af_inet.c +++ b/net/ipv4/af_inet.c @@ -831,6 +831,21 @@ int inet_sendmsg(struct socket *sock, struct msghdr *msg, size_t size) } EXPORT_SYMBOL(inet_sendmsg); +void inet_splice_eof(struct socket *sock) +{ + const struct proto *prot; + struct sock *sk = sock->sk; + + if (unlikely(inet_send_prepare(sk))) + return; + + /* IPV6_ADDRFORM can change sk->sk_prot under us. */ + prot = READ_ONCE(sk->sk_prot); + if (prot->splice_eof) + prot->splice_eof(sock); +} +EXPORT_SYMBOL_GPL(inet_splice_eof); + ssize_t inet_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags) { @@ -1050,6 +1065,7 @@ const struct proto_ops inet_stream_ops = { #ifdef CONFIG_MMU .mmap = tcp_mmap, #endif + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .splice_read = tcp_splice_read, .read_sock = tcp_read_sock, @@ -1084,6 +1100,7 @@ const struct proto_ops inet_dgram_ops = { .read_skb = udp_read_skb, .recvmsg = inet_recvmsg, .mmap = sock_no_mmap, + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .set_peek_off = sk_set_peek_off, #ifdef CONFIG_COMPAT @@ -1115,6 +1132,7 @@ static const struct proto_ops inet_sockraw_ops = { .sendmsg = inet_sendmsg, .recvmsg = inet_recvmsg, .mmap = sock_no_mmap, + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, #ifdef CONFIG_COMPAT .compat_ioctl = inet_compat_ioctl, diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 53b7751b68e1..09f03221a6f1 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -1371,6 +1371,22 @@ int tcp_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) } EXPORT_SYMBOL(tcp_sendmsg); +void tcp_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct tcp_sock *tp = tcp_sk(sk); + int mss_now, size_goal; + + if (!tcp_write_queue_tail(sk)) + return; + + lock_sock(sk); + mss_now = tcp_send_mss(sk, &size_goal, 0); + tcp_push(sk, 0, mss_now, tp->nonagle, size_goal); + release_sock(sk); +} +EXPORT_SYMBOL_GPL(tcp_splice_eof); + /* * Handle reading urgent data. BSD has very simple semantics for * this, no blocking and very strange errors 8) diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 53e9ce2f05bb..84a5d557dc1a 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -3116,6 +3116,7 @@ struct proto tcp_prot = { .keepalive = tcp_set_keepalive, .recvmsg = tcp_recvmsg, .sendmsg = tcp_sendmsg, + .splice_eof = tcp_splice_eof, .sendpage = tcp_sendpage, .backlog_rcv = tcp_v4_do_rcv, .release_cb = tcp_release_cb, diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index fd3dae081f3a..df5e407286d7 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -1324,6 +1324,21 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } EXPORT_SYMBOL(udp_sendmsg); +void udp_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct udp_sock *up = udp_sk(sk); + + if (!up->pending || READ_ONCE(up->corkflag)) + return; + + lock_sock(sk); + if (up->pending && !READ_ONCE(up->corkflag)) + udp_push_pending_frames(sk); + release_sock(sk); +} +EXPORT_SYMBOL_GPL(udp_splice_eof); + int udp_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { @@ -2918,6 +2933,7 @@ struct proto udp_prot = { .getsockopt = udp_getsockopt, .sendmsg = udp_sendmsg, .recvmsg = udp_recvmsg, + .splice_eof = udp_splice_eof, .sendpage = udp_sendpage, .release_cb = ip4_datagram_release_cb, .hash = udp_lib_hash, diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c index 2bbf13216a3d..564942bee067 100644 --- a/net/ipv6/af_inet6.c +++ b/net/ipv6/af_inet6.c @@ -695,6 +695,7 @@ const struct proto_ops inet6_stream_ops = { #ifdef CONFIG_MMU .mmap = tcp_mmap, #endif + .splice_eof = inet_splice_eof, .sendpage = inet_sendpage, .sendmsg_locked = tcp_sendmsg_locked, .sendpage_locked = tcp_sendpage_locked, diff --git a/net/ipv6/tcp_ipv6.c b/net/ipv6/tcp_ipv6.c index d657713d1c71..c17c8ff94b79 100644 --- a/net/ipv6/tcp_ipv6.c +++ b/net/ipv6/tcp_ipv6.c @@ -2150,6 +2150,7 @@ struct proto tcpv6_prot = { .keepalive = tcp_set_keepalive, .recvmsg = tcp_recvmsg, .sendmsg = tcp_sendmsg, + .splice_eof = tcp_splice_eof, .sendpage = tcp_sendpage, .backlog_rcv = tcp_v6_do_rcv, .release_cb = tcp_release_cb, diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index e5a337e6b970..317b01c9bc39 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -1653,6 +1653,20 @@ int udpv6_sendmsg(struct sock *sk, struct msghdr *msg, size_t len) } EXPORT_SYMBOL(udpv6_sendmsg); +static void udpv6_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct udp_sock *up = udp_sk(sk); + + if (!up->pending || READ_ONCE(up->corkflag)) + return; + + lock_sock(sk); + if (up->pending && !READ_ONCE(up->corkflag)) + udp_v6_push_pending_frames(sk); + release_sock(sk); +} + void udpv6_destroy_sock(struct sock *sk) { struct udp_sock *up = udp_sk(sk); @@ -1764,6 +1778,7 @@ struct proto udpv6_prot = { .getsockopt = udpv6_getsockopt, .sendmsg = udpv6_sendmsg, .recvmsg = udpv6_recvmsg, + .splice_eof = udpv6_splice_eof, .release_cb = ip6_datagram_release_cb, .hash = udp_lib_hash, .unhash = udp_lib_unhash, From patchwork Wed Jun 7 18:19:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271093 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D32AFC7EE23 for ; Wed, 7 Jun 2023 18:20:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7963C8E0001; Wed, 7 Jun 2023 14:20:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 71F39900004; Wed, 7 Jun 2023 14:20:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 57012900007; Wed, 7 Jun 2023 14:20:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 467EA900004 for ; Wed, 7 Jun 2023 14:20:17 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 2180580354 for ; Wed, 7 Jun 2023 18:20:17 +0000 (UTC) X-FDA: 80876766474.12.6B87C5E Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf11.hostedemail.com (Postfix) with ESMTP id 7328240008 for ; Wed, 7 Jun 2023 18:20:15 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gqR5U08G; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162015; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=axbQ2fmWT7oadEpv+RKPywBYD27/saJPi8A/5AZqeFc=; b=IQz/OHPlOzamw/2T3RLaxRXeCKXWbgCgkLrd27tAzn4AdVtqvUAlBOAsREUBWq/hIUUCfp yV8qF6CdXZCGfkoJb7WQjL4TrDjOrs0ybXUQXl5sUvrG0ezoFSysJEt8BsnFikt3UY6ayl efULSRyWQWcSmanjjIkICSEcprC24is= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162015; a=rsa-sha256; cv=none; b=JXXJzr7hr8w4kMfTOv9Ol1S7ZUE+Y2ZqCTkRBP2oO18igmTqeP8tKCtuynjrlZq5EiUVXh DMaX9PBQvmR/RWP2Kx81yOaI8J7F6kHIMpcYKsr5NHBTq29J8p3JWPimr9BOgjDaFdV33s bE16pwfdQSfLc1XWqrYOjmfs9ft1cNM= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=gqR5U08G; spf=pass (imf11.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162014; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=axbQ2fmWT7oadEpv+RKPywBYD27/saJPi8A/5AZqeFc=; b=gqR5U08Gcm756+2sKWslptqLrAHrRF1cnTudLTB7NQi5Uk/cqxxWBe+qa8SF3lzheXrndQ frjoOwEIv4Ld6En5QXR3H+2GBkFKZyACnrQCLTPDor2pWB/68EYItEkAOVFi5lVjZlcXs1 XsqbCHVD4HHoGZg2Hic6G1fHYHF+sIc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-296-TW2Be_pnM5qS_UH9e9Xw1g-1; Wed, 07 Jun 2023 14:20:11 -0400 X-MC-Unique: TW2Be_pnM5qS_UH9e9Xw1g-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1A99C80331C; Wed, 7 Jun 2023 18:20:10 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8D78C2026D49; Wed, 7 Jun 2023 18:20:07 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Ayush Sawal Subject: [PATCH net-next v6 08/14] chelsio/chtls: Use splice_eof() to flush Date: Wed, 7 Jun 2023 19:19:14 +0100 Message-ID: <20230607181920.2294972-9-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Rspamd-Queue-Id: 7328240008 X-Rspam-User: X-Stat-Signature: sdqntc8iwcydmsz7xh9wd5mwfrybtwjy X-Rspamd-Server: rspam03 X-HE-Tag: 1686162015-478572 X-HE-Meta: U2FsdGVkX18NPEXnK7l30Ua/VRb7n0FOzxNBZLb4RpCtJOhmQPXegdMfBVd73A07vybGdCZTDy0QnhG/XZCzkRyr3RPhHmtZc8X9LHuIMt6eaORbQnpYazX/P5wJFljWFsYYKf/zME+apSXFelwFTs5OWGeAOh3FSxEslC2k0bZuoJtzRfmHdFGWUH2nU/94fFynkAvr1Bpf+PvCMLW22MuitaoPQKBWqFDctUQgrTB8RDBnIBx+7AQdXgxD1Yob+KjIuI01hdadKsjIxlimclAzyqkxscmN6m2O0W2rindmIAeV2mzzmMJbCVKGv7rjqXxMmxBZcRLH1obdvNxt5XPhpc8vymUT8G07G6uI8DkUjbULgkAJ6LeYsSMIawDnSCkXPrtUI+9fPie6kWGfycHZRKOkoFLqRtDWlwnmRj22lN9e+Csh2dpD632102GGwIc9pcQMT6knZOdgYKsHVaC3Lv9Y0sOLZwBfMcgauVHi3MsToqa+x8htE//SRgNuknqvQWdDjmrwMzRiq6HY5Zg4hj86b4FnvrdokuXBz8AaQm37Dc+bcAEbRWgp83ji/JOeVSfBCXFl2rCI2n83qiokeXnCZHzhpOJL7Oiue9m9oUvjn3z6FLuwiTh4BlfsCFaF6t5x8zsDqxZGw0uPcVgOHnsra8ZhcJZ1y7p9Z2MMey1/+EuH4DNxj7wr+xUfoDSuejYdmAsmhb4t7Xv2xHjjNAGCFrgp6b4soT3vPC2w1d4LNvkqYTbuYPfC3P/YApH2iB6N4tdLnjbjDdGRgWzLH2UC4PPtU4mbWVx9DaqPXRu8UcZ8vpgldxhTtUUp8A7qLDoRIsJ2f6SPE5RQZw2flmxzGOgE5K/b8f3lo6IA1pMCr1ZJQ4muvKJbc1+g9gRRFWLzIPLa9f14pJR7hyoAPilquOJRxg93wTRMCZ0C9N55sgicLAZ3fjBU1emoFJ+A1LtCAwEINoEa4eF ORp9yqNy TzPHKIddXXwGlE2P9Z3iJEaO+h9M7IxHuJt3Noel9KnkUv7ODjEawrOKM5O+0M//3Or9LrBR9/dDmChy3PoynzaBTp7KdDgIIC+kHlsRyJiVYjqZxJ1yFI4xxKkq0Ayy3f/pAhyvVdy2j42EY7BMkR1MYl/sUb7qir820QNFIrOTNt6JpOkf8uejl2HP7JCSpyeqNl4VkT1Pw1UYpi0SawvdcRIPq0iE1f10Ws9yFOjW7aLuFyy1saGfJdEknBr8HbSba2FcxcBHig+Lnsd4FiMAwgfiY20LIF+PJh8ZqBv6HtMGmV/pEHBqGvLSeUNzTxmP/ZbqdNA244H4d2z3VJyBUxi1gXjZhbHcNdEIBBt/TDr+qv57NTmenIvdPGZ683b5cw1iOZpc/TI15JDsxyAK5Ll3PeV/LjPLd9X/Bff485Pc6Krf0e6pb4Cxn9eDZZWFtPmY1BpL7cmCVn56yENlFUSCG/dc6WTmZgY9uZ5oMxJw+9Ueus8i0aeeWotrOe9XBVBevkLTF5IIA43zyoC5cC2Qs5O2tQObJdQ+VDG9e4S2GJmZdU7rqOEB4ncEdafPjDm1wKqeLqNj/L1u5hWHUNwZIRFwrsjUDwzF0lOjRjW5MLWp4hLha0fet/DtD+o+5 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to end a Chelsio TLS record after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Ayush Sawal cc: "David S. Miller" cc: Eric Dumazet cc: Jakub Kicinski cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h | 1 + .../net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c | 9 +++++++++ .../ethernet/chelsio/inline_crypto/chtls/chtls_main.c | 1 + 3 files changed, 11 insertions(+) diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h index 41714203ace8..da4818d2c856 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls.h @@ -568,6 +568,7 @@ void chtls_destroy_sock(struct sock *sk); int chtls_sendmsg(struct sock *sk, struct msghdr *msg, size_t size); int chtls_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int flags, int *addr_len); +void chtls_splice_eof(struct socket *sock); int chtls_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags); int send_tx_flowc_wr(struct sock *sk, int compl, diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c index 5724bbbb6ee0..e08ac960c967 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_io.c @@ -1237,6 +1237,15 @@ int chtls_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto done; } +void chtls_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + + lock_sock(sk); + chtls_tcp_push(sk, 0); + release_sock(sk); +} + int chtls_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { diff --git a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c index 1e55b12fee51..6b6787eafd2f 100644 --- a/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c +++ b/drivers/net/ethernet/chelsio/inline_crypto/chtls/chtls_main.c @@ -606,6 +606,7 @@ static void __init chtls_init_ulp_ops(void) chtls_cpl_prot.destroy = chtls_destroy_sock; chtls_cpl_prot.shutdown = chtls_shutdown; chtls_cpl_prot.sendmsg = chtls_sendmsg; + chtls_cpl_prot.splice_eof = chtls_splice_eof; chtls_cpl_prot.sendpage = chtls_sendpage; chtls_cpl_prot.recvmsg = chtls_recvmsg; chtls_cpl_prot.setsockopt = chtls_setsockopt; From patchwork Wed Jun 7 18:19:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271094 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BB34C7EE23 for ; Wed, 7 Jun 2023 18:20:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 12C07900007; Wed, 7 Jun 2023 14:20:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B60D900004; Wed, 7 Jun 2023 14:20:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E7364900007; Wed, 7 Jun 2023 14:20:19 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id D04FD900004 for ; Wed, 7 Jun 2023 14:20:19 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 80D711C79A4 for ; Wed, 7 Jun 2023 18:20:19 +0000 (UTC) X-FDA: 80876766558.15.DD3BABD Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf18.hostedemail.com (Postfix) with ESMTP id B8DD51C0013 for ; Wed, 7 Jun 2023 18:20:17 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IyQ7cMcy; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162017; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Xw9eDWXEwpwNsn0HYqixkrc65IaqtCUycHScIiVECUM=; b=zWEo2JEj3xIRu9h6ANiHUzrKTJopEkhlwTvx6TqoRDW2mbtrXOvR8soK5k3wFklH5alGJh RP/eqAMXupABt3pfxARrEPjdby1Ms+srH9pd/incfxr/AmtPSCSQX/URD1H6LSrqmF3vE2 26XZq/pVcM78Ovvh9fYy/F7d7feacTM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162017; a=rsa-sha256; cv=none; b=ujq0+v9ptrswqa27K1iwf6nN4JYdaYHaOJfsrl13naiAeSs3mYetlNVlS7B2ysqK3URLrO xk0f7DMCU1dwdDJJogsgA+Bc6AISjYWyZB7ad4NLT4uF3Vy3vV525rzmEJYTwngIgRr7jW sJh2PsSf+ZYdDVOrIdIeF/lkICUr/GE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IyQ7cMcy; spf=pass (imf18.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Xw9eDWXEwpwNsn0HYqixkrc65IaqtCUycHScIiVECUM=; b=IyQ7cMcyGOgiTco7GHXmd96vHiCp/2cONToNf6xX7l1hqo3pvZDK5/zYteEfC8ew6y6Ma+ pqZWpsoAyVUmY5DIGFTkQQO6hSa13jZ11oSwZNAd/SqfXgM2otgeEMvpCSqwsWPS4G1tA9 PMDAC9Fv274z3l+1OJOYxDOaVOkhNK8= Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-629-YaAi_LsyPgCqEao-0jof3A-1; Wed, 07 Jun 2023 14:20:13 -0400 X-MC-Unique: YaAi_LsyPgCqEao-0jof3A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DA1683806704; Wed, 7 Jun 2023 18:20:12 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id B9CA2403176; Wed, 7 Jun 2023 18:20:10 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Tom Herbert , Tom Herbert , Cong Wang Subject: [PATCH net-next v6 09/14] kcm: Use splice_eof() to flush Date: Wed, 7 Jun 2023 19:19:15 +0100 Message-ID: <20230607181920.2294972-10-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspamd-Queue-Id: B8DD51C0013 X-Rspam-User: X-Stat-Signature: 1j1r45xc4t3oatdyiqg4am33hhiy73q1 X-Rspamd-Server: rspam03 X-HE-Tag: 1686162017-557462 X-HE-Meta: U2FsdGVkX1+s702g4QZz6pslkv430+oNTdh/Cur1TmhVSiugPa2HWYnhTzFRFfCfOKS6DPHxOIlabPd+8vujSkK8Da+qoEU52eUeqGp61I55ckpKtiuQ7fCiCiyjH5sTPEAXIAWJjJ8Dfp/V0NVi0MC0GePAVhzx3cdD/SS+NHn7YJnC7iIZO4VrlYsGGZSV/CLSALuj3/nJvR8Q8Y9rjy8mfit/I0jwjRP9of890o/QpSeofuZLlLWdGgXJjVBqU8p27VRG+k+3deytXf2Uve6xkN8Lzc+XFEEl4FvW4W/SwbrfRIgWcYisSx9XOna6vQHW7k/kAe2UgmJJfdrU5J9hm9M5DToZ4J1toVz4w8YyvzMig+1opZUw1SsJiol3gaS/nqCFMGc7GK+0mH2m5T4b/Gyv85u6vy+tUW8LV1kgYgKRl3FCFak8XWksPxVby/cBFBbszDe43iLfieSV0ZNgr9Ioyy8cM8wOONqb29TskfVh8wJBFuiiyH6TBdcsMphh/9k6MkSir84GPnciRLKAC2IRTDkxKlVzLI0E49RSqxpbPi0C7i/j4HRRnG97ckvgrKKphBpytzTK7zDksGx3N27OqDLbFD0Ojl34DVsWcTT6buTwLXTHdtFq0QAisqENRXxMUoD8YBJaRFq5vx2xmKbd//VgbjKx6VplegjaH7TdZk0AdB1d6ky2NfLxLVqBhcpMgLQXW9o00GSizYEA6ERAZ35KybGP4Af53mY1Qe1GM8c1BgChs1NjqiA0CW+YuEWQkTSHRWJhVRngpFj3/inyQi831VoPeSrozD5kkl/Kt5ULO1hOzIz3Oy2R9Tt9nTW27Nb7YHT+StWQbVbZDE4/Yi76QOdRi/Cp7OYWYtTBWDJkmx69PvK0yMlAJyUFFu0IHhqfuTOuBFbZN5k1sKCQSO8dLG6+7MkUQONYqqrpHez6UxNdqZgKKpPog3YDUDP3R0o81OsTprf Sk8ot5Mr gLu5AHU8vIUO1Ftj58Cvybf4A+SMFx1QkTUE4DntrO2EReka13O47yr5nYUg1T/nEBFmhEZqbQQ+bzUn/S7ceeJ25qeA0nAnBaiM69GLgK1Ne+laEP/Fdw9fc1BR8vnzTqkpl1MqTq3NTQtCOQdmbPk16sLKqJFf43zgUWL/KmTHl1n8BCOnBJvQh8blSgh0byxs5gynk0HGeJ+jQTDUDLtMEk5RrAsDJ+1MAfkYSeYtyQj7pktmKObch15z7HaKp3ltf0mP7+Y43VfZaI2kvlAbPmb3tQCRXBBJUVlPfIEm6hN47fJMJD6NomokjblefbwQgyg2Z9Y1tVWx+i6VWKbj3bo0Q63xYeXLQFwbncM2aJ4EVbk/3sbDe2b0Oi4TnXuODig+U85mQxzDBfUTmfAeOuFCqM2iofyDFEUD2JhHaYuZsFqfG102j456gPfmOmNhTaty+AkLOTqMqPrKR18/cw5SJPl50/J2vYanfhhSwEl36vEm927lPtobG2AHbf5/XpWizHs2e9jA0Vk4zSWvgCZ1CheHhQ3w/Nwr2cipKyoQdNbGbIRfyU2vO8GC3qWzJkUfuEcyZpOsNYH2PwcOD7Elm1IsH4mkjVR+doVrGHGDk/CTXvOjWtJE34Ga9aW3cauNE/oAMQCRlY7zrJNMOsxK9andM2dXYEOeGheMsJabbl3sr8FA6MKHqhUXIRMLUxUVwNHvI/yePHYq0HR5CgOImVaiw+qyxd812RAzvA89RSwTfKGPC5w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Allow splice to undo the effects of MSG_MORE after prematurely ending a splice/sendfile due to getting an EOF condition (->splice_read() returned 0) after splice had called sendmsg() with MSG_MORE set when the user didn't set MSG_MORE. Suggested-by: Linus Torvalds Link: https://lore.kernel.org/r/CAHk-=wh=V579PDYvkpnTobCLGczbgxpMgGmmhqiTyE34Cpi5Gg@mail.gmail.com/ Signed-off-by: David Howells cc: Tom Herbert cc: Tom Herbert cc: Cong Wang cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - In kcm_splice_eof(), use skb_queue_empty_lockless(). net/kcm/kcmsock.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/net/kcm/kcmsock.c b/net/kcm/kcmsock.c index ba22af16b96d..7dee74430b59 100644 --- a/net/kcm/kcmsock.c +++ b/net/kcm/kcmsock.c @@ -968,6 +968,19 @@ static int kcm_sendmsg(struct socket *sock, struct msghdr *msg, size_t len) return err; } +static void kcm_splice_eof(struct socket *sock) +{ + struct sock *sk = sock->sk; + struct kcm_sock *kcm = kcm_sk(sk); + + if (skb_queue_empty_lockless(&sk->sk_write_queue)) + return; + + lock_sock(sk); + kcm_write_msgs(kcm); + release_sock(sk); +} + static ssize_t kcm_sendpage(struct socket *sock, struct page *page, int offset, size_t size, int flags) @@ -1773,6 +1786,7 @@ static const struct proto_ops kcm_dgram_ops = { .sendmsg = kcm_sendmsg, .recvmsg = kcm_recvmsg, .mmap = sock_no_mmap, + .splice_eof = kcm_splice_eof, .sendpage = kcm_sendpage, }; @@ -1794,6 +1808,7 @@ static const struct proto_ops kcm_seqpacket_ops = { .sendmsg = kcm_sendmsg, .recvmsg = kcm_recvmsg, .mmap = sock_no_mmap, + .splice_eof = kcm_splice_eof, .sendpage = kcm_sendpage, .splice_read = kcm_splice_read, }; From patchwork Wed Jun 7 18:19:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D669C7EE23 for ; Wed, 7 Jun 2023 18:20:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C505F900008; Wed, 7 Jun 2023 14:20:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id BD834900004; Wed, 7 Jun 2023 14:20:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A520F900008; Wed, 7 Jun 2023 14:20:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 915E5900004 for ; Wed, 7 Jun 2023 14:20:23 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 57960A03C4 for ; Wed, 7 Jun 2023 18:20:23 +0000 (UTC) X-FDA: 80876766726.07.EF96011 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf04.hostedemail.com (Postfix) with ESMTP id 7D70D4001A for ; Wed, 7 Jun 2023 18:20:21 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XmzZmhrH; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162021; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VfP0KSvbwDpdDK3r99Lo/bGf0PzU9/ETfdNk/1wpqNM=; b=CQ6msQEJRJ3AUb6aW5sIGKgbEUaX8gd/6ICjibAO4w503LYmHmMWxEcL2KQ1FV0c6RjmAq R6DY3o+5z8Nx+a6VoI4ifS2aDmvsjyvQOiv3Ebbmxy2c8dc8LO9VbHjWgE0Q6X041Orru9 Y48ho5DoU4wGHMfVUryIqAMqoVw6iww= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162021; a=rsa-sha256; cv=none; b=Ad5UNuXfksqd44GtfCLCdb7J0z/xuzb+IKUYAwhf5Yt/0mR5DGM5bddDVQ6AsfgvGiqJBK 93eD/O7wkw2TDulQ3M2Hz+0QTiAg5M7jTIURnJNd/yGQoRJkQko2OcyHcjrlEjsdC8pJqA Cx4LuSIaNAxY48JgCdiokPslfWXJls8= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=XmzZmhrH; spf=pass (imf04.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162020; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VfP0KSvbwDpdDK3r99Lo/bGf0PzU9/ETfdNk/1wpqNM=; b=XmzZmhrHBvC7TbWUbxI4pImqdBI0VWl8HktIUzdmKIgK9wjD1g5Odb0J1CdK3PY+K7PgJT pUkzaD9EsaLUYjq/5nyejZ8Yie5Dy+TslI2sH6qX9JypI50JQJcBT8PXHU3wzEOL//0KSG jVAQnIiZwIkcpauj6y2G+FY0raBk3ys= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-643-JvJ2kpHPNDC1LBuwFYWsFg-1; Wed, 07 Jun 2023 14:20:17 -0400 X-MC-Unique: JvJ2kpHPNDC1LBuwFYWsFg-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.rdu2.redhat.com [10.11.54.4]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 665F285A5BA; Wed, 7 Jun 2023 18:20:16 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 90EEE2026D49; Wed, 7 Jun 2023 18:20:13 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Christoph Hellwig , Al Viro , Jan Kara , Jeff Layton , David Hildenbrand , Christian Brauner , linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Subject: [PATCH net-next v6 10/14] splice, net: Fix SPLICE_F_MORE signalling in splice_direct_to_actor() Date: Wed, 7 Jun 2023 19:19:16 +0100 Message-ID: <20230607181920.2294972-11-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.4 X-Stat-Signature: qenj8noa5oemtirb7jmqcaxh6srd8ywi X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 7D70D4001A X-Rspam-User: X-HE-Tag: 1686162021-181403 X-HE-Meta: U2FsdGVkX185olYDOBQPR3s5cPYmO0ObRa98kLstCMYu9iaBuQ+Xd3nQJiSAOa+nKSNI9ETr7fVnOrcP9X7f4GvWwLSfU2rAFZVV65lwcbU6PJK8ZBKoclu2PetOUwWVtpJMa8N5PkglsdYSxB7l0gHQfAXU5R3lQQxSUI8YINebyuwNt1mBWWYT8CU/pImhw/L7Ap7tzbIsDRaTwbgSwLJnkAZdtvEHf1mCfmj7jNRZ6atKFifh1pkhJu8SCu877ryU4SDorR4LsrKk32Y2z7iaCusVL+cYwgNTuPWQjyK/YAaISjCjL1wwpheONBgKVMHOlrHcSmKen4N4cv2b2d1IXBcIqPCzMm+w8B8ITyMk8CZxhumbjtzZCNALQB9GZW47KdzrA58OMvgCR7ASi3EH+nkpJ8BXUPS/L9CZppQeRzzRBOEtRxGafaho2ghrsFyJoTfOicnsAM3UyEk5Z2u2NV7PYWsivA7zKE11p182V2MofqpyNgQuANwpyi1lzMhnFRugBgA9ntLKeacZXcYBC6GZgTbx8GXtRMT7aA2fxFVCVl75sDsWK+ozZX057Py3QQXNPglSf4g3/23XhWVkOIHdei1Nhaza6jH9sCgqUGJnUpQKzfXuOERkSY5GTshPe1V0YBESvG8ybb8DmrKP1YfiFfodEdeOUrhpYkVMnW/v+u1hEpPYZaxkV9dRAmPuiRphhv8ON64Cv4ejM0JXlZ6fGc65AngKT0hEifCHMWeJ4PEJ2SF4HTxeGBOPiGDHhp/adLHfP36WS90W+/oyu9UYy6nGCCyuigzgbuvNAJan73NvK0KN3bt8dOTRnz1DGymlUuqRuQmDhAwSC4Op5GVKtA4znQ/ixNTxh3WQp+iW1NVQLWEuUeVXmnDT+751R+dmnubA7pBdNRDBNrbCBpBLHybV76glZbl/ztNrysg1x8NHlMO5WfVKKwIe0kI4aTi2Vu/xbZWqZj6 0eIsEXmJ T4hBlweUdd4NUuwOfqHJ0pb2lA502UGQrzmgkq0iVgwq7JubEydJAzrKeLBQMIFjC2JX7YYbUPh4E+a40vyUW/ERA+uJDQ9EjqZIfFJ6tR6REiblXFtqXna5gm5MXJTLb+QLoMv0a5qx+Bzjey0r9AYNl+hMMNOj1rTB48Utc2ye2XCKrSpfokwjiymoOiX6VxyqescsOdjTlSnCGoUxkhUdcVnDPig7oiBlNADdDM1rdvBfWRVo0knCol6mG072Jj7MSQT47NQEN1kiCD3vfvddafF8tXP91+bulyGa6FQJuQRwwl293v6cEdAze+oUmSdY326eE1EVqRnDJ+T28Cl0HbwSgZGm6Nu1SoAp4wXjpwRvejXjJDa4qP4+QYOnH/XRKOfPrJChGFu3mIngNIV3HToIWIauvSjlwJ612nyYnhu5VR9lno/JUdJnSy8PwV5asrYHMqGZkZgBiya5V1+jaisZV1GO/PMTRdaWhtRw32QyUH0P2eLTURXMwAO31pP29gmleR2u8VGK1+/KbKIEcn/VAU5/pEnevm5NVPTagOZehIyd1cGqnhltW9ib/DQDEPVm/59C69FGPjoHVYu4ixAAFTlREWRFALm27K5/biZ+2UTrNxNXx6XEpQxQFIJ67+gYmMfD9fyKAyAxVSJfmMlpD919Az1NK4l1Oeud1FmciEh6167FO23grfgzVUWsxSTG62/YcJM1zcFVqmBEksaY/Q/n0qko4B5PS7K9sh3shz2u9wqBZ3qDROr33m8aMF+L8WPGHsekqKS/MTIeTTgTmQz81gENQsJYU86KTRVj+1BAYDdxhIxqo9tKUVBeTa3DCT5CbLiuOk4owUEdYTbshzjL4xieoPxmlFYmKIS4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: splice_direct_to_actor() doesn't manage SPLICE_F_MORE correctly[1] - and, as a result, it incorrectly signals/fails to signal MSG_MORE when splicing to a socket. The problem I'm seeing happens when a short splice occurs because we got a short read due to hitting the EOF on a file: as the length read (read_len) is less than the remaining size to be spliced (len), SPLICE_F_MORE (and thus MSG_MORE) is set. The issue is that, for the moment, we have no way to know *why* the short read occurred and so can't make a good decision on whether we *should* keep MSG_MORE set. MSG_SENDPAGE_NOTLAST was added to work around this, but that is also set incorrectly under some circumstances - for example if a short read fills a single pipe_buffer, but the next read would return more (seqfile can do this). This was observed with the multi_chunk_sendfile tests in the tls kselftest program. Some of those tests would hang and time out when the last chunk of file was less than the sendfile request size: build/kselftest/net/tls -r tls.12_aes_gcm.multi_chunk_sendfile This has been observed before[2] and worked around in AF_TLS[3]. Fix this by making splice_direct_to_actor() always signal SPLICE_F_MORE if we haven't yet hit the requested operation size. SPLICE_F_MORE remains signalled if the user passed it in to splice() but otherwise gets cleared when we've read sufficient data to fulfill the request. If, however, we get a premature EOF from ->splice_read(), have sent at least one byte and SPLICE_F_MORE was not set by the caller, ->splice_eof() will be invoked. Signed-off-by: David Howells cc: Linus Torvalds cc: Jakub Kicinski cc: Jens Axboe cc: Christoph Hellwig cc: Al Viro cc: Matthew Wilcox cc: Jan Kara cc: Jeff Layton cc: David Hildenbrand cc: Christian Brauner cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: linux-fsdevel@vger.kernel.org cc: linux-block@vger.kernel.org cc: linux-mm@kvack.org cc: netdev@vger.kernel.org Link: https://lore.kernel.org/r/499791.1685485603@warthog.procyon.org.uk/ [1] Link: https://lore.kernel.org/r/1591392508-14592-1-git-send-email-pooja.trivedi@stackpath.com/ [2] Link: https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=d452d48b9f8b1a7f8152d33ef52cfd7fe1735b0a [3] --- Notes: ver #4) - Use ->splice_eof() to signal a premature EOF to the splice output. fs/splice.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/fs/splice.c b/fs/splice.c index 67dbd85db207..67ddaac1f5c5 100644 --- a/fs/splice.c +++ b/fs/splice.c @@ -1063,13 +1063,17 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, */ bytes = 0; len = sd->total_len; + + /* Don't block on output, we have to drain the direct pipe. */ flags = sd->flags; + sd->flags &= ~SPLICE_F_NONBLOCK; /* - * Don't block on output, we have to drain the direct pipe. + * We signal MORE until we've read sufficient data to fulfill the + * request and we keep signalling it if the caller set it. */ - sd->flags &= ~SPLICE_F_NONBLOCK; more = sd->flags & SPLICE_F_MORE; + sd->flags |= SPLICE_F_MORE; WARN_ON_ONCE(!pipe_empty(pipe->head, pipe->tail)); @@ -1085,14 +1089,12 @@ ssize_t splice_direct_to_actor(struct file *in, struct splice_desc *sd, sd->total_len = read_len; /* - * If more data is pending, set SPLICE_F_MORE - * If this is the last data and SPLICE_F_MORE was not set - * initially, clears it. + * If we now have sufficient data to fulfill the request then + * we clear SPLICE_F_MORE if it was not set initially. */ - if (read_len < len) - sd->flags |= SPLICE_F_MORE; - else if (!more) + if (read_len >= len && !more) sd->flags &= ~SPLICE_F_MORE; + /* * NOTE: nonblocking mode only applies to the input. We * must not do the output in nonblocking mode as then we From patchwork Wed Jun 7 18:19:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5DF5EC7EE43 for ; Wed, 7 Jun 2023 18:20:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F0E6B900009; Wed, 7 Jun 2023 14:20:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E9774900004; Wed, 7 Jun 2023 14:20:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D11D2900009; Wed, 7 Jun 2023 14:20:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id BE258900004 for ; Wed, 7 Jun 2023 14:20:27 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 7FEF7404C0 for ; Wed, 7 Jun 2023 18:20:27 +0000 (UTC) X-FDA: 80876766894.22.0A51994 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id AC0BF20023 for ; Wed, 7 Jun 2023 18:20:25 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W7WcGW+y; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162025; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6q5xbY97TlySjcT8Avdxpu9hvkIkBKUNCyWrMvUV8f8=; b=fnxSCk0Wnu8KKClB5Y9qIhYxZC/dagxLv6cBTi0hCsIl0hCkRUak0Lqe2l/x46eDxaLx8l Sf+Sby1N3jBf3kK/+rEE69HvpDf6G7l2Nze4pgUTQuWQgspshfOLIBtU8hHIb6OZisPu9G rswE6aHPZ7JTLQoLwQcMmyR9dt3oriY= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W7WcGW+y; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf03.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162025; a=rsa-sha256; cv=none; b=7vV7dtK4JkJYasvUaNo/aaHJHQJPM4Bz1TYm6u4gx1hvNcGNdw8lpkg4dNq9oMtipOFG7I RXeSG7FOyTJBDsKSs+5zv0Gs7QX+w/dHVa7O2r1Ub2jy5ARohaVijX++EkX77aw4pvNNoV 6fDP1wPHRU5/Q3OAJK4Hl5jzSBL205U= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6q5xbY97TlySjcT8Avdxpu9hvkIkBKUNCyWrMvUV8f8=; b=W7WcGW+yfoSgtS7F748lozL7QxYzZVo9/PdMFwfbCIhCrXwBgTwTot6m0Bh4se55pF5th5 BYzVfvyEmU8QTDKuhr8dtVC8elJFMOUKyZ/M1wmkwF1idF8Gu+R7V4ogEOkYMxg9Bt3/cT CGfCp0BD9PCLw/WG2p8bFP5m615QR5w= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-631-Eq-9mGkaPwauwtyewFdCog-1; Wed, 07 Jun 2023 14:20:19 -0400 X-MC-Unique: Eq-9mGkaPwauwtyewFdCog-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 25C4980027F; Wed, 7 Jun 2023 18:20:19 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0AC5C2166B25; Wed, 7 Jun 2023 18:20:16 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 11/14] tls/sw: Support MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 19:19:17 +0100 Message-ID: <20230607181920.2294972-12-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: AC0BF20023 X-Stat-Signature: ietegn8faqd5ep9p3kwejizig4nhb116 X-HE-Tag: 1686162025-171744 X-HE-Meta: U2FsdGVkX18mpWnQsBsK98uBregorpYyGJtScASeVLIiLZSFkEzkVqRCtJk+aYCrSXFsCs2E4kQERrPKE824YFsvMMf4hv9eB1zAFSSAMr04fzn8yzaiqgGEEKiKJsAm0qu8M9Xzit8hwtY7tDs7hbM5LkOiFSFHEORTfbuIl6TF9BRozi4cW30oqqZLCeUTlIiA3KpY/HjQLv5HWUzQ4K00vBqh8Gj2pgg2kEgEzNNkDZuwW4Gl/0Iu3i/GYddtMyiwbBTKOhwTM5osZy09V+yRLEmVhkX+7yuQPJlwGoB59xVfKvO8nWsgjs0Fyi1/S9Fb729hRSXgsO0Rf16DRHKzolZuWGbcy1PGTYGBKudKvWgx23Cb8nZFnT/ZuhzIjXqgnkZZUoZjzU9PzN4PWGNBBiq1kLedGXQoLEwPm5mFzOt4LppXNi6O2O0m2Gr1Jmvi1mLDbyUvcExQiBeO52UA35pp058MMJGcd0KRQ1TO1x2igvOo9PKVoScGqw00g/cNUl9F+E1RmXwUWvB//oLZPxHcCZ4dpvEGs5Y/5nmHD82TMxMHgABiRurLqWvIn1bsBNOeKNWm+Pon4bkfOieQ6Kk+t0WTY7MAvJxSjC88PO+hQRufKBSYUZbsk/gwoDsDdRGVEiinr5GmHi3BiNtkVdaAwTnY2iCtkuL3QbX+0F1xBG27MqRQ3c9It9kRaBRnV1r93YD/VmvS/6t8vQeheglQ2w4ZV6FxlNZ6nE/HLbEX8U3IoEmXqNLkQmDXTXSbY4Jt7gycoZZmeAsEWVmUu2kh3q6CNSZmeykkW9gQTBB/Q4MMZCopILFmfLiib6XW9Koe/eu8ecWtOCsYIyVUnf9crSSONrNBtqfusiAtMQ33qiAteVftezxCwoDthznwUVT+H3zmrMZhkcvt8ktMd2x8bc7W7uvGNvLijlasdQmQ2DVce2HU6SeEsaySCR79KJkYhFT5Sm7zIZM MZBDI4BJ jkyFH9VswS6/dopCAWJbKmx2B7u1FRsLWuB+3+rpemoLlx2a/JOrGIZjqHMhWWjixDTInhulcfrJXMyZgsMZlyF9kA6V/H/TUHSRQKjKFlfDmxyYJWVBSAmlqET6ZJmbF7YMOBtp/hvGHWWbInhuUVl/9yxN8FdQYBgd5tsPQsUqfdg9DgSeocuIbVLs2YhO+BqDDI1xOiAK6LoQVtsqZu3kvWlRWily+9xskNkV2fis4A9+uFduN/3KwZx92YPbqkZNodTWicuLhhHursCvA6C83ax6Zpx5m0WksENVXZ+sZzvnOfW3E2C0a4NbuNh4ZrHrKhCU5O4Wkp0GXZVVDzRE1H5OpIGJ93nkhKZ83V4h4YUDdoKmThslVBiMHE9laZGKKjoYUYm/pBDrjNb42c9Ac9cUGyJyqMBBd4V2vEwoj5I1IKyrC6RDBP+ppC8Tdo4p9x0blN0XetmKcjkAHzmTMN0RKZcDE6LrEH5K8tCSRSE7ixVyh8KDazq8VStDXUL2Mdpg/C8cmQzbIn0B48TAXJXRDZbLSTqQ6bt7BdloVBknx4zb+jzYH3g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Jakub Kicinski cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- Notes: ver #6) - In tls_sw_sendmsg_splice(), remove unused put_page. - In tls_sw_sendmsg(), don't set pending_open_record_frags twice. ver #2) - "rls_" should be "tls_". net/tls/tls_sw.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index a2fb0256ff1c..2d2bb933d2a6 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -931,6 +931,35 @@ static int tls_sw_push_pending_record(struct sock *sk, int flags) &copied, flags); } +static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, + struct sk_msg *msg_pl, size_t try_to_copy, + ssize_t *copied) +{ + struct page *page = NULL, **pages = &page; + + do { + ssize_t part; + size_t off; + + part = iov_iter_extract_pages(&msg->msg_iter, &pages, + try_to_copy, 1, 0, &off); + if (part <= 0) + return part ?: -EIO; + + if (WARN_ON_ONCE(!sendpage_ok(page))) { + iov_iter_revert(&msg->msg_iter, part); + return -EIO; + } + + sk_msg_page_add(msg_pl, page, part, off); + sk_mem_charge(sk, part); + *copied += part; + try_to_copy -= part; + } while (try_to_copy && !sk_msg_full(msg_pl)); + + return 0; +} + int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); @@ -1020,6 +1049,17 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) full_record = true; } + if (try_to_copy && (msg->msg_flags & MSG_SPLICE_PAGES)) { + ret = tls_sw_sendmsg_splice(sk, msg, msg_pl, + try_to_copy, &copied); + if (ret < 0) + goto send_end; + tls_ctx->pending_open_record_frags = true; + if (full_record || eor || sk_msg_full(msg_pl)) + goto copied; + continue; + } + if (!is_kvec && (full_record || eor) && !async_capable) { u32 first = msg_pl->sg.end; @@ -1084,6 +1124,7 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) */ tls_ctx->pending_open_record_frags = true; copied += try_to_copy; +copied: if (full_record || eor) { ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, record_type, &copied, From patchwork Wed Jun 7 18:19:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271097 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FBBDC83005 for ; Wed, 7 Jun 2023 18:20:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D69690000A; Wed, 7 Jun 2023 14:20:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 45EAD900004; Wed, 7 Jun 2023 14:20:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2D92D90000A; Wed, 7 Jun 2023 14:20:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 1BDAC900004 for ; Wed, 7 Jun 2023 14:20:28 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id CB6A9402A4 for ; Wed, 7 Jun 2023 18:20:27 +0000 (UTC) X-FDA: 80876766894.21.AB534FB Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf13.hostedemail.com (Postfix) with ESMTP id 16A5A2001E for ; Wed, 7 Jun 2023 18:20:25 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OLTMbWKx; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162026; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xIvq+KmHXpeRlpbNnQkpDzUJdGjGno6iPOPrQ1kfx1M=; b=3aNKOxqrvPW6Cshv/iN9WLyVr+B8tFmoSM4md4DWgGIJcA/b3/WgE9E9QZNrPIiDy1ib/z doGWAgj9rj0mjPdb3qj3YssBWG9lZSiYTcS1t0hBNv+gxvpiB8o1gWq6f8VFp9sTgukEcF uHBEv1b6ENBCAW4QgBZw7ZwwE5LV+xQ= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=OLTMbWKx; spf=pass (imf13.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162026; a=rsa-sha256; cv=none; b=Fa0+bYpvui8oC+4Vh6/A1kMHJ/GzagulooCfASWoTxruCi4KCDZe5AXGctbq94u1ZlisIx rOIEhwXMi3ZPs0/4jqMaYSX2ftXWlZ9H1x8VKGwXaUWl2PGTe7JwQ6AQXBhMGcAlgV4HvH D2UPVFzSUOQEz5pRkstTjXno1GpaRPU= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162025; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xIvq+KmHXpeRlpbNnQkpDzUJdGjGno6iPOPrQ1kfx1M=; b=OLTMbWKxA9ZF+jHlcPYStI5NBt2HUhLCwJE9Z1y3lbGraJi6whN3xEqQfr/1Bw+c/aiQMW ox0ccIs6jprSpCIfI76E+FqctoYEwXwMCFc2DH2Gs2Zntv0uaexPHzY8uia3sFNo+ujhWc V28eApLQfwGTaOrC/oLynL7693bBrSo= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-173-fVyYLsfrNGqu-sEnpHL0Hw-1; Wed, 07 Jun 2023 14:20:22 -0400 X-MC-Unique: fVyYLsfrNGqu-sEnpHL0Hw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E632F8015D8; Wed, 7 Jun 2023 18:20:21 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id D748D2166B25; Wed, 7 Jun 2023 18:20:19 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH net-next v6 12/14] tls/sw: Convert tls_sw_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 19:19:18 +0100 Message-ID: <20230607181920.2294972-13-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Rspamd-Queue-Id: 16A5A2001E X-Rspam-User: X-Stat-Signature: sa691em3teysahnfe9tii8ua49b6ti8t X-Rspamd-Server: rspam01 X-HE-Tag: 1686162025-418022 X-HE-Meta: U2FsdGVkX198L2h0f5jWymq7Az1T5mi9j2UfN4zDy33Dk5B/wDjXkI4L5Ctb0YbB2xawPFTt3ARkFCC0t2Q/VcyMSBPeP7QUEQ1qItQ9YjqTxwwVNSIhqyVD5D5T7ZbjjuBJBz126AlUBr87J1DvOhSSTL9+/J84lzZKLGIG0WfubRxayHFII4n5rY2lQmmrgxMHGVZBy1UOHyUeJZPLhIXKsDLJzh2QHVrlOtFXB2Vkcd5UQb8eiHLtz5rqWboKkkT+lemWmcZA017G7mhHNyu4LLRXzd79l0N/KNliHJW8vbqgdFgdTuNekSS/99Zklf3EnvzATf4Nwk7mdyVEIoUcVokVmf3Vr/gYNEFaw/HjF07NMxW/TkmiF95IonpohMzcmyq0Zw1PjzEWKzfVHTKkDZRmh+1yuZVaAwvwfxNXvDN2b+iNKzmJ0dlUMbCOF4pznF17JwO1WqEle6IPZuUgOqji96XwF96+fJvDym7+BAIkYSIAwU2LHgIKuykp5R0c98fyTm4m/mWc+fAQUGacEBmXlLUUzY3RVQpnns4rWcQ5CgDSdCxtywqu7oCtogbhV+MAb39KSXQbYgvDt8H5lGr+jg3LOHXfK7eUgPu3pBrVa+MM0ruK4/vXARYu6OVctsqPU6vre/g3b4FrbljUxyQ5+JTzG+gcWTAPfA0BEi8plIkLhykAxofAnBb7Z6bP//dEE+gjhb2M+8JXGTsgLCTxADuFkQR1MPNfDSY8srGnLGUiyqcB32uxm6az5JeLjUCL4DPiJ8Rv3ZZ8WL6DcTqdpp9qrLvAdU7viG7OIxOLKF2fuJDcw9g+VXYB+hGekclmWMiJq8Y3UendXSeRFgwcka91GoEZCwcvuKqJk4/HpNqT8nxwJAsmZHDH/B90b2Osw7aqUTRNM4UCKWNqlNjgJZ26O032ljdZ3G+4rbTpAzL2Hnrddof96nTzXFrOb4XR61OnFPeKXKI Zf5ZHVM6 AJhnuIOGXGxyibF66YDS1giMP6aVkVt0DiyXG0VQwJTR3SmF/jwR+a/P24OuDf/ibHGUMe3NXQHeEiyN420MCoQVWuYeQqzZzRWxU39z6IAa/Fkfdk9FytvhBq2Zd4KGQ45a5sVImTTGOpf1C57Q0l/CSOt+7G3J2lBwSCYvGepT/02uLBuy7bCQpa+qUkMyZhoVecVfhuKbETEEyF3z8HeyrZmpM1zj0+mkdIP3s7QZdiEm6XaVo0pQzH5LLrNJQju7GdzTN6TfA5du2OGeZwk8zVsOkLtE7nz3XvOM1OsoTrcnbHmwOUfEAMzgTXGH99SzTaD3jCnOs//2IKk6J87TeeDCDvpa56FPAmT88meAnNiH8tTYeH8rc6ZfM9gd7bmKXAXP1ZUYz/+/6R3ZIJ2UYcC0vWHYgvhFAyxvRm+D8xRaXaa8yYaFKhXwng0Us4EnxPCCoINUlv2FvorHpEiEU8DZbA/cxOSYrdndAgN+20M1rMm6fjiFmJOp5pVJxzCSP0J3IWgU+HokOZRst9OkwprQAYOiKOM3zu8hO5u55PjlkA/f4CCFAZw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_sw_sendpage() and tls_sw_sendpage_locked() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. [!] Note that tls_sw_sendpage_locked() appears to have the wrong locking upstream. I think the caller will only hold the socket lock, but it should hold tls_ctx->tx_lock too. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org cc: bpf@vger.kernel.org --- net/tls/tls_sw.c | 173 ++++++++++------------------------------------- 1 file changed, 35 insertions(+), 138 deletions(-) diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index 2d2bb933d2a6..319f61590d2c 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -960,7 +960,8 @@ static int tls_sw_sendmsg_splice(struct sock *sk, struct msghdr *msg, return 0; } -int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) +static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, + size_t size) { long timeo = sock_sndtimeo(sk, msg->msg_flags & MSG_DONTWAIT); struct tls_context *tls_ctx = tls_get_ctx(sk); @@ -983,15 +984,6 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) int ret = 0; int pending; - if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | - MSG_CMSG_COMPAT | MSG_SPLICE_PAGES)) - return -EOPNOTSUPP; - - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - if (unlikely(msg->msg_controllen)) { ret = tls_process_cmsg(sk, msg, &record_type); if (ret) { @@ -1192,10 +1184,27 @@ int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) send_end: ret = sk_stream_error(sk, msg->msg_flags, ret); + return copied > 0 ? copied : ret; +} +int tls_sw_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) +{ + struct tls_context *tls_ctx = tls_get_ctx(sk); + int ret; + + if (msg->msg_flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | + MSG_CMSG_COMPAT | MSG_SPLICE_PAGES | + MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) + return -EOPNOTSUPP; + + ret = mutex_lock_interruptible(&tls_ctx->tx_lock); + if (ret) + return ret; + lock_sock(sk); + ret = tls_sw_sendmsg_locked(sk, msg, size); release_sock(sk); mutex_unlock(&tls_ctx->tx_lock); - return copied > 0 ? copied : ret; + return ret; } /* @@ -1272,151 +1281,39 @@ void tls_sw_splice_eof(struct socket *sock) mutex_unlock(&tls_ctx->tx_lock); } -static int tls_sw_do_sendpage(struct sock *sk, struct page *page, - int offset, size_t size, int flags) -{ - long timeo = sock_sndtimeo(sk, flags & MSG_DONTWAIT); - struct tls_context *tls_ctx = tls_get_ctx(sk); - struct tls_sw_context_tx *ctx = tls_sw_ctx_tx(tls_ctx); - struct tls_prot_info *prot = &tls_ctx->prot_info; - unsigned char record_type = TLS_RECORD_TYPE_DATA; - struct sk_msg *msg_pl; - struct tls_rec *rec; - int num_async = 0; - ssize_t copied = 0; - bool full_record; - int record_room; - int ret = 0; - bool eor; - - eor = !(flags & MSG_SENDPAGE_NOTLAST); - sk_clear_bit(SOCKWQ_ASYNC_NOSPACE, sk); - - /* Call the sk_stream functions to manage the sndbuf mem. */ - while (size > 0) { - size_t copy, required_size; - - if (sk->sk_err) { - ret = -sk->sk_err; - goto sendpage_end; - } - - if (ctx->open_rec) - rec = ctx->open_rec; - else - rec = ctx->open_rec = tls_get_rec(sk); - if (!rec) { - ret = -ENOMEM; - goto sendpage_end; - } - - msg_pl = &rec->msg_plaintext; - - full_record = false; - record_room = TLS_MAX_PAYLOAD_SIZE - msg_pl->sg.size; - copy = size; - if (copy >= record_room) { - copy = record_room; - full_record = true; - } - - required_size = msg_pl->sg.size + copy + prot->overhead_size; - - if (!sk_stream_memory_free(sk)) - goto wait_for_sndbuf; -alloc_payload: - ret = tls_alloc_encrypted_msg(sk, required_size); - if (ret) { - if (ret != -ENOSPC) - goto wait_for_memory; - - /* Adjust copy according to the amount that was - * actually allocated. The difference is due - * to max sg elements limit - */ - copy -= required_size - msg_pl->sg.size; - full_record = true; - } - - sk_msg_page_add(msg_pl, page, copy, offset); - sk_mem_charge(sk, copy); - - offset += copy; - size -= copy; - copied += copy; - - tls_ctx->pending_open_record_frags = true; - if (full_record || eor || sk_msg_full(msg_pl)) { - ret = bpf_exec_tx_verdict(msg_pl, sk, full_record, - record_type, &copied, flags); - if (ret) { - if (ret == -EINPROGRESS) - num_async++; - else if (ret == -ENOMEM) - goto wait_for_memory; - else if (ret != -EAGAIN) { - if (ret == -ENOSPC) - ret = 0; - goto sendpage_end; - } - } - } - continue; -wait_for_sndbuf: - set_bit(SOCK_NOSPACE, &sk->sk_socket->flags); -wait_for_memory: - ret = sk_stream_wait_memory(sk, &timeo); - if (ret) { - if (ctx->open_rec) - tls_trim_both_msgs(sk, msg_pl->sg.size); - goto sendpage_end; - } - - if (ctx->open_rec) - goto alloc_payload; - } - - if (num_async) { - /* Transmit if any encryptions have completed */ - if (test_and_clear_bit(BIT_TX_SCHEDULED, &ctx->tx_bitmask)) { - cancel_delayed_work(&ctx->tx_work.work); - tls_tx_records(sk, flags); - } - } -sendpage_end: - ret = sk_stream_error(sk, flags, ret); - return copied > 0 ? copied : ret; -} - int tls_sw_sendpage_locked(struct sock *sk, struct page *page, int offset, size_t size, int flags) { + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; + if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY | MSG_NO_SHARED_FRAGS)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - return tls_sw_do_sendpage(sk, page, offset, size, flags); + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg_locked(sk, &msg, size); } int tls_sw_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - int ret; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & ~(MSG_MORE | MSG_DONTWAIT | MSG_NOSIGNAL | MSG_SENDPAGE_NOTLAST | MSG_SENDPAGE_NOPOLICY)) return -EOPNOTSUPP; + if (flags & MSG_SENDPAGE_NOTLAST) + msg.msg_flags |= MSG_MORE; - ret = mutex_lock_interruptible(&tls_ctx->tx_lock); - if (ret) - return ret; - lock_sock(sk); - ret = tls_sw_do_sendpage(sk, page, offset, size, flags); - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return ret; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_sw_sendmsg(sk, &msg, size); } static int From patchwork Wed Jun 7 18:19:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271099 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 894A5C7EE25 for ; Wed, 7 Jun 2023 18:20:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2B248900004; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 23BA7900003; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0DC95900004; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id ED721900003 for ; Wed, 7 Jun 2023 14:20:36 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id B5EED140492 for ; Wed, 7 Jun 2023 18:20:36 +0000 (UTC) X-FDA: 80876767272.30.3C50B60 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf14.hostedemail.com (Postfix) with ESMTP id A7655100027 for ; Wed, 7 Jun 2023 18:20:34 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="B/4/wTGG"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162034; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4vaOjJ4V8cBQxw4FBR0piHsYWVs4rIzqgZ7rDnK+p44=; b=GYC9urPZKEbVT/+UiHoCRHIdUs9IX8yXi6WTVO/mteEEbzKkcrhXHYqt/Ky2IIJJMMuxg6 OOqsjBjoF8vJ+i8ish9Vxp/BIsBD64OCwbm6sVaJ7b+NicV+ICCVwri80EWiysZxhLynAZ c++eQjpgu6560D1ladBlXCyN9k/uGP0= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b="B/4/wTGG"; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf14.hostedemail.com: domain of dhowells@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162034; a=rsa-sha256; cv=none; b=iO1AbthKsZP+agEG54NEj1htMETPJgfRXvLZH+jCKYD1uxYUAHryAeA/T6psh5B3LCgldb jOECw47aaZAhSfNwdeMB9hW/+54O6RYr9Q+7bNt74Vv508LJQ1e2jARG35liZ2p7LaKZNa 2kcrNNnfPwZ3QGZN9B2b5T0sk2D6Wlg= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4vaOjJ4V8cBQxw4FBR0piHsYWVs4rIzqgZ7rDnK+p44=; b=B/4/wTGG76x1SaJ72/IyMbos2lpiA89VOodyKBRR/SDWsypNAg24qph1bZENoLw6p3gRWt cB7T1RR82GOW6h7CQIUiy/6rp+FmGqxPPDzDZW9YItYU5oohRAQLxcP/eMMRDx1Op4ppQ4 LN+FRazBMvysXUi1+JMYLqVVTG4x0VA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-354-LRCuNYAMOOq4Tm5Pxw5q1A-1; Wed, 07 Jun 2023 14:20:25 -0400 X-MC-Unique: LRCuNYAMOOq4Tm5Pxw5q1A-1 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5340480027F; Wed, 7 Jun 2023 18:20:24 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87D23403367; Wed, 7 Jun 2023 18:20:22 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 13/14] tls/device: Support MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 19:19:19 +0100 Message-ID: <20230607181920.2294972-14-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.10 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: A7655100027 X-Stat-Signature: ubqhb1bjhxezdqgsffgo79m1grjkqzo6 X-HE-Tag: 1686162034-859303 X-HE-Meta: U2FsdGVkX1/Zf4++sjojUJbVl05ZBuYm47K9h/BMJp8t9sRuRQLVWq0J0Fzfv56ei9VUOnZMI/BdDnvVq0DGosxNmeIhKmOz7/hbf0UcprTvm2V2lbcNich1h6PHs3oAPihcqMwXfUcnfjr0bpuE1JI/yVXHCVtkbxiRyv/Sth+apc+4Mr5Ey9sc5XWrAD4/cIRbJEGJ9OcVnOKX9JrcupFwLWHVlrv0MEQpiQruFYh/CLcOvFR/lylQjdbDMoNFIFEZLk+CZAH5ALEEtkVpNAPMq3KwVoFhqpt9qZv3/K0HWGDX5yj5Cu4p/mApWGTKXImFSOl4ok65Ma78uDLPi+/lSc2d558JK+XcLjOWa3VH7J/vIB0Lid8uD8hNvUqKmi2pYivL7wioav6Le6mRNRaU6I5Bjvyi9RfvrRj+k3IXg5/O3v+i9QEnTo6J+J+iM+2yp6gcvgU9oT85TJhiQxqKW4zpa5IjDP1iNLyQBbb9RvNNmC+i/p53zwXN4FYSVu1NNZhez1WKM2FPmzOXKZnWdvjpXYfVku8loc8rDkI4gH4WaqbScE9+Ja23t+rvogjnBc2Vj3iDQDZLscW2VdOjlbJFfhZYxLJL/W1UslFX1yB8tkHFseeiobWWOw6quIXfnFKv/w5f1ggw7GMXhjvSnRZUgAXesN0Q8jLX+X+mvy2fGYi5IMkL9J7k18aEirIAN3iQWIToLEAhtE9L6oGv6WLQR3kjG3Re4V6BQ35W5rG14oFn+MiRx+UZV4tl2ztBnNSmHbKgOxU2dJTUPNd2cJSnNteWJszJz/FlHtN0yCwiwAVlPlD0VqeTuisDrpoNCWOuWRzkOBiQvfnZS1hfF0cjNTLaF25Nx/MW96JYeCb1G0unINEz0Hy+AEPsLaeiKUm3J2+NCvlUf3upNrP1TUMDAslBQ1fP8JMHinKMFE1HIxmvQ8KOTEMqkYyZCsIQgvLPrHLP/uMR/ih Fiu1do7E b9tO/B4NuP7VfGin5OUFz7E8kS9rNCj7b9yOX6Wfc5USINhtSTzgu9C14FT21Rqkl7CMipN7SJRwEbEzhZs0nYOfDVn8XsilsUxlf/BUtVVnRC6jF6dm26GLV874duWIS1xLoPh0qbEgAotv4PAaoKKR34i8ZAhJYcPFvt2gCC1rzlVJArJy1DwD+dRNDySt3TlI0aia9qmIGvREmZ/VAZsKRz+cHWTLFtia0fu0h9OiF34OoAUL87BHBdrr2Fd6HMPbXEv59hhVn2WWCat4z1Cbg6c2SGVMG76v1RpbLRLh02W0Ne8uaUxyyGJXVZEJxcbYibyPEtUXoGxCCb7wdpp4oZr6Z9Ed2zGvDvFEZFiYW8qq5vEFKm37kB7kvI5JST6IwJ4wrlXMYTvur7qxgBUhS69A2lED4esIU8uz2Vi4pNqFZ5b4YIh8BZM+mV+3LwoA10Ma/uqhnoBVOKvf9OxO/EY4nP37qdhcZPETgCxJmnEGuHO2iXQrUlV1D87tlo+7Ga20RWx8NLvCUrodzlFDfeZwz7AQRbuPbBa7jNZwPxo+HDnRSt9pWWA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Make TLS's device sendmsg() support MSG_SPLICE_PAGES. This causes pages to be spliced from the source iterator if possible. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells Reviewed-by: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 439be833dcf9..bb3bb523544e 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -509,6 +509,29 @@ static int tls_push_data(struct sock *sk, tls_append_frag(record, &zc_pfrag, copy); iter_offset.offset += copy; + } else if (copy && (flags & MSG_SPLICE_PAGES)) { + struct page_frag zc_pfrag; + struct page **pages = &zc_pfrag.page; + size_t off; + + rc = iov_iter_extract_pages(iter_offset.msg_iter, + &pages, copy, 1, 0, &off); + if (rc <= 0) { + if (rc == 0) + rc = -EIO; + goto handle_error; + } + copy = rc; + + if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { + iov_iter_revert(iter_offset.msg_iter, copy); + rc = -EIO; + goto handle_error; + } + + zc_pfrag.offset = off; + zc_pfrag.size = copy; + tls_append_frag(record, &zc_pfrag, copy); } else if (copy) { copy = min_t(size_t, copy, pfrag->size - pfrag->offset); @@ -572,6 +595,9 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) union tls_iter_offset iter; int rc; + if (!tls_ctx->zerocopy_sendfile) + msg->msg_flags &= ~MSG_SPLICE_PAGES; + mutex_lock(&tls_ctx->tx_lock); lock_sock(sk); From patchwork Wed Jun 7 18:19:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Howells X-Patchwork-Id: 13271100 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0EEC7C7EE2E for ; Wed, 7 Jun 2023 18:20:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F61090000C; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 87CF0900003; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F5FA90000C; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 5B09D900003 for ; Wed, 7 Jun 2023 14:20:37 -0400 (EDT) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 2B465AF173 for ; Wed, 7 Jun 2023 18:20:37 +0000 (UTC) X-FDA: 80876767314.22.2E8C4F2 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf20.hostedemail.com (Postfix) with ESMTP id 58E3A1C001F for ; Wed, 7 Jun 2023 18:20:35 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IFdEfuVh; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1686162035; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yPehJ2gStScOqa+9lHtmNzJyOl35lJFQAV7b2pxwHvQ=; b=6J18t/Q9xAMAKUaxr4g6xpYAUWrCM2EnsLhLAl3nHYXUEjZE/l7I+ySbETlxqCNZFQXq6A nhUo7nfqV9sJrZAFoU9chziJHggpHlQVUu3FvPEbhlF+q6yipEiC21XCK0QzhnAIAszxJZ UTMcj5ngpbzYx9bB8ku3V2KXq+VZUWw= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=IFdEfuVh; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of dhowells@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=dhowells@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1686162035; a=rsa-sha256; cv=none; b=LVqUvAuprKXENj/mfbKSR9eLrMVke02SVuBwGJE39FPRdqo0+7mxdA+e3XWtdn7GSIdIYA LXBhzD1BmrUUq0HGAuangNtPJ12CggkGuZEl3Ypoy8lyL08ZYIDSDHb5IjGXLYtHIfXZSp 1wwIj0EB8f0PBD1figYqLh0fSXvNdY0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1686162034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yPehJ2gStScOqa+9lHtmNzJyOl35lJFQAV7b2pxwHvQ=; b=IFdEfuVh0yAvGxtX5huHpXPlO6lCpYSFT7asv0773oLp2T0bSnUYyaUmjf03vlFh/EsXc6 LMyjjhUsil2Z9HSomg4z62HW6qfoa1cu1mrJYSCY5QtJyrqIR+s7HOYUvRmoY4YwL8IhRQ jrKL4mhk01VtqcOk1cMP9rYnMvmE6l0= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-127-A_VwRT9fOeylZO0u2NBFfw-1; Wed, 07 Jun 2023 14:20:30 -0400 X-MC-Unique: A_VwRT9fOeylZO0u2NBFfw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id A7D96185A791; Wed, 7 Jun 2023 18:20:27 +0000 (UTC) Received: from warthog.procyon.org.uk (unknown [10.42.28.182]) by smtp.corp.redhat.com (Postfix) with ESMTP id 146C340D1B66; Wed, 7 Jun 2023 18:20:24 +0000 (UTC) From: David Howells To: netdev@vger.kernel.org, Linus Torvalds Cc: David Howells , Chuck Lever , Boris Pismenny , John Fastabend , Jakub Kicinski , "David S. Miller" , Eric Dumazet , Paolo Abeni , Willem de Bruijn , David Ahern , Matthew Wilcox , Jens Axboe , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v6 14/14] tls/device: Convert tls_device_sendpage() to use MSG_SPLICE_PAGES Date: Wed, 7 Jun 2023 19:19:20 +0100 Message-ID: <20230607181920.2294972-15-dhowells@redhat.com> In-Reply-To: <20230607181920.2294972-1-dhowells@redhat.com> References: <20230607181920.2294972-1-dhowells@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-Rspamd-Queue-Id: 58E3A1C001F X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: mi4rcsb4wdg5xf36ptb3pook4hsnudyi X-HE-Tag: 1686162035-848997 X-HE-Meta: U2FsdGVkX19MxLIMPoCmLwAHhoASaYQ5plQyNZ9el2rerOK/ll+2paUvtJuRIdFSXcGOCTaJOrNbGRmCaEmzL1pqZMSJbUTSuOMhxrxtHcqUkw4YgTNF46AmVGrPAG9Wx6/lU1HElA8lQY8hREkE/Lv7KHiwKY1dpnA6rpXZw8JvvOILIQcNXVjizX8x6mMAr615CFK8RgkGGkYj+HCQ1UcnfwRhj7diYW4g0KcKMXbYlajCe3DZKC5JMLmHEObiZ3vHXWatVQPEQcbBv8kWqgtTPis8GBSAAaVIVzRgPHRKL0O8p8dH8Ye3f9cKNLq3BhAstv5RBLpVi4fSZdC1iMDJKSM7fakdfK3gN4OIJWU7EweEw5sfT90z/aTyo8fq1iasu0xSJPNBCmVMD4qUbhIRF4PlEzvpJKjlrG1Q8L8v4fO5fPqWbqDL0I1aRWmRvXTesmD37fYJRK4pIFk52MOa+8QOaXjaPaDQ3LcJvJYxD0BUDnOBIhC5WrYi0T+blm6XtHMM3EXfG56Vezh6qsUSJD+OYipuU4wRsc4Rm2gBI+D9em9Hpoqd+Mu0i5DNQUhREf2dXsqi8EOxIdxgp1FdsDDjRiltPV/SrKvBYpW7sni4d16iN/Usm3trJPQ8g5wmpFaFnLuHLUWGGWa1c3AivbkO195msHKq81KK60HjfLTSm5flt0XkYl+J5HWEhOkGU0uzzVplAz6HTI3AOnupXhY77Mna4hyLZOJZ+YZaUczw4YMzQb+xeNGBmxZ+RlP5HGdweiMWewQYz26dfmyUEkwapjdO8KnkKSLDj5yHKnySYNzOyeH6S0EIGPPIpjl5QvLphoFJmNQZscKYWzzTVFvh7EXxMr9cEhdKll10JlvlxQbmO+4KRrwLSBWbXNGMNyqAkxhmMW7TFCaawXQ76+eiZ0b2xbndN276VNSuYz+G0iLE4zbr1p3gfsP7emQlkdq8a+14nKo08g9 ZyndQOzi DBHYKFossR3SRgx7apdj8qiYDQTJKtowTka3xwsFEJN/ccO2Dk7oqtcxiQbS+QXvb3WEtirusylNDo/3UHSkO+skiECvZMU+7z1/KxfH9OJgunJ1uobRE4aOmNhKsgqTjRyUj0qw07Vz7SZ46ttZI+FBF2T0nrnM4cF1I+LulFyiMCqblR6eDTgoSGvOYnkrmWbuuvdoyLKp/92BdwNWaCW0N968keu8GQ2R9c9yK63x97+VU8tlr3bsk7B/7omV4n8hyh84ljrEAwTFh2+GIKVFMppEnsAF5nRVc/aiAjUNQZGlEm/jdOcn2h1dE9Vpky/5uU5U0o1lA2kLkBOjmrd4wF7j82xgl1ALjbsooLg5jh4lbYlRzeCjaW3UqNAHgOgaQ6+LLljAI26IyAtZFGqmlrVq7CQdfPHtE1ovJXYAYlh3aTJK6YAJJ8bKx5KIikUMe6MuKYfFdgjp6JAuTlJ44ttxXQp9QqDKdXq8DVJjHNbYFyN1duZdF30AfXE2h3tMnAoxij7UCWjSPaCbBVYToMYERzhDyA6aYr/h15gDFMUKgRfb/2q2JgA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Convert tls_device_sendpage() to use sendmsg() with MSG_SPLICE_PAGES rather than directly splicing in the pages itself. With that, the tls_iter_offset union is no longer necessary and can be replaced with an iov_iter pointer and the zc_page argument to tls_push_data() can also be removed. This allows ->sendpage() to be replaced by something that can handle multiple multipage folios in a single transaction. Signed-off-by: David Howells Acked-by: Jakub Kicinski cc: Chuck Lever cc: Boris Pismenny cc: John Fastabend cc: Eric Dumazet cc: "David S. Miller" cc: Paolo Abeni cc: Jens Axboe cc: Matthew Wilcox cc: netdev@vger.kernel.org --- net/tls/tls_device.c | 92 +++++++++++--------------------------------- 1 file changed, 23 insertions(+), 69 deletions(-) diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index bb3bb523544e..b4864d55900f 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -422,16 +422,10 @@ static int tls_device_copy_data(void *addr, size_t bytes, struct iov_iter *i) return 0; } -union tls_iter_offset { - struct iov_iter *msg_iter; - int offset; -}; - static int tls_push_data(struct sock *sk, - union tls_iter_offset iter_offset, + struct iov_iter *iter, size_t size, int flags, - unsigned char record_type, - struct page *zc_page) + unsigned char record_type) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_prot_info *prot = &tls_ctx->prot_info; @@ -500,22 +494,13 @@ static int tls_push_data(struct sock *sk, record = ctx->open_record; copy = min_t(size_t, size, max_open_record_len - record->len); - if (copy && zc_page) { - struct page_frag zc_pfrag; - - zc_pfrag.page = zc_page; - zc_pfrag.offset = iter_offset.offset; - zc_pfrag.size = copy; - tls_append_frag(record, &zc_pfrag, copy); - - iter_offset.offset += copy; - } else if (copy && (flags & MSG_SPLICE_PAGES)) { + if (copy && (flags & MSG_SPLICE_PAGES)) { struct page_frag zc_pfrag; struct page **pages = &zc_pfrag.page; size_t off; - rc = iov_iter_extract_pages(iter_offset.msg_iter, - &pages, copy, 1, 0, &off); + rc = iov_iter_extract_pages(iter, &pages, + copy, 1, 0, &off); if (rc <= 0) { if (rc == 0) rc = -EIO; @@ -524,7 +509,7 @@ static int tls_push_data(struct sock *sk, copy = rc; if (WARN_ON_ONCE(!sendpage_ok(zc_pfrag.page))) { - iov_iter_revert(iter_offset.msg_iter, copy); + iov_iter_revert(iter, copy); rc = -EIO; goto handle_error; } @@ -537,7 +522,7 @@ static int tls_push_data(struct sock *sk, rc = tls_device_copy_data(page_address(pfrag->page) + pfrag->offset, copy, - iter_offset.msg_iter); + iter); if (rc) goto handle_error; tls_append_frag(record, pfrag, copy); @@ -592,7 +577,6 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) { unsigned char record_type = TLS_RECORD_TYPE_DATA; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; int rc; if (!tls_ctx->zerocopy_sendfile) @@ -607,8 +591,8 @@ int tls_device_sendmsg(struct sock *sk, struct msghdr *msg, size_t size) goto out; } - iter.msg_iter = &msg->msg_iter; - rc = tls_push_data(sk, iter, size, msg->msg_flags, record_type, NULL); + rc = tls_push_data(sk, &msg->msg_iter, size, msg->msg_flags, + record_type); out: release_sock(sk); @@ -620,8 +604,7 @@ void tls_device_splice_eof(struct socket *sock) { struct sock *sk = sock->sk; struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter; - struct iov_iter iov_iter = {}; + struct iov_iter iter = {}; if (!tls_is_partially_sent_record(tls_ctx)) return; @@ -630,9 +613,8 @@ void tls_device_splice_eof(struct socket *sock) lock_sock(sk); if (tls_is_partially_sent_record(tls_ctx)) { - iov_iter_bvec(&iov_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &iov_iter; - tls_push_data(sk, iter, 0, 0, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_bvec(&iter, ITER_SOURCE, NULL, 0, 0); + tls_push_data(sk, &iter, 0, 0, TLS_RECORD_TYPE_DATA); } release_sock(sk); @@ -642,44 +624,18 @@ void tls_device_splice_eof(struct socket *sock) int tls_device_sendpage(struct sock *sk, struct page *page, int offset, size_t size, int flags) { - struct tls_context *tls_ctx = tls_get_ctx(sk); - union tls_iter_offset iter_offset; - struct iov_iter msg_iter; - char *kaddr; - struct kvec iov; - int rc; + struct bio_vec bvec; + struct msghdr msg = { .msg_flags = flags | MSG_SPLICE_PAGES, }; if (flags & MSG_SENDPAGE_NOTLAST) - flags |= MSG_MORE; - - mutex_lock(&tls_ctx->tx_lock); - lock_sock(sk); + msg.msg_flags |= MSG_MORE; - if (flags & MSG_OOB) { - rc = -EOPNOTSUPP; - goto out; - } - - if (tls_ctx->zerocopy_sendfile) { - iter_offset.offset = offset; - rc = tls_push_data(sk, iter_offset, size, - flags, TLS_RECORD_TYPE_DATA, page); - goto out; - } - - kaddr = kmap(page); - iov.iov_base = kaddr + offset; - iov.iov_len = size; - iov_iter_kvec(&msg_iter, ITER_SOURCE, &iov, 1, size); - iter_offset.msg_iter = &msg_iter; - rc = tls_push_data(sk, iter_offset, size, flags, TLS_RECORD_TYPE_DATA, - NULL); - kunmap(page); + if (flags & MSG_OOB) + return -EOPNOTSUPP; -out: - release_sock(sk); - mutex_unlock(&tls_ctx->tx_lock); - return rc; + bvec_set_page(&bvec, page, size, offset); + iov_iter_bvec(&msg.msg_iter, ITER_SOURCE, &bvec, 1, size); + return tls_device_sendmsg(sk, &msg, size); } struct tls_record_info *tls_get_record(struct tls_offload_context_tx *context, @@ -744,12 +700,10 @@ EXPORT_SYMBOL(tls_get_record); static int tls_device_push_pending_record(struct sock *sk, int flags) { - union tls_iter_offset iter; - struct iov_iter msg_iter; + struct iov_iter iter; - iov_iter_kvec(&msg_iter, ITER_SOURCE, NULL, 0, 0); - iter.msg_iter = &msg_iter; - return tls_push_data(sk, iter, 0, flags, TLS_RECORD_TYPE_DATA, NULL); + iov_iter_kvec(&iter, ITER_SOURCE, NULL, 0, 0); + return tls_push_data(sk, &iter, 0, flags, TLS_RECORD_TYPE_DATA); } void tls_device_write_space(struct sock *sk, struct tls_context *ctx)