From patchwork Tue Mar 26 14:32:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13604283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 697CDCD11DD for ; Tue, 26 Mar 2024 14:32:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB42B6B0085; Tue, 26 Mar 2024 10:32:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E6E476B0087; Tue, 26 Mar 2024 10:32:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D06C96B0088; Tue, 26 Mar 2024 10:32:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id C07A16B0085 for ; Tue, 26 Mar 2024 10:32:45 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 466DE80BF9 for ; Tue, 26 Mar 2024 14:32:45 +0000 (UTC) X-FDA: 81939431490.21.50C6082 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf30.hostedemail.com (Postfix) with ESMTP id 8934180024 for ; Tue, 26 Mar 2024 14:32:42 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W+PsGvPt; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711463562; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jCQ18XUYGXxAblag6bvSJIJrwTPD6Hhm3KxukBaiG2A=; b=763j5Ahcho9npTg02AXV6SZkORijXuQObY93zi2xap1UwzZs9ktirC1KlnoAx6SYzNn8Jt DWnZRRfqmZdtbkpLRxZGQmZnzLGePfR7OQoQJ+3L00MfgFpBBP3M0UqBdcOImLrqJCVL+/ IRTrbbzMdbMt3S1+XhfGMAyBd3rgChE= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=W+PsGvPt; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf30.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711463562; a=rsa-sha256; cv=none; b=Lo4q2wLwuIN2SsL5D2Y8nwlnFOWcetFaxmjUj3qAppLz/J9OBEFEf/kf3Kqc2jJov1aJRk rADwyg752Wz/6ELA1WgxKUZKcWB5Lxf7H6VyHnPARwYuIVTrgMizPqNEayDlGp5bVP8DcR nApPCDo/EQTmjTJSkHap9M7KX1rbSLM= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463561; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jCQ18XUYGXxAblag6bvSJIJrwTPD6Hhm3KxukBaiG2A=; b=W+PsGvPto/7dV1XNioWFiWvQemJgIGerwYHPPYBPk0Nlukjx7FW34eU5wclCSi7KLJWmP7 yaEhusXhG0Jy5kB1URnwyPGQDbUOwm+8kBGcT6iZRPYw6Bx81T9ds3qAH27PSkWRjWiBZt xbYlTDSkmSygB1prZ7YGKRg/O20g1as= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-130-SUIk7uSiPqOSHPDYataC4g-1; Tue, 26 Mar 2024 10:32:38 -0400 X-MC-Unique: SUIk7uSiPqOSHPDYataC4g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 95BD23C0C889; Tue, 26 Mar 2024 14:32:37 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6784340C6CBE; Tue, 26 Mar 2024 14:32:34 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun , Miklos Szeredi , stable@vger.kernel.org Subject: [PATCH v2 1/3] mm/secretmem: fix GUP-fast succeeding on secretmem folios Date: Tue, 26 Mar 2024 15:32:08 +0100 Message-ID: <20240326143210.291116-2-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Rspamd-Queue-Id: 8934180024 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: bc88pryzfcgwrp8sqh3r664ccpmsfjk4 X-HE-Tag: 1711463562-191136 X-HE-Meta: U2FsdGVkX1+Uuli2uoCVXntUhIPKddl/ixIUeVJHsw/Ti5I4rhd8ewrPu33GfaoXR7TzoIG5rNAPIA4vWDzFoJEAsz7MkzONn6cFc1VV5uo/1aEMLpVmpIoYBh0XqdSxP5eLeGSf+Kvqu/a8YCnt7Epy9vGMasXJZfGfURq94Le7kB09trDExk69PlX6nOJUxKFqC9/h7eVbBx5W2nq4VjTghmaUBCB3/ZW4j9r8MRr+55ZFtw+ArfWCiVBlh1VFtkaQHvAoZZZkohLvZ5vUnQrVVmGAY471JUsjScyqinyaDUQGW9qjzgRw08VrW7pvIl1B1QFtJnCGl4YGfIHjmMu65AMR6dRlp/xqAJSSFogajWD4s5BzPCcFje/nIfR1ONlsF07QXB0yi8OA6ffmiXJCW27r1plXLvNAzlYX4jOf/DxVqR2pWnrOMmpT6KLJ4DFg1PzRB83LCjyCh6X4nsZ63R/Vju/6USEc/0CrJfilDqldOm18U+qoTZOwZ/82iB84lIJvatT6Dv9ocUxdee0nhksdDXcX+wyIF14LWL1U+qpn/oOnxfZAVccjzR7+N8BXJvhFkTip75rfqj6cgcN+igVgCvfKPzUJBGx/zg4/SBqwXyQ8bFrzS3ZFFpIsGnNBgaWYq4jwey5q5Wr1R8RpCJztHY/TsNZE9lmQZAr4COmOAgfnXAG/LW023rAXUIvOUNP05eYyACLG8xWuKJ0HWT6kLGo6Axshf9+hDRr37X4e87yvAKx9CjG/cV93cQYX13uI7pecXv9bHWPt50cGgZUlNhnVCl7lgGk16EwH+SofViTmoQauXRwFl/tSllPUfIGKGDXStZid74kdbre35acoTTaaJOa6hhbbWpMqS0krnZd6BYCUTSURsZxz8GF62PVa1g5EZ7nyNsUW6zdOMHxbZ/IlT1x9SdWJdd5qDsFQRjEhxL6UsAi1zCQMkYG5nxXtihwfD7nnI+6 dOx2QK4g N7ybjqGMiJMJbnkzgb6cr06mElg6U9es0ABjy+9NP2B4EUUNSf1AuM6nCW1dPdunYGRDnFgE77QVB9jOWnKQgDUIuHzjQVKniE8rjoLfzULT89xnM0sx2xcCS4734zCGKj6uuvvcFPICo/cUWVB5wY27M1SFjF635pB+4Ya6BG0sXhKZ590CN/EkxL/pmBLNCjlheVpFdIkO9YFXe20aRJQqu/2YKkqU3x+eRg+dqOD9rrqoe4h4ZEj9YF6ZgTK7W2AVp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: folio_is_secretmem() currently relies on secretmem folios being LRU folios, to save some cycles. However, folios might reside in a folio batch without the LRU flag set, or temporarily have their LRU flag cleared. Consequently, the LRU flag is unreliable for this purpose. In particular, this is the case when secretmem_fault() allocates a fresh page and calls filemap_add_folio()->folio_add_lru(). The folio might be added to the per-cpu folio batch and won't get the LRU flag set until the batch was drained using e.g., lru_add_drain(). Consequently, folio_is_secretmem() might not detect secretmem folios and GUP-fast can succeed in grabbing a secretmem folio, crashing the kernel when we would later try reading/writing to the folio, because the folio has been unmapped from the directmap. Fix it by removing that unreliable check. Reported-by: xingwei lee Reported-by: yue sun Closes: https://lore.kernel.org/lkml/CABOYnLyevJeravW=QrH0JUPYEcDN160aZFb7kwndm-J2rmz0HQ@mail.gmail.com/ Debugged-by: Miklos Szeredi Tested-by: Miklos Szeredi Fixes: 1507f51255c9 ("mm: introduce memfd_secret system call to create "secret" memory areas") Cc: Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) --- include/linux/secretmem.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index 35f3a4a8ceb1..acf7e1a3f3de 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -13,10 +13,10 @@ static inline bool folio_is_secretmem(struct folio *folio) /* * Using folio_mapping() is quite slow because of the actual call * instruction. - * We know that secretmem pages are not compound and LRU so we can + * We know that secretmem pages are not compound, so we can * save a couple of cycles here. */ - if (folio_test_large(folio) || !folio_test_lru(folio)) + if (folio_test_large(folio)) return false; mapping = (struct address_space *) From patchwork Tue Mar 26 14:32:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13604284 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A78BC6FD1F for ; Tue, 26 Mar 2024 14:32:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 899CE6B0088; Tue, 26 Mar 2024 10:32:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 849876B0089; Tue, 26 Mar 2024 10:32:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6755A6B008A; Tue, 26 Mar 2024 10:32:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 55A076B0088 for ; Tue, 26 Mar 2024 10:32:47 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2C0CD1C0D7C for ; Tue, 26 Mar 2024 14:32:47 +0000 (UTC) X-FDA: 81939431574.30.463CAE8 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id F3BD21C002F for ; Tue, 26 Mar 2024 14:32:44 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DfPDCD9f; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711463565; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=82tUC6dlxq3xFltomUX9Np5kaF2oYCvvKygUhhN8Yg8=; b=ivTBZYeuJTpynVZc4jjDYWPZC3/VgOUTu2TbCl9/YtXV9DiLaTVybb5VqcqAV4b9GCyTHy NIAtX60D8OfUYi7Th+fOeVUNmzM0dyuje7o+F2+26t9PgaJ1VPZQuWwF3MgQCR5iWi4j7s ITM6XI/WTBX7gPI5AWTVEMtXxYETYA0= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711463565; a=rsa-sha256; cv=none; b=IwFf2RLKDAbFiH+NWteQeybesaN7Syug6g/aTrU/kF7xq65UcR0cJs+10Au1R69doclrm+ AxcozBOCjwgNiWpQ1+bopMKvvTIOW3TfT9TY4/TJemyXzETMeq1UAAkBPApai8cY5TPxyk 1+8SGlCT0iimdO1KRIH3abXQ7Rm6pgs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=DfPDCD9f; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=82tUC6dlxq3xFltomUX9Np5kaF2oYCvvKygUhhN8Yg8=; b=DfPDCD9fVKVmJxMpTQtAk/hrjdA3Ad7syZ8y7ABgLQWK/+pwz9X20HyKXluZbRdU5toSsJ gdLWFZCiAMMjbZNWUoBhS9mQkl/CTrncr/542oscE5aWcGkifjU+su+8vdhD49mS16F2uo 3p5UZqERLS9m6/yQhYVF9+WPwOGn4kc= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-318-Xgt-kVftPyGYoQ5JQWdHag-1; Tue, 26 Mar 2024 10:32:40 -0400 X-MC-Unique: Xgt-kVftPyGYoQ5JQWdHag-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id EE76C382C468; Tue, 26 Mar 2024 14:32:39 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 01750400D698; Tue, 26 Mar 2024 14:32:37 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun Subject: [PATCH v2 2/3] selftests/memfd_secret: add vmsplice() test Date: Tue, 26 Mar 2024 15:32:09 +0100 Message-ID: <20240326143210.291116-3-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Rspamd-Queue-Id: F3BD21C002F X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: dd178xn1bo4ahehjx9y4qq37azg7erqu X-HE-Tag: 1711463564-440878 X-HE-Meta: U2FsdGVkX1+tyxliBWA4Gt3RISziyT5Ba9Gr8tB9n61VlpeXh0qbFvZmNDFT7Czp/X1KgxnM0jb24vXbp5pMaExbr5Fo3ONli/DLElHJkxpmfD3CwRsz2J8T+BKRA0aXZaUcRdSFrZkVXdenqVpD1WODJeMw1fZyU+N+T9boHOPz252Gh/WDHz9wFrOsk4VexidzDeVy7HEOcM/+4wB04O1Zd25SO3SZ5grjg7ZZelxEH366HUInfYplXkWuaxh/5tCEwmT5hcugREbbnlJ+MtGnng1YjtKwtjqyLqOMUSqY21lZR28N0qtrLtvZcoQbhsBGbRJODmlQftsZPUaxIW5/iwIcj+NTipj2nO3zSZhbOFX7bGeLGxQ4H8YqOZQ9XwD5uNwyUmrGz0pOIPU6Kr5lNcjRfrYWEYdqBxrKFDOxTL155wD24e0nQT8Kmdj8nwsNGdtqRQFkUbvrEBT7oe1cqDeABR769rKJWdRSanIiBghN84LtI45snKM3G2E8tM9SYdtr5JUKcJIqWLg4vZgZK+MpChRWvpM1Z1yGS+ePs08cJIcIiRwq2lFDpUifAkY874iJCR4bmP67BYXr0b3QkhWL477nJoBWgr4IVW+/Mhbgwtk/Roo6LtCFrqV6bwoHQdVCNSlqxIdzCqg1kKtBBLrMNw1rt9FpraqodO1FEoFwGS6EFrt1mDOsZ5zxCYOpuEkktg5iBXnmAEjJMkPGI7Y3PSZy0CMlHEAd3Ka+Jh5aL+dAm/NArSM/iU22SVSxl3eVG9aPzUWasvv3rhmlMleqQ1mva4iNV8mRsUo56fok7cCYMV69tZLV7J/FInMKGB5xvN3fnsz3lwtb/oHfxAFKVTWSYFYf7QYiJc1VEH4/b7uR/oWdq1pjqkgSMvjSSsiE53tOzsE0Fp9BUDtRC8QB56fOSVRPCbrTwyy4vqBjO6yluyqAwganNxVKghwffYHeEP6LAxpampx VJ9MVVdc 5G0PvpSsSwV4uTLD+j56IONR256lkeGjVuJ1mbeg8BEHf7K6pXuxL+fMMJc6VM6UpqO3CBVdnTJQ6YDHbu1hK5Nn9T9HbJbH2BHiMXuGUehkENA0mbbaJswayrCpEV/40x787xWBERJ7pqw137JbkefRT69s991kDzDnga8kxZX2TTCT1Fz1Ru5QPcpuv5NphJJqa3ZBGJfyhE+zicJkK4yWT9Do0JPEuHPzY0u18/8XX5Z7BObBldlSIMRWLP6ymA+KWsjoFa8l1JJBjvmfd53EXBvc2pr1lpoPZpx6UtZjlmRE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Let's add a simple reproducer for a scenario where GUP-fast could succeed on secretmem folios, making vmsplice() succeed instead of failing. The reproducer is based on a reproducer [1] by Miklos Szeredi. We want to perform two tests: vmsplice() when a fresh page was just faulted in, and vmsplice() on an existing page after munmap() that would drain certain LRU caches/batches in the kernel. In an ideal world, we could use fallocate(FALLOC_FL_PUNCH_HOLE) / MADV_REMOVE to remove any existing page. As that is currently not possible, run the test before any other tests that would allocate memory in the secretmem fd. Perform the ftruncate() only once, and check the return value. [1] https://lkml.kernel.org/r/CAJfpegt3UCsMmxd0taOY11Uaw5U=eS1fE5dn0wZX3HF0oy8-oQ@mail.gmail.com Signed-off-by: David Hildenbrand Reviewed-by: Mike Rapoport (IBM) --- tools/testing/selftests/mm/memfd_secret.c | 51 ++++++++++++++++++++++- 1 file changed, 49 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/mm/memfd_secret.c b/tools/testing/selftests/mm/memfd_secret.c index 9b298f6a04b3..9a0597310a76 100644 --- a/tools/testing/selftests/mm/memfd_secret.c +++ b/tools/testing/selftests/mm/memfd_secret.c @@ -20,6 +20,7 @@ #include #include #include +#include #include "../kselftest.h" @@ -83,6 +84,45 @@ static void test_mlock_limit(int fd) pass("mlock limit is respected\n"); } +static void test_vmsplice(int fd, const char *desc) +{ + ssize_t transferred; + struct iovec iov; + int pipefd[2]; + char *mem; + + if (pipe(pipefd)) { + fail("pipe failed: %s\n", strerror(errno)); + return; + } + + mem = mmap(NULL, page_size, prot, mode, fd, 0); + if (mem == MAP_FAILED) { + fail("Unable to mmap secret memory\n"); + goto close_pipe; + } + + /* + * vmsplice() may use GUP-fast, which must also fail. Prefault the + * page table, so GUP-fast could find it. + */ + memset(mem, PATTERN, page_size); + + iov.iov_base = mem; + iov.iov_len = page_size; + transferred = vmsplice(pipefd[1], &iov, 1, 0); + + if (transferred < 0 && errno == EFAULT) + pass("vmsplice is blocked as expected with %s\n", desc); + else + fail("vmsplice: unexpected memory access with %s\n", desc); + + munmap(mem, page_size); +close_pipe: + close(pipefd[0]); + close(pipefd[1]); +} + static void try_process_vm_read(int fd, int pipefd[2]) { struct iovec liov, riov; @@ -187,7 +227,6 @@ static void test_remote_access(int fd, const char *name, return; } - ftruncate(fd, page_size); memset(mem, PATTERN, page_size); if (write(pipefd[1], &mem, sizeof(mem)) < 0) { @@ -258,7 +297,7 @@ static void prepare(void) strerror(errno)); } -#define NUM_TESTS 4 +#define NUM_TESTS 6 int main(int argc, char *argv[]) { @@ -277,9 +316,17 @@ int main(int argc, char *argv[]) ksft_exit_fail_msg("memfd_secret failed: %s\n", strerror(errno)); } + if (ftruncate(fd, page_size)) + ksft_exit_fail_msg("ftruncate failed: %s\n", strerror(errno)); test_mlock_limit(fd); test_file_apis(fd); + /* + * We have to run the first vmsplice test before any secretmem page was + * allocated for this fd. + */ + test_vmsplice(fd, "fresh page"); + test_vmsplice(fd, "existing page"); test_process_vm_read(fd); test_ptrace(fd); From patchwork Tue Mar 26 14:32:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13604285 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 332AEC6FD1F for ; Tue, 26 Mar 2024 14:32:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7637E6B0092; Tue, 26 Mar 2024 10:32:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6EF446B0093; Tue, 26 Mar 2024 10:32:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4F02D6B0095; Tue, 26 Mar 2024 10:32:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 376D96B0092 for ; Tue, 26 Mar 2024 10:32:50 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id F0469C0C2F for ; Tue, 26 Mar 2024 14:32:49 +0000 (UTC) X-FDA: 81939431658.11.A6356EC Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf20.hostedemail.com (Postfix) with ESMTP id CE5921C0015 for ; Tue, 26 Mar 2024 14:32:47 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LYNcYh7U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711463567; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cZdlw1b/kcLUrflUxwTe3odXoIWBotMqjI0wjKr5kQg=; b=IoEL/aXgBMSqIRu1NeZ/2eM8Da2MLMHR0LA7KkMF/KCElBFdR2R8BwQPbL5HDu5Gc3zL2z SNpmdgJuEzvO7T5d4Fw0Z0fJeX/cgsh0VAbbn1JzhHE8rd7g1hiFtfArUpWbDXagNHRrAa R28IrkYW+r0OExPWIBknJDH9E23RzIs= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=LYNcYh7U; dmarc=pass (policy=none) header.from=redhat.com; spf=pass (imf20.hostedemail.com: domain of david@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=david@redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711463567; a=rsa-sha256; cv=none; b=CrXraIa1AuTV1rzA37mc+uKpvqSax8NzSM/TCaOZ4V96RA9QAS27MrTfp1j0NE2uzSUcyi ggGWC1sLj29eBE7nMBQS4LE5qZmDOxSaQrmm7oFQ0j9wFol1tcqHoc0cHKg2Gm/+nqf0TF AU0SXviQATz9ETj18sOHWcRonOctm+Y= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711463567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cZdlw1b/kcLUrflUxwTe3odXoIWBotMqjI0wjKr5kQg=; b=LYNcYh7UQ5Q+o2W1IZqd3DEaU4NUl3RjfrC1E7yI9EN5jB7J3X+8ukRpMzIP0Jy/rWP/Tz 8Rga+tkjrcan2EyFJomiJzxUrttEc8+NJwYFEW32qcgOuENOAgvMfwgp5CMaN4+XWIIFoe wbvVmBJL4uIpPB0riCTBBwEy1B8BmAQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-468-AuwEDpdsMcGdxkD-3r8M6g-1; Tue, 26 Mar 2024 10:32:43 -0400 X-MC-Unique: AuwEDpdsMcGdxkD-3r8M6g-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B404D800266; Tue, 26 Mar 2024 14:32:42 +0000 (UTC) Received: from t14s.fritz.box (unknown [10.39.192.164]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43F7540D2982; Tue, 26 Mar 2024 14:32:40 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , Mike Rapoport , Miklos Szeredi , Lorenzo Stoakes , xingwei lee , yue sun Subject: [PATCH v2 3/3] mm: merge folio_is_secretmem() and folio_fast_pin_allowed() into gup_fast_folio_allowed() Date: Tue, 26 Mar 2024 15:32:10 +0100 Message-ID: <20240326143210.291116-4-david@redhat.com> In-Reply-To: <20240326143210.291116-1-david@redhat.com> References: <20240326143210.291116-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Rspam-User: X-Stat-Signature: etdet1b8ib1b1hoef6wracud4q6ob7ua X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CE5921C0015 X-HE-Tag: 1711463567-539294 X-HE-Meta: U2FsdGVkX1+YHg8ZVRj6AjX6ajuAiZvjtjQcBqePSObs9qLBbl9RVzNqjoZj6TjsVpc7bzOfQlnS6ZMND8VPLuYf8WFLNiX9ZwljzGiiFrL6njE+GhTFygI8TO6gYoP1QDeNzwZ45p9b4F7ZmNctqfW4BnvkNqM0kwNkQ1rI2fJiqdvhB3q3N2f3iESsUkDUfbAacgPdhpJd3DoBuj8bQE6XzKQnJUI/HHLxd5ouJf6k7P81bF/UHWlg7RzijviNU7AkPZLAGpjnJHd5YzvKK9EUACCnaRU0K5HVv9/dXagaSI45/VHQDLb9pu+VuxYYerfmKyko07S0ccGHODri6nDzrlO3wD7eI0O0F5PhJLbZlSua6kwGuCWiOtLQTfL280xglIvN8JxVz/KKsjXk6HfhlZ6EMC52RY9E9xDXWhoOfVwbDMNPdA1Ca8Jcn8554M/NnYR21fa/r9mB5lYoWgM72X14MqLtNBCc4sqgmrP/jPM1Yw32MkORTBpacDrZelEmFNaPN0+6uQALcm5p+ZmW8yO4c9r0VLUPKWdJ97z3adTuVCHI4xHCsPdWZGgseDaFZtIBxhhnY3mwJxh8g0C/fs3CX7U7NA/EqDWMVt+DpVrfWrMrhmkiGyNnoKVJFFsnDBWS7S/2AG5D5YPc+TFpKPaWRst18sr/VVyz2fCiCil+hgEZzlG/OA6laEeb68YdJRL/oJ4owhOovoIR8ZA1wecw0gv9+s1TgsHHJEcXchFDsIGEwDrUdTX8x39v1qJFm2/JMlOZUOmvgRS97u7fbmwe/zcmW0dZ49yUADhnr9DsRX8CzsGKXLhvPtoPzmCxA2xZ+mw5rJ5pi6LOB0CYytkiFufWVi5D/0KfJZJQR2KsxZ6ZnmPD2cLDZFo+R2MDZibiAxJCGIieIoOYKNRcnvwwfq89NxX3s3xovwfY8lN9F4p4hebo1iIE3xUQiRWXu+MTx+A1W6NeKKO 4tjY4Cv6 Vu9Ld+cjVXdhoRDL/S0oZJ1Jqt/Gm8tl5Uq3OtxVm2YBDpg6qHNqlSflVfAgDIWJnTAOdrKdnUvr3s4OFYLqwJeJm3Lv7+7Tv9N4VgJz3uE/v9IN4wpAcR0TVS9umtJGe0Ps+0hRhREN6nD4pK6OL2OCcnG4UgltGt0O9jyfj52lcAzNNs2rQVzCkI/t0SrSuMhoHLaJbwUjzEa+6kir3u33/XuGzd63pUexpBx1wAsvr6cbkLqS3FfbalfqR5CU9DFfFbaBwdTNtNhYW4OTx9jj8mg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: folio_is_secretmem() is currently only used during GUP-fast. Nowadays, folio_fast_pin_allowed() performs similar checks during GUP-fast and contains a lot of careful handling -- READ_ONCE() -- , sanity checks -- lockdep_assert_irqs_disabled() -- and helpful comments on how this handling is safe and correct. So let's merge folio_is_secretmem() into folio_fast_pin_allowed(). Rename folio_fast_pin_allowed() to gup_fast_folio_allowed(), to better match the new semantics. Reviewed-by: Mike Rapoport (IBM) Signed-off-by: David Hildenbrand --- include/linux/secretmem.h | 21 ++--------------- mm/gup.c | 48 +++++++++++++++++++++++---------------- 2 files changed, 30 insertions(+), 39 deletions(-) diff --git a/include/linux/secretmem.h b/include/linux/secretmem.h index acf7e1a3f3de..e918f96881f5 100644 --- a/include/linux/secretmem.h +++ b/include/linux/secretmem.h @@ -6,25 +6,8 @@ extern const struct address_space_operations secretmem_aops; -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { - struct address_space *mapping; - - /* - * Using folio_mapping() is quite slow because of the actual call - * instruction. - * We know that secretmem pages are not compound, so we can - * save a couple of cycles here. - */ - if (folio_test_large(folio)) - return false; - - mapping = (struct address_space *) - ((unsigned long)folio->mapping & ~PAGE_MAPPING_FLAGS); - - if (!mapping || mapping != folio->mapping) - return false; - return mapping->a_ops == &secretmem_aops; } @@ -38,7 +21,7 @@ static inline bool vma_is_secretmem(struct vm_area_struct *vma) return false; } -static inline bool folio_is_secretmem(struct folio *folio) +static inline bool secretmem_mapping(struct address_space *mapping) { return false; } diff --git a/mm/gup.c b/mm/gup.c index e7510b6ce765..03b74b148e30 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -2466,12 +2466,14 @@ EXPORT_SYMBOL(get_user_pages_unlocked); #ifdef CONFIG_HAVE_FAST_GUP /* - * Used in the GUP-fast path to determine whether a pin is permitted for a - * specific folio. + * Used in the GUP-fast path to determine whether GUP is permitted to work on + * a specific folio. * * This call assumes the caller has pinned the folio, that the lowest page table * level still points to this folio, and that interrupts have been disabled. * + * GUP-fast must reject all secretmem folios. + * * Writing to pinned file-backed dirty tracked folios is inherently problematic * (see comment describing the writable_file_mapping_allowed() function). We * therefore try to avoid the most egregious case of a long-term mapping doing @@ -2481,25 +2483,34 @@ EXPORT_SYMBOL(get_user_pages_unlocked); * in the fast path, so instead we whitelist known good cases and if in doubt, * fall back to the slow path. */ -static bool folio_fast_pin_allowed(struct folio *folio, unsigned int flags) +static bool gup_fast_folio_allowed(struct folio *folio, unsigned int flags) { + bool reject_file_backed = false; struct address_space *mapping; + bool check_secretmem = false; unsigned long mapping_flags; /* * If we aren't pinning then no problematic write can occur. A long term * pin is the most egregious case so this is the one we disallow. */ - if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) != + if ((flags & (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) == (FOLL_PIN | FOLL_LONGTERM | FOLL_WRITE)) - return true; + reject_file_backed = true; + + /* We hold a folio reference, so we can safely access folio fields. */ - /* The folio is pinned, so we can safely access folio fields. */ + /* secretmem folios are always order-0 folios. */ + if (IS_ENABLED(CONFIG_SECRETMEM) && !folio_test_large(folio)) + check_secretmem = true; + + if (!reject_file_backed && !check_secretmem) + return true; if (WARN_ON_ONCE(folio_test_slab(folio))) return false; - /* hugetlb mappings do not require dirty-tracking. */ + /* hugetlb neither requires dirty-tracking nor can be secretmem. */ if (folio_test_hugetlb(folio)) return true; @@ -2535,10 +2546,12 @@ static bool folio_fast_pin_allowed(struct folio *folio, unsigned int flags) /* * At this point, we know the mapping is non-null and points to an - * address_space object. The only remaining whitelisted file system is - * shmem. + * address_space object. */ - return shmem_mapping(mapping); + if (check_secretmem && secretmem_mapping(mapping)) + return false; + /* The only remaining allowed file system is shmem. */ + return !reject_file_backed || shmem_mapping(mapping); } static void __maybe_unused undo_dev_pagemap(int *nr, int nr_start, @@ -2624,18 +2637,13 @@ static int gup_pte_range(pmd_t pmd, pmd_t *pmdp, unsigned long addr, if (!folio) goto pte_unmap; - if (unlikely(folio_is_secretmem(folio))) { - gup_put_folio(folio, 1, flags); - goto pte_unmap; - } - if (unlikely(pmd_val(pmd) != pmd_val(*pmdp)) || unlikely(pte_val(pte) != pte_val(ptep_get(ptep)))) { gup_put_folio(folio, 1, flags); goto pte_unmap; } - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, 1, flags); goto pte_unmap; } @@ -2832,7 +2840,7 @@ static int gup_hugepte(pte_t *ptep, unsigned long sz, unsigned long addr, return 0; } - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2903,7 +2911,7 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, unsigned long addr, return 0; } - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2947,7 +2955,7 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, unsigned long addr, return 0; } - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; } @@ -2992,7 +3000,7 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, unsigned long addr, return 0; } - if (!folio_fast_pin_allowed(folio, flags)) { + if (!gup_fast_folio_allowed(folio, flags)) { gup_put_folio(folio, refs, flags); return 0; }