From patchwork Tue Dec 10 08:44:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Qi Zheng X-Patchwork-Id: 13901034 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07C53E7717F for ; Tue, 10 Dec 2024 08:45:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8F2496B013D; Tue, 10 Dec 2024 03:45:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 8A2656B013E; Tue, 10 Dec 2024 03:45:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 769F96B013F; Tue, 10 Dec 2024 03:45:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 5C4A46B013D for ; Tue, 10 Dec 2024 03:45:19 -0500 (EST) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 0677B1604AA for ; Tue, 10 Dec 2024 08:45:19 +0000 (UTC) X-FDA: 82878414738.04.891DCC8 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf24.hostedemail.com (Postfix) with ESMTP id 6EF59180005 for ; Tue, 10 Dec 2024 08:45:14 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IGQ01uNV; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733820296; a=rsa-sha256; cv=none; b=qVOQKTpAf++uB+bqVIhB8N8/PBdHFxUHCXcK2VKVvj3CYDihw979zg2GWdQbqZvAv+9ptL ZJZUnnY7lNV/3qugHNpCymjNKG6/DXpIPvhs2jTQFHliAtUEdpS+HI2nNS+cjn71xpT5RU 5cRsmL7Q5bJhcTGkXd1FKgBmiqnLTQk= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b=IGQ01uNV; spf=pass (imf24.hostedemail.com: domain of zhengqi.arch@bytedance.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=zhengqi.arch@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733820296; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Niwmr5rUid6tm0Hp/quWe/OSvUvk3d2Sf0tLqBvPqbs=; b=JW7tq0GtPXbZ8vaoqJd96f1DbaVbOiiENAMOUx1Qy7UtLF7sV/C43bdYithrGqAShXiNZ4 GPBxEYqnLtLpGdo72ODCQ+xN2J0YlOBonrEPh1g7/hNPoYNFjEh7BIuTU2/au02qS8DhQn W8+t+tj7tBByfzkuv4q1s0oAyOFkav4= Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2ee67e9287fso4717858a91.0 for ; Tue, 10 Dec 2024 00:45:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1733820316; x=1734425116; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Niwmr5rUid6tm0Hp/quWe/OSvUvk3d2Sf0tLqBvPqbs=; b=IGQ01uNV+9Wohhx1e5obwS+79Z3oRarpoYQiKoJavSAb7RjoXWtf1pWQFxjCgLUdXi eKKH9EjqqqkLMB/HlvFQ2yFb1IN6PshfZ+4DYcoSPJyagvps0x7vEboaEiCMMBKxKzi9 ymEVI7oNOYg6tlarVqvCITWgxnQ478dkgCIegqqlwkS5heZOIuPyjUkO87eCVxg6VSEx giY7jR+aba2z2nLsc1qnHWkxM4OqBRw/y/byiwYPWzhOWGKjQ+D2KIrS1H0hlT20AFLu r23x/dNXhPc7MwZmvYl4xuaUZwAJR5kIyK0QeRArWIBmLAR+enmrRDUxW+hEUH4/Lfb6 GbEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733820316; x=1734425116; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Niwmr5rUid6tm0Hp/quWe/OSvUvk3d2Sf0tLqBvPqbs=; b=YPNZjcM1YMqPGqV2ju2H7wGLNDYM1Lxr/Vy9nNQm4efznjAHMQJMXYMWv87OC54qcb YDQz+slaoEfruEV7U15d+bD2Ma+Zc3JCKD3bZ4jpp6y7c20V7pAyv0PSSr0Bk12MveRu l7udyjapfH/KMhxfBFDt+oCzoBSGFA8rYsq/BPWhbolaiZNupwJOFJc2v9LV6D58/Nzk FpetUR78KjoYED6pyW4PntlbdH2IpntuRNRWeMbigsYlF0eQ7vYg2Re7ydIVxjlkVNdp bOoyONIQcGlZeKouVOVY1/zGQIwz86VHw4lOtcuBihAvzaEL1x4rmsrO02bg3rwap//C YeWA== X-Gm-Message-State: AOJu0YzCU6uqKv8VUHtqrmVzmGJ2kPIGAQNrXn5f6tnyqrYOeSajGKlA U8H1RB2VPNqonqRS5H+lL6rK+TduE6LDbh7D1CRv6xALGB+yywnUEZOhsYtioLM= X-Gm-Gg: ASbGncsM59vbo2pztMT6rkT2zKo8/SWl3lnjhn1wJXZB/Neu4Pe53x9dCcwxukxqKL7 WMZwIO1dtXtYTp8bK2GqtQJZ/mZAVnk/e+uVVId8umw0pLwe4oR0mOdiLEiZuKTGBqiL/XepblW xhhAYVD1KK39XPHjO6ywW3YrznlBQxp5hFRcVAoaRkUIIR0ZdoEofCUAofuAwId8j1Av7P0ZC91 QdbpKYdc1RnzzTORgPtBsN6gKLjWmeeZJzAY3iGNK/aPjxn/4JrfpmD6DyuUB5YORR6/HwmNnG7 xuRWca24ZFn9NFAGaJs= X-Google-Smtp-Source: AGHT+IEoD4PcTbX2Xdaqk8HuoEBvaFrwzOLStkOiWJYI9VBD46uedAzdCImEWINRDJRn2KW+Js8ZLg== X-Received: by 2002:a17:90a:c88f:b0:2ee:48bf:7dc9 with SMTP id 98e67ed59e1d1-2ef69e16df7mr24803575a91.15.1733820315774; Tue, 10 Dec 2024 00:45:15 -0800 (PST) Received: from C02DW0BEMD6R.bytedance.net ([203.208.167.150]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ef7d302606sm6016395a91.18.2024.12.10.00.45.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 10 Dec 2024 00:45:15 -0800 (PST) From: Qi Zheng To: akpm@linux-foundation.org, david@redhat.com, jannh@google.com, hughd@google.com, muchun.song@linux.dev Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Qi Zheng , syzbot+1c58afed1cfd2f57efee@syzkaller.appspotmail.com Subject: [PATCH v4 12/11] mm: pgtable: make ptlock be freed by RCU Date: Tue, 10 Dec 2024 16:44:31 +0800 Message-Id: <20241210084431.91414-1-zhengqi.arch@bytedance.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) In-Reply-To: <841c1f35478d5354872d307888979c9e20de9c09.1733305182.git.zhengqi.arch@bytedance.com> References: <841c1f35478d5354872d307888979c9e20de9c09.1733305182.git.zhengqi.arch@bytedance.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 6EF59180005 X-Stat-Signature: wjjawx6kxwfnbnrg9ncxzekjqupm848j X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733820314-408107 X-HE-Meta: U2FsdGVkX1+dyYajj9IfeMx+PfdOiXHbgdU27FstEPkBNvq4HTPt84C+vpyyhbXyA9FudbeU1UekCeFURvoE3u325u3wuSuoMSMor+aILZmg3AJV8RSLQc1G61+LWL0k870/c794BZZmB2U+CEo+dPC+1UVPObY2WSANjqdb3G81CXHQXLAgu7ASRrjt22IsNYb0iPxLYTD+sWKywI5XVxnzfcMjH010ksRKgoFBxkh1eLY+xHYkYS1cKeoBX5zJd4LvnLOSC9abfMYM18OJNJvOSYVND4LFcBiDB4ST4RcSSmdH1Fn5BNhgQLjitGk8BmJMObHuFKBBicN3m1OMnlI8WRUbQlp0U0m7WS/vhidWKpqWVjRaGzoR4yBu2lbDbEbdX7kAadQEAOf0+3NnCYGrusRi4umP7atJF/UfEWIhOxupx48KbOCYhRj/yWxXpbD049QSMuOaKKgL2b5uOyluPzody7oVluB/eq+/9h+7j7g4w7ymlIRHgzTjsPV6MGyc2KhR3KZM/qnj5h9DkvpfTpRE/IelEJ1+Qfxsa8ADk4EWeMwBl9gEuL2JpM+fnxkmshQkdHsqbX4IjHIxtsAuCfDKkge2enZMsw4+qtj6oUDeje2c/Cvov/9JlVLGzqP4OnNB6vP1eFbvD7VyQCfFzqLGFOQqSUnhE+OP0ae2kLv7q3j1z08LDTpk/ynT00XzwDmZpoaZMIYX1YgB2b3i6sJ/5HDo8grZdNkNLskRLzfcckgzeL/5dNK1DpkS1QZolmJNWel+vN+7QtwnzNKMI/EW8GZWufDonT/J0+mcEGZMjj/uJHPL5hmT4smTIANiCnh5iQuyCH98IF89vU6K6qf+PrEvEjHIfkinAYFYGi4se6cZ3UKlzxphyRvpJfTGc6oPRG326dUfc99suqqh2yqe6GE6Q7m4E0mZmwzNtrO83jEhnc/qqUjNwQ9X7k7eyynAAC6mUBx4EEf E/gwBZDD X+if6myvR+bsKOSKjYmxoRcddYrOc8kL6YO4BZBIWp7DsHmDzEcICqsLjoqXamTD8czUqvgVs+92he6JKQm6HAQQoF8ZrT3CwZ76qXdXs+25iiwS4aMXw3gEn3s9KCLFLXPONDd5eWfm3K+N6ilp8JMHmINzuFbP/4T2AzYIEuHNy7rCGVHc4Chu5w2CpnqI7NUV0LqbiRAFeSG1xH7km00jiT84EHb5jw5URaxLNdy4EJ6uPtw/ht6dBTieEu2dNlXECgWjUlI1j1nu+OptVHwW+WWSBA62Q6osX+Wpe4m8SZsvPhj423fhK2ZdWcQNE+hIjIRvSO/M6hxv4jjv6b7QVuFNmo/VrxFC4m4kAmRtoULFEitaDh8ERsU0Hcd2FYvFKBXDmGtGzDGwnAJa7YGG7agjYy4xxfl1YpU4vNBaX4Ohl8B7IJvsfgdL0FJ4ERw2PdfN/rxQjTfi0sOKNE4vuKA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If ALLOC_SPLIT_PTLOCKS is enabled, the ptdesc->ptl will be a pointer and a ptlock will be allocated for it, and it will be freed immediately before the PTE page is freed. Once we support empty PTE page reclaimation, it may result in the following use-after-free problem: CPU 0 CPU 1 pte_offset_map_rw_nolock(&ptlock) --> rcu_read_lock() madvise(MADV_DONTNEED) --> ptlock_free (free ptlock immediately!) free PTE page via RCU /* UAF!! */ spin_lock(ptlock) To avoid this problem, make ptlock also be freed by RCU. Reported-by: syzbot+1c58afed1cfd2f57efee@syzkaller.appspotmail.com Tested-by: syzbot+1c58afed1cfd2f57efee@syzkaller.appspotmail.com Signed-off-by: Qi Zheng --- include/linux/mm.h | 2 +- include/linux/mm_types.h | 9 ++++++++- mm/memory.c | 22 ++++++++++++++++------ 3 files changed, 25 insertions(+), 8 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index e2d38c5867b32..e836ef6291265 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2988,7 +2988,7 @@ void ptlock_free(struct ptdesc *ptdesc); static inline spinlock_t *ptlock_ptr(struct ptdesc *ptdesc) { - return ptdesc->ptl; + return &(ptdesc->ptl->ptl); } #else /* ALLOC_SPLIT_PTLOCKS */ static inline void ptlock_cache_init(void) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5d8779997266e..df8f5152644ec 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -434,6 +434,13 @@ FOLIO_MATCH(flags, _flags_2a); FOLIO_MATCH(compound_head, _head_2a); #undef FOLIO_MATCH +#if ALLOC_SPLIT_PTLOCKS +struct pt_lock { + spinlock_t ptl; + struct rcu_head rcu; +}; +#endif + /** * struct ptdesc - Memory descriptor for page tables. * @__page_flags: Same as page flags. Powerpc only. @@ -478,7 +485,7 @@ struct ptdesc { union { unsigned long _pt_pad_2; #if ALLOC_SPLIT_PTLOCKS - spinlock_t *ptl; + struct pt_lock *ptl; #else spinlock_t ptl; #endif diff --git a/mm/memory.c b/mm/memory.c index 91900a1479322..b5babc4bc36bc 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -7044,24 +7044,34 @@ static struct kmem_cache *page_ptl_cachep; void __init ptlock_cache_init(void) { - page_ptl_cachep = kmem_cache_create("page->ptl", sizeof(spinlock_t), 0, + page_ptl_cachep = kmem_cache_create("page->ptl", sizeof(struct pt_lock), 0, SLAB_PANIC, NULL); } bool ptlock_alloc(struct ptdesc *ptdesc) { - spinlock_t *ptl; + struct pt_lock *pt_lock; - ptl = kmem_cache_alloc(page_ptl_cachep, GFP_KERNEL); - if (!ptl) + pt_lock = kmem_cache_alloc(page_ptl_cachep, GFP_KERNEL); + if (!pt_lock) return false; - ptdesc->ptl = ptl; + ptdesc->ptl = pt_lock; return true; } +static void ptlock_free_rcu(struct rcu_head *head) +{ + struct pt_lock *pt_lock; + + pt_lock = container_of(head, struct pt_lock, rcu); + kmem_cache_free(page_ptl_cachep, pt_lock); +} + void ptlock_free(struct ptdesc *ptdesc) { - kmem_cache_free(page_ptl_cachep, ptdesc->ptl); + struct pt_lock *pt_lock = ptdesc->ptl; + + call_rcu(&pt_lock->rcu, ptlock_free_rcu); } #endif