From patchwork Thu Dec 26 17:06:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13921382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01191E7718F for ; Thu, 26 Dec 2024 17:07:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 17C166B0083; Thu, 26 Dec 2024 12:07:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DE676B0085; Thu, 26 Dec 2024 12:07:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E729A6B0088; Thu, 26 Dec 2024 12:07:18 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C79956B0083 for ; Thu, 26 Dec 2024 12:07:18 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 704871610AC for ; Thu, 26 Dec 2024 17:07:18 +0000 (UTC) X-FDA: 82937740578.14.453AF24 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf20.hostedemail.com (Postfix) with ESMTP id 2EE101C000F for ; Thu, 26 Dec 2024 17:06:32 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JsCkxbTw; spf=pass (imf20.hostedemail.com: domain of 3Q41tZwYKCFYGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3Q41tZwYKCFYGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735232817; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lZeoQJ48TvPY6kfbB0nyOCq9qEQQ9w4ETlSFBxsMiUM=; b=0nEphQNtjkcjDxwCnqQIaRp3MV4cvubpWqf9sIr3p45Kg9YQegl2vk1IxaFMKWV4sS7hYE +hO5j7ckakN/LkfcpxVRBAL2hxQtrKHz3BY0tJNSqwOr+qMGfM4BaTpVEzPgUxeyPOM6iS 0A5PKuiOithsZv5KdSUXW3FxYmXwqfo= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=JsCkxbTw; spf=pass (imf20.hostedemail.com: domain of 3Q41tZwYKCFYGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3Q41tZwYKCFYGIF2Bz4CC492.0CA96BIL-AA8Jy08.CF4@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735232817; a=rsa-sha256; cv=none; b=IXk2/nBYpzv/YCN8Vq/+OajgVVRvZYpRK6svriAZ4Ve7RZkMrZWIu8PGGYZK1q6bvw9Kki fSKYYDjg//5p6Av4xAjcbpReW9wnzct9OG+LEZ3dK+UzRzLYoh4QAqpwJZ5iBKrDb23MdT ma9IT/vZAtlaf/A5BeByeOUKee9kiK0= Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-21631cbf87dso73144945ad.3 for ; Thu, 26 Dec 2024 09:07:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1735232835; x=1735837635; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=lZeoQJ48TvPY6kfbB0nyOCq9qEQQ9w4ETlSFBxsMiUM=; b=JsCkxbTwrd3Vg7lKGmR7gUWX+bvx2T/zW9W7yeEIY3eaqYETH+Wj56RJN01/A9ds6A xo+9NNFT1+D7RnbjE1jkPSvsKMkUZR0z/gU+MEk6NYn9ENDLVbFDnRIsPywbNG+DDnze ushhHrngJKN2/s1qOXdKg0ad5LrUxwFYmUDpRQKl6LuM9Xpor7aNIJpcieP5IpS7t1fC N+dbH1s7/URzwS/4VT/pngpaGBw+TN/JWpjTf0w0E9MUVbpLGwcrT2i8aUeoDbxYYlaj VBxNJ5dU0k/ZgD3n7CfuhLdSeUEGScRwVOR2pq5cGVsP6iA25jq6ZZpI8jFdu9x4d/C7 qf5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735232835; x=1735837635; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=lZeoQJ48TvPY6kfbB0nyOCq9qEQQ9w4ETlSFBxsMiUM=; b=OqqggNmfRw1YEtZ7zlSOonX+uyrM720aeiMHmZWbKdVprVXBB6yovomXNTBrdSuppq XZ8v9qwbWYEMXD2f57ycMrQQ/8NFiltvrnnGFkIRZ5Hf63aYGSo5URcOvWk7o79JHGm0 ihAQH1eeWSAjuodtcC+SW0vDz/WcQgUe+pqpEqtGQhk++ioG4xkS2YcJ2sdgzGWKevfO quG3TtePhwbnY5P6otO41uPf6/BYuL8T+6ZcD9Ja5mPIQZ+RzLnmdsn6rfJlpHCVTTpM Ld21Vp76v132gWiQvOlepmLsQdNkzDH6c7niGGS8a3Cu+aaa4cf7H3Xxz64HC3o0NZc4 ag6A== X-Forwarded-Encrypted: i=1; AJvYcCVVzKj3+ewn3YJ+fD/zu7l26hjd6v6Yfjp4dlxFw78D0d84hPaGHOX7VTzcRh16wnENlzXT5nvYug==@kvack.org X-Gm-Message-State: AOJu0YzRQKNjrB8EzKBHyyTGaHXs+Z5+Aj+hCeKihIcrNDL0LoG3L/vI Bz/aJ5VrmTg4nF4ljjD52pzcfG8TEDI6clF1AN2KYpIfSBCvQGnnresfoUss1cGlNfMolksqd+b TYw== X-Google-Smtp-Source: AGHT+IGWdbrm7iCFwz72a6IFmBjvmmjFFdB3W+njMZ9BDPMzH23Ov9tsL+sC5puGq870a2X6Uofv9F0zU/g= X-Received: from plty1.prod.google.com ([2002:a17:902:8641:b0:216:2f96:8ba1]) (user=surenb job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:d587:b0:216:5af7:eb2a with SMTP id d9443c01a7336-219e6ebb70bmr361013645ad.33.1735232835405; Thu, 26 Dec 2024 09:07:15 -0800 (PST) Date: Thu, 26 Dec 2024 09:06:53 -0800 In-Reply-To: <20241226170710.1159679-1-surenb@google.com> Mime-Version: 1.0 References: <20241226170710.1159679-1-surenb@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241226170710.1159679-2-surenb@google.com> Subject: [PATCH v7 01/17] mm: introduce vma_start_read_locked{_nested} helpers From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: peterz@infradead.org, willy@infradead.org, liam.howlett@oracle.com, lorenzo.stoakes@oracle.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mjguzik@gmail.com, oliver.sang@intel.com, mgorman@techsingularity.net, david@redhat.com, peterx@redhat.com, oleg@redhat.com, dave@stgolabs.net, paulmck@kernel.org, brauner@kernel.org, dhowells@redhat.com, hdanton@sina.com, hughd@google.com, lokeshgidra@google.com, minchan@google.com, jannh@google.com, shakeel.butt@linux.dev, souravpanda@google.com, pasha.tatashin@soleen.com, klarasmodin@gmail.com, corbet@lwn.net, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, kernel-team@android.com, surenb@google.com X-Rspamd-Queue-Id: 2EE101C000F X-Stat-Signature: 1frfpjwkkq731ydgq9qjw8id7h7rgzwu X-Rspam-User: X-Rspamd-Server: rspam11 X-HE-Tag: 1735232792-434625 X-HE-Meta: U2FsdGVkX19dA/k7QBQPbqzNh+xQBbBLLOe2iq584KGYuHkGIp16nLlUau7vcZ17j9yaKQorNc3z12ADN5PCa9bDhAH48BDm1mwfzcwZHHmO39jtowB0k0HupsiKYcr2vjhXrY/qXG8HncYc4+5VB4lIVVJjFSSaTzvKrsRobXlLDjwPWLt98uZr/pQGgXG2bTtSMB6roI8f5wHDcFFcZNenJnB5uqTrqGlvJ1+XQy9UEtm6dHpndjVi3nNhvS55yIgOL/zFxle+eho5OX/5gL71oC9rXjdthO+rH6ov8mLAPoAsmxddiT9EsBdw10zXx3tm2VE5dLtaJEy8hhKChSHS8Dib63tp1Ceiwq7qkFnRoc9lQMdZPMk2f+gyCHqXbUT1Jhe7ZSY5XB9QK+7+0wtKzSj+W3EGvxVMqYs1HFkdZv9Ew3v9iMiu1/7Jsf2YLJ4P4PStw5Dnh2CZUfQbbQWGYJDHUs4hk9TwjTaYvSCwbZiXJt3ug1mapdQrCpBtPZgdt30K70TsHOSvRZ8YCK3M8vvUimMt5d+YPt+ysVJjzI/86GeHljvQqil7Ahujr45tRlQMH68P6IuPh/VCbTqzuXNblRZdcd0+xsCG3pYyUV+FSIZeWzQRrFOs+v3J02xa2RxAey7WJ2gJ3P0jB0dbptu9r3FY9xT78yj/8SXGtSXpGK6SaFKZLVlQ4EAQROY6dYxLLhVpoyfAsA4NuLsU456RhHPxKbBsLwjx+3TmQ5rzJrySIVlW4mb9oMto4gMt+JYZV/+2pQOhoIwcawRUvYV+ONjQwnxgGsWyRmO6JAiYMG3Hg29b+A1KREZNewdrO/mwbxk7J1oryrQpA+kU4gl0tkr0dCVP8jE+RMvCqEAFPyhE74UJSHy8zNQHvfNsuLsniMUMFS7LY0QTdXirQCrekY8bE15DR0wJR44KfCCcxw0oLt93yTAzuEaf1EobOzx4m8vFgHqBU4B qe2iGz1r I8LxSZ+dYnszcuW3+ZyvKxtwn0GBH7DL9NVQJcXeeaB4HfTiv2XXwpexksUXbWn6w+yBbvGAem29H2axU94CpgO6cZDLNkGiWqX+bnElrvnvqCq2coDoNYjBbi1YzB+/3lVGnxNEf/G28Hgk7+9pLpCTT+cWH5V5+42qDN6iTvPRgUtGS0yNuATDzqTh8lxvaerpsSZRbxQQlD9Zo5tuPUtcu2foPBuoxYBfkiWqKREHPV8XghM88CKVzkykw42GvhDkBCAVpRN/wKfjcpEMSAcRiCZTzpygxyRDOsLH6F0U+RZtAYWOK3f038GfLDT2trjctL1dPdf0kOsRsEvFOy87o+OsBJvFwVOHX77xE755V/7Tn2X75SHKLygEVLuq+K8AqJZC9GVLjLcNzPobyYK0c16+2bruDsq9X2GbgrKKQsx205PFtGsSR0vF61rhOfU8uzklP4Vk7X3eHwNr6tINR6LWjav7tAC9KstKDYghAc/fopqJA34E1+RwIkv40Z1/x0Ni5Oa6KBY3LMSWv5TGOsRBJMWW1iUgE4MkQ+rEhgW4jSO7FAvkgFbL8RF2fJMMuoJaNP+PVvusJkRhLmSHZdG18VWz8D1Xq43xtgbsDOu/SRSJhjgWVYx7AibP8xcL9AKpIO2wauFEnveVKTWZpMzd494mA1OkdzlQGy11TVOw= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduce helper functions which can be used to read-lock a VMA when holding mmap_lock for read. Replace direct accesses to vma->vm_lock with these new helpers. Signed-off-by: Suren Baghdasaryan Reviewed-by: Lorenzo Stoakes Reviewed-by: Davidlohr Bueso Reviewed-by: Shakeel Butt Reviewed-by: Vlastimil Babka --- include/linux/mm.h | 24 ++++++++++++++++++++++++ mm/userfaultfd.c | 22 +++++----------------- 2 files changed, 29 insertions(+), 17 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 406b981af881..a48e207d25f2 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -735,6 +735,30 @@ static inline bool vma_start_read(struct vm_area_struct *vma) return true; } +/* + * Use only while holding mmap read lock which guarantees that locking will not + * fail (nobody can concurrently write-lock the vma). vma_start_read() should + * not be used in such cases because it might fail due to mm_lock_seq overflow. + * This functionality is used to obtain vma read lock and drop the mmap read lock. + */ +static inline void vma_start_read_locked_nested(struct vm_area_struct *vma, int subclass) +{ + mmap_assert_locked(vma->vm_mm); + down_read_nested(&vma->vm_lock->lock, subclass); +} + +/* + * Use only while holding mmap read lock which guarantees that locking will not + * fail (nobody can concurrently write-lock the vma). vma_start_read() should + * not be used in such cases because it might fail due to mm_lock_seq overflow. + * This functionality is used to obtain vma read lock and drop the mmap read lock. + */ +static inline void vma_start_read_locked(struct vm_area_struct *vma) +{ + mmap_assert_locked(vma->vm_mm); + down_read(&vma->vm_lock->lock); +} + static inline void vma_end_read(struct vm_area_struct *vma) { rcu_read_lock(); /* keeps vma alive till the end of up_read */ diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index af3dfc3633db..4527c385935b 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -84,16 +84,8 @@ static struct vm_area_struct *uffd_lock_vma(struct mm_struct *mm, mmap_read_lock(mm); vma = find_vma_and_prepare_anon(mm, address); - if (!IS_ERR(vma)) { - /* - * We cannot use vma_start_read() as it may fail due to - * false locked (see comment in vma_start_read()). We - * can avoid that by directly locking vm_lock under - * mmap_lock, which guarantees that nobody can lock the - * vma for write (vma_start_write()) under us. - */ - down_read(&vma->vm_lock->lock); - } + if (!IS_ERR(vma)) + vma_start_read_locked(vma); mmap_read_unlock(mm); return vma; @@ -1491,14 +1483,10 @@ static int uffd_move_lock(struct mm_struct *mm, mmap_read_lock(mm); err = find_vmas_mm_locked(mm, dst_start, src_start, dst_vmap, src_vmap); if (!err) { - /* - * See comment in uffd_lock_vma() as to why not using - * vma_start_read() here. - */ - down_read(&(*dst_vmap)->vm_lock->lock); + vma_start_read_locked(*dst_vmap); if (*dst_vmap != *src_vmap) - down_read_nested(&(*src_vmap)->vm_lock->lock, - SINGLE_DEPTH_NESTING); + vma_start_read_locked_nested(*src_vmap, + SINGLE_DEPTH_NESTING); } mmap_read_unlock(mm); return err;