From patchwork Mon Aug 29 21:25:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 12958426 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67FD0C0502E for ; Mon, 29 Aug 2022 21:25:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D91186B0075; Mon, 29 Aug 2022 17:25:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CCDF3940008; Mon, 29 Aug 2022 17:25:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B6D26940007; Mon, 29 Aug 2022 17:25:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 9BBB26B0075 for ; Mon, 29 Aug 2022 17:25:40 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6B63C80AB0 for ; Mon, 29 Aug 2022 21:25:40 +0000 (UTC) X-FDA: 79853912040.12.F294252 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 2C0BBC0057 for ; Mon, 29 Aug 2022 21:25:40 +0000 (UTC) Received: by mail-pl1-f202.google.com with SMTP id t13-20020a170902e84d00b00174b03be629so3065914plg.16 for ; Mon, 29 Aug 2022 14:25:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc; bh=cPhW2Jsm5VYfGL+tNkfF3HbuaaNlLZ4rflaa6thsEZ0=; b=Ef83aGWTdm10niN8twSSpG3ds5UuJPZXIlmBxHFg2pzrVEdcovVnDh3P/RYOinFzyL 56g3s4OBiiWq4sYS4zyOn+I3sJdcQzzrlMjpsAwGpe/6DNb56LumMLV46gfcGkdOoroQ Xn2Ez1+n3LVumJ9ul/e3Fo4g/eC812QXSiR/GOAlZ/X37IBJZ8GhOEEF+zxhXhrBOhrA wyQ3RxsgNeUblE3TddZPmpCUfJPQ1JoHkMsqbRXnrreQjskaAOAntkxj5MpJLWS1ppBQ s3y364HBN4NsSi7BBvtTGWNjoBJVZ6VaguwCOOzVv7z+u5EVx5HkEViGicNoS6cS9car 5ukQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc; bh=cPhW2Jsm5VYfGL+tNkfF3HbuaaNlLZ4rflaa6thsEZ0=; b=vYlNP7HB5vyz262M1rgjnZk+931LL5PD7NbqbunJg32TcyFL50t33Lcoi4G7S5p84f YvbucZxmSSeZPbhgzFoeycfr+fLix72nzkflOOsGB4DDNTup911gpJBKAzFkl/QYikqY aRx9aVnZaGwOjw6RkNCB1FrhMwNbc+YYSZ6roZPw0yCnuIiofYDNaFoVhceX1rh1G5RM 1iqECVZjTlFeJyW9tYH0BrWZBFhU5ei9zU/RZiQvkcai3CgfdHwxmU3zXAd12sDYpcBW 73MtcT2bM77cdS/mdw4/bTM+dNkgsh0JZio6dLm1CLUdoXkvXWg4smptNycY40XNILxH 1gjA== X-Gm-Message-State: ACgBeo1JNUy3GDD4Gfv9ZRf2BR8kHfpMZzgkDj/4VHQhB+QyGRokKq64 /lZY6ekzvuczcxNVmotmOtaGOH68rqzJB2MFZAAbmXxyfcESGpCKToRem2HCa/edoX9cER6Q9GP sibXBMx0= X-Google-Smtp-Source: AA6agR4h8d6XU8wf7Rba89JuzvO6/M5QQD9xRgO2erAMFyngZx2nEIGpax2LUa1YM3lFKVFmr9HgbHhBg9U= X-Received: from surenb-spec.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:3e30]) (user=surenb job=sendgmr) by 2002:a17:902:694a:b0:16e:e270:1f84 with SMTP id k10-20020a170902694a00b0016ee2701f84mr17988146plt.89.1661808339339; Mon, 29 Aug 2022 14:25:39 -0700 (PDT) Date: Mon, 29 Aug 2022 21:25:05 +0000 In-Reply-To: <20220829212531.3184856-1-surenb@google.com> Mime-Version: 1.0 References: <20220829212531.3184856-1-surenb@google.com> X-Mailer: git-send-email 2.37.2.672.g94769d06f0-goog Message-ID: <20220829212531.3184856-3-surenb@google.com> Subject: [RFC PATCH 02/28] mm: rcu safe VMA freeing From: Suren Baghdasaryan To: akpm@linux-foundation.org Cc: michel@lespinasse.org, jglisse@google.com, mhocko@suse.com, vbabka@suse.cz, hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net, willy@infradead.org, liam.howlett@oracle.com, peterz@infradead.org, ldufour@linux.ibm.com, laurent.dufour@fr.ibm.com, paulmck@kernel.org, riel@surriel.com, luto@kernel.org, songliubraving@fb.com, peterx@redhat.com, david@redhat.com, dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de, kent.overstreet@linux.dev, rientjes@google.com, axelrasmussen@google.com, joelaf@google.com, minchan@google.com, surenb@google.com, kernel-team@android.com, linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org, linux-kernel@vger.kernel.org X-ccpol: medium ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ef83aGWT; spf=pass (imf22.hostedemail.com: domain of 30y4NYwYKCIQ02zmvjowwotm.kwutqv25-uus3iks.wzo@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=30y4NYwYKCIQ02zmvjowwotm.kwutqv25-uus3iks.wzo@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1661808340; a=rsa-sha256; cv=none; b=I2Z+/RnIqOctwSFwqtXI9UYrmO3SGiL40hdJ7QBvqaz6MIXwaGyEolX0Dc5S5bnRDFCdG4 MaGV783lBjDvNQ90/E3ySf1hMODwJ81FAh/m5uLwH+Hlj+RXD+7RvFDMxnMRDeORLI54fi AgNep79wVTl2VVaW9xIDv7A/OpIt7JQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1661808340; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cPhW2Jsm5VYfGL+tNkfF3HbuaaNlLZ4rflaa6thsEZ0=; b=LNbh6M4QSJ+E6X9kqqvNpo2WVSRXBf0UsrYHn8vcVkUs+fqOaoeljvnfzXeQlDz8oSQl4m 36R21AJoWyyU1JdyyYPx30umb7RtmUPyZnfGRjOId+TWrL6cvNQcQyegL2eA/kx6w7g3cA YDqie0OyFojyaxi5tbYRi+amoXprGuM= Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=Ef83aGWT; spf=pass (imf22.hostedemail.com: domain of 30y4NYwYKCIQ02zmvjowwotm.kwutqv25-uus3iks.wzo@flex--surenb.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=30y4NYwYKCIQ02zmvjowwotm.kwutqv25-uus3iks.wzo@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Rspamd-Server: rspam06 X-Rspam-User: X-Stat-Signature: fnen5yhah7c1h3acorayn6h99u61zd4z X-Rspamd-Queue-Id: 2C0BBC0057 X-HE-Tag: 1661808340-924395 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Michel Lespinasse This prepares for page faults handling under VMA lock, looking up VMAs under protection of an rcu read lock, instead of the usual mmap read lock. Signed-off-by: Michel Lespinasse Signed-off-by: Suren Baghdasaryan --- include/linux/mm_types.h | 16 +++++++++++----- kernel/fork.c | 13 +++++++++++++ 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index cf97f3884fda..bed25ef7c994 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -403,12 +403,18 @@ struct anon_vma_name { struct vm_area_struct { /* The first cache line has the info for VMA tree walking. */ - unsigned long vm_start; /* Our start address within vm_mm. */ - unsigned long vm_end; /* The first byte after our end address - within vm_mm. */ + union { + struct { + /* VMA covers [vm_start; vm_end) addresses within mm */ + unsigned long vm_start, vm_end; - /* linked list of VM areas per task, sorted by address */ - struct vm_area_struct *vm_next, *vm_prev; + /* linked list of VMAs per task, sorted by address */ + struct vm_area_struct *vm_next, *vm_prev; + }; +#ifdef CONFIG_PER_VMA_LOCK + struct rcu_head vm_rcu; /* Used for deferred freeing. */ +#endif + }; struct rb_node vm_rb; diff --git a/kernel/fork.c b/kernel/fork.c index 90c85b17bf69..614872438393 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -481,10 +481,23 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig) return new; } +#ifdef CONFIG_PER_VMA_LOCK +static void __vm_area_free(struct rcu_head *head) +{ + struct vm_area_struct *vma = container_of(head, struct vm_area_struct, + vm_rcu); + kmem_cache_free(vm_area_cachep, vma); +} +#endif + void vm_area_free(struct vm_area_struct *vma) { free_anon_vma_name(vma); +#ifdef CONFIG_PER_VMA_LOCK + call_rcu(&vma->vm_rcu, __vm_area_free); +#else kmem_cache_free(vm_area_cachep, vma); +#endif } static void account_kernel_stack(struct task_struct *tsk, int account)