From patchwork Sat Jul 8 19:12:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13305702 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7F890EB64DA for ; Sat, 8 Jul 2023 19:12:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1FDC8D0001; Sat, 8 Jul 2023 15:12:18 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD0B06B0072; Sat, 8 Jul 2023 15:12:18 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C97868D0001; Sat, 8 Jul 2023 15:12:18 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id B9D866B0071 for ; Sat, 8 Jul 2023 15:12:18 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 83DB8C01F0 for ; Sat, 8 Jul 2023 19:12:18 +0000 (UTC) X-FDA: 80989390356.27.23B646D Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf04.hostedemail.com (Postfix) with ESMTP id CEDA940019 for ; Sat, 8 Jul 2023 19:12:16 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=e+tppejd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3D7WpZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3D7WpZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688843536; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=JxWcwQCt64OXVUnpGWZTSfPA4gPHWHeYyOn+4bzO86s=; b=SsN1TOpBl8oqC81vuE2arvyzMZM8c8wHRpoRmP14KUb+R0VhYd/nefDxdeCYuJ6Gazy9B0 SEd8d9389CFl8iLiHRE77lzONtioXEu01VUPBppUQ9U51EI9Rg0tNKfoqum52HMxHr/eGv jrY8ZhKMpWlUOHVlV9IBkX9NHSxIBB4= ARC-Authentication-Results: i=1; imf04.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=e+tppejd; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf04.hostedemail.com: domain of 3D7WpZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3D7WpZAYKCEw685s1pu22uzs.q20zw18B-00y9oqy.25u@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688843536; a=rsa-sha256; cv=none; b=NR3W6ieJBZX7w/DYt+NGUepUHlkECpw8KQzxA/4WmxfnrjJZLTT1818nZS0gqjX/32UiIb 5Huj0yCcNtl5eT2HiqBTxaAMcL5o0fEmwusRcxu45Ng5B6Gvb1MOgwhRGzgyv+TVI09LAJ IETawT/rYzanppen8fFH3JPVpkYl2hY= Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-c64ef5bde93so3496346276.0 for ; Sat, 08 Jul 2023 12:12:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688843536; x=1691435536; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=JxWcwQCt64OXVUnpGWZTSfPA4gPHWHeYyOn+4bzO86s=; b=e+tppejdoz5kN8B3C5WlD+0nrZGu/5EqfAONbG0hP/lWZ7G+LSD9YpK/YdiONHY+ET COyN68AHpyc5cvZcCJI9Kdj/VXVTOhzXyqaYxVOU/XE8oiqZCtvDT9Ohjc4lwpKs8uIs Q4SGdk/Hg/xVzMRwTNMA5Eb7fUdiGxUNFD+tTkYwspdAJA3H1WkY5sp8/o9lg62jTXI9 vPMevfc26zzweXBMLZnrlY/VDaPVf886Cj6pjLGQXsKtCnCzJ7R7L8Gjg1qP169pUZdf 1fQiHyVHhW2J/bG8DUWZndT612qIhsJaOJnKsT3WA5FG5hybdpqBg5J7YK8LJ6GcI8+m tGNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688843536; x=1691435536; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=JxWcwQCt64OXVUnpGWZTSfPA4gPHWHeYyOn+4bzO86s=; b=FzNFQU6q5mTvLeu3NBUPRqTQApqesWPdpW+Kk9rgrJa8mM21uhnWws/IsD+uZzs6++ 4PpvyYN2pMJ3AoypVIWHq8ErIUosxD8KHzbk9B2PG5tbmpIDzOaSqIee7FsxQW5tpFAW F1n7C67eQcngrV4L/vW3Atoj4RXCjz/NHj+A3+I+JSg1r9srzS3BAUchbPzPeJB1WaBp c1HIvcNg7RpIFHxUo/H+1Mohz4IqUCW31kPCl6XJK4Ej5AMYAjnvMANiNTvCs36OH3EQ 0UZhZpsADY0c2cRoKXQNuNCKf7Ekj0X8fu2AciNbX/bfPkX9qbmyndWS/GEh0Rja/O1p 8cBw== X-Gm-Message-State: ABy/qLbRx89tkPqhF8kHhpgJ9W4CD2/OOp0JAWzOMqWaBCPuTZ/F82on Zu6Y/textV5058Oa2vO5Qy+bXk0wG1s= X-Google-Smtp-Source: APBJJlHhdNayqx2O2N+8ynu4MlCffNN+ZnIRARK/gUWveaZn8/1xJL36UJUekD5xd/lYzHt6QBE2wx7GXfo= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6f0:5193:79db:25b3]) (user=surenb job=sendgmr) by 2002:a5b:10e:0:b0:c39:d6f6:481f with SMTP id 14-20020a5b010e000000b00c39d6f6481fmr73329ybx.10.1688843535891; Sat, 08 Jul 2023 12:12:15 -0700 (PDT) Date: Sat, 8 Jul 2023 12:12:10 -0700 Mime-Version: 1.0 X-Mailer: git-send-email 2.41.0.390.g38632f3daf-goog Message-ID: <20230708191212.4147700-1-surenb@google.com> Subject: [PATCH v2 1/3] mm: lock a vma before stack expansion From: Suren Baghdasaryan To: torvalds@linux-foundation.org Cc: akpm@linux-foundation.org, regressions@leemhuis.info, bagasdotme@gmail.com, jacobly.alt@gmail.com, willy@infradead.org, liam.howlett@oracle.com, david@redhat.com, peterx@redhat.com, ldufour@linux.ibm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, regressions@lists.linux.dev, Suren Baghdasaryan , stable@vger.kernel.org X-Rspamd-Queue-Id: CEDA940019 X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: jczh3b7eh6qg5caw1pdmw93m96mb3hx9 X-HE-Tag: 1688843536-393972 X-HE-Meta: U2FsdGVkX18RF5xDt0hSpNsG2gfRAEDo27MTFtvx0Pou8OWESuFgcwPFC28z4ur46TxwDF1Md3RxM051wD6qG434+svPiPin2LGwurMzrCMQNC9qMyrjTHbEdMrKyqjZ9LRG8u7j4Mn4u0ssuJlU/1H08g/YZrri2f7eySQAMS0FLVKgWzavfUZ/HIN4YrbdWcN20ZnbiKZoSeS/3Pd17MXJTldAbfusjcrIazLAPo9N+tsZPZVIs7lxxdvFkbK4N+DtKTu+th7/QMOnE1pUq3lXwvuN99oImnegIdNubLoBrw3RZ9esOQAS8qlYpdwUfTCkL59km5uOn+ekz1ylvWwB1PaovRPgZWDMaXfsZWU3kh+Ir3Pk5bdwyXbGyJ+i7joa6j5CzpKUfI3uloZuXYN0gmyBl+eNayXY3d7b1bzuGCXHWpNTob3nwQGanQDEwi4oaJgMHyZUopbNvKYyld3rjKTj8DJtIc2sBSojI661o+hhC3OWsbxspnGRsuvbIWmzvTNxWAae0l63n9xdzMVUog4lAd2Vv8x/s8/HHUyIN+dXt/CS+Hn0KkuMAjdmRnWm2R4cfw05EQGu7ShP0okXGPvIeG8CccOFF7a7tYpPudQ8RIZFEGAaeMfH7jIec+YWZ6BxRWtvtMRBkdchdmBnU6bYmaUAfCHaWzH74Hs0WvBVgikiELv+c/Mp5ghG7TJnLvxTjkm8vTH3Fioehiu4Z9S4DBmk3YLEbnLEyVTxCsyOHhDHD+Q5GmRmfKRBcTbdrfgB/cereDxnb6o8WBH+o/b5yvkhCe4b3N1Kc68fYrl7xJc4jBEBqrPR8CdWCaLJaQMXA2vh3MygJj70OhiMrd7ZOyIC0Mx7NfWD9CxSoeIIypxyBUf0DV+xMzkdISq2m3HNDCuF4PUWNzV2Big4bVFVPe+eJqiv+2v5D5t/LO2tw41Y2IGRpcRNzb7K6wob5Jx2hoxlmGIYO4c l00uTwKK 91gIZc/YyM61SQbIsXYv/7X9VYTF8VNNGTeuljo67uQHWmQfzK91rPwJX3cfsHInal4jkL3sUKhE+1hM5/fmRkQbniAGmMYbWL/RXEE5W9AIJtbETUOFBgp3KlpL/DFO+2lIgLE9cszYZXnU9g9bCZ6/8TWVeRLMxMXXYheZOVY/ZQnZmlFCX9Q6q9fGd+KTTFAvIS7gJA4E8JH3xHvxS3B2E7JIJB8xRJUvc3+/e/YM413ovqK8DKHZZ4e+4W5yiIGfBrEcQ+19nt6vcnl5ng9w6FxHxxRu699zGXFz5kYkR3UK5WfE94HUtc5X8WnC19DqsI/BfgXzjVvKhmnx9cJ9j84jqEoBu7Gkfnj34NoTwaLIFvyRYsQrkkD+pyxTzKGKr6cs+V2vrphqIFguaxiwv02Kxpl2NCWFA21GJ2KPd9clxs51s0Ikt5xBijLbEd3zQDnAztW9dHNR1MQTe3lNfQI9A9kGaqv3Mu1YMWHV0XXth4Ei1r28zL9rZDq2S6DT0kbmXr+52c0Q+n3utPL16kScahCifk6R6Vy4PQRWiRcUK7rfZf6QSGNmW43ABthfDF35tZhTqOuSgH4wH8/yoWqazpV3XFlp5j3QixioeRgttnsIU3gzMH/Uq/kR9edSCuymBswyeBstmbX7mkwF44Xv7VOFbrJpx0UzkjOCToRnIqcjxXeax9tDcRlY52pmzFO/3TbInj+DYwWREDyHxspBX7jNbkPXYZa9J36dYyhY= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With recent changes necessitating mmap_lock to be held for write while expanding a stack, per-VMA locks should follow the same rules and be write-locked to prevent page faults into the VMA being expanded. Add the necessary locking. Cc: stable@vger.kernel.org Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index 204ddcd52625..c66e4622a557 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1977,6 +1977,8 @@ static int expand_upwards(struct vm_area_struct *vma, unsigned long address) return -ENOMEM; } + /* Lock the VMA before expanding to prevent concurrent page faults */ + vma_start_write(vma); /* * vma->vm_start/vm_end cannot change under us because the caller * is required to hold the mmap_lock in read mode. We need the @@ -2064,6 +2066,8 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address) return -ENOMEM; } + /* Lock the VMA before expanding to prevent concurrent page faults */ + vma_start_write(vma); /* * vma->vm_start/vm_end cannot change under us because the caller * is required to hold the mmap_lock in read mode. We need the From patchwork Sat Jul 8 19:12:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13305703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50D76EB64DC for ; Sat, 8 Jul 2023 19:12:22 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E1FF28D0002; Sat, 8 Jul 2023 15:12:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id DD0CF6B0072; Sat, 8 Jul 2023 15:12:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C98CA8D0002; Sat, 8 Jul 2023 15:12:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id BCB2F6B0071 for ; Sat, 8 Jul 2023 15:12:21 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 863DB1C7C0E for ; Sat, 8 Jul 2023 19:12:21 +0000 (UTC) X-FDA: 80989390482.03.E111F7E Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf28.hostedemail.com (Postfix) with ESMTP id B75CAC000F for ; Sat, 8 Jul 2023 19:12:19 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=rDyzKJgv; spf=pass (imf28.hostedemail.com: domain of 3ErWpZAYKCE89B8v4sx55x2v.t532z4BE-331Crt1.58x@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ErWpZAYKCE89B8v4sx55x2v.t532z4BE-331Crt1.58x@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688843539; a=rsa-sha256; cv=none; b=0c9Xn5EH+GY6qksAXeFsnz+lOLcH+l5KuftaQ0yGB0nWtTmyTUX9/76haNbjL67duy2xZb FHvVX98iOOyVxO8W8aIHgW4nhAxCt4i7UfkHxTxqwZTj68bT53gjtsuf9W/a/mls+Dk/jS BaIXRuTTKYiN42fHcQxkY3hV6bPJ+To= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=rDyzKJgv; spf=pass (imf28.hostedemail.com: domain of 3ErWpZAYKCE89B8v4sx55x2v.t532z4BE-331Crt1.58x@flex--surenb.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3ErWpZAYKCE89B8v4sx55x2v.t532z4BE-331Crt1.58x@flex--surenb.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688843539; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Z4rAZiN8CFc+wkQPvmF3sDwgaLYdtc/sQi/Gu45S9YQ=; b=3qvia3pl4tTjgj19aVSzg9+PxrdFN2Jf+Amrs4iIp+YhiYnAGH87znb4Cqy8QmYWwBa8o3 YWX8dUHlHdJIhOSUeoQgFJHB7woa8m6ANyO1hGNGvsLdEelyzXiVF7LoJTxytLFzLS/e5/ JqcslAbEbcHbWVN7N9QruKd/eC10fRs= Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-c6dd0e46a52so1950081276.2 for ; Sat, 08 Jul 2023 12:12:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688843539; x=1691435539; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Z4rAZiN8CFc+wkQPvmF3sDwgaLYdtc/sQi/Gu45S9YQ=; b=rDyzKJgvSb4iVFr4MU6yl+skVCEaDOmrNnUTQxW2AR6lwndLcFZirFs7K8NVbOet/l Hz1ezj81fpfDhby24QFHC8W4yNrvRhOZmkRPzGk0HoDpmSzVRvUFh46uDhRrvrsHj0T+ bWfsgAAuSrJUkxovFXIvpjT+3mNLViAPukrwkqTZEVh67uYSSdkV3/RB6xiSA1ML8ef8 x4SI1Wv4IyWtXrra9hN2OOMwSs1hHi6cndf596Frps7d08xkQhXDxEudvyECcS4PhaDT YUU3z/ec3eCdOsvrUHORdE6FKycuZJ6n6CZZT2tr9aAc7ZTwuikUkUWUt+zO7jA28nYB wf5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688843539; x=1691435539; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Z4rAZiN8CFc+wkQPvmF3sDwgaLYdtc/sQi/Gu45S9YQ=; b=JAfcL9zFlRp/jpnAxZU5Jaio6Ek6+7cOUFEiuJz0In9A9X7w/z1OFOb+v6TKcM4Bvy NJIggNirN1jFuoCVvo+IYrCu/mNDCEzStn64XDvgAEqxVdzlkkGjeCzlMWm38b76J7ZN 26EJO37RIlVeHJAG4POx4jYchdlrdwMQhsjdCC0vHBkqcCD5OYkKN+W5W0hKMueOv70t Nh+VC42zHZotrUyrsjkSn/h834pfgaClFtk39wjc+3OtjUm/2Zsdo6OZUlbHrgKW18xK gmUywjG3jbpj3AfaPhMTzpEezjA9d7bhkeTFmwvtefKSPwYBbsVOW/k/uPgn4LlU2eL5 i4Uw== X-Gm-Message-State: ABy/qLY1sO1ZTlGYYFQhdW8XEWdYewBr+ScEqaR8yn9zPvu8Q6ZbjLi+ Dr0kGKCXhC/fYzpyuAkh0hPffeKOaF4= X-Google-Smtp-Source: APBJJlHlFE3mHfSxux3qV7DD1eHyTN3Bj7OluqfqRLk3Ndt0avHOdBy2KrtMLyVg6El02lkP8km17cLgxIM= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6f0:5193:79db:25b3]) (user=surenb job=sendgmr) by 2002:a25:b05:0:b0:c24:7d1c:6145 with SMTP id 5-20020a250b05000000b00c247d1c6145mr69068ybl.2.1688843538817; Sat, 08 Jul 2023 12:12:18 -0700 (PDT) Date: Sat, 8 Jul 2023 12:12:11 -0700 In-Reply-To: <20230708191212.4147700-1-surenb@google.com> Mime-Version: 1.0 References: <20230708191212.4147700-1-surenb@google.com> X-Mailer: git-send-email 2.41.0.390.g38632f3daf-goog Message-ID: <20230708191212.4147700-2-surenb@google.com> Subject: [PATCH v2 2/3] mm: lock newly mapped VMA which can be modified after it becomes visible From: Suren Baghdasaryan To: torvalds@linux-foundation.org Cc: akpm@linux-foundation.org, regressions@leemhuis.info, bagasdotme@gmail.com, jacobly.alt@gmail.com, willy@infradead.org, liam.howlett@oracle.com, david@redhat.com, peterx@redhat.com, ldufour@linux.ibm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, regressions@lists.linux.dev, Suren Baghdasaryan , stable@vger.kernel.org X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: B75CAC000F X-Stat-Signature: p4b8w385pp34jfpkrt9oymxa89hpk1ap X-Rspam-User: X-HE-Tag: 1688843539-394212 X-HE-Meta: U2FsdGVkX1/dEETuYrfOcbWUM+g7egmglMM/8u9vbxu0xd78RNyPScLBsf84XAPwghqlNdIadOMtzCeirMjG/TqG3nCYI7TrgfQpZRclbcU5UKeAVNvL7CxesKkhx5ZtwnOmADBWZJkdaphJfHnSeMdXj5w3AeAD5oY3f9NtUdm43FHYksj7dCTYCQshxfLDT7vnuc7Sh0eQSUNwWeFX0I6HKTV6T5GP8Xtat8R/YK2CIwLCpZXIL90jkmfsU6RgBNmtM9ZaG63cdV0FciZuBly3AtGHYuyNhcPTIb9/6og2nbEH+c8UxmKEsGHDjoNR245oZ2Bvy8FQGFsXXiHtv8YTmItpYNLC4FVvyLRpBbHWcVuqoSCfLguwqV5hq3chtlECgmhnLMknbm+QwpWmy9vS2ehW/rf1qccGcr8RxRPR1MJyBOP54PdIQQED0wCD0j7cUYut/O4QnxCSCy04VoskhvXPS6wfOizQdEAAakHSUI5YStqwQGaP59vkw4j3w1ww7cs8iA1w82lq78d644Dg1cZvQYHud5sHrejLFfNYzyQ8UHHxTgFT9kG6CpQjRwol6YU8w+Wacrcl7xHTs8JvHaKpc2jgw95Q5E1tLv3X5OknAUPOqOL6mH2s/ApAcWl1QLBE4IXu7ZEcviLu0RiPr9J02dMFhTy76dCmm8lfLJY14wCb+6PLY3v+EP7n1WcfppELVW20C9+E1iZ+thiIORrD2SnoHEDIIZ6WRlpyj8QKSJPCWvigjKLNyLywDFGZFnklydOMKtnn7RzudCkUCwnLsncSUG2yutvfWT95brtHALUqlfGh5bbJZSkY3HRsrk2+f7nWVezI0CMCbXcNaJVMA7xfSXOCByjRag07862/gei44vR80ffJb35K4pLZzz60vsJ6yDmudLE5ER/yfuOkSQZht/Mi5SmmVeiU+H6Ty+msshHs2oxjD7MMbq2C40N4PG6+3ABpAZf LuXmjbbs 3vcsETvDWHxl1xVe5D0QZtFR6+0A2ZhtgFrpN+em2BONyXKOp+LACUMJK8+WdWoR5UAljc23gJZ7SqtaCC75B84GWjPyTLLu7D7ZK79uoW0c6vVFO+kDlFS7j9HP2+scR0tXzwyDsdvFmKQ6GFNWGgscozkNdEWqPJ3L83zDv5HxBsza5gipDX5x84+fydbc5ufxFRZ9/LyMVKq5PqanNCik4selAq1rkJNVXM5ndXSeUKPBF1b5ozWXoEB6MBTo3Pm4vTBaM3zOol7UqUBgLP3GywpgHKfFcyMroGeJObVfJuEOeDGZA0rNZ+v8ORWJlrfRJZZzBI1bwVc76BBgH1l8zoXPjbDwv0ct72PmTjN+5SSVYpqXbc39f9xZLDag2+VXiJEEaQZ5g8P4PHVoj22gCe3yUr+A55Q2OVAVmYd/S7xRbZZhDTr5baZJCPqtCcGL/guhiW6BvF7jTKdrxYK93L6paG7xXY/A7N0XrAkNxdfKR4Cvoy+11Q1G7RkWI3wAh7YtzqkjbGiwJIUhvqdpmaFV4SuJsOJclMboqLADBOnKsKsvoPSifv7SlTIOf8TvTOESm6EWhkcashG6M6u2B6HJp8dF/Hmj+v7vK33hbdkDOfaksWedATOG8b13wYTIEY5wS6dr/0iLwVfUUXCYchgoZGkpP6zGooImS38gAaWMpGLLrpkgCDzb+Ql9BTOMOXRi7m/ZGCw2CSRdcE9zH6W8jNHt0nnVuZ4rqEoVGtBE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: mmap_region adds a newly created VMA into VMA tree and might modify it afterwards before dropping the mmap_lock. This poses a problem for page faults handled under per-VMA locks because they don't take the mmap_lock and can stumble on this VMA while it's still being modified. Currently this does not pose a problem since post-addition modifications are done only for file-backed VMAs, which are not handled under per-VMA lock. However, once support for handling file-backed page faults with per-VMA locks is added, this will become a race. Fix this by write-locking the VMA before inserting it into the VMA tree. Other places where a new VMA is added into VMA tree do not modify it after the insertion, so do not need the same locking. Cc: stable@vger.kernel.org Signed-off-by: Suren Baghdasaryan --- mm/mmap.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/mm/mmap.c b/mm/mmap.c index c66e4622a557..84c71431a527 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -2812,6 +2812,8 @@ unsigned long mmap_region(struct file *file, unsigned long addr, if (vma->vm_file) i_mmap_lock_write(vma->vm_file->f_mapping); + /* Lock the VMA since it is modified after insertion into VMA tree */ + vma_start_write(vma); vma_iter_store(&vmi, vma); mm->map_count++; if (vma->vm_file) { From patchwork Sat Jul 8 19:12:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Suren Baghdasaryan X-Patchwork-Id: 13305704 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 891D9C001DC for ; Sat, 8 Jul 2023 19:12:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 23B108D0003; Sat, 8 Jul 2023 15:12:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E9656B0072; Sat, 8 Jul 2023 15:12:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0B2548D0003; Sat, 8 Jul 2023 15:12:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id F11776B0071 for ; Sat, 8 Jul 2023 15:12:23 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id C4D421601C0 for ; Sat, 8 Jul 2023 19:12:23 +0000 (UTC) X-FDA: 80989390566.18.1CF695A Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) by imf21.hostedemail.com (Postfix) with ESMTP id F389E1C0006 for ; Sat, 8 Jul 2023 19:12:21 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1k2h2Lzi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3FbWpZAYKCFICEBy7v08805y.w86527EH-664Fuw4.8B0@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3FbWpZAYKCFICEBy7v08805y.w86527EH-664Fuw4.8B0@flex--surenb.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1688843542; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oTV0xihdJDZ3LmT5b0VvbfWVPo2l/pYZxiWhaA2z/Wc=; b=E++D8xqdCiOBWUEJn3Vz8khfFy2l2oPnE5KPBZ/qclmbQMZYbNhjily4mhRgsD2IPr5xLE X5zRQhVB3mjUq60Ma+EF2gBhGZoEGTWiwiV8jHbNQpCc+DbbuRtGnCpPgKuipNE8kig0AV GYZSuqbHJKRBeHmtFrWwTECRVKHk8P4= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=1k2h2Lzi; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf21.hostedemail.com: domain of 3FbWpZAYKCFICEBy7v08805y.w86527EH-664Fuw4.8B0@flex--surenb.bounces.google.com designates 209.85.128.201 as permitted sender) smtp.mailfrom=3FbWpZAYKCFICEBy7v08805y.w86527EH-664Fuw4.8B0@flex--surenb.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1688843542; a=rsa-sha256; cv=none; b=YPeKvK13D/vQh8NYfP737vrSk+Fy+d8kgFePhKLOvG0L6lgXP0hwopn/lxqccQM9kG2oBu 0fTMSf6G3kUaxfZNCwip9huYQ5/zerDcK9AM+E0Bz36gfLywR3ob4p8F98pZ0G9hIVX4GX jZQGDhTSjwa4ii4QumP1tL+aHz+QD/g= Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-573cacf4804so32511607b3.1 for ; Sat, 08 Jul 2023 12:12:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1688843541; x=1691435541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=oTV0xihdJDZ3LmT5b0VvbfWVPo2l/pYZxiWhaA2z/Wc=; b=1k2h2Lzi2Y7Zso2QCXABxD6tKxYBi69Od4eg/BJzd/oZKr/yo1QC9zYTy+F7bkwUsr MJ4Cs8yuMwfbeH9sKlCNAwVo49wzFYe05nOmOIFdL12CZV8/kFSu5wJrvCOBni6b6XLW llu6z1SxXl9jUD2la6uHivLOfnp3yDTzL+wbIb/uZmz+uMYQ0KGYXgadTEUmWm9Uz2p8 OvPqWDoLPuHzUoISVlQIJ3HwqRiXh8i6S85Nr9mydxCOVHJ3byBf5nAejApz8KtI+7VG nVka+9EAa6kAPb4o2CIYQlx+qV3ecK270Nq5eVaYu0hTPSFmvNlGf9LGupgmu1sf8Ore 83wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1688843541; x=1691435541; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=oTV0xihdJDZ3LmT5b0VvbfWVPo2l/pYZxiWhaA2z/Wc=; b=WEfG8yUGYVF/t0ArmDcWaDRaYp9w0xhrZHW319xeoj5v0c7HJovNrgl/hpUaBl2cP7 Ziv2a7dBkhvq2Neuq3fJo0xwp99HfKTDl65kFatoqDInA7f1bC4k1Zrwf6op6l7ehKOs hQU+Zx1SGT5trk3Jc4wI7+/z5RySOyJz1Y+Sijwbw/U4e/ouD9JvspDbjsv+3kgzF9ny GbmKYlgbH0EyfkK5djl8FXXRyyEcqa5jWLK8kPBBKVYZAdmGOHfhJoli+mEZ/C6nLJhG uD1EINS4V8aT4n3YkL/zSt0dqK5nuuhezq6BROdS0Qtw2PIETDSGF7exzVlBaoTXpNp9 7aFw== X-Gm-Message-State: ABy/qLb8FXtpu8fWgTDXci2JeCzl5tUfILGX5grK84HW83jr+jUoOtls X0kQqi5pIf8Ie40sUpQNbtmQQxQUrrA= X-Google-Smtp-Source: APBJJlHVrRFpZyyO6dwy1Gjn4/30Rl1D+ek+P+dzMKAoVpdfL0WWnvznDWyGgdeVaC3RV2FWAanbuAUiF7I= X-Received: from surenb-desktop.mtv.corp.google.com ([2620:15c:211:201:6f0:5193:79db:25b3]) (user=surenb job=sendgmr) by 2002:a81:ac20:0:b0:565:b269:5ef7 with SMTP id k32-20020a81ac20000000b00565b2695ef7mr58334ywh.1.1688843541040; Sat, 08 Jul 2023 12:12:21 -0700 (PDT) Date: Sat, 8 Jul 2023 12:12:12 -0700 In-Reply-To: <20230708191212.4147700-1-surenb@google.com> Mime-Version: 1.0 References: <20230708191212.4147700-1-surenb@google.com> X-Mailer: git-send-email 2.41.0.390.g38632f3daf-goog Message-ID: <20230708191212.4147700-3-surenb@google.com> Subject: [PATCH v2 3/3] fork: lock VMAs of the parent process when forking From: Suren Baghdasaryan To: torvalds@linux-foundation.org Cc: akpm@linux-foundation.org, regressions@leemhuis.info, bagasdotme@gmail.com, jacobly.alt@gmail.com, willy@infradead.org, liam.howlett@oracle.com, david@redhat.com, peterx@redhat.com, ldufour@linux.ibm.com, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linuxppc-dev@lists.ozlabs.org, linux-arm-kernel@lists.infradead.org, gregkh@linuxfoundation.org, regressions@lists.linux.dev, Suren Baghdasaryan , Jiri Slaby , " =?utf-8?q?Holger_Hoffst=C3=A4tte?= " , stable@vger.kernel.org X-Rspamd-Queue-Id: F389E1C0006 X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: rrxgyypubd5inyjbmfzr15bgix79qx58 X-HE-Tag: 1688843541-975679 X-HE-Meta: U2FsdGVkX1++Gmv7u0mQvjSgjiq/x2zf+cYfyNTJ3G8oKmdrxx1yB3MH/0n9rQcYnON4jxS0zLtfycpXPImfWJ98v00/C+5DdzdYf3ppL+8Qs7ey2OEWCmasrFVHwVQJQqS+ZAHuD2A6I8s8Ro2LuwNPVK2WJGf09FMRw8Yt6tJiTyLYRQfxS7n1M0XYBneFuiq6Pyh3L/sa6AFa0nAGT3hEwhytEKlIQHZXDXg3U5txvAqMOMkLgqvMNCA2xzTo53OTuAf3bCmY+a+iVnmLFTuzcIrN+dawRsPp8oTlpU4mUAcchKGnlwhHEeAPgvhN8mZpBOUg7DsJUWrw8Ta66Y6ME2jN5K5f2NtS9n3+/JQR3JRurzv4RX0d7bPr8PcKsaVVbXbLLNxAixzC2wodktAS2bxU408ZNLqm7WBCSt/+9v3U1KuLA8iFmn5m5kLS3EAQ4QGPVhA3m212tI/Tf/xhVyvqiWUyGAwvp27xVme3PatuXOVI3ZgNacq2lK1CZ+QhfL0gHWZueGV2fc1l2WzLyrhLkEivFmP3vqRtEjqcJQ1jHGIQ5/ApZWCRJR2LHFeLw5hBYE+w6RdBxLN1udlk1QecdW73XEGgu6Km13CKRactAD41Qm/z3LrZCPkaoKK/OqadjtzGvD6CUZceLxnpzLjzowCWHue9PbjtNmJkys0chwv13/KjZ1K6rjDfC5NepteTkohleBdIadRDAAhFpoSOg96Sp8kkyDSQiEIPryfO1RskWvjVEsaXgC49PKUVxxOc4J9EtKzlhelemOPFWUk5FcIV1PZru4fsq1T7LVhmcY0mUx2AlDCI6yowTbjgBLsLrT4pEgPyNbEF3DGkOHVmmtug693sG5cEXzxB3UhDgjnGqCtnp88rnnv5pXOeCpJkra8ZuqpqlpU9qbNwWLdPNS1cGtmhejKUxqSYPrr7P0gWDjccFS8ka5P+wgrcbJLXK8zzYNWEfLY hQ0ompOz lvijCMnreoRGWyz+syzQhB8Z1nkJR7oJM+hwF88kgiQAeeWadZ1a+5EIublcwJl+LBFAMgXdvHk565XDTPfQPA6H51GyS1XzfBx0snxiakgY637D4JP0mkoEaAPNXfD0qREJ42VeITfqpeHM4dGvtFP89TidazU5Nl1b86tr6sifcr5MNFCqYn6ciwtBDxE31zTQYJc2d/vBqmb6DAuABlQYNJ1zjsQArwwUd1zKaaijmvblvUIOjPJM8Qc1DQtZaUNSNzNNirtOjpT3YUY4Db5BSlhFqgF2VUOGTyQvjBtLsANkxlf0JHQWfk5gMHQIXSaVtT/o2wFZFhWeeZDbvPKxBJL2QYzHeBmG+y+gwyimlztP4g47tQ5Tl/TNBeIuu89HrN2cKXg2a7n+ROhrHR6j6DfO+qrA2OfzGnP2jtVpcqeECVn+C6CQumflDtUj+q4VUwWWS403mQOP1Rp9c5UzOU8BFYbZCHgCkv7XOtXsO3mhm919IMn03kTO/+IBinD+4O318kTghLZIYkXL6QAAX+UywZOgGE2Rjw5pzbWM4yndY7+8ZxqwQjPfc9ARPmz4B19f4pojLlA9K7bOnDZsGLF1iMf7/Axh19q8KLjONnhEZ2dV1ccTi6a0hIZcpoHoN/x4aQwONPbsLN3j6hk+Wbe3cGOLl8ox3owF97h4UdJEuPliCpHN7x9xhZ4HxPjNWqc7yMKpmeOYSmfuGLh+y+cWj2l3Oq63lrsXGnwBPNNtjJPMCGQxwqrgndVwtgEMCM05rKoEZwhZtbGZYw5cgaQVnO7Tkw3kmlcnjIsD7LljeNBaEeSvv7fGxcRDMRovgRY01YXSC7RT2YP1jcLY0oA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When forking a child process, parent write-protects an anonymous page and COW-shares it with the child being forked using copy_present_pte(). Parent's TLB is flushed right before we drop the parent's mmap_lock in dup_mmap(). If we get a write-fault before that TLB flush in the parent, and we end up replacing that anonymous page in the parent process in do_wp_page() (because, COW-shared with the child), this might lead to some stale writable TLB entries targeting the wrong (old) page. Similar issue happened in the past with userfaultfd (see flush_tlb_page() call inside do_wp_page()). Lock VMAs of the parent process when forking a child, which prevents concurrent page faults during fork operation and avoids this issue. This fix can potentially regress some fork-heavy workloads. Kernel build time did not show noticeable regression on a 56-core machine while a stress test mapping 10000 VMAs and forking 5000 times in a tight loop shows ~5% regression. If such fork time regression is unacceptable, disabling CONFIG_PER_VMA_LOCK should restore its performance. Further optimizations are possible if this regression proves to be problematic. Suggested-by: David Hildenbrand Reported-by: Jiri Slaby Closes: https://lore.kernel.org/all/dbdef34c-3a07-5951-e1ae-e9c6e3cdf51b@kernel.org/ Reported-by: Holger Hoffstätte Closes: https://lore.kernel.org/all/b198d649-f4bf-b971-31d0-e8433ec2a34c@applied-asynchrony.com/ Reported-by: Jacob Young Closes: https://bugzilla.kernel.org/show_bug.cgi?id=217624 Fixes: 0bff0aaea03e ("x86/mm: try VMA lock-based page fault handling first") Cc: stable@vger.kernel.org Signed-off-by: Suren Baghdasaryan --- kernel/fork.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/fork.c b/kernel/fork.c index b85814e614a5..d2e12b6d2b18 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -686,6 +686,7 @@ static __latent_entropy int dup_mmap(struct mm_struct *mm, for_each_vma(old_vmi, mpnt) { struct file *file; + vma_start_write(mpnt); if (mpnt->vm_flags & VM_DONTCOPY) { vm_stat_account(mm, mpnt->vm_flags, -vma_pages(mpnt)); continue;