From patchwork Thu Mar 14 16:13:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Hildenbrand X-Patchwork-Id: 13592600 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4F11C54E60 for ; Thu, 14 Mar 2024 16:13:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3EF50800BE; Thu, 14 Mar 2024 12:13:17 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 39F53800B4; Thu, 14 Mar 2024 12:13:17 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0BA5A800BE; Thu, 14 Mar 2024 12:13:17 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id F0C37800B4 for ; Thu, 14 Mar 2024 12:13:16 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 9ECCFA1411 for ; Thu, 14 Mar 2024 16:13:16 +0000 (UTC) X-FDA: 81896139192.09.81D1518 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by imf03.hostedemail.com (Postfix) with ESMTP id E8C9020012 for ; Thu, 14 Mar 2024 16:13:14 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HwdKetgE; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710432795; a=rsa-sha256; cv=none; b=pxRfcwGotUExrMsWpH5v8Xw0ehTcjE69C6sMcKzxSiCMY2OzhAwMnf3PX7vUB3AxwSGiIQ gUasZ9u+/xSE0xCmcTLc8rcuPkaIExS7CQuagNster9yVtRzJULp2AqONyVzS0snkc2OYn DzuL+Eci8BxTlUeT0j6UACgbpq+7abI= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=HwdKetgE; spf=pass (imf03.hostedemail.com: domain of david@redhat.com designates 170.10.129.124 as permitted sender) smtp.mailfrom=david@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710432795; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tioFn9Iuis64TbgOqSI9aPe4MpctazXkMOa/EAVTGPk=; b=mmDKKaIBsBIB0Iz0deVwxaIE86b3BXPMQzi8+CXdDpAtSgpxz1FQV6WQWjQHJtRq6uGu1Z Lh60LouKGL5Gi651UE4J1KAYEoPVCWWUJzfzCNh7M0n9117AF2Ej1k8+ThRg/V/D/1/VH+ taDA3GsKqzTpdImKjTh6HQbzy7DgM6k= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1710432794; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tioFn9Iuis64TbgOqSI9aPe4MpctazXkMOa/EAVTGPk=; b=HwdKetgEVE2eQGbgTVC2NPJFym+RSAn9x8QivgvLN0FRaeMjaazEo+m1qkeF1hawEOVert mz/lZ6t79F4vk0YZEUm1lOXepNrPin/4vzfO58XM19n0PCBUu2RlHcTZsYTdzPqI3jpnfV CIUVP83LVJJKyQBi2Rxd76LSE6AoEzo= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-591-0LIaEzhXNZS88K9aMt_lnA-1; Thu, 14 Mar 2024 12:13:08 -0400 X-MC-Unique: 0LIaEzhXNZS88K9aMt_lnA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 187293C0CF06; Thu, 14 Mar 2024 16:13:08 +0000 (UTC) Received: from t14s.redhat.com (unknown [10.39.193.74]) by smtp.corp.redhat.com (Postfix) with ESMTP id 406F540C6CBB; Thu, 14 Mar 2024 16:13:06 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, David Hildenbrand , Andrew Morton , "Darrick J . Wong" , John Hubbard , Jason Gunthorpe , Hugh Dickins Subject: [PATCH v1 2/2] mm/madvise: don't perform madvise VMA walk for MADV_POPULATE_(READ|WRITE) Date: Thu, 14 Mar 2024 17:13:00 +0100 Message-ID: <20240314161300.382526-3-david@redhat.com> In-Reply-To: <20240314161300.382526-1-david@redhat.com> References: <20240314161300.382526-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.2 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: E8C9020012 X-Stat-Signature: rygs6dn3y7us67edj8h6ozntunaxmi5k X-Rspam-User: X-HE-Tag: 1710432794-744709 X-HE-Meta: U2FsdGVkX1/mO28TI3OsefCt80IqyZZB++lZ+ADLR0artpuA8sAgLi9a/4h3ePX/PRdsQ3xWfB3VZk0SzzyX0B9qgAJoIMCVP87iDH7j6Tih8HLhJF7zii5ADkXSHCJ0UD7W1hb917dwwVn+epv94hMzC+pH61hG0h5aul28pPGRekh0o3HbeD5/2kZurrNY1XoRIUm7S1hf5t/T6U09RLEfs2cDEg9+O42IELVTyikP0aSe+1vL+kXlizOn4Xl5EcTkX94++282CAWs9vYBGiM65sudmdKw3T8/fRJQUH393pssR1mNDmTuUCak0Kx9rm8EZ7JWigfEgymJkut8M91jyIDecNhVdjG7Zc+87JyE5edkTZdNxGWCEb4bEGvpa8+RI7YuUGlH4dqVporUaBPiSmPP415+gTci3ZEJ0VgSQAuygjRnuXIbIYMJlmaKuGG6pjBhBOQuEYrjhnencClDhzSVDDFOhhcZtMkh42s3DoVQk76vMbF06Gq3BV7rhM1o+HFo7LakLLVFQCIZJEoCTZdOXor0MgTljFFh8GhnJD/jGF0XSJu1Q1814lfnwTx7+r2wX1pwYmGrnB9glIla5b6aVXPAZQoELTSZJMU6bdGJO88ZpYtpF0WqYi1w1FMKYK0QX08DLpIKCJGVHb8xcWoWS6rfxb4pz+XCHfAPStsZOpxIhhNHpaRovtOmiaKG4p7BXAmaLJz+ynRx+8VoL/6+1ZXZ/OgtVsaCiccA3lUDohLOVvhppbNGkU50LA/vkUlUKtl5kP6pkOyp/VsEKFT6GSDRjZPiTxkVPzPuQpejiGUe1Blzn1zBafr1cqllAleVMhpZ4jg+qwPmk2JVjcHsg46DhXxBjK1WCY0kLdjRRWaXNo1Myxbb8zKXImJh4wX4EVUJmUVct5pK+tVDza2C362LkcWarKf2e24PFfHcef/fm/1I0TyhqmRx0+SmDc8VeWKEHyOyn6A hCYscu6G RocyI9e+mjTTJsydbM9su3KUnynKIWVDIlgDhZrjrsP+xV034WhDHejkgJgzVHONws+PpxRT3LuJBHLOdobtceIiNLlX+GsCpo7b/49iV/63G3bcjNceFOp4EKsQEmmr4bwt3FayW6x9XNi3ry6D3O3g1N3bbxxT8ROZn6UIWUOQvA43XK3PV8V8RWXU7iSy9JOJwYX+N9J3fQmWaj/YtITf+hg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We changed faultin_page_range() to no longer consume a VMA, because faultin_page_range() might internally release the mm lock to lookup the VMA again -- required to cleanly handle VM_FAULT_RETRY. But independent of that, __get_user_pages() will always lookup the VMA itself. Now that we let __get_user_pages() just handle VMA checks in a way that is suitable for MADV_POPULATE_(READ|WRITE), the VMA walk in madvise() is just overhead. So let's just call madvise_populate() on the full range instead. There is one change in behavior: madvise_walk_vmas() would skip any VMA holes, and if everything succeeded, it would return -ENOMEM after processing all VMAs. However, for MADV_POPULATE_(READ|WRITE) it's unlikely for the caller to notice any difference: -ENOMEM might either indicate that there were VMA holes or that populating page tables failed because there was not enough memory. So it's unlikely that user space will notice the difference, and that special handling likely only makes sense for some other madvise() actions. Further, we'd already fail with -ENOMEM early in the past if looking up the VMA after dropping the MM lock failed because of concurrent VMA modifications. So let's just keep it simple and avoid the madvise VMA walk, and consistently fail early if we find a VMA hole. Signed-off-by: David Hildenbrand --- mm/madvise.c | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/mm/madvise.c b/mm/madvise.c index 1a073fcc4c0c0..a2dd70c4a2e6b 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -901,26 +901,19 @@ static long madvise_dontneed_free(struct vm_area_struct *vma, return -EINVAL; } -static long madvise_populate(struct vm_area_struct *vma, - struct vm_area_struct **prev, - unsigned long start, unsigned long end, - int behavior) +static long madvise_populate(struct mm_struct *mm, unsigned long start, + unsigned long end, int behavior) { const bool write = behavior == MADV_POPULATE_WRITE; - struct mm_struct *mm = vma->vm_mm; int locked = 1; long pages; - *prev = vma; - while (start < end) { /* Populate (prefault) page tables readable/writable. */ pages = faultin_page_range(mm, start, end, write, &locked); if (!locked) { mmap_read_lock(mm); locked = 1; - *prev = NULL; - vma = NULL; } if (pages < 0) { switch (pages) { @@ -1021,9 +1014,6 @@ static int madvise_vma_behavior(struct vm_area_struct *vma, case MADV_DONTNEED: case MADV_DONTNEED_LOCKED: return madvise_dontneed_free(vma, prev, start, end, behavior); - case MADV_POPULATE_READ: - case MADV_POPULATE_WRITE: - return madvise_populate(vma, prev, start, end, behavior); case MADV_NORMAL: new_flags = new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; break; @@ -1425,8 +1415,16 @@ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int beh end = start + len; blk_start_plug(&plug); - error = madvise_walk_vmas(mm, start, end, behavior, - madvise_vma_behavior); + switch (behavior) { + case MADV_POPULATE_READ: + case MADV_POPULATE_WRITE: + error = madvise_populate(mm, start, end, behavior); + break; + default: + error = madvise_walk_vmas(mm, start, end, behavior, + madvise_vma_behavior); + break; + } blk_finish_plug(&plug); if (write) mmap_write_unlock(mm);