From patchwork Mon Sep 2 22:50:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Barry Song <21cnbao@gmail.com> X-Patchwork-Id: 13787765 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEF6CCD3420 for ; Mon, 2 Sep 2024 22:50:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 52F4F8D0116; Mon, 2 Sep 2024 18:50:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4E0A98D00EF; Mon, 2 Sep 2024 18:50:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 359328D0116; Mon, 2 Sep 2024 18:50:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 18ACE8D00EF for ; Mon, 2 Sep 2024 18:50:24 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A9A021A01AB for ; Mon, 2 Sep 2024 22:50:23 +0000 (UTC) X-FDA: 82521293526.17.1BEECAF Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) by imf25.hostedemail.com (Postfix) with ESMTP id CC550A0015 for ; Mon, 2 Sep 2024 22:50:21 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nAz2hHnN; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725317347; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=0w7lkXpo1g+CHmrmflvgRnAVSJTT4Wmu8BjVnTGsv2g=; b=fmlqHvK9U84FCrdWh8dn3dOMVghYX2IHlTTx/eTfc6aPtiSpR27NRwVx5TJgzliw0CxE1R WYqnBzPTiL27c5gOMey89sBF0pCTe1WMuagTl11g7QosqbTrOgQML+zGyvHA+MKFlP9yuY F/g0SRZfI2e1kaBQbbzJ8TF5Ieo4Ixo= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nAz2hHnN; spf=pass (imf25.hostedemail.com: domain of 21cnbao@gmail.com designates 209.85.210.174 as permitted sender) smtp.mailfrom=21cnbao@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725317347; a=rsa-sha256; cv=none; b=dbjHgqe7nN8cN6VpcMp5PqRKaDrSfJJls+lCeFKnvFM3fhkDxDUDVG0KZ1kIRUNsrs6iAp HYmIYmEb9bv97fq2Rx+y0XLMdK4jYE74f4rxYQW4d3tH/lUATp97DLUyg7G8yDr6VQWeG2 NIXKCTkIQOwQbYD6Wv5QsxVka09SlOg= Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-71423273c62so3512946b3a.0 for ; Mon, 02 Sep 2024 15:50:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1725317420; x=1725922220; darn=kvack.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=0w7lkXpo1g+CHmrmflvgRnAVSJTT4Wmu8BjVnTGsv2g=; b=nAz2hHnNzFc4FRSkZOzX153WMyWzE12YSDFtauPw4VMi1rm6sHz+O4xXpFiwzvgaLM rv7Ux7BZC9uvtgah/FcrjbzI/lBcJYyBzYInFnrnhGf3fCLJ54r6dX4hGDNlxu9Gy1Tl P93zDYaQh9zktNrWjMvKSZAGXGUGRjTcXHQn0Iy5x3QQTExOcTJqkZpLH9MyAlwsjlD0 LzhJszovYySHb1s5xYFm6BlszyO3nP5HC6xAe60LWH3aggDEKdmZv/0xkype0VvOgfhN 8Aa/LQEkyGG7gzHOs+2IuEiSdXFp8KyEHsHBYoOl+3Wtih0F4HNAodH1h5fiG5j1/G8o 2hUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1725317420; x=1725922220; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=0w7lkXpo1g+CHmrmflvgRnAVSJTT4Wmu8BjVnTGsv2g=; b=cVg72/MYikNCNmf+n+1uIx12ebGrzG0DL9AR0lN/GA2VmM8d99ZN7w/HKmOOohcuSB KI4XFhi9bLHCxDMSvf624RJhBMEvaB/qsR7yC1Zpo6lgt5qoQh86WQxXwEFD2tm+UauE VsMeT8c4rsvbVlMS/+hBs2JDc/MHOnFDaa4Vkzjsb/R+VsZRByqEhvCK3WdEuQbwJfLc LD+zxwfp9eO38KSg2tWKtN5KXcWuTTu9py1Xd0GfV23XJurzYF4R+JXaR7Gi0XLNZ4ul a+YYrmcZDmQ82nU5CMxVZA4/btWgtfD4ZZBOJPyQPvZFJktfpgTX2QVUeSuRUa6nWBgf aVJw== X-Forwarded-Encrypted: i=1; AJvYcCWNrfHU3OUich8C37y4qGLeUmks4JFwhA7pjA3spC82V1sQFah4fBN1p2R3VnMw7pTOA58qtsx7ww==@kvack.org X-Gm-Message-State: AOJu0Yy1OuRDYFnXcu2Hgi21FmTeKWaBQPP7FJZchTkyDom/akl5C5Zm NnmyGd0pak6Yus+7IE26n8nOjwsezDUIis+eU9G1Bl/oIbW+z6MyyI4+PLkH X-Google-Smtp-Source: AGHT+IGOOEgseY3taJzvPKMOGi4cJwnXK3ju8YNTsH2qFP+1eIRtKzWkc/8EV0t5wRt0gIkoTyO4Lg== X-Received: by 2002:a05:6a00:1a93:b0:705:b0c0:d7d7 with SMTP id d2e1a72fcca58-7173fa3dcb9mr7623930b3a.7.1725317420331; Mon, 02 Sep 2024 15:50:20 -0700 (PDT) Received: from Barrys-MBP.hub ([2407:7000:8942:5500:c91e:9f1a:31db:cd2e]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-715e5577263sm7338284b3a.21.2024.09.02.15.50.14 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 02 Sep 2024 15:50:19 -0700 (PDT) From: Barry Song <21cnbao@gmail.com> To: akpm@linux-foundation.org, linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Barry Song , Greg Kroah-Hartman , =?utf-8?q?Arve_Hj=C3=B8nne?= =?utf-8?q?v=C3=A5g?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Christian Brauner , Carlos Llamas , Suren Baghdasaryan , Tangquan Zheng Subject: [PATCH] binder_alloc: Move alloc_page() out of mmap_rwsem to reduce the lock duration Date: Tue, 3 Sep 2024 10:50:09 +1200 Message-Id: <20240902225009.34576-1-21cnbao@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-146) MIME-Version: 1.0 X-Stat-Signature: rs688hu5oo5bshhbbo5t96mtf5sqrxcg X-Rspam-User: X-Rspamd-Queue-Id: CC550A0015 X-Rspamd-Server: rspam02 X-HE-Tag: 1725317421-249618 X-HE-Meta: U2FsdGVkX1/OZWdp/BM2lBOYnlsAZSIUHogmNTwChXF9gpA/gfXFvBBTvamMZJuMYygLO/f7tZD+/0oC2T6tNFeXkh3RlWr858/QnFttvrcwHf2GpSZKAVBCv1Pnmm7bFhBurB71I6dSesjlikigQSAYjqaVL3QiGXNVYy7lNuE83ccCglvSHKy2Da02HqK4w2WdqbXmsO2cUIFjbK1A1c/Y6C5gIYv0GoZ9JmlWlWjrJgIteN1akxJbRMDrWuBpYdqsH5dSjsN05tDSHpqEv8jB7tgPx5idV6wynWfF2gzHcORNcA5CM79De3wSKcnJrsRMQQRfT5MKzfGPG6DsRTraWRaZw/Wv5VkmEJaZwM2Zj6ulH4Xr0xI9hAML/MPP0rlzZd75EjfF9g+VV+naCAr5Yzy0XUo0Iw3K5M6yh7566cHcihTqO4QRlOXThjR1RDofs4xA3RxzWJC+GKdwokTTcLvIi3MNDgzzbOeKubfMvKHoVEbEXdjWnMslBXWles5qlYhU4PBcCYEWUMwbjjBh9g9fiLDMtfTnfU11TUCESDYANeLQAvnUdjrCxsSG4zT0HJN/T3ggq9CoigK6poxM76ey0l+UL3nUKvA3JEu0PX3W6l3+lsmuAL1lNrWvMToz9WyZwcmuQFfElxRzbV1gU/q8jHvy13YkSymKOBvbTeZmI9C2W+aStT4K5+7LjDFlS5b2zhI+Nbw/IRGI9BA0zXZR+Ld1aI+sos+HmkxHlVk7bpjyDLc1JXObGSbPQlXyhGlUukUNxF8A7GZSxg08mt0Rdrg/D4//hwBxvaca/CgqP2xXFN1tA342S3OguaFrlCszOtgX/Y4/qN9G7sQfQNB00NaQFOHBjsh9Mofv3Wjc/EXs9bZBIAqzwfr5RykYtgXDYoxjXfi4osBhTEVvHh2AaevEzCJD0nLchU3D96Gd3Zvp9qpIKo49DOliL68Sth+gM/sK3IbtxzG cHqjGTAu 0MuhYZwazk95OZx5aPx+SJR/ybec6aR3NHLBwpWO2QgnNnUsk36L4VsA+1MZe5A9hhb5kVmuXNWPH2nYOybNf1WtmNj5XP3MoBKQsVNnbS1t80hmHHakw5ebqK2969ZVaHTyAbEA0/jgTdgF/lz1Fxpqnr2KIHWly7tgFiFfkuunklYV36rYlBxUel+vwdtKDBbzcUMc5gLkCUhOe8kfldw9uns9C4IRPeu9znNFAGaVyU19W8AhzuIoy5wQqAFIdcM0HZW2LsCVU7w+01KidEO1rg2EjQGIjgnZ1ZTICo/D+Eski5w84L6KllCQcaKWRyrZDwVjdlbj5VtjvOELo3tzkBwwoGx6/6Ay73EuT89YtqlLakpHFu5F9UFjnF/kCVgYhY2WK6xZldl/ZWp4hv5NRRB9A66cMX9R6sMykFbZMh0GqJteZMriYoPXI2ZNduzlrE56eBUji2nFVSUb2vzW8LMrRjr7CVkwTZhQHmbqQu9PqVuZhA8X6DHFPRai/EE/X9UOsslXkQPeA46eIcOZ3gubbgYHYbr+Yo5auKcbRVGmuB6c2irGCGiid873ZSmJvHyyKbkky+m+7os5yqjF3b1ZhHb+N3tRZizdG2keSmoiKErlt0t8pVzAnjCh4+BW4ojIukWOS3Xc4pn412A54l3tlRgDqfpla/fzXf5MNHtZpBbAiemshtchoPv/KHxeh X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Barry Song The mmap_write_lock() can block all access to the VMAs, for example page faults. Performing memory allocation while holding this lock may trigger direct reclamation, leading to others being queued in the rwsem for an extended period. We've observed that the allocation can sometimes take more than 300ms, significantly blocking other threads. The user interface sometimes becomes less responsive as a result. To prevent this, let's move the allocation outside of the write lock. A potential side effect could be an extra alloc_page() for the second thread executing binder_install_single_page() while the first thread has done it earlier. However, according to Tangquan's 48-hour profiling using monkey, the likelihood of this occurring is minimal, with a ratio of only 1 in 2400. Compared to the significantly costly rwsem, this is negligible. On the other hand, holding a write lock without making any VMA modifications appears questionable and likely incorrect. While this patch focuses on reducing the lock duration, future updates may aim to eliminate the write lock entirely. Cc: Greg Kroah-Hartman Cc: "Arve Hjønnevåg" Cc: Todd Kjos Cc: Martijn Coenen Cc: Joel Fernandes Cc: Christian Brauner Cc: Carlos Llamas Cc: Suren Baghdasaryan Tested-by: Tangquan Zheng Signed-off-by: Barry Song Signed-off-by: Barry Song Signed-off-by: Barry Song --- drivers/android/binder_alloc.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c index b3acbc4174fb..f20074e23a7c 100644 --- a/drivers/android/binder_alloc.c +++ b/drivers/android/binder_alloc.c @@ -227,13 +227,23 @@ static int binder_install_single_page(struct binder_alloc *alloc, if (!mmget_not_zero(alloc->mm)) return -ESRCH; + /* + * Don't allocate page in mmap_write_lock, this can block + * mmap_rwsem for a long time; Meanwhile, allocation failure + * doesn't necessarily need to return -ENOMEM, if lru_page + * has been installed, we can still return 0(success). + */ + page = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); + /* * Protected with mmap_sem in write mode as multiple tasks * might race to install the same page. */ mmap_write_lock(alloc->mm); - if (binder_get_installed_page(lru_page)) + if (binder_get_installed_page(lru_page)) { + ret = 1; goto out; + } if (!alloc->vma) { pr_err("%d: %s failed, no vma\n", alloc->pid, __func__); @@ -241,7 +251,6 @@ static int binder_install_single_page(struct binder_alloc *alloc, goto out; } - page = alloc_page(GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO); if (!page) { pr_err("%d: failed to allocate page\n", alloc->pid); ret = -ENOMEM; @@ -252,7 +261,6 @@ static int binder_install_single_page(struct binder_alloc *alloc, if (ret) { pr_err("%d: %s failed to insert page at offset %lx with %d\n", alloc->pid, __func__, addr - alloc->buffer, ret); - __free_page(page); ret = -ENOMEM; goto out; } @@ -262,7 +270,9 @@ static int binder_install_single_page(struct binder_alloc *alloc, out: mmap_write_unlock(alloc->mm); mmput_async(alloc->mm); - return ret; + if (ret && page) + __free_page(page); + return ret < 0 ? ret : 0; } static int binder_install_buffer_pages(struct binder_alloc *alloc,