From patchwork Fri Apr 21 17:40:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13220540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 834AAC77B78 for ; Fri, 21 Apr 2023 17:40:29 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1A5896B0074; Fri, 21 Apr 2023 13:40:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 156526B0075; Fri, 21 Apr 2023 13:40:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F14F76B0078; Fri, 21 Apr 2023 13:40:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id DD4BE6B0074 for ; Fri, 21 Apr 2023 13:40:28 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id AEF04AC4AF for ; Fri, 21 Apr 2023 17:40:28 +0000 (UTC) X-FDA: 80706112536.24.7CC9ABB Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf28.hostedemail.com (Postfix) with ESMTP id CF605C0007 for ; Fri, 21 Apr 2023 17:40:25 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=XiTBjh0u; spf=pass (imf28.hostedemail.com: domain of 3iMpCZAoKCFEH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3iMpCZAoKCFEH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682098825; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=WtMmjj+4e6y9XFpMlRAEAH1/JxluFHL3mc0pUNIc8tVUCIhwElp5DZ89ldYbgmsVA9gufg w18EeKKaod3lKvl/DVldFbkifKq0qCX3qEnxMTUWIPemsOAOoc13SI+5iarCjJYosGeO5L WvX+qp4uR1ngmPQdD7yPiwQkBOfrPEo= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=XiTBjh0u; spf=pass (imf28.hostedemail.com: domain of 3iMpCZAoKCFEH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3iMpCZAoKCFEH7BAHt05xwz77z4x.v75416DG-553Etv3.7Az@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682098825; a=rsa-sha256; cv=none; b=aYOblge5Y6Ynb6lrVclvXgc/Y51df0/ovk+H02LFYx1DOBM5BSd2IvW8/71WOdHUbhaOk9 CAs8fOYgjnB2ylBtI3gG7wV/heJQ9VLkvpRaYLr/dMwsM3LO3gXzlxl3DJ4/+b9Hse+Fnj yj9VwFJh6jzj8dlHr5NLlh2/RqSLNH4= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-2479c766e91so1391786a91.1 for ; Fri, 21 Apr 2023 10:40:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098824; x=1684690824; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=XiTBjh0uy9kBhESmjZrsmo27Fd2heaOUEeGy+wI7JgY+55BFi83KpJfUElFkBAdtm2 /9Y+dOWIDQQOwVTZr/9l2zpPw4NhLuSad5LwurXNF9rM5PxMETajnAk6IdSR9itjleO5 5RCZOU6uWoQn/XlBfG6L6Ut73QqfPkSzpxRKMZq9c5dNC03EQysfXGAmQsX7vQIaMlXI wrYrPEcgEz4/6R89vEXhItTLbmEid9U911aGVkkoSTQdm4KxH3ssyAravljY6M6dDyLy P40RfGznmXP1ykBLl7M1nQBK+thu7rJX9Fwoe+MatBRuu3PDwmPEoG+B+msi8iGqptyL Fwng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098824; x=1684690824; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=abvwWBWOn0vLHIokwPNC/tHyu1Fa1R/g8ZqTWIaeK3o=; b=dZI6Z3mpn8bYfKhRD0wz4dHKO7CCv5BNjbFsXQPKS/5gu36A0SKV5MCT1w8rQAldJX 8pPH8NWWwaCxEH6E58W9wWcdeeYai8sUdEtBbCQCLd69wGU9mAQi0MKUAgGqReiZJ7zW 1ilQFXXjlKECQ6jl1DkHzQHTD31o2ggpMKx939IQyXJl4mX3FkPCgjBWtT+YaJMwCi9w Es8DbHYtBox4CVR5XWNhxJaeio+LNWDB6DZ+sFTHQPwhLsJQLg0T5T2Lz45iwfgDKggy NILOC+m95mWPIy04YGnZ8I1/hZcoRA/hKvgfzt35+NTDutvSBN1Q/atNGxrTv14azSYH B0Sg== X-Gm-Message-State: AAQBX9fEyMJZsEPHWpHbLq6Z0R72YyKuNxBcIw0S0sL6ZkCXpkbC3nx6 AZXcgqNajBOTG/Hl1IwQj/C8Hypdow1fljRj X-Google-Smtp-Source: AKy350YbjANQ50zXR9lpXjPZgwQ40mwqLJ7O/YqrPonNpx/xppRCva3J0ueIG/FDAbEMT7LPHpueSbGCVpFSyjJ8 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90a:290:b0:247:2677:77b with SMTP id w16-20020a17090a029000b002472677077bmr1455398pja.8.1682098824669; Fri, 21 Apr 2023 10:40:24 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:16 +0000 In-Reply-To: <20230421174020.2994750-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230421174020.2994750-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-2-yosryahmed@google.com> Subject: [PATCH v5 1/5] writeback: move wb_over_bg_thresh() call outside lock section From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Mich?= =?utf-8?q?al_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: CF605C0007 X-Rspam-User: X-Stat-Signature: 5ke59kbzyu1pqzn5gd8qon571my4s1fw X-HE-Tag: 1682098825-109107 X-HE-Meta: U2FsdGVkX1+dO8kOntENu0iq0ZOSVGoYa6N4qHoX/UQhl2UJN9n/fC2j0DrstCv/ixFIOPFk/dMvRHOrchMsuIDpQzFqCHmZ1DXg077+EgDrJGLUM+GE/IuID1JplLU0Cm+GD04VRORjbFlkLXUwTcKIOEfl4nLjt7CazmV++B8uYaJScwl9XgzHkbLJWM37OG5oI5zmpmDur98as4FSwCmBl2WHUg8JwXbbFAQgsOa5aUHvjSrOQxQ7kG7nW6/vLQfL+aRoaYlwLeJnsfFGJL5wRnaSxzTAonSBAuT+KRgP/gN4Wr9FEb4tXGMSlSvyppIoXls9+nI8gPjE1cFU2gI2TGahghaK8jacWWra7ti70LT1d1nV469u7BRMutxZJB8JXbuUEvyXf/7uQEG8PItsgRm3KZMVmMSZucwW8I2YSIIoE9jskN320gL5sqUmAV3WYdrzBC2ZPg5nTpXchfScaRpT2dHeLgwZWK92vzr0kFlPyjIC0SbrHv8TmxWOHYLBC4SgwvXu+enyqbUEbE4JsZvL1S/zMRIra3heAw/BKkMTPIGLxD6fj2i1OaFHmr+R4MYDjzRvZxrcUaj+NNHqFiQ4sCyFJuOfVRbhCfj14IF8/nyji8d2RRRi/v0Jb7OPKmh3wZYyOWWdY0OO+7fkOjm1NFCypw/Rkk/c68AUrP5mO/CAhPxsmVLDqkru3ThuhQVoQMugoRLcsoDyIN/Ggvwe77Y3pSKx2xUYFp9Y1QwgMdA5PE2S6lR3cBtVL5K9uOJK5NjB9/svxCkjEA/RNjxJ3Hn25ut3RF+iUrkAFekfmWEdE2j9pHXaIMu8ekJM62USYClPuT+FM1UiirDKe1r6ufH2FlI42IB+4aIOZbn98TARXUv1owozt4iZtvXjKtDiykpPJC4cCbiM+OpOKf8GhDBcvoaPMnGiUdLqzplDyhogVR+oJlUhBSr0du94+kG+XdAFFaJhSUL 463gw0XA oWL1Z5Xq33STdTkdgx9yTbgOHyli6+/Xj5Whs/943DY/RUUkW1/q7O4FF0oubYJK2ppSqlf5W1S3yswFyA39wO4j38wZDISSs7XSyhjlIpwgX2e4KcE+1TkuAmjSQwsB17aMd/7gv2VtIsqeU6N3A1yYhBHcYGqHOTE+tGSGF7KivmIrVctGOslXIiNSXafY/40lSwZvPjtJSVzghq2cIJ3OyUcwllZNW9VRmCY2jd+UtS7ko+nv8ProuUbFpsMjxLJZC4dKnG0lQUmbizhUrqz7vJCCvR4pvmR1ejnsYWPtCrjdGkCc3iBR4iYbTjIf0W3rHncVxrDqSRj+cIhzQ4PFk7FCd9+YeBVHjq40VimBmXuWqa00AjY9yThFP+FZQjcBbXvovg/cr7Xm9BHGzbrYdS0Z/eTwIg1K5vtGj1wsRuNyjB0wV4oX98LTnB70zH2KFE+VPexNOKs41uUvYuw4lzQhIH2pZF4drkwx4lxyzOobH6zHMncCmXwXgyZnC1zzaIWLFrYhelzWhjHgSLn9LY+DjZjoqDNbzJOmhckSPyCEFfN4clZRdYxh/O9S71IDldrNlGJvwx3m+/Lt6U9cSc4cWrCYi6hSew91RnkYSXFSUgg+86ODHkUqENpJeKCP1Ps+vcqNZrQG1uql4HqjQ323fJ+L7Me3uafqb2idVrF1HAgMaxq4NOSxkrAeb290rqlJUCVUMSkXSw+h5hGayLDlEM9WsPkczj/nz/6GK600rVngHkE4tpg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: wb_over_bg_thresh() calls mem_cgroup_wb_stats() which invokes an rstat flush, which can be expensive on large systems. Currently, wb_writeback() calls wb_over_bg_thresh() within a lock section, so we have to do the rstat flush atomically. On systems with a lot of cpus and/or cgroups, this can cause us to disable irqs for a long time, potentially causing problems. Move the call to wb_over_bg_thresh() outside the lock section in preparation to make the rstat flush in mem_cgroup_wb_stats() non-atomic. The list_empty(&wb->work_list) check should be okay outside the lock section of wb->list_lock as it is protected by a separate lock (wb->work_lock), and wb_over_bg_thresh() doesn't seem like it is modifying any of wb->b_* lists the wb->list_lock is protecting. Also, the loop seems to be already releasing and reacquring the lock, so this refactoring looks safe. Signed-off-by: Yosry Ahmed Reviewed-by: Michal Koutný Reviewed-by: Jan Kara Acked-by: Shakeel Butt Acked-by: Tejun Heo --- fs/fs-writeback.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index 1db3e3c24b43..11aa1652fb84 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -2024,7 +2024,6 @@ static long wb_writeback(struct bdi_writeback *wb, struct blk_plug plug; blk_start_plug(&plug); - spin_lock(&wb->list_lock); for (;;) { /* * Stop writeback when nr_pages has been consumed @@ -2049,6 +2048,9 @@ static long wb_writeback(struct bdi_writeback *wb, if (work->for_background && !wb_over_bg_thresh(wb)) break; + + spin_lock(&wb->list_lock); + /* * Kupdate and background works are special and we want to * include all inodes that need writing. Livelock avoidance is @@ -2078,13 +2080,19 @@ static long wb_writeback(struct bdi_writeback *wb, * mean the overall work is done. So we keep looping as long * as made some progress on cleaning pages or inodes. */ - if (progress) + if (progress) { + spin_unlock(&wb->list_lock); continue; + } + /* * No more inodes for IO, bail */ - if (list_empty(&wb->b_more_io)) + if (list_empty(&wb->b_more_io)) { + spin_unlock(&wb->list_lock); break; + } + /* * Nothing written. Wait for some inode to * become available for writeback. Otherwise @@ -2096,9 +2104,7 @@ static long wb_writeback(struct bdi_writeback *wb, spin_unlock(&wb->list_lock); /* This function drops i_lock... */ inode_sleep_on_writeback(inode); - spin_lock(&wb->list_lock); } - spin_unlock(&wb->list_lock); blk_finish_plug(&plug); return nr_pages - work->nr_pages;