From patchwork Tue Dec 24 14:37:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13920189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AE6AE77188 for ; Tue, 24 Dec 2024 14:39:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 19C8D6B0085; Tue, 24 Dec 2024 09:39:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 125A16B0088; Tue, 24 Dec 2024 09:39:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F06BA6B0089; Tue, 24 Dec 2024 09:39:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D34AE6B0085 for ; Tue, 24 Dec 2024 09:39:35 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 8BB18160BC6 for ; Tue, 24 Dec 2024 14:39:35 +0000 (UTC) X-FDA: 82930110648.02.DC27E8D Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) by imf22.hostedemail.com (Postfix) with ESMTP id A4FD9C000D for ; Tue, 24 Dec 2024 14:38:52 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VRCwjlAC; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735051128; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=OgYKQ0ZtCYd3YVntGndgFqG1xG6LNy98qDU3ugr/8khxiCFmuRVLZRIVdJ6ZlpDCtz1vqv j766pDrt7VPwLObhhuvSUov9/fGPOalPwTIwsCZs183JTPQ4Q7sT6LxDsnZ12mhXmNPNx8 +MDM01ZSJ5gRLlreLFV/2gURBpKeG2w= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735051128; a=rsa-sha256; cv=none; b=wsJXaXWoc/Py1DhyBH0hmwX1upB8sh4PvWReflKnxympk16hwRAXHeFMqZ4h7n51AXaHSB LDOj+1MKvJomLTVXsn7QstGwyS85fKUgWePTJEjpGGNwsTnVak4d3Mypr3VEAnxXkEOUVA TVgA+/86TX96ulFnFUkCVJDBTvivcJs= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=VRCwjlAC; spf=pass (imf22.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.214.180 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2165cb60719so54434495ad.0 for ; Tue, 24 Dec 2024 06:39:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1735051172; x=1735655972; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=VRCwjlACYIYX4eefkxaooJzOU5qeyNkbtBXIukA1zfMiE05rhyttkaMF+TOdRHJoLG Di2RaNzZh8T21w32zPWOKToJzGl/ZIRajq08wjgOl+72whsLrUqPUhapo+iVJ1TUSILf RGWnUri1wjIODnD0bEulSm1Uwxkio8Mz0SXz75SsSKor7EGOsQNLoEx5B2BepEYfCmJ3 Ie3AhjvzRBkm+wxWWZ3sF8jFuOP9WGTPGEOYY7T4713kcicJAkHmxfA+lKAErJslVHSE +11B2+WLecvu5LQzq+TxirA1HTRHP+Jw5Suruw6qBZiKoANTx+PVB/j0CkjATje0sn08 WWmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735051172; x=1735655972; h=content-transfer-encoding:mime-version:reply-to:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ByQEC8FsvBO2uOH/tvc24OCSxXwsTLkJSMXrTAjB9ew=; b=SREzv9FD2jZk3XqaRxAi6mA3OsK/EEUZ107RBefaOs2Jvus51dCkg7JMwxCqiVzPM5 CfyHpUegf9OKal2hs0D0omo6nFvqK4ohcVCWJaHw6YHmqc1G5gRt/lrbNifX1EmFPUld todL5RIYWKPH5HiFk9c/PbbB+DpK2whGlJbzt78ndel9wJSoFicg5/rnIQaF5dmN3vMc Woqp70nElWaf6YQy4ZF9vQDbUNvSN1lA0y8LpWDsUHqirIX+uH17eiUrFT6nSJOXg0++ bB1GReEniMVMbjUFuo9T3qYhkzInFoVixIj0sRs4myOFJxQnYB02TkB5b9ULr9JTeljI b3SA== X-Gm-Message-State: AOJu0YxF5mTWh9bTiLk0O8HMOywT9SyTX5uu3+TI0fk0dTOsaN8Xlrt7 7g5FVwxYtXx7ju4jcMkdSgumHcivTtOan1pTKRjr8BR87mIiEvezr34wFzAZZ10= X-Gm-Gg: ASbGncuhbuEx92i/o8uFddoqvOeOxH0o711kkXxJ/BVBFBNnPMlmAbMPv5e+vjj1oj5 v7B2Rxa4uKujG6+MdjadK+TXYpijmDQ7UJ9uIcrpehBmYW10wVWbq22l7qBdRlZPXP1O+93vahy V/iZTk3IqYRE0+PjZ9vKRIt0nVnwT5GXpvUaKahcyK3xmudOAoJYsyMqJVsrj6vmESbWPoON5uH UIJlfCNjQVOsUU6HQWnokK9XmVX0DjViVmoTAcPEKTFiYLxFYqXkiz/SnKg6aJ7AEDZ8624njQT Mw== X-Google-Smtp-Source: AGHT+IERM809iT3lGHUyfSmwzz85A7RBdI6+sHZNonAv7QDefMy2AtxZq5MCVgChbGhf0YMVvbHZ6g== X-Received: by 2002:a17:902:c407:b0:216:50c6:6b42 with SMTP id d9443c01a7336-219e6f25ddbmr196266805ad.56.1735051171988; Tue, 24 Dec 2024 06:39:31 -0800 (PST) Received: from KASONG-MC4.tencent.com ([115.171.41.189]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dc9cdec1sm90598735ad.136.2024.12.24.06.39.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 24 Dec 2024 06:39:31 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Chris Li , Barry Song , Ryan Roberts , Hugh Dickins , Yosry Ahmed , "Huang, Ying" , Nhat Pham , Johannes Weiner , Kalesh Singh , linux-kernel@vger.kernel.org, Kairui Song Subject: [PATCH v2 01/13] mm, swap: minor clean up for swap entry allocation Date: Tue, 24 Dec 2024 22:37:59 +0800 Message-ID: <20241224143811.33462-2-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241224143811.33462-1-ryncsn@gmail.com> References: <20241224143811.33462-1-ryncsn@gmail.com> Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Queue-Id: A4FD9C000D X-Rspam-User: X-Rspamd-Server: rspam07 X-Stat-Signature: kkifmghegnbxkhjr1sx9qhr4ehj5ohb7 X-HE-Tag: 1735051132-518881 X-HE-Meta: U2FsdGVkX1+P8a60dJFh886p+WjBL1CQ5svslQxllz+MUgv2tH1qFmx0aJlkf7FAJqWLb3xPELLyzE8/T/aBksEeopr8z6dWs5gB2Lqb51BDZH//rzqkY/iZ38Ko0r481f+P9vu48bA2AA6uIK8msEn7tROmhKfCAr5EEGHgpbqySQ99WmlIV5Tsp+9HRq7qhsihhw0/z7PVFjeVYnp6uZORAVJsMtFS/+tf6v2XvKHNXpK/F2bHs5AtXKhbmcQNjVQiuuB6IAY0eCEnk9WDKAZXODGGGsqIg8mZDFC6eVRKg6VgWWY7LpeoiRWUB2h9Hthpt94ONvSKPbwTOPXRMMNaFeWuIhD4JtM/rW0bNRfrCXAo+8+etL7TV//c648FRgH/bKU7hTQyUBSsnoc/zmVUfYvdDasf1J0w8idxbMiBzq71SFRmXCkIVZqDsdqCaMSzXRjkSX+mj10t5e4QE/3Gv3OwcGduqrVcQIF5hbtv4/79R820jS3CspXA0FdS6nLgDot3syn2wOuWF6sWe0B+nghT5L7wDYw1LzdOSY/2CdR96yBFu5hLKscgWJMAuYFrFRvlr7UrouuGpaOVctwgrHGs8nLCVwuUHc/h9GklOBIZnSBgfchOfj+hjalrrSeyZN9lpiRKG1OyITpOOnXqP2c6gU/zKODOrVslvCM5UHf33Qu3EF+UbR5Avbq2OaaahbJpDXPj8lVyX7+BY3IIym42Rn+CGgaMpkWMdd2D0t6/3m/d1IK8K5wPaoclz2AMCSh+I6o8vqSx3u6ZkIAXSwKhJUf9+lfxGmU+RhW8r09SZfxM2lp/yJAeS9edLostBm9O/a1buQpyoSXUV9wWc2FlwK2SJNL23VMag0ngczMjyK0gF7GTUndT+jkW7EamA3FdvchdKhnCWhwDg5KTJkvax21IKXRblh/0zyrBZZ5RWopN2q/KeMbfxWSwBOaaj5hhBBj7eqkcVJN htSx7wJ7 bPaO+p/nytwKP7WPSSWsc7wgG1V1gIle5gP8IK5Ps3TiYAGWl7tW0HmpA5bU+A66emvdML1zL15X4/KsrbTfrVvtoZJ+MgTpG/YApEEBG4DkOE59/bgDTTZaSbEQuFJN/T7OLzn2DeA/0i6lAs86iVYX5gUIaT2KTfGlaZK4sZbA4O81hxoKeIvWzkxTUjwj0EZSzk+nBEZx99dD4SMM5eBXwLZGhdKMZwkhekT8GQGmVTbKIOMJd0e4tJT9WXlEDlHgn02VDd95N+lifIjE09t4LoLXIKYAtqamp57MjpGyI8xpTJ83+UApmEfPI/y16j6Ko6c9Su1mqhxMEZqnAHfppGT1gD4vx3hrma521BNA+F096PkZhKt5HX5GqD3qxIVmrzJ8J3E/xsKd3VrcOsWxcMSY9ySCPhjEdq9cWF+8jZKeQKOcPwDrzgaOa6tC1SJzOWynSxtpYCKkjF0Yo9TPb3VM7IrKFeNoTbhKsXNoT5Mjff3HvdtonxEnjrUtirp/+KsS+xbPE2gt+GNgu6tH8Bsllf4XizByQ4Qfs/Itcqbwpnim24FkWIg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.001076, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Direct reclaim can skip the whole folio after reclaimed a set of folio based slots. Also simplify the code for allocation, reduce indention. Signed-off-by: Kairui Song --- mm/swapfile.c | 59 +++++++++++++++++++++++++-------------------------- 1 file changed, 29 insertions(+), 30 deletions(-) diff --git a/mm/swapfile.c b/mm/swapfile.c index b0a9071cfe1d..f8002f110104 100644 --- a/mm/swapfile.c +++ b/mm/swapfile.c @@ -604,23 +604,28 @@ static bool cluster_reclaim_range(struct swap_info_struct *si, unsigned long start, unsigned long end) { unsigned char *map = si->swap_map; - unsigned long offset; + unsigned long offset = start; + int nr_reclaim; spin_unlock(&ci->lock); spin_unlock(&si->lock); - for (offset = start; offset < end; offset++) { + do { switch (READ_ONCE(map[offset])) { case 0: - continue; + offset++; + break; case SWAP_HAS_CACHE: - if (__try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT) > 0) - continue; - goto out; + nr_reclaim = __try_to_reclaim_swap(si, offset, TTRS_ANYWAY | TTRS_DIRECT); + if (nr_reclaim > 0) + offset += nr_reclaim; + else + goto out; + break; default: goto out; } - } + } while (offset < end); out: spin_lock(&si->lock); spin_lock(&ci->lock); @@ -838,35 +843,30 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o &found, order, usage); frags++; if (found) - break; + goto done; } - if (!found) { + /* + * Nonfull clusters are moved to frag tail if we reached + * here, count them too, don't over scan the frag list. + */ + while (frags < si->frag_cluster_nr[order]) { + ci = list_first_entry(&si->frag_clusters[order], + struct swap_cluster_info, list); /* - * Nonfull clusters are moved to frag tail if we reached - * here, count them too, don't over scan the frag list. + * Rotate the frag list to iterate, they were all failing + * high order allocation or moved here due to per-CPU usage, + * this help keeping usable cluster ahead. */ - while (frags < si->frag_cluster_nr[order]) { - ci = list_first_entry(&si->frag_clusters[order], - struct swap_cluster_info, list); - /* - * Rotate the frag list to iterate, they were all failing - * high order allocation or moved here due to per-CPU usage, - * this help keeping usable cluster ahead. - */ - list_move_tail(&ci->list, &si->frag_clusters[order]); - offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), - &found, order, usage); - frags++; - if (found) - break; - } + list_move_tail(&ci->list, &si->frag_clusters[order]); + offset = alloc_swap_scan_cluster(si, cluster_offset(si, ci), + &found, order, usage); + frags++; + if (found) + goto done; } } - if (found) - goto done; - if (!list_empty(&si->discard_clusters)) { /* * we don't have free cluster but have some clusters in @@ -904,7 +904,6 @@ static unsigned long cluster_alloc_swap_entry(struct swap_info_struct *si, int o goto done; } } - done: cluster->next[order] = offset; return found;