From patchwork Tue Mar 28 09:58:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Muchun Song X-Patchwork-Id: 13190787 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1ED9AC761A6 for ; Tue, 28 Mar 2023 09:58:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 8D47A6B0074; Tue, 28 Mar 2023 05:58:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 8369D6B0075; Tue, 28 Mar 2023 05:58:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 689126B0078; Tue, 28 Mar 2023 05:58:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 5784A6B0074 for ; Tue, 28 Mar 2023 05:58:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 25DC5C0774 for ; Tue, 28 Mar 2023 09:58:36 +0000 (UTC) X-FDA: 80617857432.11.B260B63 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) by imf21.hostedemail.com (Postfix) with ESMTP id 517831C0014 for ; Tue, 28 Mar 2023 09:58:34 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="GNYV/SpQ"; spf=pass (imf21.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1679997514; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=oXReBzDsI5mUppZ3rGyjpCu82GIXuKJKMV0lZiD0JNk=; b=oNKgZxdMDOHd3ACbJGwIUtNgHQDuQ4uoC5d2cgeBaZUpTKOs32CIG1+45AQz1RhsUITQIm 4V6qZdWMmAKtuJchZztUpQaQQgrRjcCy+bI8QH4wPRibKP55HjBkIrkPaBQVtBTmAzDx0L miAnh3sjlR38JTl8GOHqeaJrBmY4aBM= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=bytedance.com header.s=google header.b="GNYV/SpQ"; spf=pass (imf21.hostedemail.com: domain of songmuchun@bytedance.com designates 209.85.215.176 as permitted sender) smtp.mailfrom=songmuchun@bytedance.com; dmarc=pass (policy=quarantine) header.from=bytedance.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1679997514; a=rsa-sha256; cv=none; b=A+n0Ksw/oJtNhVCRQwVVSf3SYjRE6WYIfq0luiAQxcP9/OZ6xVZxxC9raRNWd1b6gvpmkD SV9WXUVUjBdkKHys1elkk3XwShMV8hcWwecW8n2U7DU3S7ty8Cfas9qtdTcM2WEigtJm3H MB1sGhnKEYOG5Ty4MqlvP+JzifDMNw0= Received: by mail-pg1-f176.google.com with SMTP id y19so6849491pgk.5 for ; Tue, 28 Mar 2023 02:58:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1679997513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oXReBzDsI5mUppZ3rGyjpCu82GIXuKJKMV0lZiD0JNk=; b=GNYV/SpQ4PF1Rk9guHwx0aYyiVbzxvCe8LgWKY4fKwXFWvJ0gHYKR9uI9Hyt7SuIba lqZhGe+iNAZBDttHfVPfrJtzXlSY4W0BwXLoi8+mDBLpZrOaqvyR3cnJuUvQfnijfFWF DgXTjFXJbvdNpSpbmeMZZq9T6E47bY/Bfun3K0PrAfzuydbRgjCh+tcBC+yQ0FegohJ2 LDusHWD4zsYHXyzzOsDLqMZyFzXOZExMZ6+6Nz5aE4HYnhqn7ndtS3fNHvJhBx1KBdK6 QyPuMZMYaMgxyNGf7ut9mP8hd0RslzKwLhfQJ46QWZOfUXnZ+YLqil4EchVkeDWoIrO7 Af0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679997513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oXReBzDsI5mUppZ3rGyjpCu82GIXuKJKMV0lZiD0JNk=; b=qaT4P1SysDd6NY/owuAo9Yx87cZ9GTmSDPt3q/bH+GYKw3jUGlXmHgQXLMlgf9S5LJ 6Bt0TJkFIP5ZlOJhoAYpny4RrzdltHUrx8x57cmGZl2PuJE1ptUy57/RAY2n/s+EsyOf PCJK9vIQn1wU/k2wnmG5BfZZOMy+gBlVqslxOChAgP2EjTjjIAyEk16V7afBdpxh50dU oguUSe81dWZ165nBDcBjshuCn6gLdxV9YI/+O9hLmoToKCDFUPxDFKmYBsVF9pJesUq2 /vlvKnx29Dnv4RLV2/pPGDPfxRbfE8u01jJ0OE7PZixDwVNIXEqEBJET4xmSOjWTp0yj yLcw== X-Gm-Message-State: AAQBX9ffHLV7ckRaZE7H3XvPinZ6rtxUgKfwBdIilQfoJ48/GvyIZSkV gaz2j1djKY134XTo7i3iQzwGyw== X-Google-Smtp-Source: AKy350a8z+tgtFIvSjXB4Jv/VaYEvsSRl5/1SyRdomnZTLXT1oaUTne7LXl5YF+58ZBqG0o4zrBD6g== X-Received: by 2002:aa7:96f8:0:b0:600:cc40:2589 with SMTP id i24-20020aa796f8000000b00600cc402589mr3361407pfq.3.1679997512966; Tue, 28 Mar 2023 02:58:32 -0700 (PDT) Received: from PXLDJ45XCM.bytedance.net ([139.177.225.236]) by smtp.gmail.com with ESMTPSA id m26-20020aa78a1a000000b005a8a5be96b2sm17207556pfa.104.2023.03.28.02.58.28 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Tue, 28 Mar 2023 02:58:32 -0700 (PDT) From: Muchun Song To: glider@google.com, elver@google.com, dvyukov@google.com, akpm@linux-foundation.org, jannh@google.com, sjpark@amazon.de, muchun.song@linux.dev Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, linux-kernel@vger.kernel.org, Muchun Song Subject: [PATCH 1/6] mm: kfence: simplify kfence pool initialization Date: Tue, 28 Mar 2023 17:58:02 +0800 Message-Id: <20230328095807.7014-2-songmuchun@bytedance.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) In-Reply-To: <20230328095807.7014-1-songmuchun@bytedance.com> References: <20230328095807.7014-1-songmuchun@bytedance.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 517831C0014 X-Stat-Signature: d7qhqaiq7ghwqosxyybnzez9u519wub6 X-HE-Tag: 1679997514-737210 X-HE-Meta: U2FsdGVkX19+jm6DKZApb3QIOiaqfISIkUyKHine1rcuwYtxcknoTzzkRkCA31jHhENX5KblOyVETZSQ7TMKKcsb3OrmE9xXe3Dz72jxSeTmsS7fRDuAFWE4cQKxHRrWqhXE8cPZnOs6ebK8ERc22ywP9oORZ9btesY/9jOaHNEDON9ss/RQYvTSLh7fP3foX1ChO9iVXwOWFtBWOSBAPxI2ND9ffVMD3ZkFFZoJ4zGFCzEeLfepnADu92LXYo2DDwQiflwjZI0kbmf8GYVmrUuPsQz22oI7683X7PUXavK/kWVdxdR9Bpuh9ZD0sQjVdNGbhCdgQquiER2End6AUT/fZ75XfrvNdQuKh7//QGzYYgCBR4wgzIk+inRS9djd1DV2Mcpy/E1QyeGgNuGupk43Bly/HzLoEYaURC3FwmXxsWSMu+uSe1q3cC2qfazwvaI1IcOz4UfybH3sE6hDnzynQx3sLZ2LaBSSh11wvWrqZdBMt1fv+kcjuJUSSnCwt+mXlKbs/LxcpmGpytY4RI+SAi2OSLedl6tBHyEIN7iTEG7DtToZU/qekP7gstldzzqMduvT6X7FTqLDpeq+/OWIVrryNhvpga8DIOv8RU9j/zuOus7tdl/hHt5t/Fa05hIFVj2gj7LQ0PYInS3UWEG6T3HMc9dfB0kD8txh3hOAYcu0Rv+BzJmgzO3KCePBWvoMMUn6apT4TZU7OsTE/Fy5FPkKlhLqCb+wm86ukTAvFbr9hf4b56iScWANshhfJODR4P3bro+q8p6SjV77y16svsStzYb87yXCiqE1DSDzR66WVTVen7ky9wsiehdW3qeUDGsuNzKsZkJu+WFyQpl/Cnyvb9fE0BrqSsBEzEup7gxU4P0J2X5uzxJ90ga9Ok+VsVWvYWT4JJ+HoqIFWFs3tMHSs+kIJ/FIBvtSJjsu/8OD2Zge8AeY+zHmraz6APeX7Zi+1TfBQ0wubUD 0D67s8mK gblGDuXWmVL0lEkvcF5MCQewFvCt12eGlQqmzuIuJS3ElO0uFzXu3FkmxVzPMswpCyzAy7h1kxWpL8s1oxUOxYg1aHJpAJ+BdQPqGq3pWM+8ptJa9f7AaAC2pvJ5el7pnaL27z/b7ovc9eScFteUff+SIAusH+Aa9FIzGOjEdy3TARNtJDP27aqmM84DEWIlDZ3Rdysi5G3W8Nm6LMjb3jMT5SG1iWeWfdFz6muHqd47CjT2/h9ZmGyb33ldYIcDOslmmxr7zDuTaPleV5fjISHPGph6iRvxv0uMzw17n0h8MtIUsUWsbokhWtuZ+ow18sIgTBzYYxCELokA8wZFm6IjpxzWeDwoh1poWfjxrjIkxsOVoGsPBEtbT7RD5+wpTy6hY/i7c5K2DYICiNeDeGbxtd2IMrzzCXK3jLUT/Q/4wny6ShBbpUq08zNVUiIm9pN0L X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: There are three similar loops to initialize kfence pool, we could merge all of them into one loop to simplify the code and make code more efficient. Signed-off-by: Muchun Song Reviewed-by: Marco Elver --- mm/kfence/core.c | 47 ++++++----------------------------------------- 1 file changed, 6 insertions(+), 41 deletions(-) diff --git a/mm/kfence/core.c b/mm/kfence/core.c index 7d01a2c76e80..de62a84d4830 100644 --- a/mm/kfence/core.c +++ b/mm/kfence/core.c @@ -539,35 +539,10 @@ static void rcu_guarded_free(struct rcu_head *h) static unsigned long kfence_init_pool(void) { unsigned long addr = (unsigned long)__kfence_pool; - struct page *pages; int i; if (!arch_kfence_init_pool()) return addr; - - pages = virt_to_page(__kfence_pool); - - /* - * Set up object pages: they must have PG_slab set, to avoid freeing - * these as real pages. - * - * We also want to avoid inserting kfence_free() in the kfree() - * fast-path in SLUB, and therefore need to ensure kfree() correctly - * enters __slab_free() slow-path. - */ - for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) { - struct slab *slab = page_slab(nth_page(pages, i)); - - if (!i || (i % 2)) - continue; - - __folio_set_slab(slab_folio(slab)); -#ifdef CONFIG_MEMCG - slab->memcg_data = (unsigned long)&kfence_metadata[i / 2 - 1].objcg | - MEMCG_DATA_OBJCGS; -#endif - } - /* * Protect the first 2 pages. The first page is mostly unnecessary, and * merely serves as an extended guard page. However, adding one @@ -581,8 +556,9 @@ static unsigned long kfence_init_pool(void) addr += PAGE_SIZE; } - for (i = 0; i < CONFIG_KFENCE_NUM_OBJECTS; i++) { + for (i = 0; i < CONFIG_KFENCE_NUM_OBJECTS; i++, addr += 2 * PAGE_SIZE) { struct kfence_metadata *meta = &kfence_metadata[i]; + struct slab *slab = page_slab(virt_to_page(addr)); /* Initialize metadata. */ INIT_LIST_HEAD(&meta->list); @@ -593,26 +569,15 @@ static unsigned long kfence_init_pool(void) /* Protect the right redzone. */ if (unlikely(!kfence_protect(addr + PAGE_SIZE))) - goto reset_slab; - - addr += 2 * PAGE_SIZE; - } - - return 0; - -reset_slab: - for (i = 0; i < KFENCE_POOL_SIZE / PAGE_SIZE; i++) { - struct slab *slab = page_slab(nth_page(pages, i)); + return addr; - if (!i || (i % 2)) - continue; + __folio_set_slab(slab_folio(slab)); #ifdef CONFIG_MEMCG - slab->memcg_data = 0; + slab->memcg_data = (unsigned long)&meta->objcg | MEMCG_DATA_OBJCGS; #endif - __folio_clear_slab(slab_folio(slab)); } - return addr; + return 0; } static bool __init kfence_init_pool_early(void)