From patchwork Wed Jan 25 07:34:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Leonardo Bras X-Patchwork-Id: 13115046 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21DF3C27C76 for ; Wed, 25 Jan 2023 07:35:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B9AC6B0075; Wed, 25 Jan 2023 02:35:54 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9695A6B0078; Wed, 25 Jan 2023 02:35:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 80C236B007B; Wed, 25 Jan 2023 02:35:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 713FA6B0075 for ; Wed, 25 Jan 2023 02:35:54 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 4DD45C04BD for ; Wed, 25 Jan 2023 07:35:54 +0000 (UTC) X-FDA: 80392512228.12.B733B25 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by imf25.hostedemail.com (Postfix) with ESMTP id 4F9D9A0004 for ; Wed, 25 Jan 2023 07:35:52 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iRRbMiVl; spf=pass (imf25.hostedemail.com: domain of leobras@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674632152; a=rsa-sha256; cv=none; b=t0ce6o3wpPwrT+mwz1LHDSDxXuPf6Av3e6x6fzUnKGSUXJyB++WAS46GGOjvB35I02+rg9 JzNY6h2XLmNAjfiaT/LjgIUWdBd0Ziu2vpleXV2LLyuwd9SHKEEy5OxBht2l62bma7xRxN LUFtqAy5rf9KBU2fZbsphFTM0ECzqkc= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=pass header.d=redhat.com header.s=mimecast20190719 header.b=iRRbMiVl; spf=pass (imf25.hostedemail.com: domain of leobras@redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=leobras@redhat.com; dmarc=pass (policy=none) header.from=redhat.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674632152; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KVG76aWPU3FY34u+v28shR/7Qg472WeM1BX+arjDbFU=; b=N+uYeyJX8vKfdzHthYXT2ZLfufQcU5iLXc4+64XL/NcUwZCKEHrCrsjFyO7bhIFeSBz+Tr jvDBW2GevCXx/ememkV2m6KFx5vW/GfliZjoX3ADrJlMj+7/9Ovl9HYWPUJZR3hBUe1o0h j2qcfpZ90XzbDpK1rM318M8MQ6bL8B8= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1674632151; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=KVG76aWPU3FY34u+v28shR/7Qg472WeM1BX+arjDbFU=; b=iRRbMiVl6RZKBLIn0RBTwIA/asmhzguVrV+5U1CCgd8BBrwPbW6IZyw7vuS4W9zwW/wlPP hqXHhPbNv+kjqgb1Hwf6WK1nYrg5f6yrfkJRv8A3D4H5hjFmOou2Vz0M8FSR0tODs2O3Un /uXvyzFG7hd4/7hk3Ke9NHq1Dt9fdR8= Received: from mail-oi1-f200.google.com (mail-oi1-f200.google.com [209.85.167.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_128_GCM_SHA256) id us-mta-614-qtfqlQ_FP2ip4pllzwjdIw-1; Wed, 25 Jan 2023 02:35:50 -0500 X-MC-Unique: qtfqlQ_FP2ip4pllzwjdIw-1 Received: by mail-oi1-f200.google.com with SMTP id z8-20020a056808064800b0036aae651f2aso5586512oih.17 for ; Tue, 24 Jan 2023 23:35:50 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KVG76aWPU3FY34u+v28shR/7Qg472WeM1BX+arjDbFU=; b=4saPrwpmbF4J/lk9c+QYSbmd3WIkdE9cgjdK29H39L0uIjolge603J8FtOOoFVezzB Y6WXt5KXgw1aN4Zc7mzSmOSzcCZvu8KzexlPmUFow1Vt08nVQWvKASzvqk9efPuhk1ll 3a6V3OxJ7pwabRyIMHfwKlDjToXRLZ9LlRXvhWhL9gGoSEtU7MjWnp1KnUHfsLOi/fcj 0EYzlSJbXY4BErj996b+mkm9yRg8du/YboNUIuGLvjAiysqyWf00WucpSD3ESPt6G4+U JJTXYVYHucXi7CqAnU45m4jFt/KPCHaHQdjMGTAKWJxb91d1uswsahkaQLMcO27bRBJ7 3lqQ== X-Gm-Message-State: AFqh2koKK14o5t0mB7s9NSnb/cb6DBFS1bZB3/SdAOlkbSzIAXukdieN cN6iUnMMO9kxrRY7PS5fA+SIW3ZNVjQDPQd5WoSZer61sZ/ZksWWnkaKO+qaT6GslvxHeltXZWP 5MwCIKBLc8/4= X-Received: by 2002:a05:6870:8dc5:b0:15f:74b2:9168 with SMTP id lq5-20020a0568708dc500b0015f74b29168mr14415646oab.4.1674632149720; Tue, 24 Jan 2023 23:35:49 -0800 (PST) X-Google-Smtp-Source: AMrXdXva0ztnWMLaJL80saQFKdei152TK3so9LzytflfDAcnAzjhuVwqsk6VSRdFSyzYlYX+q5Awzg== X-Received: by 2002:a05:6870:8dc5:b0:15f:74b2:9168 with SMTP id lq5-20020a0568708dc500b0015f74b29168mr14415637oab.4.1674632149469; Tue, 24 Jan 2023 23:35:49 -0800 (PST) Received: from LeoBras.redhat.com ([2804:1b3:a800:14fa:9361:c141:6c70:c877]) by smtp.gmail.com with ESMTPSA id x189-20020a4a41c6000000b0050dc79bb80esm1538802ooa.27.2023.01.24.23.35.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 24 Jan 2023 23:35:48 -0800 (PST) From: Leonardo Bras To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Marcelo Tosatti Cc: Leonardo Bras , cgroups@vger.kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/5] mm/memcontrol: Align percpu memcg_stock to cache Date: Wed, 25 Jan 2023 04:34:58 -0300 Message-Id: <20230125073502.743446-2-leobras@redhat.com> X-Mailer: git-send-email 2.39.1 In-Reply-To: <20230125073502.743446-1-leobras@redhat.com> References: <20230125073502.743446-1-leobras@redhat.com> MIME-Version: 1.0 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-Rspam-User: X-Rspamd-Queue-Id: 4F9D9A0004 X-Rspamd-Server: rspam01 X-Stat-Signature: 8mfa9asj8jiitjtxrw6cg3su3gnmmo8p X-HE-Tag: 1674632152-251831 X-HE-Meta: U2FsdGVkX18eF4NnfspS/suUjE8BtvQVikdLW8LeYtlGwQJaMz/QQG+axD70LOCg0iXq5AxCmISSSx1sEQO8x96q1KsaRfousnWrAQTK52CDjF/Ny2d4ymC5w4vVHOLRFPyyWA0MiTl0VExpgyuvBMoNlVRucCUEP2a8lMV9AN0Zr/znn/zg+NbvYIUcf5NfTyfhIdfP/5/yr8S7GaEC9VX7NCa9u8HLy9R9KvtzMD11fXgyfxDibAe3RsKmtxzwIapzBLeP3vHYuv0pte3P9rtpCahVMiv8+zs/cPd7RrtAMU+WoRlOpdl6VP2Zr/gBTCX2QD1iTiLdYlHlWDolfNvBbn3VR3yAmi8LLDxQj6lUvzysfayHAVpqi4jP7jFahVae46zLhCdKMy5bJHUsgu9WTjFsEnPel9PKi4K+anXUArkUirkP/SRwQ4g1P4F+HquOGnCjHq3SFhBt9GzZRNy6JcTpS6b4O0XZHNhGCLAOSQGmu/RthEUR82x3da5kZa9AFQVaxBHNfI+35Lf6H+qAUtXVcfA1jvMFmIadlCVgwX+sk4peYedv1XA/o4+pDXUMLEgf7Mn/1UQOi6bMesvw7Se9rKgaASUSJ+JjoEnXMPkSvmDKxXdAogaRf3sQl/YJfUCgwLXzT8r9P+J3tobSNB99s9slpKsPJlAn8nADUBBB/qrD/UTN/Hec7MEomarqDIIJYgTQWxbsH2TZXF+QyDQuBOUgtDXxh5rw+WHKUn4FOvioS65BfhMP1ITeB4i/2aMRZsnljtDlb8wQNhBgxs3XwXuRdn095voM+BB1VpYc0UO2u6iC9MEgwQR/k2Uroghdskcm9SbNxT/H7z6lsytvXYFXgtbTeCBwcYloGrUFa6TU98ATxS5LVLp8muPY6OO+q9Ml2B+d/TxPjXX7jOyFQdzYIvFTGeCEkWkrfCPZ1aotr2bj4DJvY6I7lI/t++sw6vVsK98dQGe X3K28boM 3FkP6jq18W5uZUEXuydKWtatGn7lGWP0amY3TUHH1k8Lm8KlVRag4EBs30H4LF8zfwbnsNhv4cbmY/eJ027sJAEb6V0ZSFN6+21bhlBWAk0pdn3ngpSTJ4XcmEbGb1CVfFzsnESfrVdvs95KTcNOd7bzKB3Bzh03pQjG3jn/nG8QxG7TQFmUr83GTdbkEfVAOovVRFw13hodvFDAH1ujk5uUSC/qXjYAZEvBzX+haGCutxBsVO9JjgeYIxZ8jGpAw4EZ3vgvK4bD44/4utmNt18RvbVsio/ajn5mN/UpQmhg7BIkMpk5bWnKJ40XPl7VmyIA11jb4XOJOi10wAp9F4Jd8zDdN/qJPRpoPX3nrmS4eSYS3q/6tUkr7SinDSUusujWdkhrrDQU/8SrFRTllVfsNvnXX6WeL6jLhxr5ePHrYE7egSjZJqiFGLw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When a struct smaller than a cacheline has an instance that is not aligned to cache block size, it could happen that data can be spread across two cachelines. For memcg_stock this could mean the need to fetch and get cache-exclusivity in 2 cachelines instead of 1 when we bounce the cacheline between local cpu functions and drain_all_stock(), which does remote read/write. This could also mean some false-sharing costs being paid due to the cacheline being shared between 2 unrelated structures. Avoid this issue by getting memcg_stock cacheline-aligned. Signed-off-by: Leonardo Bras --- mm/memcontrol.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index ab457f0394ab6..f8e86b88b3c7a 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2188,7 +2188,8 @@ struct memcg_stock_pcp { unsigned long flags; #define FLUSHING_CACHED_CHARGE 0 }; -static DEFINE_PER_CPU(struct memcg_stock_pcp, memcg_stock) = { + +static DEFINE_PER_CPU_SHARED_ALIGNED(struct memcg_stock_pcp, memcg_stock) = { .stock_lock = INIT_LOCAL_LOCK(stock_lock), }; static DEFINE_MUTEX(percpu_charge_mutex);