From patchwork Wed Apr 14 11:23:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 12202385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C30C1C433ED for ; Wed, 14 Apr 2021 11:27:34 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E53461164 for ; Wed, 14 Apr 2021 11:27:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E53461164 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=46VVG/gG/of2UJ0t6XjlnRwsXRKRZvQ+V9nUArgUUmU=; b=OglUmfHgUvjyRu1Fiwgkq+RcR 3UaEdlRByl3xuGG8PPUVPLfvDe6SCxoutsggt179zf87SYRXz1NbvfMWAEqKtCmAEZr9MOh5EoX90 bsqqkTwyPlCrfNgtx3Y/jPW6MkfclZRmo0QuxGGD/EFsFH1s966SkQ0eG7tRvc+I4BnQCx+YjGQO7 DW7q/vHVDKzgRH2+v43SMCTl7mnPGGysfuAaupMyvzSsA5AHodfB2zNWfk9EVMpjcdrD7yL92Z4vq oV2TzXI2b1eekjIv7oFqGr/ab+80/0Pfr49hqdmlcfK7Q/xM4LyAOsgcwmSnbm6sdRq0RqaAy8v2F +tH/XY8zg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWdeQ-00COuQ-Nj; Wed, 14 Apr 2021 11:25:43 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWddd-00COj3-Ho for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 11:24:54 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=WE9jGHcsvG/h5S1UesM49EOD5usYge8drRPUrtL2KdA=; b=E4RXPAok9SIkgtKHbEjK1wWax2 O3OfGIxQIwZ2RtUexOOdzZ+nfONV6Qv9tjWtQTQjz2eitLoaLiKyD1RZVFLQ0qzrXQRv/zmewZHe+ fHx7fFikxnbjFc3ushvC0lQ4QrRKxlNd0FV2kXBPHSftFQcixDmk0zfvrnC4ZkUJAdFtUqi8xho9L cecMU+jjVaiRwMwYuQ8lnvTzqY+znhaBohP7M5kB5MSoZknGA8Q9rTVYOPEAW6VyUIOUO+i3+W4cp 0aBLSKaIyS1KEcp4AY0XSQlInRw9aw84YH9CtWL0osZYhdFU8RAz8pAODd43AdNhCYy0gCeDO7xoW RKZlf4yw==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWdda-007i17-K4 for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 11:24:52 +0000 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.59]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4FL0SS0dntztW9W; Wed, 14 Apr 2021 19:22:32 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.47.82.32) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.498.0; Wed, 14 Apr 2021 19:24:39 +0800 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [PATCH v4 04/16] arm64/mm: Move the variable lock and tlb_flush_pending to asid_info Date: Wed, 14 Apr 2021 12:23:00 +0100 Message-ID: <20210414112312.13704-5-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20210414112312.13704-1-shameerali.kolothum.thodi@huawei.com> References: <20210414112312.13704-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.47.82.32] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210414_042451_016616_CCCCD46A X-CRM114-Status: GOOD ( 13.02 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Julien Grall The variables lock and tlb_flush_pending holds information for a given ASID allocator. So move them to the asid_info structure. Signed-off-by: Julien Grall Signed-off-by: Shameer Kolothum --- arch/arm64/mm/context.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 1fd40a42955c..139ebc161acb 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -17,8 +17,6 @@ #include #include -static DEFINE_RAW_SPINLOCK(cpu_asid_lock); - static struct asid_info { atomic64_t generation; @@ -27,6 +25,9 @@ static struct asid_info atomic64_t __percpu *active; u64 __percpu *reserved; u32 bits; + raw_spinlock_t lock; + /* Which CPU requires context flush on next call */ + cpumask_t flush_pending; } asid_info; #define active_asid(info, cpu) (*per_cpu_ptr((info)->active, cpu)) @@ -34,7 +35,6 @@ static struct asid_info static DEFINE_PER_CPU(atomic64_t, active_asids); static DEFINE_PER_CPU(u64, reserved_asids); -static cpumask_t tlb_flush_pending; static unsigned long max_pinned_asids; static unsigned long nr_pinned_asids; @@ -137,7 +137,7 @@ static void flush_context(struct asid_info *info) * Queue a TLB invalidation for each CPU to perform on next * context-switch */ - cpumask_setall(&tlb_flush_pending); + cpumask_setall(&info->flush_pending); } static bool check_update_reserved_asid(struct asid_info *info, u64 asid, @@ -253,7 +253,7 @@ void check_and_switch_context(struct mm_struct *mm) old_active_asid, asid)) goto switch_mm_fastpath; - raw_spin_lock_irqsave(&cpu_asid_lock, flags); + raw_spin_lock_irqsave(&info->lock, flags); /* Check that our ASID belongs to the current generation. */ asid = atomic64_read(&mm->context.id); if (!asid_gen_match(asid, info)) { @@ -262,11 +262,11 @@ void check_and_switch_context(struct mm_struct *mm) } cpu = smp_processor_id(); - if (cpumask_test_and_clear_cpu(cpu, &tlb_flush_pending)) + if (cpumask_test_and_clear_cpu(cpu, &info->flush_pending)) local_flush_tlb_all(); atomic64_set(&active_asid(info, cpu), asid); - raw_spin_unlock_irqrestore(&cpu_asid_lock, flags); + raw_spin_unlock_irqrestore(&info->lock, flags); switch_mm_fastpath: @@ -289,7 +289,7 @@ unsigned long arm64_mm_context_get(struct mm_struct *mm) if (!pinned_asid_map) return 0; - raw_spin_lock_irqsave(&cpu_asid_lock, flags); + raw_spin_lock_irqsave(&info->lock, flags); asid = atomic64_read(&mm->context.id); @@ -315,7 +315,7 @@ unsigned long arm64_mm_context_get(struct mm_struct *mm) refcount_set(&mm->context.pinned, 1); out_unlock: - raw_spin_unlock_irqrestore(&cpu_asid_lock, flags); + raw_spin_unlock_irqrestore(&info->lock, flags); asid &= ~ASID_MASK(info); @@ -336,14 +336,14 @@ void arm64_mm_context_put(struct mm_struct *mm) if (!pinned_asid_map) return; - raw_spin_lock_irqsave(&cpu_asid_lock, flags); + raw_spin_lock_irqsave(&info->lock, flags); if (refcount_dec_and_test(&mm->context.pinned)) { __clear_bit(asid2idx(info, asid), pinned_asid_map); nr_pinned_asids--; } - raw_spin_unlock_irqrestore(&cpu_asid_lock, flags); + raw_spin_unlock_irqrestore(&info->lock, flags); } EXPORT_SYMBOL_GPL(arm64_mm_context_put); @@ -426,6 +426,7 @@ static int asids_init(void) info->map_idx = 1; info->active = &active_asids; info->reserved = &reserved_asids; + raw_spin_lock_init(&info->lock); pinned_asid_map = kcalloc(BITS_TO_LONGS(NUM_USER_ASIDS(info)), sizeof(*pinned_asid_map), GFP_KERNEL);