From patchwork Wed Apr 14 11:23:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shameerali Kolothum Thodi X-Patchwork-Id: 12202395 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A86FDC433ED for ; Wed, 14 Apr 2021 11:29:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1F9B3613C8 for ; Wed, 14 Apr 2021 11:29:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1F9B3613C8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=huawei.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-ID:Date: Subject:CC:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=04xsZ+w47zLmJu8fRzkEe6+xQ0v9W57wlHb/uPuBhrI=; b=BHw8mw0aJeINbMEbTTw1l9glh 03Y6mNhOT+bI7KIU+XzrNwb1HabShjdYnfrCgbrvK8G/gIOwZZtWGQ8Cwm3xLgyMfcuLge6RbKlll OT2TLh6POPmWH2ReGd8bfG21Wq6DFhCu5D7Y9G6vmmu5fFGa0ao5iIXX7AyTJ4K6XLMujm6AuJO2S oJ3ONH2eDtJ4IsbaE+ghJwRcIaz937YV3Fhxvf4tShnCbkv/tCAAUgm3r+Sc2QCQKQf0cYYRFfKkf fGxGYC3UIv4kvHlgUSQ3vf/aCUaTz5jb7UIn1Dvt+7tWVLg7Kf1SyfpRJL3ypth6Bi3fEdYcxgVOm 8tR+UZ1iw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWdfs-00CPSZ-C5; Wed, 14 Apr 2021 11:27:12 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWddx-00COnq-Ew for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 11:25:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Date:Subject:CC:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=1KUBSehEABzbFhWQ2fXSchVUTX8EIdLJ6+HFAPBk/TQ=; b=Ry3GDxd0MSLNgej2+5ZXm/7X8X 12eYGLLvXsqvLXhD0wRUAQKSs6gw+YkmMBTmJ+g3twOHRLuwqfJ1w7njHFaNPmM+37OR8ZoiygY5O V4czhaJ8bS0vBsfSKmqhyG7mvst7xpQRTQgmcxaYybdM8P4UwOP7fOPB5eQTBl7K+LMUD4hjjkgXJ XpOAVEfmecFio4mmURCR5QwAhPkJ36G6Pe7+1bC2b1+nd+BYLU/PACil/2hAS38l14mMKz4IGydwS SnPtmddOJlih06qODNeKjd+HrKhBscglfJIbkfz0t+xFgqNXgojzLoaiDpqJP7Q32V1nRkVMxduO6 O8jGFbcg==; Received: from szxga05-in.huawei.com ([45.249.212.191]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWddu-007i3e-Ir for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 11:25:12 +0000 Received: from DGGEMS414-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4FL0S74W29zNtWp; Wed, 14 Apr 2021 19:22:15 +0800 (CST) Received: from S00345302A-PC.china.huawei.com (10.47.82.32) by DGGEMS414-HUB.china.huawei.com (10.3.19.214) with Microsoft SMTP Server id 14.3.498.0; Wed, 14 Apr 2021 19:25:00 +0800 From: Shameer Kolothum To: , , CC: , , , , , , , , Subject: [PATCH v4 09/16] arm64/mm: Split the function check_and_switch_context in 3 parts Date: Wed, 14 Apr 2021 12:23:05 +0100 Message-ID: <20210414112312.13704-10-shameerali.kolothum.thodi@huawei.com> X-Mailer: git-send-email 2.12.0.windows.1 In-Reply-To: <20210414112312.13704-1-shameerali.kolothum.thodi@huawei.com> References: <20210414112312.13704-1-shameerali.kolothum.thodi@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.47.82.32] X-CFilter-Loop: Reflected X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210414_042510_973179_6F3BA29A X-CRM114-Status: GOOD ( 17.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Julien Grall The function check_and_switch_context is used to: 1) Check whether the ASID is still valid 2) Generate a new one if it is not valid 3) Switch the context While the latter is specific to the MM subsystem, the rest could be part of the generic ASID allocator. After this patch, the function is now split in 3 parts which corresponds to the use of the functions: 1) asid_check_context: Check if the ASID is still valid 2) asid_new_context: Generate a new ASID for the context 3) check_and_switch_context: Call 1) and 2) and switch the context 1) and 2) have not been merged in a single function because we want to avoid to add a branch in when the ASID is still valid. This will matter when the code will be moved in separate file later on as 1) will reside in the header as a static inline function. Signed-off-by: Julien Grall Signed-off-by: Shameer Kolothum --- v3 comment: Will wants to avoid to add a branch when the ASID is still valid. So 1) and 2) are in separates function. The former will move to a new header and make static inline. --- arch/arm64/mm/context.c | 70 ++++++++++++++++++++++++++++------------- 1 file changed, 48 insertions(+), 22 deletions(-) diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 041c3c5e0216..40ef013c90c3 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -222,17 +222,49 @@ static u64 new_context(struct asid_info *info, atomic64_t *pasid, return idx2asid(info, asid) | generation; } -void check_and_switch_context(struct mm_struct *mm) +/* + * Generate a new ASID for the context. + * + * @pasid: Pointer to the current ASID batch allocated. It will be updated + * with the new ASID batch. + * @pinned: refcount if asid is pinned. + * Caller needs to make sure preempt is disabled before calling this function. + */ +static void asid_new_context(struct asid_info *info, atomic64_t *pasid, + refcount_t *pinned) { unsigned long flags; - unsigned int cpu; - u64 asid, old_active_asid; - struct asid_info *info = &asid_info; + u64 asid; + unsigned int cpu = smp_processor_id(); - if (system_supports_cnp()) - cpu_set_reserved_ttbr0(); + raw_spin_lock_irqsave(&info->lock, flags); + /* Check that our ASID belongs to the current generation. */ + asid = atomic64_read(pasid); + if (!asid_gen_match(asid, info)) { + asid = new_context(info, pasid, pinned); + atomic64_set(pasid, asid); + } - asid = atomic64_read(&mm->context.id); + if (cpumask_test_and_clear_cpu(cpu, &info->flush_pending)) + local_flush_tlb_all(); + + atomic64_set(&active_asid(info, cpu), asid); + raw_spin_unlock_irqrestore(&info->lock, flags); +} + +/* + * Check the ASID is still valid for the context. If not generate a new ASID. + * + * @pasid: Pointer to the current ASID batch + * @pinned: refcount if asid is pinned + * Caller needs to make sure preempt is disabled before calling this function. + */ +static void asid_check_context(struct asid_info *info, atomic64_t *pasid, + refcount_t *pinned) +{ + u64 asid, old_active_asid; + + asid = atomic64_read(pasid); /* * The memory ordering here is subtle. @@ -252,24 +284,18 @@ void check_and_switch_context(struct mm_struct *mm) if (old_active_asid && asid_gen_match(asid, info) && atomic64_cmpxchg_relaxed(this_cpu_ptr(info->active), old_active_asid, asid)) - goto switch_mm_fastpath; - - raw_spin_lock_irqsave(&info->lock, flags); - /* Check that our ASID belongs to the current generation. */ - asid = atomic64_read(&mm->context.id); - if (!asid_gen_match(asid, info)) { - asid = new_context(info, &mm->context.id, &mm->context.pinned); - atomic64_set(&mm->context.id, asid); - } + return; - cpu = smp_processor_id(); - if (cpumask_test_and_clear_cpu(cpu, &info->flush_pending)) - local_flush_tlb_all(); + asid_new_context(info, pasid, pinned); +} - atomic64_set(&active_asid(info, cpu), asid); - raw_spin_unlock_irqrestore(&info->lock, flags); +void check_and_switch_context(struct mm_struct *mm) +{ + if (system_supports_cnp()) + cpu_set_reserved_ttbr0(); -switch_mm_fastpath: + asid_check_context(&asid_info, &mm->context.id, + &mm->context.pinned); arm64_apply_bp_hardening();