From patchwork Tue Mar 30 10:57:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Pingfan Liu X-Patchwork-Id: 12172269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81CCFC433DB for ; Tue, 30 Mar 2021 11:00:34 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 55CEA6196C for ; Tue, 30 Mar 2021 11:00:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 55CEA6196C Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=YSKSXJqrTlovf1ImRPCb0LGyzIpyAnX/AA9N1AXW04M=; b=TPP8HeeqjN+wWlSGnzGOrWeVu 8zcorXLn+ZcE/iGdICtCu4QjPm6lmX5/i7Og7dztbILxgumP2CJUmxT5XdZzBVxkKNntd/rbjjzCD gfqSbkZ4PKrkAW5qJUpWogvNUMd5D0zULQkDM0ltMSF/l/DBJ0XYDu7NPFE5V6NPbw7uwpxH3pziy y+8+LvMvTKJokkheLaHEfaVOA6KWsTXn/T2cWr9iZIHj5FnD99G3N+uDnBmldBSnR91F0ldIoxcOk hH8ZMoHwuK0UXaYWhQk+SDXgdWyr0KZ1bQIybbc+Ka/zrgvGHxXFEMztH2hMl7/2ml7bWgJZYmsLy i7nx0VmVA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lRC54-003ULR-H1; Tue, 30 Mar 2021 10:58:42 +0000 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lRC4K-003UDs-Hz for linux-arm-kernel@lists.infradead.org; Tue, 30 Mar 2021 10:58:02 +0000 Received: by mail-pf1-x42b.google.com with SMTP id j25so11911318pfe.2 for ; Tue, 30 Mar 2021 03:57:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fJSbPvojFnbEF/MBW0oxVAl1MiFmeqCZww4wOyxMW9k=; b=a/ENeiYnVcIy2JAnFWn+h4IBhPsLtxDD7Sn9KlheEGfSyOh9G8MZnBa0LDCynGJHKQ nVN0zSoSICHf6yHga9VATv5iKnTQuwLbtBvzThlNQOGCcso2fekJXXYbx0/jqOCOZw57 pLqnyXVmJcOV7rcEInvNTJ8/c2cmk4RQ9G2RGkP5p7UwhOO+gIB9i7Tu6jdX2lkdwNtR /xEhWy0mBgvXf3rkozdn2Jh6WRnaD0cBBKYcZ8JO8GSNVEaoUkABTiHjh/qCiO5acX6/ A0ztdcYg3vVURMZ+vu+I2UBe8N4ED49bfWsMbR6GnT6myb39BV2XThwS9msxQpFkRnvX kjdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fJSbPvojFnbEF/MBW0oxVAl1MiFmeqCZww4wOyxMW9k=; b=UHtzmXjjw/TpQ4CYnEYyCveKyEvKFtL4pA6cxwLFAl1waNsOwJ+lJQsjP7qwNHNe5T UqRJCd6sWLHXec1Hj2YPou/sHzO24rlrI88xUiwMZxVbiPMS4P+t3mq3sThfVIPDDl3R +NQu6Ms9jEyfPXWwQh0Xx2ycoZEH3As6QyZ7/ypq1Z24acIfAamdWpwdV0MLf3Dt+xae Kbgtn4kU6TrdltPeaZGOdakEOMvz/fPGMu8Kr85Q70GehiFSoeP/tx40FUlGXnuBvYoU kGRQ9OxuVjY6hB069Gs1uhNTYclh0AXKvTxew8j3OytAE4pglNShElqrmggLBfBaViM7 0WLw== X-Gm-Message-State: AOAM530ApnDyIzxa9Pv8qJqLEWMIyPCIIHnlT8DFYTSc0YpaKh2/Haar S3tPE+Gf9XYjYbOmXh/3tUvfK3i4MQ== X-Google-Smtp-Source: ABdhPJyETbcV75D2ImaUUc5rTmhTgLolq8KDOX/OVNaVIefsyEHcWDIMqqOv8SLOGTuv6cHHr18pZw== X-Received: by 2002:a65:5a4a:: with SMTP id z10mr27337238pgs.240.1617101874788; Tue, 30 Mar 2021 03:57:54 -0700 (PDT) Received: from x1pad.redhat.com ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id 144sm20000439pfy.75.2021.03.30.03.57.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Mar 2021 03:57:54 -0700 (PDT) From: Pingfan Liu To: linux-arm-kernel@lists.infradead.org Cc: Pingfan Liu , Catalin Marinas , Will Deacon , Vincenzo Frascino , Thomas Gleixner , Mark Rutland , Andrei Vagin , Marc Zyngier Subject: [PATCH 2/2] arm64/arch_timer: replace arch_counter_enforce_ordering() with isb Date: Tue, 30 Mar 2021 18:57:19 +0800 Message-Id: <20210330105719.47760-3-kernelfans@gmail.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20210330105719.47760-1-kernelfans@gmail.com> References: <20210330105719.47760-1-kernelfans@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210330_115757_233515_DA3208FA X-CRM114-Status: GOOD ( 15.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The description of getting counter value is not very clear. [1] 'mrs Xt, cntpct' may execute out of program order, either forward or backward. Now taking a look at this group of getting counter routines. All of them are called from sched_clock(). And there is an isb to protect forward speculation. But there is no isb for the backward speculation. The current code enforces read dependency instructions anchored on getting counter. But it is not enough to protect against other no dependency instructions, and even function call can not prevent the speculation between getting counter and them. Replacing arch_counter_enforce_ordering() with isb to achieve the aim. [1]: AArch64 Programmer's Guides Generic Timer: 3.1. Count and frequency Signed-off-by: Pingfan Liu Cc: Catalin Marinas Cc: Will Deacon Cc: Vincenzo Frascino Cc: Thomas Gleixner Cc: Mark Rutland Cc: Andrei Vagin Cc: Marc Zyngier To: linux-arm-kernel@lists.infradead.org --- arch/arm64/include/asm/arch_timer.h | 34 ++++++++--------------------- 1 file changed, 9 insertions(+), 25 deletions(-) diff --git a/arch/arm64/include/asm/arch_timer.h b/arch/arm64/include/asm/arch_timer.h index 9f0ec21d6327..233ade46390f 100644 --- a/arch/arm64/include/asm/arch_timer.h +++ b/arch/arm64/include/asm/arch_timer.h @@ -165,32 +165,18 @@ static inline void arch_timer_set_cntkctl(u32 cntkctl) isb(); } -/* - * Ensure that reads of the counter are treated the same as memory reads - * for the purposes of ordering by subsequent memory barriers. - * - * This insanity brought to you by speculative system register reads, - * out-of-order memory accesses, sequence locks and Thomas Gleixner. - * - * http://lists.infradead.org/pipermail/linux-arm-kernel/2019-February/631195.html - */ -#define arch_counter_enforce_ordering(val) do { \ - u64 tmp, _val = (val); \ - \ - asm volatile( \ - " eor %0, %1, %1\n" \ - " add %0, sp, %0\n" \ - " ldr xzr, [%0]" \ - : "=r" (tmp) : "r" (_val)); \ -} while (0) - static __always_inline u64 __arch_counter_get_cntpct_stable(void) { u64 cnt; isb(); cnt = arch_timer_reg_read_stable(cntpct_el0); - arch_counter_enforce_ordering(cnt); + /* + * read dependency does not help against the sepculation between getting counter + * and no dependency instructions, which cause the overwritten of register value. + * So here needs isb. + */ + isb(); return cnt; } @@ -200,7 +186,7 @@ static __always_inline u64 __arch_counter_get_cntpct(void) isb(); cnt = read_sysreg(cntpct_el0); - arch_counter_enforce_ordering(cnt); + isb(); return cnt; } @@ -210,7 +196,7 @@ static __always_inline u64 __arch_counter_get_cntvct_stable(void) isb(); cnt = arch_timer_reg_read_stable(cntvct_el0); - arch_counter_enforce_ordering(cnt); + isb(); return cnt; } @@ -220,12 +206,10 @@ static __always_inline u64 __arch_counter_get_cntvct(void) isb(); cnt = read_sysreg(cntvct_el0); - arch_counter_enforce_ordering(cnt); + isb(); return cnt; } -#undef arch_counter_enforce_ordering - static inline int arch_timer_arch_init(void) { return 0;