From patchwork Fri Mar 9 09:06:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 10270087 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 62BE060236 for ; Fri, 9 Mar 2018 09:10:41 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5AF2029D9A for ; Fri, 9 Mar 2018 09:10:41 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4FBEE29D9F; Fri, 9 Mar 2018 09:10:41 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9E98C29D9A for ; Fri, 9 Mar 2018 09:10:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=ja3oI4KLBV7vqKAH97MgvgUdI16kiJFWZoB+M6WNbnQ=; b=TWoq/7ls5nHu2LOA1isSVQOAML QCMyMO3N9Wah2AtRkH2RhhpJ6xoHXF2e3FFURDRc86aZXVlllWn1Qzll3RzaSRxAv8kCzj4K8GdXn 8HGJ7tJVkcpHFotseS0+ZOVTgxRgT9xyE6KfhHtut31yevZHz3uX+EyjSVku1tsb0xtt2oE6Cn004 z+aSKYeo+rctfsSzFx6xcZ/3QqkxXL0V1MAqJdFotwmymDKdqAgrl+IZRx95c32HwT+iEKd7thBni TJqrNOKkWZACJsJBxtdyladQSq2XfjXVoZHfAeRpCrkAcD4IaN4VnPNMgYdCkuf7nYPtyJISx8sOF 4mI87LSQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1euE2q-0005SA-MF; Fri, 09 Mar 2018 09:10:32 +0000 Received: from mail-pl0-x243.google.com ([2607:f8b0:400e:c01::243]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1euE12-0003RK-Aq for linux-arm-kernel@lists.infradead.org; Fri, 09 Mar 2018 09:08:48 +0000 Received: by mail-pl0-x243.google.com with SMTP id u13-v6so4941536plq.1 for ; Fri, 09 Mar 2018 01:08:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=PReSpBXxelZ0QGBS5R1wbJaAcUtP4eazdaP7TKgJ8AY=; b=dvq/e/Ba7NOzDL/+oVBdBuwvJSrvetdOoQzVh3fdzdQuMz7SwuuLbNNg4nHl4ycP43 qBJQOqZvZ/E39LNJk/shSx7EAwtJeyV+r2bh0FJYLuN3Iu9b6tAM6jEvsqqaWmHdrq42 mRgQBpospje9aQNyc56rDHesBUNJTLhHMIoZw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=PReSpBXxelZ0QGBS5R1wbJaAcUtP4eazdaP7TKgJ8AY=; b=mnPCTXAFnuCIHB/47AC4qC2NLrYpO4TcGDcDV75jQ56ExZ9JdgulDbN2iBPC8U3kxd VSyXc7jc6pFVMq2RqcqgVRnripCBOvy86b+H7dydUKTitRq+KanH7F7o7ZYG5RwwtWA7 omGwmnE3Ib0ryyEYGIya5g6jIknCXp2B96VTH6IwNsCdsk7w6E9Kwl2WYJrzPeThpC1Y iA+spl/+eVKIpuj/Hq7ywPLjecORVAcCg7JHQEfVZxHrVD3vvB9PYlB5ZOZdF9oS8Hin /Gto4e0zf6RiUeqdM0AHD/GU9iwsuTTKnHLsgNYKgRDnFaXKyAyHkhqPFi72rPcZvvxT /UrA== X-Gm-Message-State: AElRT7Hu0W3EJ15gaEiCWBD0pdcs5s+XqxCCKLs5DOgqC+PEsYDvPZK2 D8TemIY4BKPrHPvTBhCR6yfF2A== X-Google-Smtp-Source: AG47ELvQQebn9x1U1QolL26ltS/xEd6uXSDvfmemgq0kHcJ1xQJ62DnUuPqlcdCxjdoWpogG3YtMUQ== X-Received: by 2002:a17:902:b691:: with SMTP id c17-v6mr176376pls.308.1520586517719; Fri, 09 Mar 2018 01:08:37 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id c7sm1961752pfg.36.2018.03.09.01.08.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 09 Mar 2018 01:08:37 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , broonie@linaro.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH 05/39] arm64: Make USER_DS an inclusive limit Date: Fri, 9 Mar 2018 17:06:48 +0800 Message-Id: <20180309090722.26279-6-alex.shi@linaro.org> X-Mailer: git-send-email 2.16.2.440.gc6284da In-Reply-To: <20180309090722.26279-1-alex.shi@linaro.org> References: <20180309090722.26279-1-alex.shi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180309_010841_069905_FF5ADC90 X-CRM114-Status: GOOD ( 16.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Shi , Robin Murphy MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Robin Murphy Commit 51369e398d0d upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/include/asm/uaccess.h | 45 +++++++++++++++++------------- arch/arm64/kernel/entry.S | 2 +- 3 files changed, 30 insertions(+), 20 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 12d5b2b97f04..c49597ae529d 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 829fa6d3e561..c625cc5531fc 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -56,10 +56,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -87,22 +84,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -111,7 +118,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs /* diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index fbb4fdd370b9..274898465a3f 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -99,7 +99,7 @@ /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] .endif /* \el == 0 */ mrs x22, elr_el1