From patchwork Thu Mar 1 12:53:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 10251411 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id C4DB460211 for ; Thu, 1 Mar 2018 13:32:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B1DC31FF65 for ; Thu, 1 Mar 2018 13:32:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A69FA201B1; Thu, 1 Mar 2018 13:32:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 391081FF65 for ; Thu, 1 Mar 2018 13:32:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=fkqEILu7ozlV48ImzIirzAtAQPH8iSx4Qxi5xqnppIY=; b=rGFyisiGhIfD4Lc9B5FR31DZ5r NjAl/RqM2ZIgizoi6v9nHxvsYmURw7yvsXkBqwnqduepWybisSGzRKgmJWuzeQ6QwFIT7S+wAWT7+ GjOfvpEyN1sXaVb2ISWbmlhyXOXXDfO6QihlJDRyrl5NZ7wDHkkWSxmF/Ft0r+fZC/Knr2MVMcJCN pEOSWDJMIO3WofxDY3xa7SLEPlQ/rssUvdhJBfsGOZz8wGeOg3LBhUY0+gKmX0VnAQREJ/pOThN71 PwOBMu0tXY8Bo35UeXfh/2UXiDI0mNAyZaykODWJdrUq20y0uvoCoHXafy6ApBvdjBw4Ghq7+QZlz 7BFz/DLg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1erOJw-0004jV-92; Thu, 01 Mar 2018 13:32:28 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1erO0a-0005gV-La for linux-arm-kernel@bombadil.infradead.org; Thu, 01 Mar 2018 13:12:28 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=References:In-Reply-To:Message-Id:Date: Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=j0ATirbPRkpiwskTS+miQg40S RxYfW9VgQ7i1veXcSYF9+25jLxY3ExOqYvVHpvvRSFzjOfNoZRdl0LyuyuXUfKaWz5Wrg0DOwXpaD PekSWvb+HbzdyqyFa/TEVh0xio+1LvHKGn8Xn5gL3zWAwWu2KvHdAcAU64GudAX+LJzgWe+DWLNqe ZdokMj73Hco7Q9Xns2dEq/3L5zzpBlybawJrERayUVjSWIXuhdf2rHWXB4TNeEOdfY3G+cEZOKGZC IaEn9XOWZREtNVCPE812e/YJkdSICrJV7zOuM7JMhyA3VCpdrrs1KB6eZ0LVdIzF+qHO4FRN5mTLr AK4HsCf6w==; Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]) by casper.infradead.org with esmtps (Exim 4.89 #1 (Red Hat Linux)) id 1erNlD-00050O-Ey for linux-arm-kernel@lists.infradead.org; Thu, 01 Mar 2018 12:56:37 +0000 Received: by mail-pf0-x241.google.com with SMTP id f80so2398104pfa.8 for ; Thu, 01 Mar 2018 04:56:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=TWc8MaydIeTgVwL1bNsWb6speWePBk5cwbeu8tE+wxKd8SsD5cVfgBhwWptS4TtVbN wty/oSLnY4XVK8XAfxr+3+LIXVfEnx+pHfKFbR1prFS0wzbjhvJWGXHXEqmyLXloSg1Q VEnFT/xBJHIOyP5e6cVXuVBA4TQn4CnmF3Mt4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=pkSwvmKMHNNLRhymKkKAhX1GDqTiFiQOpUbZ8tKMBgk=; b=Wqp4InVsRjlWkWW3yQti1oe0GP0sbbFbsDMzy0SeXThyqinMeZ0OGtZ81beV9UAlge 2398M1hYrOxxQ277avhj4BNYjjKHM6xmiRVcrFSgaLCxnEoLR/6KvlXCaYG8/zRDbFEs nKVd/VBeZMh5eoPIcjPP03ppL1DX6jg+dR7Jo6oqCztceioLlFYckwa342l+b1RCbcBs f9sVuFQLEN5h3KojyTeiYYfraNBTJdk6zxgRo2un5oye7pw05RI4ModvlAMa2kjeYVzb HywVsbgv8+M79J9GGbI/X9lBP307qbjngiNy7UWfcZJ2CBUeDUnugm25+jmHzHeOLt7Y Ggsg== X-Gm-Message-State: APf1xPDJ8pYsz7zHvquoAPwLMZEKcAPUf9u9RBAay2GXrsmnjtof7NUh 2rYwYLvqbXVc9usjOtdCjSQWuA== X-Google-Smtp-Source: AG47ELsvIplkm0XEHK0OqbtpxkC0UEHc61Va9etLnkxO9CF4MVtSNqgxPRgmFmnB8zxrcsKBQ7DNCw== X-Received: by 10.101.86.198 with SMTP id w6mr1462142pgs.434.1519908983165; Thu, 01 Mar 2018 04:56:23 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id x4sm2289655pfb.46.2018.03.01.04.56.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 01 Mar 2018 04:56:22 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH 09/45] arm64: Make USER_DS an inclusive limit Date: Thu, 1 Mar 2018 20:53:46 +0800 Message-Id: <1519908862-11425-10-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> References: <1519908862-11425-1-git-send-email-alex.shi@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180301_125635_502395_C1BE3A7D X-CRM114-Status: GOOD ( 26.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Shi , Robin Murphy MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP From: Robin Murphy commit 51369e398d0d upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas (cherry picked from commit 83b20dff71ea949431cf57c6aebaaf7ebd5c1991) Signed-off-by: Alex Shi --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/uaccess.h | 46 ++++++++++++++++++++++---------------- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/fault.c | 2 +- 4 files changed, 32 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 4258f4d..5917147 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 09c9b59..7b1eb49 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -36,6 +36,7 @@ #include #include #include +#include #define VERIFY_READ 0 #define VERIFY_WRITE 1 @@ -62,10 +63,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -90,22 +88,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -114,7 +122,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs #define _ASM_EXTABLE(from, to) \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 478f0fe..6915697 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -103,7 +103,7 @@ /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] /* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */ .endif /* \el == 0 */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 403fe9e..4df70c9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -332,7 +332,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (is_permission_fault(esr) && (addr < USER_DS)) { + if (is_permission_fault(esr) && (addr < TASK_SIZE)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die("Accessing user space memory with fs=KERNEL_DS", regs, esr);