From patchwork Tue Aug 21 13:50:59 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 1355131 Return-Path: X-Original-To: patchwork-linux-arm@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork2.kernel.org Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) by patchwork2.kernel.org (Postfix) with ESMTP id DCB30DFB34 for ; Tue, 21 Aug 2012 13:59:25 +0000 (UTC) Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.76 #1 (Red Hat Linux)) id 1T3orb-0008Cz-Hh; Tue, 21 Aug 2012 13:51:23 +0000 Received: from cam-admin0.cambridge.arm.com ([217.140.96.50]) by merlin.infradead.org with esmtp (Exim 4.76 #1 (Red Hat Linux)) id 1T3orX-0008AM-Lv for linux-arm-kernel@lists.infradead.org; Tue, 21 Aug 2012 13:51:20 +0000 Received: from mudshark.cambridge.arm.com (mudshark.cambridge.arm.com [10.1.79.58]) by cam-admin0.cambridge.arm.com (8.12.6/8.12.6) with ESMTP id q7LDp8E7014305; Tue, 21 Aug 2012 14:51:08 +0100 (BST) Received: by mudshark.cambridge.arm.com (Postfix, from userid 1000) id A78BBC1964; Tue, 21 Aug 2012 14:51:04 +0100 (BST) From: Will Deacon To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2] ARM: io: avoid writeback addressing modes for __raw_ accessors Date: Tue, 21 Aug 2012 14:50:59 +0100 Message-Id: <1345557059-6922-1-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 1.7.4.1 X-Spam-Note: CRM114 invocation failed X-Spam-Score: -7.1 (-------) X-Spam-Report: SpamAssassin version 3.3.2 on merlin.infradead.org summary: Content analysis details: (-7.1 points) pts rule name description ---- ---------------------- -------------------------------------------------- -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high trust [217.140.96.50 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.2 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Cc: Marc Zyngier , Will Deacon , nico@fluxnic.net X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: linux-arm-kernel-bounces@lists.infradead.org Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org Data aborts taken to hyp mode do not provide a valid instruction syndrome field in the HSR if the faulting instruction is a memory access using a writeback addressing mode. For hypervisors emulating MMIO accesses to virtual peripherals, taking such an exception requires disassembling the faulting instruction in order to determine the behaviour of the access. Since this requires manually walking the two stages of translation, the world must be stopped to prevent races against page aging in the guest, where the first-stage translation is invalidated after the hypervisor has translated to an IPA and the physical page is reused for something else. This patch avoids taking this heavy performance penalty when running Linux as a guest by ensuring that our I/O accessors do not make use of writeback addressing modes. Cc: Marc Zyngier Reviewed-by: Nicolas Pitre Signed-off-by: Will Deacon Reviewed-by: Arnd Bergmann --- Changes since v1: - Use "Qo" to allow for pre-indexing addressing modes arch/arm/include/asm/io.h | 67 +++++++++++++++++++++++++++++++++++++++++---- 1 files changed, 61 insertions(+), 6 deletions(-) diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h index 815c669..09c4628 100644 --- a/arch/arm/include/asm/io.h +++ b/arch/arm/include/asm/io.h @@ -47,13 +47,68 @@ extern void __raw_readsb(const void __iomem *addr, void *data, int bytelen); extern void __raw_readsw(const void __iomem *addr, void *data, int wordlen); extern void __raw_readsl(const void __iomem *addr, void *data, int longlen); -#define __raw_writeb(v,a) ((void)(__chk_io_ptr(a), *(volatile unsigned char __force *)(a) = (v))) -#define __raw_writew(v,a) ((void)(__chk_io_ptr(a), *(volatile unsigned short __force *)(a) = (v))) -#define __raw_writel(v,a) ((void)(__chk_io_ptr(a), *(volatile unsigned int __force *)(a) = (v))) +#if __LINUX_ARM_ARCH__ < 6 +/* + * Half-word accesses are problematic with RiscPC due to limitations of + * the bus. Rather than special-case the machine, just let the compiler + * generate the access for CPUs prior to ARMv6. + */ +#define __raw_readw(a) (__chk_io_ptr(a), *(volatile unsigned short __force *)(a)) +#define __raw_writew(v,a) ((void)(__chk_io_ptr(a), *(volatile unsigned short __force *)(a) = (v))) +#else +/* + * When running under a hypervisor, we want to avoid I/O accesses with + * writeback addressing modes as these incur a significant performance + * overhead (the address generation must be emulated in software). + */ +static inline void __raw_writew(u16 val, volatile void __iomem *addr) +{ + asm volatile("strh %1, %0" + : "+Qo" (*(volatile u16 __force *)addr) + : "r" (val)); +} + +static inline u16 __raw_readw(const volatile void __iomem *addr) +{ + u16 val; + asm volatile("ldrh %1, %0" + : "+Qo" (*(volatile u16 __force *)addr), + "=r" (val)); + return val; +} +#endif -#define __raw_readb(a) (__chk_io_ptr(a), *(volatile unsigned char __force *)(a)) -#define __raw_readw(a) (__chk_io_ptr(a), *(volatile unsigned short __force *)(a)) -#define __raw_readl(a) (__chk_io_ptr(a), *(volatile unsigned int __force *)(a)) +static inline void __raw_writeb(u8 val, volatile void __iomem *addr) +{ + asm volatile("strb %1, %0" + : "+Qo" (*(volatile u8 __force *)addr) + : "r" (val)); +} + +static inline void __raw_writel(u32 val, volatile void __iomem *addr) +{ + asm volatile("str %1, %0" + : "+Qo" (*(volatile u32 __force *)addr) + : "r" (val)); +} + +static inline u8 __raw_readb(const volatile void __iomem *addr) +{ + u8 val; + asm volatile("ldrb %1, %0" + : "+Qo" (*(volatile u8 __force *)addr), + "=r" (val)); + return val; +} + +static inline u32 __raw_readl(const volatile void __iomem *addr) +{ + u32 val; + asm volatile("ldr %1, %0" + : "+Qo" (*(volatile u32 __force *)addr), + "=r" (val)); + return val; +} /* * Architecture ioremap implementation.