From patchwork Fri Nov 30 17:06:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 10706857 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CD83513A4 for ; Fri, 30 Nov 2018 17:06:35 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B7E37304E5 for ; Fri, 30 Nov 2018 17:06:35 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB8BD304E4; Fri, 30 Nov 2018 17:06:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 02DC5304E5 for ; Fri, 30 Nov 2018 17:06:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 529606B5946; Fri, 30 Nov 2018 12:06:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 43BB06B5947; Fri, 30 Nov 2018 12:06:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 28D076B5949; Fri, 30 Nov 2018 12:06:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f69.google.com (mail-io1-f69.google.com [209.85.166.69]) by kanga.kvack.org (Postfix) with ESMTP id EEA026B5946 for ; Fri, 30 Nov 2018 12:06:19 -0500 (EST) Received: by mail-io1-f69.google.com with SMTP id k4so6081077ioc.10 for ; Fri, 30 Nov 2018 09:06:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:subject; bh=n6K3JZp3Sv6EziJOAs43HEtQARW9logLUVWDabZXNe8=; b=ZkW3uEC8m+5PpBGu+3PcH2uGBzkLLg4vd9YJFRQxKMw6vnHU7raqJb6Ls2zM1wvTXS CzNcCpxCWY8C2qgc2UYYkVkvjiBwebyL57csuM0tSzFu2UFXdsMmmvIMIFgQony17sVN wrHmmjZYCIzY1Y2M3Bd3C2eV+Ax+j2dAVnazydpxB+hxmg7p6UQf28xzdrQ0r2f9rf1R OmDA+HHbXuaYfDreQuVvkFZZPDK5yNdcK88hxf/PRxFqsJVuHrNjwxepkIKSM9JiAjVF K6aO7a7uENjwIpHvnbf8OwAOlZ+NiHLfRQ/ah1zsckqxpstCpzqWzyx1GLLdaDHOZMBL eY6w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com X-Gm-Message-State: AA+aEWY97wFPA7uMxXFCy3ZsJ6/qWPC4III84WY7sZIAYhDJLECNrGkB HHiKGy8fYiuTn1mzqOY2jcq1xJm5DY1OoMhYROgjpj0smUVRjE5WLSP1pMepDWkIYexw5agSBBy x2DChzymjUr0QRgFNOAjCYPDOyAWL3baLqsoGm4d9uZa8e0hlQpwHkABggpdkxHBRwg== X-Received: by 2002:a24:a08a:: with SMTP id o132mr6037669ite.1.1543597579739; Fri, 30 Nov 2018 09:06:19 -0800 (PST) X-Google-Smtp-Source: AFSGD/XO5HfVQ7ErxhRsxP/uyPf+yiBO41jJ6NFusxjgZ3jkIoMTIWp88I/b7gIgAQ7LAGtf9vgC X-Received: by 2002:a24:a08a:: with SMTP id o132mr6037613ite.1.1543597578958; Fri, 30 Nov 2018 09:06:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543597578; cv=none; d=google.com; s=arc-20160816; b=KmX22eOcwrQJx7+EP1VG+ixac/jsLdkX0imMgzovn71txkrZPO7PPMW+yRQ6Y5F5OG u4nnRWnroUEeqj5x9HpTb6BFpDVc4nrMDBlke25+dMBkNldg15nXoTUa4+baRyxcMuJk sknSUDQvZQX3pUoRk7y3dwchX4tKx7JPglZgZhxfBGAHmwQD+Jiuu+GWLyATP6LoBrcJ u6iGBszm4ZdMaRrCm2JmYn3EoLAzuI+DWIqw8GL2YoCYj5KDP5yWe84JpiYRacxENjI8 40Z4S6B8hJL6m7fRgAjtrYqLg8cjfDcAc8c5FpIq3HRfJQe3G2XUdW05jCy1YIQNBtQE bhgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=subject:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:cc:to:from; bh=n6K3JZp3Sv6EziJOAs43HEtQARW9logLUVWDabZXNe8=; b=qov4zTCHasJLhrwfxqNAu8TztEX+2hPlwPhk99/PK5YgO5oGV2dx4RneMeEgYzQK5M 7lnAlP62k02VbhIlTzyL1DMpy9z+a+ojcWEioj5xfClSLjLH+u333/Rxvv2m95dof0/P SHnH8tuqbU0SLxgkeFzaxuAWGirHn5OzaAewBD2Sn3Fqvep/LxF7M3IcfxNyDkmlUWJE 3NxXDimPDvXI0K1+BMwtkPMx+4nFXgpCCK44+mPtwxeqqNkyiwEogvv9tuCuBuGMC1oW PH2U3QnOP1kzsLoSDzeUsX3SZ6d2yvgDlUiSrsRIlQHFBeRl8DicFJIubajpX+vpHXjV /2xQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com Received: from ale.deltatee.com (ale.deltatee.com. [207.54.116.67]) by mx.google.com with ESMTPS id j204si3269505iof.88.2018.11.30.09.06.18 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Nov 2018 09:06:18 -0800 (PST) Received-SPF: pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) client-ip=207.54.116.67; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gSmF4-0000qo-2F; Fri, 30 Nov 2018 10:06:17 -0700 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.89) (envelope-from ) id 1gSmEx-0004VC-Mv; Fri, 30 Nov 2018 10:06:07 -0700 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-ntb@googlegroups.com, linux-crypto@vger.kernel.org, linux-mm@kvack.org, Andrew Morton Cc: Arnd Bergmann , Greg Kroah-Hartman , Andy Shevchenko , =?utf-8?q?Horia_Geant=C4=83?= , Logan Gunthorpe , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Suresh Warrier , Nicholas Piggin Date: Fri, 30 Nov 2018 10:06:03 -0700 Message-Id: <20181130170606.17252-4-logang@deltatee.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20181130170606.17252-1-logang@deltatee.com> References: <20181130170606.17252-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-ntb@googlegroups.com, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-crypto@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, arnd@arndb.de, gregkh@linuxfoundation.org, horia.geanta@nxp.com, logang@deltatee.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, warrier@linux.vnet.ibm.com, andy.shevchenko@gmail.com, npiggin@gmail.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v24 3/6] iomap: introduce io{read|write}64_{lo_hi|hi_lo} X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In order to provide non-atomic functions for io{read|write}64 that will use readq and writeq when appropriate. We define a number of variants of these functions in the generic iomap that will do non-atomic operations on pio but atomic operations on mmio. These functions are only defined if readq and writeq are defined. If they are not, then the wrappers that always use non-atomic operations from include/linux/io-64-nonatomic*.h will be used. Signed-off-by: Logan Gunthorpe Reviewed-by: Andy Shevchenko Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Arnd Bergmann Cc: Suresh Warrier Cc: Nicholas Piggin --- arch/powerpc/include/asm/io.h | 2 + include/asm-generic/iomap.h | 22 ++++++ lib/iomap.c | 132 ++++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+) diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h index e746becd9d6f..093a61795b14 100644 --- a/arch/powerpc/include/asm/io.h +++ b/arch/powerpc/include/asm/io.h @@ -781,8 +781,10 @@ extern void __iounmap_at(void *ea, unsigned long size); #define mmio_read16be(addr) readw_be(addr) #define mmio_read32be(addr) readl_be(addr) +#define mmio_read64be(addr) readq_be(addr) #define mmio_write16be(val, addr) writew_be(val, addr) #define mmio_write32be(val, addr) writel_be(val, addr) +#define mmio_write64be(val, addr) writeq_be(val, addr) #define mmio_insb(addr, dst, count) readsb(addr, dst, count) #define mmio_insw(addr, dst, count) readsw(addr, dst, count) #define mmio_insl(addr, dst, count) readsl(addr, dst, count) diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index 5b63b94ef6b5..a008f504a2d0 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -36,6 +36,17 @@ extern u64 ioread64(void __iomem *); extern u64 ioread64be(void __iomem *); #endif +#ifdef readq +#define ioread64_lo_hi ioread64_lo_hi +#define ioread64_hi_lo ioread64_hi_lo +#define ioread64be_lo_hi ioread64be_lo_hi +#define ioread64be_hi_lo ioread64be_hi_lo +extern u64 ioread64_lo_hi(void __iomem *addr); +extern u64 ioread64_hi_lo(void __iomem *addr); +extern u64 ioread64be_lo_hi(void __iomem *addr); +extern u64 ioread64be_hi_lo(void __iomem *addr); +#endif + extern void iowrite8(u8, void __iomem *); extern void iowrite16(u16, void __iomem *); extern void iowrite16be(u16, void __iomem *); @@ -46,6 +57,17 @@ extern void iowrite64(u64, void __iomem *); extern void iowrite64be(u64, void __iomem *); #endif +#ifdef writeq +#define iowrite64_lo_hi iowrite64_lo_hi +#define iowrite64_hi_lo iowrite64_hi_lo +#define iowrite64be_lo_hi iowrite64be_lo_hi +#define iowrite64be_hi_lo iowrite64be_hi_lo +extern void iowrite64_lo_hi(u64 val, void __iomem *addr); +extern void iowrite64_hi_lo(u64 val, void __iomem *addr); +extern void iowrite64be_lo_hi(u64 val, void __iomem *addr); +extern void iowrite64be_hi_lo(u64 val, void __iomem *addr); +#endif + /* * "string" versions of the above. Note that they * use native byte ordering for the accesses (on diff --git a/lib/iomap.c b/lib/iomap.c index 2c293b22569f..e909ab71e995 100644 --- a/lib/iomap.c +++ b/lib/iomap.c @@ -67,6 +67,7 @@ static void bad_io_access(unsigned long port, const char *access) #ifndef mmio_read16be #define mmio_read16be(addr) swab16(readw(addr)) #define mmio_read32be(addr) swab32(readl(addr)) +#define mmio_read64be(addr) swab64(readq(addr)) #endif unsigned int ioread8(void __iomem *addr) @@ -100,6 +101,80 @@ EXPORT_SYMBOL(ioread16be); EXPORT_SYMBOL(ioread32); EXPORT_SYMBOL(ioread32be); +#ifdef readq +static u64 pio_read64_lo_hi(unsigned long port) +{ + u64 lo, hi; + + lo = inl(port); + hi = inl(port + sizeof(u32)); + + return lo | (hi << 32); +} + +static u64 pio_read64_hi_lo(unsigned long port) +{ + u64 lo, hi; + + hi = inl(port + sizeof(u32)); + lo = inl(port); + + return lo | (hi << 32); +} + +static u64 pio_read64be_lo_hi(unsigned long port) +{ + u64 lo, hi; + + lo = pio_read32be(port + sizeof(u32)); + hi = pio_read32be(port); + + return lo | (hi << 32); +} + +static u64 pio_read64be_hi_lo(unsigned long port) +{ + u64 lo, hi; + + hi = pio_read32be(port); + lo = pio_read32be(port + sizeof(u32)); + + return lo | (hi << 32); +} + +u64 ioread64_lo_hi(void __iomem *addr) +{ + IO_COND(addr, return pio_read64_lo_hi(port), return readq(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64_hi_lo(void __iomem *addr) +{ + IO_COND(addr, return pio_read64_hi_lo(port), return readq(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64be_lo_hi(void __iomem *addr) +{ + IO_COND(addr, return pio_read64be_lo_hi(port), + return mmio_read64be(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64be_hi_lo(void __iomem *addr) +{ + IO_COND(addr, return pio_read64be_hi_lo(port), + return mmio_read64be(addr)); + return 0xffffffffffffffffULL; +} + +EXPORT_SYMBOL(ioread64_lo_hi); +EXPORT_SYMBOL(ioread64_hi_lo); +EXPORT_SYMBOL(ioread64be_lo_hi); +EXPORT_SYMBOL(ioread64be_hi_lo); + +#endif /* readq */ + #ifndef pio_write16be #define pio_write16be(val,port) outw(swab16(val),port) #define pio_write32be(val,port) outl(swab32(val),port) @@ -108,6 +183,7 @@ EXPORT_SYMBOL(ioread32be); #ifndef mmio_write16be #define mmio_write16be(val,port) writew(swab16(val),port) #define mmio_write32be(val,port) writel(swab32(val),port) +#define mmio_write64be(val,port) writeq(swab64(val),port) #endif void iowrite8(u8 val, void __iomem *addr) @@ -136,6 +212,62 @@ EXPORT_SYMBOL(iowrite16be); EXPORT_SYMBOL(iowrite32); EXPORT_SYMBOL(iowrite32be); +#ifdef writeq +static void pio_write64_lo_hi(u64 val, unsigned long port) +{ + outl(val, port); + outl(val >> 32, port + sizeof(u32)); +} + +static void pio_write64_hi_lo(u64 val, unsigned long port) +{ + outl(val >> 32, port + sizeof(u32)); + outl(val, port); +} + +static void pio_write64be_lo_hi(u64 val, unsigned long port) +{ + pio_write32be(val, port + sizeof(u32)); + pio_write32be(val >> 32, port); +} + +static void pio_write64be_hi_lo(u64 val, unsigned long port) +{ + pio_write32be(val >> 32, port); + pio_write32be(val, port + sizeof(u32)); +} + +void iowrite64_lo_hi(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64_lo_hi(val, port), + writeq(val, addr)); +} + +void iowrite64_hi_lo(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64_hi_lo(val, port), + writeq(val, addr)); +} + +void iowrite64be_lo_hi(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64be_lo_hi(val, port), + mmio_write64be(val, addr)); +} + +void iowrite64be_hi_lo(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64be_hi_lo(val, port), + mmio_write64be(val, addr)); +} + +EXPORT_SYMBOL(iowrite64_lo_hi); +EXPORT_SYMBOL(iowrite64_hi_lo); +EXPORT_SYMBOL(iowrite64be_lo_hi); +EXPORT_SYMBOL(iowrite64be_hi_lo); + +#endif /* readq */ + /* * These are the "repeat MMIO read/write" functions. * Note the "__raw" accesses, since we don't want to