From patchwork Wed Jan 16 18:25:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 10766683 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8A12313BF for ; Wed, 16 Jan 2019 18:25:50 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7ADF12F3E1 for ; Wed, 16 Jan 2019 18:25:50 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6E83E2F3F4; Wed, 16 Jan 2019 18:25:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D4D622F3E1 for ; Wed, 16 Jan 2019 18:25:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B77C38E0011; Wed, 16 Jan 2019 13:25:37 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id A16E38E0004; Wed, 16 Jan 2019 13:25:37 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7EC738E0013; Wed, 16 Jan 2019 13:25:37 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-io1-f70.google.com (mail-io1-f70.google.com [209.85.166.70]) by kanga.kvack.org (Postfix) with ESMTP id 43FD08E0011 for ; Wed, 16 Jan 2019 13:25:37 -0500 (EST) Received: by mail-io1-f70.google.com with SMTP id c4so5341373ioh.16 for ; Wed, 16 Jan 2019 10:25:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :date:message-id:in-reply-to:references:mime-version :content-transfer-encoding:subject; bh=REEhkDTxDlUiod/mFrWUzBoDhFBOMry6cHe+vAASCag=; b=dY60tSu8t44al442IjwvLJj82jW4qAU1gwYX4gHrePPFOMwRJwHnI07sOqvoRnJOEG qRcmN83dpiCzRW3GbitwAJ8ITUslbLzL1XHJFf1V+qu37jTlw89gt9az7gCAaXMqO1XT xIiNughTge2DBHHZLc8XM4u3oJ8l4cLuBZld/YREX+NCnQTJH9mK5OIMKxHfKrNLidDD DTjEst9w/k4coGhsPMU7O/ZWjJpRcVTjDCcU0r4fJ9uKB1M578acPqUh8i8rRp3ojtKf CXJeqMxFmymzMeTCciTkHE9SM42jOxsT2xqmtP+oKgzGCK43smWO5/CDsMxCIsEyo7rf weTQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com X-Gm-Message-State: AJcUukdP9yjVm25Ou8lkhsQWBxTfpC3mVy3YhbhUD6KKYrOQPYBCUIRQ As7pq+UJUcST6P0ZuJifKS6e+bJjpeyHyezO/C4RVSiM57Jd515U/CsGQQKWJdh8XxsrR2tsztO jYVyaDn1YMfmKz4YB7vNI3mTpgQsnF0xX60cw77D2UgLIkmy4LmVuQXrOATEPyYpuWQ== X-Received: by 2002:a02:8785:: with SMTP id t5mr6144045jai.73.1547663136964; Wed, 16 Jan 2019 10:25:36 -0800 (PST) X-Google-Smtp-Source: ALg8bN5bpPbSeTEnKMkgjHHLUM8Tqj59IoZmAANXxcfCjssua8Tr8YelTSopON0ZCPXXpcLoj1Db X-Received: by 2002:a02:8785:: with SMTP id t5mr6144009jai.73.1547663136186; Wed, 16 Jan 2019 10:25:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547663136; cv=none; d=google.com; s=arc-20160816; b=FBZtGOVLK7zmL+WHzHHNc0sNbXg7/D8hoOcjwBcnDHNP3e2epJhvtm21sKTR9suShy Z7Vi3PkA8UfTELoyJMybGpduOxCUKtwBu1AhNfCISvy1e5bZQZS5T3zUPyJA9q6BVEsa LchtK4zHPNxdRVppil97FBKr6ozqINYX9/Q/dXdHPN7X3TMqRCyKAXSsu21fz46M6K42 5oA9ol1VO47wj+X5Yiy3Xt93YvAYdvEU6J7cpsummAYXteQBgQewEcKpHSaYqPNOyJIV 01yJaKYij5BxKscem3erUTLCWFxUNmSVDr0i1O8jGtFG5fo2Y230CYNTeX10xRGYepbT ZVaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=subject:content-transfer-encoding:mime-version:references :in-reply-to:message-id:date:cc:to:from; bh=REEhkDTxDlUiod/mFrWUzBoDhFBOMry6cHe+vAASCag=; b=rrkMVK8a0XjTF6ltUN9DemFmOgVyySNxeM67Iypsp7qBASC/+RmfZVoZKatYZogcrt fSXQ0mXLYoeg3+e+A+kj/nrrXS7fe2qkrpFDUXATPkfaUADqHiOExhAi5UgrZVJKcR9R yUTX7/aLYhmGeUhfklFj2iuvMA+0s1h/G8Ak5h5vazSpbueqhdMC38feI55MZTuxGcWX 9UNpOccyLpy8L0bc9U9k7OMUgQgQHyJ3YqyIVWU90lFAY5PyS9M1eyH2CixWlNzboug3 hrZrWnwlyLAGhD+rHosV+v65RRZGwXJynwOkJlyPgbQF5O7z0yT6bPpsKsCe8DZUYUPJ Ts2g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com Received: from ale.deltatee.com (ale.deltatee.com. [207.54.116.67]) by mx.google.com with ESMTPS id 130si4091127jaz.43.2019.01.16.10.25.35 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 16 Jan 2019 10:25:36 -0800 (PST) Received-SPF: pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) client-ip=207.54.116.67; Authentication-Results: mx.google.com; spf=pass (google.com: domain of gunthorp@deltatee.com designates 207.54.116.67 as permitted sender) smtp.mailfrom=gunthorp@deltatee.com Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1gjpsY-0003ND-0K; Wed, 16 Jan 2019 11:25:34 -0700 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.89) (envelope-from ) id 1gjpsU-00054a-37; Wed, 16 Jan 2019 11:25:26 -0700 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-ntb@googlegroups.com, linux-crypto@vger.kernel.org, linux-mm@kvack.org, Andrew Morton Cc: Arnd Bergmann , Greg Kroah-Hartman , Andy Shevchenko , =?utf-8?q?Horia_Geant=C4=83?= , Logan Gunthorpe , Benjamin Herrenschmidt , Paul Mackerras , Michael Ellerman , Suresh Warrier , Nicholas Piggin Date: Wed, 16 Jan 2019 11:25:20 -0700 Message-Id: <20190116182523.19446-4-logang@deltatee.com> X-Mailer: git-send-email 2.19.0 In-Reply-To: <20190116182523.19446-1-logang@deltatee.com> References: <20190116182523.19446-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-ntb@googlegroups.com, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-crypto@vger.kernel.org, linux-mm@kvack.org, akpm@linux-foundation.org, arnd@arndb.de, gregkh@linuxfoundation.org, horia.geanta@nxp.com, logang@deltatee.com, benh@kernel.crashing.org, paulus@samba.org, mpe@ellerman.id.au, warrier@linux.vnet.ibm.com, andy.shevchenko@gmail.com, npiggin@gmail.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v25 3/6] iomap: introduce io{read|write}64_{lo_hi|hi_lo} X-SA-Exim-Version: 4.2.1 (built Tue, 02 Aug 2016 21:08:31 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP In order to provide non-atomic functions for io{read|write}64 that will use readq and writeq when appropriate. We define a number of variants of these functions in the generic iomap that will do non-atomic operations on pio but atomic operations on mmio. These functions are only defined if readq and writeq are defined. If they are not, then the wrappers that always use non-atomic operations from include/linux/io-64-nonatomic*.h will be used. Signed-off-by: Logan Gunthorpe Reviewed-by: Andy Shevchenko Cc: Benjamin Herrenschmidt Cc: Paul Mackerras Cc: Michael Ellerman Cc: Arnd Bergmann Cc: Suresh Warrier Cc: Nicholas Piggin --- arch/powerpc/include/asm/io.h | 2 + include/asm-generic/iomap.h | 22 ++++++ lib/iomap.c | 132 ++++++++++++++++++++++++++++++++++ 3 files changed, 156 insertions(+) diff --git a/arch/powerpc/include/asm/io.h b/arch/powerpc/include/asm/io.h index 7f19fbd3ba55..4b73847e9b95 100644 --- a/arch/powerpc/include/asm/io.h +++ b/arch/powerpc/include/asm/io.h @@ -783,8 +783,10 @@ extern void __iounmap_at(void *ea, unsigned long size); #define mmio_read16be(addr) readw_be(addr) #define mmio_read32be(addr) readl_be(addr) +#define mmio_read64be(addr) readq_be(addr) #define mmio_write16be(val, addr) writew_be(val, addr) #define mmio_write32be(val, addr) writel_be(val, addr) +#define mmio_write64be(val, addr) writeq_be(val, addr) #define mmio_insb(addr, dst, count) readsb(addr, dst, count) #define mmio_insw(addr, dst, count) readsw(addr, dst, count) #define mmio_insl(addr, dst, count) readsl(addr, dst, count) diff --git a/include/asm-generic/iomap.h b/include/asm-generic/iomap.h index 5b63b94ef6b5..a008f504a2d0 100644 --- a/include/asm-generic/iomap.h +++ b/include/asm-generic/iomap.h @@ -36,6 +36,17 @@ extern u64 ioread64(void __iomem *); extern u64 ioread64be(void __iomem *); #endif +#ifdef readq +#define ioread64_lo_hi ioread64_lo_hi +#define ioread64_hi_lo ioread64_hi_lo +#define ioread64be_lo_hi ioread64be_lo_hi +#define ioread64be_hi_lo ioread64be_hi_lo +extern u64 ioread64_lo_hi(void __iomem *addr); +extern u64 ioread64_hi_lo(void __iomem *addr); +extern u64 ioread64be_lo_hi(void __iomem *addr); +extern u64 ioread64be_hi_lo(void __iomem *addr); +#endif + extern void iowrite8(u8, void __iomem *); extern void iowrite16(u16, void __iomem *); extern void iowrite16be(u16, void __iomem *); @@ -46,6 +57,17 @@ extern void iowrite64(u64, void __iomem *); extern void iowrite64be(u64, void __iomem *); #endif +#ifdef writeq +#define iowrite64_lo_hi iowrite64_lo_hi +#define iowrite64_hi_lo iowrite64_hi_lo +#define iowrite64be_lo_hi iowrite64be_lo_hi +#define iowrite64be_hi_lo iowrite64be_hi_lo +extern void iowrite64_lo_hi(u64 val, void __iomem *addr); +extern void iowrite64_hi_lo(u64 val, void __iomem *addr); +extern void iowrite64be_lo_hi(u64 val, void __iomem *addr); +extern void iowrite64be_hi_lo(u64 val, void __iomem *addr); +#endif + /* * "string" versions of the above. Note that they * use native byte ordering for the accesses (on diff --git a/lib/iomap.c b/lib/iomap.c index 2c293b22569f..e909ab71e995 100644 --- a/lib/iomap.c +++ b/lib/iomap.c @@ -67,6 +67,7 @@ static void bad_io_access(unsigned long port, const char *access) #ifndef mmio_read16be #define mmio_read16be(addr) swab16(readw(addr)) #define mmio_read32be(addr) swab32(readl(addr)) +#define mmio_read64be(addr) swab64(readq(addr)) #endif unsigned int ioread8(void __iomem *addr) @@ -100,6 +101,80 @@ EXPORT_SYMBOL(ioread16be); EXPORT_SYMBOL(ioread32); EXPORT_SYMBOL(ioread32be); +#ifdef readq +static u64 pio_read64_lo_hi(unsigned long port) +{ + u64 lo, hi; + + lo = inl(port); + hi = inl(port + sizeof(u32)); + + return lo | (hi << 32); +} + +static u64 pio_read64_hi_lo(unsigned long port) +{ + u64 lo, hi; + + hi = inl(port + sizeof(u32)); + lo = inl(port); + + return lo | (hi << 32); +} + +static u64 pio_read64be_lo_hi(unsigned long port) +{ + u64 lo, hi; + + lo = pio_read32be(port + sizeof(u32)); + hi = pio_read32be(port); + + return lo | (hi << 32); +} + +static u64 pio_read64be_hi_lo(unsigned long port) +{ + u64 lo, hi; + + hi = pio_read32be(port); + lo = pio_read32be(port + sizeof(u32)); + + return lo | (hi << 32); +} + +u64 ioread64_lo_hi(void __iomem *addr) +{ + IO_COND(addr, return pio_read64_lo_hi(port), return readq(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64_hi_lo(void __iomem *addr) +{ + IO_COND(addr, return pio_read64_hi_lo(port), return readq(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64be_lo_hi(void __iomem *addr) +{ + IO_COND(addr, return pio_read64be_lo_hi(port), + return mmio_read64be(addr)); + return 0xffffffffffffffffULL; +} + +u64 ioread64be_hi_lo(void __iomem *addr) +{ + IO_COND(addr, return pio_read64be_hi_lo(port), + return mmio_read64be(addr)); + return 0xffffffffffffffffULL; +} + +EXPORT_SYMBOL(ioread64_lo_hi); +EXPORT_SYMBOL(ioread64_hi_lo); +EXPORT_SYMBOL(ioread64be_lo_hi); +EXPORT_SYMBOL(ioread64be_hi_lo); + +#endif /* readq */ + #ifndef pio_write16be #define pio_write16be(val,port) outw(swab16(val),port) #define pio_write32be(val,port) outl(swab32(val),port) @@ -108,6 +183,7 @@ EXPORT_SYMBOL(ioread32be); #ifndef mmio_write16be #define mmio_write16be(val,port) writew(swab16(val),port) #define mmio_write32be(val,port) writel(swab32(val),port) +#define mmio_write64be(val,port) writeq(swab64(val),port) #endif void iowrite8(u8 val, void __iomem *addr) @@ -136,6 +212,62 @@ EXPORT_SYMBOL(iowrite16be); EXPORT_SYMBOL(iowrite32); EXPORT_SYMBOL(iowrite32be); +#ifdef writeq +static void pio_write64_lo_hi(u64 val, unsigned long port) +{ + outl(val, port); + outl(val >> 32, port + sizeof(u32)); +} + +static void pio_write64_hi_lo(u64 val, unsigned long port) +{ + outl(val >> 32, port + sizeof(u32)); + outl(val, port); +} + +static void pio_write64be_lo_hi(u64 val, unsigned long port) +{ + pio_write32be(val, port + sizeof(u32)); + pio_write32be(val >> 32, port); +} + +static void pio_write64be_hi_lo(u64 val, unsigned long port) +{ + pio_write32be(val >> 32, port); + pio_write32be(val, port + sizeof(u32)); +} + +void iowrite64_lo_hi(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64_lo_hi(val, port), + writeq(val, addr)); +} + +void iowrite64_hi_lo(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64_hi_lo(val, port), + writeq(val, addr)); +} + +void iowrite64be_lo_hi(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64be_lo_hi(val, port), + mmio_write64be(val, addr)); +} + +void iowrite64be_hi_lo(u64 val, void __iomem *addr) +{ + IO_COND(addr, pio_write64be_hi_lo(val, port), + mmio_write64be(val, addr)); +} + +EXPORT_SYMBOL(iowrite64_lo_hi); +EXPORT_SYMBOL(iowrite64_hi_lo); +EXPORT_SYMBOL(iowrite64be_lo_hi); +EXPORT_SYMBOL(iowrite64be_hi_lo); + +#endif /* readq */ + /* * These are the "repeat MMIO read/write" functions. * Note the "__raw" accesses, since we don't want to