From patchwork Thu Mar 2 08:53:32 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Edgar E. Iglesias" X-Patchwork-Id: 9599723 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 97D3C60453 for ; Thu, 2 Mar 2017 08:55:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7698327F89 for ; Thu, 2 Mar 2017 08:55:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6B79C28587; Thu, 2 Mar 2017 08:55:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, RCVD_IN_SORBS_SPAM, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 65BC627F89 for ; Thu, 2 Mar 2017 08:55:41 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjMUX-0003Gg-4M; Thu, 02 Mar 2017 08:53:41 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1cjMUV-0003GW-Pe for xen-devel@lists.xenproject.org; Thu, 02 Mar 2017 08:53:39 +0000 Received: from [193.109.254.147] by server-7.bemta-6.messagelabs.com id ED/6B-04817-39DD7B85; Thu, 02 Mar 2017 08:53:39 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrBIsWRWlGSWpSXmKPExsVyMfS6o27/3e0 RBovXyVp83zKZyYHR4/CHKywBjFGsmXlJ+RUJrBm/f/azFfTJVdz+3MrewPhHrIuRi0NIYDqj xIa2bSwgDovAfFaJ6T0PmUAcCYHdrBITP19k62LkBHJiJE6tbIeySyU2fpzKDmILCahLnH63g w1i1DQmiYtdLSwgCRYBFYnv+9eD2WwCJhJ7/zxgArFFBIwljh7cDmYzC6RJPFt+gxXEFhbwkn i2cxkjiM0roCFx7f0xFogFWRI7T29lg4gLSpyc+YQFoldL4sa/l0BzOIBsaYnl/zhAwpxA4bv dL9hAwqJAJ7w6WD+BUXgWkuZZSJpnITQvYGRexahRnFpUllqka2iol1SUmZ5RkpuYmaNraGCm l5taXJyYnpqTmFSsl5yfu4kRGOYMQLCD8dOygEOMkhxMSqK8h69vjxDiS8pPqcxILM6ILyrNS S0+xCjDwaEkwXvjNlBOsCg1PbUiLTMHGHEwaQkOHiURXs87QGne4oLE3OLMdIjUKUZ7jgendr 1h4pgzezeQvNWwB0h+6j/8hkmIJS8/L1VKnFcKpE0ApC2jNA9uKCxBXGKUlRLmZQQ6U4inILU oN7MEVf4VozgHo5IwrwjIFJ7MvBK43a+AzmICOuuFylaQs0oSEVJSDYyL2Gaax9ic+X3MJ/FF RiXvnDO+PorpVUcc+a0ePmHZENl/8eM140+rxbZskNzS9+s3xxpFqaSMvRpnflVxCu6SzC08f d1/5VTZ588q3LoPFr9nmzRPu8/8Zudr5hfnvycweceVzpoSlW0684Elw3PtOWYrdW4u+qS61l dT8GnHe5ednAIFzy4osRRnJBpqMRcVJwIAu0yE1wsDAAA= X-Env-Sender: edgar.iglesias@gmail.com X-Msg-Ref: server-3.tower-27.messagelabs.com!1488444814!89331423!1 X-Originating-IP: [209.85.215.65] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 9.4.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 59165 invoked from network); 2 Mar 2017 08:53:35 -0000 Received: from mail-lf0-f65.google.com (HELO mail-lf0-f65.google.com) (209.85.215.65) by server-3.tower-27.messagelabs.com with AES128-GCM-SHA256 encrypted SMTP; 2 Mar 2017 08:53:35 -0000 Received: by mail-lf0-f65.google.com with SMTP id g70so4157974lfh.3 for ; Thu, 02 Mar 2017 00:53:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=nKyau0MexAtxxw6PmQCQ4t7qY0hvLUNgIK5WMqWHvqw=; b=l0ycPrLur/qjoJZVN96kFJJL2B+qSKAXueQVEIhirkb+ysRBxw3TTg3V+okJ/6Fz8c 5f6tXP1T5rYD0QYks/hykiV+1JP7kVtGsoiLqEQueTXdMfrptlQBJwzZH4hrxrBwFycX 4dbhryGMxQo4VHWuWRhUvwtCenUxE4f7DbPwqWZu+1acKosenA9actij6Ut4jqkNwpRL IV2CAJKDjAOHtP5M9M5TK3s0LNQ9uCuxHg1adDA3pEU/NlvT7SBIRIfXoHqXQbQgDKm/ HJU0d3AFBWu6mMClee/LKF2p6a8wSXgtPSO1VN4o5LY4vdU0ugvwUUUsBm9YrBuQ28e5 qlzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=nKyau0MexAtxxw6PmQCQ4t7qY0hvLUNgIK5WMqWHvqw=; b=RFx7tOLTwLBQKN7y7SdbIg7YY1JqrnTGlmfoaLOYnmz5oilhzS00NQcQTfb+LLkFlQ omgoS3IaF50NmI99/vqKmL+8QZYeYUfiJ//VDN/1jk7QcvvT6FIpW87jvLR6UXhWI1C1 ktFaHB2b0LUymduZJgfYi8CEpdGPL7rv1glQsxnSeQRSZyDU0YUj16FmOyVdCR1XliwB mRs40k83d2DmgBvk2F3tSLhvujWcD2f/t82RqntLV0178RIB/XerNxw5Jj6ZaRs/iYuL ZZQhMAKf3xbZBLoranC00Jxd1z17jO7oGQ25bASG7U4P8ZwWwneNQNS9D8CLAy+KHDu+ rBAA== X-Gm-Message-State: AMke39kbwPkAY2MrA1ckeRNceKtdmI9gwRAcreQxSHDalrlgpOjkJ0HHGtiEdEaEAfMlaQ== X-Received: by 10.46.15.9 with SMTP id 9mr4536712ljp.108.1488444814492; Thu, 02 Mar 2017 00:53:34 -0800 (PST) Received: from localhost (81-231-233-234-no56.tbcn.telia.com. [81.231.233.234]) by smtp.gmail.com with ESMTPSA id 1sm1609367ljv.40.2017.03.02.00.53.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Mar 2017 00:53:33 -0800 (PST) Date: Thu, 2 Mar 2017 09:53:32 +0100 From: "Edgar E. Iglesias" To: "Edgar E. Iglesias" Message-ID: <20170302085332.GU9606@toto> References: <20170302083837.GB23726@toto> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170302083837.GB23726@toto> User-Agent: Mutt/1.5.24 (2015-08-30) Cc: xen-devel@lists.xenproject.org, julien.grall@arm.com, Stefano Stabellini Subject: Re: [Xen-devel] xen/arm and swiotlb-xen: possible data corruption X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP On Thu, Mar 02, 2017 at 09:38:37AM +0100, Edgar E. Iglesias wrote: > On Wed, Mar 01, 2017 at 05:05:21PM -0800, Stefano Stabellini wrote: > > Hi all, > > > > Edgar reported a data corruption on network packets in dom0 when the > > swiotlb-xen is in use. He also reported that the following patch "fixes" > > the problem for him: > > > > static void __xen_dma_page_cpu_to_dev(struct device *hwdev, dma_addr_t handle, > > size_t size, enum dma_data_direction dir) > > { > > - dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, DMA_MAP); > > + printk("%s: addr=%lx size=%zd\n", __func__, handle, size); > > + dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size + 64, dir, DMA_MAP); > > > > I am thinking that the problem has something to do with cacheline > > alignment on the Xen side > > (xen/common/grant_table.c:__gnttab_cache_flush). > > > > If op == GNTTAB_CACHE_INVAL, we call invalidate_dcache_va_range; if op > > == GNTTAB_CACHE_CLEAN, we call clean_dcache_va_range instead. The > > parameter, v, could be non-cacheline aligned. > > > > invalidate_dcache_va_range is capable of handling a not aligned address, > > while clean_dcache_va_range does not. > > > > Edgar, does the appended patch fix the problem for you? > > > Thanks Stefano, > > This does indeed fix the issue for me. Hi again, Looking at the code, the problem here is that we may flush one cache line less than expected. This smaller patch fixes it for me too: Anyway, I'm OK with either fix. Cheers, Edgar > > Cheers, > Edgar > > > > > > --- > > > > diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h > > index 86de0b6..9cdf2fb 100644 > > --- a/xen/include/asm-arm/page.h > > +++ b/xen/include/asm-arm/page.h > > @@ -322,10 +322,30 @@ static inline int invalidate_dcache_va_range(const void *p, unsigned long size) > > > > static inline int clean_dcache_va_range(const void *p, unsigned long size) > > { > > - const void *end; > > + size_t off; > > + const void *end = p + size; > > + > > dsb(sy); /* So the CPU issues all writes to the range */ > > - for ( end = p + size; p < end; p += cacheline_bytes ) > > + > > + off = (unsigned long)p % cacheline_bytes; > > + if ( off ) > > + { > > + p -= off; > > asm volatile (__clean_dcache_one(0) : : "r" (p)); > > + p += cacheline_bytes; > > + size -= cacheline_bytes - off; > > + } > > + off = (unsigned long)end % cacheline_bytes; > > + if ( off ) > > + { > > + end -= off; > > + size -= off; > > + asm volatile (__clean_dcache_one(0) : : "r" (end)); > > + } > > + > > + for ( ; p < end; p += cacheline_bytes ) > > + asm volatile (__clean_dcache_one(0) : : "r" (p)); > > + > > dsb(sy); /* So we know the flushes happen before continuing */ > > /* ARM callers assume that dcache_* functions cannot fail. */ > > return 0; > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h index c492d6d..fa1b4dd 100644 --- a/xen/include/asm-arm/page.h +++ b/xen/include/asm-arm/page.h @@ -325,7 +325,9 @@ static inline int clean_dcache_va_range(const void *p, unsigned long size) { const void *end; dsb(sy); /* So the CPU issues all writes to the range */ - for ( end = p + size; p < end; p += cacheline_bytes ) + + end = (void *)ROUNDUP((uintptr_t)p + size, cacheline_bytes); + for ( ; p < end; p += cacheline_bytes ) asm volatile (__clean_dcache_one(0) : : "r" (p)); dsb(sy); /* So we know the flushes happen before continuing */ /* ARM callers assume that dcache_* functions cannot fail. */