[RFC,v2,00/11] riscv: Add DMA_COHERENT support for Allwinner D1

Message ID	1622970249-50770-1-git-send-email-guoren@kernel.org (mailing list archive)
Headers	show Return-Path: <SRS0=9QI1=LA=lists.infradead.org=linux-riscv-bounces+linux-riscv=archiver.kernel.org@kernel.org> DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 725CC6120F From: guoren@kernel.org To: guoren@kernel.org, anup.patel@wdc.com, palmerdabbelt@google.com, arnd@arndb.de, wens@csie.org, maxime@cerno.tech, drew@beagleboard.org, liush@allwinnertech.com, lazyparser@gmail.com, wefu@redhat.com Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-sunxi@lists.linux.dev, Guo Ren <guoren@linux.alibaba.com> Subject: [RFC PATCH v2 00/11] riscv: Add DMA_COHERENT support for Allwinner D1 Date: Sun, 6 Jun 2021 09:03:55 +0000 Message-Id: <1622970249-50770-1-git-send-email-guoren@kernel.org> Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" <linux-riscv-bounces@lists.infradead.org> Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org
Series	riscv: Add DMA_COHERENT support for Allwinner D1 \| expand [RFC,v2,00/11] riscv: Add DMA_COHERENT support for Allwinner D1 [V5,1/3] riscv: Use global mappings for kernel pages [V5,2/3] riscv: Add ASID-based tlbflushing methods [V5,3/3] riscv: tlbflush: Optimize coding convention [RFC,v2,04/11] riscv: pgtable: Fixup _PAGE_CHG_MASK usage [RFC,v2,05/11] riscv: pgtable: Add custom protection_map init [RFC,v2,06/11] riscv: pgtable: Add DMA_COHERENT with custom PTE attributes [RFC,v2,07/11] riscv: cmo: Add dma-noncoherency support [RFC,v2,08/11] riscv: cmo: Add vendor custom icache sync [RFC,v2,09/11] riscv: soc: Initial DTS for Allwinner D1 NeZha board [RFC,v2,10/11] riscv: soc: Add Allwinner SoC kconfig option [RFC,v2,11/11] riscv: soc: Allwinner D1 GMAC driver only for temp use

Guo Ren June 6, 2021, 9:03 a.m. UTC

From: Guo Ren <guoren@linux.alibaba.com>

The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
vendors define the custom properties of memory regions in PTE.

This patchset helps SOC vendors to support their own custom interconnect
coherent solution with PTE attributes.

For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906 has
two modes in MMU:
 - Compatible mode, the same as the definitions in spec.
 - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
   not mentioned in spec.

Allwinner D1 needs the enhanced mode to support the DMA type device with
non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
attribute bits in PTE.

The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been
tested on D1:
 - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
   conflict with RISC-V spec, and hope these patches soon could be
   approved.
 - pgtable: Using a image-hdr to pass vendor specific information and
   setup custom PTE attributes in a global struct variable during boot
   stage. Also it needs define custom protection_map in linux/mm.
 - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
   In this patchset, I just show you how T-HEAD C9xx work, and seems Atish
   is working for the DMA infrustructure, please let me know the idea.
 - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.

The patchset could work with linux-5.13-rc4, here is the steps for D1:
 - Download linux-5.13-rc4 and apply the patchset
 - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
 - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
 - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e 0x00200000 -n Linux -d arch/riscv/boot/Image uImage
 - Download newest opensbi [2], build with [3], and get fw_dynamic.bin
 - Copy uImage, fw_dynamic.bin, allwinner-d1-nezha-kit.dtb into boot
   partition of TF card.
 - Plugin the TF card and power on D1.

Link: https://linux-sunxi.org/D1 [1]
Link: https://github.com/riscv/opensbi branch:master [2]
Link: https://github.com/riscv/opensbi/blob/master/docs/platform/thead-c9xx.md [3]

Changes since v1:
 - Rebase on linux-5.13-rc4
 - Support defconfig for different PTE attributes
 - Support C906 icache_sync
 - Add Allwinner D1 dts & Kconfig & gmac for testing
 - Add asid optimization for D1 usage

Guo Ren (10):
  riscv: asid: Use global mappings for kernel pages
  riscv: asid: Add ASID-based tlbflushing methods
  riscv: asid: Optimize tlbflush coding convention
  riscv: pgtable: Fixup _PAGE_CHG_MASK usage
  riscv: pgtable: Add custom protection_map init
  riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
  riscv: cmo: Add dma-noncoherency support
  riscv: cmo: Add vendor custom icache sync
  riscv: soc: Initial DTS for Allwinner D1 NeZha board
  riscv: soc: Add Allwinner SoC kconfig option

liush (1):
  riscv: soc: Allwinner D1 GMAC driver only for temp use

 arch/riscv/Kconfig                                 |    9 +
 arch/riscv/Kconfig.socs                            |   12 +
 arch/riscv/boot/dts/Makefile                       |    1 +
 arch/riscv/boot/dts/allwinner/Makefile             |    2 +
 .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
 arch/riscv/configs/defconfig                       |    1 +
 arch/riscv/include/asm/cacheflush.h                |   48 +-
 arch/riscv/include/asm/mmu_context.h               |    2 +
 arch/riscv/include/asm/pgtable-64.h                |    8 +-
 arch/riscv/include/asm/pgtable-bits.h              |   20 +-
 arch/riscv/include/asm/pgtable.h                   |   44 +-
 arch/riscv/include/asm/sbi.h                       |   15 +
 arch/riscv/include/asm/soc.h                       |    1 +
 arch/riscv/include/asm/tlbflush.h                  |   22 +
 arch/riscv/include/asm/vendorid_list.h             |    1 +
 arch/riscv/kernel/sbi.c                            |   19 +
 arch/riscv/kernel/soc.c                            |   22 +
 arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
 arch/riscv/mm/Makefile                             |    1 +
 arch/riscv/mm/cacheflush.c                         |    3 +-
 arch/riscv/mm/context.c                            |    2 +-
 arch/riscv/mm/dma-mapping.c                        |   53 +
 arch/riscv/mm/init.c                               |   26 +
 arch/riscv/mm/tlbflush.c                           |   57 +-
 drivers/net/ethernet/Kconfig                       |    1 +
 drivers/net/ethernet/Makefile                      |    1 +
 drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
 drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240 ++++++++++++++++++++
 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h     |  258 +++
 drivers/net/phy/realtek.c                          |    2 +-
 mm/mmap.c                                          |    4 +
 34 files changed, 3714 insertions(+), 37 deletions(-)
 create mode 100644 arch/riscv/boot/dts/allwinner/Makefile
 create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-kit.dts
 create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi
 create mode 100644 arch/riscv/mm/dma-mapping.c
 create mode 100644 drivers/net/ethernet/allwinnertmp/Kconfig
 create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
 create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
 create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
 create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h

Jernej Škrabec June 6, 2021, 4:29 p.m. UTC | #1

Hi!

Dne nedelja, 06. junij 2021 ob 11:03:55 CEST je guoren@kernel.org napisal(a):
> From: Guo Ren <guoren@linux.alibaba.com>
> 
> The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> vendors define the custom properties of memory regions in PTE.
> 
> This patchset helps SOC vendors to support their own custom interconnect
> coherent solution with PTE attributes.
> 
> For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906 has
> two modes in MMU:
>  - Compatible mode, the same as the definitions in spec.
>  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
>    not mentioned in spec.
> 
> Allwinner D1 needs the enhanced mode to support the DMA type device with
> non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> attribute bits in PTE.
> 
> The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been
> tested on D1:
>  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
>    conflict with RISC-V spec, and hope these patches soon could be
>    approved.
>  - pgtable: Using a image-hdr to pass vendor specific information and
>    setup custom PTE attributes in a global struct variable during boot
>    stage. Also it needs define custom protection_map in linux/mm.
>  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
>    In this patchset, I just show you how T-HEAD C9xx work, and seems Atish
>    is working for the DMA infrustructure, please let me know the idea.
>  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> 
> The patchset could work with linux-5.13-rc4, here is the steps for D1:
>  - Download linux-5.13-rc4 and apply the patchset
>  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
>  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
>  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e 0x00200000
> -n Linux -d arch/riscv/boot/Image uImage - Download newest opensbi [2],
> build with [3], and get fw_dynamic.bin - Copy uImage, fw_dynamic.bin,
> allwinner-d1-nezha-kit.dtb into boot partition of TF card.
>  - Plugin the TF card and power on D1.
> 
> Link: https://linux-sunxi.org/D1 [1]
> Link: https://github.com/riscv/opensbi branch:master [2]
> Link:
> https://github.com/riscv/opensbi/blob/master/docs/platform/thead-c9xx.md
> [3]
> 

Some patches are marked with v2 and some V5. It's very confusing. Mark them 
with same version in next revision.

Best regards,
Jernej

> Changes since v1:
>  - Rebase on linux-5.13-rc4
>  - Support defconfig for different PTE attributes
>  - Support C906 icache_sync
>  - Add Allwinner D1 dts & Kconfig & gmac for testing
>  - Add asid optimization for D1 usage
> 
> Guo Ren (10):
>   riscv: asid: Use global mappings for kernel pages
>   riscv: asid: Add ASID-based tlbflushing methods
>   riscv: asid: Optimize tlbflush coding convention
>   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
>   riscv: pgtable: Add custom protection_map init
>   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
>   riscv: cmo: Add dma-noncoherency support
>   riscv: cmo: Add vendor custom icache sync
>   riscv: soc: Initial DTS for Allwinner D1 NeZha board
>   riscv: soc: Add Allwinner SoC kconfig option
> 
> liush (1):
>   riscv: soc: Allwinner D1 GMAC driver only for temp use
> 
>  arch/riscv/Kconfig                                 |    9 +
>  arch/riscv/Kconfig.socs                            |   12 +
>  arch/riscv/boot/dts/Makefile                       |    1 +
>  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
>  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
>  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
>  arch/riscv/configs/defconfig                       |    1 +
>  arch/riscv/include/asm/cacheflush.h                |   48 +-
>  arch/riscv/include/asm/mmu_context.h               |    2 +
>  arch/riscv/include/asm/pgtable-64.h                |    8 +-
>  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
>  arch/riscv/include/asm/pgtable.h                   |   44 +-
>  arch/riscv/include/asm/sbi.h                       |   15 +
>  arch/riscv/include/asm/soc.h                       |    1 +
>  arch/riscv/include/asm/tlbflush.h                  |   22 +
>  arch/riscv/include/asm/vendorid_list.h             |    1 +
>  arch/riscv/kernel/sbi.c                            |   19 +
>  arch/riscv/kernel/soc.c                            |   22 +
>  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
>  arch/riscv/mm/Makefile                             |    1 +
>  arch/riscv/mm/cacheflush.c                         |    3 +-
>  arch/riscv/mm/context.c                            |    2 +-
>  arch/riscv/mm/dma-mapping.c                        |   53 +
>  arch/riscv/mm/init.c                               |   26 +
>  arch/riscv/mm/tlbflush.c                           |   57 +-
>  drivers/net/ethernet/Kconfig                       |    1 +
>  drivers/net/ethernet/Makefile                      |    1 +
>  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
>  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
>  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
>  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> ++++++++++++++++++++ drivers/net/ethernet/allwinnertmp/sunxi-gmac.h     | 
> 258 +++
>  drivers/net/phy/realtek.c                          |    2 +-
>  mm/mmap.c                                          |    4 +
>  34 files changed, 3714 insertions(+), 37 deletions(-)
>  create mode 100644 arch/riscv/boot/dts/allwinner/Makefile
>  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-kit.dts
> create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi create
> mode 100644 arch/riscv/mm/dma-mapping.c
>  create mode 100644 drivers/net/ethernet/allwinnertmp/Kconfig
>  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h

Guo Ren June 6, 2021, 4:54 p.m. UTC | #2

V5 is not related to the patch series.

On Mon, Jun 7, 2021 at 12:29 AM Jernej Škrabec <jernej.skrabec@gmail.com> wrote:
>
> Hi!
>
> Dne nedelja, 06. junij 2021 ob 11:03:55 CEST je guoren@kernel.org napisal(a):
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> > vendors define the custom properties of memory regions in PTE.
> >
> > This patchset helps SOC vendors to support their own custom interconnect
> > coherent solution with PTE attributes.
> >
> > For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906 has
> > two modes in MMU:
> >  - Compatible mode, the same as the definitions in spec.
> >  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
> >    not mentioned in spec.
> >
> > Allwinner D1 needs the enhanced mode to support the DMA type device with
> > non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> > attribute bits in PTE.
> >
> > The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been
> > tested on D1:
> >  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
> >    conflict with RISC-V spec, and hope these patches soon could be
> >    approved.
> >  - pgtable: Using a image-hdr to pass vendor specific information and
> >    setup custom PTE attributes in a global struct variable during boot
> >    stage. Also it needs define custom protection_map in linux/mm.
> >  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
> >    In this patchset, I just show you how T-HEAD C9xx work, and seems Atish
> >    is working for the DMA infrustructure, please let me know the idea.
> >  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> >
> > The patchset could work with linux-5.13-rc4, here is the steps for D1:
> >  - Download linux-5.13-rc4 and apply the patchset
> >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
> >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
> >  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e 0x00200000
> > -n Linux -d arch/riscv/boot/Image uImage - Download newest opensbi [2],
> > build with [3], and get fw_dynamic.bin - Copy uImage, fw_dynamic.bin,
> > allwinner-d1-nezha-kit.dtb into boot partition of TF card.
> >  - Plugin the TF card and power on D1.
> >
> > Link: https://linux-sunxi.org/D1 [1]
> > Link: https://github.com/riscv/opensbi branch:master [2]
> > Link:
> > https://github.com/riscv/opensbi/blob/master/docs/platform/thead-c9xx.md
> > [3]
> >
>
> Some patches are marked with v2 and some V5. It's very confusing. Mark them
> with same version in next revision.
>
> Best regards,
> Jernej
>
> > Changes since v1:
> >  - Rebase on linux-5.13-rc4
> >  - Support defconfig for different PTE attributes
> >  - Support C906 icache_sync
> >  - Add Allwinner D1 dts & Kconfig & gmac for testing
> >  - Add asid optimization for D1 usage
> >
> > Guo Ren (10):
> >   riscv: asid: Use global mappings for kernel pages
> >   riscv: asid: Add ASID-based tlbflushing methods
> >   riscv: asid: Optimize tlbflush coding convention
> >   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
> >   riscv: pgtable: Add custom protection_map init
> >   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
> >   riscv: cmo: Add dma-noncoherency support
> >   riscv: cmo: Add vendor custom icache sync
> >   riscv: soc: Initial DTS for Allwinner D1 NeZha board
> >   riscv: soc: Add Allwinner SoC kconfig option
> >
> > liush (1):
> >   riscv: soc: Allwinner D1 GMAC driver only for temp use
> >
> >  arch/riscv/Kconfig                                 |    9 +
> >  arch/riscv/Kconfig.socs                            |   12 +
> >  arch/riscv/boot/dts/Makefile                       |    1 +
> >  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
> >  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
> >  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
> >  arch/riscv/configs/defconfig                       |    1 +
> >  arch/riscv/include/asm/cacheflush.h                |   48 +-
> >  arch/riscv/include/asm/mmu_context.h               |    2 +
> >  arch/riscv/include/asm/pgtable-64.h                |    8 +-
> >  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
> >  arch/riscv/include/asm/pgtable.h                   |   44 +-
> >  arch/riscv/include/asm/sbi.h                       |   15 +
> >  arch/riscv/include/asm/soc.h                       |    1 +
> >  arch/riscv/include/asm/tlbflush.h                  |   22 +
> >  arch/riscv/include/asm/vendorid_list.h             |    1 +
> >  arch/riscv/kernel/sbi.c                            |   19 +
> >  arch/riscv/kernel/soc.c                            |   22 +
> >  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
> >  arch/riscv/mm/Makefile                             |    1 +
> >  arch/riscv/mm/cacheflush.c                         |    3 +-
> >  arch/riscv/mm/context.c                            |    2 +-
> >  arch/riscv/mm/dma-mapping.c                        |   53 +
> >  arch/riscv/mm/init.c                               |   26 +
> >  arch/riscv/mm/tlbflush.c                           |   57 +-
> >  drivers/net/ethernet/Kconfig                       |    1 +
> >  drivers/net/ethernet/Makefile                      |    1 +
> >  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
> >  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
> >  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
> >  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> > ++++++++++++++++++++ drivers/net/ethernet/allwinnertmp/sunxi-gmac.h     |
> > 258 +++
> >  drivers/net/phy/realtek.c                          |    2 +-
> >  mm/mmap.c                                          |    4 +
> >  34 files changed, 3714 insertions(+), 37 deletions(-)
> >  create mode 100644 arch/riscv/boot/dts/allwinner/Makefile
> >  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-kit.dts
> > create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi create
> > mode 100644 arch/riscv/mm/dma-mapping.c
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/Kconfig
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h
>
>
>
>

Jernej Škrabec June 6, 2021, 5:14 p.m. UTC | #3

Dne nedelja, 06. junij 2021 ob 18:54:05 CEST je Guo Ren napisal(a):
> V5 is not related to the patch series.

Don't top post.

If that's the case, then your e-mail client messed things up. I got V5 patches 
in the same thread as board enablement patches and so did mailing list [1].

Best regards,
Jernej

[1] https://lore.kernel.org/linux-sunxi/
CAJF2gTQ1yvSNthJ2yJkqaYGaJ6OYmf0vbG=hm4ocgnw4DiZbOw@mail.gmail.com/T/#t

> 
> On Mon, Jun 7, 2021 at 12:29 AM Jernej Škrabec <jernej.skrabec@gmail.com> 
wrote:
> > Hi!
> > 
> > Dne nedelja, 06. junij 2021 ob 11:03:55 CEST je guoren@kernel.org 
napisal(a):
> > > From: Guo Ren <guoren@linux.alibaba.com>
> > > 
> > > The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> > > vendors define the custom properties of memory regions in PTE.
> > > 
> > > This patchset helps SOC vendors to support their own custom interconnect
> > > coherent solution with PTE attributes.
> > > 
> > > For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906
> > > has
> > > 
> > > two modes in MMU:
> > >  - Compatible mode, the same as the definitions in spec.
> > >  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
> > >  
> > >    not mentioned in spec.
> > > 
> > > Allwinner D1 needs the enhanced mode to support the DMA type device with
> > > non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> > > attribute bits in PTE.
> > > 
> > > The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been
> > > 
> > > tested on D1:
> > >  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
> > >  
> > >    conflict with RISC-V spec, and hope these patches soon could be
> > >    approved.
> > >  
> > >  - pgtable: Using a image-hdr to pass vendor specific information and
> > >  
> > >    setup custom PTE attributes in a global struct variable during boot
> > >    stage. Also it needs define custom protection_map in linux/mm.
> > >  
> > >  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
> > >  
> > >    In this patchset, I just show you how T-HEAD C9xx work, and seems
> > >    Atish
> > >    is working for the DMA infrustructure, please let me know the idea.
> > >  
> > >  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> > > 
> > > The patchset could work with linux-5.13-rc4, here is the steps for D1:
> > >  - Download linux-5.13-rc4 and apply the patchset
> > >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
> > >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
> > >  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e
> > >  0x00200000
> > > 
> > > -n Linux -d arch/riscv/boot/Image uImage - Download newest opensbi [2],
> > > build with [3], and get fw_dynamic.bin - Copy uImage, fw_dynamic.bin,
> > > allwinner-d1-nezha-kit.dtb into boot partition of TF card.
> > > 
> > >  - Plugin the TF card and power on D1.
> > > 
> > > Link: https://linux-sunxi.org/D1 [1]
> > > Link: https://github.com/riscv/opensbi branch:master [2]
> > > Link:
> > > https://github.com/riscv/opensbi/blob/master/docs/platform/thead-c9xx.md
> > > [3]
> > 
> > Some patches are marked with v2 and some V5. It's very confusing. Mark
> > them
> > with same version in next revision.
> > 
> > Best regards,
> > Jernej
> > 
> > > Changes since v1:
> > >  - Rebase on linux-5.13-rc4
> > >  - Support defconfig for different PTE attributes
> > >  - Support C906 icache_sync
> > >  - Add Allwinner D1 dts & Kconfig & gmac for testing
> > >  - Add asid optimization for D1 usage
> > > 
> > > Guo Ren (10):
> > >   riscv: asid: Use global mappings for kernel pages
> > >   riscv: asid: Add ASID-based tlbflushing methods
> > >   riscv: asid: Optimize tlbflush coding convention
> > >   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
> > >   riscv: pgtable: Add custom protection_map init
> > >   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
> > >   riscv: cmo: Add dma-noncoherency support
> > >   riscv: cmo: Add vendor custom icache sync
> > >   riscv: soc: Initial DTS for Allwinner D1 NeZha board
> > >   riscv: soc: Add Allwinner SoC kconfig option
> > > 
> > > liush (1):
> > >   riscv: soc: Allwinner D1 GMAC driver only for temp use
> > >  
> > >  arch/riscv/Kconfig                                 |    9 +
> > >  arch/riscv/Kconfig.socs                            |   12 +
> > >  arch/riscv/boot/dts/Makefile                       |    1 +
> > >  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
> > >  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
> > >  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
> > >  arch/riscv/configs/defconfig                       |    1 +
> > >  arch/riscv/include/asm/cacheflush.h                |   48 +-
> > >  arch/riscv/include/asm/mmu_context.h               |    2 +
> > >  arch/riscv/include/asm/pgtable-64.h                |    8 +-
> > >  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
> > >  arch/riscv/include/asm/pgtable.h                   |   44 +-
> > >  arch/riscv/include/asm/sbi.h                       |   15 +
> > >  arch/riscv/include/asm/soc.h                       |    1 +
> > >  arch/riscv/include/asm/tlbflush.h                  |   22 +
> > >  arch/riscv/include/asm/vendorid_list.h             |    1 +
> > >  arch/riscv/kernel/sbi.c                            |   19 +
> > >  arch/riscv/kernel/soc.c                            |   22 +
> > >  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
> > >  arch/riscv/mm/Makefile                             |    1 +
> > >  arch/riscv/mm/cacheflush.c                         |    3 +-
> > >  arch/riscv/mm/context.c                            |    2 +-
> > >  arch/riscv/mm/dma-mapping.c                        |   53 +
> > >  arch/riscv/mm/init.c                               |   26 +
> > >  arch/riscv/mm/tlbflush.c                           |   57 +-
> > >  drivers/net/ethernet/Kconfig                       |    1 +
> > >  drivers/net/ethernet/Makefile                      |    1 +
> > >  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
> > >  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
> > >  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
> > >  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> > > 
> > > ++++++++++++++++++++ drivers/net/ethernet/allwinnertmp/sunxi-gmac.h    
> > > |
> > > 258 +++
> > > 
> > >  drivers/net/phy/realtek.c                          |    2 +-
> > >  mm/mmap.c                                          |    4 +
> > >  34 files changed, 3714 insertions(+), 37 deletions(-)
> > >  create mode 100644 arch/riscv/boot/dts/allwinner/Makefile
> > >  create mode 100644
> > >  arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-kit.dts
> > > 
> > > create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi
> > > create
> > > mode 100644 arch/riscv/mm/dma-mapping.c
> > > 
> > >  create mode 100644 drivers/net/ethernet/allwinnertmp/Kconfig
> > >  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
> > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
> > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
> > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h

Guo Ren June 6, 2021, 11:42 p.m. UTC | #4

On Mon, Jun 7, 2021 at 1:14 AM Jernej Škrabec <jernej.skrabec@gmail.com> wrote:
>
> Dne nedelja, 06. junij 2021 ob 18:54:05 CEST je Guo Ren napisal(a):
> > V5 is not related to the patch series.
>
> Don't top post.
>
> If that's the case, then your e-mail client messed things up. I got V5 patches
> in the same thread as board enablement patches and so did mailing list [1].
I made mistake that make two patchset mixed in my linux directory.

git format-patch -11
git send-email *.patch

But there are still some last posted patches in *.patch.

I'll prevent it in the next patchset version.

>
> Best regards,
> Jernej
>
> [1] https://lore.kernel.org/linux-sunxi/
> CAJF2gTQ1yvSNthJ2yJkqaYGaJ6OYmf0vbG=hm4ocgnw4DiZbOw@mail.gmail.com/T/#t
>
> >
> > On Mon, Jun 7, 2021 at 12:29 AM Jernej Škrabec <jernej.skrabec@gmail.com>
> wrote:
> > > Hi!
> > >
> > > Dne nedelja, 06. junij 2021 ob 11:03:55 CEST je guoren@kernel.org
> napisal(a):
> > > > From: Guo Ren <guoren@linux.alibaba.com>
> > > >
> > > > The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> > > > vendors define the custom properties of memory regions in PTE.
> > > >
> > > > This patchset helps SOC vendors to support their own custom interconnect
> > > > coherent solution with PTE attributes.
> > > >
> > > > For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906
> > > > has
> > > >
> > > > two modes in MMU:
> > > >  - Compatible mode, the same as the definitions in spec.
> > > >  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
> > > >
> > > >    not mentioned in spec.
> > > >
> > > > Allwinner D1 needs the enhanced mode to support the DMA type device with
> > > > non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> > > > attribute bits in PTE.
> > > >
> > > > The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been
> > > >
> > > > tested on D1:
> > > >  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
> > > >
> > > >    conflict with RISC-V spec, and hope these patches soon could be
> > > >    approved.
> > > >
> > > >  - pgtable: Using a image-hdr to pass vendor specific information and
> > > >
> > > >    setup custom PTE attributes in a global struct variable during boot
> > > >    stage. Also it needs define custom protection_map in linux/mm.
> > > >
> > > >  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
> > > >
> > > >    In this patchset, I just show you how T-HEAD C9xx work, and seems
> > > >    Atish
> > > >    is working for the DMA infrustructure, please let me know the idea.
> > > >
> > > >  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> > > >
> > > > The patchset could work with linux-5.13-rc4, here is the steps for D1:
> > > >  - Download linux-5.13-rc4 and apply the patchset
> > > >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
> > > >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
> > > >  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e
> > > >  0x00200000
> > > >
> > > > -n Linux -d arch/riscv/boot/Image uImage - Download newest opensbi [2],
> > > > build with [3], and get fw_dynamic.bin - Copy uImage, fw_dynamic.bin,
> > > > allwinner-d1-nezha-kit.dtb into boot partition of TF card.
> > > >
> > > >  - Plugin the TF card and power on D1.
> > > >
> > > > Link: https://linux-sunxi.org/D1 [1]
> > > > Link: https://github.com/riscv/opensbi branch:master [2]
> > > > Link:
> > > > https://github.com/riscv/opensbi/blob/master/docs/platform/thead-c9xx.md
> > > > [3]
> > >
> > > Some patches are marked with v2 and some V5. It's very confusing. Mark
> > > them
> > > with same version in next revision.
> > >
> > > Best regards,
> > > Jernej
> > >
> > > > Changes since v1:
> > > >  - Rebase on linux-5.13-rc4
> > > >  - Support defconfig for different PTE attributes
> > > >  - Support C906 icache_sync
> > > >  - Add Allwinner D1 dts & Kconfig & gmac for testing
> > > >  - Add asid optimization for D1 usage
> > > >
> > > > Guo Ren (10):
> > > >   riscv: asid: Use global mappings for kernel pages
> > > >   riscv: asid: Add ASID-based tlbflushing methods
> > > >   riscv: asid: Optimize tlbflush coding convention
> > > >   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
> > > >   riscv: pgtable: Add custom protection_map init
> > > >   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
> > > >   riscv: cmo: Add dma-noncoherency support
> > > >   riscv: cmo: Add vendor custom icache sync
> > > >   riscv: soc: Initial DTS for Allwinner D1 NeZha board
> > > >   riscv: soc: Add Allwinner SoC kconfig option
> > > >
> > > > liush (1):
> > > >   riscv: soc: Allwinner D1 GMAC driver only for temp use
> > > >
> > > >  arch/riscv/Kconfig                                 |    9 +
> > > >  arch/riscv/Kconfig.socs                            |   12 +
> > > >  arch/riscv/boot/dts/Makefile                       |    1 +
> > > >  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
> > > >  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
> > > >  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
> > > >  arch/riscv/configs/defconfig                       |    1 +
> > > >  arch/riscv/include/asm/cacheflush.h                |   48 +-
> > > >  arch/riscv/include/asm/mmu_context.h               |    2 +
> > > >  arch/riscv/include/asm/pgtable-64.h                |    8 +-
> > > >  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
> > > >  arch/riscv/include/asm/pgtable.h                   |   44 +-
> > > >  arch/riscv/include/asm/sbi.h                       |   15 +
> > > >  arch/riscv/include/asm/soc.h                       |    1 +
> > > >  arch/riscv/include/asm/tlbflush.h                  |   22 +
> > > >  arch/riscv/include/asm/vendorid_list.h             |    1 +
> > > >  arch/riscv/kernel/sbi.c                            |   19 +
> > > >  arch/riscv/kernel/soc.c                            |   22 +
> > > >  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
> > > >  arch/riscv/mm/Makefile                             |    1 +
> > > >  arch/riscv/mm/cacheflush.c                         |    3 +-
> > > >  arch/riscv/mm/context.c                            |    2 +-
> > > >  arch/riscv/mm/dma-mapping.c                        |   53 +
> > > >  arch/riscv/mm/init.c                               |   26 +
> > > >  arch/riscv/mm/tlbflush.c                           |   57 +-
> > > >  drivers/net/ethernet/Kconfig                       |    1 +
> > > >  drivers/net/ethernet/Makefile                      |    1 +
> > > >  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
> > > >  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
> > > >  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
> > > >  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> > > >
> > > > ++++++++++++++++++++ drivers/net/ethernet/allwinnertmp/sunxi-gmac.h
> > > > |
> > > > 258 +++
> > > >
> > > >  drivers/net/phy/realtek.c                          |    2 +-
> > > >  mm/mmap.c                                          |    4 +
> > > >  34 files changed, 3714 insertions(+), 37 deletions(-)
> > > >  create mode 100644 arch/riscv/boot/dts/allwinner/Makefile
> > > >  create mode 100644
> > > >  arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-kit.dts
> > > >
> > > > create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi
> > > > create
> > > > mode 100644 arch/riscv/mm/dma-mapping.c
> > > >
> > > >  create mode 100644 drivers/net/ethernet/allwinnertmp/Kconfig
> > > >  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
> > > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
> > > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
> > > >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h
>
>
>
>

Anup Patel June 7, 2021, 3:44 a.m. UTC | #5

> -----Original Message-----
> From: guoren@kernel.org <guoren@kernel.org>
> Sent: 06 June 2021 14:34
> To: guoren@kernel.org; Anup Patel <Anup.Patel@wdc.com>;
> palmerdabbelt@google.com; arnd@arndb.de; wens@csie.org;
> maxime@cerno.tech; drew@beagleboard.org; liush@allwinnertech.com;
> lazyparser@gmail.com; wefu@redhat.com
> Cc: linux-riscv@lists.infradead.org; linux-kernel@vger.kernel.org; linux-
> arch@vger.kernel.org; linux-sunxi@lists.linux.dev; Guo Ren
> <guoren@linux.alibaba.com>
> Subject: [RFC PATCH v2 00/11] riscv: Add DMA_COHERENT support for
> Allwinner D1
> 
> From: Guo Ren <guoren@linux.alibaba.com>
> 
> The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> vendors define the custom properties of memory regions in PTE.
> 
> This patchset helps SOC vendors to support their own custom interconnect
> coherent solution with PTE attributes.
> 
> For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906 has
> two modes in MMU:
>  - Compatible mode, the same as the definitions in spec.
>  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
>    not mentioned in spec.
> 
> Allwinner D1 needs the enhanced mode to support the DMA type device with
> non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> attribute bits in PTE.
> 
> The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been tested
> on D1:
>  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
>    conflict with RISC-V spec, and hope these patches soon could be
>    approved.
>  - pgtable: Using a image-hdr to pass vendor specific information and
>    setup custom PTE attributes in a global struct variable during boot
>    stage. Also it needs define custom protection_map in linux/mm.
>  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
>    In this patchset, I just show you how T-HEAD C9xx work, and seems Atish
>    is working for the DMA infrustructure, please let me know the idea.
>  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> 
> The patchset could work with linux-5.13-rc4, here is the steps for D1:
>  - Download linux-5.13-rc4 and apply the patchset
>  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
>  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
>  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e 0x00200000 -
> n Linux -d arch/riscv/boot/Image uImage
>  - Download newest opensbi [2], build with [3], and get fw_dynamic.bin
>  - Copy uImage, fw_dynamic.bin, allwinner-d1-nezha-kit.dtb into boot
>    partition of TF card.
>  - Plugin the TF card and power on D1.
> 
> Link: https://linux-sunxi.org/D1 [1]
> Link: https://github.com/riscv/opensbi branch:master [2]
> Link: https://github.com/riscv/opensbi/blob/master/docs/platform/thead-
> c9xx.md [3]
> 
> Changes since v1:
>  - Rebase on linux-5.13-rc4
>  - Support defconfig for different PTE attributes
>  - Support C906 icache_sync
>  - Add Allwinner D1 dts & Kconfig & gmac for testing
>  - Add asid optimization for D1 usage
> 
> Guo Ren (10):
>   riscv: asid: Use global mappings for kernel pages
>   riscv: asid: Add ASID-based tlbflushing methods
>   riscv: asid: Optimize tlbflush coding convention
>   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
>   riscv: pgtable: Add custom protection_map init
>   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
>   riscv: cmo: Add dma-noncoherency support
>   riscv: cmo: Add vendor custom icache sync
>   riscv: soc: Initial DTS for Allwinner D1 NeZha board
>   riscv: soc: Add Allwinner SoC kconfig option

The series cover letter says DMA_COHERENT support but
it is doing lot of stuff not related to DMA.

Please keep the first three patches separate. They belong
to your ASID series.

I also dislike the fact that you are continuously sending
SBI DMA sync patches without any discussion on the
UnixPlatform mailing list for SBI spec changes.

Regards,
Anup

> 
> liush (1):
>   riscv: soc: Allwinner D1 GMAC driver only for temp use
> 
>  arch/riscv/Kconfig                                 |    9 +
>  arch/riscv/Kconfig.socs                            |   12 +
>  arch/riscv/boot/dts/Makefile                       |    1 +
>  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
>  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
>  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
>  arch/riscv/configs/defconfig                       |    1 +
>  arch/riscv/include/asm/cacheflush.h                |   48 +-
>  arch/riscv/include/asm/mmu_context.h               |    2 +
>  arch/riscv/include/asm/pgtable-64.h                |    8 +-
>  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
>  arch/riscv/include/asm/pgtable.h                   |   44 +-
>  arch/riscv/include/asm/sbi.h                       |   15 +
>  arch/riscv/include/asm/soc.h                       |    1 +
>  arch/riscv/include/asm/tlbflush.h                  |   22 +
>  arch/riscv/include/asm/vendorid_list.h             |    1 +
>  arch/riscv/kernel/sbi.c                            |   19 +
>  arch/riscv/kernel/soc.c                            |   22 +
>  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
>  arch/riscv/mm/Makefile                             |    1 +
>  arch/riscv/mm/cacheflush.c                         |    3 +-
>  arch/riscv/mm/context.c                            |    2 +-
>  arch/riscv/mm/dma-mapping.c                        |   53 +
>  arch/riscv/mm/init.c                               |   26 +
>  arch/riscv/mm/tlbflush.c                           |   57 +-
>  drivers/net/ethernet/Kconfig                       |    1 +
>  drivers/net/ethernet/Makefile                      |    1 +
>  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
>  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
>  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
>  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> ++++++++++++++++++++
>  drivers/net/ethernet/allwinnertmp/sunxi-gmac.h     |  258 +++
>  drivers/net/phy/realtek.c                          |    2 +-
>  mm/mmap.c                                          |    4 +
>  34 files changed, 3714 insertions(+), 37 deletions(-)  create mode 100644
> arch/riscv/boot/dts/allwinner/Makefile
>  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-
> kit.dts
>  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi
>  create mode 100644 arch/riscv/mm/dma-mapping.c  create mode 100644
> drivers/net/ethernet/allwinnertmp/Kconfig
>  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
>  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h
> 
> --
> 2.7.4

Guo Ren June 7, 2021, 4:36 a.m. UTC | #6

Hi Anup,

On Mon, Jun 7, 2021 at 11:44 AM Anup Patel <Anup.Patel@wdc.com> wrote:
>
>
>
> > -----Original Message-----
> > From: guoren@kernel.org <guoren@kernel.org>
> > Sent: 06 June 2021 14:34
> > To: guoren@kernel.org; Anup Patel <Anup.Patel@wdc.com>;
> > palmerdabbelt@google.com; arnd@arndb.de; wens@csie.org;
> > maxime@cerno.tech; drew@beagleboard.org; liush@allwinnertech.com;
> > lazyparser@gmail.com; wefu@redhat.com
> > Cc: linux-riscv@lists.infradead.org; linux-kernel@vger.kernel.org; linux-
> > arch@vger.kernel.org; linux-sunxi@lists.linux.dev; Guo Ren
> > <guoren@linux.alibaba.com>
> > Subject: [RFC PATCH v2 00/11] riscv: Add DMA_COHERENT support for
> > Allwinner D1
> >
> > From: Guo Ren <guoren@linux.alibaba.com>
> >
> > The RISC-V ISA doesn't yet specify how to query or modify PMAs, so let
> > vendors define the custom properties of memory regions in PTE.
> >
> > This patchset helps SOC vendors to support their own custom interconnect
> > coherent solution with PTE attributes.
> >
> > For example, allwinner D1[1] uses T-HEAD C906 as main processor, C906 has
> > two modes in MMU:
> >  - Compatible mode, the same as the definitions in spec.
> >  - Enhanced mode, add custom DMA_COHERENT attribute bits in PTE which
> >    not mentioned in spec.
> >
> > Allwinner D1 needs the enhanced mode to support the DMA type device with
> > non-coherent interconnect in its SOC. C906 uses BITS(63 - 59) as custom
> > attribute bits in PTE.
> >
> > The patchset contain 4 parts (asid, pgtable, cmo, soc) which have been tested
> > on D1:
> >  - asid: T-HEAD C906 of D1 contains full asid hw facilities which has no
> >    conflict with RISC-V spec, and hope these patches soon could be
> >    approved.
> >  - pgtable: Using a image-hdr to pass vendor specific information and
> >    setup custom PTE attributes in a global struct variable during boot
> >    stage. Also it needs define custom protection_map in linux/mm.
> >  - cmo: We need deal with dma_sync & icache_sync & __vdso_icache_sync.
> >    In this patchset, I just show you how T-HEAD C9xx work, and seems Atish
> >    is working for the DMA infrustructure, please let me know the idea.
> >  - soc: Add allwinner gmac driver & dts & Kconfig for sunxi test.
> >
> > The patchset could work with linux-5.13-rc4, here is the steps for D1:
> >  - Download linux-5.13-rc4 and apply the patchset
> >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- defconfig
> >  - make ARCH=riscv CROSS_COMPILE=riscv64-linux- Image modules dtbs
> >  - mkimage -A riscv -O linux -T kernel -C none -a 0x00200000 -e 0x00200000 -
> > n Linux -d arch/riscv/boot/Image uImage
> >  - Download newest opensbi [2], build with [3], and get fw_dynamic.bin
> >  - Copy uImage, fw_dynamic.bin, allwinner-d1-nezha-kit.dtb into boot
> >    partition of TF card.
> >  - Plugin the TF card and power on D1.
> >
> > Link: https://linux-sunxi.org/D1 [1]
> > Link: https://github.com/riscv/opensbi branch:master [2]
> > Link: https://github.com/riscv/opensbi/blob/master/docs/platform/thead-
> > c9xx.md [3]
> >
> > Changes since v1:
> >  - Rebase on linux-5.13-rc4
> >  - Support defconfig for different PTE attributes
> >  - Support C906 icache_sync
> >  - Add Allwinner D1 dts & Kconfig & gmac for testing
> >  - Add asid optimization for D1 usage
> >
> > Guo Ren (10):
> >   riscv: asid: Use global mappings for kernel pages
> >   riscv: asid: Add ASID-based tlbflushing methods
> >   riscv: asid: Optimize tlbflush coding convention
> >   riscv: pgtable: Fixup _PAGE_CHG_MASK usage
> >   riscv: pgtable: Add custom protection_map init
> >   riscv: pgtable: Add DMA_COHERENT with custom PTE attributes
> >   riscv: cmo: Add dma-noncoherency support
> >   riscv: cmo: Add vendor custom icache sync
> >   riscv: soc: Initial DTS for Allwinner D1 NeZha board
> >   riscv: soc: Add Allwinner SoC kconfig option
>
> The series cover letter says DMA_COHERENT support but
> it is doing lot of stuff not related to DMA.
>
> Please keep the first three patches separate. They belong
> to your ASID series.
I just want to give out a whole view of how allwinner D1 works, which
could help soc folks run their mini system.

Christoph Hellwig has helped to update the ASID series, and I've given
the tested-by on my hardware 4*910 SMP & 906 D1
https://lore.kernel.org/linux-riscv/20210606152050.636038-1-hch@lst.de/T/#t

Ok, you won't see it in the next version patchset.

>
> I also dislike the fact that you are continuously sending
> SBI DMA sync patches without any discussion on the
> UnixPlatform mailing list for SBI spec changes.
SBI DMA sync is not critical for us, I could follow your definition
when you are ready. Even CBO trap emulation is okay for me.

>
> Regards,
> Anup
>
> >
> > liush (1):
> >   riscv: soc: Allwinner D1 GMAC driver only for temp use
> >
> >  arch/riscv/Kconfig                                 |    9 +
> >  arch/riscv/Kconfig.socs                            |   12 +
> >  arch/riscv/boot/dts/Makefile                       |    1 +
> >  arch/riscv/boot/dts/allwinner/Makefile             |    2 +
> >  .../boot/dts/allwinner/allwinner-d1-nezha-kit.dts  |   29 +
> >  arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi    |  100 +
> >  arch/riscv/configs/defconfig                       |    1 +
> >  arch/riscv/include/asm/cacheflush.h                |   48 +-
> >  arch/riscv/include/asm/mmu_context.h               |    2 +
> >  arch/riscv/include/asm/pgtable-64.h                |    8 +-
> >  arch/riscv/include/asm/pgtable-bits.h              |   20 +-
> >  arch/riscv/include/asm/pgtable.h                   |   44 +-
> >  arch/riscv/include/asm/sbi.h                       |   15 +
> >  arch/riscv/include/asm/soc.h                       |    1 +
> >  arch/riscv/include/asm/tlbflush.h                  |   22 +
> >  arch/riscv/include/asm/vendorid_list.h             |    1 +
> >  arch/riscv/kernel/sbi.c                            |   19 +
> >  arch/riscv/kernel/soc.c                            |   22 +
> >  arch/riscv/kernel/vdso/flush_icache.S              |   33 +-
> >  arch/riscv/mm/Makefile                             |    1 +
> >  arch/riscv/mm/cacheflush.c                         |    3 +-
> >  arch/riscv/mm/context.c                            |    2 +-
> >  arch/riscv/mm/dma-mapping.c                        |   53 +
> >  arch/riscv/mm/init.c                               |   26 +
> >  arch/riscv/mm/tlbflush.c                           |   57 +-
> >  drivers/net/ethernet/Kconfig                       |    1 +
> >  drivers/net/ethernet/Makefile                      |    1 +
> >  drivers/net/ethernet/allwinnertmp/Kconfig          |   17 +
> >  drivers/net/ethernet/allwinnertmp/Makefile         |    7 +
> >  drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c |  690 ++++++
> >  drivers/net/ethernet/allwinnertmp/sunxi-gmac.c     | 2240
> > ++++++++++++++++++++
> >  drivers/net/ethernet/allwinnertmp/sunxi-gmac.h     |  258 +++
> >  drivers/net/phy/realtek.c                          |    2 +-
> >  mm/mmap.c                                          |    4 +
> >  34 files changed, 3714 insertions(+), 37 deletions(-)  create mode 100644
> > arch/riscv/boot/dts/allwinner/Makefile
> >  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1-nezha-
> > kit.dts
> >  create mode 100644 arch/riscv/boot/dts/allwinner/allwinner-d1.dtsi
> >  create mode 100644 arch/riscv/mm/dma-mapping.c  create mode 100644
> > drivers/net/ethernet/allwinnertmp/Kconfig
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/Makefile
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac-ops.c
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.c
> >  create mode 100644 drivers/net/ethernet/allwinnertmp/sunxi-gmac.h
> >
> > --
> > 2.7.4
>

[RFC,v2,00/11] riscv: Add DMA_COHERENT support for Allwinner D1

Message

Comments