mbox series

[v13,0/5] DMA-BUF Heaps (destaging ION)

Message ID 20191021190310.85221-1-john.stultz@linaro.org (mailing list archive)
Headers show
Series DMA-BUF Heaps (destaging ION) | expand

Message

John Stultz Oct. 21, 2019, 7:03 p.m. UTC
Lucky number 13! :)

Last week in v12 I had re-added some symbol exports to support
later patches I have pending to enable loading heaps from
modules. He reminded me that back around v3 (its been awhile!) I
had removed those exports due to concerns about the fact that we
don't support module removal.

So I'm respinning the patches, removing the exports again. I'll
submit a patch to re-add them in a later series enabling moduels
which can be reviewed indepently.

With that done, lets get on to the boilerplate!

The patchset implements per-heap devices which can be opened
directly and then an ioctl is used to allocate a dmabuf from the
heap.

The interface is similar, but much simpler then IONs, only
providing an ALLOC ioctl.

Also, I've provided relatively simple system and cma heaps.

I've booted and tested these patches with AOSP on the HiKey960
using the kernel tree here:
  https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap

And the userspace changes here:
  https://android-review.googlesource.com/c/device/linaro/hikey/+/909436

Compared to ION, this patchset is missing the system-contig,
carveout and chunk heaps, as I don't have a device that uses
those, so I'm unable to do much useful validation there.
Additionally we have no upstream users of chunk or carveout,
and the system-contig has been deprecated in the common/andoid-*
kernels, so this should be ok.

I've also removed the stats accounting, since any such
accounting should be implemented by dma-buf core or the heaps
themselves.

New in v13:
* Re-remove symbol exports, per discussion with Brian. I'll
  resubmit these separately in a later patch so they can be
  independently reviewed

thanks
-john

Cc: Laura Abbott <labbott@redhat.com>
Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
Cc: Sumit Semwal <sumit.semwal@linaro.org>
Cc: Liam Mark <lmark@codeaurora.org>
Cc: Pratik Patel <pratikp@codeaurora.org>
Cc: Brian Starkey <Brian.Starkey@arm.com>
Cc: Vincent Donnefort <Vincent.Donnefort@arm.com>
Cc: Sudipto Paul <Sudipto.Paul@arm.com>
Cc: Andrew F. Davis <afd@ti.com>
Cc: Christoph Hellwig <hch@infradead.org>
Cc: Chenbo Feng <fengc@google.com>
Cc: Alistair Strachan <astrachan@google.com>
Cc: Hridya Valsaraju <hridya@google.com>
Cc: Hillf Danton <hdanton@sina.com>
Cc: dri-devel@lists.freedesktop.org



Andrew F. Davis (1):
  dma-buf: Add dma-buf heaps framework

John Stultz (4):
  dma-buf: heaps: Add heap helpers
  dma-buf: heaps: Add system heap to dmabuf heaps
  dma-buf: heaps: Add CMA heap to dmabuf heaps
  kselftests: Add dma-heap test

 MAINTAINERS                                   |  18 ++
 drivers/dma-buf/Kconfig                       |  11 +
 drivers/dma-buf/Makefile                      |   2 +
 drivers/dma-buf/dma-heap.c                    | 269 ++++++++++++++++++
 drivers/dma-buf/heaps/Kconfig                 |  14 +
 drivers/dma-buf/heaps/Makefile                |   4 +
 drivers/dma-buf/heaps/cma_heap.c              | 178 ++++++++++++
 drivers/dma-buf/heaps/heap-helpers.c          | 268 +++++++++++++++++
 drivers/dma-buf/heaps/heap-helpers.h          |  55 ++++
 drivers/dma-buf/heaps/system_heap.c           | 124 ++++++++
 include/linux/dma-heap.h                      |  59 ++++
 include/uapi/linux/dma-heap.h                 |  55 ++++
 tools/testing/selftests/dmabuf-heaps/Makefile |   9 +
 .../selftests/dmabuf-heaps/dmabuf-heap.c      | 238 ++++++++++++++++
 14 files changed, 1304 insertions(+)
 create mode 100644 drivers/dma-buf/dma-heap.c
 create mode 100644 drivers/dma-buf/heaps/Kconfig
 create mode 100644 drivers/dma-buf/heaps/Makefile
 create mode 100644 drivers/dma-buf/heaps/cma_heap.c
 create mode 100644 drivers/dma-buf/heaps/heap-helpers.c
 create mode 100644 drivers/dma-buf/heaps/heap-helpers.h
 create mode 100644 drivers/dma-buf/heaps/system_heap.c
 create mode 100644 include/linux/dma-heap.h
 create mode 100644 include/uapi/linux/dma-heap.h
 create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile
 create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c

Comments

John Stultz Oct. 21, 2019, 7:04 p.m. UTC | #1
On Mon, Oct 21, 2019 at 12:03 PM John Stultz <john.stultz@linaro.org> wrote:
>
> Lucky number 13! :)
>
> Last week in v12 I had re-added some symbol exports to support
> later patches I have pending to enable loading heaps from
> modules. He reminded me that back around v3 (its been awhile!) I

By "He" I mean Brian here.

thanks
-john
Neil Armstrong Oct. 22, 2019, 8:21 a.m. UTC | #2
Hi John,

On 21/10/2019 21:03, John Stultz wrote:
> Lucky number 13! :)
> 
> Last week in v12 I had re-added some symbol exports to support
> later patches I have pending to enable loading heaps from
> modules. He reminded me that back around v3 (its been awhile!) I
> had removed those exports due to concerns about the fact that we
> don't support module removal.
> 
> So I'm respinning the patches, removing the exports again. I'll
> submit a patch to re-add them in a later series enabling moduels
> which can be reviewed indepently.
> 
> With that done, lets get on to the boilerplate!
> 
> The patchset implements per-heap devices which can be opened
> directly and then an ioctl is used to allocate a dmabuf from the
> heap.
> 
> The interface is similar, but much simpler then IONs, only
> providing an ALLOC ioctl.
> 
> Also, I've provided relatively simple system and cma heaps.
> 
> I've booted and tested these patches with AOSP on the HiKey960
> using the kernel tree here:
>   https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap

Do you have a 4.19 tree with the changes ? I tried but the xarray idr replacement
is missing... so I can't test with our android-amlogic-bmeson-4.19 tree.

If you can provide, I'll be happy to test the serie and the gralloc changes.

Neil

> 
> And the userspace changes here:
>   https://android-review.googlesource.com/c/device/linaro/hikey/+/909436
> 
> Compared to ION, this patchset is missing the system-contig,
> carveout and chunk heaps, as I don't have a device that uses
> those, so I'm unable to do much useful validation there.
> Additionally we have no upstream users of chunk or carveout,
> and the system-contig has been deprecated in the common/andoid-*
> kernels, so this should be ok.
> 
> I've also removed the stats accounting, since any such
> accounting should be implemented by dma-buf core or the heaps
> themselves.
> 
> New in v13:
> * Re-remove symbol exports, per discussion with Brian. I'll
>   resubmit these separately in a later patch so they can be
>   independently reviewed
> 
> thanks
> -john
> 
> Cc: Laura Abbott <labbott@redhat.com>
> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
> Cc: Sumit Semwal <sumit.semwal@linaro.org>
> Cc: Liam Mark <lmark@codeaurora.org>
> Cc: Pratik Patel <pratikp@codeaurora.org>
> Cc: Brian Starkey <Brian.Starkey@arm.com>
> Cc: Vincent Donnefort <Vincent.Donnefort@arm.com>
> Cc: Sudipto Paul <Sudipto.Paul@arm.com>
> Cc: Andrew F. Davis <afd@ti.com>
> Cc: Christoph Hellwig <hch@infradead.org>
> Cc: Chenbo Feng <fengc@google.com>
> Cc: Alistair Strachan <astrachan@google.com>
> Cc: Hridya Valsaraju <hridya@google.com>
> Cc: Hillf Danton <hdanton@sina.com>
> Cc: dri-devel@lists.freedesktop.org
> 
> 
> 
> Andrew F. Davis (1):
>   dma-buf: Add dma-buf heaps framework
> 
> John Stultz (4):
>   dma-buf: heaps: Add heap helpers
>   dma-buf: heaps: Add system heap to dmabuf heaps
>   dma-buf: heaps: Add CMA heap to dmabuf heaps
>   kselftests: Add dma-heap test
> 
>  MAINTAINERS                                   |  18 ++
>  drivers/dma-buf/Kconfig                       |  11 +
>  drivers/dma-buf/Makefile                      |   2 +
>  drivers/dma-buf/dma-heap.c                    | 269 ++++++++++++++++++
>  drivers/dma-buf/heaps/Kconfig                 |  14 +
>  drivers/dma-buf/heaps/Makefile                |   4 +
>  drivers/dma-buf/heaps/cma_heap.c              | 178 ++++++++++++
>  drivers/dma-buf/heaps/heap-helpers.c          | 268 +++++++++++++++++
>  drivers/dma-buf/heaps/heap-helpers.h          |  55 ++++
>  drivers/dma-buf/heaps/system_heap.c           | 124 ++++++++
>  include/linux/dma-heap.h                      |  59 ++++
>  include/uapi/linux/dma-heap.h                 |  55 ++++
>  tools/testing/selftests/dmabuf-heaps/Makefile |   9 +
>  .../selftests/dmabuf-heaps/dmabuf-heap.c      | 238 ++++++++++++++++
>  14 files changed, 1304 insertions(+)
>  create mode 100644 drivers/dma-buf/dma-heap.c
>  create mode 100644 drivers/dma-buf/heaps/Kconfig
>  create mode 100644 drivers/dma-buf/heaps/Makefile
>  create mode 100644 drivers/dma-buf/heaps/cma_heap.c
>  create mode 100644 drivers/dma-buf/heaps/heap-helpers.c
>  create mode 100644 drivers/dma-buf/heaps/heap-helpers.h
>  create mode 100644 drivers/dma-buf/heaps/system_heap.c
>  create mode 100644 include/linux/dma-heap.h
>  create mode 100644 include/uapi/linux/dma-heap.h
>  create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile
>  create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
>
John Stultz Oct. 22, 2019, 3:56 p.m. UTC | #3
On Tue, Oct 22, 2019 at 1:21 AM Neil Armstrong <narmstrong@baylibre.com> wrote:
>
> Hi John,
>
> On 21/10/2019 21:03, John Stultz wrote:
> > Lucky number 13! :)
> >
> > Last week in v12 I had re-added some symbol exports to support
> > later patches I have pending to enable loading heaps from
> > modules. He reminded me that back around v3 (its been awhile!) I
> > had removed those exports due to concerns about the fact that we
> > don't support module removal.
> >
> > So I'm respinning the patches, removing the exports again. I'll
> > submit a patch to re-add them in a later series enabling moduels
> > which can be reviewed indepently.
> >
> > With that done, lets get on to the boilerplate!
> >
> > The patchset implements per-heap devices which can be opened
> > directly and then an ioctl is used to allocate a dmabuf from the
> > heap.
> >
> > The interface is similar, but much simpler then IONs, only
> > providing an ALLOC ioctl.
> >
> > Also, I've provided relatively simple system and cma heaps.
> >
> > I've booted and tested these patches with AOSP on the HiKey960
> > using the kernel tree here:
> >   https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
>
> Do you have a 4.19 tree with the changes ? I tried but the xarray idr replacement
> is missing... so I can't test with our android-amlogic-bmeson-4.19 tree.
>
> If you can provide, I'll be happy to test the serie and the gralloc changes.

Unfortunately I don't have a 4.19 version of dmabuf heaps (all the
work has been done this year, post 4.19). I'm planning to backport to
5.4 for AOSP, but I've not really thought about 4.19. Most likely I
won't have time to look at it until after the changes are upstream and
the 5.4 backport is done.

Is the bmeson tree likely to only stay at 4.19? Or will it move forward?

thanks
-john
Andrew Davis Oct. 22, 2019, 4:01 p.m. UTC | #4
On 10/22/19 3:21 AM, Neil Armstrong wrote:
> Hi John,
> 
> On 21/10/2019 21:03, John Stultz wrote:
>> Lucky number 13! :)
>>
>> Last week in v12 I had re-added some symbol exports to support
>> later patches I have pending to enable loading heaps from
>> modules. He reminded me that back around v3 (its been awhile!) I
>> had removed those exports due to concerns about the fact that we
>> don't support module removal.
>>
>> So I'm respinning the patches, removing the exports again. I'll
>> submit a patch to re-add them in a later series enabling moduels
>> which can be reviewed indepently.
>>
>> With that done, lets get on to the boilerplate!
>>
>> The patchset implements per-heap devices which can be opened
>> directly and then an ioctl is used to allocate a dmabuf from the
>> heap.
>>
>> The interface is similar, but much simpler then IONs, only
>> providing an ALLOC ioctl.
>>
>> Also, I've provided relatively simple system and cma heaps.
>>
>> I've booted and tested these patches with AOSP on the HiKey960
>> using the kernel tree here:
>>   https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
> 
> Do you have a 4.19 tree with the changes ? I tried but the xarray idr replacement
> is missing... so I can't test with our android-amlogic-bmeson-4.19 tree.
> 


Just a note, we switched to xarray around v4 time frame of this series,
so you may be able to find the IDR based code and plug it in for a 4.19
port.

Andrew


> If you can provide, I'll be happy to test the serie and the gralloc changes.
> 
> Neil
> 
>>
>> And the userspace changes here:
>>   https://android-review.googlesource.com/c/device/linaro/hikey/+/909436
>>
>> Compared to ION, this patchset is missing the system-contig,
>> carveout and chunk heaps, as I don't have a device that uses
>> those, so I'm unable to do much useful validation there.
>> Additionally we have no upstream users of chunk or carveout,
>> and the system-contig has been deprecated in the common/andoid-*
>> kernels, so this should be ok.
>>
>> I've also removed the stats accounting, since any such
>> accounting should be implemented by dma-buf core or the heaps
>> themselves.
>>
>> New in v13:
>> * Re-remove symbol exports, per discussion with Brian. I'll
>>   resubmit these separately in a later patch so they can be
>>   independently reviewed
>>
>> thanks
>> -john
>>
>> Cc: Laura Abbott <labbott@redhat.com>
>> Cc: Benjamin Gaignard <benjamin.gaignard@linaro.org>
>> Cc: Sumit Semwal <sumit.semwal@linaro.org>
>> Cc: Liam Mark <lmark@codeaurora.org>
>> Cc: Pratik Patel <pratikp@codeaurora.org>
>> Cc: Brian Starkey <Brian.Starkey@arm.com>
>> Cc: Vincent Donnefort <Vincent.Donnefort@arm.com>
>> Cc: Sudipto Paul <Sudipto.Paul@arm.com>
>> Cc: Andrew F. Davis <afd@ti.com>
>> Cc: Christoph Hellwig <hch@infradead.org>
>> Cc: Chenbo Feng <fengc@google.com>
>> Cc: Alistair Strachan <astrachan@google.com>
>> Cc: Hridya Valsaraju <hridya@google.com>
>> Cc: Hillf Danton <hdanton@sina.com>
>> Cc: dri-devel@lists.freedesktop.org
>>
>>
>>
>> Andrew F. Davis (1):
>>   dma-buf: Add dma-buf heaps framework
>>
>> John Stultz (4):
>>   dma-buf: heaps: Add heap helpers
>>   dma-buf: heaps: Add system heap to dmabuf heaps
>>   dma-buf: heaps: Add CMA heap to dmabuf heaps
>>   kselftests: Add dma-heap test
>>
>>  MAINTAINERS                                   |  18 ++
>>  drivers/dma-buf/Kconfig                       |  11 +
>>  drivers/dma-buf/Makefile                      |   2 +
>>  drivers/dma-buf/dma-heap.c                    | 269 ++++++++++++++++++
>>  drivers/dma-buf/heaps/Kconfig                 |  14 +
>>  drivers/dma-buf/heaps/Makefile                |   4 +
>>  drivers/dma-buf/heaps/cma_heap.c              | 178 ++++++++++++
>>  drivers/dma-buf/heaps/heap-helpers.c          | 268 +++++++++++++++++
>>  drivers/dma-buf/heaps/heap-helpers.h          |  55 ++++
>>  drivers/dma-buf/heaps/system_heap.c           | 124 ++++++++
>>  include/linux/dma-heap.h                      |  59 ++++
>>  include/uapi/linux/dma-heap.h                 |  55 ++++
>>  tools/testing/selftests/dmabuf-heaps/Makefile |   9 +
>>  .../selftests/dmabuf-heaps/dmabuf-heap.c      | 238 ++++++++++++++++
>>  14 files changed, 1304 insertions(+)
>>  create mode 100644 drivers/dma-buf/dma-heap.c
>>  create mode 100644 drivers/dma-buf/heaps/Kconfig
>>  create mode 100644 drivers/dma-buf/heaps/Makefile
>>  create mode 100644 drivers/dma-buf/heaps/cma_heap.c
>>  create mode 100644 drivers/dma-buf/heaps/heap-helpers.c
>>  create mode 100644 drivers/dma-buf/heaps/heap-helpers.h
>>  create mode 100644 drivers/dma-buf/heaps/system_heap.c
>>  create mode 100644 include/linux/dma-heap.h
>>  create mode 100644 include/uapi/linux/dma-heap.h
>>  create mode 100644 tools/testing/selftests/dmabuf-heaps/Makefile
>>  create mode 100644 tools/testing/selftests/dmabuf-heaps/dmabuf-heap.c
>>
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
>
Neil Armstrong Oct. 23, 2019, 7:32 a.m. UTC | #5
On 22/10/2019 17:56, John Stultz wrote:
> On Tue, Oct 22, 2019 at 1:21 AM Neil Armstrong <narmstrong@baylibre.com> wrote:
>>
>> Hi John,
>>
>> On 21/10/2019 21:03, John Stultz wrote:
>>> Lucky number 13! :)
>>>
>>> Last week in v12 I had re-added some symbol exports to support
>>> later patches I have pending to enable loading heaps from
>>> modules. He reminded me that back around v3 (its been awhile!) I
>>> had removed those exports due to concerns about the fact that we
>>> don't support module removal.
>>>
>>> So I'm respinning the patches, removing the exports again. I'll
>>> submit a patch to re-add them in a later series enabling moduels
>>> which can be reviewed indepently.
>>>
>>> With that done, lets get on to the boilerplate!
>>>
>>> The patchset implements per-heap devices which can be opened
>>> directly and then an ioctl is used to allocate a dmabuf from the
>>> heap.
>>>
>>> The interface is similar, but much simpler then IONs, only
>>> providing an ALLOC ioctl.
>>>
>>> Also, I've provided relatively simple system and cma heaps.
>>>
>>> I've booted and tested these patches with AOSP on the HiKey960
>>> using the kernel tree here:
>>>   https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
>>
>> Do you have a 4.19 tree with the changes ? I tried but the xarray idr replacement
>> is missing... so I can't test with our android-amlogic-bmeson-4.19 tree.
>>
>> If you can provide, I'll be happy to test the serie and the gralloc changes.
> 
> Unfortunately I don't have a 4.19 version of dmabuf heaps (all the
> work has been done this year, post 4.19). I'm planning to backport to
> 5.4 for AOSP, but I've not really thought about 4.19. Most likely I
> won't have time to look at it until after the changes are upstream and
> the 5.4 backport is done.
> 
> Is the bmeson tree likely to only stay at 4.19? Or will it move forward?

No idea, I don't have any details on the future plans.
Since we did an upstream-first support, 90% will be available on the future android-5.4 tree anyway.

Neil

> 
> thanks
> -john
>
Sumit Semwal Oct. 25, 2019, 5:56 a.m. UTC | #6
Hi John,

On Tue, 22 Oct 2019 at 00:33, John Stultz <john.stultz@linaro.org> wrote:
>
> Lucky number 13! :)
>
> Last week in v12 I had re-added some symbol exports to support
> later patches I have pending to enable loading heaps from
> modules. He reminded me that back around v3 (its been awhile!) I
> had removed those exports due to concerns about the fact that we
> don't support module removal.
>
> So I'm respinning the patches, removing the exports again. I'll
> submit a patch to re-add them in a later series enabling moduels
> which can be reviewed indepently.

This looks good to me, and hasn't seen any more comments, so I am
going to merge it via drm-misc-next today.
>
> With that done, lets get on to the boilerplate!
>
> The patchset implements per-heap devices which can be opened
> directly and then an ioctl is used to allocate a dmabuf from the
> heap.
>
> The interface is similar, but much simpler then IONs, only
> providing an ALLOC ioctl.
>
> Also, I've provided relatively simple system and cma heaps.
>
> I've booted and tested these patches with AOSP on the HiKey960
> using the kernel tree here:
>   https://git.linaro.org/people/john.stultz/android-dev.git/log/?h=dev/dma-buf-heap
>
> And the userspace changes here:
>   https://android-review.googlesource.com/c/device/linaro/hikey/+/909436
>
> Compared to ION, this patchset is missing the system-contig,
> carveout and chunk heaps, as I don't have a device that uses
> those, so I'm unable to do much useful validation there.
> Additionally we have no upstream users of chunk or carveout,
> and the system-contig has been deprecated in the common/andoid-*
> kernels, so this should be ok.
>
> I've also removed the stats accounting, since any such
> accounting should be implemented by dma-buf core or the heaps
> themselves.
>
> New in v13:
> * Re-remove symbol exports, per discussion with Brian. I'll
>   resubmit these separately in a later patch so they can be
>   independently reviewed
>
> thanks
> -john

<snip>

Best,
Sumit.
Sumit Semwal Oct. 25, 2019, 11:34 a.m. UTC | #7
On Fri, 25 Oct 2019 at 11:26, Sumit Semwal <sumit.semwal@linaro.org> wrote:
>
> Hi John,
>
> On Tue, 22 Oct 2019 at 00:33, John Stultz <john.stultz@linaro.org> wrote:
> >
> > Lucky number 13! :)
> >
> > Last week in v12 I had re-added some symbol exports to support
> > later patches I have pending to enable loading heaps from
> > modules. He reminded me that back around v3 (its been awhile!) I
> > had removed those exports due to concerns about the fact that we
> > don't support module removal.
> >
> > So I'm respinning the patches, removing the exports again. I'll
> > submit a patch to re-add them in a later series enabling moduels
> > which can be reviewed indepently.
>
> This looks good to me, and hasn't seen any more comments, so I am
> going to merge it via drm-misc-next today.
Done; merged to drm-misc-next.

<snip>

Best,
Sumit.