mbox series

[0/4] Replace CONFIG_DMABUF_SYSFS_STATS with BPF

Message ID 20250414225227.3642618-1-tjmercier@google.com (mailing list archive)
Headers show
Series Replace CONFIG_DMABUF_SYSFS_STATS with BPF | expand

Message

T.J. Mercier April 14, 2025, 10:52 p.m. UTC
Until CONFIG_DMABUF_SYSFS_STATS was added [1] it was only possible to
perform per-buffer accounting with debugfs which is not suitable for
production environments. Eventually we discovered the overhead with
per-buffer sysfs file creation/removal was significantly impacting
allocation and free times, and exacerbated kernfs lock contention. [2]
dma_buf_stats_setup() is responsible for 39% of single-page buffer
creation duration, or 74% of single-page dma_buf_export() duration when
stressing dmabuf allocations and frees.

I prototyped a change from per-buffer to per-exporter statistics with a
RCU protected list of exporter allocations that accommodates most (but
not all) of our use-cases and avoids almost all of the sysfs overhead.
While that adds less overhead than per-buffer sysfs, and less even than
the maintenance of the dmabuf debugfs_list, it's still *additional*
overhead on top of the debugfs_list and doesn't give us per-buffer info.

This series uses the existing dmabuf debugfs_list to implement a BPF
dmabuf iterator, which adds no overhead to buffer allocation/free and
provides per-buffer info. While the kernel must have CONFIG_DEBUG_FS for
the dmabuf_iter to be available, debugfs does not need to be mounted.
The BPF program loaded by userspace that extracts per-buffer information
gets to define its own interface which avoids the lack of ABI stability
with debugfs (even if it were mounted).

As this is a replacement for our use of CONFIG_DMABUF_SYSFS_STATS, the
last patch is a RFC for removing it from the kernel. Please see my
suggestion there regarding the timeline for that.

[1] https://lore.kernel.org/linux-media/20201210044400.1080308-1-hridya@google.com/
[2] https://lore.kernel.org/all/20220516171315.2400578-1-tjmercier@google.com/

T.J. Mercier (4):
  dma-buf: Rename and expose debugfs symbols
  bpf: Add dmabuf iterator
  selftests/bpf: Add test for dmabuf_iter
  RFC: dma-buf: Remove DMA-BUF statistics

 .../ABI/testing/sysfs-kernel-dmabuf-buffers   |  24 ---
 Documentation/driver-api/dma-buf.rst          |   5 -
 drivers/dma-buf/Kconfig                       |  15 --
 drivers/dma-buf/Makefile                      |   1 -
 drivers/dma-buf/dma-buf-sysfs-stats.c         | 202 ------------------
 drivers/dma-buf/dma-buf-sysfs-stats.h         |  35 ---
 drivers/dma-buf/dma-buf.c                     |  40 +---
 include/linux/btf_ids.h                       |   1 +
 include/linux/dma-buf.h                       |   6 +
 kernel/bpf/Makefile                           |   3 +
 kernel/bpf/dmabuf_iter.c                      | 130 +++++++++++
 tools/testing/selftests/bpf/config            |   1 +
 .../selftests/bpf/prog_tests/dmabuf_iter.c    | 116 ++++++++++
 .../testing/selftests/bpf/progs/dmabuf_iter.c |  31 +++
 14 files changed, 299 insertions(+), 311 deletions(-)
 delete mode 100644 Documentation/ABI/testing/sysfs-kernel-dmabuf-buffers
 delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.c
 delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.h
 create mode 100644 kernel/bpf/dmabuf_iter.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/dmabuf_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/dmabuf_iter.c

Comments

Christian König April 15, 2025, 9:03 a.m. UTC | #1
Am 15.04.25 um 00:52 schrieb T.J. Mercier:
> Until CONFIG_DMABUF_SYSFS_STATS was added [1] it was only possible to
> perform per-buffer accounting with debugfs which is not suitable for
> production environments. Eventually we discovered the overhead with
> per-buffer sysfs file creation/removal was significantly impacting
> allocation and free times, and exacerbated kernfs lock contention. [2]
> dma_buf_stats_setup() is responsible for 39% of single-page buffer
> creation duration, or 74% of single-page dma_buf_export() duration when
> stressing dmabuf allocations and frees.
>
> I prototyped a change from per-buffer to per-exporter statistics with a
> RCU protected list of exporter allocations that accommodates most (but
> not all) of our use-cases and avoids almost all of the sysfs overhead.
> While that adds less overhead than per-buffer sysfs, and less even than
> the maintenance of the dmabuf debugfs_list, it's still *additional*
> overhead on top of the debugfs_list and doesn't give us per-buffer info.
>
> This series uses the existing dmabuf debugfs_list to implement a BPF
> dmabuf iterator, which adds no overhead to buffer allocation/free and
> provides per-buffer info.

Really interesting suggestion. I was expecting something like cgroups, but bpf is certainly an option as well.

How do you then use bpf to account the buffers? E.g. are you interacting with cgroups or have sysfs procedure to expose the list or how does that work?

Additional to that why using DMA-buf for accounting in the first place? See DMA-buf is for sharing buffers and only a minimal fraction of buffers usually need to get shared. Everything else is just massive overhead.

> While the kernel must have CONFIG_DEBUG_FS for
> the dmabuf_iter to be available, debugfs does not need to be mounted.
> The BPF program loaded by userspace that extracts per-buffer information
> gets to define its own interface which avoids the lack of ABI stability
> with debugfs (even if it were mounted).

I think we can make the buffer list independent of CONFIG_DEBUG_FS.

> As this is a replacement for our use of CONFIG_DMABUF_SYSFS_STATS, the
> last patch is a RFC for removing it from the kernel. Please see my
> suggestion there regarding the timeline for that.

Oh, yes please!

Regards,
Christian.

>
> [1] https://lore.kernel.org/linux-media/20201210044400.1080308-1-hridya@google.com/
> [2] https://lore.kernel.org/all/20220516171315.2400578-1-tjmercier@google.com/
>
> T.J. Mercier (4):
>   dma-buf: Rename and expose debugfs symbols
>   bpf: Add dmabuf iterator
>   selftests/bpf: Add test for dmabuf_iter
>   RFC: dma-buf: Remove DMA-BUF statistics
>
>  .../ABI/testing/sysfs-kernel-dmabuf-buffers   |  24 ---
>  Documentation/driver-api/dma-buf.rst          |   5 -
>  drivers/dma-buf/Kconfig                       |  15 --
>  drivers/dma-buf/Makefile                      |   1 -
>  drivers/dma-buf/dma-buf-sysfs-stats.c         | 202 ------------------
>  drivers/dma-buf/dma-buf-sysfs-stats.h         |  35 ---
>  drivers/dma-buf/dma-buf.c                     |  40 +---
>  include/linux/btf_ids.h                       |   1 +
>  include/linux/dma-buf.h                       |   6 +
>  kernel/bpf/Makefile                           |   3 +
>  kernel/bpf/dmabuf_iter.c                      | 130 +++++++++++
>  tools/testing/selftests/bpf/config            |   1 +
>  .../selftests/bpf/prog_tests/dmabuf_iter.c    | 116 ++++++++++
>  .../testing/selftests/bpf/progs/dmabuf_iter.c |  31 +++
>  14 files changed, 299 insertions(+), 311 deletions(-)
>  delete mode 100644 Documentation/ABI/testing/sysfs-kernel-dmabuf-buffers
>  delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.c
>  delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.h
>  create mode 100644 kernel/bpf/dmabuf_iter.c
>  create mode 100644 tools/testing/selftests/bpf/prog_tests/dmabuf_iter.c
>  create mode 100644 tools/testing/selftests/bpf/progs/dmabuf_iter.c
>
T.J. Mercier April 15, 2025, 6:13 p.m. UTC | #2
On Tue, Apr 15, 2025 at 2:03 AM Christian König
<christian.koenig@amd.com> wrote:
>
> Am 15.04.25 um 00:52 schrieb T.J. Mercier:
> > Until CONFIG_DMABUF_SYSFS_STATS was added [1] it was only possible to
> > perform per-buffer accounting with debugfs which is not suitable for
> > production environments. Eventually we discovered the overhead with
> > per-buffer sysfs file creation/removal was significantly impacting
> > allocation and free times, and exacerbated kernfs lock contention. [2]
> > dma_buf_stats_setup() is responsible for 39% of single-page buffer
> > creation duration, or 74% of single-page dma_buf_export() duration when
> > stressing dmabuf allocations and frees.
> >
> > I prototyped a change from per-buffer to per-exporter statistics with a
> > RCU protected list of exporter allocations that accommodates most (but
> > not all) of our use-cases and avoids almost all of the sysfs overhead.
> > While that adds less overhead than per-buffer sysfs, and less even than
> > the maintenance of the dmabuf debugfs_list, it's still *additional*
> > overhead on top of the debugfs_list and doesn't give us per-buffer info.
> >
> > This series uses the existing dmabuf debugfs_list to implement a BPF
> > dmabuf iterator, which adds no overhead to buffer allocation/free and
> > provides per-buffer info.
>
> Really interesting suggestion. I was expecting something like cgroups, but bpf is certainly an option as well.
>
> How do you then use bpf to account the buffers? E.g. are you interacting with cgroups or have sysfs procedure to expose the list or how does that work?

Where currently we read through all of /sys/kernel/dmabuf/buffers/,
with this we can load or pin a bpf program (like
tools/testing/selftests/bpf/progs/dmabuf_iter.c) and then just cat
(and parse) /sys/fs/bpf/dmabufs to get all per-buffer info one go.

The attribution of buffers to processes is currently done by looking
through procfs for fd and map references to dmabufs. That part is
still slow, and provides no limitation on who can allocate how much,
so I think cgroups is still the main potential tool for that. We have
a program that does all the scanning work which is called on-demand
for some use cases, and also manually by users:
https://cs.android.com/android/platform/superproject/main/+/main:system/memory/libmeminfo/libdmabufinfo/tools/dmabuf_dump.cpp

The per-buffer information is used for accounting kernel-only buffers
that don't show up in procfs, and for partially mapped buffers without
fd references where the total buffer size isn't otherwise known. Also
sometimes (manual debugging or bugreports) it's useful just to know
how much memory in total is tied up in dmabufs regardless of who
allocated it because it can be gigabytes due to bugs or crazy program
behaviors; the per buffer info is a faster way to get that then
reading through all of procfs even if you assume everything is
viewable in procfs.

> Additional to that why using DMA-buf for accounting in the first place? See DMA-buf is for sharing buffers and only a minimal fraction of buffers usually need to get shared. Everything else is just massive overhead.

Well we need some way to account all DMA-buf memory because it
consumes a significant portion of total device memory. Even more so
lately where they're used to store >1G AI models for execution on
accelerator hardware. I've attached an example of dmabuf_dump output
to give you an idea of how many buffers we're talking about, and most
of those are (or will be, when an app goes to foreground) shared among
multiple processes and/or drivers.

> > While the kernel must have CONFIG_DEBUG_FS for
> > the dmabuf_iter to be available, debugfs does not need to be mounted.
> > The BPF program loaded by userspace that extracts per-buffer information
> > gets to define its own interface which avoids the lack of ABI stability
> > with debugfs (even if it were mounted).
>
> I think we can make the buffer list independent of CONFIG_DEBUG_FS.

This would be nice. It's a fairly small overhead, and we can make it
less with RCU too. (__dma_buf_debugfs_list_add.png)

> > As this is a replacement for our use of CONFIG_DMABUF_SYSFS_STATS, the
> > last patch is a RFC for removing it from the kernel. Please see my
> > suggestion there regarding the timeline for that.
>
> Oh, yes please!

I thought you might be happy about this. :)

> Regards,
> Christian.
>
> >
> > [1] https://lore.kernel.org/linux-media/20201210044400.1080308-1-hridya@google.com/
> > [2] https://lore.kernel.org/all/20220516171315.2400578-1-tjmercier@google.com/
> >
> > T.J. Mercier (4):
> >   dma-buf: Rename and expose debugfs symbols
> >   bpf: Add dmabuf iterator
> >   selftests/bpf: Add test for dmabuf_iter
> >   RFC: dma-buf: Remove DMA-BUF statistics
> >
> >  .../ABI/testing/sysfs-kernel-dmabuf-buffers   |  24 ---
> >  Documentation/driver-api/dma-buf.rst          |   5 -
> >  drivers/dma-buf/Kconfig                       |  15 --
> >  drivers/dma-buf/Makefile                      |   1 -
> >  drivers/dma-buf/dma-buf-sysfs-stats.c         | 202 ------------------
> >  drivers/dma-buf/dma-buf-sysfs-stats.h         |  35 ---
> >  drivers/dma-buf/dma-buf.c                     |  40 +---
> >  include/linux/btf_ids.h                       |   1 +
> >  include/linux/dma-buf.h                       |   6 +
> >  kernel/bpf/Makefile                           |   3 +
> >  kernel/bpf/dmabuf_iter.c                      | 130 +++++++++++
> >  tools/testing/selftests/bpf/config            |   1 +
> >  .../selftests/bpf/prog_tests/dmabuf_iter.c    | 116 ++++++++++
> >  .../testing/selftests/bpf/progs/dmabuf_iter.c |  31 +++
> >  14 files changed, 299 insertions(+), 311 deletions(-)
> >  delete mode 100644 Documentation/ABI/testing/sysfs-kernel-dmabuf-buffers
> >  delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.c
> >  delete mode 100644 drivers/dma-buf/dma-buf-sysfs-stats.h
> >  create mode 100644 kernel/bpf/dmabuf_iter.c
> >  create mode 100644 tools/testing/selftests/bpf/prog_tests/dmabuf_iter.c
> >  create mode 100644 tools/testing/selftests/bpf/progs/dmabuf_iter.c
> >
>
raven:/ # dmabuf_dump                                                                                                                                                                                                                                
 id.GoogleCamera:27290
                  Name              Rss              Pss         nr_procs            Inode               Exporter
                system          1024 kB          1024 kB                1              435                  system
                system             4 kB             4 kB                1              436                  system
                system             4 kB             4 kB                1              437                  system
         PROCESS TOTAL          1032 kB          1032 kB                 
----------------------
 putmethod.latin:4860 
                  Name              Rss              Pss         nr_procs            Inode               Exporter
AHardwareBuffer pid [4860]            52 kB            52 kB                1              539         system-uncached
                system             8 kB             8 kB                1              540                  system
AHardwareBuffer pid [4860]            52 kB            52 kB                1              546         system-uncached
                system             8 kB             8 kB                1              547                  system
         PROCESS TOTAL           120 kB           120 kB                 
----------------------
 android.hardwar:940  
                  Name              Rss              Pss         nr_procs            Inode               Exporter
             <unknown>            32 kB            32 kB                1              277                  system
             <unknown>         16384 kB         16384 kB                1              278                  system
         PROCESS TOTAL         16416 kB         16416 kB                 
----------------------
    binder:892_2:892  
                  Name              Rss              Pss         nr_procs            Inode               Exporter
                system          1024 kB          1024 kB                1               33                  system
                system             4 kB             4 kB                1               34                  system
         PROCESS TOTAL          1028 kB          1028 kB                 
----------------------
 s.nexuslauncher:2316 
                  Name              Rss              Pss         nr_procs            Inode               Exporter
ImageReader-64x64f1u2816m2-2316            20 kB            20 kB                1              183         system-uncached
                system             8 kB             8 kB                1              184                  system
ImageReader-60x60f1u2816m2-2316            20 kB            20 kB                1              185         system-uncached
                system             8 kB             8 kB                1              186                  system
ImageReader-2x2f1u2816m2-2316-2             4 kB             4 kB                1              187         system-uncached
                system             8 kB             8 kB                1              188                  system
ImageReader-231x231f1u2816m2-23           232 kB           232 kB                1              189         system-uncached
                system             8 kB             8 kB                1              190                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              191         system-uncached
                system             8 kB             8 kB                1              192                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              195         system-uncached
                system             8 kB             8 kB                1              196                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              197         system-uncached
                system             8 kB             8 kB                1              198                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              199         system-uncached
                system             8 kB             8 kB                1              200                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              201         system-uncached
                system             8 kB             8 kB                1              202                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              203         system-uncached
                system             8 kB             8 kB                1              204                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              205         system-uncached
                system             8 kB             8 kB                1              206                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              207         system-uncached
                system             8 kB             8 kB                1              208                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              209         system-uncached
                system             8 kB             8 kB                1              210                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              211         system-uncached
                system             8 kB             4 kB                2              212                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           184 kB                3              213         system-uncached
                system             8 kB             2 kB                3              214                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              215         system-uncached
                system             8 kB             4 kB                2              216                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              217         system-uncached
                system             8 kB             4 kB                2              218                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              219         system-uncached
                system             8 kB             4 kB                2              220                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              221         system-uncached
                system             8 kB             8 kB                1              222                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              225         system-uncached
                system             8 kB             8 kB                1              226                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              227         system-uncached
                system             8 kB             8 kB                1              228                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              229         system-uncached
                system             8 kB             8 kB                1              230                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              231         system-uncached
                system             8 kB             8 kB                1              232                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              233         system-uncached
                system             8 kB             8 kB                1              234                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              235         system-uncached
                system             8 kB             8 kB                1              236                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              237         system-uncached
                system             8 kB             8 kB                1              238                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              239         system-uncached
                system             8 kB             8 kB                1              240                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              243         system-uncached
                system             8 kB             8 kB                1              244                  system
ImageReader-231x231f1u2816m2-23           232 kB           232 kB                1              245         system-uncached
                system             8 kB             8 kB                1              246                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              247         system-uncached
                system             8 kB             8 kB                1              248                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              249         system-uncached
                system             8 kB             8 kB                1              250                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              258         system-uncached
                system             8 kB             8 kB                1              259                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              415         system-uncached
                system             8 kB             8 kB                1              416                  system
AHardwareBuffer pid [2316]           228 kB           228 kB                1              442         system-uncached
                system             8 kB             8 kB                1              443                  system
AHardwareBuffer pid [1394]          4412 kB          4412 kB                1              510         system-uncached
                system             8 kB             8 kB                1              511                  system
AHardwareBuffer pid [1394]          4412 kB          4412 kB                1              513         system-uncached
                system             8 kB             8 kB                1              514                  system
            screenshot         11236 kB          5618 kB                2              634                  system
                system             8 kB             4 kB                2              635                  system
         PROCESS TOTAL         29096 kB         21980 kB                 
----------------------
 android.hardwar:1155 
                  Name              Rss              Pss         nr_procs            Inode               Exporter
             <unknown>           404 kB           404 kB                1               35                 system

         PROCESS TOTAL           404 kB           404 kB                 
----------------------
 camera.provider:880  
                  Name              Rss              Pss         nr_procs            Inode               Exporter
                system          1024 kB          1024 kB                1               36                  system
                system             4 kB             4 kB                1               37                  system
                system             4 kB             4 kB                1               38                  system
                system             4 kB             4 kB                1               40                  system
                system           576 kB           576 kB                1               41                  system
                system             8 kB             8 kB                1               42                  system
                system           576 kB           576 kB                1               43                  system
                system             8 kB             8 kB                1               44                  system
         PROCESS TOTAL          2204 kB          2204 kB                 
----------------------
  surfaceflinger:550  
                  Name              Rss              Pss         nr_procs            Inode               Exporter
         unnamed-550-0         17828 kB          8914 kB                2                2         system-uncached
                system             8 kB             4 kB                2                3                  system
         unnamed-550-0         17828 kB          8914 kB                2                4         system-uncached
                system             8 kB             4 kB                2                5                  system
         unnamed-550-0         17828 kB          8914 kB                2                6         system-uncached
                system             8 kB             4 kB                2                7                  system
               Planner         17828 kB         17828 kB                1                8         system-uncached
                system             8 kB             8 kB                1                9                  system
               Planner         17828 kB          8914 kB                2               10         system-uncached
                system             8 kB             4 kB                2               11                  system
               Planner         17828 kB          8914 kB                2               12         system-uncached
                system             8 kB             4 kB                2               13                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              101         system-uncached
                system             8 kB             2 kB                3              102                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              103         system-uncached
                system             8 kB             2 kB                3              104                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              105         system-uncached
                system             8 kB             2 kB                3              106                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              107         system-uncached
                system             8 kB             2 kB                3              108                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              109         system-uncached
                system             8 kB             2 kB                3              110                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              116         system-uncached
                system             8 kB             2 kB                3              117                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              118         system-uncached
                system             8 kB             2 kB                3              119                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              120         system-uncached
                system             8 kB             2 kB                3              121                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              122         system-uncached
                system             8 kB             2 kB                3              123                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              125         system-uncached
                system             8 kB             2 kB                3              126                  system
Wallpaper#1(BLAST Consumer)1         17828 kB          5942 kB                3              129         system-uncached
                system             8 kB             2 kB                3              130                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              161         system-uncached
                system             8 kB             2 kB                3              162                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              163         system-uncached
                system             8 kB             2 kB                3              164                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              165         system-uncached
                system             8 kB             2 kB                3              166                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              167         system-uncached
                system             8 kB             2 kB                3              168                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              169         system-uncached
                system             8 kB             2 kB                3              170                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              211         system-uncached
                system             8 kB             4 kB                2              212                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           184 kB                3              213         system-uncached
                system             8 kB             2 kB                3              214                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              215         system-uncached
                system             8 kB             4 kB                2              216                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              217         system-uncached
                system             8 kB             4 kB                2              218                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           276 kB                2              219         system-uncached
                system             8 kB             4 kB                2              220                  system
  RegionSamplingThread            28 kB            28 kB                1              260                  system
                system             8 kB             8 kB                1              261                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              636         system-uncached
                system             8 kB             2 kB                3              637                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              638         system-uncached
                system             8 kB             2 kB                3              639                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              640         system-uncached
                system             8 kB             2 kB                3              641                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              642         system-uncached
                system             8 kB             2 kB                3              643                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              644         system-uncached
                system             8 kB             2 kB                3              645                  system
ColorFade#6(BLAST Consumer)6         17828 kB          8914 kB                2              648         system-uncached
                system             8 kB             4 kB                2              649                  system
ColorFade#6(BLAST Consumer)6         17828 kB          5942 kB                3              650         system-uncached
                system             8 kB             2 kB                3              651                  system
         PROCESS TOTAL        267320 kB        119230 kB                 
----------------------
 ndroid.systemui:1991 
                  Name              Rss              Pss         nr_procs            Inode               Exporter
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              101         system-uncached
                system             8 kB             2 kB                3              102                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              103         system-uncached
                system             8 kB             2 kB                3              104                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              105         system-uncached
                system             8 kB             2 kB                3              106                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              107         system-uncached
                system             8 kB             2 kB                3              108                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              109         system-uncached
                system             8 kB             2 kB                3              110                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              116         system-uncached
                system             8 kB             2 kB                3              117                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              118         system-uncached
                system             8 kB             2 kB                3              119                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              120         system-uncached
                system             8 kB             2 kB                3              121                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              122         system-uncached
                system             8 kB             2 kB                3              123                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              125         system-uncached
                system             8 kB             2 kB                3              126                  system
Wallpaper#1(BLAST Consumer)1         17828 kB          5942 kB                3              129         system-uncached
                system             8 kB             2 kB                3              130                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              161         system-uncached
                system             8 kB             2 kB                3              162                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              163         system-uncached
                system             8 kB             2 kB                3              164                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              165         system-uncached
                system             8 kB             2 kB                3              166                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              167         system-uncached
                system             8 kB             2 kB                3              168                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              169         system-uncached
                system             8 kB             2 kB                3              170                  system
ImageReader-168x168f1u2816m2-19           124 kB           124 kB                1              293         system-uncached
                system             8 kB             8 kB                1              294                  system
ImageReader-168x168f1u2816m2-19           124 kB           124 kB                1              297         system-uncached
                system             8 kB             8 kB                1              298                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              636         system-uncached
                system             8 kB             2 kB                3              637                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              638         system-uncached
                system             8 kB             2 kB                3              639                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              640         system-uncached
                system             8 kB             2 kB                3              641                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              642         system-uncached
                system             8 kB             2 kB                3              643                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              644         system-uncached
                system             8 kB             2 kB                3              645                  system
         PROCESS TOTAL        122060 kB         40862 kB                 
----------------------
   system_server:1394 
                  Name              Rss              Pss         nr_procs            Inode               Exporter
            screenshot         11236 kB          5618 kB                2              634                  system
                system             8 kB             4 kB                2              635                  system
ColorFade#6(BLAST Consumer)6         17828 kB          5942 kB                3              650         system-uncached
                system             8 kB             2 kB                3              651                  system
         PROCESS TOTAL         29080 kB         11567 kB                 
----------------------
    binder:554_2:554  
                  Name              Rss              Pss         nr_procs            Inode               Exporter
         unnamed-550-0         17828 kB          8914 kB                2                2         system-uncached
                system             8 kB             4 kB                2                3                  system
         unnamed-550-0         17828 kB          8914 kB                2                4         system-uncached
                system             8 kB             4 kB                2                5                  system
         unnamed-550-0         17828 kB          8914 kB                2                6         system-uncached
                system             8 kB             4 kB                2                7                  system
               Planner         17828 kB          8914 kB                2               10         system-uncached
                system             8 kB             4 kB                2               11                  system
               Planner         17828 kB          8914 kB                2               12         system-uncached
                system             8 kB             4 kB                2               13                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              101         system-uncached
                system             8 kB             2 kB                3              102                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              103         system-uncached
                system             8 kB             2 kB                3              104                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              105         system-uncached
                system             8 kB             2 kB                3              106                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              107         system-uncached
                system             8 kB             2 kB                3              108                  system
VRI[ScreenDecorOverlay]#0(BLAST          1008 kB           336 kB                3              109         system-uncached
                system             8 kB             2 kB                3              110                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              116         system-uncached
                system             8 kB             2 kB                3              117                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              118         system-uncached
                system             8 kB             2 kB                3              119                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              120         system-uncached
                system             8 kB             2 kB                3              121                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              122         system-uncached
                system             8 kB             2 kB                3              123                  system
VRI[ScreenDecorOverlayBottom]#2          1008 kB           336 kB                3              125         system-uncached
                system             8 kB             2 kB                3              126                  system
Wallpaper#1(BLAST Consumer)1         17828 kB          5942 kB                3              129         system-uncached
                system             8 kB             2 kB                3              130                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              161         system-uncached
                system             8 kB             2 kB                3              162                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              163         system-uncached
                system             8 kB             2 kB                3              164                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              165         system-uncached
                system             8 kB             2 kB                3              166                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              167         system-uncached
                system             8 kB             2 kB                3              168                  system
VRI[StatusBar]#4(BLAST Consumer           916 kB           305 kB                3              169         system-uncached
                system             8 kB             2 kB                3              170                  system
VRI[Taskbar]#2(BLAST Consumer)2           552 kB           184 kB                3              213         system-uncached
                system             8 kB             2 kB                3              214                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              636         system-uncached
                system             8 kB             2 kB                3              637                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              638         system-uncached
                system             8 kB             2 kB                3              639                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              640         system-uncached
                system             8 kB             2 kB                3              641                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              642         system-uncached
                system             8 kB             2 kB                3              643                  system
VRI[NotificationShade]#10(BLAST         17828 kB          5942 kB                3              644         system-uncached
                system             8 kB             2 kB                3              645                  system
ColorFade#6(BLAST Consumer)6         17828 kB          8914 kB                2              648         system-uncached
                system             8 kB             4 kB                2              649                  system
ColorFade#6(BLAST Consumer)6         17828 kB          5942 kB                3              650         system-uncached
                system             8 kB             2 kB                3              651                  system
ColorFade#6(BLAST Consumer)6         17828 kB         17828 kB                1              652         system-uncached
                system             8 kB             8 kB                1              653                  system
ColorFade#6(BLAST Consumer)6         17828 kB         17828 kB                1              654         system-uncached
                system             8 kB             8 kB                1              655                  system
ColorFade#6(BLAST Consumer)6         17828 kB         17828 kB                1              656         system-uncached
                system             8 kB             8 kB                1              657                  system
         PROCESS TOTAL        300716 kB        153746 kB                 
----------------------
dmabuf total: 369616 kB kernel_rss: 1024 kB userspace_rss: 769476 kB userspace_pss: 368591 kB