diff mbox series

mm: Avoid swapping in interrupt context

Message ID 1538387115-2363-1-git-send-email-amhetre@nvidia.com (mailing list archive)
State New, archived
Headers show
Series mm: Avoid swapping in interrupt context | expand

Commit Message

Ashish Mhetre Oct. 1, 2018, 9:45 a.m. UTC
From: Sri Krishna chowdary <schowdary@nvidia.com>

Pages can be swapped out from interrupt context as well.
ZRAM uses zsmalloc allocator to make room for these pages.
But zsmalloc is not made to be used from interrupt context.
This can result in a kernel Oops.

Signed-off-by: Sri Krishna chowdary <schowdary@nvidia.com>
Signed-off-by: Ashish Mhetre <amhetre@nvidia.com>
---
 mm/vmscan.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

Comments

Michal Hocko Oct. 1, 2018, 12:24 p.m. UTC | #1
On Mon 01-10-18 15:15:15, Ashish Mhetre wrote:
> From: Sri Krishna chowdary <schowdary@nvidia.com>
> 
> Pages can be swapped out from interrupt context as well.

How? No allocation request from the interrupt context can use a
sleepable allocation context and that means that no reclaim is allowed
from the IRQ context.

> ZRAM uses zsmalloc allocator to make room for these pages.
> But zsmalloc is not made to be used from interrupt context.
> This can result in a kernel Oops.

Could you provide the Oops message?
Sergey Senozhatsky Oct. 2, 2018, 7:54 a.m. UTC | #2
On (10/01/18 15:15), Ashish Mhetre wrote:
> From: Sri Krishna chowdary <schowdary@nvidia.com>
> 
> Pages can be swapped out from interrupt context as well.

Well, if you do kmalloc(GFP_KERNEL) from IRQ then that's the bug
you need to fix in the first place.

> ZRAM uses zsmalloc allocator to make room for these pages.
> 
> But zsmalloc is not made to be used from interrupt context.
> This can result in a kernel Oops.

Most like not just "can" but "will" result in panic(). We have
BUG_ON(in_interrupt()) in zsmalloc.

	-ss
Ashish Mhetre Oct. 3, 2018, 10:48 a.m. UTC | #3
>How? No allocation request from the interrupt context can use a
>sleepable allocation context and that means that no reclaim is allowed
>from the IRQ context.
Kernel Oops happened when ZRAM was used as swap with zsmalloc as alloctor
under memory pressure condition.
This is probably because of kmalloc() from IRQ as pointed out by Sergey.

>Could you provide the Oops message?
BUG_ON() got triggered at https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324 with Oops message:
[ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
[ 264.088350] Modules linked in:
[ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W 
3.10.33-g990282b #1
[ 264.099572] Workqueue: events netstat_work_func
[ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
[ 264.109485] PC is at zs_map_object+0x180/0x18c
[ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
[ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
[ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
[ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
[ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
[ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000


On Monday 01 October 2018 05:54 PM, Michal Hocko wrote:
> On Mon 01-10-18 15:15:15, Ashish Mhetre wrote:
>> From: Sri Krishna chowdary <schowdary@nvidia.com>
>>
>> Pages can be swapped out from interrupt context as well.
> How? No allocation request from the interrupt context can use a
> sleepable allocation context and that means that no reclaim is allowed
> from the IRQ context.
>
>> ZRAM uses zsmalloc allocator to make room for these pages.
>> But zsmalloc is not made to be used from interrupt context.
>> This can result in a kernel Oops.
> Could you provide the Oops message?
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <pre wrap="">&gt;How? No allocation request from the interrupt context can use a
&gt;sleepable allocation context and that means that no reclaim is allowed
&gt;from the IRQ context.
Kernel Oops happened when ZRAM was used as swap with zsmalloc as alloctor
under memory pressure condition.
This is probably because of kmalloc() from IRQ as pointed out by Sergey.

&gt;Could you provide the Oops message?
BUG_ON() got triggered at <a class="moz-txt-link-freetext" href="https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324">https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324</a> with Oops message:
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.088350] Modules linked in:<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W 3.10.33-g990282b #1<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.099572] Workqueue: events netstat_work_func<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.109485] PC is at zs_map_object+0x180/0x18c<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.118956] pc : [&lt;c01581e8&gt;] lr : [&lt;c0456618&gt;] psr: 200f0013<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000<span> 
</span></span></pre>
    <br>
    <div class="moz-cite-prefix">On Monday 01 October 2018 05:54 PM,
      Michal Hocko wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20181001122400.GF18290@dhcp22.suse.cz">
      <pre wrap="">On Mon 01-10-18 15:15:15, Ashish Mhetre wrote:
</pre>
      <blockquote type="cite">
        <pre wrap="">From: Sri Krishna chowdary <a class="moz-txt-link-rfc2396E" href="mailto:schowdary@nvidia.com">&lt;schowdary@nvidia.com&gt;</a>

Pages can be swapped out from interrupt context as well.
</pre>
      </blockquote>
      <pre wrap="">
How? No allocation request from the interrupt context can use a
sleepable allocation context and that means that no reclaim is allowed
from the IRQ context.

</pre>
      <blockquote type="cite">
        <pre wrap="">ZRAM uses zsmalloc allocator to make room for these pages.
But zsmalloc is not made to be used from interrupt context.
This can result in a kernel Oops.
</pre>
      </blockquote>
      <pre wrap="">
Could you provide the Oops message?
</pre>
    </blockquote>
    <br>
  </body>
</html>
Michal Hocko Oct. 3, 2018, 11:01 a.m. UTC | #4
On Wed 03-10-18 16:18:37, Ashish Mhetre wrote:
> > How? No allocation request from the interrupt context can use a
> > sleepable allocation context and that means that no reclaim is allowed
> > from the IRQ context.
> Kernel Oops happened when ZRAM was used as swap with zsmalloc as alloctor
> under memory pressure condition.
> This is probably because of kmalloc() from IRQ as pointed out by Sergey.

Yes most likely and that should be fixed.
 
> > Could you provide the Oops message?
> BUG_ON() got triggered at https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324 with Oops message:
> [ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
> [ 264.088350] Modules linked in:
> [ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W
> 3.10.33-g990282b #1
> [ 264.099572] Workqueue: events netstat_work_func
> [ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
> [ 264.109485] PC is at zs_map_object+0x180/0x18c
> [ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
> [ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
> [ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
> [ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
> [ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
> [ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000

This doesn't show the backtrace part which contains the allocation
AFAICS.
Ashish Mhetre Oct. 3, 2018, 11:50 a.m. UTC | #5
>This doesn't show the backtrace part which contains the allocation
AFAICS.

My bad. Here is a complete dump:
[ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
[ 264.088350] Modules linked in:
[ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W 
3.10.33-g990282b #1
[ 264.099572] Workqueue: events netstat_work_func
[ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
[ 264.109485] PC is at zs_map_object+0x180/0x18c
[ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
[ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
[ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
[ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
[ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
[ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000

--------
[ 265.772426] [<c01581e8>] (zs_map_object+0x180/0x18c) from [<c0456618>] 
(zram_bvec_rw.isra.15+0x304/0x88c)
[ 265.781973] [<c0456618>] (zram_bvec_rw.isra.15+0x304/0x88c) from 
[<c0456d78>] (zram_make_request+0x1d8/0x378)
[ 265.791868] [<c0456d78>] (zram_make_request+0x1d8/0x378) from 
[<c02c7afc>] (generic_make_request+0xb0/0xdc)
[ 265.801588] [<c02c7afc>] (generic_make_request+0xb0/0xdc) from 
[<c02c7bb0>] (submit_bio+0x88/0x140)
[ 265.810617] [<c02c7bb0>] (submit_bio+0x88/0x140) from [<c01459d4>] 
(__swap_writepage+0x198/0x230)
[ 265.819471] [<c01459d4>] (__swap_writepage+0x198/0x230) from 
[<c011fc50>] (shrink_page_list+0x4e0/0x974)
[ 265.828930] [<c011fc50>] (shrink_page_list+0x4e0/0x974) from 
[<c0120644>] (shrink_inactive_list+0x150/0x3c8)
[ 265.838736] [<c0120644>] (shrink_inactive_list+0x150/0x3c8) from 
[<c0120de8>] (shrink_lruvec+0x20c/0x448)
[ 265.848282] [<c0120de8>] (shrink_lruvec+0x20c/0x448) from [<c012109c>] 
(shrink_zone+0x78/0x188)
[ 265.856960] [<c012109c>] (shrink_zone+0x78/0x188) from [<c01212ac>] 
(do_try_to_free_pages+0x100/0x544)
[ 265.866246] [<c01212ac>] (do_try_to_free_pages+0x100/0x544) from 
[<c0121928>] (try_to_free_pages+0x238/0x428)
[ 265.876140] [<c0121928>] (try_to_free_pages+0x238/0x428) from 
[<c01179cc>] (__alloc_pages_nodemask+0x5b0/0x90c)
[ 265.886207] [<c01179cc>] (__alloc_pages_nodemask+0x5b0/0x90c) from 
[<c0117d44>] (__get_free_pages+0x1c/0x34)
[ 265.896014] [<c0117d44>] (__get_free_pages+0x1c/0x34) from 
[<c0844a4c>] (tcp4_seq_show+0x248/0x4b4)
[ 265.905042] [<c0844a4c>] (tcp4_seq_show+0x248/0x4b4) from [<c017a844>] 
(seq_read+0x1e4/0x484)
[ 265.913550] [<c017a844>] (seq_read+0x1e4/0x484) from [<c01a84f0>] 
(proc_reg_read+0x60/0x88)
[ 265.921884] [<c01a84f0>] (proc_reg_read+0x60/0x88) from [<c015aa44>] 
(vfs_read+0xa0/0x14c)
[ 265.930129] [<c015aa44>] (vfs_read+0xa0/0x14c) from [<c015b0c4>] 
(SyS_read+0x44/0x80)
[ 265.937942] [<c015b0c4>] (SyS_read+0x44/0x80) from [<c052e98c>] 
(netstat_work_func+0x54/0xec)
[ 265.946450] [<c052e98c>] (netstat_work_func+0x54/0xec) from 
[<c0086700>] (process_one_work+0x13c/0x454)
[ 265.955823] [<c0086700>] (process_one_work+0x13c/0x454) from 
[<c008745c>] (worker_thread+0x140/0x3dc)
265.965022] [<c008745c>] (worker_thread+0x140/0x3dc) from [<c008cf4c>] 
(kthread+0xe0/0xe4)
[ 265.973269] [<c008cf4c>] (kthread+0xe0/0xe4) from [<c000ef98>] 
(ret_from_fork+0x14/0x20)
[ 264.148640] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
[ 264.155930] Control: 30c5387d Table: aaf7c000 DAC: fffffffd

On Wednesday 03 October 2018 04:31 PM, Michal Hocko wrote:
> On Wed 03-10-18 16:18:37, Ashish Mhetre wrote:
>>> How? No allocation request from the interrupt context can use a
>>> sleepable allocation context and that means that no reclaim is allowed
>>> from the IRQ context.
>> Kernel Oops happened when ZRAM was used as swap with zsmalloc as alloctor
>> under memory pressure condition.
>> This is probably because of kmalloc() from IRQ as pointed out by Sergey.
> Yes most likely and that should be fixed.
>   
>>> Could you provide the Oops message?
>> BUG_ON() got triggered at https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324 with Oops message:
>> [ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>> [ 264.088350] Modules linked in:
>> [ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W
>> 3.10.33-g990282b #1
>> [ 264.099572] Workqueue: events netstat_work_func
>> [ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
>> [ 264.109485] PC is at zs_map_object+0x180/0x18c
>> [ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
>> [ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
>> [ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
>> [ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
>> [ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
>> [ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000
> This doesn't show the backtrace part which contains the allocation
> AFAICS.
<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <pre wrap="">&gt;This doesn't show the backtrace part which contains the allocation
AFAICS.

My bad. Here is a complete dump:
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.088350] Modules linked in:<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W 3.10.33-g990282b #1<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.099572] Workqueue: events netstat_work_func<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.109485] PC is at zs_map_object+0x180/0x18c<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.118956] pc : [&lt;c01581e8&gt;] lr : [&lt;c0456618&gt;] psr: 200f0013<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000<span> </span></span>

<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">--------<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.772426] [&lt;c01581e8&gt;] (zs_map_object+0x180/0x18c) from [&lt;c0456618&gt;] (zram_bvec_rw.isra.15+0x304/0x88c)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.781973] [&lt;c0456618&gt;] (zram_bvec_rw.isra.15+0x304/0x88c) from [&lt;c0456d78&gt;] (zram_make_request+0x1d8/0x378)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.791868] [&lt;c0456d78&gt;] (zram_make_request+0x1d8/0x378) from [&lt;c02c7afc&gt;] (generic_make_request+0xb0/0xdc)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.801588] [&lt;c02c7afc&gt;] (generic_make_request+0xb0/0xdc) from [&lt;c02c7bb0&gt;] (submit_bio+0x88/0x140)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.810617] [&lt;c02c7bb0&gt;] (submit_bio+0x88/0x140) from [&lt;c01459d4&gt;] (__swap_writepage+0x198/0x230)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.819471] [&lt;c01459d4&gt;] (__swap_writepage+0x198/0x230) from [&lt;c011fc50&gt;] (shrink_page_list+0x4e0/0x974)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.828930] [&lt;c011fc50&gt;] (shrink_page_list+0x4e0/0x974) from [&lt;c0120644&gt;] (shrink_inactive_list+0x150/0x3c8)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.838736] [&lt;c0120644&gt;] (shrink_inactive_list+0x150/0x3c8) from [&lt;c0120de8&gt;] (shrink_lruvec+0x20c/0x448)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.848282] [&lt;c0120de8&gt;] (shrink_lruvec+0x20c/0x448) from [&lt;c012109c&gt;] (shrink_zone+0x78/0x188)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.856960] [&lt;c012109c&gt;] (shrink_zone+0x78/0x188) from [&lt;c01212ac&gt;] (do_try_to_free_pages+0x100/0x544)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.866246] [&lt;c01212ac&gt;] (do_try_to_free_pages+0x100/0x544) from [&lt;c0121928&gt;] (try_to_free_pages+0x238/0x428)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.876140] [&lt;c0121928&gt;] (try_to_free_pages+0x238/0x428) from [&lt;c01179cc&gt;] (__alloc_pages_nodemask+0x5b0/0x90c)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.886207] [&lt;c01179cc&gt;] (__alloc_pages_nodemask+0x5b0/0x90c) from [&lt;c0117d44&gt;] (__get_free_pages+0x1c/0x34)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.896014] [&lt;c0117d44&gt;] (__get_free_pages+0x1c/0x34) from [&lt;c0844a4c&gt;] (tcp4_seq_show+0x248/0x4b4)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.905042] [&lt;c0844a4c&gt;] (tcp4_seq_show+0x248/0x4b4) from [&lt;c017a844&gt;] (seq_read+0x1e4/0x484)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.913550] [&lt;c017a844&gt;] (seq_read+0x1e4/0x484) from [&lt;c01a84f0&gt;] (proc_reg_read+0x60/0x88)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.921884] [&lt;c01a84f0&gt;] (proc_reg_read+0x60/0x88) from [&lt;c015aa44&gt;] (vfs_read+0xa0/0x14c)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.930129] [&lt;c015aa44&gt;] (vfs_read+0xa0/0x14c) from [&lt;c015b0c4&gt;] (SyS_read+0x44/0x80)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.937942] [&lt;c015b0c4&gt;] (SyS_read+0x44/0x80) from [&lt;c052e98c&gt;] (netstat_work_func+0x54/0xec)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.946450] [&lt;c052e98c&gt;] (netstat_work_func+0x54/0xec) from [&lt;c0086700&gt;] (process_one_work+0x13c/0x454)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.955823] [&lt;c0086700&gt;] (process_one_work+0x13c/0x454) from [&lt;c008745c&gt;] (worker_thread+0x140/0x3dc)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">265.965022] [&lt;c008745c&gt;] (worker_thread+0x140/0x3dc) from [&lt;c008cf4c&gt;] (kthread+0xe0/0xe4)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 265.973269] [&lt;c008cf4c&gt;] (kthread+0xe0/0xe4) from [&lt;c000ef98&gt;] (ret_from_fork+0x14/0x20)<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.148640] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel<span> </span></span>
<span style="color: rgb(0, 0, 0); font-family: Verdana, Arial, Helvetica, sans-serif; font-size: 13px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; background-color: rgb(255, 255, 255); text-decoration-style: initial; text-decoration-color: initial; display: inline !important; float: none;">[ 264.155930] Control: 30c5387d Table: aaf7c000 DAC: fffffffd<span> 
</span></span></pre>
    <div class="moz-cite-prefix">On Wednesday 03 October 2018 04:31 PM,
      Michal Hocko wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:20181003110146.GB4714@dhcp22.suse.cz">
      <pre wrap="">On Wed 03-10-18 16:18:37, Ashish Mhetre wrote:
</pre>
      <blockquote type="cite">
        <blockquote type="cite">
          <pre wrap="">How? No allocation request from the interrupt context can use a
sleepable allocation context and that means that no reclaim is allowed
from the IRQ context.
</pre>
        </blockquote>
        <pre wrap="">Kernel Oops happened when ZRAM was used as swap with zsmalloc as alloctor
under memory pressure condition.
This is probably because of kmalloc() from IRQ as pointed out by Sergey.
</pre>
      </blockquote>
      <pre wrap="">
Yes most likely and that should be fixed.
 
</pre>
      <blockquote type="cite">
        <blockquote type="cite">
          <pre wrap="">Could you provide the Oops message?
</pre>
        </blockquote>
        <pre wrap="">BUG_ON() got triggered at <a class="moz-txt-link-freetext" href="https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324">https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/mm/zsmalloc.c?h=next-20181002#n1324</a> with Oops message:
[ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
[ 264.088350] Modules linked in:
[ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W
3.10.33-g990282b #1
[ 264.099572] Workqueue: events netstat_work_func
[ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
[ 264.109485] PC is at zs_map_object+0x180/0x18c
[ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
[ 264.118956] pc : [&lt;c01581e8&gt;] lr : [&lt;c0456618&gt;] psr: 200f0013
[ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
[ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
[ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
[ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000
</pre>
      </blockquote>
      <pre wrap="">
This doesn't show the backtrace part which contains the allocation
AFAICS.
</pre>
    </blockquote>
    <br>
  </body>
</html>
Michal Hocko Oct. 3, 2018, 11:53 a.m. UTC | #6
On Wed 03-10-18 17:20:15, Ashish Mhetre wrote:
> > This doesn't show the backtrace part which contains the allocation
> AFAICS.
> 
> My bad. Here is a complete dump:
> [ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
> [ 264.088350] Modules linked in:
> [ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W
> 3.10.33-g990282b #1
> [ 264.099572] Workqueue: events netstat_work_func
> [ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
> [ 264.109485] PC is at zs_map_object+0x180/0x18c
> [ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
> [ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
> [ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
> [ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
> [ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
> [ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000
> 
> --------
> [ 265.772426] [<c01581e8>] (zs_map_object+0x180/0x18c) from [<c0456618>]
> (zram_bvec_rw.isra.15+0x304/0x88c)
> [ 265.781973] [<c0456618>] (zram_bvec_rw.isra.15+0x304/0x88c) from
> [<c0456d78>] (zram_make_request+0x1d8/0x378)
> [ 265.791868] [<c0456d78>] (zram_make_request+0x1d8/0x378) from [<c02c7afc>]
> (generic_make_request+0xb0/0xdc)
> [ 265.801588] [<c02c7afc>] (generic_make_request+0xb0/0xdc) from
> [<c02c7bb0>] (submit_bio+0x88/0x140)
> [ 265.810617] [<c02c7bb0>] (submit_bio+0x88/0x140) from [<c01459d4>]
> (__swap_writepage+0x198/0x230)
> [ 265.819471] [<c01459d4>] (__swap_writepage+0x198/0x230) from [<c011fc50>]
> (shrink_page_list+0x4e0/0x974)
> [ 265.828930] [<c011fc50>] (shrink_page_list+0x4e0/0x974) from [<c0120644>]
> (shrink_inactive_list+0x150/0x3c8)
> [ 265.838736] [<c0120644>] (shrink_inactive_list+0x150/0x3c8) from
> [<c0120de8>] (shrink_lruvec+0x20c/0x448)
> [ 265.848282] [<c0120de8>] (shrink_lruvec+0x20c/0x448) from [<c012109c>]
> (shrink_zone+0x78/0x188)
> [ 265.856960] [<c012109c>] (shrink_zone+0x78/0x188) from [<c01212ac>]
> (do_try_to_free_pages+0x100/0x544)
> [ 265.866246] [<c01212ac>] (do_try_to_free_pages+0x100/0x544) from
> [<c0121928>] (try_to_free_pages+0x238/0x428)
> [ 265.876140] [<c0121928>] (try_to_free_pages+0x238/0x428) from [<c01179cc>]
> (__alloc_pages_nodemask+0x5b0/0x90c)
> [ 265.886207] [<c01179cc>] (__alloc_pages_nodemask+0x5b0/0x90c) from
> [<c0117d44>] (__get_free_pages+0x1c/0x34)
> [ 265.896014] [<c0117d44>] (__get_free_pages+0x1c/0x34) from [<c0844a4c>]
> (tcp4_seq_show+0x248/0x4b4)
> [ 265.905042] [<c0844a4c>] (tcp4_seq_show+0x248/0x4b4) from [<c017a844>]
> (seq_read+0x1e4/0x484)
> [ 265.913550] [<c017a844>] (seq_read+0x1e4/0x484) from [<c01a84f0>]
> (proc_reg_read+0x60/0x88)
> [ 265.921884] [<c01a84f0>] (proc_reg_read+0x60/0x88) from [<c015aa44>]
> (vfs_read+0xa0/0x14c)
> [ 265.930129] [<c015aa44>] (vfs_read+0xa0/0x14c) from [<c015b0c4>]
> (SyS_read+0x44/0x80)
> [ 265.937942] [<c015b0c4>] (SyS_read+0x44/0x80) from [<c052e98c>]
> (netstat_work_func+0x54/0xec)
> [ 265.946450] [<c052e98c>] (netstat_work_func+0x54/0xec) from [<c0086700>]
> (process_one_work+0x13c/0x454)
> [ 265.955823] [<c0086700>] (process_one_work+0x13c/0x454) from [<c008745c>]
> (worker_thread+0x140/0x3dc)
> 265.965022] [<c008745c>] (worker_thread+0x140/0x3dc) from [<c008cf4c>]
> (kthread+0xe0/0xe4)
> [ 265.973269] [<c008cf4c>] (kthread+0xe0/0xe4) from [<c000ef98>]
> (ret_from_fork+0x14/0x20)
> [ 264.148640] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
> [ 264.155930] Control: 30c5387d Table: aaf7c000 DAC: fffffffd

This looks like a regular syscall path. How have you concluded this is
due to an IRQ context?
Ashish Mhetre Oct. 16, 2018, 9:35 a.m. UTC | #7
Sorry for the delayed response.
Yes, you are correct. There is no call from IRQ in dump.
I take back our assertion. Thanks for the insights.


On Wednesday 03 October 2018 05:23 PM, Michal Hocko wrote:
> On Wed 03-10-18 17:20:15, Ashish Mhetre wrote:
>>> This doesn't show the backtrace part which contains the allocation
>> AFAICS.
>>
>> My bad. Here is a complete dump:
>> [ 264.082531] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP ARM
>> [ 264.088350] Modules linked in:
>> [ 264.091406] CPU: 0 PID: 3805 Comm: kworker/0:4 Tainted: G W
>> 3.10.33-g990282b #1
>> [ 264.099572] Workqueue: events netstat_work_func
>> [ 264.104097] task: e7b12040 ti: dc7d4000 task.ti: dc7d4000
>> [ 264.109485] PC is at zs_map_object+0x180/0x18c
>> [ 264.113918] LR is at zram_bvec_rw.isra.15+0x304/0x88c
>> [ 264.118956] pc : [<c01581e8>] lr : [<c0456618>] psr: 200f0013
>> [ 264.118956] sp : dc7d5460 ip : fff00814 fp : 00000002
>> [ 264.130407] r10: ea8ec000 r9 : ebc93340 r8 : 00000000
>> [ 264.135618] r7 : c191502c r6 : dc7d4020 r5 : d25f5684 r4 : ec3158c0
>> [ 264.142128] r3 : 00000200 r2 : 00000002 r1 : c191502c r0 : ea8ec000
>>
>> --------
>> [ 265.772426] [<c01581e8>] (zs_map_object+0x180/0x18c) from [<c0456618>]
>> (zram_bvec_rw.isra.15+0x304/0x88c)
>> [ 265.781973] [<c0456618>] (zram_bvec_rw.isra.15+0x304/0x88c) from
>> [<c0456d78>] (zram_make_request+0x1d8/0x378)
>> [ 265.791868] [<c0456d78>] (zram_make_request+0x1d8/0x378) from [<c02c7afc>]
>> (generic_make_request+0xb0/0xdc)
>> [ 265.801588] [<c02c7afc>] (generic_make_request+0xb0/0xdc) from
>> [<c02c7bb0>] (submit_bio+0x88/0x140)
>> [ 265.810617] [<c02c7bb0>] (submit_bio+0x88/0x140) from [<c01459d4>]
>> (__swap_writepage+0x198/0x230)
>> [ 265.819471] [<c01459d4>] (__swap_writepage+0x198/0x230) from [<c011fc50>]
>> (shrink_page_list+0x4e0/0x974)
>> [ 265.828930] [<c011fc50>] (shrink_page_list+0x4e0/0x974) from [<c0120644>]
>> (shrink_inactive_list+0x150/0x3c8)
>> [ 265.838736] [<c0120644>] (shrink_inactive_list+0x150/0x3c8) from
>> [<c0120de8>] (shrink_lruvec+0x20c/0x448)
>> [ 265.848282] [<c0120de8>] (shrink_lruvec+0x20c/0x448) from [<c012109c>]
>> (shrink_zone+0x78/0x188)
>> [ 265.856960] [<c012109c>] (shrink_zone+0x78/0x188) from [<c01212ac>]
>> (do_try_to_free_pages+0x100/0x544)
>> [ 265.866246] [<c01212ac>] (do_try_to_free_pages+0x100/0x544) from
>> [<c0121928>] (try_to_free_pages+0x238/0x428)
>> [ 265.876140] [<c0121928>] (try_to_free_pages+0x238/0x428) from [<c01179cc>]
>> (__alloc_pages_nodemask+0x5b0/0x90c)
>> [ 265.886207] [<c01179cc>] (__alloc_pages_nodemask+0x5b0/0x90c) from
>> [<c0117d44>] (__get_free_pages+0x1c/0x34)
>> [ 265.896014] [<c0117d44>] (__get_free_pages+0x1c/0x34) from [<c0844a4c>]
>> (tcp4_seq_show+0x248/0x4b4)
>> [ 265.905042] [<c0844a4c>] (tcp4_seq_show+0x248/0x4b4) from [<c017a844>]
>> (seq_read+0x1e4/0x484)
>> [ 265.913550] [<c017a844>] (seq_read+0x1e4/0x484) from [<c01a84f0>]
>> (proc_reg_read+0x60/0x88)
>> [ 265.921884] [<c01a84f0>] (proc_reg_read+0x60/0x88) from [<c015aa44>]
>> (vfs_read+0xa0/0x14c)
>> [ 265.930129] [<c015aa44>] (vfs_read+0xa0/0x14c) from [<c015b0c4>]
>> (SyS_read+0x44/0x80)
>> [ 265.937942] [<c015b0c4>] (SyS_read+0x44/0x80) from [<c052e98c>]
>> (netstat_work_func+0x54/0xec)
>> [ 265.946450] [<c052e98c>] (netstat_work_func+0x54/0xec) from [<c0086700>]
>> (process_one_work+0x13c/0x454)
>> [ 265.955823] [<c0086700>] (process_one_work+0x13c/0x454) from [<c008745c>]
>> (worker_thread+0x140/0x3dc)
>> 265.965022] [<c008745c>] (worker_thread+0x140/0x3dc) from [<c008cf4c>]
>> (kthread+0xe0/0xe4)
>> [ 265.973269] [<c008cf4c>] (kthread+0xe0/0xe4) from [<c000ef98>]
>> (ret_from_fork+0x14/0x20)
>> [ 264.148640] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment kernel
>> [ 264.155930] Control: 30c5387d Table: aaf7c000 DAC: fffffffd
> This looks like a regular syscall path. How have you concluded this is
> due to an IRQ context?
diff mbox series

Patch

diff --git a/mm/vmscan.c b/mm/vmscan.c
index 0b63d9a..d9d36a5 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -365,6 +365,16 @@  unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone
 
 }
 
+/* If zram is being used as swap, and zram is using zsmalloc allocator
+ * then there is a potential bug when reclaim happens from interrupt
+ * context
+ */
+static void adjust_scan_control(struct scan_control *sc)
+{
+	if (in_interrupt() && IS_ENABLED(ZSMALLOC) && total_swap_pages)
+		sc->may_swap = 0;
+}
+
 /*
  * Add a shrinker callback to be called from the vm.
  */
@@ -1519,6 +1529,7 @@  unsigned long reclaim_clean_pages_from_list(struct zone *zone,
 	unsigned long ret;
 	struct page *page, *next;
 	LIST_HEAD(clean_pages);
+	adjust_scan_control(&sc);
 
 	list_for_each_entry_safe(page, next, page_list, lru) {
 		if (page_is_file_cache(page) && !PageDirty(page) &&
@@ -3232,6 +3243,8 @@  unsigned long try_to_free_pages(struct zonelist *zonelist, int order,
 		.may_swap = 1,
 	};
 
+	adjust_scan_control(&sc);
+
 	/*
 	 * scan_control uses s8 fields for order, priority, and reclaim_idx.
 	 * Confirm they are large enough for max values.
@@ -3277,6 +3290,8 @@  unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg,
 	};
 	unsigned long lru_pages;
 
+	adjust_scan_control(&sc);
+
 	sc.gfp_mask = (gfp_mask & GFP_RECLAIM_MASK) |
 			(GFP_HIGHUSER_MOVABLE & ~GFP_RECLAIM_MASK);
 
@@ -3322,6 +3337,7 @@  unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg,
 		.may_swap = may_swap,
 	};
 
+	adjust_scan_control(&sc);
 	/*
 	 * Unlike direct reclaim via alloc_pages(), memcg's reclaim doesn't
 	 * take care of from where we get pages. So the node where we start the
@@ -3518,6 +3534,7 @@  static int balance_pgdat(pg_data_t *pgdat, int order, int classzone_idx)
 		.may_swap = 1,
 	};
 
+	adjust_scan_control(&sc);
 	psi_memstall_enter(&pflags);
 	__fs_reclaim_acquire();
 
@@ -3891,6 +3908,7 @@  unsigned long shrink_all_memory(unsigned long nr_to_reclaim)
 	unsigned long nr_reclaimed;
 	unsigned int noreclaim_flag;
 
+	adjust_scan_control(&sc);
 	fs_reclaim_acquire(sc.gfp_mask);
 	noreclaim_flag = memalloc_noreclaim_save();
 	reclaim_state.reclaimed_slab = 0;
@@ -4076,6 +4094,7 @@  static int __node_reclaim(struct pglist_data *pgdat, gfp_t gfp_mask, unsigned in
 		.reclaim_idx = gfp_zone(gfp_mask),
 	};
 
+	adjust_scan_control(&sc);
 	cond_resched();
 	fs_reclaim_acquire(sc.gfp_mask);
 	/*