| Message ID | 20231011084228.77615-1-jefflexu@linux.alibaba.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | writeback, cgroup: switch inodes with dirty timestamps to release dying cgwbs | expand |
On Wed, Oct 11, 2023 at 04:42:28PM +0800, Jingbo Xu wrote: > The cgwb cleanup routine will try to release the dying cgwb by switching > the attached inodes. It fetches the attached inodes from wb->b_attached > list, omitting the fact that inodes only with dirty timestamps reside in > wb->b_dirty_time list, which is the case when lazytime is enabled. This > causes enormous zombie memory cgroup when lazytime is enabled, as inodes > with dirty timestamps can not be switched to a live cgwb for a long time. > > It is reasonable not to switch cgwb for inodes with dirty data, as > otherwise it may break the bandwidth restrictions. However since the > writeback of inode metadata is not accounted, let's also switch inodes > with dirty timestamps to avoid zombie memory and block cgroups when > laztytime is enabled. > > Fixs: c22d70a162d3 ("writeback, cgroup: release dying cgwbs by switching attached inodes") > Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com> The patch looks fine to me. ... > + restart = isw_prepare_wbs_switch(isw, &wb->b_attached, &nr); > + if (!restart) > + restart = isw_prepare_wbs_switch(isw, &wb->b_dirty_time, &nr); But can you add a comment explaining why we're also migrating b_dirty_time? Thanks.
On 10/13/23 1:31 AM, Tejun Heo wrote: > On Wed, Oct 11, 2023 at 04:42:28PM +0800, Jingbo Xu wrote: >> The cgwb cleanup routine will try to release the dying cgwb by switching >> the attached inodes. It fetches the attached inodes from wb->b_attached >> list, omitting the fact that inodes only with dirty timestamps reside in >> wb->b_dirty_time list, which is the case when lazytime is enabled. This >> causes enormous zombie memory cgroup when lazytime is enabled, as inodes >> with dirty timestamps can not be switched to a live cgwb for a long time. >> >> It is reasonable not to switch cgwb for inodes with dirty data, as >> otherwise it may break the bandwidth restrictions. However since the >> writeback of inode metadata is not accounted, let's also switch inodes >> with dirty timestamps to avoid zombie memory and block cgroups when >> laztytime is enabled. >> >> Fixs: c22d70a162d3 ("writeback, cgroup: release dying cgwbs by switching attached inodes") >> Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com> > > The patch looks fine to me. > > ... >> + restart = isw_prepare_wbs_switch(isw, &wb->b_attached, &nr); >> + if (!restart) >> + restart = isw_prepare_wbs_switch(isw, &wb->b_dirty_time, &nr); > > But can you add a comment explaining why we're also migrating b_dirty_time? Will add the comment in the next version. Thanks.
diff --git a/fs/fs-writeback.c b/fs/fs-writeback.c index c1af01b2c42d..89125760e4ad 100644 --- a/fs/fs-writeback.c +++ b/fs/fs-writeback.c @@ -613,6 +613,24 @@ static void inode_switch_wbs(struct inode *inode, int new_wb_id) kfree(isw); } +static bool isw_prepare_wbs_switch(struct inode_switch_wbs_context *isw, + struct list_head *list, int *nr) +{ + struct inode *inode; + + list_for_each_entry(inode, list, i_io_list) { + if (!inode_prepare_wbs_switch(inode, isw->new_wb)) + continue; + + isw->inodes[*nr] = inode; + (*nr)++; + + if (*nr >= WB_MAX_INODES_PER_ISW - 1) + return true; + } + return false; +} + /** * cleanup_offline_cgwb - detach associated inodes * @wb: target wb @@ -625,7 +643,6 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb) { struct cgroup_subsys_state *memcg_css; struct inode_switch_wbs_context *isw; - struct inode *inode; int nr; bool restart = false; @@ -647,17 +664,9 @@ bool cleanup_offline_cgwb(struct bdi_writeback *wb) nr = 0; spin_lock(&wb->list_lock); - list_for_each_entry(inode, &wb->b_attached, i_io_list) { - if (!inode_prepare_wbs_switch(inode, isw->new_wb)) - continue; - - isw->inodes[nr++] = inode; - - if (nr >= WB_MAX_INODES_PER_ISW - 1) { - restart = true; - break; - } - } + restart = isw_prepare_wbs_switch(isw, &wb->b_attached, &nr); + if (!restart) + restart = isw_prepare_wbs_switch(isw, &wb->b_dirty_time, &nr); spin_unlock(&wb->list_lock); /* no attached inodes? bail out */
The cgwb cleanup routine will try to release the dying cgwb by switching the attached inodes. It fetches the attached inodes from wb->b_attached list, omitting the fact that inodes only with dirty timestamps reside in wb->b_dirty_time list, which is the case when lazytime is enabled. This causes enormous zombie memory cgroup when lazytime is enabled, as inodes with dirty timestamps can not be switched to a live cgwb for a long time. It is reasonable not to switch cgwb for inodes with dirty data, as otherwise it may break the bandwidth restrictions. However since the writeback of inode metadata is not accounted, let's also switch inodes with dirty timestamps to avoid zombie memory and block cgroups when laztytime is enabled. Fixs: c22d70a162d3 ("writeback, cgroup: release dying cgwbs by switching attached inodes") Signed-off-by: Jingbo Xu <jefflexu@linux.alibaba.com> --- This issue is reported on our production environment (5.10 kernel with the dying cgwbs optimization[1] backported, and ext4 mounted with "-o relatime,lazytime"), while I can also reproduce it on the latest mainline kernel. [1] c22d70a162d3 ("writeback, cgroup: release dying cgwbs by switching attached inodes") --- fs/fs-writeback.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)