[v2] mm: shmem: implement POSIX_FADV_[WILL|DONT]NEED for shmem

Commit Message

Charan Teja Kalla Dec. 2, 2021, 10:50 a.m. UTC

From: Charan Teja Reddy <[e-mail]>

Currently fadvise(2) is supported only for the files that doesn't
associated with noop_backing_dev_info thus for the files, like shmem,
fadvise results into NOP. But then there is file_operations->fadvise()
that lets the file systems to implement their own fadvise
implementation. Use this support to implement some of the POSIX_FADV_XXX
functionality for shmem files.

This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED
advices to shmem files which can be helpful for the drivers who may want
to manage the shmem pages of the files that are created through
shmem_file_setup[_with_mnt]().  An example usecase may be like, driver
can create the shmem file of the size equal to its requirements and
map the pages for DMA and then pass the fd to user. The user who knows
well about the usage of these pages can now decide when these pages are
not required push them to swap through DONTNEED thus free up memory well
in advance rather than relying on the reclaim and use WILLNEED when it
decide that they are useful in the near future. IOW, it lets the clients
to free up/read the memory when it wants to. Another usecase is that GEM
objets which are currenlty allocated and managed through shmem files can
use vfs_fadvise(DONT|WILLNEED) on shmem fd when the driver comes to
know(like through some hints from user space) that GEM objects are not
going to use/will need in the near future.

Signed-off-by: Charan Teja Reddy <[e-mail]>
---

Changes in V2:
  -- Rearranged the code to not to sleep with rcu_lock while using xas_() functionality.
  -- Addressed the comments from Suren.

changes in V1:
  -- Created the interface for fadvise(2) to work on shmem files.
  -- https://patchwork.kernel.org/project/linux-mm/patch/[e-mail]/
       

 mm/shmem.c | 167 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 167 insertions(+)

Comments

Matthew Wilcox Dec. 2, 2021, 1:27 p.m. UTC | #1

On Thu, Dec 02, 2021 at 04:20:53PM +0530, Charan Teja Reddy wrote:
> +static int shmem_fadvise_willneed(struct address_space *mapping,
> +				 pgoff_t start, pgoff_t long end)
> +{
> +	XA_STATE(xas, &mapping->i_pages, start);
> +	struct page *page;
> +
> +	rcu_read_lock();
> +	page = xas_find(&xas, end);
> +	rcu_read_unlock();
> +
> +	while (page) {
> +		if (xa_is_value(page)) {
> +			page = shmem_read_mapping_page(mapping, xas.xa_index);
> +			if (!IS_ERR(page))
> +				put_page(page);
> +		}
> +
> +		if (need_resched()) {
> +			xas_pause(&xas);
> +			cond_resched();
> +		}
> +
> +		rcu_read_lock();
> +		page = xas_next_entry(&xas, end);
> +		rcu_read_unlock();
> +	}
> +
> +	return 0;
> +}

What part of the XArray documentation led you to believe that this is a
safe thing to do?  Because it needs to be rewritten immediately!

Matthew Wilcox Dec. 2, 2021, 2:59 p.m. UTC | #2

On Thu, Dec 02, 2021 at 04:20:53PM +0530, Charan Teja Reddy wrote:
> +static void shmem_isolate_pages_range(struct address_space *mapping, loff_t start,
> +				loff_t end, struct list_head *list)
> +{
> +	XA_STATE(xas, &mapping->i_pages, start);
> +	struct page *page;
> +
> +	rcu_read_lock();
> +	xas_for_each(&xas, page, end) {
> +		if (xas_retry(&xas, page))
> +			continue;
> +		if (xa_is_value(page))
> +			continue;
> +		if (!get_page_unless_zero(page))
> +			continue;
> +		if (isolate_lru_page(page))
> +			continue;
> +
> +		list_add(&page->lru, list);
> +		inc_node_page_state(page, NR_ISOLATED_ANON +
> +						page_is_file_lru(page));

... what if the page is a THP?

Charan Teja Kalla Dec. 2, 2021, 3:29 p.m. UTC | #3

Thanks Matthew for the comments!!

On 12/2/2021 6:57 PM, Matthew Wilcox wrote:
> On Thu, Dec 02, 2021 at 04:20:53PM +0530, Charan Teja Reddy wrote:
>> +static int shmem_fadvise_willneed(struct address_space *mapping,
>> +				 pgoff_t start, pgoff_t long end)
>> +{
>> +	XA_STATE(xas, &mapping->i_pages, start);
>> +	struct page *page;
>> +
>> +	rcu_read_lock();
>> +	page = xas_find(&xas, end);
>> +	rcu_read_unlock();
>> +
>> +	while (page) {
>> +		if (xa_is_value(page)) {
>> +			page = shmem_read_mapping_page(mapping, xas.xa_index);
>> +			if (!IS_ERR(page))
>> +				put_page(page);
>> +		}
>> +
>> +		if (need_resched()) {
>> +			xas_pause(&xas);
>> +			cond_resched();
>> +		}
>> +
>> +		rcu_read_lock();
>> +		page = xas_next_entry(&xas, end);
>> +		rcu_read_unlock();
>> +	}
>> +
>> +	return 0;
>> +}
> 
> What part of the XArray documentation led you to believe that this is a
> safe thing to do?  Because it needs to be rewritten immediately!

The above code changes made from my understanding of both the
Documentation and the implementation of xa_for_each(). The Locking
section of the document[1] says that xa_for_each() takes the rcu lock
thus can be used without any explicit locking and the "Advanced API"
section says that users need to take xa_lock/rcu lock as no locking done
for you.

Further I have looked at the xa_for_each() implementation details,
where, it is taking the rcu_lock just across xas_find() in both
xa_find() and xa_find_after() which made me to think that it just needs
to take the rcu lock just across the xas_find().

But a comment from you saying that this implementation is wrong making
me to think that I lack very trivial understanding about xarray usage.

[1] https://www.kernel.org/doc/html/latest/core-api/xarray.html

>

Matthew Wilcox Dec. 2, 2021, 3:54 p.m. UTC | #4

On Thu, Dec 02, 2021 at 08:59:52PM +0530, Charan Teja Kalla wrote:
> > What part of the XArray documentation led you to believe that this is a
> > safe thing to do?  Because it needs to be rewritten immediately!
> 
> The above code changes made from my understanding of both the
> Documentation and the implementation of xa_for_each(). The Locking
> section of the document[1] says that xa_for_each() takes the rcu lock
> thus can be used without any explicit locking and the "Advanced API"
> section says that users need to take xa_lock/rcu lock as no locking done
> for you.
> 
> Further I have looked at the xa_for_each() implementation details,
> where, it is taking the rcu_lock just across xas_find() in both
> xa_find() and xa_find_after() which made me to think that it just needs
> to take the rcu lock just across the xas_find().
> 
> But a comment from you saying that this implementation is wrong making
> me to think that I lack very trivial understanding about xarray usage.

Would this change to the documentation have prevented you from making
this mistake?

 The advanced API is based around the xa_state.  This is an opaque data
 structure which you declare on the stack using the XA_STATE()
 macro.  This macro initialises the xa_state ready to start walking
 around the XArray.  It is used as a cursor to maintain the position
 in the XArray and let you compose various operations together without
-having to restart from the top every time.
+having to restart from the top every time.  The contents of the xa_state
+are protected by the rcu_read_lock() or the xas_lock().  If you need to
+drop whichever of those locks is protecting your state and tree, you must
+call xas_pause() so that future calls do not rely on the parts of the
+state which were left unprotected.

Shakeel Butt Dec. 2, 2021, 5:54 p.m. UTC | #5

On Thu, Dec 2, 2021 at 2:51 AM Charan Teja Reddy
<[e-mail]> wrote:
>
> From: Charan Teja Reddy <[e-mail]>
>
> Currently fadvise(2) is supported only for the files that doesn't
> associated with noop_backing_dev_info thus for the files, like shmem,
> fadvise results into NOP. But then there is file_operations->fadvise()
> that lets the file systems to implement their own fadvise
> implementation. Use this support to implement some of the POSIX_FADV_XXX
> functionality for shmem files.
>
> This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED
> advices to shmem files which can be helpful for the drivers who may want
> to manage the shmem pages of the files that are created through
> shmem_file_setup[_with_mnt]().  An example usecase may be like, driver
> can create the shmem file of the size equal to its requirements and
> map the pages for DMA and then pass the fd to user. The user who knows
> well about the usage of these pages can now decide when these pages are
> not required push them to swap through DONTNEED thus free up memory well
> in advance rather than relying on the reclaim and use WILLNEED when it
> decide that they are useful in the near future. IOW, it lets the clients
> to free up/read the memory when it wants to. Another usecase is that GEM
> objets which are currenlty allocated and managed through shmem files can
> use vfs_fadvise(DONT|WILLNEED) on shmem fd when the driver comes to
> know(like through some hints from user space) that GEM objects are not
> going to use/will need in the near future.

The proposed semantics of POSIX_FADV_DONTNEED is actually similar to
MADV_PAGEOUT and different from MADV_DONTNEED. This is a user facing
API and this difference will cause confusion.

Charan Teja Kalla Dec. 3, 2021, 11:55 a.m. UTC | #6

()

On 12/2/2021 9:24 PM, Matthew Wilcox wrote:
> Would this change to the documentation have prevented you from making
> this mistake?
> 
>  The advanced API is based around the xa_state.  This is an opaque data
>  structure which you declare on the stack using the XA_STATE()
>  macro.  This macro initialises the xa_state ready to start walking
>  around the XArray.  It is used as a cursor to maintain the position
>  in the XArray and let you compose various operations together without
> -having to restart from the top every time.
> +having to restart from the top every time.  The contents of the xa_state
> +are protected by the rcu_read_lock() or the xas_lock().  If you need to
> +drop whichever of those locks is protecting your state and tree, you must
> +call xas_pause() so that future calls do not rely on the parts of the
> +state which were left unprotected.

Yes, this looks much clear to me.

Charan Teja Kalla Dec. 3, 2021, 1:02 p.m. UTC | #7

On 12/2/2021 8:29 PM, Matthew Wilcox wrote:
>> +		list_add(&page->lru, list);
>> +		inc_node_page_state(page, NR_ISOLATED_ANON +
>> +						page_is_file_lru(page));
> ... what if the page is a THP?

Will take care of THP pages in the next spin. Thanks for pointing it out.

Charan Teja Kalla Dec. 6, 2021, 7:29 a.m. UTC | #8

Thanks Shakeel for your valuable inputs!!

On 12/2/2021 11:24 PM, Shakeel Butt wrote:
> On Thu, Dec 2, 2021 at 2:51 AM Charan Teja Reddy
> <[e-mail]> wrote:
>>
>> From: Charan Teja Reddy <[e-mail]>
>>
>> Currently fadvise(2) is supported only for the files that doesn't
>> associated with noop_backing_dev_info thus for the files, like shmem,
>> fadvise results into NOP. But then there is file_operations->fadvise()
>> that lets the file systems to implement their own fadvise
>> implementation. Use this support to implement some of the POSIX_FADV_XXX
>> functionality for shmem files.
>>
>> This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED
>> advices to shmem files which can be helpful for the drivers who may want
>> to manage the shmem pages of the files that are created through
>> shmem_file_setup[_with_mnt]().  An example usecase may be like, driver
>> can create the shmem file of the size equal to its requirements and
>> map the pages for DMA and then pass the fd to user. The user who knows
>> well about the usage of these pages can now decide when these pages are
>> not required push them to swap through DONTNEED thus free up memory well
>> in advance rather than relying on the reclaim and use WILLNEED when it
>> decide that they are useful in the near future. IOW, it lets the clients
>> to free up/read the memory when it wants to. Another usecase is that GEM
>> objets which are currenlty allocated and managed through shmem files can
>> use vfs_fadvise(DONT|WILLNEED) on shmem fd when the driver comes to
>> know(like through some hints from user space) that GEM objects are not
>> going to use/will need in the near future.
> 
> The proposed semantics of POSIX_FADV_DONTNEED is actually similar to
> MADV_PAGEOUT and different from MADV_DONTNEED. This is a user facing
> API and this difference will cause confusion.

man pages [1] says that "POSIX_FADV_DONTNEED attempts to free cached
pages associated with the specified region." This statement, IIUC,  on
issuing this FADV, it is expected to free the file cache pages. And it
is implementation defined If the dirty pages may be attempted to
writeback. And the unwritten dirty pages will not be freed.  And thus
for Shmem files this is being done by dirtying the page and pushing out
to swap.

So, Isn't the FADV_DONTNEED also covers the semantics of MADV_PAGEOUT
for file pages? IOW, what is the purpose of PAGEOUT for the file pages.
Or I am missing some trivial logic in your comment here?

Coming to MADV_DONTNEED[2], on the mapped shmem files doesn't have any
effect as the pages of those shmem files can still be in RAM. Subsequent
accesses of pages in the range will succeed from the up-to-date contents
of the underlying mapped file. IOW, the pages are still be present in
the cache. Am I wrong here?

[1] https://man7.org/linux/man-pages/man2/posix_fadvise.2.html
[2] https://man7.org/linux/man-pages/man2/madvise.2.html


>

Charan Teja Kalla Jan. 5, 2022, 3:17 p.m. UTC | #9

Firstly, Apologies for taking such a long time for the next spin. Posted
V3 @
https://patchwork.kernel.org/project/linux-mm/patch/[e-mail]/
. Please provide your comments.

On 12/6/2021 12:59 PM, Charan Teja Kalla wrote:
> Thanks Shakeel for your valuable inputs!!
> 
> On 12/2/2021 11:24 PM, Shakeel Butt wrote:
>> On Thu, Dec 2, 2021 at 2:51 AM Charan Teja Reddy
>> <[e-mail]> wrote:
>>>
>>> From: Charan Teja Reddy <[e-mail]>
>>>
>>> Currently fadvise(2) is supported only for the files that doesn't
>>> associated with noop_backing_dev_info thus for the files, like shmem,
>>> fadvise results into NOP. But then there is file_operations->fadvise()
>>> that lets the file systems to implement their own fadvise
>>> implementation. Use this support to implement some of the POSIX_FADV_XXX
>>> functionality for shmem files.
>>>
>>> This patch aims to implement POSIX_FADV_WILLNEED and POSIX_FADV_DONTNEED
>>> advices to shmem files which can be helpful for the drivers who may want
>>> to manage the shmem pages of the files that are created through
>>> shmem_file_setup[_with_mnt]().  An example usecase may be like, driver
>>> can create the shmem file of the size equal to its requirements and
>>> map the pages for DMA and then pass the fd to user. The user who knows
>>> well about the usage of these pages can now decide when these pages are
>>> not required push them to swap through DONTNEED thus free up memory well
>>> in advance rather than relying on the reclaim and use WILLNEED when it
>>> decide that they are useful in the near future. IOW, it lets the clients
>>> to free up/read the memory when it wants to. Another usecase is that GEM
>>> objets which are currenlty allocated and managed through shmem files can
>>> use vfs_fadvise(DONT|WILLNEED) on shmem fd when the driver comes to
>>> know(like through some hints from user space) that GEM objects are not
>>> going to use/will need in the near future.
>>
>> The proposed semantics of POSIX_FADV_DONTNEED is actually similar to
>> MADV_PAGEOUT and different from MADV_DONTNEED. This is a user facing
>> API and this difference will cause confusion.
> 
> man pages [1] says that "POSIX_FADV_DONTNEED attempts to free cached
> pages associated with the specified region." This statement, IIUC,  on
> issuing this FADV, it is expected to free the file cache pages. And it
> is implementation defined If the dirty pages may be attempted to
> writeback. And the unwritten dirty pages will not be freed.  And thus
> for Shmem files this is being done by dirtying the page and pushing out
> to swap.
> 
> So, Isn't the FADV_DONTNEED also covers the semantics of MADV_PAGEOUT
> for file pages? IOW, what is the purpose of PAGEOUT for the file pages.
> Or I am missing some trivial logic in your comment here?
> 
> Coming to MADV_DONTNEED[2], on the mapped shmem files doesn't have any
> effect as the pages of those shmem files can still be in RAM. Subsequent
> accesses of pages in the range will succeed from the up-to-date contents
> of the underlying mapped file. IOW, the pages are still be present in
> the cache. Am I wrong here?
> 
> [1] https://man7.org/linux/man-pages/man2/posix_fadvise.2.html
> [2] https://man7.org/linux/man-pages/man2/madvise.2.html
> 
> 
>>
>

Patch

diff --git a/mm/shmem.c b/mm/shmem.c
index 70d9ce2..4c4685f 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -38,6 +38,8 @@ 
 #include <linux/hugetlb.h>
 #include <linux/frontswap.h>
 #include <linux/fs_parser.h>
+#include <linux/mm_inline.h>
+#include <linux/fadvise.h>
 
 #include <asm/tlbflush.h> /* for arch/microblaze update_mmu_cache() */
 
@@ -2792,6 +2794,170 @@  static long shmem_fallocate(struct file *file, int mode, loff_t offset,
 	return error;
 }
 
+static void shmem_isolate_pages_range(struct address_space *mapping, loff_t start,
+				loff_t end, struct list_head *list)
+{
+	XA_STATE(xas, &mapping->i_pages, start);
+	struct page *page;
+
+	rcu_read_lock();
+	xas_for_each(&xas, page, end) {
+		if (xas_retry(&xas, page))
+			continue;
+		if (xa_is_value(page))
+			continue;
+		if (!get_page_unless_zero(page))
+			continue;
+		if (isolate_lru_page(page))
+			continue;
+
+		list_add(&page->lru, list);
+		inc_node_page_state(page, NR_ISOLATED_ANON +
+						page_is_file_lru(page));
+		put_page(page);
+		if (need_resched()) {
+			xas_pause(&xas);
+			cond_resched_rcu();
+		}
+	}
+	rcu_read_unlock();
+}
+
+static int shmem_fadvise_dontneed(struct address_space *mapping, loff_t start,
+				loff_t end)
+{
+	int ret;
+	struct page *page;
+	LIST_HEAD(list);
+
+	if (!shmem_mapping(mapping))
+		return -EINVAL;
+
+	if (!total_swap_pages)
+		return 0;
+
+	lru_add_drain();
+	shmem_isolate_pages_range(mapping, start, end, &list);
+
+	while (!list_empty(&list)) {
+		page = lru_to_page(&list);
+		list_del(&page->lru);
+		lock_page(page);
+		if (clear_page_dirty_for_io(page)) {
+			struct writeback_control wbc = {
+				.sync_mode = WB_SYNC_NONE,
+				.nr_to_write = LONG_MAX,
+				.range_start = 0,
+				.range_end = LLONG_MAX,
+				.for_reclaim = 1,
+			};
+
+			SetPageReclaim(page);
+			ret = shmem_writepage(page, &wbc);
+			if (ret || PageWriteback(page)) {
+				if (ret)
+					unlock_page(page);
+				putback_lru_page(page);
+				dec_node_page_state(page, NR_ISOLATED_ANON +
+						page_is_file_lru(page));
+				continue;
+			}
+		}
+
+		if (!PageWriteback(page))
+			ClearPageReclaim(page);
+
+		/*
+		 * shmem_writepage() place the page in the swapcache.
+		 * Delete the page from the swapcache and release the
+		 * page.
+		 */
+		dec_node_page_state(page, NR_ISOLATED_ANON +
+						page_is_file_lru(page));
+		lock_page(page);
+		delete_from_swap_cache(page);
+		unlock_page(page);
+		put_page(page);
+
+	}
+
+	return 0;
+}
+
+static int shmem_fadvise_willneed(struct address_space *mapping,
+				 pgoff_t start, pgoff_t long end)
+{
+	XA_STATE(xas, &mapping->i_pages, start);
+	struct page *page;
+
+	rcu_read_lock();
+	page = xas_find(&xas, end);
+	rcu_read_unlock();
+
+	while (page) {
+		if (xa_is_value(page)) {
+			page = shmem_read_mapping_page(mapping, xas.xa_index);
+			if (!IS_ERR(page))
+				put_page(page);
+		}
+
+		if (need_resched()) {
+			xas_pause(&xas);
+			cond_resched();
+		}
+
+		rcu_read_lock();
+		page = xas_next_entry(&xas, end);
+		rcu_read_unlock();
+	}
+
+	return 0;
+}
+
+static int shmem_fadvise(struct file *file, loff_t offset, loff_t len, int advice)
+{
+	loff_t endbyte;
+	pgoff_t start_index;
+	pgoff_t end_index;
+	struct address_space *mapping;
+	int ret = 0;
+
+	mapping = file->f_mapping;
+	if (!mapping || len < 0)
+		return -EINVAL;
+
+	endbyte = (u64)offset + (u64)len;
+	if (!len || endbyte < len)
+		endbyte = -1;
+	else
+		endbyte--;
+
+
+	start_index = offset >> PAGE_SHIFT;
+	end_index   = endbyte >> PAGE_SHIFT;
+	switch (advice) {
+	case POSIX_FADV_DONTNEED:
+		ret = shmem_fadvise_dontneed(mapping, start_index, end_index);
+		break;
+	case POSIX_FADV_WILLNEED:
+		ret = shmem_fadvise_willneed(mapping, start_index, end_index);
+		break;
+	case POSIX_FADV_NORMAL:
+	case POSIX_FADV_RANDOM:
+	case POSIX_FADV_SEQUENTIAL:
+	case POSIX_FADV_NOREUSE:
+		/*
+		 * No bad return value, but ignore advice. May have to
+		 * implement in future.
+		 */
+		break;
+	default:
+		return -EINVAL;
+	}
+
+	return ret;
+}
+
 static int shmem_statfs(struct dentry *dentry, struct kstatfs *buf)
 {
 	struct shmem_sb_info *sbinfo = SHMEM_SB(dentry->d_sb);
@@ -3799,6 +3965,7 @@  static const struct file_operations shmem_file_operations = {
 	.splice_write	= iter_file_splice_write,
 	.fallocate	= shmem_fallocate,
 #endif
+	.fadvise	= shmem_fadvise,
 };
 
 static const struct inode_operations shmem_inode_operations = {