[v2] fs: Replace kmap{,_atomic}() with kmap_local_page()

Commit Message

Fabio M. De Francesco Aug. 3, 2022, 6:28 p.m. UTC

The use of kmap() and kmap_atomic() are being deprecated in favor of
kmap_local_page().

There are two main problems with kmap(): (1) It comes with an overhead as
mapping space is restricted and protected by a global lock for
synchronization and (2) it also requires global TLB invalidation when the
kmap’s pool wraps and it might block when the mapping space is fully
utilized until a slot becomes available.

With kmap_local_page() the mappings are per thread, CPU local, can take
page faults, and can be called from any context (including interrupts).
It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore,
the tasks can be preempted and, when they are scheduled to run again, the
kernel virtual addresses are restored and are still valid.

Since the use of kmap_local_page() in exec.c is safe, it should be
preferred everywhere in exec.c.

As said, since kmap_local_page() can be also called from atomic context,
and since remove_arg_zero() doesn't (and shouldn't ever) rely on an
implicit preempt_disable(), this function can also safely replace
kmap_atomic().

Therefore, replace kmap() and kmap_atomic() with kmap_local_page() in
fs/exec.c.

Tested with xfstests on a QEMU/KVM x86_32 VM, 6GB RAM, booting a kernel
with HIGHMEM64GB enabled.

Cc: Eric W. Biederman <[e-mail]>
Suggested-by: Ira Weiny <[e-mail]>
Reviewed-by: Ira Weiny <[e-mail]>
Signed-off-by: Fabio M. De Francesco <[e-mail]>
---

v1->v2: Added more information to the commit log to address some
objections expressed by Eric W. Biederman[1] in reply to v1. No changes
have been made to the code. Forwarded a tag from Ira Weiny (thanks!).

[1] https://lore.kernel.org/lkml/[e-mail]/

 fs/exec.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

Comments

Fabio M. De Francesco Aug. 13, 2022, 1:36 p.m. UTC | #1

On mercoledì 3 agosto 2022 20:28:56 CEST Fabio M. De Francesco wrote:
> The use of kmap() and kmap_atomic() are being deprecated in favor of
> kmap_local_page().
> 
> There are two main problems with kmap(): (1) It comes with an overhead as
> mapping space is restricted and protected by a global lock for
> synchronization and (2) it also requires global TLB invalidation when the
> kmap’s pool wraps and it might block when the mapping space is fully
> utilized until a slot becomes available.
> 
> With kmap_local_page() the mappings are per thread, CPU local, can take
> page faults, and can be called from any context (including interrupts).
> It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore,
> the tasks can be preempted and, when they are scheduled to run again, the
> kernel virtual addresses are restored and are still valid.
> 
> Since the use of kmap_local_page() in exec.c is safe, it should be
> preferred everywhere in exec.c.
> 
> As said, since kmap_local_page() can be also called from atomic context,
> and since remove_arg_zero() doesn't (and shouldn't ever) rely on an
> implicit preempt_disable(), this function can also safely replace
> kmap_atomic().
> 
> Therefore, replace kmap() and kmap_atomic() with kmap_local_page() in
> fs/exec.c.
> 
> Tested with xfstests on a QEMU/KVM x86_32 VM, 6GB RAM, booting a kernel
> with HIGHMEM64GB enabled.
> 
> Cc: Eric W. Biederman <[e-mail]>
> Suggested-by: Ira Weiny <[e-mail]>
> Reviewed-by: Ira Weiny <[e-mail]>
> Signed-off-by: Fabio M. De Francesco <[e-mail]>
> ---
> 
> v1->v2: Added more information to the commit log to address some
> objections expressed by Eric W. Biederman[1] in reply to v1. No changes
> have been made to the code. Forwarded a tag from Ira Weiny (thanks!).
> 
> [1]
> https://lore.kernel.org/lkml/[e-mail]/
>
>  fs/exec.c | 14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
> 

Hi Kees,

After that thread about the report from Syzbot, and the subsequent discussion, 
I noticed that you didn't yet take this other patch for exec.c.

I suppose that the two patches would better go out together. So I'm writing 
for sending a gentle ping.

As I said, no changes have been made to the code with respect to v1 (which I 
submitted in June). However, later I thought that adding more information 
might have helped reviewers and maintainers to better understand the why of 
this patch.

Thanks,

Fabio

Kees Cook Aug. 16, 2022, 7:08 p.m. UTC | #2

On Sat, Aug 13, 2022 at 03:36:53PM +0200, Fabio M. De Francesco wrote:
> On mercoledì 3 agosto 2022 20:28:56 CEST Fabio M. De Francesco wrote:
> > The use of kmap() and kmap_atomic() are being deprecated in favor of
> > kmap_local_page().
> > 
> > There are two main problems with kmap(): (1) It comes with an overhead as
> > mapping space is restricted and protected by a global lock for
> > synchronization and (2) it also requires global TLB invalidation when the
> > kmap’s pool wraps and it might block when the mapping space is fully
> > utilized until a slot becomes available.
> > 
> > With kmap_local_page() the mappings are per thread, CPU local, can take
> > page faults, and can be called from any context (including interrupts).
> > It is faster than kmap() in kernels with HIGHMEM enabled. Furthermore,
> > the tasks can be preempted and, when they are scheduled to run again, the
> > kernel virtual addresses are restored and are still valid.
> > 
> > Since the use of kmap_local_page() in exec.c is safe, it should be
> > preferred everywhere in exec.c.
> > 
> > As said, since kmap_local_page() can be also called from atomic context,
> > and since remove_arg_zero() doesn't (and shouldn't ever) rely on an
> > implicit preempt_disable(), this function can also safely replace
> > kmap_atomic().
> > 
> > Therefore, replace kmap() and kmap_atomic() with kmap_local_page() in
> > fs/exec.c.
> > 
> > Tested with xfstests on a QEMU/KVM x86_32 VM, 6GB RAM, booting a kernel
> > with HIGHMEM64GB enabled.
> > 
> > Cc: Eric W. Biederman <[e-mail]>
> > Suggested-by: Ira Weiny <[e-mail]>
> > Reviewed-by: Ira Weiny <[e-mail]>
> > Signed-off-by: Fabio M. De Francesco <[e-mail]>
> > ---
> > 
> > v1->v2: Added more information to the commit log to address some
> > objections expressed by Eric W. Biederman[1] in reply to v1. No changes
> > have been made to the code. Forwarded a tag from Ira Weiny (thanks!).
> > 
> > [1]
> > https://lore.kernel.org/lkml/[e-mail]/
> >
> >  fs/exec.c | 14 +++++++-------
> >  1 file changed, 7 insertions(+), 7 deletions(-)
> > 
> 
> Hi Kees,
> 
> After that thread about the report from Syzbot, and the subsequent discussion, 
> I noticed that you didn't yet take this other patch for exec.c.
> 
> I suppose that the two patches would better go out together. So I'm writing 
> for sending a gentle ping.
> 
> As I said, no changes have been made to the code with respect to v1 (which I 
> submitted in June). However, later I thought that adding more information 
> might have helped reviewers and maintainers to better understand the why of 
> this patch.

Oops, thanks for the ping. I'll pull this now.

Kees Cook Aug. 16, 2022, 7:13 p.m. UTC | #3

On Wed, 3 Aug 2022 20:28:56 +0200, Fabio M. De Francesco wrote:
> The use of kmap() and kmap_atomic() are being deprecated in favor of
> kmap_local_page().
> 
> There are two main problems with kmap(): (1) It comes with an overhead as
> mapping space is restricted and protected by a global lock for
> synchronization and (2) it also requires global TLB invalidation when the
> kmap’s pool wraps and it might block when the mapping space is fully
> utilized until a slot becomes available.
> 
> [...]

Applied to for-next/execve, thanks!

[1/1] fs: Replace kmap{,_atomic}() with kmap_local_page()
      https://git.kernel.org/kees/c/3a608cfee97e

Patch

diff --git a/fs/exec.c b/fs/exec.c
index 5fd73915c62c..b51dd14e7388 100644
--- a/fs/exec.c
+++ b/fs/exec.c
@@ -584,11 +584,11 @@  static int copy_strings(int argc, struct user_arg_ptr argv,
 
 				if (kmapped_page) {
 					flush_dcache_page(kmapped_page);
-					kunmap(kmapped_page);
+					kunmap_local(kaddr);
 					put_arg_page(kmapped_page);
 				}
 				kmapped_page = page;
-				kaddr = kmap(kmapped_page);
+				kaddr = kmap_local_page(kmapped_page);
 				kpos = pos & PAGE_MASK;
 				flush_arg_page(bprm, kpos, kmapped_page);
 			}
@@ -602,7 +602,7 @@  static int copy_strings(int argc, struct user_arg_ptr argv,
 out:
 	if (kmapped_page) {
 		flush_dcache_page(kmapped_page);
-		kunmap(kmapped_page);
+		kunmap_local(kaddr);
 		put_arg_page(kmapped_page);
 	}
 	return ret;
@@ -880,11 +880,11 @@  int transfer_args_to_stack(struct linux_binprm *bprm,
 
 	for (index = MAX_ARG_PAGES - 1; index >= stop; index--) {
 		unsigned int offset = index == stop ? bprm->p & ~PAGE_MASK : 0;
-		char *src = kmap(bprm->page[index]) + offset;
+		char *src = kmap_local_page(bprm->page[index]) + offset;
 		sp -= PAGE_SIZE - offset;
 		if (copy_to_user((void *) sp, src, PAGE_SIZE - offset) != 0)
 			ret = -EFAULT;
-		kunmap(bprm->page[index]);
+		kunmap_local(src);
 		if (ret)
 			goto out;
 	}
@@ -1683,13 +1683,13 @@  int remove_arg_zero(struct linux_binprm *bprm)
 			ret = -EFAULT;
 			goto out;
 		}
-		kaddr = kmap_atomic(page);
+		kaddr = kmap_local_page(page);
 
 		for (; offset < PAGE_SIZE && kaddr[offset];
 				offset++, bprm->p++)
 			;
 
-		kunmap_atomic(kaddr);
+		kunmap_local(kaddr);
 		put_arg_page(page);
 	} while (offset == PAGE_SIZE);