| Message ID | ZUKxT1CL9/0Dn6NE@bombadil.infradead.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL] Modules changes for v6.7-rc1 | expand |
On Wed, 1 Nov 2023 at 10:13, Luis Chamberlain <mcgrof@kernel.org> wrote: > > The only thing worth highligthing is that gzip moves to use vmalloc() instead of > kmalloc just as we had a fix for this for zstd on v6.6-rc1. Actually, that's almost certainly entirely the wrong thing to do. Unless you *know* that the allocation is large, you shouldn't use vmalloc(). And since kmalloc() has worked fine, you most definitely don't know that. So we have 'kvmalloc()' *exactly* for this reason, which is a "use kmalloc, unless that is too small, then use vmalloc". kmalloc() isn't just about "use physically contiguous allocations". It's also more memory-efficient, and a *lot* faster than vmalloc(), which has to play VM tricks. So this "just switch to vmalloc()" is entirely wrong. Linus
On Wed, 1 Nov 2023 at 21:02, Linus Torvalds <torvalds@linux-foundation.org> wrote: > > kmalloc() isn't just about "use physically contiguous allocations". > It's also more memory-efficient, and a *lot* faster than vmalloc(), > which has to play VM tricks. I've pulled this, but I think you should do something like the attached (UNTESTED!) patch. Linus
On Wed, Nov 01, 2023 at 09:02:51PM -1000, Linus Torvalds wrote: > On Wed, 1 Nov 2023 at 10:13, Luis Chamberlain <mcgrof@kernel.org> wrote: > > > > The only thing worth highligthing is that gzip moves to use vmalloc() instead of > > kmalloc just as we had a fix for this for zstd on v6.6-rc1. > > Actually, that's almost certainly entirely the wrong thing to do. > > Unless you *know* that the allocation is large, you shouldn't use > vmalloc(). And since kmalloc() has worked fine, you most definitely > don't know that. > > So we have 'kvmalloc()' *exactly* for this reason, which is a "use > kmalloc, unless that is too small, then use vmalloc". > > kmalloc() isn't just about "use physically contiguous allocations". > It's also more memory-efficient, and a *lot* faster than vmalloc(), > which has to play VM tricks. > > So this "just switch to vmalloc()" is entirely wrong. > > Linus I proposed that change mostlfy for consistency with the zstd case, but I haven't experience any issue with gzip compressed modules (that seem to require less memory, even with larger modules). So, yes, it probably makes sense to drop this change for now and I can send another patch to switch to kvmalloc() for all the decompress cases. Thanks, -Andrea
The pull request you sent on Wed, 1 Nov 2023 13:13:03 -0700:
> git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/modules-6.7-rc1
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/21e80f3841c01aeaf32d7aee7bbc87b3db1aa0c6
Thank you!
On Wed, Nov 01, 2023 at 09:21:09PM -1000, Linus Torvalds wrote: > On Wed, 1 Nov 2023 at 21:02, Linus Torvalds > <torvalds@linux-foundation.org> wrote: > > > > kmalloc() isn't just about "use physically contiguous allocations". > > It's also more memory-efficient, and a *lot* faster than vmalloc(), > > which has to play VM tricks. > > I've pulled this, but I think you should do something like the > attached (UNTESTED!) patch. > > Linus Looks good to me, I'll give it a try ASAP. -Andrea > kernel/module/decompress.c | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > diff --git a/kernel/module/decompress.c b/kernel/module/decompress.c > index 4156d59be440..474e68f0f063 100644 > --- a/kernel/module/decompress.c > +++ b/kernel/module/decompress.c > @@ -100,7 +100,7 @@ static ssize_t module_gzip_decompress(struct load_info *info, > s.next_in = buf + gzip_hdr_len; > s.avail_in = size - gzip_hdr_len; > > - s.workspace = vmalloc(zlib_inflate_workspacesize()); > + s.workspace = kvmalloc(zlib_inflate_workspacesize(), GFP_KERNEL); > if (!s.workspace) > return -ENOMEM; > > @@ -138,7 +138,7 @@ static ssize_t module_gzip_decompress(struct load_info *info, > out_inflate_end: > zlib_inflateEnd(&s); > out: > - vfree(s.workspace); > + kvfree(s.workspace); > return retval; > } > #elif defined(CONFIG_MODULE_COMPRESS_XZ) > @@ -241,7 +241,7 @@ static ssize_t module_zstd_decompress(struct load_info *info, > } > > wksp_size = zstd_dstream_workspace_bound(header.windowSize); > - wksp = vmalloc(wksp_size); > + wksp = kvmalloc(wksp_size, GFP_KERNEL); > if (!wksp) { > retval = -ENOMEM; > goto out; > @@ -284,7 +284,7 @@ static ssize_t module_zstd_decompress(struct load_info *info, > retval = new_size; > > out: > - vfree(wksp); > + kvfree(wksp); > return retval; > } > #else
On Thu, Nov 02, 2023 at 08:29:17AM +0100, Andrea Righi wrote: > On Wed, Nov 01, 2023 at 09:21:09PM -1000, Linus Torvalds wrote: > > On Wed, 1 Nov 2023 at 21:02, Linus Torvalds > > <torvalds@linux-foundation.org> wrote: > > > > > > kmalloc() isn't just about "use physically contiguous allocations". > > > It's also more memory-efficient, and a *lot* faster than vmalloc(), > > > which has to play VM tricks. > > > > I've pulled this, but I think you should do something like the > > attached (UNTESTED!) patch. > > > > Linus > > Looks good to me, I'll give it a try ASAP. > > -Andrea Just tested this both with zstd and gzip module compression, all good. You can add my: Tested-by: Andrea Righi <andrea.righi@canonical.com> Or if you need a proper paperwork: -- From: Andrea Righi <andrea.righi@canonical.com> Subject: [PATCH] module/decompress: use kvmalloc() consistently We consistently switched from kmalloc() to vmalloc() in module decompression to prevent potential memory allocation failures with large modules, however vmalloc() is not as memory-efficient and fast as kmalloc(). Since we don't know in general the size of the workspace required by the decompression algorithm, it is more reasonable to use kvmalloc() consistently, also considering that we don't have special memory requirements here. Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org> Tested-by: Andrea Righi <andrea.righi@canonical.com> Signed-off-by: Andrea Righi <andrea.righi@canonical.com> --- kernel/module/decompress.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/kernel/module/decompress.c b/kernel/module/decompress.c index 4156d59be440..474e68f0f063 100644 --- a/kernel/module/decompress.c +++ b/kernel/module/decompress.c @@ -100,7 +100,7 @@ static ssize_t module_gzip_decompress(struct load_info *info, s.next_in = buf + gzip_hdr_len; s.avail_in = size - gzip_hdr_len; - s.workspace = vmalloc(zlib_inflate_workspacesize()); + s.workspace = kvmalloc(zlib_inflate_workspacesize(), GFP_KERNEL); if (!s.workspace) return -ENOMEM; @@ -138,7 +138,7 @@ static ssize_t module_gzip_decompress(struct load_info *info, out_inflate_end: zlib_inflateEnd(&s); out: - vfree(s.workspace); + kvfree(s.workspace); return retval; } #elif defined(CONFIG_MODULE_COMPRESS_XZ) @@ -241,7 +241,7 @@ static ssize_t module_zstd_decompress(struct load_info *info, } wksp_size = zstd_dstream_workspace_bound(header.windowSize); - wksp = vmalloc(wksp_size); + wksp = kvmalloc(wksp_size, GFP_KERNEL); if (!wksp) { retval = -ENOMEM; goto out; @@ -284,7 +284,7 @@ static ssize_t module_zstd_decompress(struct load_info *info, retval = new_size; out: - vfree(wksp); + kvfree(wksp); return retval; } #else
The following changes since commit 7d461b291e65938f15f56fe58da2303b07578a76: Merge tag 'drm-next-2023-10-31-1' of git://anongit.freedesktop.org/drm/drm (2023-11-01 06:28:35 -1000) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/modules-6.7-rc1 for you to fetch changes up to ea0b0bcef4917a2640ecc100c768b8e785784834: module: Annotate struct module_notes_attrs with __counted_by (2023-11-01 13:07:32 -0700) ---------------------------------------------------------------- Modules changes for v6.7-rc1 The only thing worth highligthing is that gzip moves to use vmalloc() instead of kmalloc just as we had a fix for this for zstd on v6.6-rc1. The rest is regular house keeping, keeping things neat, tidy, and boring. Oh and this has been on linux-next for over a month. ---------------------------------------------------------------- Andrea Righi (1): module/decompress: use vmalloc() for gzip decompression workspace Kees Cook (2): module: Clarify documentation of module_param_call() module: Annotate struct module_notes_attrs with __counted_by Luis Chamberlain (1): MAINTAINERS: add include/linux/module*.h to modules Tiezhu Yang (2): module: Make is_mapping_symbol() return bool module: Make is_valid_name() return bool Zhu Mao (1): module: Fix comment typo MAINTAINERS | 2 +- include/linux/module_symbol.h | 2 +- include/linux/moduleparam.h | 6 +++++- kernel/module/decompress.c | 4 ++-- kernel/module/stats.c | 2 +- kernel/module/sysfs.c | 2 +- scripts/mod/modpost.c | 4 ++-- 7 files changed, 13 insertions(+), 9 deletions(-)