[v2,0/4] export/modpost: avoid renaming __ksymtab entries for symbol namespaces
diff mbox

Message ID 20191018093143.15997-1-maennich@google.com
State New
Headers show

Commit Message

Matthias Maennich Oct. 18, 2019, 9:31 a.m. UTC
The introduction of the symbol namespace patches changed the way symbols are
named in the ksymtab entries. That caused userland tools to fail (such as
kmod's depmod). As depmod is used as part of the kernel build it was worth
having another look whether this name change can be avoided.

The main purpose of this series is to restore the original ksymtab entry names.
For that to happen and to remove some rough edges around that, the relevant
parts in modpost got a small refactoring as to when and how namespaces are
evaluated and set in the symbol struct.

Eventually, the namespace values can be read from __kstrtabns_ entries and
their corresponding __ksymtab_strings values. That removes the need to carry
the namespace names within the (anyway unique) symbol name entries.

The last patch of this series is adopted from Masahiro [1]. By allowing 'no
namespace' to be represented as empty string, large chunks of
include/linux/export.h could be consolidated. Technically, this last patch is
not absolutely necessary to fix functionality. It addresses concerns about
maintainability and readability. While I strongly suggest sending all of the
patches for 5.4, the last one could possible deferred to the next merge window.

This patch applies to the modules-linus [2] branch.

Changes since v2:
 - restored correct authorship for [4/4]
 - add missing contributor tags
 - fixed typos and code style (spaces/tabs)

[1] https://lore.kernel.org/lkml/20190927093603.9140-5-yamada.masahiro@socionext.com/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux.git/log/?h=modules-linus

Cc: Jessica Yu <jeyu@kernel.org>
Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
Cc: Martijn Coenen <maco@android.com>
Cc: Lucas De Marchi <lucas.de.marchi@gmail.com>
Cc: Shaun Ruffell <sruffell@sruffell.net>
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Will Deacon <will@kernel.org>
Cc: linux-kbuild@vger.kernel.org
Cc: linux-modules@vger.kernel.org


Masahiro Yamada (1):
  export: avoid code duplication in include/linux/export.h

Matthias Maennich (3):
  modpost: delegate updating namespaces to separate function
  modpost: make updating the symbol namespace explicit
  symbol namespaces: revert to previous __ksymtab name scheme

 include/linux/export.h | 97 +++++++++++++-----------------------------
 kernel/module.c        |  2 +-
 scripts/mod/modpost.c  | 59 ++++++++++++++++---------
 scripts/mod/modpost.h  |  1 +
 4 files changed, 71 insertions(+), 88 deletions(-)

Interdiff against v1:

Comments

Jessica Yu Oct. 21, 2019, 1:31 p.m. UTC | #1
+++ Matthias Maennich [18/10/19 10:31 +0100]:
>The introduction of the symbol namespace patches changed the way symbols are
>named in the ksymtab entries. That caused userland tools to fail (such as
>kmod's depmod). As depmod is used as part of the kernel build it was worth
>having another look whether this name change can be avoided.
>
>The main purpose of this series is to restore the original ksymtab entry names.
>For that to happen and to remove some rough edges around that, the relevant
>parts in modpost got a small refactoring as to when and how namespaces are
>evaluated and set in the symbol struct.
>
>Eventually, the namespace values can be read from __kstrtabns_ entries and
>their corresponding __ksymtab_strings values. That removes the need to carry
>the namespace names within the (anyway unique) symbol name entries.
>
>The last patch of this series is adopted from Masahiro [1]. By allowing 'no
>namespace' to be represented as empty string, large chunks of
>include/linux/export.h could be consolidated. Technically, this last patch is
>not absolutely necessary to fix functionality. It addresses concerns about
>maintainability and readability. While I strongly suggest sending all of the
>patches for 5.4, the last one could possible deferred to the next merge window.
>
>This patch applies to the modules-linus [2] branch.

Hi!

I've applied the first three patches to modules-linus, inbound for -rc5.

Patch 4/4 was agreed to be for 5.5, and will be applied to
modules-next once I rebase that branch against -rc5.

Thanks!

Jessica

>Changes since v2:
> - restored correct authorship for [4/4]
> - add missing contributor tags
> - fixed typos and code style (spaces/tabs)
>
>[1] https://lore.kernel.org/lkml/20190927093603.9140-5-yamada.masahiro@socionext.com/
>[2] https://git.kernel.org/pub/scm/linux/kernel/git/jeyu/linux.git/log/?h=modules-linus
>
>Cc: Jessica Yu <jeyu@kernel.org>
>Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
>Cc: Martijn Coenen <maco@android.com>
>Cc: Lucas De Marchi <lucas.de.marchi@gmail.com>
>Cc: Shaun Ruffell <sruffell@sruffell.net>
>Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>Cc: Will Deacon <will@kernel.org>
>Cc: linux-kbuild@vger.kernel.org
>Cc: linux-modules@vger.kernel.org
>
>
>Masahiro Yamada (1):
>  export: avoid code duplication in include/linux/export.h
>
>Matthias Maennich (3):
>  modpost: delegate updating namespaces to separate function
>  modpost: make updating the symbol namespace explicit
>  symbol namespaces: revert to previous __ksymtab name scheme
>
> include/linux/export.h | 97 +++++++++++++-----------------------------
> kernel/module.c        |  2 +-
> scripts/mod/modpost.c  | 59 ++++++++++++++++---------
> scripts/mod/modpost.h  |  1 +
> 4 files changed, 71 insertions(+), 88 deletions(-)
>
>Interdiff against v1:
>diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
>index 7cf0065ac95f..0bf7eab80d9f 100644
>--- a/scripts/mod/modpost.c
>+++ b/scripts/mod/modpost.c
>@@ -357,18 +357,21 @@ static const char *namespace_from_kstrtabns(struct elf_info *info,
>
> static void sym_update_namespace(const char *symname, const char *namespace)
> {
>-       struct symbol *s = find_symbol(symname);
>-       /* That symbol should have been created earlier and thus this is
>-        * actually an assertion. */
>-       if (!s) {
>-               merror("Could not update namespace(%s) for symbol %s\n",
>-                      namespace, symname);
>-               return;
>-       }
>-
>-       free(s->namespace);
>-       s->namespace =
>-	       namespace && namespace[0] ? NOFAIL(strdup(namespace)) : NULL;
>+	struct symbol *s = find_symbol(symname);
>+
>+	/*
>+	 * That symbol should have been created earlier and thus this is
>+	 * actually an assertion.
>+	 */
>+	if (!s) {
>+		merror("Could not update namespace(%s) for symbol %s\n",
>+		       namespace, symname);
>+		return;
>+	}
>+
>+	free(s->namespace);
>+	s->namespace =
>+		namespace && namespace[0] ? NOFAIL(strdup(namespace)) : NULL;
> }
>
> /**
>-- 
>2.23.0.866.gb869b98d4c-goog
>
Luis Chamberlain Oct. 23, 2019, 12:22 p.m. UTC | #2
On Fri, Oct 18, 2019 at 10:31:39AM +0100, Matthias Maennich wrote:
> The introduction of the symbol namespace patches changed the way symbols are
> named in the ksymtab entries. That caused userland tools to fail (such as
> kmod's depmod). As depmod is used as part of the kernel build it was worth
> having another look whether this name change can be avoided.

Why have this as a default feature? What about having an option to
disable this feature? The benefit being that without a full swing of
tests to avoid regressions its not clear what other issues may creep
up. With this as optional, those wanting the mechanism can enable it
and happilly find the issues for those more conservative.

  Luis
Matthias Maennich Oct. 24, 2019, 9:35 a.m. UTC | #3
On Wed, Oct 23, 2019 at 12:22:22PM +0000, Luis Chamberlain wrote:
>On Fri, Oct 18, 2019 at 10:31:39AM +0100, Matthias Maennich wrote:
>> The introduction of the symbol namespace patches changed the way symbols are
>> named in the ksymtab entries. That caused userland tools to fail (such as
>> kmod's depmod). As depmod is used as part of the kernel build it was worth
>> having another look whether this name change can be avoided.
>
>Why have this as a default feature? What about having an option to
>disable this feature? The benefit being that without a full swing of
>tests to avoid regressions its not clear what other issues may creep
>up. With this as optional, those wanting the mechanism can enable it
>and happilly find the issues for those more conservative.

The strongest argument against that is, that the 'conservative' people
would constantly break things for the more 'adventurous' ones. They
would introduce namespace requirements by just using symbols without
correctly adjusting the imports.

Second, vmlinux and modules would have to be compiled in the same
configuration. Otherwise they are incompatible and we would likely have
to maintain code in the module loader to catch issues caused by that.
In general, I think for the adoption of this feature and one of its
purposes - making unexpected use of symbols across the tree visible
already at review time - we should not make this an optional one.
Enforcing the imports at module load time is optional (there is an
option).

And finally, having that code configurable for both options introduces
quite some complexity in kernel/module.c, modpost and
include/linux/export.h that would make the code hard to maintain and
complex to test. Hence that would likely introduce more issues.

I know the feature came with some rough edges. Sorry about that. I
think, we got most of them worked out pretty well (big thanks to
Masahiro and Jessica and others helping with that). Now the actual
change to the surface exposed to userland tools is much smaller and the
feature itself less intrusive.

Cheers,
Matthias

>
>  Luis
Luis Chamberlain Oct. 24, 2019, 10:24 a.m. UTC | #4
On Thu, Oct 24, 2019 at 10:35:46AM +0100, Matthias Maennich wrote:
> On Wed, Oct 23, 2019 at 12:22:22PM +0000, Luis Chamberlain wrote:
> > On Fri, Oct 18, 2019 at 10:31:39AM +0100, Matthias Maennich wrote:
> > > The introduction of the symbol namespace patches changed the way symbols are
> > > named in the ksymtab entries. That caused userland tools to fail (such as
> > > kmod's depmod). As depmod is used as part of the kernel build it was worth
> > > having another look whether this name change can be avoided.
> > 
> > Why have this as a default feature? What about having an option to
> > disable this feature? The benefit being that without a full swing of
> > tests to avoid regressions its not clear what other issues may creep
> > up. With this as optional, those wanting the mechanism can enable it
> > and happilly find the issues for those more conservative.
> 
> The strongest argument against that is, that the 'conservative' people
> would constantly break things for the more 'adventurous' ones. They
> would introduce namespace requirements by just using symbols without
> correctly adjusting the imports.
> 
> Second, vmlinux and modules would have to be compiled in the same
> configuration. Otherwise they are incompatible and we would likely have
> to maintain code in the module loader to catch issues caused by that.
> In general, I think for the adoption of this feature and one of its
> purposes - making unexpected use of symbols across the tree visible
> already at review time - we should not make this an optional one.
> Enforcing the imports at module load time is optional (there is an
> option).
> 
> And finally, having that code configurable for both options introduces
> quite some complexity in kernel/module.c, modpost and
> include/linux/export.h that would make the code hard to maintain and
> complex to test. Hence that would likely introduce more issues.
> 
> I know the feature came with some rough edges. Sorry about that. I
> think, we got most of them worked out pretty well (big thanks to
> Masahiro and Jessica and others helping with that). Now the actual
> change to the surface exposed to userland tools is much smaller and the
> feature itself less intrusive.

This logic makes sense, the complexity over module loading is already
high and supporting yet another division would be a burden for review
and maintenace.

However I'd feel much more inclined to support such decisions when and if
we had a series of test cases to prevent possible regressions. Since
effort with testing will move forward, I'm happy with the status quo.

  Luis

Patch
diff mbox

diff --git a/scripts/mod/modpost.c b/scripts/mod/modpost.c
index 7cf0065ac95f..0bf7eab80d9f 100644
--- a/scripts/mod/modpost.c
+++ b/scripts/mod/modpost.c
@@ -357,18 +357,21 @@  static const char *namespace_from_kstrtabns(struct elf_info *info,
 
 static void sym_update_namespace(const char *symname, const char *namespace)
 {
-       struct symbol *s = find_symbol(symname);
-       /* That symbol should have been created earlier and thus this is
-        * actually an assertion. */
-       if (!s) {
-               merror("Could not update namespace(%s) for symbol %s\n",
-                      namespace, symname);
-               return;
-       }
-
-       free(s->namespace);
-       s->namespace =
-	       namespace && namespace[0] ? NOFAIL(strdup(namespace)) : NULL;
+	struct symbol *s = find_symbol(symname);
+
+	/*
+	 * That symbol should have been created earlier and thus this is
+	 * actually an assertion.
+	 */
+	if (!s) {
+		merror("Could not update namespace(%s) for symbol %s\n",
+		       namespace, symname);
+		return;
+	}
+
+	free(s->namespace);
+	s->namespace =
+		namespace && namespace[0] ? NOFAIL(strdup(namespace)) : NULL;
 }
 
 /**