From patchwork Wed Aug 21 11:49:23 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: =?utf-8?q?Matthias_M=C3=A4nnich?= X-Patchwork-Id: 11106489 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 734A71864 for ; Wed, 21 Aug 2019 11:54:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 50B15233FF for ; Wed, 21 Aug 2019 11:54:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="mutT78ZX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727435AbfHULy4 (ORCPT ); Wed, 21 Aug 2019 07:54:56 -0400 Received: from mail-pg1-f202.google.com ([209.85.215.202]:51952 "EHLO mail-pg1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728086AbfHULy4 (ORCPT ); Wed, 21 Aug 2019 07:54:56 -0400 Received: by mail-pg1-f202.google.com with SMTP id c9so1117060pgm.18 for ; Wed, 21 Aug 2019 04:54:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=KsiMtetsi+HhY2srbiOwM0cOvpdAc3wAuabal/+OVi0=; b=mutT78ZXlUyLbX1O8lAIS2dxK+tHhjG4ikLhmlNus3sBNCCUZxpCRbQnFDpnTqcDTt ON9QuBOc1VrLnzQrLbpFxi2aQsazF49Y4r9tqTJ98LBw/NRdGYpgh2NrfQarx1FEGrGf phbPc/h52Iux0whH3oQxKBAQycUEXKg5y4k8QGExq6pzuj7qN6zAEkvi3038YUC5wW+0 blmdeWExzwfLCwrd+MJxQZLdCz/9KzRpVEx0LBaejgNNF/wOs/HjkOt7tRct2IZ5On+V FX2YvZVtgXevEeF1fZOAJYzggU2PC1EHvyoppIWTVPnHFJEUwXJqW9ml+vdbAOA0M9hO EFqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=KsiMtetsi+HhY2srbiOwM0cOvpdAc3wAuabal/+OVi0=; b=WAhbsa9EzYsdGfhi+b97K0UJ85eVHGSf9TtjJGXzjkDPN94QncHc4VkjgSk7UyPFM6 Pn9l7zTdcvf8exicXiPEi9yXybaxwQdvQJcR145c/NVj1jULPEcq0BIFNFUuwGOLFFZM DSawxuFYLLJWvMQ6KSAQkfY1TlRJsVQ9o1pkSexooBQw2PyJggJkiIbL8ezY1OwOY4Ah 22hXRUSh4MG8Femj/Sf7cX0ZMc/xN0cZjMpxz3gSxKaiwHENpTq8cEsbSE68y+TphTct r591RQj0HUBhy7+xW2qge+s5rqwANd2+SsZppy1PtxdF1c2OMFovBMscyTEX4DuU+/Qd UIhg== X-Gm-Message-State: APjAAAXJ6C7+8eoEfKk3ptTcPtfQHrXi8gwZz0AhErWwMpjt3HWuXYbp OBKHW3K/mNUumH1FdYM9FsdscPKT43wi9g== X-Google-Smtp-Source: APXvYqxqNAwircYBl6VHLZSUXbQpQkN6h8o+zPQfAF2yPdOhRMJYlEyLJzz6sxiW447WxjGKmodGax2sMqbunw== X-Received: by 2002:a63:9e56:: with SMTP id r22mr28815115pgo.221.1566388494681; Wed, 21 Aug 2019 04:54:54 -0700 (PDT) Date: Wed, 21 Aug 2019 12:49:23 +0100 In-Reply-To: <20190821114955.12788-1-maennich@google.com> Message-Id: <20190821114955.12788-9-maennich@google.com> Mime-Version: 1.0 References: <20190813121733.52480-1-maennich@google.com> <20190821114955.12788-1-maennich@google.com> X-Mailer: git-send-email 2.23.0.rc1.153.gdeed80330f-goog Subject: [PATCH v3 08/11] scripts: Coccinelle script for namespace dependencies. From: Matthias Maennich To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, maennich@google.com, arnd@arndb.de, geert@linux-m68k.org, gregkh@linuxfoundation.org, hpa@zytor.com, jeyu@kernel.org, joel@joelfernandes.org, kstewart@linuxfoundation.org, linux-arch@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-modules@vger.kernel.org, linux-scsi@vger.kernel.org, linux-usb@vger.kernel.org, lucas.de.marchi@gmail.com, maco@android.com, maco@google.com, michal.lkml@markovi.net, mingo@redhat.com, oneukum@suse.com, pombredanne@nexb.com, sam@ravnborg.org, sspatil@google.com, stern@rowland.harvard.edu, tglx@linutronix.de, usb-storage@lists.one-eyed-alien.net, x86@kernel.org, yamada.masahiro@socionext.com, Julia Lawall , Julia Lawall , Gilles Muller , Nicolas Palix , Mauro Carvalho Chehab , "David S. Miller" , Nicolas Ferre , cocci@systeme.lip6.fr Sender: linux-usb-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-usb@vger.kernel.org A script that uses the '.ns_deps' files generated by modpost to automatically add the required symbol namespace dependencies to each module. Usage: 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define DEFAULT_SYMBOL_NAMESPACE 2) Run 'make' (or 'make modules') and get warnings about modules not importing that namespace. 3) Run 'make nsdeps' to automatically add required import statements to said modules. This makes it easer for subsystem maintainers to introduce and maintain symbol namespaces into their codebase. Co-developed-by: Martijn Coenen Signed-off-by: Martijn Coenen Acked-by: Julia Lawall Reviewed-by: Greg Kroah-Hartman Signed-off-by: Matthias Maennich --- MAINTAINERS | 5 ++ Makefile | 12 +++++ scripts/Makefile.modpost | 4 +- scripts/coccinelle/misc/add_namespace.cocci | 23 +++++++++ scripts/nsdeps | 56 +++++++++++++++++++++ 5 files changed, 99 insertions(+), 1 deletion(-) create mode 100644 scripts/coccinelle/misc/add_namespace.cocci create mode 100644 scripts/nsdeps diff --git a/MAINTAINERS b/MAINTAINERS index 08176d64eed5..dd5b37b49a07 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -11428,6 +11428,11 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git F: tools/include/nolibc/ +NSDEPS +M: Matthias Maennich +S: Maintained +F: scripts/nsdeps + NTB AMD DRIVER M: Shyam Sundar S K L: linux-ntb@googlegroups.com diff --git a/Makefile b/Makefile index a89870188c09..40311f583ee1 100644 --- a/Makefile +++ b/Makefile @@ -1500,6 +1500,9 @@ help: @echo ' headerdep - Detect inclusion cycles in headers' @echo ' coccicheck - Check with Coccinelle' @echo '' + @echo 'Tools:' + @echo ' nsdeps - Generate missing symbol namespace dependencies' + @echo '' @echo 'Kernel selftest:' @echo ' kselftest - Build and run kernel selftest (run as root)' @echo ' Build, install, and boot kernel before' @@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN $@ tags TAGS cscope gtags: FORCE $(call cmd,tags) +# Script to generate missing namespace dependencies +# --------------------------------------------------------------------------- + +PHONY += nsdeps + +nsdeps: + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps + $(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@ + # Scripts to check various things for consistency # --------------------------------------------------------------------------- diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index 26e6574ecd08..743fe3a2e885 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost \ $(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS))) \ $(if $(KBUILD_EXTMOD),-o $(modulesymfile)) \ $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \ - $(if $(KBUILD_MODPOST_WARN),-w) + $(if $(KBUILD_MODPOST_WARN),-w) \ + $(if $(filter nsdeps,$(MAKECMDGOALS)),-d) ifdef MODPOST_VMLINUX @@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE targets += $(modules) +nsdeps: __modpost # Add FORCE to the prequisites of a target to force it to be always rebuilt. # --------------------------------------------------------------------------- diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci new file mode 100644 index 000000000000..c832bb6445a8 --- /dev/null +++ b/scripts/coccinelle/misc/add_namespace.cocci @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +/// Adds missing MODULE_IMPORT_NS statements to source files +/// +/// This script is usually called from scripts/nsdeps with -D ns= to +/// add a missing namespace tag to a module source file. +/// + +@has_ns_import@ +declarer name MODULE_IMPORT_NS; +identifier virtual.ns; +@@ +MODULE_IMPORT_NS(ns); + +// Add missing imports, but only adjacent to a MODULE_LICENSE statement. +// That ensures we are adding it only to the main module source file. +@do_import depends on !has_ns_import@ +declarer name MODULE_LICENSE; +expression license; +identifier virtual.ns; +@@ +MODULE_LICENSE(license); ++ MODULE_IMPORT_NS(ns); diff --git a/scripts/nsdeps b/scripts/nsdeps new file mode 100644 index 000000000000..3b5995a61e65 --- /dev/null +++ b/scripts/nsdeps @@ -0,0 +1,56 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Linux kernel symbol namespace import generator +# +# This script requires a minimum spatch version. +SPATCH_REQ_VERSION="1.0.4" + +DIR="$(dirname $(readlink -f $0))/.." +SPATCH="`which ${SPATCH:=spatch}`" +if [ ! -x "$SPATCH" ]; then + echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/' + exit 1 +fi + +SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh) +SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}') +SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh) + +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then + echo "spatch needs to be version $SPATCH_REQ_VERSION or higher" + exit 1 +fi + +generate_deps_for_ns() { + $SPATCH --very-quiet --in-place --sp-file \ + $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2 +} + +generate_deps() { + local mod_name=`basename $@ .ko` + local mod_file=`echo $@ | sed -e 's/\.ko/\.mod/'` + local ns_deps_file=`echo $@ | sed -e 's/\.ko/\.ns_deps/'` + if [ ! -f "$ns_deps_file" ]; then return; fi + local mod_source_files=`cat $mod_file | sed -n 1p | sed -e 's/\.o/\.c/g'` + for ns in `cat $ns_deps_file`; do + echo "Adding namespace $ns to module $mod_name (if needed)." + generate_deps_for_ns $ns $mod_source_files + # sort the imports + for source_file in $mod_source_files; do + sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp + offset=$(wc -l ${source_file}.tmp | awk '{print $1;}') + cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp + tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp + if ! diff -q ${source_file} ${source_file}.tmp; then + mv ${source_file}.tmp ${source_file} + else + rm ${source_file}.tmp + fi + done + done +} + +for f in `cat $srctree/modules.order`; do + generate_deps $f +done +