[v6,3/3] firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module

Commit Message

John Stultz Nov. 6, 2020, 4:27 a.m. UTC

Allow the qcom_scm driver to be loadable as a permenent module.

This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to
ensure that drivers that call into the qcom_scm driver are
also built as modules. While not ideal in some cases its the
only safe way I can find to avoid build errors without having
those drivers select QCOM_SCM and have to force it on (as
QCOM_SCM=n can be valid for those drivers).

Cc: Catalin Marinas <[e-mail]>
Cc: Will Deacon <[e-mail]>
Cc: Andy Gross <[e-mail]>
Cc: Bjorn Andersson <[e-mail]>
Cc: Joerg Roedel <[e-mail]>
Cc: Thomas Gleixner <[e-mail]>
Cc: Jason Cooper <[e-mail]>
Cc: Marc Zyngier <[e-mail]>
Cc: Linus Walleij <[e-mail]>
Cc: Vinod Koul <[e-mail]>
Cc: Kalle Valo <[e-mail]>
Cc: Maulik Shah <[e-mail]>
Cc: Lina Iyer <[e-mail]>
Cc: Saravana Kannan <[e-mail]>
Cc: Todd Kjos <[e-mail]>
Cc: Greg Kroah-Hartman <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Acked-by: Kalle Valo <[e-mail]>
Acked-by: Greg Kroah-Hartman <[e-mail]>
Reviewed-by: Bjorn Andersson <[e-mail]>
Signed-off-by: John Stultz <[e-mail]>
---
v3:
* Fix __arm_smccc_smc build issue reported by
  kernel test robot <[e-mail]>
v4:
* Add "depends on QCOM_SCM || !QCOM_SCM" bit to ath10k
  config that requires it.
v5:
* Fix QCOM_QCM typo in Kconfig, it should be QCOM_SCM
---
 drivers/firmware/Kconfig                | 4 ++--
 drivers/firmware/Makefile               | 3 ++-
 drivers/firmware/qcom_scm.c             | 4 ++++
 drivers/iommu/Kconfig                   | 2 ++
 drivers/net/wireless/ath/ath10k/Kconfig | 1 +
 5 files changed, 11 insertions(+), 3 deletions(-)

Comments

Linus Walleij Nov. 10, 2020, 1:35 p.m. UTC | #1

On Fri, Nov 6, 2020 at 5:27 AM John Stultz <[e-mail]> wrote:

> Allow the qcom_scm driver to be loadable as a permenent module.
>
> This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to
> ensure that drivers that call into the qcom_scm driver are
> also built as modules. While not ideal in some cases its the
> only safe way I can find to avoid build errors without having
> those drivers select QCOM_SCM and have to force it on (as
> QCOM_SCM=n can be valid for those drivers).
>
> Cc: Catalin Marinas <[e-mail]>
> Cc: Will Deacon <[e-mail]>
> Cc: Andy Gross <[e-mail]>
> Cc: Bjorn Andersson <[e-mail]>
> Cc: Joerg Roedel <[e-mail]>
> Cc: Thomas Gleixner <[e-mail]>
> Cc: Jason Cooper <[e-mail]>
> Cc: Marc Zyngier <[e-mail]>
> Cc: Linus Walleij <[e-mail]>
> Cc: Vinod Koul <[e-mail]>
> Cc: Kalle Valo <[e-mail]>
> Cc: Maulik Shah <[e-mail]>
> Cc: Lina Iyer <[e-mail]>
> Cc: Saravana Kannan <[e-mail]>
> Cc: Todd Kjos <[e-mail]>
> Cc: Greg Kroah-Hartman <[e-mail]>
> Cc: [e-mail]
> Cc: [e-mail]
> Cc: [e-mail]
> Acked-by: Kalle Valo <[e-mail]>
> Acked-by: Greg Kroah-Hartman <[e-mail]>
> Reviewed-by: Bjorn Andersson <[e-mail]>
> Signed-off-by: John Stultz <[e-mail]>

I applied this patch to the pinctrl tree as well, I suppose
that was the intention. If someone gets upset I can always
pull it out.

Thanks for your perseverance in driving this John.

Yours,
Linus Walleij

John Stultz Nov. 10, 2020, 6:51 p.m. UTC | #2

On Tue, Nov 10, 2020 at 5:35 AM Linus Walleij <[e-mail]> wrote:
> On Fri, Nov 6, 2020 at 5:27 AM John Stultz <[e-mail]> wrote:
>
> > Allow the qcom_scm driver to be loadable as a permenent module.
> >
...
> I applied this patch to the pinctrl tree as well, I suppose
> that was the intention. If someone gets upset I can always
> pull it out.

Will: You ok with this?

thanks
-john

Will Deacon Nov. 12, 2020, 5:37 p.m. UTC | #3

On Tue, Nov 10, 2020 at 10:51:46AM -0800, John Stultz wrote:
> On Tue, Nov 10, 2020 at 5:35 AM Linus Walleij <[e-mail]> wrote:
> > On Fri, Nov 6, 2020 at 5:27 AM John Stultz <[e-mail]> wrote:
> >
> > > Allow the qcom_scm driver to be loadable as a permenent module.
> > >
> ...
> > I applied this patch to the pinctrl tree as well, I suppose
> > that was the intention. If someone gets upset I can always
> > pull it out.
> 
> Will: You ok with this?

We didn't come up with something better, so I can live with it. Not sure
about the otehr issues that were reported by Robin though -- your RFC for
fixing those looked a bit more controversial ;)

Will

John Stultz Nov. 12, 2020, 7:17 p.m. UTC | #4

On Thu, Nov 12, 2020 at 9:37 AM Will Deacon <[e-mail]> wrote:
> On Tue, Nov 10, 2020 at 10:51:46AM -0800, John Stultz wrote:
> > On Tue, Nov 10, 2020 at 5:35 AM Linus Walleij <[e-mail]> wrote:
> > > On Fri, Nov 6, 2020 at 5:27 AM John Stultz <[e-mail]> wrote:
> > >
> > > > Allow the qcom_scm driver to be loadable as a permenent module.
> > > >
> > ...
> > > I applied this patch to the pinctrl tree as well, I suppose
> > > that was the intention. If someone gets upset I can always
> > > pull it out.
> >
> > Will: You ok with this?
>
> We didn't come up with something better, so I can live with it.

Ok, thanks!

> Not sure
> about the otehr issues that were reported by Robin though -- your RFC for
> fixing those looked a bit more controversial ;)

Huh, I hadn't heard anything back on that series and was going to
resend it. Do let me know if you have more thoughts on that one.

thanks
-john

Thierry Reding Nov. 16, 2020, 3:59 p.m. UTC | #5

On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> Allow the qcom_scm driver to be loadable as a permenent module.
> 
> This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to
> ensure that drivers that call into the qcom_scm driver are
> also built as modules. While not ideal in some cases its the
> only safe way I can find to avoid build errors without having
> those drivers select QCOM_SCM and have to force it on (as
> QCOM_SCM=n can be valid for those drivers).
> 
> Cc: Catalin Marinas <[e-mail]>
> Cc: Will Deacon <[e-mail]>
> Cc: Andy Gross <[e-mail]>
> Cc: Bjorn Andersson <[e-mail]>
> Cc: Joerg Roedel <[e-mail]>
> Cc: Thomas Gleixner <[e-mail]>
> Cc: Jason Cooper <[e-mail]>
> Cc: Marc Zyngier <[e-mail]>
> Cc: Linus Walleij <[e-mail]>
> Cc: Vinod Koul <[e-mail]>
> Cc: Kalle Valo <[e-mail]>
> Cc: Maulik Shah <[e-mail]>
> Cc: Lina Iyer <[e-mail]>
> Cc: Saravana Kannan <[e-mail]>
> Cc: Todd Kjos <[e-mail]>
> Cc: Greg Kroah-Hartman <[e-mail]>
> Cc: [e-mail]
> Cc: [e-mail]
> Cc: [e-mail]
> Acked-by: Kalle Valo <[e-mail]>
> Acked-by: Greg Kroah-Hartman <[e-mail]>
> Reviewed-by: Bjorn Andersson <[e-mail]>
> Signed-off-by: John Stultz <[e-mail]>
> ---
> v3:
> * Fix __arm_smccc_smc build issue reported by
>   kernel test robot <[e-mail]>
> v4:
> * Add "depends on QCOM_SCM || !QCOM_SCM" bit to ath10k
>   config that requires it.
> v5:
> * Fix QCOM_QCM typo in Kconfig, it should be QCOM_SCM
> ---
>  drivers/firmware/Kconfig                | 4 ++--
>  drivers/firmware/Makefile               | 3 ++-
>  drivers/firmware/qcom_scm.c             | 4 ++++
>  drivers/iommu/Kconfig                   | 2 ++
>  drivers/net/wireless/ath/ath10k/Kconfig | 1 +
>  5 files changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
> index 3315e3c215864..5e369928bc567 100644
> --- a/drivers/firmware/Kconfig
> +++ b/drivers/firmware/Kconfig
> @@ -235,8 +235,8 @@ config INTEL_STRATIX10_RSU
>  	  Say Y here if you want Intel RSU support.
>  
>  config QCOM_SCM
> -	bool
> -	depends on ARM || ARM64
> +	tristate "Qcom SCM driver"
> +	depends on (ARM && HAVE_ARM_SMCCC) || ARM64
>  	select RESET_CONTROLLER
>  
>  config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
> diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
> index 5e013b6a3692e..523173cbff335 100644
> --- a/drivers/firmware/Makefile
> +++ b/drivers/firmware/Makefile
> @@ -17,7 +17,8 @@ obj-$(CONFIG_ISCSI_IBFT)	+= iscsi_ibft.o
>  obj-$(CONFIG_FIRMWARE_MEMMAP)	+= memmap.o
>  obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
>  obj-$(CONFIG_FW_CFG_SYSFS)	+= qemu_fw_cfg.o
> -obj-$(CONFIG_QCOM_SCM)		+= qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
> +obj-$(CONFIG_QCOM_SCM)		+= qcom-scm.o
> +qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
>  obj-$(CONFIG_TI_SCI_PROTOCOL)	+= ti_sci.o
>  obj-$(CONFIG_TRUSTED_FOUNDATIONS) += trusted_foundations.o
>  obj-$(CONFIG_TURRIS_MOX_RWTM)	+= turris-mox-rwtm.o
> diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c
> index 7be48c1bec96d..6f431b73e617d 100644
> --- a/drivers/firmware/qcom_scm.c
> +++ b/drivers/firmware/qcom_scm.c
> @@ -1280,6 +1280,7 @@ static const struct of_device_id qcom_scm_dt_match[] = {
>  	{ .compatible = "qcom,scm" },
>  	{}
>  };
> +MODULE_DEVICE_TABLE(of, qcom_scm_dt_match);
>  
>  static struct platform_driver qcom_scm_driver = {
>  	.driver = {
> @@ -1295,3 +1296,6 @@ static int __init qcom_scm_init(void)
>  	return platform_driver_register(&qcom_scm_driver);
>  }
>  subsys_initcall(qcom_scm_init);
> +
> +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. SCM driver");
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
> index 04878caf6da49..c64d7a2b65134 100644
> --- a/drivers/iommu/Kconfig
> +++ b/drivers/iommu/Kconfig
> @@ -248,6 +248,7 @@ config SPAPR_TCE_IOMMU
>  config ARM_SMMU
>  	tristate "ARM Ltd. System MMU (SMMU) Support"
>  	depends on ARM64 || ARM || (COMPILE_TEST && !GENERIC_ATOMIC64)
> +	depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
>  	select IOMMU_API
>  	select IOMMU_IO_PGTABLE_LPAE
>  	select ARM_DMA_USE_IOMMU if ARM

This, in conjunction with deferred probe timeout, causes mayhem on
Tegra186. The problem, as far as I can tell, is that there are various
devices that are hooked up to the ARM SMMU, but if ARM SMMU ends up
being built as a loadable module, then those devices will initialize
without IOMMU support (because deferred probe will timeout before the
ARM SMMU module can be loaded from the root filesystem).

Unfortunately, the ARM SMMU module will eventually end up being loaded
once the root filesystem has been mounted (for example via SDHCI or
Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
initialize, configuring as "fault by default", which then results from a
slew of SMMU faults from all the devices that have previously configured
themselves without IOMMU support.

One way to work around this is to just disable all QCOM-related drivers
for the build so that ARM SMMU will be built-in again. I'm going to
guess that distributions aren't going to be too happy about having to
make that kind of choice.

Another way would be for the ARM SMMU module to be included in the
initial ramdisk, which /should/ solve this as well, though I haven't
actually tested that yet. That's not ideal, because it means that users
will have to use an initial ramdisk in order to make this work, and not
all of them may want to.

Perhaps a better solution for now would be to make QCOM_SCM always
built-in, so that ARM SMMU can also always be built-in? I suspect that
this will be a problem not only on Tegra but on any platform that uses
an ARM SMMU. I think this is also not directly related to the QCOM_SCM
code because this would also happen if ARM SMMU were built as a module
for a kernel that doesn't have any QCOM drivers enabled. So in general
any configuration that builds ARM SMMU as a module seems like it would
currently be broken (if it also keeps the "fault by default" default).
Is this something that people have extensively tested? I can't see how
that would currently work, since there's no way for an ARM SMMU master
to somehow recover and switch to IOMMU-backed DMA API dynamically once
the ARM SMMU becomes available.

I guess yet another possibility would be for the ARM SMMU driver to
detect whether it was loaded after all of its consumers and switch to
"bypass by default" automatically in such a situation. That should allow
any driver probed after the ARM SMMU to still take advantage of IOVA
translation, but will not impact the devices probed before the ARM SMMU.

Thierry

Will Deacon Nov. 16, 2020, 4:36 p.m. UTC | #6

On Mon, Nov 16, 2020 at 04:59:36PM +0100, Thierry Reding wrote:
> On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> > diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c
> > index 7be48c1bec96d..6f431b73e617d 100644
> > --- a/drivers/firmware/qcom_scm.c
> > +++ b/drivers/firmware/qcom_scm.c
> > @@ -1280,6 +1280,7 @@ static const struct of_device_id qcom_scm_dt_match[] = {
> >  	{ .compatible = "qcom,scm" },
> >  	{}
> >  };
> > +MODULE_DEVICE_TABLE(of, qcom_scm_dt_match);
> >  
> >  static struct platform_driver qcom_scm_driver = {
> >  	.driver = {
> > @@ -1295,3 +1296,6 @@ static int __init qcom_scm_init(void)
> >  	return platform_driver_register(&qcom_scm_driver);
> >  }
> >  subsys_initcall(qcom_scm_init);
> > +
> > +MODULE_DESCRIPTION("Qualcomm Technologies, Inc. SCM driver");
> > +MODULE_LICENSE("GPL v2");
> > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
> > index 04878caf6da49..c64d7a2b65134 100644
> > --- a/drivers/iommu/Kconfig
> > +++ b/drivers/iommu/Kconfig
> > @@ -248,6 +248,7 @@ config SPAPR_TCE_IOMMU
> >  config ARM_SMMU
> >  	tristate "ARM Ltd. System MMU (SMMU) Support"
> >  	depends on ARM64 || ARM || (COMPILE_TEST && !GENERIC_ATOMIC64)
> > +	depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
> >  	select IOMMU_API
> >  	select IOMMU_IO_PGTABLE_LPAE
> >  	select ARM_DMA_USE_IOMMU if ARM
> 
> This, in conjunction with deferred probe timeout, causes mayhem on
> Tegra186. The problem, as far as I can tell, is that there are various
> devices that are hooked up to the ARM SMMU, but if ARM SMMU ends up
> being built as a loadable module, then those devices will initialize
> without IOMMU support (because deferred probe will timeout before the
> ARM SMMU module can be loaded from the root filesystem).
> 
> Unfortunately, the ARM SMMU module will eventually end up being loaded
> once the root filesystem has been mounted (for example via SDHCI or
> Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
> initialize, configuring as "fault by default", which then results from a
> slew of SMMU faults from all the devices that have previously configured
> themselves without IOMMU support.

I wonder if fw_devlink=on would help here?

But either way, I'd be more inclined to revert this change if it's causing
problems for !QCOM devices.

Linus -- please can you drop this one (patch 3/3) for now, given that it's
causing problems?

Cheers,

Will

John Stultz Nov. 16, 2020, 7:40 p.m. UTC | #7

On Mon, Nov 16, 2020 at 7:59 AM Thierry Reding <[e-mail]> wrote:
>
> On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> > diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
> > index 04878caf6da49..c64d7a2b65134 100644
> > --- a/drivers/iommu/Kconfig
> > +++ b/drivers/iommu/Kconfig
> > @@ -248,6 +248,7 @@ config SPAPR_TCE_IOMMU
> >  config ARM_SMMU
> >       tristate "ARM Ltd. System MMU (SMMU) Support"
> >       depends on ARM64 || ARM || (COMPILE_TEST && !GENERIC_ATOMIC64)
> > +     depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
> >       select IOMMU_API
> >       select IOMMU_IO_PGTABLE_LPAE
> >       select ARM_DMA_USE_IOMMU if ARM
>
> This, in conjunction with deferred probe timeout, causes mayhem on
> Tegra186. The problem, as far as I can tell, is that there are various
> devices that are hooked up to the ARM SMMU, but if ARM SMMU ends up
> being built as a loadable module, then those devices will initialize
> without IOMMU support (because deferred probe will timeout before the
> ARM SMMU module can be loaded from the root filesystem).
>
> Unfortunately, the ARM SMMU module will eventually end up being loaded
> once the root filesystem has been mounted (for example via SDHCI or
> Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
> initialize, configuring as "fault by default", which then results from a
> slew of SMMU faults from all the devices that have previously configured
> themselves without IOMMU support.

Oof. My apologies for the trouble. Thanks so much for the report.

Out of curiosity, does booting with deferred_probe_timeout=30 avoid
the issue for you?

thanks
-john

John Stultz Nov. 16, 2020, 7:48 p.m. UTC | #8

On Mon, Nov 16, 2020 at 8:36 AM Will Deacon <[e-mail]> wrote:
> On Mon, Nov 16, 2020 at 04:59:36PM +0100, Thierry Reding wrote:
> > On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> > Unfortunately, the ARM SMMU module will eventually end up being loaded
> > once the root filesystem has been mounted (for example via SDHCI or
> > Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
> > initialize, configuring as "fault by default", which then results from a
> > slew of SMMU faults from all the devices that have previously configured
> > themselves without IOMMU support.
>
> I wonder if fw_devlink=on would help here?
>
> But either way, I'd be more inclined to revert this change if it's causing
> problems for !QCOM devices.
>
> Linus -- please can you drop this one (patch 3/3) for now, given that it's
> causing problems?

Agreed. Apologies again for the trouble.

I do feel like the probe timeout to handle optional links is causing a
lot of the trouble here. I expect fw_devlink would solve this, but it
may be awhile before it can be always enabled.  I may see about
pushing the default probe timeout value to be a little further out
than init (I backed away from my last attempt as I didn't want to
cause long (30 second) delays for cases like NFS root, but maybe 2-5
seconds would be enough to make things work better for everyone).

thanks
-john

Thierry Reding Nov. 17, 2020, 1:47 p.m. UTC | #9

On Mon, Nov 16, 2020 at 11:48:39AM -0800, John Stultz wrote:
> On Mon, Nov 16, 2020 at 8:36 AM Will Deacon <[e-mail]> wrote:
> > On Mon, Nov 16, 2020 at 04:59:36PM +0100, Thierry Reding wrote:
> > > On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> > > Unfortunately, the ARM SMMU module will eventually end up being loaded
> > > once the root filesystem has been mounted (for example via SDHCI or
> > > Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
> > > initialize, configuring as "fault by default", which then results from a
> > > slew of SMMU faults from all the devices that have previously configured
> > > themselves without IOMMU support.
> >
> > I wonder if fw_devlink=on would help here?
> >
> > But either way, I'd be more inclined to revert this change if it's causing
> > problems for !QCOM devices.
> >
> > Linus -- please can you drop this one (patch 3/3) for now, given that it's
> > causing problems?
> 
> Agreed. Apologies again for the trouble.
> 
> I do feel like the probe timeout to handle optional links is causing a
> lot of the trouble here. I expect fw_devlink would solve this, but it
> may be awhile before it can be always enabled.  I may see about
> pushing the default probe timeout value to be a little further out
> than init (I backed away from my last attempt as I didn't want to
> cause long (30 second) delays for cases like NFS root, but maybe 2-5
> seconds would be enough to make things work better for everyone).

I think there are two problems here: 1) the deferred probe timeout can
cause a mismatch between what SMMU masters and the SMMU think is going
on and 2) a logistical problem of dealing with the SMMU driver being a
loadable module.

The second problem can be dealt with by shipping the module in the
initial ramdisk. That's a bit annoying, but perhaps the right thing to
do. At least on Tegra we need this because all the devices that carry
the root filesystem (Ethernet for NFS and SDHCI/USB/SATA/PCI for disk
boot) are SMMU masters and will start to fault once the SMMU driver is
loaded.

The first problem is trickier, but if the ARM SMMU driver is built as a
module and shipped in the initial ramdisk it should work. Like I said,
this is annoying because it makes the development a bit more complicated
than just rebuilding a kernel image and flashing it (or boot it straight
from TFTP) because now everytime the ARM SMMU module is built the
initial ramdisk needs to be updated (and potentially flashed) as well.

Thierry

P.S.: Interestingly this is very similar to the problem that I've been
trying to address for display hardware that's left on by the bootloader.
Given that, one potential solution would be to somehow retrieve memory
allocations done by these devices and create identity mappings in the
ARM SMMU address spaces for such devices, much like we plan to do for
devices left on by the bootloader (like the display controller for
showing a boot splash). I suspect that it's not really worth doing this
for devices that are only initialized by the kernel because we have a
bit of control over when we enable them, so I'd prefer if we just kept
things reasonably simple and made sure the SMMU was either always used
by a device from the start or not at all. Dynamically switching between
SMMU and no-SMMU seems a bit eccentric.

Will Deacon Nov. 19, 2020, 9:33 a.m. UTC | #10

On Tue, Nov 17, 2020 at 02:47:54PM +0100, Thierry Reding wrote:
> On Mon, Nov 16, 2020 at 11:48:39AM -0800, John Stultz wrote:
> > On Mon, Nov 16, 2020 at 8:36 AM Will Deacon <[e-mail]> wrote:
> > > On Mon, Nov 16, 2020 at 04:59:36PM +0100, Thierry Reding wrote:
> > > > On Fri, Nov 06, 2020 at 04:27:10AM +0000, John Stultz wrote:
> > > > Unfortunately, the ARM SMMU module will eventually end up being loaded
> > > > once the root filesystem has been mounted (for example via SDHCI or
> > > > Ethernet, both with using just plain, non-IOMMU-backed DMA API) and then
> > > > initialize, configuring as "fault by default", which then results from a
> > > > slew of SMMU faults from all the devices that have previously configured
> > > > themselves without IOMMU support.
> > >
> > > I wonder if fw_devlink=on would help here?
> > >
> > > But either way, I'd be more inclined to revert this change if it's causing
> > > problems for !QCOM devices.
> > >
> > > Linus -- please can you drop this one (patch 3/3) for now, given that it's
> > > causing problems?
> > 
> > Agreed. Apologies again for the trouble.
> > 
> > I do feel like the probe timeout to handle optional links is causing a
> > lot of the trouble here. I expect fw_devlink would solve this, but it
> > may be awhile before it can be always enabled.  I may see about
> > pushing the default probe timeout value to be a little further out
> > than init (I backed away from my last attempt as I didn't want to
> > cause long (30 second) delays for cases like NFS root, but maybe 2-5
> > seconds would be enough to make things work better for everyone).
> 
> I think there are two problems here: 1) the deferred probe timeout can
> cause a mismatch between what SMMU masters and the SMMU think is going
> on and 2) a logistical problem of dealing with the SMMU driver being a
> loadable module.
> 
> The second problem can be dealt with by shipping the module in the
> initial ramdisk. That's a bit annoying, but perhaps the right thing to
> do. At least on Tegra we need this because all the devices that carry
> the root filesystem (Ethernet for NFS and SDHCI/USB/SATA/PCI for disk
> boot) are SMMU masters and will start to fault once the SMMU driver is
> loaded.

Realistically, if you're building an IOMMU driver as a module then it needs
to be part of the initrd and fw_devlink needs to be enabled. Relying on
timeouts and the phase of the moon is not going to be reliable.

But back to the original issue, I think we should revert the Kconfig patch
from Linus' tree. Please can you send a revert for that?

Will

Linus Walleij Nov. 23, 2020, 12:35 p.m. UTC | #11

On Mon, Nov 16, 2020 at 5:36 PM Will Deacon <[e-mail]> wrote:

> Linus -- please can you drop this one (patch 3/3) for now, given that it's
> causing problems?

Reverted now, sorry for missing to do this earlier.

Yours,
Linus Walleij

Will Deacon Nov. 23, 2020, 1:06 p.m. UTC | #12

On Mon, Nov 23, 2020 at 01:35:14PM +0100, Linus Walleij wrote:
> On Mon, Nov 16, 2020 at 5:36 PM Will Deacon <[e-mail]> wrote:
> 
> > Linus -- please can you drop this one (patch 3/3) for now, given that it's
> > causing problems?
> 
> Reverted now, sorry for missing to do this earlier.

Cheers, Linus!

Will

Patch

diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index 3315e3c215864..5e369928bc567 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -235,8 +235,8 @@  config INTEL_STRATIX10_RSU
 	  Say Y here if you want Intel RSU support.
 
 config QCOM_SCM
-	bool
-	depends on ARM || ARM64
+	tristate "Qcom SCM driver"
+	depends on (ARM && HAVE_ARM_SMCCC) || ARM64
 	select RESET_CONTROLLER
 
 config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 5e013b6a3692e..523173cbff335 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -17,7 +17,8 @@  obj-$(CONFIG_ISCSI_IBFT)	+= iscsi_ibft.o
 obj-$(CONFIG_FIRMWARE_MEMMAP)	+= memmap.o
 obj-$(CONFIG_RASPBERRYPI_FIRMWARE) += raspberrypi.o
 obj-$(CONFIG_FW_CFG_SYSFS)	+= qemu_fw_cfg.o
-obj-$(CONFIG_QCOM_SCM)		+= qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
+obj-$(CONFIG_QCOM_SCM)		+= qcom-scm.o
+qcom-scm-objs += qcom_scm.o qcom_scm-smc.o qcom_scm-legacy.o
 obj-$(CONFIG_TI_SCI_PROTOCOL)	+= ti_sci.o
 obj-$(CONFIG_TRUSTED_FOUNDATIONS) += trusted_foundations.o
 obj-$(CONFIG_TURRIS_MOX_RWTM)	+= turris-mox-rwtm.o
diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c
index 7be48c1bec96d..6f431b73e617d 100644
--- a/drivers/firmware/qcom_scm.c
+++ b/drivers/firmware/qcom_scm.c
@@ -1280,6 +1280,7 @@  static const struct of_device_id qcom_scm_dt_match[] = {
 	{ .compatible = "qcom,scm" },
 	{}
 };
+MODULE_DEVICE_TABLE(of, qcom_scm_dt_match);
 
 static struct platform_driver qcom_scm_driver = {
 	.driver = {
@@ -1295,3 +1296,6 @@  static int __init qcom_scm_init(void)
 	return platform_driver_register(&qcom_scm_driver);
 }
 subsys_initcall(qcom_scm_init);
+
+MODULE_DESCRIPTION("Qualcomm Technologies, Inc. SCM driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 04878caf6da49..c64d7a2b65134 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -248,6 +248,7 @@  config SPAPR_TCE_IOMMU
 config ARM_SMMU
 	tristate "ARM Ltd. System MMU (SMMU) Support"
 	depends on ARM64 || ARM || (COMPILE_TEST && !GENERIC_ATOMIC64)
+	depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
 	select IOMMU_API
 	select IOMMU_IO_PGTABLE_LPAE
 	select ARM_DMA_USE_IOMMU if ARM
@@ -375,6 +376,7 @@  config QCOM_IOMMU
 	# Note: iommu drivers cannot (yet?) be built as modules
 	bool "Qualcomm IOMMU Support"
 	depends on ARCH_QCOM || (COMPILE_TEST && !GENERIC_ATOMIC64)
+	depends on QCOM_SCM=y
 	select IOMMU_API
 	select IOMMU_IO_PGTABLE_LPAE
 	select ARM_DMA_USE_IOMMU
diff --git a/drivers/net/wireless/ath/ath10k/Kconfig b/drivers/net/wireless/ath/ath10k/Kconfig
index 40f91bc8514d8..741289e385d59 100644
--- a/drivers/net/wireless/ath/ath10k/Kconfig
+++ b/drivers/net/wireless/ath/ath10k/Kconfig
@@ -44,6 +44,7 @@  config ATH10K_SNOC
 	tristate "Qualcomm ath10k SNOC support"
 	depends on ATH10K
 	depends on ARCH_QCOM || COMPILE_TEST
+	depends on QCOM_SCM || !QCOM_SCM #if QCOM_SCM=m this can't be =y
 	select QCOM_QMI_HELPERS
 	help
 	  This module adds support for integrated WCN3990 chip connected