diff mbox series

[V11,3/5] kbuild: Allow .dtso format for overlay source files

Message ID 170e086a5fa076869e7b37de8eea850fa7c39118.1615354376.git.viresh.kumar@linaro.org (mailing list archive)
State New, archived
Headers show
Series dt: Add fdtoverlay rule and statically build unittest | expand

Commit Message

Viresh Kumar March 10, 2021, 5:35 a.m. UTC 
Since the overlays dtb files are now named as .dtbo, there is a lot of
interest in similarly naming the overlay source dts files as .dtso.

This patch makes the necessary changes to allow .dtso format for overlay
source files.

Note that we still support generating .dtbo files from .dts files. This
is required for the source files present in drivers/of/unittest-data/,
because they can't be renamed to .dtso as they are used for some runtime
testing as well.

Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
---
 scripts/Makefile.lib | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

Comments

Masahiro Yamada March 10, 2021, 11:24 a.m. UTC | #1 
On Wed, Mar 10, 2021 at 2:35 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> Since the overlays dtb files are now named as .dtbo, there is a lot of
> interest in similarly naming the overlay source dts files as .dtso.
>
> This patch makes the necessary changes to allow .dtso format for overlay
> source files.
>
> Note that we still support generating .dtbo files from .dts files. This
> is required for the source files present in drivers/of/unittest-data/,
> because they can't be renamed to .dtso as they are used for some runtime
> testing as well.
>
> Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
> Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> ---
>  scripts/Makefile.lib | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index bc045a54a34e..59e86f67f9e0 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -339,7 +339,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
>
>  quiet_cmd_dtc = DTC     $@
>  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
> -       $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> +       $(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \

Even without "-I dts",

   inform = guess_input_format(arg, "dts");

seems to fall back to "dts" anyway,
but I guess you wanted to make this explicit, correct?

I will drop the ugly -O.
https://patchwork.kernel.org/project/linux-kbuild/patch/20210310110824.782209-1-masahiroy@kernel.org/

I will queue it to linux-kbuild/fixes.



>                 $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
>                 -d $(depfile).dtc.tmp $(dtc-tmp) ; \
>         cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
> @@ -347,9 +347,13 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
>  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
>         $(call if_changed_dep,dtc)
>
> +# Required for of unit-test files as they can't be renamed to .dtso

If you go with *.dtso, I think you will rename
all *.dts under the drivers/ directory.

What is blocking you from making this consistent?






>  $(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
>         $(call if_changed_dep,dtc)
>
> +$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
> +       $(call if_changed_dep,dtc)
> +
>  overlay-y := $(addprefix $(obj)/, $(overlay-y))
>
>  quiet_cmd_fdtoverlay = DTOVL   $@
> @@ -375,6 +379,9 @@ endef
>  $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
>         $(call if_changed_rule,dtc,yaml)
>
> +$(obj)/%.dt.yaml: $(src)/%.dtso $(DTC) $(DT_TMP_SCHEMA) FORCE
> +       $(call if_changed_rule,dtc,yaml)
> +
>  dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
>
>  # Bzip2
> --
> 2.25.0.rc1.19.g042ed3e048af
>
Masahiro Yamada March 10, 2021, 11:29 a.m. UTC | #2 
On Wed, Mar 10, 2021 at 8:24 PM Masahiro Yamada <masahiroy@kernel.org> wrote:
>
> On Wed, Mar 10, 2021 at 2:35 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > Since the overlays dtb files are now named as .dtbo, there is a lot of
> > interest in similarly naming the overlay source dts files as .dtso.
> >
> > This patch makes the necessary changes to allow .dtso format for overlay
> > source files.
> >
> > Note that we still support generating .dtbo files from .dts files. This
> > is required for the source files present in drivers/of/unittest-data/,
> > because they can't be renamed to .dtso as they are used for some runtime
> > testing as well.
> >
> > Reviewed-by: Geert Uytterhoeven <geert+renesas@glider.be>
> > Tested-by: Geert Uytterhoeven <geert+renesas@glider.be>
> > Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
> > ---
> >  scripts/Makefile.lib | 9 ++++++++-
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index bc045a54a34e..59e86f67f9e0 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -339,7 +339,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
> >
> >  quiet_cmd_dtc = DTC     $@
> >  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
> > -       $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> > +       $(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
>
> Even without "-I dts",
>
>    inform = guess_input_format(arg, "dts");
>
> seems to fall back to "dts" anyway,
> but I guess you wanted to make this explicit, correct?
>
> I will drop the ugly -O.
> https://patchwork.kernel.org/project/linux-kbuild/patch/20210310110824.782209-1-masahiroy@kernel.org/
>
> I will queue it to linux-kbuild/fixes.
>
>


BTW, is the attached patch good for DTC?

I do not know when '-O dtbo' is useful,
unless I am missing something.
Viresh Kumar March 10, 2021, 2:47 p.m. UTC | #3 
On 10-03-21, 20:24, Masahiro Yamada wrote:
> On Wed, Mar 10, 2021 at 2:35 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > index bc045a54a34e..59e86f67f9e0 100644
> > --- a/scripts/Makefile.lib
> > +++ b/scripts/Makefile.lib
> > @@ -339,7 +339,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
> >
> >  quiet_cmd_dtc = DTC     $@
> >  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
> > -       $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> > +       $(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> 
> Even without "-I dts",
> 
>    inform = guess_input_format(arg, "dts");
> 
> seems to fall back to "dts" anyway,

I missed this TBH.

> but I guess you wanted to make this explicit, correct?

That can be a reason now :)

> I will drop the ugly -O.
> https://patchwork.kernel.org/project/linux-kbuild/patch/20210310110824.782209-1-masahiroy@kernel.org/

But if we are going to depend on DTC to guess it right, then we
shouldn't add -I at all..

> I will queue it to linux-kbuild/fixes.
> 
> 
> 
> >                 $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
> >                 -d $(depfile).dtc.tmp $(dtc-tmp) ; \
> >         cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
> > @@ -347,9 +347,13 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
> >  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
> >         $(call if_changed_dep,dtc)
> >
> > +# Required for of unit-test files as they can't be renamed to .dtso
> 
> If you go with *.dtso, I think you will rename
> all *.dts under the drivers/ directory.
> 
> What is blocking you from making this consistent?

The unit-test dts files are designed differently (we have had lots of
discussion between Frank and David on that) and they aren't purely
overlay or base files. They are designed to do some tricky testing and
renaming them to .dtso won't be right, we are just reusing them to do
static (build time) testing as well.

I think it would be better if we can drop the existing %.dtbo rule
here (i.e. dtbo from .dts) and do some magic in unit-test's Makefile,
so it is localised at least instead of it here.

Any ideas for that ?
Viresh Kumar March 10, 2021, 2:48 p.m. UTC | #4 
On 10-03-21, 20:29, Masahiro Yamada wrote:
> BTW, is the attached patch good for DTC?
> 
> I do not know when '-O dtbo' is useful,
> unless I am missing something.

It is useful if we are sending the -O option all the time (I have
already given more details to your patch) as outform will not be NULL.
Masahiro Yamada March 10, 2021, 3:15 p.m. UTC | #5 
On Wed, Mar 10, 2021 at 11:47 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> On 10-03-21, 20:24, Masahiro Yamada wrote:
> > On Wed, Mar 10, 2021 at 2:35 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > > diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> > > index bc045a54a34e..59e86f67f9e0 100644
> > > --- a/scripts/Makefile.lib
> > > +++ b/scripts/Makefile.lib
> > > @@ -339,7 +339,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
> > >
> > >  quiet_cmd_dtc = DTC     $@
> > >  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
> > > -       $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> > > +       $(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
> >
> > Even without "-I dts",
> >
> >    inform = guess_input_format(arg, "dts");
> >
> > seems to fall back to "dts" anyway,
>
> I missed this TBH.
>
> > but I guess you wanted to make this explicit, correct?
>
> That can be a reason now :)
>
> > I will drop the ugly -O.
> > https://patchwork.kernel.org/project/linux-kbuild/patch/20210310110824.782209-1-masahiroy@kernel.org/
>
> But if we are going to depend on DTC to guess it right, then we
> shouldn't add -I at all..
>
> > I will queue it to linux-kbuild/fixes.
> >
> >
> >
> > >                 $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
> > >                 -d $(depfile).dtc.tmp $(dtc-tmp) ; \
> > >         cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
> > > @@ -347,9 +347,13 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
> > >  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
> > >         $(call if_changed_dep,dtc)
> > >
> > > +# Required for of unit-test files as they can't be renamed to .dtso
> >
> > If you go with *.dtso, I think you will rename
> > all *.dts under the drivers/ directory.
> >
> > What is blocking you from making this consistent?
>
> The unit-test dts files are designed differently (we have had lots of
> discussion between Frank and David on that) and they aren't purely
> overlay or base files. They are designed to do some tricky testing and
> renaming them to .dtso won't be right, we are just reusing them to do
> static (build time) testing as well.


I still do not understand.

If they are not overlay files, why
do you need to have them suffixed with .dtbo?

".dts -> .dtb" should be enough.

Why do you need to do ".dts  -> .dtbo" ?




> I think it would be better if we can drop the existing %.dtbo rule
> here (i.e. dtbo from .dts) and do some magic in unit-test's Makefile,
> so it is localised at least instead of it here.
>
> Any ideas for that ?

I do not know.

My impression is you are doing something fishy.
Masahiro Yamada March 10, 2021, 3:18 p.m. UTC | #6 
On Wed, Mar 10, 2021 at 11:48 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> On 10-03-21, 20:29, Masahiro Yamada wrote:
> > BTW, is the attached patch good for DTC?
> >
> > I do not know when '-O dtbo' is useful,
> > unless I am missing something.
>
> It is useful if we are sending the -O option all the time (I have
> already given more details to your patch) as outform will not be NULL.


"-O dtbo" was useful to make your buggy patch work.

That is not justification.


.yaml   ->  -O yaml
.dtb    ->  -O dtb
.dtbo   ->  -O dtb

is the correct if you want to give -O explicitly.
Frank Rowand March 10, 2021, 10:42 p.m. UTC | #7 
On 3/10/21 9:15 AM, Masahiro Yamada wrote:
> On Wed, Mar 10, 2021 at 11:47 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>>
>> On 10-03-21, 20:24, Masahiro Yamada wrote:
>>> On Wed, Mar 10, 2021 at 2:35 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>>>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>>>> index bc045a54a34e..59e86f67f9e0 100644
>>>> --- a/scripts/Makefile.lib
>>>> +++ b/scripts/Makefile.lib
>>>> @@ -339,7 +339,7 @@ $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
>>>>
>>>>  quiet_cmd_dtc = DTC     $@
>>>>  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
>>>> -       $(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
>>>> +       $(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
>>>
>>> Even without "-I dts",
>>>
>>>    inform = guess_input_format(arg, "dts");
>>>
>>> seems to fall back to "dts" anyway,
>>
>> I missed this TBH.
>>
>>> but I guess you wanted to make this explicit, correct?
>>
>> That can be a reason now :)
>>
>>> I will drop the ugly -O.
>>> https://patchwork.kernel.org/project/linux-kbuild/patch/20210310110824.782209-1-masahiroy@kernel.org/
>>
>> But if we are going to depend on DTC to guess it right, then we
>> shouldn't add -I at all..
>>
>>> I will queue it to linux-kbuild/fixes.
>>>
>>>
>>>
>>>>                 $(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
>>>>                 -d $(depfile).dtc.tmp $(dtc-tmp) ; \
>>>>         cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
>>>> @@ -347,9 +347,13 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
>>>>  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
>>>>         $(call if_changed_dep,dtc)
>>>>
>>>> +# Required for of unit-test files as they can't be renamed to .dtso
>>>
>>> If you go with *.dtso, I think you will rename
>>> all *.dts under the drivers/ directory.
>>>
>>> What is blocking you from making this consistent?
>>
>> The unit-test dts files are designed differently (we have had lots of
>> discussion between Frank and David on that) and they aren't purely
>> overlay or base files. They are designed to do some tricky testing and
>> renaming them to .dtso won't be right, we are just reusing them to do
>> static (build time) testing as well.
> 
> 
> I still do not understand.
> 
> If they are not overlay files, why
> do you need to have them suffixed with .dtbo?
> 
> ".dts -> .dtb" should be enough.
> 
> Why do you need to do ".dts  -> .dtbo" ?
> 
> 
> 
> 
>> I think it would be better if we can drop the existing %.dtbo rule
>> here (i.e. dtbo from .dts) and do some magic in unit-test's Makefile,
>> so it is localised at least instead of it here.
>>
>> Any ideas for that ?
> 
> I do not know.
> 


> My impression is you are doing something fishy.

That is accurate.  Devicetree unittest plays some tricks to enable
testing to occur.  These tricks will never be used anywhere else
in the kernel.

-Frank
Viresh Kumar March 12, 2021, 3:52 a.m. UTC | #8 
On 11-03-21, 00:18, Masahiro Yamada wrote:
> On Wed, Mar 10, 2021 at 11:48 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > On 10-03-21, 20:29, Masahiro Yamada wrote:
> > > BTW, is the attached patch good for DTC?
> > >
> > > I do not know when '-O dtbo' is useful,
> > > unless I am missing something.
> >
> > It is useful if we are sending the -O option all the time (I have
> > already given more details to your patch) as outform will not be NULL.
> 
> 
> "-O dtbo" was useful to make your buggy patch work.
> 
> That is not justification.

I wasn't giving justification, but rather saying why it was required
earlier. And I agree that it isn't required once we drop the -O
parameter here.
Viresh Kumar March 12, 2021, 4:47 a.m. UTC | #9 
On 10-03-21, 20:24, Masahiro Yamada wrote:
> Even without "-I dts",
> 
>    inform = guess_input_format(arg, "dts");
> 
> seems to fall back to "dts" anyway,
> but I guess you wanted to make this explicit, correct?


> > +# Required for of unit-test files as they can't be renamed to .dtso
> 
> If you go with *.dtso, I think you will rename
> all *.dts under the drivers/ directory.
> 
> What is blocking you from making this consistent?

What about this patch instead ? This localizes the dts->dtbo hack to
unitest's Makefile at least.

diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
index a5d2d9254b2c..9f3426ec3fab 100644
--- a/drivers/of/unittest-data/Makefile
+++ b/drivers/of/unittest-data/Makefile
@@ -86,3 +86,7 @@ static_test_1-dtbs := static_base_1.dtb $(apply_static_overlay_1)
 static_test_2-dtbs := static_base_2.dtb $(apply_static_overlay_2)
 
 dtb-$(CONFIG_OF_OVERLAY) += static_test_1.dtb static_test_2.dtb
+
+# Required for of unittest files as they can't be renamed to .dtso
+$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+       $(call if_changed_dep,dtc)
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index bc045a54a34e..77a9be055e51 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -347,7 +347,7 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
 $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
        $(call if_changed_dep,dtc)
 
-$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
        $(call if_changed_dep,dtc)
 
 overlay-y := $(addprefix $(obj)/, $(overlay-y))
@@ -375,6 +375,9 @@ endef
 $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
        $(call if_changed_rule,dtc,yaml)
 
+$(obj)/%.dt.yaml: $(src)/%.dtso $(DTC) $(DT_TMP_SCHEMA) FORCE
+       $(call if_changed_rule,dtc,yaml)
+
 dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
 
 # Bzip2
Frank Rowand March 12, 2021, 7:03 a.m. UTC | #10 
Hi Viresh,

On 3/11/21 10:47 PM, Viresh Kumar wrote:
> On 10-03-21, 20:24, Masahiro Yamada wrote:
>> Even without "-I dts",
>>
>>    inform = guess_input_format(arg, "dts");
>>
>> seems to fall back to "dts" anyway,
>> but I guess you wanted to make this explicit, correct?
> 
> 
>>> +# Required for of unit-test files as they can't be renamed to .dtso
>>
>> If you go with *.dtso, I think you will rename
>> all *.dts under the drivers/ directory.
>>
>> What is blocking you from making this consistent?
> 
> What about this patch instead ? This localizes the dts->dtbo hack to
> unitest's Makefile at least.

It is late here, so I am not going to take the time to actually try what
I am going to suggest.  I apologize in advance if I send you off on a
wild goose chase.

Would it work to create a .dtso file for each of the unittest overlay .dts
files, where the .dtso would simply #include the .dts file.  Then the corresponding
.dtbo files could be added to the obj-$(CONFIG_OF_OVERLAY) list.

I would like to avoid having the unitest-data/Makefile have different rules to
build objects because then the normal build rule is not being tested.

-Frank

> 
> diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
> index a5d2d9254b2c..9f3426ec3fab 100644
> --- a/drivers/of/unittest-data/Makefile
> +++ b/drivers/of/unittest-data/Makefile
> @@ -86,3 +86,7 @@ static_test_1-dtbs := static_base_1.dtb $(apply_static_overlay_1)
>  static_test_2-dtbs := static_base_2.dtb $(apply_static_overlay_2)
>  
>  dtb-$(CONFIG_OF_OVERLAY) += static_test_1.dtb static_test_2.dtb
> +
> +# Required for of unittest files as they can't be renamed to .dtso
> +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> +       $(call if_changed_dep,dtc)
> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
> index bc045a54a34e..77a9be055e51 100644
> --- a/scripts/Makefile.lib
> +++ b/scripts/Makefile.lib
> @@ -347,7 +347,7 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
>  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
>         $(call if_changed_dep,dtc)
>  
> -$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> +$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
>         $(call if_changed_dep,dtc)
>  
>  overlay-y := $(addprefix $(obj)/, $(overlay-y))
> @@ -375,6 +375,9 @@ endef
>  $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
>         $(call if_changed_rule,dtc,yaml)
>  
> +$(obj)/%.dt.yaml: $(src)/%.dtso $(DTC) $(DT_TMP_SCHEMA) FORCE
> +       $(call if_changed_rule,dtc,yaml)
> +
>  dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
>  
>  # Bzip2
>
Frank Rowand March 12, 2021, 7:09 a.m. UTC | #11 
On 3/12/21 1:03 AM, Frank Rowand wrote:
> Hi Viresh,
> 
> On 3/11/21 10:47 PM, Viresh Kumar wrote:
>> On 10-03-21, 20:24, Masahiro Yamada wrote:
>>> Even without "-I dts",
>>>
>>>    inform = guess_input_format(arg, "dts");
>>>
>>> seems to fall back to "dts" anyway,
>>> but I guess you wanted to make this explicit, correct?
>>
>>
>>>> +# Required for of unit-test files as they can't be renamed to .dtso
>>>
>>> If you go with *.dtso, I think you will rename
>>> all *.dts under the drivers/ directory.
>>>
>>> What is blocking you from making this consistent?
>>
>> What about this patch instead ? This localizes the dts->dtbo hack to
>> unitest's Makefile at least.
> 
> It is late here, so I am not going to take the time to actually try what
> I am going to suggest.  I apologize in advance if I send you off on a
> wild goose chase.
> 
> Would it work to create a .dtso file for each of the unittest overlay .dts
> files, where the .dtso would simply #include the .dts file.  Then the corresponding
> .dtbo files could be added to the obj-$(CONFIG_OF_OVERLAY) list.

I suggested having the .dtso files include the .dts file because that is a relatively
small and easy change to test.  What would probably make more sense is the rename
the existing overlay .dts files to be .dtso files and then for each overlay .dtso
file create a new .dts file that #includes the corresponding .dtso file.  This is
more work and churn, but easier to document that the .dts files are a hack that is
needed so that the corresponding .dtb.S files will be generated.

If it works, I am fine with either approach.

-Frank

> 
> I would like to avoid having the unitest-data/Makefile have different rules to
> build objects because then the normal build rule is not being tested.
> 
> -Frank
> 
>>
>> diff --git a/drivers/of/unittest-data/Makefile b/drivers/of/unittest-data/Makefile
>> index a5d2d9254b2c..9f3426ec3fab 100644
>> --- a/drivers/of/unittest-data/Makefile
>> +++ b/drivers/of/unittest-data/Makefile
>> @@ -86,3 +86,7 @@ static_test_1-dtbs := static_base_1.dtb $(apply_static_overlay_1)
>>  static_test_2-dtbs := static_base_2.dtb $(apply_static_overlay_2)
>>  
>>  dtb-$(CONFIG_OF_OVERLAY) += static_test_1.dtb static_test_2.dtb
>> +
>> +# Required for of unittest files as they can't be renamed to .dtso
>> +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
>> +       $(call if_changed_dep,dtc)
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index bc045a54a34e..77a9be055e51 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -347,7 +347,7 @@ cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
>>  $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
>>         $(call if_changed_dep,dtc)
>>  
>> -$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
>> +$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
>>         $(call if_changed_dep,dtc)
>>  
>>  overlay-y := $(addprefix $(obj)/, $(overlay-y))
>> @@ -375,6 +375,9 @@ endef
>>  $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
>>         $(call if_changed_rule,dtc,yaml)
>>  
>> +$(obj)/%.dt.yaml: $(src)/%.dtso $(DTC) $(DT_TMP_SCHEMA) FORCE
>> +       $(call if_changed_rule,dtc,yaml)
>> +
>>  dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
>>  
>>  # Bzip2
>>
>
Viresh Kumar March 12, 2021, 7:13 a.m. UTC | #12 
On 12-03-21, 01:09, Frank Rowand wrote:
> I suggested having the .dtso files include the .dts file because that is a relatively
> small and easy change to test.  What would probably make more sense is the rename
> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
> file create a new .dts file that #includes the corresponding .dtso file.  This is
> more work and churn, but easier to document that the .dts files are a hack that is
> needed so that the corresponding .dtb.S files will be generated.

What about creating links instead then ?
Frank Rowand March 13, 2021, 5:11 a.m. UTC | #13 
On 3/12/21 1:13 AM, Viresh Kumar wrote:
> On 12-03-21, 01:09, Frank Rowand wrote:
>> I suggested having the .dtso files include the .dts file because that is a relatively
>> small and easy change to test.  What would probably make more sense is the rename
>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
>> file create a new .dts file that #includes the corresponding .dtso file.  This is
>> more work and churn, but easier to document that the .dts files are a hack that is
>> needed so that the corresponding .dtb.S files will be generated.
> 
> What about creating links instead then ?
> 

I don't really like the idea of using links here.

Maybe it is best to make the changes needed to allow the unittest
overlays to be .dtso instead of .dts.

Off the top of my head:

  scripts/Makefile.lib:
     The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
     overlay data in section .dtb.init.rodata, with a label
     pointing to the beginning of the overlay __dtb_XXX_begin and
     a label pointing to the end of the overlay __dtb_XXX_end,
     for the overlay named XXX.  I _think_ that you could simply
     add a corresponding rule for %.dtbo.S using a new command
     cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
     __dtbo_XXX_begin and __dtbo_XXX_end).

  drivers/of/unittest.o:
     would need to have the #define of OVERLAY_INFO() changed to
     reflect the changed label names (use __dtbo_##overlayname##begin
     and __dtb_##overlay_name##_end).

  drivers/of/unittest-data/Makefile:
     In obj-$(CONFIG_OF_OVERLAY) change the *.dtb.o names to *.dtbo.o

     I'm not sure how the DTC_FLAGS_... += -@ differentiates between
     .dts / .dtb and .dtso / .dtbo  That is worth looking at.

-Frank
Frank Rowand March 15, 2021, 1:16 a.m. UTC | #14 
Hi VIresh,

On 3/12/21 11:11 PM, Frank Rowand wrote:
> On 3/12/21 1:13 AM, Viresh Kumar wrote:
>> On 12-03-21, 01:09, Frank Rowand wrote:
>>> I suggested having the .dtso files include the .dts file because that is a relatively
>>> small and easy change to test.  What would probably make more sense is the rename
>>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
>>> file create a new .dts file that #includes the corresponding .dtso file.  This is
>>> more work and churn, but easier to document that the .dts files are a hack that is
>>> needed so that the corresponding .dtb.S files will be generated.
>>
>> What about creating links instead then ?
>>
> 
> I don't really like the idea of using links here.
> 
> Maybe it is best to make the changes needed to allow the unittest
> overlays to be .dtso instead of .dts.
> 
> Off the top of my head:
> 
>   scripts/Makefile.lib:
>      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
>      overlay data in section .dtb.init.rodata, with a label
>      pointing to the beginning of the overlay __dtb_XXX_begin and
>      a label pointing to the end of the overlay __dtb_XXX_end,
>      for the overlay named XXX.  I _think_ that you could simply
>      add a corresponding rule for %.dtbo.S using a new command
>      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
>      __dtbo_XXX_begin and __dtbo_XXX_end).

If you do the above, please put it in drivers/of/unittest-data/Makefile
instead of scripts/Makefile.lib because it is unittest.c specific and
not meant to be anywhere else in the kernel.

-Frank

> 
>   drivers/of/unittest.o:
>      would need to have the #define of OVERLAY_INFO() changed to
>      reflect the changed label names (use __dtbo_##overlayname##begin
>      and __dtb_##overlay_name##_end).
> 
>   drivers/of/unittest-data/Makefile:
>      In obj-$(CONFIG_OF_OVERLAY) change the *.dtb.o names to *.dtbo.o
> 
>      I'm not sure how the DTC_FLAGS_... += -@ differentiates between
>      .dts / .dtb and .dtso / .dtbo  That is worth looking at.
> 
> -Frank
>
Viresh Kumar March 15, 2021, 6:40 a.m. UTC | #15 
On 14-03-21, 20:16, Frank Rowand wrote:
> On 3/12/21 11:11 PM, Frank Rowand wrote:
> > On 3/12/21 1:13 AM, Viresh Kumar wrote:
> >> On 12-03-21, 01:09, Frank Rowand wrote:
> >>> I suggested having the .dtso files include the .dts file because that is a relatively
> >>> small and easy change to test.  What would probably make more sense is the rename
> >>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
> >>> file create a new .dts file that #includes the corresponding .dtso file.  This is
> >>> more work and churn, but easier to document that the .dts files are a hack that is
> >>> needed so that the corresponding .dtb.S files will be generated.
> >>
> >> What about creating links instead then ?
> >>
> > 
> > I don't really like the idea of using links here.
> > 
> > Maybe it is best to make the changes needed to allow the unittest
> > overlays to be .dtso instead of .dts.
> > 
> > Off the top of my head:
> > 
> >   scripts/Makefile.lib:
> >      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
> >      overlay data in section .dtb.init.rodata, with a label
> >      pointing to the beginning of the overlay __dtb_XXX_begin and
> >      a label pointing to the end of the overlay __dtb_XXX_end,
> >      for the overlay named XXX.  I _think_ that you could simply
> >      add a corresponding rule for %.dtbo.S using a new command
> >      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
> >      __dtbo_XXX_begin and __dtbo_XXX_end).
> 
> If you do the above, please put it in drivers/of/unittest-data/Makefile
> instead of scripts/Makefile.lib because it is unittest.c specific and
> not meant to be anywhere else in the kernel.

What about doing this then in unittest's Makefile instead (which I
already suggested earlier), that will make everything work just fine
without any other changes ?

+# Required for of unittest files as they can't be renamed to .dtso
+$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
+       $(call if_changed_dep,dtc)
Masahiro Yamada March 15, 2021, 5:43 p.m. UTC | #16 
On Mon, Mar 15, 2021 at 3:40 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
>
> On 14-03-21, 20:16, Frank Rowand wrote:
> > On 3/12/21 11:11 PM, Frank Rowand wrote:
> > > On 3/12/21 1:13 AM, Viresh Kumar wrote:
> > >> On 12-03-21, 01:09, Frank Rowand wrote:
> > >>> I suggested having the .dtso files include the .dts file because that is a relatively
> > >>> small and easy change to test.  What would probably make more sense is the rename
> > >>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
> > >>> file create a new .dts file that #includes the corresponding .dtso file.  This is
> > >>> more work and churn, but easier to document that the .dts files are a hack that is
> > >>> needed so that the corresponding .dtb.S files will be generated.
> > >>
> > >> What about creating links instead then ?
> > >>
> > >
> > > I don't really like the idea of using links here.
> > >
> > > Maybe it is best to make the changes needed to allow the unittest
> > > overlays to be .dtso instead of .dts.
> > >
> > > Off the top of my head:
> > >
> > >   scripts/Makefile.lib:
> > >      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
> > >      overlay data in section .dtb.init.rodata, with a label
> > >      pointing to the beginning of the overlay __dtb_XXX_begin and
> > >      a label pointing to the end of the overlay __dtb_XXX_end,
> > >      for the overlay named XXX.  I _think_ that you could simply
> > >      add a corresponding rule for %.dtbo.S using a new command
> > >      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
> > >      __dtbo_XXX_begin and __dtbo_XXX_end).
> >
> > If you do the above, please put it in drivers/of/unittest-data/Makefile
> > instead of scripts/Makefile.lib because it is unittest.c specific and
> > not meant to be anywhere else in the kernel.
>
> What about doing this then in unittest's Makefile instead (which I
> already suggested earlier), that will make everything work just fine
> without any other changes ?
>
> +# Required for of unittest files as they can't be renamed to .dtso
> +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> +       $(call if_changed_dep,dtc)
>
> --
> viresh


If those rules are only needed by drivers/of/unittest-data/Makefile,
they should not be located in scripts/Makefile.lib.



But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
if these are doing bad things.
They seem to be overlay files even though the file name suffix is .dts



$ find drivers -name '*.dts'
drivers/staging/pi433/Documentation/devicetree/pi433-overlay.dts
drivers/staging/mt7621-dts/gbpc2.dts
drivers/staging/mt7621-dts/gbpc1.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts
drivers/of/unittest-data/overlay_1.dts
drivers/of/unittest-data/testcases.dts
drivers/of/unittest-data/overlay_bad_add_dup_node.dts
drivers/of/unittest-data/overlay_bad_symbol.dts
drivers/of/unittest-data/overlay_0.dts
drivers/of/unittest-data/overlay_11.dts
drivers/of/unittest-data/overlay_gpio_03.dts
drivers/of/unittest-data/overlay_gpio_04a.dts
drivers/of/unittest-data/overlay_gpio_04b.dts
drivers/of/unittest-data/overlay_5.dts
drivers/of/unittest-data/overlay_bad_add_dup_prop.dts
drivers/of/unittest-data/overlay_gpio_01.dts
drivers/of/unittest-data/overlay_10.dts
drivers/of/unittest-data/overlay_7.dts
drivers/of/unittest-data/overlay_bad_phandle.dts
drivers/of/unittest-data/overlay_3.dts
drivers/of/unittest-data/overlay_6.dts
drivers/of/unittest-data/overlay_8.dts
drivers/of/unittest-data/overlay_12.dts
drivers/of/unittest-data/overlay_gpio_02a.dts
drivers/of/unittest-data/overlay_gpio_02b.dts
drivers/of/unittest-data/overlay_4.dts
drivers/of/unittest-data/overlay.dts
drivers/of/unittest-data/overlay_9.dts
drivers/of/unittest-data/overlay_2.dts
drivers/of/unittest-data/overlay_15.dts
drivers/of/unittest-data/overlay_base.dts
drivers/of/unittest-data/overlay_13.dts
Laurent Pinchart March 15, 2021, 10:12 p.m. UTC | #17 
Hi Yamada-san,

On Tue, Mar 16, 2021 at 02:43:45AM +0900, Masahiro Yamada wrote:
> On Mon, Mar 15, 2021 at 3:40 PM Viresh Kumar wrote:
> > On 14-03-21, 20:16, Frank Rowand wrote:
> > > On 3/12/21 11:11 PM, Frank Rowand wrote:
> > > > On 3/12/21 1:13 AM, Viresh Kumar wrote:
> > > >> On 12-03-21, 01:09, Frank Rowand wrote:
> > > >>> I suggested having the .dtso files include the .dts file because that is a relatively
> > > >>> small and easy change to test.  What would probably make more sense is the rename
> > > >>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
> > > >>> file create a new .dts file that #includes the corresponding .dtso file.  This is
> > > >>> more work and churn, but easier to document that the .dts files are a hack that is
> > > >>> needed so that the corresponding .dtb.S files will be generated.
> > > >>
> > > >> What about creating links instead then ?
> > > >>
> > > >
> > > > I don't really like the idea of using links here.
> > > >
> > > > Maybe it is best to make the changes needed to allow the unittest
> > > > overlays to be .dtso instead of .dts.
> > > >
> > > > Off the top of my head:
> > > >
> > > >   scripts/Makefile.lib:
> > > >      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
> > > >      overlay data in section .dtb.init.rodata, with a label
> > > >      pointing to the beginning of the overlay __dtb_XXX_begin and
> > > >      a label pointing to the end of the overlay __dtb_XXX_end,
> > > >      for the overlay named XXX.  I _think_ that you could simply
> > > >      add a corresponding rule for %.dtbo.S using a new command
> > > >      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
> > > >      __dtbo_XXX_begin and __dtbo_XXX_end).
> > >
> > > If you do the above, please put it in drivers/of/unittest-data/Makefile
> > > instead of scripts/Makefile.lib because it is unittest.c specific and
> > > not meant to be anywhere else in the kernel.
> >
> > What about doing this then in unittest's Makefile instead (which I
> > already suggested earlier), that will make everything work just fine
> > without any other changes ?
> >
> > +# Required for of unittest files as they can't be renamed to .dtso
> > +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> > +       $(call if_changed_dep,dtc)
> >
> 
> If those rules are only needed by drivers/of/unittest-data/Makefile,
> they should not be located in scripts/Makefile.lib.
> 
> But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
> if these are doing bad things.
> They seem to be overlay files even though the file name suffix is .dts

That is correct, they are overlays. I have no issue with those files
being renamed to .dtso if that can help (but I haven't checked if that
would have any adverse effect on the R-Car DU driver).

These files are there to ensure backward compatibility with older DT
bindings. The change was made 3 years ago and I wouldn't object to
dropping this completely, but I understand I may not be the most
cautious person when it comes to ensuring DT backward compatibility :-)

> $ find drivers -name '*.dts'
> drivers/staging/pi433/Documentation/devicetree/pi433-overlay.dts
> drivers/staging/mt7621-dts/gbpc2.dts
> drivers/staging/mt7621-dts/gbpc1.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts
> drivers/of/unittest-data/overlay_1.dts
> drivers/of/unittest-data/testcases.dts
> drivers/of/unittest-data/overlay_bad_add_dup_node.dts
> drivers/of/unittest-data/overlay_bad_symbol.dts
> drivers/of/unittest-data/overlay_0.dts
> drivers/of/unittest-data/overlay_11.dts
> drivers/of/unittest-data/overlay_gpio_03.dts
> drivers/of/unittest-data/overlay_gpio_04a.dts
> drivers/of/unittest-data/overlay_gpio_04b.dts
> drivers/of/unittest-data/overlay_5.dts
> drivers/of/unittest-data/overlay_bad_add_dup_prop.dts
> drivers/of/unittest-data/overlay_gpio_01.dts
> drivers/of/unittest-data/overlay_10.dts
> drivers/of/unittest-data/overlay_7.dts
> drivers/of/unittest-data/overlay_bad_phandle.dts
> drivers/of/unittest-data/overlay_3.dts
> drivers/of/unittest-data/overlay_6.dts
> drivers/of/unittest-data/overlay_8.dts
> drivers/of/unittest-data/overlay_12.dts
> drivers/of/unittest-data/overlay_gpio_02a.dts
> drivers/of/unittest-data/overlay_gpio_02b.dts
> drivers/of/unittest-data/overlay_4.dts
> drivers/of/unittest-data/overlay.dts
> drivers/of/unittest-data/overlay_9.dts
> drivers/of/unittest-data/overlay_2.dts
> drivers/of/unittest-data/overlay_15.dts
> drivers/of/unittest-data/overlay_base.dts
> drivers/of/unittest-data/overlay_13.dts
Viresh Kumar March 16, 2021, 5:27 a.m. UTC | #18 
On 16-03-21, 02:43, Masahiro Yamada wrote:
> On Mon, Mar 15, 2021 at 3:40 PM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> > On 14-03-21, 20:16, Frank Rowand wrote:
> > What about doing this then in unittest's Makefile instead (which I
> > already suggested earlier), that will make everything work just fine
> > without any other changes ?
> >
> > +# Required for of unittest files as they can't be renamed to .dtso
> > +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> > +       $(call if_changed_dep,dtc)
> 
> If those rules are only needed by drivers/of/unittest-data/Makefile,
> they should not be located in scripts/Makefile.lib.

Right, this is exactly what I suggested.

> But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
> if these are doing bad things.
> They seem to be overlay files even though the file name suffix is .dts
> 
> $ find drivers -name '*.dts'
> drivers/staging/pi433/Documentation/devicetree/pi433-overlay.dts
> drivers/staging/mt7621-dts/gbpc2.dts
> drivers/staging/mt7621-dts/gbpc1.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts

For all the above files, even if they are really overlay files, we
won't use fdtoverlay tool to apply them to some base dtb and so if we
leave them as is, i.e. .dts->.dtb, it won't break anything.

The problem only happens if someone wants to generate .dtbo for them
instead and then they should be named .dtso as we won't allow .dts ->
.dtbo conversion there.
Frank Rowand March 16, 2021, 5:36 a.m. UTC | #19 
On 3/15/21 1:40 AM, Viresh Kumar wrote:
> On 14-03-21, 20:16, Frank Rowand wrote:
>> On 3/12/21 11:11 PM, Frank Rowand wrote:
>>> On 3/12/21 1:13 AM, Viresh Kumar wrote:
>>>> On 12-03-21, 01:09, Frank Rowand wrote:
>>>>> I suggested having the .dtso files include the .dts file because that is a relatively
>>>>> small and easy change to test.  What would probably make more sense is the rename
>>>>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
>>>>> file create a new .dts file that #includes the corresponding .dtso file.  This is
>>>>> more work and churn, but easier to document that the .dts files are a hack that is
>>>>> needed so that the corresponding .dtb.S files will be generated.
>>>>
>>>> What about creating links instead then ?
>>>>
>>>
>>> I don't really like the idea of using links here.
>>>
>>> Maybe it is best to make the changes needed to allow the unittest
>>> overlays to be .dtso instead of .dts.
>>>
>>> Off the top of my head:
>>>
>>>   scripts/Makefile.lib:
>>>      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
>>>      overlay data in section .dtb.init.rodata, with a label
>>>      pointing to the beginning of the overlay __dtb_XXX_begin and
>>>      a label pointing to the end of the overlay __dtb_XXX_end,
>>>      for the overlay named XXX.  I _think_ that you could simply
>>>      add a corresponding rule for %.dtbo.S using a new command
>>>      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
>>>      __dtbo_XXX_begin and __dtbo_XXX_end).
>>
>> If you do the above, please put it in drivers/of/unittest-data/Makefile
>> instead of scripts/Makefile.lib because it is unittest.c specific and
>> not meant to be anywhere else in the kernel.
> 
> What about doing this then in unittest's Makefile instead (which I
> already suggested earlier), that will make everything work just fine
> without any other changes ?
> 
> +# Required for of unittest files as they can't be renamed to .dtso
> +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
> +       $(call if_changed_dep,dtc)
> 

I should have looked at patch 3/5 more carefully instead of counting on
Masahiro to check it out and simply build testing.

Patch 3/5 does not seem correct.  I'll look over all the makefile related
changes that have been accepted (if any) and patch 3/5 later today (Tuesday).

-Frank
Frank Rowand March 16, 2021, 5:39 a.m. UTC | #20 
On 3/15/21 5:12 PM, Laurent Pinchart wrote:
> Hi Yamada-san,
> 
> On Tue, Mar 16, 2021 at 02:43:45AM +0900, Masahiro Yamada wrote:
>> On Mon, Mar 15, 2021 at 3:40 PM Viresh Kumar wrote:
>>> On 14-03-21, 20:16, Frank Rowand wrote:
>>>> On 3/12/21 11:11 PM, Frank Rowand wrote:
>>>>> On 3/12/21 1:13 AM, Viresh Kumar wrote:
>>>>>> On 12-03-21, 01:09, Frank Rowand wrote:
>>>>>>> I suggested having the .dtso files include the .dts file because that is a relatively
>>>>>>> small and easy change to test.  What would probably make more sense is the rename
>>>>>>> the existing overlay .dts files to be .dtso files and then for each overlay .dtso
>>>>>>> file create a new .dts file that #includes the corresponding .dtso file.  This is
>>>>>>> more work and churn, but easier to document that the .dts files are a hack that is
>>>>>>> needed so that the corresponding .dtb.S files will be generated.
>>>>>>
>>>>>> What about creating links instead then ?
>>>>>>
>>>>>
>>>>> I don't really like the idea of using links here.
>>>>>
>>>>> Maybe it is best to make the changes needed to allow the unittest
>>>>> overlays to be .dtso instead of .dts.
>>>>>
>>>>> Off the top of my head:
>>>>>
>>>>>   scripts/Makefile.lib:
>>>>>      The rule for %.dtb.S invokes cmd_dt_S_dtb, which puts the
>>>>>      overlay data in section .dtb.init.rodata, with a label
>>>>>      pointing to the beginning of the overlay __dtb_XXX_begin and
>>>>>      a label pointing to the end of the overlay __dtb_XXX_end,
>>>>>      for the overlay named XXX.  I _think_ that you could simply
>>>>>      add a corresponding rule for %.dtbo.S using a new command
>>>>>      cmd_dt_S_dtbo (the same as cmd_dt_S_dtb, except use labels
>>>>>      __dtbo_XXX_begin and __dtbo_XXX_end).
>>>>
>>>> If you do the above, please put it in drivers/of/unittest-data/Makefile
>>>> instead of scripts/Makefile.lib because it is unittest.c specific and
>>>> not meant to be anywhere else in the kernel.
>>>
>>> What about doing this then in unittest's Makefile instead (which I
>>> already suggested earlier), that will make everything work just fine
>>> without any other changes ?
>>>
>>> +# Required for of unittest files as they can't be renamed to .dtso
>>> +$(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
>>> +       $(call if_changed_dep,dtc)
>>>
>>
>> If those rules are only needed by drivers/of/unittest-data/Makefile,
>> they should not be located in scripts/Makefile.lib.
>>
>> But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
>> if these are doing bad things.
>> They seem to be overlay files even though the file name suffix is .dts
> 
> That is correct, they are overlays. I have no issue with those files
> being renamed to .dtso if that can help (but I haven't checked if that
> would have any adverse effect on the R-Car DU driver).

As Laurent replied, yes these are overlays.  They were grandfathered in
as a deprecated use of overlays.

> 
> These files are there to ensure backward compatibility with older DT
> bindings. The change was made 3 years ago and I wouldn't object to
> dropping this completely, but I understand I may not be the most
> cautious person when it comes to ensuring DT backward compatibility :-)

My memory is that the goal was to eventually remove these overlays
at some point in the future.  If everyone agrees that today is the
proper time, it would be helpful to go ahead and remove these .dts
files and the code that uses them.

-Frank

> 
>> $ find drivers -name '*.dts'
>> drivers/staging/pi433/Documentation/devicetree/pi433-overlay.dts
>> drivers/staging/mt7621-dts/gbpc2.dts
>> drivers/staging/mt7621-dts/gbpc1.dts
>> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
>> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
>> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
>> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
>> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts
>> drivers/of/unittest-data/overlay_1.dts
>> drivers/of/unittest-data/testcases.dts
>> drivers/of/unittest-data/overlay_bad_add_dup_node.dts
>> drivers/of/unittest-data/overlay_bad_symbol.dts
>> drivers/of/unittest-data/overlay_0.dts
>> drivers/of/unittest-data/overlay_11.dts
>> drivers/of/unittest-data/overlay_gpio_03.dts
>> drivers/of/unittest-data/overlay_gpio_04a.dts
>> drivers/of/unittest-data/overlay_gpio_04b.dts
>> drivers/of/unittest-data/overlay_5.dts
>> drivers/of/unittest-data/overlay_bad_add_dup_prop.dts
>> drivers/of/unittest-data/overlay_gpio_01.dts
>> drivers/of/unittest-data/overlay_10.dts
>> drivers/of/unittest-data/overlay_7.dts
>> drivers/of/unittest-data/overlay_bad_phandle.dts
>> drivers/of/unittest-data/overlay_3.dts
>> drivers/of/unittest-data/overlay_6.dts
>> drivers/of/unittest-data/overlay_8.dts
>> drivers/of/unittest-data/overlay_12.dts
>> drivers/of/unittest-data/overlay_gpio_02a.dts
>> drivers/of/unittest-data/overlay_gpio_02b.dts
>> drivers/of/unittest-data/overlay_4.dts
>> drivers/of/unittest-data/overlay.dts
>> drivers/of/unittest-data/overlay_9.dts
>> drivers/of/unittest-data/overlay_2.dts
>> drivers/of/unittest-data/overlay_15.dts
>> drivers/of/unittest-data/overlay_base.dts
>> drivers/of/unittest-data/overlay_13.dts
>
Viresh Kumar March 16, 2021, 5:42 a.m. UTC | #21 
On 16-03-21, 00:36, Frank Rowand wrote:
> I should have looked at patch 3/5 more carefully instead of counting on
> Masahiro to check it out and simply build testing.
> 
> Patch 3/5 does not seem correct.  I'll look over all the makefile related
> changes that have been accepted (if any) and patch 3/5 later today (Tuesday).

What is already merged by Rob is what everyone reviewed and it wasn't
related to this .dtso/.dtbo discussion we are having. I will resend a
new patchset with the stuff left for merging (which will involve the
thing we are discussing here), so we can get a clear picture of it.
Geert Uytterhoeven March 16, 2021, 8:01 a.m. UTC | #22 
Hi Frank,

On Tue, Mar 16, 2021 at 6:39 AM Frank Rowand <frowand.list@gmail.com> wrote:
> On 3/15/21 5:12 PM, Laurent Pinchart wrote:
> > On Tue, Mar 16, 2021 at 02:43:45AM +0900, Masahiro Yamada wrote:
> >> But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
> >> if these are doing bad things.
> >> They seem to be overlay files even though the file name suffix is .dts
> >
> > That is correct, they are overlays. I have no issue with those files
> > being renamed to .dtso if that can help (but I haven't checked if that
> > would have any adverse effect on the R-Car DU driver).
>
> As Laurent replied, yes these are overlays.  They were grandfathered in
> as a deprecated use of overlays.
>
> > These files are there to ensure backward compatibility with older DT
> > bindings. The change was made 3 years ago and I wouldn't object to
> > dropping this completely, but I understand I may not be the most
> > cautious person when it comes to ensuring DT backward compatibility :-)
>
> My memory is that the goal was to eventually remove these overlays
> at some point in the future.  If everyone agrees that today is the
> proper time, it would be helpful to go ahead and remove these .dts
> files and the code that uses them.

Given [1][2][3] were merged in v4.17, and [4] was merged in v4.20, and
all were backported to the old v4.14-based R-Car BSP v3.8.0, I think
it's safe to assume all users have the DTS updates, so the backward
compatibility mode can be removed?

> >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
> >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
> >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
> >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
> >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts

[1] 15a1ff30d8f9bd83 ("ARM: dts: r8a7790: Convert to new LVDS DT bindings")
[2] e5c3f4707f3956a2 ("ARM: dts: r8a7791: Convert to new LVDS DT bindings")
[3] edb0c3affe5214a2 ("ARM: dts: r8a7793: Convert to new LVDS DT bindings")
[4] 58e8ed2ee9abe718 ("arm64: dts: renesas: Convert to new LVDS DT bindings")

Gr{oetje,eeting}s,

                        Geert


--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
Frank Rowand March 24, 2021, 7:34 a.m. UTC | #23 
On 3/16/21 12:42 AM, Viresh Kumar wrote:
> On 16-03-21, 00:36, Frank Rowand wrote:
>> I should have looked at patch 3/5 more carefully instead of counting on
>> Masahiro to check it out and simply build testing.
>>
>> Patch 3/5 does not seem correct.  I'll look over all the makefile related
>> changes that have been accepted (if any) and patch 3/5 later today (Tuesday).
> 
> What is already merged by Rob is what everyone reviewed and it wasn't
> related to this .dtso/.dtbo discussion we are having. I will resend a
> new patchset with the stuff left for merging (which will involve the
> thing we are discussing here), so we can get a clear picture of it.
> 

Instead of doing what I suggested in my 16-03-21, 00:36 email (only
partly quoted above), I have made most of the changes to unittest.c
and drivers/of/unitest/unittest-data/Makefile to allow the unittest
.dts files to be .dtso source files and .dtbo FDT files, and tested.
One final step remaining in my work is to actually rename the *.dts
files to *.dtso.

I hope to finish this later today (Wednesday) after getting some
sleep.

-Frank
Frank Rowand March 24, 2021, 10:45 p.m. UTC | #24 
Hi Viresh,

On 3/24/21 2:34 AM, Frank Rowand wrote:
> On 3/16/21 12:42 AM, Viresh Kumar wrote:
>> On 16-03-21, 00:36, Frank Rowand wrote:
>>> I should have looked at patch 3/5 more carefully instead of counting on
>>> Masahiro to check it out and simply build testing.
>>>
>>> Patch 3/5 does not seem correct.  I'll look over all the makefile related
>>> changes that have been accepted (if any) and patch 3/5 later today (Tuesday).
>>
>> What is already merged by Rob is what everyone reviewed and it wasn't
>> related to this .dtso/.dtbo discussion we are having. I will resend a
>> new patchset with the stuff left for merging (which will involve the
>> thing we are discussing here), so we can get a clear picture of it.
>>
> 
> Instead of doing what I suggested in my 16-03-21, 00:36 email (only
> partly quoted above), I have made most of the changes to unittest.c
> and drivers/of/unitest/unittest-data/Makefile to allow the unittest
> .dts files to be .dtso source files and .dtbo FDT files, and tested.
> One final step remaining in my work is to actually rename the *.dts
> files to *.dtso.
> 
> I hope to finish this later today (Wednesday) after getting some
> sleep.

I have submitted a patch that I _think_ removes the need for most
of patch 3/5, _other_ than the yaml rule, which I would assume is
still needed, though I did not examine it.

My patch is 
https://lore.kernel.org/r/20210324223713.1334666-1-frowand.list@gmail.com

Can you do a new patch with just the yaml rule (if Rob accepts my patch).

-Frank
Masahiro Yamada April 9, 2021, 6:03 a.m. UTC | #25 
On Tue, Mar 16, 2021 at 5:01 PM Geert Uytterhoeven <geert@linux-m68k.org> wrote:
>
> Hi Frank,
>
> On Tue, Mar 16, 2021 at 6:39 AM Frank Rowand <frowand.list@gmail.com> wrote:
> > On 3/15/21 5:12 PM, Laurent Pinchart wrote:
> > > On Tue, Mar 16, 2021 at 02:43:45AM +0900, Masahiro Yamada wrote:
> > >> But how can we fix drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a779*.dts
> > >> if these are doing bad things.
> > >> They seem to be overlay files even though the file name suffix is .dts
> > >
> > > That is correct, they are overlays. I have no issue with those files
> > > being renamed to .dtso if that can help (but I haven't checked if that
> > > would have any adverse effect on the R-Car DU driver).
> >
> > As Laurent replied, yes these are overlays.  They were grandfathered in
> > as a deprecated use of overlays.
> >
> > > These files are there to ensure backward compatibility with older DT
> > > bindings. The change was made 3 years ago and I wouldn't object to
> > > dropping this completely, but I understand I may not be the most
> > > cautious person when it comes to ensuring DT backward compatibility :-)
> >
> > My memory is that the goal was to eventually remove these overlays
> > at some point in the future.  If everyone agrees that today is the
> > proper time, it would be helpful to go ahead and remove these .dts
> > files and the code that uses them.
>
> Given [1][2][3] were merged in v4.17, and [4] was merged in v4.20, and
> all were backported to the old v4.14-based R-Car BSP v3.8.0, I think
> it's safe to assume all users have the DTS updates, so the backward
> compatibility mode can be removed?
>
> > >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
> > >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
> > >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
> > >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
> > >> drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts
>
> [1] 15a1ff30d8f9bd83 ("ARM: dts: r8a7790: Convert to new LVDS DT bindings")
> [2] e5c3f4707f3956a2 ("ARM: dts: r8a7791: Convert to new LVDS DT bindings")
> [3] edb0c3affe5214a2 ("ARM: dts: r8a7793: Convert to new LVDS DT bindings")
> [4] 58e8ed2ee9abe718 ("arm64: dts: renesas: Convert to new LVDS DT bindings")


Can we remove all of drivers/gpu/drm/rcar-du/*.dts ?

I see some more under drivers/staging/.


masahiro@grover:~/ref/linux-next$ find  drivers  -name  '*.dts'  |
grep -v drivers/of/unittest-data
drivers/staging/pi433/Documentation/devicetree/pi433-overlay.dts
drivers/staging/mt7621-dts/gbpc2.dts
drivers/staging/mt7621-dts/gbpc1.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7791.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7795.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7796.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7793.dts
drivers/gpu/drm/rcar-du/rcar_du_of_lvds_r8a7790.dts







Best Regards

Masahiro Yamada
diff mbox series

Patch

diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index bc045a54a34e..59e86f67f9e0 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -339,7 +339,7 @@  $(obj)/%.dtb.S: $(obj)/%.dtb FORCE
 
 quiet_cmd_dtc = DTC     $@
 cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ; \
-	$(DTC) -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
+	$(DTC) -I dts -O $(patsubst .%,%,$(suffix $@)) -o $@ -b 0 \
 		$(addprefix -i,$(dir $<) $(DTC_INCLUDE)) $(DTC_FLAGS) \
 		-d $(depfile).dtc.tmp $(dtc-tmp) ; \
 	cat $(depfile).pre.tmp $(depfile).dtc.tmp > $(depfile)
@@ -347,9 +347,13 @@  cmd_dtc = $(HOSTCC) -E $(dtc_cpp_flags) -x assembler-with-cpp -o $(dtc-tmp) $< ;
 $(obj)/%.dtb: $(src)/%.dts $(DTC) FORCE
 	$(call if_changed_dep,dtc)
 
+# Required for of unit-test files as they can't be renamed to .dtso
 $(obj)/%.dtbo: $(src)/%.dts $(DTC) FORCE
 	$(call if_changed_dep,dtc)
 
+$(obj)/%.dtbo: $(src)/%.dtso $(DTC) FORCE
+	$(call if_changed_dep,dtc)
+
 overlay-y := $(addprefix $(obj)/, $(overlay-y))
 
 quiet_cmd_fdtoverlay = DTOVL   $@
@@ -375,6 +379,9 @@  endef
 $(obj)/%.dt.yaml: $(src)/%.dts $(DTC) $(DT_TMP_SCHEMA) FORCE
 	$(call if_changed_rule,dtc,yaml)
 
+$(obj)/%.dt.yaml: $(src)/%.dtso $(DTC) $(DT_TMP_SCHEMA) FORCE
+	$(call if_changed_rule,dtc,yaml)
+
 dtc-tmp = $(subst $(comma),_,$(dot-target).dts.tmp)
 
 # Bzip2