diff mbox

[RESEND] ARM: add dtbImage.<dt> and dtbuImage.<dt> rules

Message ID 1348733441-11626-1-git-send-email-richard.genoud@gmail.com (mailing list archive)
State New, archived
Headers show

Commit Message

Richard Genoud Sept. 27, 2012, 8:10 a.m. UTC
[add Grant Likely and Rob Herring in To: since this is maybe more OF
related than ARM related]

This rules are useful for appended device tree conforming to the
CONFIG_ARM_APPENDED_DTB kernel option.

The rule dtbImage.<dt> is actually just a:
cat zImage <dt>.dtb > dtbImage.<dt>

The dtbuImage.<dt> makes an uImage out of the dtbImage.<dt> file.

KernelVersion: v3.6-rc5

Signed-off-by: Richard Genoud <richard.genoud@gmail.com>
Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
---
 arch/arm/Makefile      |   30 ++++++++++++++++--------------
 arch/arm/boot/Makefile |   11 +++++++++++
 scripts/Makefile.lib   |    6 ++++++
 3 files changed, 33 insertions(+), 14 deletions(-)

Comments

Jean-Christophe PLAGNIOL-VILLARD Sept. 27, 2012, 8:29 a.m. UTC | #1
On 10:10 Thu 27 Sep     , Richard Genoud wrote:
> [add Grant Likely and Rob Herring in To: since this is maybe more OF
> related than ARM related]
> 
> This rules are useful for appended device tree conforming to the
> CONFIG_ARM_APPENDED_DTB kernel option.
> 
> The rule dtbImage.<dt> is actually just a:
> cat zImage <dt>.dtb > dtbImage.<dt>
> 
> The dtbuImage.<dt> makes an uImage out of the dtbImage.<dt> file.
> 
> KernelVersion: v3.6-rc5
> 
> Signed-off-by: Richard Genoud <richard.genoud@gmail.com>
> Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
as discuss with Grant no this is a temporary solution so no Makefile Target

so nack

Best Regards,
J.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Richard Genoud Sept. 27, 2012, 9:08 a.m. UTC | #2
2012/9/27 Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>:
>> Signed-off-by: Richard Genoud <richard.genoud@gmail.com>
>> Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
> as discuss with Grant no this is a temporary solution so no Makefile Target
>
> so nack

hum, it doesn't seems to be temporary on powerpc.
cf 25431333813686654907ab987fb5de10c10a16db
and
ARCH=powerpc make help

So what's the problem on arm ?
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Gerlando Falauto Sept. 24, 2013, 10:55 a.m. UTC | #3
Hi everyone,

apologies for picking up a topic that's more than 360 days old...

We're in the position where we'd like to have FDT support just because 
it simplifies our code a whole lot; yet we want to be able to run such a 
kernel on older u-boots (completely unaware of FTDs) as well.
I believe this must be a very common use case indeed.
So a dtbuImage.<dtb> would be very useful in this case.

If I understand it correctly, dtbImage.<dtb> is indeed just a cat of 
zImage+dtb, whereas dtbuImage.<dtb> is the uimage of such a dtbImage.<dtb>.
So it shouldn't be such a crazy idea to create a dtbuImage.<dtb> within 
Kbuild (as opposed to from an external wrapper for make).

Is there any reason why nothing like that was ever mainlined?
Or maybe I'm just missing something?

Thanks!
Gerlando

On 09/27/2012 11:08 AM, Richard Genoud wrote:
> 2012/9/27 Jean-Christophe PLAGNIOL-VILLARD <plagnioj@jcrosoft.com>:
>>> Signed-off-by: Richard Genoud <richard.genoud@gmail.com>
>>> Tested-by: Gregory CLEMENT <gregory.clement@free-electrons.com>
>> as discuss with Grant no this is a temporary solution so no Makefile Target
>>
>> so nack
>
> hum, it doesn't seems to be temporary on powerpc.
> cf 25431333813686654907ab987fb5de10c10a16db
> and
> ARCH=powerpc make help
>
> So what's the problem on arm ?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jason Cooper Sept. 24, 2013, 12:01 p.m. UTC | #4
Gerlando,

On Tue, Sep 24, 2013 at 12:55:19PM +0200, Gerlando Falauto wrote:
> We're in the position where we'd like to have FDT support just
> because it simplifies our code a whole lot;

Great!

> yet we want to be able to run such a kernel on older u-boots
> (completely unaware of FTDs) as well.
> I believe this must be a very common use case indeed.

Yes, very much so.  With the understanding that this is an artifact of
the transition, hopefully not a permanent state. ;-)

> So a dtbuImage.<dtb> would be very useful in this case.

well...

> If I understand it correctly, dtbImage.<dtb> is indeed just a cat of
> zImage+dtb, whereas dtbuImage.<dtb> is the uimage of such a
> dtbImage.<dtb>.

Yes.

> So it shouldn't be such a crazy idea to create a dtbuImage.<dtb>
> within Kbuild (as opposed to from an external wrapper for make).

Yes :)  The problem is that we are moving to multiplatform zImages.  In
that scenario, the uImage target isn't very helpful because it needs to
be custom to each board.  Additionally, appended-dtb was intended as a
developer tool and isn't intended as a deployable item.  It also
defeats the purpose of multiplatform.

> Is there any reason why nothing like that was ever mainlined?
> Or maybe I'm just missing something?

We are strongly encouraging all vendors to upgrade their bootloaders to
support field upgradeable dtbs.  eg having the dtb on a separate
partition of the flash (or whereever the bootloader environment is)

iirc, keymile is using an odd flavor of Kirkwood.  Have you tried kwboot
over serial?  It can be found in barebox and u-boot source repositories.

The above is the long-term goal, in the short term, please take a look
at:

https://github.com/zonque/pxa-impedance-matcher

You can use this to boot a modern fdt kernel on a legacy bootloader,
with one or more dtb's linked in.  I tested it on the dreamplug, so it's
mostly there for you.

The big outstanding TODO I have on it is to parse the atags and modify
the selected dtb before passing off to the kernel.  Patches welcome! :)

hth,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Russell King - ARM Linux Sept. 24, 2013, 12:19 p.m. UTC | #5
On Tue, Sep 24, 2013 at 08:01:52AM -0400, Jason Cooper wrote:
> On Tue, Sep 24, 2013 at 12:55:19PM +0200, Gerlando Falauto wrote:
> > So it shouldn't be such a crazy idea to create a dtbuImage.<dtb>
> > within Kbuild (as opposed to from an external wrapper for make).
> 
> Yes :)  The problem is that we are moving to multiplatform zImages.  In
> that scenario, the uImage target isn't very helpful because it needs to
> be custom to each board.  Additionally, appended-dtb was intended as a
> developer tool and isn't intended as a deployable item.  It also
> defeats the purpose of multiplatform.

And we've already taken the decision that we will not be augmenting
kbuild with any more ubooot specific stuff, especially with anything
that goes against the long term goal of having a single kernel image
which is bootable across multiple platforms.

Having kbuild wrap the zImage + DTB up into a uImage goes directly
against this goal, because it turns the produced kernel into something
which is specific to a single target.  It's much worse than what we
had before we went to DT, because with this it can only be used on a
_single_ target, whereas prior to DT we could boot a single kernel
on a variety of compatible targets.

Moreover, the support for appending the DTB to the zImage is a stop-gap
solution for the transition to a full DT based environment - don't
expect it to be around into the long term future.  Modern platforms
should be using a DT compatible boot loader, which means you don't
want to use the DTB appended to zImage method.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Richard Genoud Sept. 24, 2013, 12:34 p.m. UTC | #6
2013/9/24 Russell King - ARM Linux <linux@arm.linux.org.uk>:
> Modern platforms
> should be using a DT compatible boot loader, which means you don't
> want to use the DTB appended to zImage method.

Even small bootloaders like at91bootstrap can handle DT blob now (that
was my main problem at the time I wrote this patch)
So I don't use it anymore. (but as a developer tool, this can be convenient).

Richard.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Gerlando Falauto Sept. 24, 2013, 3:14 p.m. UTC | #7
Hi Jason,

On 09/24/2013 02:01 PM, Jason Cooper wrote:
> Gerlando,
[...]

> We are strongly encouraging all vendors to upgrade their bootloaders to
> support field upgradeable dtbs.  eg having the dtb on a separate
> partition of the flash (or whereever the bootloader environment is)

This will most definitely be the case for future boards, but we sort of 
follow the unspoken guideline of not upgrading u-boot on field-deployed 
units. Arguable at least, but that's the way it is.

> iirc, keymile is using an odd flavor of Kirkwood.

Correct, it's a kirkwood + a switch on the same die.

> Have you tried kwboot over serial?  It can be found in barebox and u-boot source repositories.

Uhm, yes, I've used it a few times... but what are you suggesting?
That u-boot can be upgraded? Yes, I believe that... ;-)

> The above is the long-term goal, in the short term, please take a look
> at:
>
> https://github.com/zonque/pxa-impedance-matcher
>
> You can use this to boot a modern fdt kernel on a legacy bootloader,
> with one or more dtb's linked in.  I tested it on the dreamplug, so it's
> mostly there for you.
>
> The big outstanding TODO I have on it is to parse the atags and modify
> the selected dtb before passing off to the kernel.  Patches welcome! :)

Thank you, I had a quick look at the README and it sounds great!
But CONFIG_APPENDED_DTB already does the job of filling this gap for me...

>
> hth,

It sure did. I really appreciate it. Thanks!!!

Gerlando
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Gerlando Falauto Sept. 24, 2013, 3:14 p.m. UTC | #8
Hi guys,

On 09/24/2013 02:34 PM, Richard Genoud wrote:
> 2013/9/24 Russell King - ARM Linux <linux@arm.linux.org.uk>:
>> Modern platforms
>> should be using a DT compatible boot loader, which means you don't
>> want to use the DTB appended to zImage method.
>
> Even small bootloaders like at91bootstrap can handle DT blob now (that
> was my main problem at the time I wrote this patch)
> So I don't use it anymore. (but as a developer tool, this can be convenient).
>
> Richard.
>

Thanks a lot for your replies.
I now understand your point, your reasons and, most importantly, where 
we're headed and what to expect in the future.

Thanks a lot!
Gerlando
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Jason Cooper Sept. 24, 2013, 3:54 p.m. UTC | #9
On Tue, Sep 24, 2013 at 05:14:26PM +0200, Gerlando Falauto wrote:
> On 09/24/2013 02:01 PM, Jason Cooper wrote:
> >We are strongly encouraging all vendors to upgrade their bootloaders to
> >support field upgradeable dtbs.  eg having the dtb on a separate
> >partition of the flash (or whereever the bootloader environment is)
> 
> This will most definitely be the case for future boards, but we sort
> of follow the unspoken guideline of not upgrading u-boot on
> field-deployed units. Arguable at least, but that's the way it is.

Understandable.

> >Have you tried kwboot over serial?  It can be found in barebox and u-boot source repositories.
> 
> Uhm, yes, I've used it a few times... but what are you suggesting?
> That u-boot can be upgraded? Yes, I believe that... ;-)

I'm suggesting that I primarily talk to hobbyists and tinkerers, and
occasionally forget there are people that have jtag access :-P

> >The above is the long-term goal, in the short term, please take a look
> >at:
> >
> >https://github.com/zonque/pxa-impedance-matcher
> >
> >You can use this to boot a modern fdt kernel on a legacy bootloader,
> >with one or more dtb's linked in.  I tested it on the dreamplug, so it's
> >mostly there for you.
> >
> >The big outstanding TODO I have on it is to parse the atags and modify
> >the selected dtb before passing off to the kernel.  Patches welcome! :)
> 
> Thank you, I had a quick look at the README and it sounds great!
> But CONFIG_APPENDED_DTB already does the job of filling this gap for me...

Right, it's intended as a replacement for APPENDED_DTB since it can
handle multiple dtbs (so you can do one multiplatform zImage), and you
can hack around custom vendor atags.  eg ethernet mac addresses, Thomas'
CP15 register base address hack, etc.  All that goodness without
polluting the kernel source.  :)

thx,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Tim Bird Sept. 30, 2013, 6:14 p.m. UTC | #10
On Tue, Sep 24, 2013 at 5:19 AM, Russell King - ARM Linux
<linux@arm.linux.org.uk> wrote:
> On Tue, Sep 24, 2013 at 08:01:52AM -0400, Jason Cooper wrote:
>> On Tue, Sep 24, 2013 at 12:55:19PM +0200, Gerlando Falauto wrote:
>> > So it shouldn't be such a crazy idea to create a dtbuImage.<dtb>
>> > within Kbuild (as opposed to from an external wrapper for make).
>>
>> Yes :)  The problem is that we are moving to multiplatform zImages.  In
>> that scenario, the uImage target isn't very helpful because it needs to
>> be custom to each board.  Additionally, appended-dtb was intended as a
>> developer tool and isn't intended as a deployable item.  It also
>> defeats the purpose of multiplatform.
>
> And we've already taken the decision that we will not be augmenting
> kbuild with any more ubooot specific stuff, especially with anything
> that goes against the long term goal of having a single kernel image
> which is bootable across multiple platforms.
>
> Having kbuild wrap the zImage + DTB up into a uImage goes directly
> against this goal, because it turns the produced kernel into something
> which is specific to a single target.  It's much worse than what we
> had before we went to DT, because with this it can only be used on a
> _single_ target, whereas prior to DT we could boot a single kernel
> on a variety of compatible targets.
>
> Moreover, the support for appending the DTB to the zImage is a stop-gap
> solution for the transition to a full DT based environment - don't
> expect it to be around into the long term future.  Modern platforms
> should be using a DT compatible boot loader, which means you don't
> want to use the DTB appended to zImage method.
>

What is the long-term mechanism/methodology for creating a kernel
that runs on a single system?  So many drivers now have device-tree
dependencies that it seems unlikely we'll be able to avoid using
device tree.  But if we don't care or don't want to support more than
a single platform, what should we do (if appending DTB to the zImage
will eventually go away).  Will the requirement be that all future
Linux bootloaders will be DT-aware?

 -- Tim Bird
Senior Software Engineer, Sony Mobile
Architecture Group Chair, CE Workgroup, Linux Foundation
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Maxime Bizon Sept. 30, 2013, 8:21 p.m. UTC | #11
On Tue, 2013-09-24 at 13:19 +0100, Russell King - ARM Linux wrote:

Hi Russell,
> 
> Having kbuild wrap the zImage + DTB up into a uImage goes directly
> against this goal, because it turns the produced kernel into something
> which is specific to a single target.  It's much worse than what we
> had before we went to DT, because with this it can only be used on a
> _single_ target, whereas prior to DT we could boot a single kernel
> on a variety of compatible targets.

What would you think of a patch allowing multiple DTB selected using
ATAG number ?

I don't believe telling people to upgrade bootloaders in the field is a
reasonable thing to ask.

We can make fun of vendors for leaving devices with outdated kernel
versions, but this is making life harder for the one who played by the
rules.
Jason Cooper Sept. 30, 2013, 8:42 p.m. UTC | #12
On Mon, Sep 30, 2013 at 10:21:23PM +0200, Maxime Bizon wrote:
> On Tue, 2013-09-24 at 13:19 +0100, Russell King - ARM Linux wrote:
> > Having kbuild wrap the zImage + DTB up into a uImage goes directly
> > against this goal, because it turns the produced kernel into something
> > which is specific to a single target.  It's much worse than what we
> > had before we went to DT, because with this it can only be used on a
> > _single_ target, whereas prior to DT we could boot a single kernel
> > on a variety of compatible targets.
> 
> What would you think of a patch allowing multiple DTB selected using
> ATAG number ?

Nicolas Pitre already suggested an impedance matcher that keeps this
kind of hacky code out of the kernel, and allows users to do exactly
that.  Daniel Mack took him up on the suggestion and implemented it [1].
I contributed some patches to make it work generically.

Please take a look.  Currently, the only outstanding blocker is to convert
atags to fdt.  Patches welcome as I'm currently occupied with other
things. ;-)

hth,

Jason.

[1] https://github.com/zonque/pxa-impedance-matcher
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Maxime Bizon Sept. 30, 2013, 9:07 p.m. UTC | #13
On Mon, 2013-09-30 at 16:42 -0400, Jason Cooper wrote:

> Nicolas Pitre already suggested an impedance matcher that keeps this
> kind of hacky code out of the kernel, and allows users to do exactly

I know that, my proposal was to bring this back into the kernel.

That "kind of hacky" ATAG thing was actually the only supported way to
boot a platform not so long ago :)

In the end, people needing the impedance matcher will somehow integrate
it in the kernel build system anyway.
Rob Herring Sept. 30, 2013, 10:44 p.m. UTC | #14
On Mon, Sep 30, 2013 at 1:14 PM, Tim Bird <tbird20d@gmail.com> wrote:
> On Tue, Sep 24, 2013 at 5:19 AM, Russell King - ARM Linux
> <linux@arm.linux.org.uk> wrote:
>> On Tue, Sep 24, 2013 at 08:01:52AM -0400, Jason Cooper wrote:
>>> On Tue, Sep 24, 2013 at 12:55:19PM +0200, Gerlando Falauto wrote:
>>> > So it shouldn't be such a crazy idea to create a dtbuImage.<dtb>
>>> > within Kbuild (as opposed to from an external wrapper for make).
>>>
>>> Yes :)  The problem is that we are moving to multiplatform zImages.  In
>>> that scenario, the uImage target isn't very helpful because it needs to
>>> be custom to each board.  Additionally, appended-dtb was intended as a
>>> developer tool and isn't intended as a deployable item.  It also
>>> defeats the purpose of multiplatform.
>>
>> And we've already taken the decision that we will not be augmenting
>> kbuild with any more ubooot specific stuff, especially with anything
>> that goes against the long term goal of having a single kernel image
>> which is bootable across multiple platforms.
>>
>> Having kbuild wrap the zImage + DTB up into a uImage goes directly
>> against this goal, because it turns the produced kernel into something
>> which is specific to a single target.  It's much worse than what we
>> had before we went to DT, because with this it can only be used on a
>> _single_ target, whereas prior to DT we could boot a single kernel
>> on a variety of compatible targets.
>>
>> Moreover, the support for appending the DTB to the zImage is a stop-gap
>> solution for the transition to a full DT based environment - don't
>> expect it to be around into the long term future.  Modern platforms
>> should be using a DT compatible boot loader, which means you don't
>> want to use the DTB appended to zImage method.
>>
>
> What is the long-term mechanism/methodology for creating a kernel
> that runs on a single system?  So many drivers now have device-tree
> dependencies that it seems unlikely we'll be able to avoid using
> device tree.  But if we don't care or don't want to support more than
> a single platform, what should we do (if appending DTB to the zImage
> will eventually go away).  Will the requirement be that all future
> Linux bootloaders will be DT-aware?

I don't think appended DTB will go away. In fact, use of it is now
being added to flash-kernel in debian/ubuntu. However, this is only
done at OS install time on running on the h/w. The problem goes beyond
just having a DT aware bootloader. You also need the bootloader to
ship with the DTB. This isn't happening yet, so the distros are having
to install their own DTB. Since there is not a standard way to install
a dtb and tell the bootloader where it is, the appended dtb is the
quick solution.

Rob
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Nicolas Pitre Oct. 1, 2013, 2:40 a.m. UTC | #15
On Mon, 30 Sep 2013, Tim Bird wrote:

> What is the long-term mechanism/methodology for creating a kernel
> that runs on a single system?  So many drivers now have device-tree
> dependencies that it seems unlikely we'll be able to avoid using
> device tree.  But if we don't care or don't want to support more than
> a single platform, what should we do (if appending DTB to the zImage
> will eventually go away).  Will the requirement be that all future
> Linux bootloaders will be DT-aware?

The ability to append a DTB to zImage is not going away tomorrow, 
although I encourage you to use another solution if possible.

What can be done on a simple single platform setup is to load the DTB 
from flash into RAM just like another blob along with the ramdisk image 
and the zImage itself.  Then you just have to branch into zImage with 
the address of the DTB in RAM into r2.  Most bootloaders are able to 
load arbitrary binaries into memory, and most bootloaders (except for 
U-Boot) are able to set arbitrary values into r0-r3 before branching to 
some location in memory.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Nicolas Pitre Oct. 1, 2013, 3:14 a.m. UTC | #16
On Mon, 30 Sep 2013, Maxime Bizon wrote:

> 
> On Mon, 2013-09-30 at 16:42 -0400, Jason Cooper wrote:
> 
> > Nicolas Pitre already suggested an impedance matcher that keeps this
> > kind of hacky code out of the kernel, and allows users to do exactly
> 
> I know that, my proposal was to bring this back into the kernel.

And obviously I disagree with such a proposal.

> That "kind of hacky" ATAG thing was actually the only supported way to
> boot a platform not so long ago :)

Strictly speaking, this was never supported in mainline.

> In the end, people needing the impedance matcher will somehow integrate
> it in the kernel build system anyway.

As long as it remains their own cost to maintain then nobody will 
object, which means "not in mainline".

The ultimate idea with the impedance matcher was to augment existing 
bootloaders with sort of a semi second stage bootloader that would be 
separate from zImage and installed permanently, and then only the zImage 
itself would be updateable.  This way you'd get a DT capable environment 
without replacing the actual bootloader.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Nicolas Pitre Oct. 1, 2013, 3:21 a.m. UTC | #17
On Mon, 30 Sep 2013, Rob Herring wrote:

> I don't think appended DTB will go away. In fact, use of it is now
> being added to flash-kernel in debian/ubuntu. However, this is only
> done at OS install time on running on the h/w.

Excellent!  That is exactly the use case this feature was designed for.  
Otherwise we would simply have linked the DTB into the kernel image 
directly.

Appending a DTB to zImage should be done at kernel installation time, 
not at build time.  The nuance might not mean a lot for some 
environment, but for a distro this is clearly separate operations.


Nicolas
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Maxime Bizon Oct. 1, 2013, 10:41 a.m. UTC | #18
On Monday 30 Sep 2013 à 23:14:49 (-0400), Nicolas Pitre wrote:

> > That "kind of hacky" ATAG thing was actually the only supported way to
> > boot a platform not so long ago :)
> 
> Strictly speaking, this was never supported in mainline.

I meant mach-type, which was ofc documented

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/Documentation/arm/Booting?id=1da177e4c3f41524e886b7f1b8a0c1fc7321cac2

> The ultimate idea with the impedance matcher was to augment existing 
> bootloaders with sort of a semi second stage bootloader that would be 
> separate from zImage and installed permanently, and then only the zImage 
> itself would be updateable.  This way you'd get a DT capable environment 
> without replacing the actual bootloader.

That's true if you have access to the bootloader in some way (serial,
flash env).

But for devices in the field for which upgrading the booloader is not
an option, neither is changing anything else in the boot process
(anything before linux starts). The risk of bricking the device is
just too high.

Now of course we have the capability to support these devices
ourselves, I just find it harsh that having "designed" that boot
process in 2010, we are told 3 years later that it is obsolete and
won't be able to boot a recent kernel (we support two machines with
one kernel image, appended DTB cannot handle that).

If I were redesigning that product now, would I use a DT aware
bootloader ? you'd tell me "yes go for it", but hey, you broke my
build, can I trust you ? :)
Jason Cooper Oct. 1, 2013, 11:12 a.m. UTC | #19
On Tue, Oct 01, 2013 at 12:41:15PM +0200, Maxime Bizon wrote:
> On Monday 30 Sep 2013 à 23:14:49 (-0400), Nicolas Pitre wrote:
> 
> > > That "kind of hacky" ATAG thing was actually the only supported way to
> > > boot a platform not so long ago :)
> > 
> > Strictly speaking, this was never supported in mainline.

I was referring to the non-approved atags many vendors arbitrarily added
to their bootloaders and kernels.  Ethernet mac addresses, etc.  It's
not proper to go cluttering the kernel with every hair-brained, one-off
tag a board vendor came up with to ship a product.

> > The ultimate idea with the impedance matcher was to augment existing 
> > bootloaders with sort of a semi second stage bootloader that would be 
> > separate from zImage and installed permanently, and then only the zImage 
> > itself would be updateable.  This way you'd get a DT capable environment 
> > without replacing the actual bootloader.
> 
> That's true if you have access to the bootloader in some way (serial,
> flash env).
> 
> But for devices in the field for which upgrading the booloader is not
> an option, neither is changing anything else in the boot process
> (anything before linux starts). The risk of bricking the device is
> just too high.

This is the exact scenario we are solving with the impedance matcher.
It's a wrapper around a kernel and a series of dtbs.  To the bootloader,
it looks _exactly_ like the kernel, but it translates the info passed to
it into terms newer kernels can understand.  No need for a separate
binary.

hth,

Jason.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Russell King - ARM Linux Oct. 2, 2013, 9:14 p.m. UTC | #20
On Tue, Oct 01, 2013 at 12:41:15PM +0200, Maxime Bizon wrote:
> But for devices in the field for which upgrading the booloader is not
> an option, neither is changing anything else in the boot process
> (anything before linux starts). The risk of bricking the device is
> just too high.

Well, how does the kernel get on to these devices in the field?  Is
it burned into the flash using some laser type device, or does it
involve running some kind of a script?

If it's running a script, then that script can be modified.  The script
can take the generic zImage, and it can identify what the target hardware
is, and it can pick the appropriate DT image to append to the kernel
and then create whatever format your target uboot wants (whether that
be the new format or the uImage format.)

I'm not sure why people think this is soo difficult.  I've had my build/
boot system doing that since OMAP4 went DT only.  I'm now using it on the
Cubox-i Carrier-1 board.  I'm doing it manually there - concatenating the
kernel and DTB and then using mkimage.

If you really must have all the DT blobs as part of the kernel and select
one at boot time, that's precisely where having the kind of solution
Nicolas is suggesting comes in.

We can't keep augmenting the kernel build system with every different
magic way to generate the final platform image - it's got to the point
where there are just _far_ too many different options and to keep
adding more and more is getting really stupid.

Remember too that the plan is to drop the DT files from the kernel
tarball and maintain them in a separate _external_ tree to the kernel.
This means that any "solution" we merge today which allows you to build
a "final platform image" out of a DT file plus the zImage will only
need to be removed again when that happens.  It's far better not to
encourage these lazy methods which will only end up causing us kernel
developers more pain in the future.
--
To unsubscribe from this list: send the line "unsubscribe linux-kbuild" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
diff mbox

Patch

diff --git a/arch/arm/Makefile b/arch/arm/Makefile
index 30eae87..c5f2673 100644
--- a/arch/arm/Makefile
+++ b/arch/arm/Makefile
@@ -278,7 +278,7 @@  archprepare:
 # Convert bzImage to zImage
 bzImage: zImage
 
-zImage Image xipImage bootpImage uImage: vmlinux
+zImage Image xipImage bootpImage uImage dtbImage.% dtbuImage.% : vmlinux
 	$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $(boot)/$@
 
 zinstall uinstall install: vmlinux
@@ -300,17 +300,19 @@  i zi:;	$(Q)$(MAKE) $(build)=$(boot) MACHINE=$(MACHINE) $@
 
 
 define archhelp
-  echo  '* zImage        - Compressed kernel image (arch/$(ARCH)/boot/zImage)'
-  echo  '  Image         - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
-  echo  '* xipImage      - XIP kernel image, if configured (arch/$(ARCH)/boot/xipImage)'
-  echo  '  uImage        - U-Boot wrapped zImage'
-  echo  '  bootpImage    - Combined zImage and initial RAM disk' 
-  echo  '                  (supply initrd image via make variable INITRD=<path>)'
-  echo  '  dtbs          - Build device tree blobs for enabled boards'
-  echo  '  install       - Install uncompressed kernel'
-  echo  '  zinstall      - Install compressed kernel'
-  echo  '  uinstall      - Install U-Boot wrapped compressed kernel'
-  echo  '                  Install using (your) ~/bin/$(INSTALLKERNEL) or'
-  echo  '                  (distribution) /sbin/$(INSTALLKERNEL) or'
-  echo  '                  install to $$(INSTALL_PATH) and run lilo'
+  echo  '* zImage         - Compressed kernel image (arch/$(ARCH)/boot/zImage)'
+  echo  '  Image          - Uncompressed kernel image (arch/$(ARCH)/boot/Image)'
+  echo  '* xipImage       - XIP kernel image, if configured (arch/$(ARCH)/boot/xipImage)'
+  echo  '  uImage         - U-Boot wrapped zImage'
+  echo  '  bootpImage     - Combined zImage and initial RAM disk'
+  echo  '                   (supply initrd image via make variable INITRD=<path>)'
+  echo  '  dtbs           - Build device tree blobs for enabled boards'
+  echo  '  dtbImage.<dt>  - zImage with an appended device tree blob'
+  echo  '  dtbuImage.<dt> - uImage with an embedded device tree blob'
+  echo  '  install        - Install uncompressed kernel'
+  echo  '  zinstall       - Install compressed kernel'
+  echo  '  uinstall       - Install U-Boot wrapped compressed kernel'
+  echo  '                   Install using (your) ~/bin/$(INSTALLKERNEL) or'
+  echo  '                   (distribution) /sbin/$(INSTALLKERNEL) or'
+  echo  '                   install to $$(INSTALL_PATH) and run lilo'
 endef
diff --git a/arch/arm/boot/Makefile b/arch/arm/boot/Makefile
index c877087..35c01a8 100644
--- a/arch/arm/boot/Makefile
+++ b/arch/arm/boot/Makefile
@@ -98,6 +98,17 @@  $(obj)/bootpImage: $(obj)/bootp/bootp FORCE
 	$(call if_changed,objcopy)
 	@echo '  Kernel: $@ is ready'
 
+# dtbImage.% - a dtbImage is a zImage with an appended device tree blob
+$(obj)/dtbImage.%: $(obj)/zImage $(obj)/%.dtb FORCE
+	$(call if_changed,cat)
+	@echo '  Kernel: $@ is ready'
+
+# dtbuImage.% - a dtbuImage is a uImage with an embedded device tree blob
+$(obj)/dtbuImage.%: $(obj)/dtbImage.% FORCE
+	@$(check_for_multiple_loadaddr)
+	$(call if_changed,uimage)
+	@echo '  Image $@ is ready'
+
 PHONY += initrd FORCE
 initrd:
 	@test "$(INITRD_PHYS)" != "" || \
diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
index 0be6f11..8550466 100644
--- a/scripts/Makefile.lib
+++ b/scripts/Makefile.lib
@@ -235,6 +235,12 @@  cmd_ld = $(LD) $(LDFLAGS) $(ldflags-y) $(LDFLAGS_$(@F)) \
 quiet_cmd_objcopy = OBJCOPY $@
 cmd_objcopy = $(OBJCOPY) $(OBJCOPYFLAGS) $(OBJCOPYFLAGS_$(@F)) $< $@
 
+# Cat
+# ---------------------------------------------------------------------------
+
+quiet_cmd_cat = CAT     $@
+cmd_cat = (cat $(filter-out FORCE,$^) > $@) || (rm -f $@ ; false)
+
 # Gzip
 # ---------------------------------------------------------------------------