mbox series

[v3,0/9] MIPS: Unify low-level debugging functionalities

Message ID 20240502-mips_debug_ll-v3-0-3b61f30e484c@flygoat.com (mailing list archive)
Headers show
Series MIPS: Unify low-level debugging functionalities | expand

Message

Jiaxun Yang May 2, 2024, 9:59 a.m. UTC
Hi all,

This is a attempt to bring all low-level debugging print functions
together and provide a arm-like low-level debugging interface and
a further capability to debug early exceptions.

This patch elimiate platform specific early_printk, zboot printing
functions and cps-vec-ns16550 by newly introduced debug_ll.

Hope you'll find them handy :-)

Happy hacking!

Thanks

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
---
Changes in v3:
- Collect review tags
- Fix an indentation
- Link to v2: https://lore.kernel.org/r/20240326-mips_debug_ll-v2-0-b64abc76f2a1@flygoat.com

---
Jiaxun Yang (9):
      MIPS: asm: Move strings to .rodata.str section
      MIPS: debug: Implement low-level debugging functions
      MIPS: debug: Hook up DEBUG_LL with early printk
      MIPS: debug: Provide an early exception vector for low-level debugging
      MIPS: debug_ll: Add Kconfig symbols for some 8250 uarts
      MIPS: debug_ll: Implement support for Alchemy uarts
      MIPS: debug_ll: Implement support for AR933X uarts
      MIPS: zboot: Convert to use debug_ll facilities
      MIPS: CPS: Convert to use debug_ll facilities

 arch/mips/Kconfig                        |  12 +-
 arch/mips/Kconfig.debug                  | 240 +++++++++++++++++++++++--------
 arch/mips/boot/compressed/Makefile       |   9 +-
 arch/mips/boot/compressed/dbg.c          |  39 -----
 arch/mips/boot/compressed/debug-vec.S    |   3 +
 arch/mips/boot/compressed/debug.S        |   3 +
 arch/mips/boot/compressed/decompress.h   |   8 +-
 arch/mips/boot/compressed/head.S         |   6 +
 arch/mips/boot/compressed/uart-16550.c   |  49 -------
 arch/mips/boot/compressed/uart-alchemy.c |   9 --
 arch/mips/boot/compressed/uart-ath79.c   |   2 -
 arch/mips/boot/compressed/uart-prom.c    |   9 --
 arch/mips/include/asm/asm.h              |   2 +-
 arch/mips/include/debug/8250.S           |  60 ++++++++
 arch/mips/include/debug/alchemy.S        |  46 ++++++
 arch/mips/include/debug/ar933x.S         |  41 ++++++
 arch/mips/include/debug/uhi.S            |  48 +++++++
 arch/mips/kernel/Makefile                |   4 +-
 arch/mips/kernel/cps-vec.S               |  16 +--
 arch/mips/kernel/debug-vec.S             | 194 +++++++++++++++++++++++++
 arch/mips/kernel/debug.S                 | 130 +++++++++++++++++
 arch/mips/kernel/early_printk.c          |  19 +++
 arch/mips/kernel/head.S                  |   4 +
 23 files changed, 756 insertions(+), 197 deletions(-)
---
base-commit: 084c8e315db34b59d38d06e684b1a0dd07d30287
change-id: 20240326-mips_debug_ll-ce72fee1b6a2

Best regards,

Comments

Jiaxun Yang May 15, 2024, 9:28 p.m. UTC | #1
在2024年5月2日五月 上午10:59,Jiaxun Yang写道:
> Hi all,
>
> This is a attempt to bring all low-level debugging print functions
> together and provide a arm-like low-level debugging interface and
> a further capability to debug early exceptions.
>
> This patch elimiate platform specific early_printk, zboot printing
> functions and cps-vec-ns16550 by newly introduced debug_ll.
>
> Hope you'll find them handy :-)
>
> Happy hacking!
>
> Thanks
>
> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>

A gentle ping.

Our reviewing capacity is quite low recently, hope everything is fine
with Thomas.

Thanks
[...]
- Jiaxun
Jiaxun Yang May 22, 2024, 7:15 a.m. UTC | #2
在2024年5月15日五月 下午10:28,Jiaxun Yang写道:
[...]
>
> A gentle ping.
>
> Our reviewing capacity is quite low recently, hope everything is fine
> with Thomas.

Another gentle-ish ping after 6.10 merge window.

This series has been floating here for so long, if I missed the merge
window, I think I deserve a notice.

Thanks
>
> Thanks
> [...]
> - Jiaxun
Thomas Bogendoerfer May 22, 2024, 8:03 a.m. UTC | #3
On Wed, May 22, 2024 at 08:15:22AM +0100, Jiaxun Yang wrote:
> 
> 
> 在2024年5月15日五月 下午10:28,Jiaxun Yang写道:
> [...]
> >
> > A gentle ping.
> >
> > Our reviewing capacity is quite low recently, hope everything is fine
> > with Thomas.
> 
> Another gentle-ish ping after 6.10 merge window.
> 
> This series has been floating here for so long, if I missed the merge
> window, I think I deserve a notice.

hmmm, I thought I was clear enough on version 1 of this series.

I don't want an additional printk like debug interface, There is
prom_putchar() and early printk console, which always got me past
any boot issue.

Thomas.
Jiaxun Yang May 22, 2024, 8:28 a.m. UTC | #4
在2024年5月22日五月 上午9:03,Thomas Bogendoerfer写道:
[...]
>
> hmmm, I thought I was clear enough on version 1 of this series.
>
> I don't want an additional printk like debug interface, There is
> prom_putchar() and early printk console, which always got me past
> any boot issue.

So it's not an additional printk like debug interface, it actually
merged 3 existing debug interfaces, the first being zboot's assembly
print routines, the second being CPS's assembly print routines, the
third being some platform specific early printk. I think they are
all essential for debugging early faults, for zboot that's the only
way to print something at decompressing stage, for CPS as other cores
are booting in non-coherent state we can't safely use any kernel
functions, for early_printk that can help us *reduce* the amount
of early printk code by just adding UART base to config.

The only thing being added is the ability to debug very early exception,
even that is partially ported from existing CPS assembly debugging routines.

Please let me know your thoughts.

Thanks
>
> Thomas.
>
> -- 
> Crap can work. Given enough thrust pigs will fly, but it's not necessarily a
> good idea.                                                [ RFC1925, 2.3 ]
Jiaxun Yang May 22, 2024, 3:07 p.m. UTC | #5
在2024年5月22日五月 上午9:28,Jiaxun Yang写道:
> 在2024年5月22日五月 上午9:03,Thomas Bogendoerfer写道:
> [...]
>>
>> hmmm, I thought I was clear enough on version 1 of this series.
>>
>> I don't want an additional printk like debug interface, There is
>> prom_putchar() and early printk console, which always got me past
>> any boot issue.
>
> So it's not an additional printk like debug interface, it actually
> merged 3 existing debug interfaces, the first being zboot's assembly
> print routines, the second being CPS's assembly print routines, the
> third being some platform specific early printk. I think they are
> all essential for debugging early faults, for zboot that's the only
> way to print something at decompressing stage, for CPS as other cores
> are booting in non-coherent state we can't safely use any kernel
> functions, for early_printk that can help us *reduce* the amount
> of early printk code by just adding UART base to config.
>
> The only thing being added is the ability to debug very early exception,
> even that is partially ported from existing CPS assembly debugging routines.
>
> Please let me know your thoughts.

That being said, have you noticed that prom_putchar and early_printk is
a non-extant on generic mach, ingenic, ralink etc? That's because we
really don't want to introduce any platform specific UART code for
early debugging on new platforms. With DEBUG_LL introduced by Arm it's
only a Kconfig option to do the trick.

I've got review tags in PATCH v2, that means not only me feeling that
this series is reasonable.

arm64 / riscv doesn't need that because they are well standardized
and it's almost guaranteed that kernel can boot into earlycon without
much drama. For MIPS that's not the case, there are too many things that
may go wrong, from zboot decompressor to cpu-probe and memblock. We really
lacks a way to debug things early, we need something that is available
at 1st instruction at kernel entry. Furthermore, many MIPS processors
don't come with JTAG or alike debugging support, that makes debugging
even harder, there is no way to debug an early exception if your firmware
doesn't handle it. That's all the motivation behind the series.

Besides, I think our communication needs to be improved. At PATCH v1
you made your point in reply, that's fair. So I also replied twice for
clarification. I heard nothing back, so I assume you want to see how
would it develop to address your concern. Then I posted PATCH v2 and
v3 to further improve the series, after that I pinged twice on PATCH v3.

That's in a 6-month timeframe with multiple transactions, you need to
inform us your intention, even if it's a NAK or you don't want to engage
on this topic further.

Quoting the maintainer handbook [1]: "If the review process or validation
for a particular change will take longer than the expected review timeline
for the subsystem, maintainer should reply to the submission indicating
that the work is being done, and when to expect full results." Radio silence
won't help anything, it's wasting time for both of us. Please, give a
shout if it's possible.

I can see some other series being slipped away this way, like I6500
multi-cluster patch, which is sent even earlier and respined many times
over. I can recall Mobileye series had a hard time on getting your
attention, luckily we went through it.

Quoting the maintainer handbook [1]: "Nonetheless when the work does
arrive (in form of patches which need review, user bug reports etc.)
it has to be acted upon promptly.". I understand Linux/MIPS is not
your day job, also you need to take breaks or go on holidays. Sometimes
you may burn out from your maintainer duties. That's fine, we are all
human beings. I'm not expecting a 1-week SLA or something, but 6 months
or longer to expect an action is appalling to me.

I'd strongly recommend you to look for a secondary maintainer, as mentioned
in maintainer handbook [1]: "Modern best practices dictate that there should
be at least two maintainers for any piece of code, no matter how trivial".
I understand you reject the idea once when Paul handed maintainership to
you, but there are clear evidences to show that something needs to be done.
You might need a hand on handling stuff promptly and understanding some
modern MIPS stuff.

I have many, many tiny improvements to MIPS kernel locally. Furthermore,
I do bring-up for both new and ancient MIPS systems. I never got a chance
to send them out because I want you prioritise on those fundamental series.

Apologise for potential aggressive tone in this email. I just can't clam
down when I think back about your reply, and I think we really need to
talk about it.

Thanks

[1]: https://docs.kernel.org/maintainer/feature-and-driver-maintainers.html
Maciej W. Rozycki May 26, 2024, 12:44 p.m. UTC | #6
On Wed, 22 May 2024, Jiaxun Yang wrote:

> That being said, have you noticed that prom_putchar and early_printk is
> a non-extant on generic mach, ingenic, ralink etc? That's because we
> really don't want to introduce any platform specific UART code for
> early debugging on new platforms. With DEBUG_LL introduced by Arm it's
> only a Kconfig option to do the trick.

 IMHO that is however the logical thing to do.  And then you need no magic 
options to fiddle with and say a distribution kernel will dump whatever it 
has to say if something wrong has happened early on.

 IOW just wire `prom_putchar' as required, using C code preferably.  NB 
YAMON does have a `print' entry point for console output, so for the Malta 
platform you can trivially use just that, no need for messy ad hoc 8250 
code.

 As to intercepting exceptions, it depends.  Again YAMON does handle that 
and dumps the register state, so with the Malta you get the information 
required.  For less capable ones it might make sense, but it ISTM like a 
candidate for an independent change, and then again I fail to see why the 
handler has to be written in the assembly language rather than C.

  Maciej
Jiaxun Yang May 26, 2024, 3:02 p.m. UTC | #7
在2024年5月26日五月 下午1:44,Maciej W. Rozycki写道:
> On Wed, 22 May 2024, Jiaxun Yang wrote:
>
>> That being said, have you noticed that prom_putchar and early_printk is
>> a non-extant on generic mach, ingenic, ralink etc? That's because we
>> really don't want to introduce any platform specific UART code for
>> early debugging on new platforms. With DEBUG_LL introduced by Arm it's
>> only a Kconfig option to do the trick.

Hi Maciej,

Thanks for your thoughts, my two cents below.
>
>  IMHO that is however the logical thing to do.  And then you need no magic 
> options to fiddle with and say a distribution kernel will dump whatever it 
> has to say if something wrong has happened early on.

This is a strict debug only options, we are not expecting any distribution
kernel to enable it. It has made itself explicit that no production device
should enable it.

Setting UART address for debug console by developers know what are they
doing is a proven approach among multiple places in multiple projects.

For kernel we have general earlycon cmdline option that would take MMIO
base address, Arm's DEBUG_LL had taken a similar approach and U-Boot have
CONFIG_DEBUG_UART_BASE, even our old zboot debug print code is taking such
approach.

It takes a balance between platform dependent code addition and bring-up
debugging capability. I fail to see why does it suddenly become an undesired
thing here.

>
>  IOW just wire `prom_putchar' as required, using C code preferably.  NB 
> YAMON does have a `print' entry point for console output, so for the Malta 
> platform you can trivially use just that, no need for messy ad hoc 8250 
> code.

Sadly for the majority of modern MIPS devices are dominated by U-Boot or
vendor's simple loader. In most cases, runtime APIs are not provided by
default. Even if it's enabled, there are still a couple of reasons preventing
it to be utilized properly. U-Boot relies on global pointer stored in K0 to
save global runtime data, kernel will clobber it very early and makes U-Boot
non-functional. On devices with limited memory, it's easy to get U-Boot memory
being clobbered by kernel and render U-Boot's runtime useless anyway.

That's why many new-ish platforms such as lantiq brings it's own UART
implementations for prom_putchar.

However, for generic platform implementing prom_putchar means we need to
introduce platform dependent code, which we had to pay all the price to
avoid. We have many in-tree and out-of tree generic platform users who
don't need to add any single line of code to bring up their platform,
thanks to DeviceTree, but they still need something to help with debugging
bring up process when devicetree went wrong or early panics.

>
>  As to intercepting exceptions, it depends.  Again YAMON does handle that 
> and dumps the register state, so with the Malta you get the information 
> required.  For less capable ones it might make sense, but it ISTM like a 
> candidate for an independent change, and then again I fail to see why the 
> handler has to be written in the assembly language rather than C.

Again for U-Boot debug exception dumping is optional and I know many devices
not shipping with that enabled. Even if it's enabled, it will stop to work
after U-Boot's memory/global pointer being clobbered or ebase being overridden
by kernel. Not to mention that Bootloader's exception dumping won't work
with CPS secondary cores.

Paul wrote cps-vec.S and cps-vec-ns16550.S in pure assembly for reasons.
Stack pointer is not initialized at second core & we really want to reduce
code footprint on secondary core to minimize side effects before coherence
is enabled.

When the infra is here, expand it to generic early exception is just tens
of lines. I fail to see the reason to bring in hundreds lines of C code for
the same functionality.

>
>   Maciej

Although it's a pure technical discussion, I still want to expand that while
I appreciate what you have done to the MIPS, sometimes I feel like we are not
on the same page because you guys are away from frontliner for so long and
missed many contexts. MIPS is still evolving, although I never appeared here
with my corp email, I'm one of those behind the scene. I draft new architecture
specs, write AVP and internal simulators, do RTL coding for future core products,
helping customers design SoC products, design software architecture and bringing
them up. I kept FOSS as my hobby and I tried my best to keep upstream in sync with
modern practices.

I love MIPS heritages, I own an SGI Indy and Algorithmics P-4032, I made some
fixes on MAME emulator for indy to keep kernel running on it. I'm still frequently
fascinated by those brilliant old designs. But I think we still need to make progress.
While maintaining compatibility with all those old things, we need to adopt common
practices that have been proven by other architectures and make our own innovations.
We need to make the development process agile, so no developer is turned away. We
need to adopt modern booting protocols like EFI and ensure generic kernel is really
generic and not being diverted because of different loading address....

I don't know if you would agree with my in both technical details and ideology,
but I think it's the time to make my intention clear.

Thanks