| Message ID | 68fe24ea-7795-24d8-211b-9d8a50affe9f@akamai.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
On 02/27/2017 02:50 PM, Jason Baron wrote: > > > On 02/27/2017 05:45 PM, David Daney wrote: >> On 02/27/2017 02:36 PM, Steven Rostedt wrote: >>> On Mon, 27 Feb 2017 14:21:21 -0800 >>> David Daney <ddaney@caviumnetworks.com> wrote: >>> >>>> See attached for mips. It seems to do the right thing. >>>> >>>> I leave it as an exercise to the reader to fix the other architectures. >>>> >>>> Consult your own binutils experts to verify that what I say is true. >>> >>> It may still just be safer to do the pointers instead. That way we >>> don't need to worry about some strange arch or off by one binutils >>> messing it up. >> >> Obviously it is your choice, but this is bog standard ELF linking. In >> theory even the arrays of power-of-2 sized objects should also supply an >> entity size. Think __ex_table and its ilk. >> >> >> The benefit of supplying an entsize is that you don't have to change the >> structure of the existing code and risk breaking something in the >> process. >> >> David Daney >> >> > > Thanks for the suggestion! I would like to see if this resolves the ppc > issue we had. I'm attaching a powerpc patch based on your suggestion. > Hopefully, Sachin can try it. > If there are problems, you could try something like: $ find . -name \*\.o | xargs mips64-octeon-linux-gnu-readelf -eW | grep 'File:\| __jump_table' File: ./drivers/firmware/built-in.o File: ./drivers/built-in.o [3249] __jump_table PROGBITS 0000000000000000 1838c8 0022c8 18 WAM 0 0 8 File: ./drivers/spi/built-in.o [82] __jump_table PROGBITS 0000000000000000 008cb0 000048 18 WAM 0 0 8 File: ./drivers/spi/spi-cavium-octeon.o File: ./drivers/spi/spi-cavium.o File: ./drivers/spi/spi.o . . . Look for files where the size of the __jump_table section is not a integer multiple of the entsize. > Thanks, > > -Jason
On Tue, 28 Feb 2017 10:25:46 +0530 Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: > File: ./net/ipv4/xfrm4_input.o > [12] __jump_table PROGBITS 0000000000000000 000639 000018 18 WAM 0 0 1 > File: ./net/ipv4/udplite.o > File: ./net/ipv4/xfrm4_output.o > [ 9] __jump_table PROGBITS 0000000000000000 000481 000018 18 WAM 0 0 1 Looks like there's some issues right there. -- Steve
On 02/28/2017 08:21 AM, Steven Rostedt wrote: > On Tue, 28 Feb 2017 10:25:46 +0530 > Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: > >> File: ./net/ipv4/xfrm4_input.o >> [12] __jump_table PROGBITS 0000000000000000 000639 000018 18 WAM 0 0 1 >> File: ./net/ipv4/udplite.o >> File: ./net/ipv4/xfrm4_output.o >> [ 9] __jump_table PROGBITS 0000000000000000 000481 000018 18 WAM 0 0 1 > > Looks like there's some issues right there. Those look good to me 18/18 = 1 with no remainder. The odd numbers are the offset of the section in the ELF file. If you look at the stack trace, it seems that it is during module loading. Are the primitives for generating the tables doing something different for the module case? I am not familiar enough with the powerpc ABIs to know. Try this: $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if $#f > 5; print $_' <~/jump_table.log There are no entries with size that is not a multiple of 0x18. I think my patch to add the ENTSIZE is not doing anything here. I suspect that the alignment of the __jump_table section in the .ko files is not correct, and you are seeing some sort of problem due to that. > > -- Steve >
On 02/28/2017 01:16 PM, David Daney wrote: > On 02/28/2017 08:21 AM, Steven Rostedt wrote: >> On Tue, 28 Feb 2017 10:25:46 +0530 >> Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: >> >>> File: ./net/ipv4/xfrm4_input.o >>> [12] __jump_table PROGBITS 0000000000000000 000639 >>> 000018 18 WAM 0 0 1 >>> File: ./net/ipv4/udplite.o >>> File: ./net/ipv4/xfrm4_output.o >>> [ 9] __jump_table PROGBITS 0000000000000000 000481 >>> 000018 18 WAM 0 0 1 >> >> Looks like there's some issues right there. > > Those look good to me 18/18 = 1 with no remainder. The odd numbers are > the offset of the section in the ELF file. > > If you look at the stack trace, it seems that it is during module loading. > > Are the primitives for generating the tables doing something different > for the module case? I am not familiar enough with the powerpc ABIs to > know. > > Try this: > > $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if > $#f > 5; print $_' <~/jump_table.log > > > There are no entries with size that is not a multiple of 0x18. > > I think my patch to add the ENTSIZE is not doing anything here. > > I suspect that the alignment of the __jump_table section in the .ko > files is not correct, and you are seeing some sort of problem due to that. > > Hi, Yes, if you look at the trace that Sachin sent the module being loaded that does the WARN_ON() is nfsd.ko. That module from Sachin's trace has: [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 18 WAM 0 0 1 So its not the size but rather the start offset '03fd77', that is the problem here. That is what the WARN_ON triggers on, that the start of the table is not 4-byte aligned. Using a ppc cross-compiler and the ENTSIZE patch that line does not change, however if I use the initial patch posted in this thread, the start does align to 4-bytes and thus the warning goes away, as Sachin verified. In fact, without the patch I found several modules that don't start at the proper alignment, however with the patch that started this thread they were all properly aligned. In terms of the '.balign' causing holes, we originally added the '_ASM_ALIGN' to x86 for precisely this reason. See commit: ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. In addition, we have a lot of runtime with the .balign in the tree and I'm not aware of any holes in the table. I think the code would blow up pretty badly if there were. A number of arches were already using the '.balign', and the patch I proposed simply added it to remaining ones, now that we added a WARN_ON() to catch this condition. Thanks, -Jason
On 02/28/2017 10:39 AM, Jason Baron wrote: > > > On 02/28/2017 01:16 PM, David Daney wrote: >> On 02/28/2017 08:21 AM, Steven Rostedt wrote: >>> On Tue, 28 Feb 2017 10:25:46 +0530 >>> Sachin Sant <sachinp@linux.vnet.ibm.com> wrote: >>> >>>> File: ./net/ipv4/xfrm4_input.o >>>> [12] __jump_table PROGBITS 0000000000000000 000639 >>>> 000018 18 WAM 0 0 1 >>>> File: ./net/ipv4/udplite.o >>>> File: ./net/ipv4/xfrm4_output.o >>>> [ 9] __jump_table PROGBITS 0000000000000000 000481 >>>> 000018 18 WAM 0 0 1 >>> >>> Looks like there's some issues right there. >> >> Those look good to me 18/18 = 1 with no remainder. The odd numbers are >> the offset of the section in the ELF file. >> >> If you look at the stack trace, it seems that it is during module >> loading. >> >> Are the primitives for generating the tables doing something different >> for the module case? I am not familiar enough with the powerpc ABIs to >> know. >> >> Try this: >> >> $ perl -n -e 's/\[ /\[/; my @f = split " "; print hex($f[5]) % 0x18 if >> $#f > 5; print $_' <~/jump_table.log >> >> >> There are no entries with size that is not a multiple of 0x18. >> >> I think my patch to add the ENTSIZE is not doing anything here. >> >> I suspect that the alignment of the __jump_table section in the .ko >> files is not correct, and you are seeing some sort of problem due to >> that. >> >> > > Hi, > > Yes, if you look at the trace that Sachin sent the module being loaded > that does the WARN_ON() is nfsd.ko. > > That module from Sachin's trace has: > > [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 > 18 WAM 0 0 1 The problem is then the section alignment (last column) for power. On mips with no patches applied, we get: [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 00 WA 0 0 8 Look, proper alignment! The question I have is why do the power ".llong" and ".long" assembler directives not force section alignment? Is there an alternative that could be used that would result in the proper alignment? Would ".word" work? If not, then I would say patch only power with your balign thing. 8-byte alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel > > So its not the size but rather the start offset '03fd77', that is the > problem here. That is what the WARN_ON triggers on, that the start of > the table is not 4-byte aligned. > > Using a ppc cross-compiler and the ENTSIZE patch that line does not > change, however if I use the initial patch posted in this thread, the > start does align to 4-bytes and thus the warning goes away, as Sachin > verified. In fact, without the patch I found several modules that don't > start at the proper alignment, however with the patch that started this > thread they were all properly aligned. > > In terms of the '.balign' causing holes, we originally added the > '_ASM_ALIGN' to x86 for precisely this reason. See commit: > ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. > > In addition, we have a lot of runtime with the .balign in the tree and > I'm not aware of any holes in the table. I think the code would blow up > pretty badly if there were. > > A number of arches were already using the '.balign', and the patch I > proposed simply added it to remaining ones, now that we added a > WARN_ON() to catch this condition. > > Thanks, > > -Jason > > > >
On 02/28/2017 11:05 AM, David Daney wrote: > On 02/28/2017 10:39 AM, Jason Baron wrote: >> [...] >>> I suspect that the alignment of the __jump_table section in the .ko >>> files is not correct, and you are seeing some sort of problem due to >>> that. >>> >>> >> >> Hi, >> >> Yes, if you look at the trace that Sachin sent the module being loaded >> that does the WARN_ON() is nfsd.ko. >> >> That module from Sachin's trace has: >> >> [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 >> 18 WAM 0 0 1 > > The problem is then the section alignment (last column) for power. > > On mips with no patches applied, we get: > > [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 > 00 WA 0 0 8 > > Look, proper alignment! > > The question I have is why do the power ".llong" and ".long" assembler > directives not force section alignment? Is there an alternative that > could be used that would result in the proper alignment? Would ".word" > work? > > If not, then I would say patch only power with your balign thing. 8-byte > alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel > I think the proper fix is either: A) Modify scripts/module-common.lds to force __jump_table alignment for all architectures. B) Add arch/powerpc/kernel/module.lds to force __jump_table alignment for powerpc only. David. > >> >> So its not the size but rather the start offset '03fd77', that is the >> problem here. That is what the WARN_ON triggers on, that the start of >> the table is not 4-byte aligned. >> >> Using a ppc cross-compiler and the ENTSIZE patch that line does not >> change, however if I use the initial patch posted in this thread, the >> start does align to 4-bytes and thus the warning goes away, as Sachin >> verified. In fact, without the patch I found several modules that don't >> start at the proper alignment, however with the patch that started this >> thread they were all properly aligned. >> >> In terms of the '.balign' causing holes, we originally added the >> '_ASM_ALIGN' to x86 for precisely this reason. See commit: >> ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. >> >> In addition, we have a lot of runtime with the .balign in the tree and >> I'm not aware of any holes in the table. I think the code would blow up >> pretty badly if there were. >> >> A number of arches were already using the '.balign', and the patch I >> proposed simply added it to remaining ones, now that we added a >> WARN_ON() to catch this condition. >> >> Thanks, >> >> -Jason >> >> >> >> > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On 02/28/2017 02:22 PM, David Daney wrote: > On 02/28/2017 11:05 AM, David Daney wrote: >> On 02/28/2017 10:39 AM, Jason Baron wrote: >>> > [...] >>>> I suspect that the alignment of the __jump_table section in the .ko >>>> files is not correct, and you are seeing some sort of problem due to >>>> that. >>>> >>>> >>> >>> Hi, >>> >>> Yes, if you look at the trace that Sachin sent the module being loaded >>> that does the WARN_ON() is nfsd.ko. >>> >>> That module from Sachin's trace has: >>> >>> [31] __jump_table PROGBITS 0000000000000000 03fd77 0000c0 >>> 18 WAM 0 0 1 >> >> The problem is then the section alignment (last column) for power. >> >> On mips with no patches applied, we get: >> >> [17] __jump_table PROGBITS 0000000000000000 00d2c0 000048 >> 00 WA 0 0 8 >> >> Look, proper alignment! >> >> The question I have is why do the power ".llong" and ".long" assembler >> directives not force section alignment? Is there an alternative that >> could be used that would result in the proper alignment? Would ".word" >> work? >> >> If not, then I would say patch only power with your balign thing. 8-byte >> alignment for 64-bit kernel, 4-byte alignment for 32-bit kernel >> > > I think the proper fix is either: > > A) Modify scripts/module-common.lds to force __jump_table alignment for > all architectures. > > B) Add arch/powerpc/kernel/module.lds to force __jump_table alignment > for powerpc only. > > David. > > Ok, I can try adding it to the linger script. FWIW, here is my before and after with the .balign thing for the nfsd.ko module on powperc (using a cross-compiler): before: [31] __jump_table PROGBITS 0000000000000000 03ee3e 0000f0 00 WA 0 0 1 after: [31] __jump_table PROGBITS 0000000000000000 03ee40 0000f0 00 WA 0 0 4 Thanks, -Jason > >> >>> >>> So its not the size but rather the start offset '03fd77', that is the >>> problem here. That is what the WARN_ON triggers on, that the start of >>> the table is not 4-byte aligned. >>> >>> Using a ppc cross-compiler and the ENTSIZE patch that line does not >>> change, however if I use the initial patch posted in this thread, the >>> start does align to 4-bytes and thus the warning goes away, as Sachin >>> verified. In fact, without the patch I found several modules that don't >>> start at the proper alignment, however with the patch that started this >>> thread they were all properly aligned. >>> >>> In terms of the '.balign' causing holes, we originally added the >>> '_ASM_ALIGN' to x86 for precisely this reason. See commit: >>> ef64789 jump label: Add _ASM_ALIGN for x86 and x86_64 and discussion. >>> >>> In addition, we have a lot of runtime with the .balign in the tree and >>> I'm not aware of any holes in the table. I think the code would blow up >>> pretty badly if there were. >>> >>> A number of arches were already using the '.balign', and the patch I >>> proposed simply added it to remaining ones, now that we added a >>> WARN_ON() to catch this condition. >>> >>> Thanks, >>> >>> -Jason >>> >>> >>> >>> >> >> >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel@lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel >
diff --git a/arch/powerpc/include/asm/jump_label.h b/arch/powerpc/include/asm/jump_label.h index 9a287e0ac8b1..3c5660e50f9a 100644 --- a/arch/powerpc/include/asm/jump_label.h +++ b/arch/powerpc/include/asm/jump_label.h @@ -19,14 +19,26 @@ #define JUMP_ENTRY_TYPE stringify_in_c(FTR_ENTRY_LONG) #define JUMP_LABEL_NOP_SIZE 4 +#ifdef CONFIG_PPC64 +typedef u64 jump_label_t; +#else +typedef u32 jump_label_t; +#endif + +struct jump_entry { + jump_label_t code; + jump_label_t target; + jump_label_t key; +}; + static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:\n\t" "nop # arch_static_branch\n\t" - ".pushsection __jump_table, \"aw\"\n\t" + ".pushsection __jump_table, \"awM\",@progbits, %1\n\t" JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" ".popsection \n\t" - : : "i" (&((char *)key)[branch]) : : l_yes); + : : "i" (&((char *)key)[branch]), "i" (sizeof(struct jump_entry)) : : l_yes); return false; l_yes: @@ -37,32 +49,24 @@ static __always_inline bool arch_static_branch_jump(struct static_key *key, bool { asm_volatile_goto("1:\n\t" "b %l[l_yes] # arch_static_branch_jump\n\t" - ".pushsection __jump_table, \"aw\"\n\t" + ".pushsection __jump_table, \"awM\",@progbits, %1\n\t" JUMP_ENTRY_TYPE "1b, %l[l_yes], %c0\n\t" ".popsection \n\t" - : : "i" (&((char *)key)[branch]) : : l_yes); + : : "i" (&((char *)key)[branch]), "i" (sizeof(struct jump_entry)) : : l_yes); return false; l_yes: return true; } -#ifdef CONFIG_PPC64 -typedef u64 jump_label_t; + #else -typedef u32 jump_label_t; -#endif -struct jump_entry { - jump_label_t code; - jump_label_t target; - jump_label_t key; -}; +#define ENTRY_SIZE (ULONG_SIZE * 3) -#else #define ARCH_STATIC_BRANCH(LABEL, KEY) \ 1098: nop; \ - .pushsection __jump_table, "aw"; \ + .pushsection __jump_table, "awM",@progbits,ENTRY_SIZE; \ FTR_ENTRY_LONG 1098b, LABEL, KEY; \ .popsection #endif