| Message ID | 20221022124525.2080-2-thunder.leizhen@huawei.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | rcu: Add RCU stall diagnosis information | expand |
> Similar to kstat_cpu_irqs_sum(), it counts the sum of all software > interrupts on a specified CPU. > > diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h > @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) > return kstat_cpu(cpu).softirqs[irq]; > } > > +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) > +{ > + int i; > + unsigned int sum = 0; > + > + for (i = 0; i < NR_SOFTIRQS; i++) > + sum += kstat_softirqs_cpu(i, cpu); > + > + return sum; > +} In the function upon which this is based: irqs_sumstruct kernel_stat { unsigned long irqs_sum; unsigned int softirqs[NR_SOFTIRQS]; }; static inline unsigned int kstat_cpu_irqs_sum(unsigned int cpu) { return kstat_cpu(cpu).irqs_sum; } kstat_cpu_irqs_sum returns an unsigned long as an unsigned int, which could cause large values to be truncated. Should that return unsigned long? The only existing caller is fs/proc/stat.c which puts it into a u64: u64 sum = 0; ... sum += kstat_cpu_irqs_sum(i); The softirqs field is an unsigned int, so the new function doesn't have this inconsistency.
On 2022/10/28 3:04, Elliott, Robert (Servers) wrote: > >> Similar to kstat_cpu_irqs_sum(), it counts the sum of all software >> interrupts on a specified CPU. >> >> diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h >> @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) >> return kstat_cpu(cpu).softirqs[irq]; >> } >> >> +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) >> +{ >> + int i; >> + unsigned int sum = 0; >> + >> + for (i = 0; i < NR_SOFTIRQS; i++) >> + sum += kstat_softirqs_cpu(i, cpu); >> + >> + return sum; >> +} > > In the function upon which this is based: > > struct kernel_stat { > unsigned long irqs_sum; > unsigned int softirqs[NR_SOFTIRQS]; > }; > > static inline unsigned int kstat_cpu_irqs_sum(unsigned int cpu) > { > return kstat_cpu(cpu).irqs_sum; > } > > kstat_cpu_irqs_sum returns an unsigned long as an unsigned int, which > could cause large values to be truncated. Should that return > unsigned long? The only existing caller is fs/proc/stat.c which This should be a mistake on: commit f2c66cd8eeddedb4 ("/proc/stat: scalability of irq num per cpu") I'll correct it to "unsigned long" in the next version. Thanks. > puts it into a u64: > u64 sum = 0; > ... > sum += kstat_cpu_irqs_sum(i); > > The softirqs field is an unsigned int, so the new function doesn't have > this inconsistency. OK. To be honest, I did the math. CONFIG_HZ=250 2^32 / 250 / 3600 / 24 / 365 = 0.545 < 1 year So, in theory, for those 32-bit processors, we should use "unsigned long long". Of course, from a programming point of view, 64-bit consists of two 32-bits, and there is an atomicity problem. I think that's probably why members of struct kernel_stat don't use u64. However, it seems that the type of member softirqs can currently be changed to unsigned long. So, at least on a 64-bit processor, it won't have a count overflow problem. > > . >
On Thu, Oct 27, 2022 at 07:04:53PM +0000, Elliott, Robert (Servers) wrote: > > > Similar to kstat_cpu_irqs_sum(), it counts the sum of all software > > interrupts on a specified CPU. > > > > diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h > > @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) > > return kstat_cpu(cpu).softirqs[irq]; > > } > > > > +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) > > +{ > > + int i; > > + unsigned int sum = 0; > > + > > + for (i = 0; i < NR_SOFTIRQS; i++) > > + sum += kstat_softirqs_cpu(i, cpu); > > + > > + return sum; > > +} > > In the function upon which this is based: > > irqs_sumstruct kernel_stat { > unsigned long irqs_sum; > unsigned int softirqs[NR_SOFTIRQS]; > }; > > static inline unsigned int kstat_cpu_irqs_sum(unsigned int cpu) > { > return kstat_cpu(cpu).irqs_sum; > } > > kstat_cpu_irqs_sum returns an unsigned long as an unsigned int, which > could cause large values to be truncated. Should that return > unsigned long? The only existing caller is fs/proc/stat.c which > puts it into a u64: > u64 sum = 0; > ... > sum += kstat_cpu_irqs_sum(i); > > The softirqs field is an unsigned int, so the new function doesn't have > this inconsistency. Good point! Zhen Lei, thoughts? Thanx, Paul
On Fri, Oct 28, 2022 at 10:38:15AM +0800, Leizhen (ThunderTown) wrote: > > > On 2022/10/28 3:04, Elliott, Robert (Servers) wrote: > > > >> Similar to kstat_cpu_irqs_sum(), it counts the sum of all software > >> interrupts on a specified CPU. > >> > >> diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h > >> @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) > >> return kstat_cpu(cpu).softirqs[irq]; > >> } > >> > >> +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) > >> +{ > >> + int i; > >> + unsigned int sum = 0; > >> + > >> + for (i = 0; i < NR_SOFTIRQS; i++) > >> + sum += kstat_softirqs_cpu(i, cpu); > >> + > >> + return sum; > >> +} > > > > In the function upon which this is based: > > > > struct kernel_stat { > > unsigned long irqs_sum; > > unsigned int softirqs[NR_SOFTIRQS]; > > }; > > > > static inline unsigned int kstat_cpu_irqs_sum(unsigned int cpu) > > { > > return kstat_cpu(cpu).irqs_sum; > > } > > > > kstat_cpu_irqs_sum returns an unsigned long as an unsigned int, which > > could cause large values to be truncated. Should that return > > unsigned long? The only existing caller is fs/proc/stat.c which > > This should be a mistake on: > commit f2c66cd8eeddedb4 ("/proc/stat: scalability of irq num per cpu") > > I'll correct it to "unsigned long" in the next version. Thanks. > > > puts it into a u64: > > u64 sum = 0; > > ... > > sum += kstat_cpu_irqs_sum(i); > > > > The softirqs field is an unsigned int, so the new function doesn't have > > this inconsistency. > > OK. > > To be honest, I did the math. CONFIG_HZ=250 > 2^32 / 250 / 3600 / 24 / 365 = 0.545 < 1 year For this to be a problem, our RCU CPU stall warning would have to be for a months-long grace period, even on systems with HZ=1000. In almost all cases, the system would have OOMed long before then. > So, in theory, for those 32-bit processors, we should use "unsigned long long". > Of course, from a programming point of view, 64-bit consists of two 32-bits, > and there is an atomicity problem. I think that's probably why members of > struct kernel_stat don't use u64. > > However, it seems that the type of member softirqs can currently be changed to > unsigned long. So, at least on a 64-bit processor, it won't have a count > overflow problem. An unsigned long should suffice. ;-) Thanx, Paul
On 2022/10/29 6:35, Paul E. McKenney wrote: > On Fri, Oct 28, 2022 at 10:38:15AM +0800, Leizhen (ThunderTown) wrote: >> >> >> On 2022/10/28 3:04, Elliott, Robert (Servers) wrote: >>> >>>> Similar to kstat_cpu_irqs_sum(), it counts the sum of all software >>>> interrupts on a specified CPU. >>>> >>>> diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h >>>> @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) >>>> return kstat_cpu(cpu).softirqs[irq]; >>>> } >>>> >>>> +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) >>>> +{ >>>> + int i; >>>> + unsigned int sum = 0; >>>> + >>>> + for (i = 0; i < NR_SOFTIRQS; i++) >>>> + sum += kstat_softirqs_cpu(i, cpu); >>>> + >>>> + return sum; >>>> +} >>> >>> In the function upon which this is based: >>> >>> struct kernel_stat { >>> unsigned long irqs_sum; >>> unsigned int softirqs[NR_SOFTIRQS]; >>> }; >>> >>> static inline unsigned int kstat_cpu_irqs_sum(unsigned int cpu) >>> { >>> return kstat_cpu(cpu).irqs_sum; >>> } >>> >>> kstat_cpu_irqs_sum returns an unsigned long as an unsigned int, which >>> could cause large values to be truncated. Should that return >>> unsigned long? The only existing caller is fs/proc/stat.c which >> >> This should be a mistake on: >> commit f2c66cd8eeddedb4 ("/proc/stat: scalability of irq num per cpu") >> >> I'll correct it to "unsigned long" in the next version. Thanks. >> >>> puts it into a u64: >>> u64 sum = 0; >>> ... >>> sum += kstat_cpu_irqs_sum(i); >>> >>> The softirqs field is an unsigned int, so the new function doesn't have >>> this inconsistency. >> >> OK. >> >> To be honest, I did the math. CONFIG_HZ=250 >> 2^32 / 250 / 3600 / 24 / 365 = 0.545 < 1 year > > For this to be a problem, our RCU CPU stall warning would have to be > for a months-long grace period, even on systems with HZ=1000. In almost > all cases, the system would have OOMed long before then. Yes. > >> So, in theory, for those 32-bit processors, we should use "unsigned long long". >> Of course, from a programming point of view, 64-bit consists of two 32-bits, >> and there is an atomicity problem. I think that's probably why members of >> struct kernel_stat don't use u64. >> >> However, it seems that the type of member softirqs can currently be changed to >> unsigned long. So, at least on a 64-bit processor, it won't have a count >> overflow problem. > > An unsigned long should suffice. ;-) include/linux/irqdesc.h:58: unsigned int __percpu *kstat_irqs; I found another place where the hard interrupt count was stored with type "unsigned int", it's used by "/proc/interrupts". Maybe the user-mode program gets it periodically and accumulates it to a 64-bit value. Of course, maybe half a year later, no one cares about the specific interrupts count anymore. So, apart from what Elliott mentioned, I won't change the rest. > > Thanx, Paul > . >
diff --git a/include/linux/kernel_stat.h b/include/linux/kernel_stat.h index ddb5a358fd829f4..61d427c1962bf1c 100644 --- a/include/linux/kernel_stat.h +++ b/include/linux/kernel_stat.h @@ -67,6 +67,17 @@ static inline unsigned int kstat_softirqs_cpu(unsigned int irq, int cpu) return kstat_cpu(cpu).softirqs[irq]; } +static inline unsigned int kstat_cpu_softirqs_sum(int cpu) +{ + int i; + unsigned int sum = 0; + + for (i = 0; i < NR_SOFTIRQS; i++) + sum += kstat_softirqs_cpu(i, cpu); + + return sum; +} + /* * Number of interrupts per specific IRQ source, since bootup */
Similar to kstat_cpu_irqs_sum(), it counts the sum of all software interrupts on a specified CPU. Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com> --- include/linux/kernel_stat.h | 11 +++++++++++ 1 file changed, 11 insertions(+)