mbox series

[GIT,PULL] sysctl changes for v6.6-rc1

Message ID ZO5Yx5JFogGi/cBo@bombadil.infradead.org (mailing list archive)
State Not Applicable
Headers show
Series [GIT,PULL] sysctl changes for v6.6-rc1 | expand

Pull-request

git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1

Message

Luis Chamberlain Aug. 29, 2023, 8:44 p.m. UTC
The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:

  Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)

are available in the Git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1

for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:

  sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)

----------------------------------------------------------------
sysctl-6.6-rc1

Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
placings sysctls to their own sybsystem or file to help avoid merge conflicts.
Matthew Wilcox pointed out though that if we're going to do that we might as
well also *save* space while at it and try to remove the extra last sysctl
entry added at the end of each array, a sentintel, instead of bloating the
kernel by adding a new sentinel with each array moved.

Doing that was not so trivial, and has required slowing down the moves of
kernel/sysctl.c arrays and measuring the impact on size by each new move.

The complex part of the effort to help reduce the size of each sysctl is being
done by the patient work of el señor Don Joel Granados. A lot of this is truly
painful code refactoring and testing and then trying to measure the savings of
each move and removing the sentinels. Although Joel already has code which does
most of this work, experience with sysctl moves in the past shows is we need to
be careful due to the slew of odd build failures that are possible due to the
amount of random Kconfig options sysctls use.

To that end Joel's work is split by first addressing the major housekeeping
needed to remove the sentinels, which is part of this merge request. The rest
of the work to actually remove the sentinels will be done later in future
kernel releases.

At first I was only going to send his first 7 patches of his patch series,
posted 1 month ago, but in retrospect due to the testing the changes have
received in linux-next and the minor changes they make this goes with the
entire set of patches Joel had planned: just sysctl house keeping. There are
networking changes but these are part of the house keeping too.

The preliminary math is showing this will all help reduce the overall build
time size of the kernel and run time memory consumed by the kernel by about
~64 bytes per array where we are able to remove each sentinel in the future.
That also means there is no more bloating the kernel with the extra ~64 bytes
per array moved as no new sentinels are created.

Most of this has been in linux-next for about a month, the last 7 patches took
a minor refresh 2 week ago based on feedback.

----------------------------------------------------------------
Joel Granados (14):
      sysctl: Prefer ctl_table_header in proc_sysctl
      sysctl: Use ctl_table_header in list_for_each_table_entry
      sysctl: Add ctl_table_size to ctl_table_header
      sysctl: Add size argument to init_header
      sysctl: Add a size arg to __register_sysctl_table
      sysctl: Add size to register_sysctl
      sysctl: Add size arg to __register_sysctl_init
      sysctl: Add size to register_net_sysctl function
      ax.25: Update to register_net_sysctl_sz
      netfilter: Update to register_net_sysctl_sz
      networking: Update to register_net_sysctl_sz
      vrf: Update to register_net_sysctl_sz
      sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
      sysctl: Use ctl_table_size as stopping criteria for list macro

 arch/arm64/kernel/armv8_deprecated.c    |  2 +-
 arch/s390/appldata/appldata_base.c      |  2 +-
 drivers/net/vrf.c                       |  3 +-
 fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
 include/linux/sysctl.h                  | 31 +++++++++---
 include/net/ipv6.h                      |  2 +
 include/net/net_namespace.h             | 10 ++--
 ipc/ipc_sysctl.c                        |  4 +-
 ipc/mq_sysctl.c                         |  4 +-
 kernel/ucount.c                         |  5 +-
 net/ax25/sysctl_net_ax25.c              |  3 +-
 net/bridge/br_netfilter_hooks.c         |  3 +-
 net/core/neighbour.c                    |  8 ++-
 net/core/sysctl_net_core.c              |  3 +-
 net/ieee802154/6lowpan/reassembly.c     |  8 ++-
 net/ipv4/devinet.c                      |  3 +-
 net/ipv4/ip_fragment.c                  |  3 +-
 net/ipv4/route.c                        |  8 ++-
 net/ipv4/sysctl_net_ipv4.c              |  3 +-
 net/ipv4/xfrm4_policy.c                 |  3 +-
 net/ipv6/addrconf.c                     |  3 +-
 net/ipv6/icmp.c                         |  5 ++
 net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
 net/ipv6/reassembly.c                   |  3 +-
 net/ipv6/route.c                        |  9 ++++
 net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
 net/ipv6/xfrm6_policy.c                 |  3 +-
 net/mpls/af_mpls.c                      |  6 ++-
 net/mptcp/ctrl.c                        |  3 +-
 net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
 net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
 net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
 net/netfilter/nf_conntrack_standalone.c |  4 +-
 net/netfilter/nf_log.c                  |  7 +--
 net/rds/tcp.c                           |  3 +-
 net/sctp/sysctl.c                       |  4 +-
 net/smc/smc_sysctl.c                    |  3 +-
 net/sysctl_net.c                        | 26 +++++++---
 net/unix/sysctl_net_unix.c              |  3 +-
 net/xfrm/xfrm_sysctl.c                  |  8 ++-
 40 files changed, 222 insertions(+), 113 deletions(-)

Comments

Alexey Gladkov Sept. 6, 2023, 11:58 a.m. UTC | #1
On Tue, Aug 29, 2023 at 01:44:55PM -0700, Luis Chamberlain wrote:
> The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> 
>   Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> 
> are available in the Git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1
> 
> for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:
> 
>   sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)
> 
> ----------------------------------------------------------------
> sysctl-6.6-rc1
> 
> Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
> placings sysctls to their own sybsystem or file to help avoid merge conflicts.
> Matthew Wilcox pointed out though that if we're going to do that we might as
> well also *save* space while at it and try to remove the extra last sysctl
> entry added at the end of each array, a sentintel, instead of bloating the
> kernel by adding a new sentinel with each array moved.
> 
> Doing that was not so trivial, and has required slowing down the moves of
> kernel/sysctl.c arrays and measuring the impact on size by each new move.
> 
> The complex part of the effort to help reduce the size of each sysctl is being
> done by the patient work of el señor Don Joel Granados. A lot of this is truly
> painful code refactoring and testing and then trying to measure the savings of
> each move and removing the sentinels. Although Joel already has code which does
> most of this work, experience with sysctl moves in the past shows is we need to
> be careful due to the slew of odd build failures that are possible due to the
> amount of random Kconfig options sysctls use.
> 
> To that end Joel's work is split by first addressing the major housekeeping
> needed to remove the sentinels, which is part of this merge request. The rest
> of the work to actually remove the sentinels will be done later in future
> kernel releases.

This is very interesting for me. I'm also refactoring sysctl based on
discussion with Linus a while ago.

Could you please add me to the Cc in the next patches?

> At first I was only going to send his first 7 patches of his patch series,
> posted 1 month ago, but in retrospect due to the testing the changes have
> received in linux-next and the minor changes they make this goes with the
> entire set of patches Joel had planned: just sysctl house keeping. There are
> networking changes but these are part of the house keeping too.
> 
> The preliminary math is showing this will all help reduce the overall build
> time size of the kernel and run time memory consumed by the kernel by about
> ~64 bytes per array where we are able to remove each sentinel in the future.
> That also means there is no more bloating the kernel with the extra ~64 bytes
> per array moved as no new sentinels are created.
> 
> Most of this has been in linux-next for about a month, the last 7 patches took
> a minor refresh 2 week ago based on feedback.
> 
> ----------------------------------------------------------------
> Joel Granados (14):
>       sysctl: Prefer ctl_table_header in proc_sysctl
>       sysctl: Use ctl_table_header in list_for_each_table_entry
>       sysctl: Add ctl_table_size to ctl_table_header
>       sysctl: Add size argument to init_header
>       sysctl: Add a size arg to __register_sysctl_table
>       sysctl: Add size to register_sysctl
>       sysctl: Add size arg to __register_sysctl_init
>       sysctl: Add size to register_net_sysctl function
>       ax.25: Update to register_net_sysctl_sz
>       netfilter: Update to register_net_sysctl_sz
>       networking: Update to register_net_sysctl_sz
>       vrf: Update to register_net_sysctl_sz
>       sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
>       sysctl: Use ctl_table_size as stopping criteria for list macro
> 
>  arch/arm64/kernel/armv8_deprecated.c    |  2 +-
>  arch/s390/appldata/appldata_base.c      |  2 +-
>  drivers/net/vrf.c                       |  3 +-
>  fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
>  include/linux/sysctl.h                  | 31 +++++++++---
>  include/net/ipv6.h                      |  2 +
>  include/net/net_namespace.h             | 10 ++--
>  ipc/ipc_sysctl.c                        |  4 +-
>  ipc/mq_sysctl.c                         |  4 +-
>  kernel/ucount.c                         |  5 +-
>  net/ax25/sysctl_net_ax25.c              |  3 +-
>  net/bridge/br_netfilter_hooks.c         |  3 +-
>  net/core/neighbour.c                    |  8 ++-
>  net/core/sysctl_net_core.c              |  3 +-
>  net/ieee802154/6lowpan/reassembly.c     |  8 ++-
>  net/ipv4/devinet.c                      |  3 +-
>  net/ipv4/ip_fragment.c                  |  3 +-
>  net/ipv4/route.c                        |  8 ++-
>  net/ipv4/sysctl_net_ipv4.c              |  3 +-
>  net/ipv4/xfrm4_policy.c                 |  3 +-
>  net/ipv6/addrconf.c                     |  3 +-
>  net/ipv6/icmp.c                         |  5 ++
>  net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
>  net/ipv6/reassembly.c                   |  3 +-
>  net/ipv6/route.c                        |  9 ++++
>  net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
>  net/ipv6/xfrm6_policy.c                 |  3 +-
>  net/mpls/af_mpls.c                      |  6 ++-
>  net/mptcp/ctrl.c                        |  3 +-
>  net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
>  net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
>  net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
>  net/netfilter/nf_conntrack_standalone.c |  4 +-
>  net/netfilter/nf_log.c                  |  7 +--
>  net/rds/tcp.c                           |  3 +-
>  net/sctp/sysctl.c                       |  4 +-
>  net/smc/smc_sysctl.c                    |  3 +-
>  net/sysctl_net.c                        | 26 +++++++---
>  net/unix/sysctl_net_unix.c              |  3 +-
>  net/xfrm/xfrm_sysctl.c                  |  8 ++-
>  40 files changed, 222 insertions(+), 113 deletions(-)
Joel Granados Sept. 6, 2023, 12:35 p.m. UTC | #2
On Wed, Sep 06, 2023 at 01:58:49PM +0200, Alexey Gladkov wrote:
> On Tue, Aug 29, 2023 at 01:44:55PM -0700, Luis Chamberlain wrote:
> > The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
> > 
> >   Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
> > 
> > are available in the Git repository at:
> > 
> >   git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/sysctl-6.6-rc1
> > 
> > for you to fetch changes up to 53f3811dfd5e39507ee3aaea1be09aabce8f9c98:
> > 
> >   sysctl: Use ctl_table_size as stopping criteria for list macro (2023-08-15 15:26:18 -0700)
> > 
> > ----------------------------------------------------------------
> > sysctl-6.6-rc1
> > 
> > Long ago we set out to remove the kitchen sink on kernel/sysctl.c arrays and
> > placings sysctls to their own sybsystem or file to help avoid merge conflicts.
> > Matthew Wilcox pointed out though that if we're going to do that we might as
> > well also *save* space while at it and try to remove the extra last sysctl
> > entry added at the end of each array, a sentintel, instead of bloating the
> > kernel by adding a new sentinel with each array moved.
> > 
> > Doing that was not so trivial, and has required slowing down the moves of
> > kernel/sysctl.c arrays and measuring the impact on size by each new move.
> > 
> > The complex part of the effort to help reduce the size of each sysctl is being
> > done by the patient work of el señor Don Joel Granados. A lot of this is truly
> > painful code refactoring and testing and then trying to measure the savings of
> > each move and removing the sentinels. Although Joel already has code which does
> > most of this work, experience with sysctl moves in the past shows is we need to
> > be careful due to the slew of odd build failures that are possible due to the
> > amount of random Kconfig options sysctls use.
> > 
> > To that end Joel's work is split by first addressing the major housekeeping
> > needed to remove the sentinels, which is part of this merge request. The rest
> > of the work to actually remove the sentinels will be done later in future
> > kernel releases.
> 
> This is very interesting for me. I'm also refactoring sysctl based on
> discussion with Linus a while ago.
> 
> Could you please add me to the Cc in the next patches?
I just sent the next batch for this set for review. You can see it here
https://lore.kernel.org/all/20230906-jag-sysctl_remove_empty_elem_arch-v1-0-3935d4854248@samsung.com/
Will add you for the next ones.

Best

> 
> > At first I was only going to send his first 7 patches of his patch series,
> > posted 1 month ago, but in retrospect due to the testing the changes have
> > received in linux-next and the minor changes they make this goes with the
> > entire set of patches Joel had planned: just sysctl house keeping. There are
> > networking changes but these are part of the house keeping too.
> > 
> > The preliminary math is showing this will all help reduce the overall build
> > time size of the kernel and run time memory consumed by the kernel by about
> > ~64 bytes per array where we are able to remove each sentinel in the future.
> > That also means there is no more bloating the kernel with the extra ~64 bytes
> > per array moved as no new sentinels are created.
> > 
> > Most of this has been in linux-next for about a month, the last 7 patches took
> > a minor refresh 2 week ago based on feedback.
> > 
> > ----------------------------------------------------------------
> > Joel Granados (14):
> >       sysctl: Prefer ctl_table_header in proc_sysctl
> >       sysctl: Use ctl_table_header in list_for_each_table_entry
> >       sysctl: Add ctl_table_size to ctl_table_header
> >       sysctl: Add size argument to init_header
> >       sysctl: Add a size arg to __register_sysctl_table
> >       sysctl: Add size to register_sysctl
> >       sysctl: Add size arg to __register_sysctl_init
> >       sysctl: Add size to register_net_sysctl function
> >       ax.25: Update to register_net_sysctl_sz
> >       netfilter: Update to register_net_sysctl_sz
> >       networking: Update to register_net_sysctl_sz
> >       vrf: Update to register_net_sysctl_sz
> >       sysctl: SIZE_MAX->ARRAY_SIZE in register_net_sysctl
> >       sysctl: Use ctl_table_size as stopping criteria for list macro
> > 
> >  arch/arm64/kernel/armv8_deprecated.c    |  2 +-
> >  arch/s390/appldata/appldata_base.c      |  2 +-
> >  drivers/net/vrf.c                       |  3 +-
> >  fs/proc/proc_sysctl.c                   | 90 +++++++++++++++++----------------
> >  include/linux/sysctl.h                  | 31 +++++++++---
> >  include/net/ipv6.h                      |  2 +
> >  include/net/net_namespace.h             | 10 ++--
> >  ipc/ipc_sysctl.c                        |  4 +-
> >  ipc/mq_sysctl.c                         |  4 +-
> >  kernel/ucount.c                         |  5 +-
> >  net/ax25/sysctl_net_ax25.c              |  3 +-
> >  net/bridge/br_netfilter_hooks.c         |  3 +-
> >  net/core/neighbour.c                    |  8 ++-
> >  net/core/sysctl_net_core.c              |  3 +-
> >  net/ieee802154/6lowpan/reassembly.c     |  8 ++-
> >  net/ipv4/devinet.c                      |  3 +-
> >  net/ipv4/ip_fragment.c                  |  3 +-
> >  net/ipv4/route.c                        |  8 ++-
> >  net/ipv4/sysctl_net_ipv4.c              |  3 +-
> >  net/ipv4/xfrm4_policy.c                 |  3 +-
> >  net/ipv6/addrconf.c                     |  3 +-
> >  net/ipv6/icmp.c                         |  5 ++
> >  net/ipv6/netfilter/nf_conntrack_reasm.c |  3 +-
> >  net/ipv6/reassembly.c                   |  3 +-
> >  net/ipv6/route.c                        |  9 ++++
> >  net/ipv6/sysctl_net_ipv6.c              | 16 ++++--
> >  net/ipv6/xfrm6_policy.c                 |  3 +-
> >  net/mpls/af_mpls.c                      |  6 ++-
> >  net/mptcp/ctrl.c                        |  3 +-
> >  net/netfilter/ipvs/ip_vs_ctl.c          |  8 ++-
> >  net/netfilter/ipvs/ip_vs_lblc.c         | 10 ++--
> >  net/netfilter/ipvs/ip_vs_lblcr.c        | 10 ++--
> >  net/netfilter/nf_conntrack_standalone.c |  4 +-
> >  net/netfilter/nf_log.c                  |  7 +--
> >  net/rds/tcp.c                           |  3 +-
> >  net/sctp/sysctl.c                       |  4 +-
> >  net/smc/smc_sysctl.c                    |  3 +-
> >  net/sysctl_net.c                        | 26 +++++++---
> >  net/unix/sysctl_net_unix.c              |  3 +-
> >  net/xfrm/xfrm_sysctl.c                  |  8 ++-
> >  40 files changed, 222 insertions(+), 113 deletions(-)
> 
> -- 
> Rgrds, legion
>