mbox series

[v2,00/11] xfs: widen timestamps to deal with y2038

Message ID 159770500809.3956827.8869892960975362931.stgit@magnolia (mailing list archive)
Headers show
Series xfs: widen timestamps to deal with y2038 | expand

Message

Darrick J. Wong Aug. 17, 2020, 10:56 p.m. UTC
Hi all,

This series performs some refactoring of our timestamp and inode
encoding functions, then retrofits the timestamp union to handle
timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
to the non-root dquot timer fields to boost their effective size to 34
bits.  These two changes enable correct time handling on XFS through the
year 2486.

v2: rebase to 5.9, having landed the quota refactoring

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=bigtime

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=bigtime

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=bigtime
---
 fs/xfs/libxfs/xfs_dquot_buf.c  |   60 +++++++++++++++++
 fs/xfs/libxfs/xfs_format.h     |  139 +++++++++++++++++++++++++++++++++++++---
 fs/xfs/libxfs/xfs_fs.h         |    1 
 fs/xfs/libxfs/xfs_inode_buf.c  |  132 ++++++++++++++++++--------------------
 fs/xfs/libxfs/xfs_inode_buf.h  |    7 +-
 fs/xfs/libxfs/xfs_log_format.h |   21 ++++--
 fs/xfs/libxfs/xfs_quota_defs.h |    9 ++-
 fs/xfs/libxfs/xfs_sb.c         |    2 +
 fs/xfs/scrub/inode.c           |   31 +++++++--
 fs/xfs/scrub/quota.c           |    8 ++
 fs/xfs/xfs_dquot.c             |   66 ++++++++++++++++---
 fs/xfs/xfs_dquot.h             |    4 +
 fs/xfs/xfs_inode.c             |   11 +++
 fs/xfs/xfs_inode_item.c        |   97 ++++++++++++++++++++++++++--
 fs/xfs/xfs_inode_item.h        |    3 +
 fs/xfs/xfs_ioctl.c             |    3 +
 fs/xfs/xfs_ondisk.h            |   30 ++++++++-
 fs/xfs/xfs_qm.c                |    2 +
 fs/xfs/xfs_qm_syscalls.c       |   18 +++--
 fs/xfs/xfs_super.c             |   14 +++-
 20 files changed, 531 insertions(+), 127 deletions(-)

Comments

Dave Chinner Aug. 18, 2020, 11:01 p.m. UTC | #1
On Mon, Aug 17, 2020 at 03:56:48PM -0700, Darrick J. Wong wrote:
> Hi all,
> 
> This series performs some refactoring of our timestamp and inode
> encoding functions, then retrofits the timestamp union to handle
> timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
> to the non-root dquot timer fields to boost their effective size to 34
> bits.  These two changes enable correct time handling on XFS through the
> year 2486.

A bit more detail would be nice :)

Like, the inode timestamp has a range of slightly greater than 2^34
because 10^9 < 2^30. i.e.

Inode timestamp range in days:

$ echo $(((2**62 / (1000*1000*1000) / 86400) * 2**2))
213500
$

While the quota timer range in days is:
$ echo $(((2**34 / 86400)))
198841
$

There's ~15,000 days difference in range here, which in years is
about 40 years. Hence the inodes have a timestamp range out to
~2485 from the 1901 epoch start, while quota timers have a range
out to only 2445 from the epoch start.

Some discussion of the different ranges, the problems it might cause
and why we don't have to worry about it would be appreciated :)

Cheers,

Dave.
Darrick J. Wong Aug. 18, 2020, 11:10 p.m. UTC | #2
On Wed, Aug 19, 2020 at 09:01:21AM +1000, Dave Chinner wrote:
> On Mon, Aug 17, 2020 at 03:56:48PM -0700, Darrick J. Wong wrote:
> > Hi all,
> > 
> > This series performs some refactoring of our timestamp and inode
> > encoding functions, then retrofits the timestamp union to handle
> > timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
> > to the non-root dquot timer fields to boost their effective size to 34
> > bits.  These two changes enable correct time handling on XFS through the
> > year 2486.
> 
> A bit more detail would be nice :)

Heh, ok.

> Like, the inode timestamp has a range of slightly greater than 2^34
> because 10^9 < 2^30. i.e.
> 
> Inode timestamp range in days:
> 
> $ echo $(((2**62 / (1000*1000*1000) / 86400) * 2**2))
> 213500
> $
> 
> While the quota timer range in days is:
> $ echo $(((2**34 / 86400)))
> 198841
> $
> 
> There's ~15,000 days difference in range here, which in years is
> about 40 years. Hence the inodes have a timestamp range out to
> ~2485 from the 1901 epoch start, while quota timers have a range
> out to only 2445 from the epoch start.

Quota timers have always treated the d_{b,i,rtb}timer value as an
unsigned 32-bit integer, which means that it has /never/ been possible
to set a timer expiration before 1/1/1970.  The quota timer range is
therefore 198,841 days *after* 1970, not after 1901.

Therefore, the quota timer range in days is:

$ echo $(( ((2**34) + (2**31)) / 86400) ))
223696

So, technically speaking, the quota timers could go beyond 2486, but the
current patchset clamps the quota counters to the same max as the
inodes.  I guess I just proved the need for more details upfront.

--D

> 
> Some discussion of the different ranges, the problems it might cause
> and why we don't have to worry about it would be appreciated :)
> 
> Cheers,
> 
> Dave.
> -- 
> Dave Chinner
> david@fromorbit.com
Dave Chinner Aug. 18, 2020, 11:41 p.m. UTC | #3
On Tue, Aug 18, 2020 at 04:10:33PM -0700, Darrick J. Wong wrote:
> On Wed, Aug 19, 2020 at 09:01:21AM +1000, Dave Chinner wrote:
> > On Mon, Aug 17, 2020 at 03:56:48PM -0700, Darrick J. Wong wrote:
> > > Hi all,
> > > 
> > > This series performs some refactoring of our timestamp and inode
> > > encoding functions, then retrofits the timestamp union to handle
> > > timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
> > > to the non-root dquot timer fields to boost their effective size to 34
> > > bits.  These two changes enable correct time handling on XFS through the
> > > year 2486.
> > 
> > A bit more detail would be nice :)
> 
> Heh, ok.
> 
> > Like, the inode timestamp has a range of slightly greater than 2^34
> > because 10^9 < 2^30. i.e.
> > 
> > Inode timestamp range in days:
> > 
> > $ echo $(((2**62 / (1000*1000*1000) / 86400) * 2**2))
> > 213500
> > $
> > 
> > While the quota timer range in days is:
> > $ echo $(((2**34 / 86400)))
> > 198841
> > $
> > 
> > There's ~15,000 days difference in range here, which in years is
> > about 40 years. Hence the inodes have a timestamp range out to
> > ~2485 from the 1901 epoch start, while quota timers have a range
> > out to only 2445 from the epoch start.
> 
> Quota timers have always treated the d_{b,i,rtb}timer value as an
> unsigned 32-bit integer, which means that it has /never/ been possible
> to set a timer expiration before 1/1/1970.  The quota timer range is
> therefore 198,841 days *after* 1970, not after 1901.
> 
> Therefore, the quota timer range in days is:
> 
> $ echo $(( ((2**34) + (2**31)) / 86400) ))
> 223696
> 
> So, technically speaking, the quota timers could go beyond 2486, but the
> current patchset clamps the quota counters to the same max as the
> inodes.  I guess I just proved the need for more details upfront.

Yeah, little things like quota timers and inode timestamps having a
different epoch value are kinda important to understand. :)

Cheers,

Dave.