mbox series

[v5,00/11] xfs: widen timestamps to deal with y2038

Message ID 159885400575.3608006.17716724192510967135.stgit@magnolia (mailing list archive)
Headers show
Series xfs: widen timestamps to deal with y2038 | expand

Message

Darrick J. Wong Aug. 31, 2020, 6:06 a.m. UTC
Hi all,

This series performs some refactoring of our timestamp and inode
encoding functions, then retrofits the timestamp union to handle
timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
to the non-root dquot timer fields to boost their effective size to 34
bits.  These two changes enable correct time handling on XFS through the
year 2486.

On a current V5 filesystem, inodes timestamps are a signed 32-bit
seconds counter, with 0 being the Unix epoch.  Quota timers are an
unsigned 32-bit seconds counter, with 0 also being the Unix epoch.

This means that inode timestamps can range from:
-(2^31-1) (13 Dec 1901) through (2^31-1) (19 Jan 2038).

And quota timers can range from:
0 (1 Jan 1970) through (2^32-1) (7 Feb 2106).

With the bigtime encoding turned on, inode timestamps are an unsigned
64-bit nanoseconds counter, with 0 being the 1901 epoch.  Quota timers
are a 34-bit unsigned second counter right shifted two bits, with 0
being the Unix epoch, and capped at the maximum inode timestamp value.

This means that inode timestamps can range from:
0 (13 Dec 1901) through (2^64-1 / 1e9) (2 Jul 2486)

Quota timers could theoretically range from:
0 (1 Jan 1970) through (((2^34-1) + (2^31-1)) & ~3) (16 Jun 2582).

But with the capping in place, the quota timers maximum is:
max((2^64-1 / 1e9) - (2^31-1), (((2^34-1) + (2^31-1)) & ~3) (2 Jul 2486).

v2: rebase to 5.9, having landed the quota refactoring
v3: various suggestions by Amir and Dave
v4: drop the timestamp unions, add "is bigtime?" predicates everywhere
v5: reintroduce timestamp unions as *legacy* timestamp unions

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This is an extraordinary way to destroy everything.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfs-linux.git/log/?h=bigtime

xfsprogs git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfsprogs-dev.git/log/?h=bigtime

fstests git tree:
https://git.kernel.org/cgit/linux/kernel/git/djwong/xfstests-dev.git/log/?h=bigtime
---
 fs/xfs/libxfs/xfs_dquot_buf.c   |   35 +++++++
 fs/xfs/libxfs/xfs_format.h      |  190 ++++++++++++++++++++++++++++++++++++++-
 fs/xfs/libxfs/xfs_fs.h          |    1 
 fs/xfs/libxfs/xfs_ialloc.c      |    4 +
 fs/xfs/libxfs/xfs_inode_buf.c   |  130 +++++++++++++--------------
 fs/xfs/libxfs/xfs_inode_buf.h   |   15 +++
 fs/xfs/libxfs/xfs_log_format.h  |    7 +
 fs/xfs/libxfs/xfs_quota_defs.h  |    8 +-
 fs/xfs/libxfs/xfs_sb.c          |    2 
 fs/xfs/libxfs/xfs_shared.h      |    3 +
 fs/xfs/libxfs/xfs_trans_inode.c |   16 +++
 fs/xfs/scrub/inode.c            |   31 +++++-
 fs/xfs/xfs_dquot.c              |   52 +++++++++--
 fs/xfs/xfs_dquot.h              |    3 +
 fs/xfs/xfs_inode.c              |    4 -
 fs/xfs/xfs_inode.h              |    5 +
 fs/xfs/xfs_inode_item.c         |   34 +++++--
 fs/xfs/xfs_inode_item_recover.c |   76 ++++++++++++++++
 fs/xfs/xfs_ioctl.c              |    3 -
 fs/xfs/xfs_ondisk.h             |   24 +++++
 fs/xfs/xfs_qm.c                 |   13 +++
 fs/xfs/xfs_qm.h                 |    4 +
 fs/xfs/xfs_qm_syscalls.c        |   18 ++--
 fs/xfs/xfs_super.c              |   14 ++-
 fs/xfs/xfs_trace.h              |   26 +++++
 fs/xfs/xfs_trans_dquot.c        |    6 +
 26 files changed, 607 insertions(+), 117 deletions(-)

Comments

Amir Goldstein Aug. 31, 2020, 8:07 a.m. UTC | #1
On Mon, Aug 31, 2020 at 9:08 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> Hi all,
>
> This series performs some refactoring of our timestamp and inode
> encoding functions, then retrofits the timestamp union to handle
> timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
> to the non-root dquot timer fields to boost their effective size to 34
> bits.  These two changes enable correct time handling on XFS through the
> year 2486.
>
> On a current V5 filesystem, inodes timestamps are a signed 32-bit
> seconds counter, with 0 being the Unix epoch.  Quota timers are an
> unsigned 32-bit seconds counter, with 0 also being the Unix epoch.
>
> This means that inode timestamps can range from:
> -(2^31-1) (13 Dec 1901) through (2^31-1) (19 Jan 2038).
>
> And quota timers can range from:
> 0 (1 Jan 1970) through (2^32-1) (7 Feb 2106).
>
> With the bigtime encoding turned on, inode timestamps are an unsigned
> 64-bit nanoseconds counter, with 0 being the 1901 epoch.  Quota timers
> are a 34-bit unsigned second counter right shifted two bits, with 0
> being the Unix epoch, and capped at the maximum inode timestamp value.
>
> This means that inode timestamps can range from:
> 0 (13 Dec 1901) through (2^64-1 / 1e9) (2 Jul 2486)
>
> Quota timers could theoretically range from:
> 0 (1 Jan 1970) through (((2^34-1) + (2^31-1)) & ~3) (16 Jun 2582).
>
> But with the capping in place, the quota timers maximum is:
> max((2^64-1 / 1e9) - (2^31-1), (((2^34-1) + (2^31-1)) & ~3) (2 Jul 2486).
>
> v2: rebase to 5.9, having landed the quota refactoring
> v3: various suggestions by Amir and Dave
> v4: drop the timestamp unions, add "is bigtime?" predicates everywhere
> v5: reintroduce timestamp unions as *legacy* timestamp unions

I went over the relevant patches briefly.
I do not have time for thorough re-review and seems like you have enough
reviewers already, but wanted to say that IMO v5 is "approachable" for
novice xfs developers and I can follow the conversions easily, so that's
probably a good thing ;-)

Thanks,
Amir.
Darrick J. Wong Aug. 31, 2020, 3:58 p.m. UTC | #2
On Mon, Aug 31, 2020 at 11:07:14AM +0300, Amir Goldstein wrote:
> On Mon, Aug 31, 2020 at 9:08 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
> >
> > Hi all,
> >
> > This series performs some refactoring of our timestamp and inode
> > encoding functions, then retrofits the timestamp union to handle
> > timestamps as a 64-bit nanosecond counter.  Next, it adds bit shifting
> > to the non-root dquot timer fields to boost their effective size to 34
> > bits.  These two changes enable correct time handling on XFS through the
> > year 2486.
> >
> > On a current V5 filesystem, inodes timestamps are a signed 32-bit
> > seconds counter, with 0 being the Unix epoch.  Quota timers are an
> > unsigned 32-bit seconds counter, with 0 also being the Unix epoch.
> >
> > This means that inode timestamps can range from:
> > -(2^31-1) (13 Dec 1901) through (2^31-1) (19 Jan 2038).
> >
> > And quota timers can range from:
> > 0 (1 Jan 1970) through (2^32-1) (7 Feb 2106).
> >
> > With the bigtime encoding turned on, inode timestamps are an unsigned
> > 64-bit nanoseconds counter, with 0 being the 1901 epoch.  Quota timers
> > are a 34-bit unsigned second counter right shifted two bits, with 0
> > being the Unix epoch, and capped at the maximum inode timestamp value.
> >
> > This means that inode timestamps can range from:
> > 0 (13 Dec 1901) through (2^64-1 / 1e9) (2 Jul 2486)
> >
> > Quota timers could theoretically range from:
> > 0 (1 Jan 1970) through (((2^34-1) + (2^31-1)) & ~3) (16 Jun 2582).
> >
> > But with the capping in place, the quota timers maximum is:
> > max((2^64-1 / 1e9) - (2^31-1), (((2^34-1) + (2^31-1)) & ~3) (2 Jul 2486).
> >
> > v2: rebase to 5.9, having landed the quota refactoring
> > v3: various suggestions by Amir and Dave
> > v4: drop the timestamp unions, add "is bigtime?" predicates everywhere
> > v5: reintroduce timestamp unions as *legacy* timestamp unions
> 
> I went over the relevant patches briefly.
> I do not have time for thorough re-review and seems like you have enough
> reviewers already, but wanted to say that IMO v5 is "approachable" for
> novice xfs developers and I can follow the conversions easily, so that's
> probably a good thing ;-)

<nod> Thanks for the review!  Good to see you at LPC last week too!

--D

> Thanks,
> Amir.