[RFC,0/2] copy_file_range in cephfs kernel client

Message

Luis Henriques Aug. 20, 2018, 4:22 p.m. UTC

Hi!

This series is my initial attempt at getting a copy_file_range syscall
implementation in the kernel cephfs client using the 'copy-from' RADOS
operation.

The idea of getting this implemented was from Greg -- or, at least, he
created a feature in the tracker [1].  I just decided to give it a try
as the feature wasn't assigned to anyone ;-)

I have this patchset sitting on my laptop for a while already, waiting
for me to revisit it, review some of its TODOs... but I finally decided
to send it out as-is instead, to get some early feedback.

The first patch implements the copy-from operation in the libceph
module.  Unfortunately, the documentation for this operation is
nonexistent and I had to do a lot of digging to figure out the details
(and I probably I missed something!).  For example, initially I was
hoping that this operation could be used to copy more than one object at
the time.  Doing an OSD request per object copy is not ideal, but
unfortunately it seems to be the only way.  Anyway, my expectations are
that this new operation will be useful for other features in the future.

The 2nd patch is where the copy_file_range is implemented and could
probably be optimised, but I didn't bother with that for now.  The
important bit is that we still may need to do some manual copies if the
offsets aren't object aligned or if the length is smaller than the
object size.  I'm using do_splice_direct() for the manual copies as it
was the easiest way to get a PoC running, but maybe there are better
ways.

I've done some functional testing on this PoC.  And it also passes the
generic xfstest suite, in particular the copy_file_range specific tests
(430-434).  But I haven't done any benchmarks to measure any performance
changes in using this syscall.

Any feedback is welcome, specially regarding the TODOs on the code.

[1] https://tracker.ceph.com/issues/21944

Luis Henriques (2):
  ceph: support the RADOS copy-from operation
  ceph: support copy_file_range file operation

 fs/ceph/file.c                  | 182 ++++++++++++++++++++++++++++++++
 include/linux/ceph/osd_client.h |  18 ++++
 include/linux/ceph/rados.h      |  19 ++++
 net/ceph/osd_client.c           |  73 +++++++++++++
 4 files changed, 291 insertions(+)

Comments

Ilya Dryomov Aug. 20, 2018, 4:47 p.m. UTC | #1

On Mon, Aug 20, 2018 at 6:21 PM Luis Henriques <[e-mail]> wrote:
>
> Hi!
>
> This series is my initial attempt at getting a copy_file_range syscall
> implementation in the kernel cephfs client using the 'copy-from' RADOS
> operation.
>
> The idea of getting this implemented was from Greg -- or, at least, he
> created a feature in the tracker [1].  I just decided to give it a try
> as the feature wasn't assigned to anyone ;-)
>
> I have this patchset sitting on my laptop for a while already, waiting
> for me to revisit it, review some of its TODOs... but I finally decided
> to send it out as-is instead, to get some early feedback.
>
> The first patch implements the copy-from operation in the libceph
> module.  Unfortunately, the documentation for this operation is
> nonexistent and I had to do a lot of digging to figure out the details
> (and I probably I missed something!).  For example, initially I was
> hoping that this operation could be used to copy more than one object at
> the time.  Doing an OSD request per object copy is not ideal, but
> unfortunately it seems to be the only way.  Anyway, my expectations are
> that this new operation will be useful for other features in the future.

A single OSD request can contain multiple OSD ops.  In the kernel
client, this number is currently limited to 16 (CEPH_OSD_MAX_OPS).

Thanks,

                Ilya

Ilya Dryomov Aug. 20, 2018, 4:54 p.m. UTC | #2

On Mon, Aug 20, 2018 at 6:47 PM Ilya Dryomov <[e-mail]> wrote:
>
> On Mon, Aug 20, 2018 at 6:21 PM Luis Henriques <[e-mail]> wrote:
> >
> > Hi!
> >
> > This series is my initial attempt at getting a copy_file_range syscall
> > implementation in the kernel cephfs client using the 'copy-from' RADOS
> > operation.
> >
> > The idea of getting this implemented was from Greg -- or, at least, he
> > created a feature in the tracker [1].  I just decided to give it a try
> > as the feature wasn't assigned to anyone ;-)
> >
> > I have this patchset sitting on my laptop for a while already, waiting
> > for me to revisit it, review some of its TODOs... but I finally decided
> > to send it out as-is instead, to get some early feedback.
> >
> > The first patch implements the copy-from operation in the libceph
> > module.  Unfortunately, the documentation for this operation is
> > nonexistent and I had to do a lot of digging to figure out the details
> > (and I probably I missed something!).  For example, initially I was
> > hoping that this operation could be used to copy more than one object at
> > the time.  Doing an OSD request per object copy is not ideal, but
> > unfortunately it seems to be the only way.  Anyway, my expectations are
> > that this new operation will be useful for other features in the future.
>
> A single OSD request can contain multiple OSD ops.  In the kernel
> client, this number is currently limited to 16 (CEPH_OSD_MAX_OPS).

... that operate on the same object, so this isn't very relevant.

Thanks,

                Ilya